Introduction to spherical harmonics for graphics programmers(gpfault.net)
gpfault.net
Introduction to spherical harmonics for graphics programmers
https://gpfault.net/posts/sph.html
29 comments
In case the author is reading this, if you're going to introduce the complex-valued harmonics you should be careful to put the complex conjugate in the inner product
<f, g> = ∫ f(ω)^* g(ω) dω
p . q = Σ_i p^*_i q_i
what level of math do I need to understand this? or the rest of the math in the post, something I can catch up on in a weekend? barely remember the last math class I took seriously, trig like 18 years ago
I think the steps would be like this:
- get understanding of ordinary vector linear algebra.
- understand what vector dot product does and why
- understand why an orthogonal set of basis vectors for the space you're working in is useful / what properties it has / how its used. like basic euclidean 3d space (1,0,0) (0,1,0) (0, 0, 1) basis vectors.
- get a refresher on basic calculus, in particular integrals
- understand this inner product, it's a generalization of dot product, except you can think of your vectors having infinite number of dimensions now.
- the properties of the dot product you know (like that two vectors are perpendicular if their dot product is 0) work for the inner product too. or perhaps its better to say that the general inner product is defined to have similar properties
- there are functions that are orthogonal to each other in the same way vectors can be orthogonal to each other, and you can use the inner product to tell which ones.
- spherical harmonics are constructed / by design orthogonal to each other. how to show this and where the intuition for finding them could come from is a whole topic...
- but once you have it, just like you can project vectors onto basis vectors (to essentially transform them into the coordinate system described by those basis vectors), you can project functions into the coordinate system represented by those orthogonal functions.
- then you have to figure out why you would even want to do this. in short is has a lot of useful properties/applications. in the graphics case you can compress some quite complex functions into just a few coefficients using this (not perfectly, there is some 'information loss', but still). integrating over two functions becomes cheaper when they are projected to SH basis. it lets you do some unintuitive stuff like combine light that goes into different directions into one common set of coefficients.
- get understanding of ordinary vector linear algebra.
- understand what vector dot product does and why
- understand why an orthogonal set of basis vectors for the space you're working in is useful / what properties it has / how its used. like basic euclidean 3d space (1,0,0) (0,1,0) (0, 0, 1) basis vectors.
- get a refresher on basic calculus, in particular integrals
- understand this inner product, it's a generalization of dot product, except you can think of your vectors having infinite number of dimensions now.
- the properties of the dot product you know (like that two vectors are perpendicular if their dot product is 0) work for the inner product too. or perhaps its better to say that the general inner product is defined to have similar properties
- there are functions that are orthogonal to each other in the same way vectors can be orthogonal to each other, and you can use the inner product to tell which ones.
- spherical harmonics are constructed / by design orthogonal to each other. how to show this and where the intuition for finding them could come from is a whole topic...
- but once you have it, just like you can project vectors onto basis vectors (to essentially transform them into the coordinate system described by those basis vectors), you can project functions into the coordinate system represented by those orthogonal functions.
- then you have to figure out why you would even want to do this. in short is has a lot of useful properties/applications. in the graphics case you can compress some quite complex functions into just a few coefficients using this (not perfectly, there is some 'information loss', but still). integrating over two functions becomes cheaper when they are projected to SH basis. it lets you do some unintuitive stuff like combine light that goes into different directions into one common set of coefficients.
> what level of math do I need to understand this?
A basic understanding of differential equations is all that's really necessary, but knowing about orthogonal polynomials would be helpful too.
> something I can catch up on in a weekend?
Probably not. If you know any calculus (even a basic high school class should be enough), two weekends would probably be enough; if you don't know calculus, then double it.
My advice would be to use an introductory level quantum physics textbook or an advanced chemistry textbook, since the spherical harmonics are used quite a bit in those fields. You could use a math textbook too, but those will tend to focus on details that are irrelevant to you.
An alternate path would be to learn about Fourier series/transformations, then what's discussed in the article will follow as a natural consequence. This is probably a harder option, but there's lots of really good learning materials online for Fourier transformations (and comparatively little for spherical harmonics), so it may end up being easier for you.
A basic understanding of differential equations is all that's really necessary, but knowing about orthogonal polynomials would be helpful too.
> something I can catch up on in a weekend?
Probably not. If you know any calculus (even a basic high school class should be enough), two weekends would probably be enough; if you don't know calculus, then double it.
My advice would be to use an introductory level quantum physics textbook or an advanced chemistry textbook, since the spherical harmonics are used quite a bit in those fields. You could use a math textbook too, but those will tend to focus on details that are irrelevant to you.
An alternate path would be to learn about Fourier series/transformations, then what's discussed in the article will follow as a natural consequence. This is probably a harder option, but there's lots of really good learning materials online for Fourier transformations (and comparatively little for spherical harmonics), so it may end up being easier for you.
This is basic integral calculus, and the sigma symbol indicates discrete summation.
If one needs to describe (and maybe compress) functions or data on a sphere, spherical harmonics are really a thing.
An alternative would be to construct a new function (or matrix) that is not only periodic in azimuth, but also in elevation (i.e., extend elevation to a full circle -pi to +pi). Then, one can simply compute two independent Fourie r transforms: along azimuth and along elevation. [1] The same idea works on matrices using the Discrete Fourier transform (DFT/FFT). However, you then have to accept things like that your data points are all equal at the poles.
[1] https://en.wikipedia.org/wiki/Double_Fourier_sphere_method
An alternative would be to construct a new function (or matrix) that is not only periodic in azimuth, but also in elevation (i.e., extend elevation to a full circle -pi to +pi). Then, one can simply compute two independent Fourie r transforms: along azimuth and along elevation. [1] The same idea works on matrices using the Discrete Fourier transform (DFT/FFT). However, you then have to accept things like that your data points are all equal at the poles.
[1] https://en.wikipedia.org/wiki/Double_Fourier_sphere_method
> spherical harmonics can have uses beyond lighting
This math is also used in Ambisonic surround sound though newer techniques use planewave expansion.
For games, the full-sphere encoding of Ambisonic B-format can be decoded for arbitrary speaker locations and the soundfield rotated around any axis. I'm not sure if its ever been used for a game though.
This math is also used in Ambisonic surround sound though newer techniques use planewave expansion.
For games, the full-sphere encoding of Ambisonic B-format can be decoded for arbitrary speaker locations and the soundfield rotated around any axis. I'm not sure if its ever been used for a game though.
... also quantum mechanics. The textbook solution for the wavefunction of the hydrogen atom involves spherical harmonics.
That fact is betrayed by the the similarity of the shapes of atomic orbitals and the sensitivity patterns of Abisonic B-format channels.
https://en.wikipedia.org/wiki/Ambisonics#Higher-order_ambiso...
https://en.wikipedia.org/wiki/Atomic_orbital#Orbitals_table
...and the same patterns appear on the unit disk with the Zernike polynomials, used to describe optical aberrations and more.
https://en.wikipedia.org/wiki/Zernike_polynomials
https://en.wikipedia.org/wiki/Ambisonics#Higher-order_ambiso...
https://en.wikipedia.org/wiki/Atomic_orbital#Orbitals_table
...and the same patterns appear on the unit disk with the Zernike polynomials, used to describe optical aberrations and more.
https://en.wikipedia.org/wiki/Zernike_polynomials
I noticed that (similarity between the graphs and the shapes of atomic orbitals), and assumed that was what the article was about. And it wasn’t, and never brought it up, so I was thinking maybe I was confused about the similarity. So thank you for showing me I was not.
lol, I was confused from the first imaging thinking this was going to be a tutorial on quantum physics then was confused even more as I scrolled
Ambisonics can give one end game audio
end-game audio?
Or it can give game audio to one end?
Or it can give game audio to one end?
A single Ambisonic B-format recording can be shipped and at runtime decoded into any coincident or near-coincident stereo pair pointing in any direction or into any surround sound format. It is a universal format that encodes the direction and intensity of arriving sound over a full sphere.
Author here. Please let me know if the sample code doesn't work for you. It's all single threaded dumb JavaScript which makes it very easy to read, but definitely not performant. I decided to stick with it for didactic reasons, but still worried that it may hang someone's browser.
Would it be practical to use high resolution spherical harmonics as a replacement for cube maps?
Not really. Besides the problems with ringing outlined in the post, the number of coefficients required to capture higher frequency detail grows quadratically, requiring not only more storage but also operations to evaluate. Which makes straightforward cubemap replacement impractical.
Loved the writeup.
I'll just drop a note here to say that these spherical harmonics are also used in creating specialized neural network layers that are useful for modeling 3D objects like point clouds and moleculues, proteins, etc. Basically whenever we want to make sure that rotating / translating the object doesnt make a new object. [0] is a good reference for this.
Even more interesting is that these are the same spherical harmonics that appear as solutions to Schrodinger's equation in quantum mechanics (s, p, d, f orbitals in an atom) [1]
[0] https://arxiv.org/abs/2312.07511. [1] https://en.wikipedia.org/wiki/Atomic_orbital
I'll just drop a note here to say that these spherical harmonics are also used in creating specialized neural network layers that are useful for modeling 3D objects like point clouds and moleculues, proteins, etc. Basically whenever we want to make sure that rotating / translating the object doesnt make a new object. [0] is a good reference for this.
Even more interesting is that these are the same spherical harmonics that appear as solutions to Schrodinger's equation in quantum mechanics (s, p, d, f orbitals in an atom) [1]
[0] https://arxiv.org/abs/2312.07511. [1] https://en.wikipedia.org/wiki/Atomic_orbital
Awesome write up. Is the appeal of spherical harmonics in graphics compression?
Like, if you know the third order harmonics that's only 16 values you have to pass around
Like, if you know the third order harmonics that's only 16 values you have to pass around
For sure that's a big reason but it's also a useful basis for doing lighting calculations because of their sphere like nature. They are quite efficient in dynamic scenes and historically used in a lot of precalc to do something akin to real time Global Illumination
This is really great. I always saw those harmonic shapes as electron orbitals, I had no idea they could be used in lighting too - so cool.
It made me wonder - why do the electron orbitals take those shapes in say a hydrogen atom? Is there a constraint on the electron and proton together that make it fit only to spherical harmonic functions?
It made me wonder - why do the electron orbitals take those shapes in say a hydrogen atom? Is there a constraint on the electron and proton together that make it fit only to spherical harmonic functions?
The reason is that electrons (like all quantum mechanical objects) are wavelike. In an isolated hydrogen atom, the electron is in a spherically symmetric environment, so the solutions to the wave equation have to be spherical standing waves, which are the spherical harmonics. The wave frequencies have to be integer divisions of 2pi or else they would destructively interfere. (Technically each solution is a product of a spherical harmonic function and a radial function that describes how fast the electron wave decays vs distance from the nucleus)
What’s interesting is if the environment is not spherically symmetric (consider an electron in a molecule) the solutions to the wave equation (the electronic wave functions) are no longer spherical harmonics, even though we like to approximate them with combinations of spherical harmonic basis functions centered on each nucleus. It’s kind of like standing waves on a circular drum head (hydrogen atom) vs standing waves on an irregular shaped drum head
Of course the nucleus also has a wave nature and in reality this interacts with the electrons, but in chemistry and materials we mostly ignore this and approximate the nucleus like a static point charge from the elctrons perspective because the electrons are so much lighter and faster
What’s interesting is if the environment is not spherically symmetric (consider an electron in a molecule) the solutions to the wave equation (the electronic wave functions) are no longer spherical harmonics, even though we like to approximate them with combinations of spherical harmonic basis functions centered on each nucleus. It’s kind of like standing waves on a circular drum head (hydrogen atom) vs standing waves on an irregular shaped drum head
Of course the nucleus also has a wave nature and in reality this interacts with the electrons, but in chemistry and materials we mostly ignore this and approximate the nucleus like a static point charge from the elctrons perspective because the electrons are so much lighter and faster
Ah amazing - thank you for the response! I have a couple of related questions - is it that the non 2 pi frequencies exist, but they destructively interfere so we can't see them? My understanding is that the radial function for the electron is zero at the nucleus - there is no possibility of it being found there - but why is that the case?
Admittedly my understanding of QM is a bit vibey but I’ll try to answer
In an atom, angular wavefunctions with wavelengths non-integer divisions of 2pi can’t exist because of the boundary conditions on the wave equation. A free electron can have any wavelength, but once you put it in a box (confine it to the potential around a proton in a Hydrogen atom) the non-integer wavelengths aren’t allowed
I think it’s instructive to think about what the wavefunction represents. It’s square is the electron probability density (technically the wavefunction is complex valued so it’s the wavefunction times it’s complex conjugate). If you have a non-integer multiple wavelength then the wavefunction goes out of phase with its complex conjugate after one period, and if you integrate over the angular domain the electron probability has to be zero everywhere.
This also answers your second question. The radial solution to the wave equation for hydrogen gives you the Laguerre polynomials. They don’t all go to zero at the nucleus though, actually the first one has a maximum at zero because it scales like exp(-r) (See fig 4.10.2 on chem.libretexts linked below). But when you do a volume integral to calculate the electron probability, the probability near the nucleus is low because the integration volume is small even though the wavefunction is large
https://en.wikipedia.org/wiki/Laguerre_polynomials
https://chem.libretexts.org/Courses/University_of_California...
In an atom, angular wavefunctions with wavelengths non-integer divisions of 2pi can’t exist because of the boundary conditions on the wave equation. A free electron can have any wavelength, but once you put it in a box (confine it to the potential around a proton in a Hydrogen atom) the non-integer wavelengths aren’t allowed
I think it’s instructive to think about what the wavefunction represents. It’s square is the electron probability density (technically the wavefunction is complex valued so it’s the wavefunction times it’s complex conjugate). If you have a non-integer multiple wavelength then the wavefunction goes out of phase with its complex conjugate after one period, and if you integrate over the angular domain the electron probability has to be zero everywhere.
This also answers your second question. The radial solution to the wave equation for hydrogen gives you the Laguerre polynomials. They don’t all go to zero at the nucleus though, actually the first one has a maximum at zero because it scales like exp(-r) (See fig 4.10.2 on chem.libretexts linked below). But when you do a volume integral to calculate the electron probability, the probability near the nucleus is low because the integration volume is small even though the wavefunction is large
https://en.wikipedia.org/wiki/Laguerre_polynomials
https://chem.libretexts.org/Courses/University_of_California...
Spherical harmonics are basically a fourier series. They're a complete orthonormal set of basis functions for functions for the unit sphere. Whereas the fourier series from calc 101 is a complete orthonormal set of basis functions on the unit interval (eg [0,1]).
In other words you can express any reasonable function on the unit sphere as a series of spherical harmonic terms. That makes them ideal for working with differential equations (eg schrodinger's equation for the hydrogen atom, or, emission from an arbitrary light source).
In other words you can express any reasonable function on the unit sphere as a series of spherical harmonic terms. That makes them ideal for working with differential equations (eg schrodinger's equation for the hydrogen atom, or, emission from an arbitrary light source).
And the number of terms you need to get a good approximation is related to the frequency. Low frequency signals like lighting work well.
this is all so interesting.. Are there any particular functions / parameters that are typically used, that say replicates 3 point light setups?
I guess at a certain point the number of terms becomes so large that it makes sense to just use a cube map?
I guess at a certain point the number of terms becomes so large that it makes sense to just use a cube map?
In the era im familiar with (ps3, 360) everyone used the first 9 coefficients. You can read the original Ramamoorthi paper for better theory applied to lighting.
But yes it’s an approximation. If you have a ton of terms it looks like a bitmap like you said.
But yes it’s an approximation. If you have a ton of terms it looks like a bitmap like you said.
Typically, spherical harmonics are introduced as a complex function over spherical coordinates, which makes them much easier to derive, but imo hides their beauty.
The real-valued, cartesian form of regular spherical harmonics is also called "solid harmonics" or "harmonic polynomials", in case you want to dig deeper.