AI Needs Enormous Computing Power. Could Light-Based Chips Help?(quantamagazine.org)
quantamagazine.org
AI Needs Enormous Computing Power. Could Light-Based Chips Help?
https://www.quantamagazine.org/ai-needs-enormous-computing-power-could-light-based-chips-help-20240520/
41 comments
A wise person once said "Bosons are for transport and fermions are for switching."
please, eli5 <3
ly5:
The universe is made of two kinds of particles, Bosons and Fermions. Light is a boson. Most of the stuff we touch is made of Fermions. Electricity is fermions also.
Bosons, like light, don't like to interact with things. They just want to go places, as fast as they possibly can. So if you want a particle to carry information from point A to point B, light is great. Point it in the right direction and off it goes, and it won't interact with anything unless it absolutely as to.
Fermions, like electrons, love bumping into each other and changing each other. They like to make atoms and collide and bounce off each other all the time. So if you want to build a switch or a microchip where lots of things connect to each other to do some big complicated task like computing lots of math then electrons work really well.
But if you try to get photons to dance together to do something complicated you'll have a hard time. They only interact with each other and the stuff around them when things are just right, and then only in limited ways.
And if you try to use electrons to move data over any kind of distance that's hard too. They're constantly stopping to chat with atoms they find along the way, turning the energy of your message into heat.
How's that?
Bosons, like light, don't like to interact with things. They just want to go places, as fast as they possibly can. So if you want a particle to carry information from point A to point B, light is great. Point it in the right direction and off it goes, and it won't interact with anything unless it absolutely as to.
Fermions, like electrons, love bumping into each other and changing each other. They like to make atoms and collide and bounce off each other all the time. So if you want to build a switch or a microchip where lots of things connect to each other to do some big complicated task like computing lots of math then electrons work really well.
But if you try to get photons to dance together to do something complicated you'll have a hard time. They only interact with each other and the stuff around them when things are just right, and then only in limited ways.
And if you try to use electrons to move data over any kind of distance that's hard too. They're constantly stopping to chat with atoms they find along the way, turning the energy of your message into heat.
How's that?
That is nice.
But then why do fiber optic cables work? The data does need to be turned into photons, and back from photons into electrons at some point. So all of that has switching and has been done for a long time.
Guess maybe lot of people like me are wondering why the technology for fiber optic cables can't be scaled down further to be used on chips.
But then why do fiber optic cables work? The data does need to be turned into photons, and back from photons into electrons at some point. So all of that has switching and has been done for a long time.
Guess maybe lot of people like me are wondering why the technology for fiber optic cables can't be scaled down further to be used on chips.
Latency? Takes time to make photons from electrons and vice versa. Folks try to step around this with light-driven gates. Lots to learn! Plenty out there!
There are bosons that carry charges and/or mass like the weak bosons and the gluons. There is nothing in principle about bosons that would make them incapable of building logic gates out of them.
Some advanced civilization (future us) may use them. Even photons interact with each other a little bit due to QED.
Some advanced civilization (future us) may use them. Even photons interact with each other a little bit due to QED.
For sure, you can if you want to. And if you pull it off there are some advantages, which the article describes.
It's just harder.
It's just harder.
I wonder if it's just harder for us, now. It used to be hard for humans to make fire: nowadays you have match stick or lighter technology to make fire just like that. I wonder if we as humanity will be able to build the technological harness around light just as well.
There's also composite bosons like cooper pairs which are just made up of two electrons.
That's a fantastic explanation. +1.
In quantum mechanics, bosons are (often massless) force carrying particles like photons or gluons. Fermions are the massive matter particles, such as electrons or quarks.
So, while I’ve never heard this saying before, I assume it’s meaning is that massless particles like photons are best for carrying information around (rather than electrons we are using in circuits today), while the electrons are best for carrying state, like in a switch.
Note, that in networking we have already made that transition by using fiber optics, rather than electric wire to transfer information over longer distances.
So, while I’ve never heard this saying before, I assume it’s meaning is that massless particles like photons are best for carrying information around (rather than electrons we are using in circuits today), while the electrons are best for carrying state, like in a switch.
Note, that in networking we have already made that transition by using fiber optics, rather than electric wire to transfer information over longer distances.
And radio! We use lots of radio as well.
I wonder what the longest information carrying electric wire is? They used to cross oceans, but not anymore. There are loads of DSL twisted pairs and co-ax cables in the "last mile" that maybe go 5km max. In rural areas maybe up to 50km with repeaters?
Is there something in between? Some old buried copper trunk cable between two university campuses or something like that?
I wonder what the longest information carrying electric wire is? They used to cross oceans, but not anymore. There are loads of DSL twisted pairs and co-ax cables in the "last mile" that maybe go 5km max. In rural areas maybe up to 50km with repeaters?
Is there something in between? Some old buried copper trunk cable between two university campuses or something like that?
Depending on your definition of wire, probably the earth. The first telegraph systems used the earth (or sea water can't remember) as the return path.
Hmm, fermions are still better for __power__ transport.
When you measure the flow of electrical power along a cable, the energy travels close to the speed of light, yet the electrons drift very, very slowly in comparison.
Electrical power is transported as a fast-moving density wave in the electromagnetic force between many electrons and protons, the former drifting slowly.
Not by the electrons themselves, as if energy were attached to them.
The force responsible for transporting the energy so much faster than the electrons themselves move, is comprised of photons (bosons not fermions)...
Electrical power is transported as a fast-moving density wave in the electromagnetic force between many electrons and protons, the former drifting slowly.
Not by the electrons themselves, as if energy were attached to them.
The force responsible for transporting the energy so much faster than the electrons themselves move, is comprised of photons (bosons not fermions)...
By that analogy information flowing in a regular IC is also photons.
> HITOP performs about 1 trillion operations per second, whereas sophisticated Nvidia chips can chug through 300 times as much data
Wow, it sounds like optical computing is already closer than I expected.
Wow, it sounds like optical computing is already closer than I expected.
If you have a laser, you can print an optical neural network using a consumer 3D printer. The present barrier to adoption, as the article describes, is the cost of transitioning from the optical space to the electronic space.
Do you know where does this cost lie? Say for an organization with no prior spending on "legacy" electronics based hardware.
Is it the sunken cost in existing hardware, or developing the rest of the "value chain" to make photonics work, or something else altogether?
Is it the sunken cost in existing hardware, or developing the rest of the "value chain" to make photonics work, or something else altogether?
You have to miniaturize the components and be able to cram billions on a die of a few square centimeters at most. The industry has lots of experience doing that with electronics. Almost none with photonics.
The silver lining is that one might need massively less photonic components to get the job done than electronic components due to the former's theoretically magnitudes higher switching frequency. And we might reserve them for graphics and AI at first.
The silver lining is that one might need massively less photonic components to get the job done than electronic components due to the former's theoretically magnitudes higher switching frequency. And we might reserve them for graphics and AI at first.
> If you have a laser, you can print an optical neural network using a consumer 3D printer.
I would love to look into this more. Do you have a reference that would give some instructions on how to give this a go?
I would love to look into this more. Do you have a reference that would give some instructions on how to give this a go?
You could think of a lens as a "Fourier transformer," not in the LLM sense directly but in the sense of something that executes a Fourier transform. See the excellent videos by Huygens Optics including this one: https://www.youtube.com/watch?v=Y9FZ4igNxNA entitled "Fourier optics used for optical pattern recognition."
Having established that, well, it's not hard to imagine that something that's good at running Fourier transforms is probably good for other situations where you need to run a lot of dot products in parallel. That, in turn, should sound like an awfully familiar problem...
Having established that, well, it's not hard to imagine that something that's good at running Fourier transforms is probably good for other situations where you need to run a lot of dot products in parallel. That, in turn, should sound like an awfully familiar problem...
Ok, but dot products and Fourier transforms are both linear operations. Maxwell's equations are linear (light also obeys them).
Neural nets, on the other hand, generally require nonlinear operations. As do most computer programs.
So the explanation here is either not complete, or perhaps applies to a very restricted type of neural network only.
Neural nets, on the other hand, generally require nonlinear operations. As do most computer programs.
So the explanation here is either not complete, or perhaps applies to a very restricted type of neural network only.
Good point. I know there are people working on optical NNs, but I have no idea how they handle nonlinear activation functions.
There’re a handful of ways to introduce electro-optical and all-optical non-linearities into optical neural networks. Here’s a fair survey of the field: https://dl.acm.org/doi/full/10.1145/3607533
> But the optical chip’s efficiency is compelling. “The game here is that we lowered the energy cost 1,000 times,” Chen said.
The second part of that paragraph is even more important. The optical equivalent is actually ~3x more efficient than the electronical version. With 300x HITOPs you would only consume 300 energy instead of the 1000 energy of the Nvidia version.
Simplified, that means 3x longer battery life, or 3x less heat, or 3x lower energy bill.
The second part of that paragraph is even more important. The optical equivalent is actually ~3x more efficient than the electronical version. With 300x HITOPs you would only consume 300 energy instead of the 1000 energy of the Nvidia version.
Simplified, that means 3x longer battery life, or 3x less heat, or 3x lower energy bill.
> 3x longer battery life, or 3x less heat, or 3x lower energy bill.
Can we not have the same battery life, same heat, and same power bill and instead do 3 times as many things?
Can we not have the same battery life, same heat, and same power bill and instead do 3 times as many things?
I see you work in the mobile phone industry
Regular I doesn't need that much power, maybe we could just use that instead?
Ok, but how do you lie to VCs to trick them into thinking there is limitless potential in RI?
It sounds like VC boards are semi-civilised bands of fairy tale creatures that have to be put to sleep with gold and fantastical tales or else you get kicked out the kingdom and they kidnap your children.
Can't we just AI driven techniques to help optimize and reduce resource usage.
I think we are all ready seeing good results. I'm super interested in the next gen chips as well.
https://stability.ai/news/stability-ai-sdxl-turbo
https://www.marktechpost.com/2023/01/05/deepminds-latest-ai-...
I think we are all ready seeing good results. I'm super interested in the next gen chips as well.
https://stability.ai/news/stability-ai-sdxl-turbo
https://www.marktechpost.com/2023/01/05/deepminds-latest-ai-...
Why not both? Just because we can make our code more efficient doesn't mean we should go back to Motorola 6800s. We should make our code more efficient and make hardware that can run that code more efficently.
[deleted]
Apart from the fact that there is no such thing as AI (it's artificial all right but far from intelligence), this might be the next cryptocurrency, i.e. huge energy waste for no perceptible gain. Except this time there are way more suckers and a hype wave the likes of which we haven't seen since the web 1.0 bubble.
I agree there's no intelligence, and that there's hype, but on the other hand there's some useful stuff being solved by Machine Learning.
In the first place, why use AI for such basic tasks as generate commit messages ?
Reminds me of lightera and infinera
it's the photonics amplifiers that are very hard to make on wafer
Quantum computing, carbon chips, biological chips, optical chips. I have heard about these supposed alternatives to classic silicon chips since at least 20 years ago. They always were 10 years from the point where they can show their super-powers.
I am not saying it can't be done, but I think we are either limited by the level of science we know or by money. We are probably still far from a major breakthrough in computing that will yield exponential growth.
And even when we will have the wonder chip, someone will port Javascript on it, so things can be slow again. :)
I am not saying it can't be done, but I think we are either limited by the level of science we know or by money. We are probably still far from a major breakthrough in computing that will yield exponential growth.
And even when we will have the wonder chip, someone will port Javascript on it, so things can be slow again. :)
The benefits have to outweigh the cost. We're finally reaching the edge of silicon based technologies, where improvements are not guaranteed or incremental. There's still, obviously, a lot of possible improvement, but it's finally worth seriously looking at alternatives.
You'll likely start to see optical networking in cars first, fwiw.
You'll likely start to see optical networking in cars first, fwiw.