John Carmack visits Starbase(twitter.com)
twitter.com
John Carmack visits Starbase
https://twitter.com/ID_AA_Carmack/status/1429907886159941638
98 comments
He must be tempted to join. I think there are few places in the world where one could make a bigger impact as an engineer. He already has experience in rocketry. He's located in Texas. And I bet he'd be interested in the stuff Tesla's doing with AI too.
https://twitter.com/ID_AA_Carmack/status/1429930662652485632
> I did kind of volunteer to help them fix what I consider very poor user interface performance on the older model S (that I drive). Their engineers have been sharing data with me.
That sounds like kinda sorta starting to join...
Working with Tesla on AI actually sounds more like what he's been working on recently... at least if he can convince them to give him a lot of flexibility on what exactly he works on.
> I did kind of volunteer to help them fix what I consider very poor user interface performance on the older model S (that I drive). Their engineers have been sharing data with me.
That sounds like kinda sorta starting to join...
Working with Tesla on AI actually sounds more like what he's been working on recently... at least if he can convince them to give him a lot of flexibility on what exactly he works on.
The big problem is that Carmack is an elite software engineer, and he's not at that same tier in rocketry/aerospace despite being good at it. Elite vs good. Which is to say, he'd have to take a back seat at SpaceX (which is running at full speed; Carmack would have to try to get back up to speed in aerospace just to contribute, difficult by itself). His ability to contribute beyond that of other industry specific talent is very questionable, and there are far better people - at rocketry/aerospace - working there than him. Carmack knows that without question. It simply doesn't make sense for him to spend his time doing that; it's not the best use of his time given that SpaceX already exists (vs trying to start what SpaceX has become, in his Armadillo effort).
It's the equivalent of Michael Jordan spending the remainder of his athletic career trying to play baseball. Sure, he could have done that. If the White Sox had allowed him on the team (ignoring the strike for a moment), more than likely all he would have been doing is taking the job of someone better at playing professional baseball than him; while his greatest ability by far rests in his basketball prowess.
Carmack already did a deep run at aerospace. He has N time left. Where can he combo make the greatest impact and enjoy the work. I'd guess that's something close to the equation, it seems to be for most people in that position/stage of life. Working at SpaceX is unlikely to be that place all things considered. What might have happened if Carmack had joined in the early days instead of doing Armadillo is an interesting thought (it's hard to see how the SpaceX outcome could be much better than it has been though, realistically).
It's the equivalent of Michael Jordan spending the remainder of his athletic career trying to play baseball. Sure, he could have done that. If the White Sox had allowed him on the team (ignoring the strike for a moment), more than likely all he would have been doing is taking the job of someone better at playing professional baseball than him; while his greatest ability by far rests in his basketball prowess.
Carmack already did a deep run at aerospace. He has N time left. Where can he combo make the greatest impact and enjoy the work. I'd guess that's something close to the equation, it seems to be for most people in that position/stage of life. Working at SpaceX is unlikely to be that place all things considered. What might have happened if Carmack had joined in the early days instead of doing Armadillo is an interesting thought (it's hard to see how the SpaceX outcome could be much better than it has been though, realistically).
> The big problem is that Carmack is an elite software engineer, and he's not at that same tier in rocketry/aerospace despite being good at it. Elite vs good. Which is to say, he'd have to take a back seat at SpaceX (which is running at full speed; Carmack would have to try to get back up to speed in aerospace just to contribute, difficult by itself). His ability to contribute beyond that of other industry specific talent is very questionable, and there are far better people - at rocketry/aerospace - working there than him.
According to the book 'Liftoff' about the genesis of SpaceX. This does not accurately reflect how they hire. Domain knowledge and skill is part of the equation, especially for the leadership team. But they'll also pick up anyone with a proven ability of solving difficult engineering problems even from completely different disciplines.
SpaceX is doing things that haven't been done before anywhere, thus the domain experience can only take you so far...
According to the book 'Liftoff' about the genesis of SpaceX. This does not accurately reflect how they hire. Domain knowledge and skill is part of the equation, especially for the leadership team. But they'll also pick up anyone with a proven ability of solving difficult engineering problems even from completely different disciplines.
SpaceX is doing things that haven't been done before anywhere, thus the domain experience can only take you so far...
> It's the equivalent of Michael Jordan spending the remainder of his athletic career trying to play baseball
Sure, but that could just as easily describe Carmack's current efforts in AGI (and I seem to recall that he even tweeted something to that effect himself, though I can't find it now). Him achieving AGI as a solo researcher seems less likely than him making some big technical contributions to SpaceX, or Tesla AI.
Edit: I think I was recalling this tweet: https://twitter.com/ID_AA_Carmack/status/1411010796285480965
Sure, but that could just as easily describe Carmack's current efforts in AGI (and I seem to recall that he even tweeted something to that effect himself, though I can't find it now). Him achieving AGI as a solo researcher seems less likely than him making some big technical contributions to SpaceX, or Tesla AI.
Edit: I think I was recalling this tweet: https://twitter.com/ID_AA_Carmack/status/1411010796285480965
[deleted]
Tempted, perhaps, but not enough to join. Carmack's working on AGI, and he's committed to a decade-long time horizon.
The impact of AGI is far greater than rockets, too.
The impact of AGI is far greater than rockets, too.
Elon is working on AI too, and Dojo is one of the most exciting things in the field if you ask me. Carmack has always been good at leveraging hardware to do amazing things on the graphics side, and AI hardware is not so dissimilar to graphics hardware.
Elon is working on ML, not AGI.
Carmack is working on artificial general intelligence. As far as I know, that's well outside of anything Elon is doing.
Carmack is working on artificial general intelligence. As far as I know, that's well outside of anything Elon is doing.
Elon's been giving some lip service to closer-to-GI capabilities anyways, e.g. when describing how it would be optimal to be able to control Tesla's humanoid robot by saying "please go to the store and get me the following groceries"...
https://youtu.be/j0z4FweCy4M?t=7775
Whether Tesla wants to fund serious work towards that may be a different question (but they do seem to be serious about at least building the robot).
https://youtu.be/j0z4FweCy4M?t=7775
Whether Tesla wants to fund serious work towards that may be a different question (but they do seem to be serious about at least building the robot).
[deleted]
I mean, Elon just announced that they're building a literal android that is supposed to take verbal directions and perform everyday tasks for you. If that's not AGI, I don't know what is. It may not be very likely to happen anytime soon, but the ambition is there at least.
If you haven't watched the Tesla AI presentation, I highly recommend it. The dancer in a spandex robot suit is getting all the headlines but the technical presentations are the real deal. Karpathy in particular is an excellent technical presenter and he didn't hold back.
If you haven't watched the Tesla AI presentation, I highly recommend it. The dancer in a spandex robot suit is getting all the headlines but the technical presentations are the real deal. Karpathy in particular is an excellent technical presenter and he didn't hold back.
> Elon just announced that they're building a literal android that is supposed to take verbal directions and perform everyday tasks for you. If that's not AGI, I don't know what is.
It's not AGI. An AGI is something that can think for itself, the way you can.
It's not AGI. An AGI is something that can think for itself, the way you can.
Elon described this robot as taking directions like "go to the store and get me this list of groceries", implying that it would be able to, like, read the list, choose a store, drive a car there, use a cart, locate the groceries, go through checkout, etc. It's going to need to do a lot of thinking for itself to complete tasks like that. And more generally he described replacing human labor to the point that a universal basic income becomes necessary. AGI is definitely the ambition here.
Nahhh... I'll paraphrase my old skool A(G)I advisor from my first year in grad school: you tell ML to get your groceries; AGI tells you to get the fucking groceries.
I think what's more likely is we'd discover how many common, useful tasks you could perform with a humanoid robot that don't require AGI, with a more Alexa-like approach of iteratively adding support for explicit discrete tasks over time. We've already moved back the goalposts for AGI in so many other areas as we realise how many tasks can be decomposed to something quite mechanical in the right vector space etc.
If anything, the lesson of the entire history of robotics is the exact opposite. I'm surprised to see so many people believe that a robot doesn't need AGI to replace most human labor. Or maybe people just don't believe that Tesla's general purpose robot is a real project. Which is a view I'm partially sympathetic to.
> and Dojo is one of the most exciting things in the field if you ask me
Why do you think this?
Why do you think this?
I believe that the major limiting factor in the speed of development of AI is hardware, not software. Now and for the foreseeable future. The most important projects in AI, therefore, are the ones pushing hardware performance as high as it can go, and the people who have the opportunity to write software for that hardware are going to make the fastest progress.
Though Carmack probably doesn't agree with me on this given his statements that "I believe that current supercomputers are actually sufficient for human level AGI" https://twitter.com/ID_AA_Carmack/status/1340369768138862592 and "I am provisionally not very interested in custom AI accelerators, because I don't think the extra performance will outweigh the reduced flexibility and immature software compared to GPUs" https://twitter.com/ID_AA_Carmack/status/1412897975370719235
Though Carmack probably doesn't agree with me on this given his statements that "I believe that current supercomputers are actually sufficient for human level AGI" https://twitter.com/ID_AA_Carmack/status/1340369768138862592 and "I am provisionally not very interested in custom AI accelerators, because I don't think the extra performance will outweigh the reduced flexibility and immature software compared to GPUs" https://twitter.com/ID_AA_Carmack/status/1412897975370719235
I'm tempted to agree with Carmack here, having worked at a now-defunct self driving project previously that used multiple server-grade GPUs and many-core CPUs pulling 1000+W in their car that couldn't even figure out how to move around a car that was parallel parked too far out in the road.
More teraflops are not what is needed.
More teraflops are not what is needed.
I mean, I partially agree with his opinions from those tweets, but my conclusions are still different. Firstly, the AGI of the future may well be able to be distilled to a form that would fit on a top supercomputer of today. But developing an AGI is going to need a whole lot more power than running an optimized version after the fact. And secondly, the flexibility and availability of GPUs is hard to beat, but I think that after a few iterations of datacenter-scale hardware we'll learn what's needed to really make it shine. So maybe the first iteration of Dojo won't be much better than a GPU cluster in practice, but the third or fourth could be spectacular.
Sure, but this has nothing to do with Dojo specifically (over say TPUs or other matrix accelerators).
Also, I think most people who have so serious deep learning believe that larger models/faster iteration will only get us so far.
Also, I think most people who have so serious deep learning believe that larger models/faster iteration will only get us so far.
Sure, TPUs and other accelerators are also exciting. Dojo is specifically interesting because of the massive off-chip communication bandwidth it has, which I believe is important. But in general, I'm excited about custom AI silicon.
OpenAI seems to believe that scaling transformers can solve many if not most problems, so I'm not alone here.
OpenAI seems to believe that scaling transformers can solve many if not most problems, so I'm not alone here.
Then don't require an AI to run in realtime. Just let it interpret the world in 0.001x speed. This way you could easily simulate hardware which will only be available in decades.
Not the person you're asking, but
- Long iteration time kills productivity, a lot of AI work has long iteration time because hardware is slow.
- It's possible that bigger models on faster hardware are actually all we need to solve some problems. For example see GPT-3.
- Long iteration time kills productivity, a lot of AI work has long iteration time because hardware is slow.
- It's possible that bigger models on faster hardware are actually all we need to solve some problems. For example see GPT-3.
Can you explain what you find exiting about the Dojo?
Making custom AI chips (other than as practice) seems a giant waste of money at the scale of it. In addition I was kinda underwhelmed how little benefit they got compared to an off the shelf solution.
Edit: Someone else asked the same thing before.
Making custom AI chips (other than as practice) seems a giant waste of money at the scale of it. In addition I was kinda underwhelmed how little benefit they got compared to an off the shelf solution.
Edit: Someone else asked the same thing before.
AGI on classical computers? is this a joke?
In a previous twitter thread linked from the post here Elon mentions having tried many times to get John to join.
The two previous tweets that triggered this are also interesting:
https://twitter.com/ID_AA_Carmack/status/1423053681520414724 > A troll was trying to get a rise out of me yesterday, suggesting that I should somehow feel bad about Elon's success after my failed aerospace venture. I can't overstate how alien that thought is to me -- my joy at these things being built is deep.
https://twitter.com/elonmusk/status/1423162437034037248 > It would be an honor to have you visit Starbase
That's an amazing way to respond to a troll.
https://twitter.com/ID_AA_Carmack/status/1423053681520414724 > A troll was trying to get a rise out of me yesterday, suggesting that I should somehow feel bad about Elon's success after my failed aerospace venture. I can't overstate how alien that thought is to me -- my joy at these things being built is deep.
https://twitter.com/elonmusk/status/1423162437034037248 > It would be an honor to have you visit Starbase
That's an amazing way to respond to a troll.
If you sold Carmack, you sold me. I have my opinions on Tesla as a company, but have always been impressed with SpaceX. Godspeed.
Carmack has been sold by SpaceX right from the start.
Musk and Carmack are both rocket nuts. Carmack ran a rocket company: Armadillo Aerospace, mostly as a hobby. From engineer to engineer, Carmack has a lot of respect for Musk, and Musk tried to offer Carmack a job at SpaceX, as in "stop playing with your toys, we have serious rockets here".
Musk and Carmack are both rocket nuts. Carmack ran a rocket company: Armadillo Aerospace, mostly as a hobby. From engineer to engineer, Carmack has a lot of respect for Musk, and Musk tried to offer Carmack a job at SpaceX, as in "stop playing with your toys, we have serious rockets here".
Musk isn't an engineer
Neither Elon Musk nor John Carmack are engineers, certainly not rocket engineers. Musk is a businessman, Carmack is a programmer. Both are world class in their respective fields.
But they are also pretty good at roleplaying engineers. I suggest watching the latest Tim Dodd videos when he tours Starbase with Elon Musk and you'll see Musk engineer character. No nonsense here, no Mars colonies or hyperloops, just the practical considerations of rocket building.
As for John Carmack, my preferred example is that short discussion with Elon Musk on Twitter. https://twitter.com/ID_AA_Carmack/status/536277962803650562
But they are also pretty good at roleplaying engineers. I suggest watching the latest Tim Dodd videos when he tours Starbase with Elon Musk and you'll see Musk engineer character. No nonsense here, no Mars colonies or hyperloops, just the practical considerations of rocket building.
As for John Carmack, my preferred example is that short discussion with Elon Musk on Twitter. https://twitter.com/ID_AA_Carmack/status/536277962803650562
I concur and I’m very vocal about the vapor ware that Tesla spews from time to time. SpaceX has delivered and I watched in complete awe when two rockets touched down near simultaneously a year or so ago.
I still think the whole passenger part of spacex might be a pipe dream but who knows.
I still think the whole passenger part of spacex might be a pipe dream but who knows.
I just look at where Tesla started, and where it is today. How they revolutionized electric cars during a time when the ICE industry was against them in order to protect the status quo. I wonder how many companies would be "all in" on electric right now had Tesla not existed and forced them to. Would the Mustang "Mach E" exist? Would Ford be changing their most profitable and popular line, the F-150 Truck, if there wasn't a Tesla? Would Porsche be making electric cars if Tesla cars 1/3 the price weren't wiping the floor with them in track races, handily embarrassing them? They accomplished this even as the car industry was able to "swing" politicians to ban "Tesla dealerships" which is a huge impediment. Do you know of any Ford, Chevy, Toyota, Nissan, GMC, or Porsche "dealerships" that don't have physical storefronts? They persevered and basically got around that by building A+ products which sold themselves on reputation and performance alone. I guess you can concentrate on "vaporware" and ignore the actual industry shake-up that they forced upon the "establishment" (existing ICE companies) and have done more concretely than any other company I can think of in terms of taking an existing problem (car pollution) and turning it around.
> vapor ware that Tesla spews from time to time
Elon Musk speaks quite aspirationally and optimistically from time to time, for both Tesla and SpaceX.
> SpaceX has delivered
They sure have! Tesla has also delivered over 1.7 million cars. They're on track to deliver about 750k cars in 2021 alone.
Elon Musk speaks quite aspirationally and optimistically from time to time, for both Tesla and SpaceX.
> SpaceX has delivered
They sure have! Tesla has also delivered over 1.7 million cars. They're on track to deliver about 750k cars in 2021 alone.
[deleted]
For those wondering, Carmack founded the now defunct Armadillo Aerospace (as well as being a well known video game developer).
IIR, Carmack spent a lot of his own money on Armadillo Aerospace and most of the people who worked on it were volunteers. They had many ideas including custom designed rocket engines and self contained composable pieces that could be built into large systems.
It was very up Musk's "first principles" alley, but wildly underestimated the capital needed to be successful.
It was very up Musk's "first principles" alley, but wildly underestimated the capital needed to be successful.
"game developer"... He created a couple of small time indie titles : DOOM, Quake, with a hobby company called ID software
I did say "well known"
But seriously, is there a better title to give him for that part of his work?
But seriously, is there a better title to give him for that part of his work?
Father of the First Person Shooter and Patron Saint of Low Latency
Think of it as what it might sound like to a Christian if you called Jesus a "well known" religious figure.
As well as being a carpenter
Pioneer? God?
Pioneer is a good suggestion, thanks :)
Pioneer – thirty years after Spacewar? Hum…
Edit: Pioneering achievements, yes, certainly, but pioneer as in "game development pioneer"…
Edit: Pioneering achievements, yes, certainly, but pioneer as in "game development pioneer"…
I believe John Romero appropriated the 'God' title, making it unavailable for Carmack, no matter how appropriate.
> But seriously, is there a better title to give him for that part of his work?
Omnissiah or Machine Spirit.
Omnissiah or Machine Spirit.
You can say "Creator of Doom and Quake" and/or "Founder of ID software"
Actually, Commander Keen could be called an indie title by today's standards. The success put them in a position to make DOOM.
At least, I think that's how the history went. I could be wrong.
At least, I think that's how the history went. I could be wrong.
Everyone always skips Wolfenstein 3D
Ah, yes. FPS #0
People tend to seem to start from #1
People tend to seem to start from #1
Wolf3D wasn't even Carmack's first FPS. Before that there was Catacomb 3-D.
They're very underground, you've probably never heard of them.
Are those not games he developed?
John is one of the great engineering minds of our generation. Don’t forget Oculus.
He was years ahead of the curve [0] on mobile-first VR. By that I don't mean "stick a phone in Google Carboad," but truly self-contained and portable VR built on mobile chipsets. The success of the Quest and Quest 2 is proof of that, especially in the context of years of slow PC-only VR user growth.
[0] https://venturebeat.com/2016/10/07/oculus-john-carmack-argue...
[0] https://venturebeat.com/2016/10/07/oculus-john-carmack-argue...
Who doesn't know Carmack on HN?
Other people don't know all the things you do and vice versa. I would bet that less than half of the monthly active users on HN know who John Carmack is.
And I'd bet that half of those that do know who John Carmack is don't know he spent awhile building rockets, it's not exactly what he is known for.
I concur, I know about Carmack since I started coding games 15 or so years ago, we've even met in person. Yet, somehow, I didn't knew about Armadillo until this thread. Wow.
Thanks for sharing these obscure facts with us, man. I guess it does not harm. At the very least you made a lot of people giggle
In any case, it's funny he's still using the I'd handle on twitter. Whatever happened to that lawsuit
Finally somewhat on topic: I really thought they had a chance when armadillo won that NASA moonlander prize.
In any case, it's funny he's still using the I'd handle on twitter. Whatever happened to that lawsuit
Finally somewhat on topic: I really thought they had a chance when armadillo won that NASA moonlander prize.
> I really thought they had a chance when armadillo won that NASA moonlander prize.
Northrop Grumman Lunar Lander Challenge.
I'd really wish John would put fingers to the keyboard and write the history of Armadillo Aerospace. From buying a peroxide engine from Juan Lozano (?) to decision not to continue, even with that grand CNC behemoth as a birthday gift.
Northrop Grumman Lunar Lander Challenge.
I'd really wish John would put fingers to the keyboard and write the history of Armadillo Aerospace. From buying a peroxide engine from Juan Lozano (?) to decision not to continue, even with that grand CNC behemoth as a birthday gift.
There are no aerospace factories that aren't impressive. Like termite nests, whether they are builing anything useful or not is another matter.
You think John Carmack, a legendary engineer who had his own aerospace company and has likely toured others, is wrong that it's extremely impressive? Which would be relative to others.
The commenter is not questioning the impressiveness of the factory.
The commenter was making the assertion that Starbase is not exceptionally unique or an advanced outlier in its impressiveness.
ULA might be inefficient and seems likely to struggle to compete with SpaceX, but Destin Sandlin's (Smarter Every Day) tour of their factory is pretty freakin sweet.
It stinks like rotten meat, but looks like the lost Deimos base. Looks like you're stuck on the shores of Hell. The only way out is through.
It's funny that the Tesla engineers are sharing data with him, since they are suing repair shops that repair Teslas. Time for class action lawsuit, methinks.
John is going to Mars. Why?, you ask...
I wonder where Elon Musk would be without Twitter.
Still wildly successful, but man that platform is an amplifier.
Still wildly successful, but man that platform is an amplifier.
Wow Tony Stark and Cap America finally meet IRL!
Saved you a click: It's a photo of 100x developers with Elon
SpaceX will now be renamed Union Aerospace Corporation (like in Doom).
Somebody call Dwayne Johnson, now!
Ooh, this is a fun crossover episode.
What is China speed? Are the Chinese known for fast manufacturing?
China's dominance in manufacturing comes from speed and agility. Cheap cost of labor is a distant third benefit of manufacturing in China.
> Cheap cost of labor is a distant third benefit of manufacturing in China.
Increasingly distant as labor costs continue to soar in China that their demographic crisis will only exacerbate. China speed now is only possible due to a never ending debt stream that gets eternally refinanced.
Increasingly distant as labor costs continue to soar in China that their demographic crisis will only exacerbate. China speed now is only possible due to a never ending debt stream that gets eternally refinanced.
To me, "speed and agility" are code words for a culture that tolerates long work weeks and hours:
https://en.wikipedia.org/wiki/996_working_hour_system
With a society that had been dealing with famine, poverty, and labor opportunities consisting of subsistence farming in living memory, working 72 hours a week to receive a steady paycheck and a full belly seems like a pretty decent trade.
When you look at the history of the Industrial Revolution and the Western labor movement, I have to have doubts that this "speed and agility" will last. People who are prosperous start to have higher standards, especially as new generations enter the labor market. There is already significant pushback against the 996 system.
It's not hard to out-produce when the average person is working almost double the Western standard.
I thought the documentary "American Factory" was an interesting window into the lives of Chinese and American factory workers. My own analysis/opinion: the factory workers in China were indeed shown to be working quickly, obediently, and working long hours. The Americans looked lazy at times.
I thought the reasoning for this difference was that American workers used to union protections already had generations of knowledge about why they should not become wage slaves risking their health and safety for their capitalist employers, despite having few other options in a town where most of the factory jobs left.
I personally wonder when the typical Chinese worker will wake up to that same reality: that speed, agility, and fast innovation aren't really worth selling your weekends and joints and tendons away just so that the boss can buy a third vacation home.
https://en.wikipedia.org/wiki/996_working_hour_system
With a society that had been dealing with famine, poverty, and labor opportunities consisting of subsistence farming in living memory, working 72 hours a week to receive a steady paycheck and a full belly seems like a pretty decent trade.
When you look at the history of the Industrial Revolution and the Western labor movement, I have to have doubts that this "speed and agility" will last. People who are prosperous start to have higher standards, especially as new generations enter the labor market. There is already significant pushback against the 996 system.
It's not hard to out-produce when the average person is working almost double the Western standard.
I thought the documentary "American Factory" was an interesting window into the lives of Chinese and American factory workers. My own analysis/opinion: the factory workers in China were indeed shown to be working quickly, obediently, and working long hours. The Americans looked lazy at times.
I thought the reasoning for this difference was that American workers used to union protections already had generations of knowledge about why they should not become wage slaves risking their health and safety for their capitalist employers, despite having few other options in a town where most of the factory jobs left.
I personally wonder when the typical Chinese worker will wake up to that same reality: that speed, agility, and fast innovation aren't really worth selling your weekends and joints and tendons away just so that the boss can buy a third vacation home.
I mean, they built the world's largest high speed rail network pretty damn quickly.
That had always been the advantage of communism and dictatorships. When something needs to be done, you can just force people to do it.
Who was forced to work on the high speed rail? Do you have a source?
Maybe GP meant forced in the sense of giving right-of-way, which is arguably the biggest impediment to long distance rail construction. As of this January, California's High Speed Rail project still did not have all necessary right-of-ways [0].
[0] https://www.city-journal.org/high-costs-construction-delays-...
[0] https://www.city-journal.org/high-costs-construction-delays-...
Yes
In the sense that if you abandon all concern for safety, engineering, the environment, human rights, money and use concentration camp slaves for labor you can achieve massive scale public works very quickly. Yes I think Nazi China is known for it's speed.