I'll be damned, I feel like I've been using GA since forever!
You're right though, just re-listened to the segment[0] and the ex-CEO mentions they were initially using AWS, then moved to their own servers because of the limitations of AWS at the time and their particular needs. Github Actions did however always run on Azure!
> And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run.
I mean, you could argue that if you take into consideration all the generations (starting from the first amoeba) that it took to get to a standard human brain today, then the total energy used to "train" that brain is far greater. But I get your point and I do agree with you that our current hardware paradigm is probably not what's going to give us "god in a box".
In the Pragmatic Engineer podcast episode with the former CEO of Github, the latter mentioned that they had their own infra for everything. If I remember correctly, this was due to the fact that Github is quite old and at the time when Github Actions became a thing, cloud providers were not really offering the kind of infra that was necessary to support the feature.