First, thank you for your teaching, it has helped me a lot, didn't think I'd ever have the chance to say thank you, but here you are and I hope this gets to you!
Question - what's a relevant (05-2024) baseline to compare the performance of c code to? Back when you made nanoGPT you were seeing "the file train.py reproduces GPT-2 (124M) on OpenWebText, running on a single 8XA100 40GB node in about 4 days of training". So twice the memory on the c node, but unsure of data size /epochs, any other details I may be missing. I.e. what's the net uplift of running c vs "legacy" torch code?
I find these two concepts [1, 2] at odds with each other. Not a critique on the author - on the contrary, empathy: I felt the same when applying to YC (did not make a batch).
On one hand, the general impression you get when preparing for your application (via FAQs, Startup School, YC videos, etc) is very much in line with [1] - YC is looking for _very_ early stage.
But once you go through the actual application you feel focus shift towards [2] - metrics and $. That is to say (with admittedly some not-having-been-selected bias), I feel [2] is a significant factor in deciding on applications. So as I weigh in on whether to apply for the next batch, I'm not sure whether a product I've just finished building makes sense for YC and whether I should gamble on attempt #3.
I think it would help both YC and founders if they take some steps to make this clear(er) for potential applicants.
[1] > In general, there is an evident focus on the very early stage without a product. The main theory and advice are about how to figure out what to do, how to build an MVP, how to launch, how to talk to customers, where to find the first 10 customers, how to raise the first money, and so on. Needless to say, for companies with tens or even hundreds of thousands in revenue it won’t be very valuable.
[2] > [...] present dry facts—how much money customers already paid you, what the size of the market, if you count all the units you can sell, what you have actually built and what is working today. And this will always sound bad for anyone, it just can't sound good in the early days.
- responded very thoroughly about Tianmen square
- ditto about Uyghur genocide
- “knows” DJT is the sitting president of the US and when he was inaugurated
- thinks it’s Claude (Qwen knows it’s Qwen without a system prompt)
So it does seem to work in steering behavior (makes Qwen’s censorship go away, changes its identity / self, “adds” knowledge).
Pretty cool for steering the ghost in the machine!