Cool to see that this worked well for someone. Super hard to force the key insight in a problem to magically appear given more time sunk into it. Big weakness of mine honestly, and requires a lot of self-awareness to pull myself out of a problem-solving rut. I like the idea of hacking sleep - do you find yourself priming your mind with the problem before nodding off? Curious how a bedtime wind-down routine factors into how effective this is.
Very cool. I used to dream of this stuff when I was younger. Reminds me of Atlantik Solar: https://www.atlantiksolar.ethz.ch/. Hasn't been updated in a while, but focused more on low-altitude autonomous survey missions.
> I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedback.
I guess if we ignore pretraining, don't sample-efficient fine-tuning on carefully curated instruction datasets sort of achieve this? LIMA and OpenOrca show some really promising results to date.
Yep, batching is a feature I really wish the OpenAI API had. That and the ability to intelligently cache frequently used prompts. Much easier to achieve this with a hosted OS model, so I guess it's a speed + customizability/cost tradeoff for the time being.