This difference between medical board examinations and real world practice is something that mirrors my real-world experience too, having finished med school and started residency a year ago.
I’ve heard others say before that real clinical education starts after medical school and once residency starts.
Does anyone know how this “user decides how much compute” is implemented architecturally? I assume it’s the same underlying model, so what factor pushes the model to <think> for longer or shorter? Just a prompt-time modification or something else?
I see a couple comments comparing llama.cpp and Ollama, and I think both have utility for different purposes. Having used both llama.cpp (which is fantastic) and Ollama, a couple things that I find valuable about Ollama out-of-the-box --
- Automatically loading/unloading models from memory - just running the Ollama server is a relatively small footprint; every time a particular model is called it is loaded into memory, and then unloaded after 5 mins of no further usage. It makes it very convenient to spin up different models for different use-cases without having to worry about memory management or manually shutting down those tools when not in use.
- OpenAI API compatibility - I run Ollama on a headless machine that has better hardware and connect via SSH port forwarding from my laptop, and with a 1 line change I can reroute any scripts on my laptop from GPT to Llama-3 (or anything else).
Overall, at least for tinkering with multiple local models and building small, personal tools, I've found the utility:maintenance ratio of Ollama to be very positive -- thanks to the team for building something so valuable! :)
I also had a similar thought process regarding understanding vs memorizing facts while transitioning from studying CS (where I emphasized understanding the underlying concepts rather than trying to memorize atomic facts that I could derive) to medicine (where having facts memorized is also a key component). Interestingly, I found that committing to memorizing facts actually helped me gain a deeper understanding of the topics themselves, which was not what I originally expected! (I wrote a little bit about the above a few months ago -- https://samrawal.substack.com/p/on-the-relationship-between-...)