I feel like both this comment and the parent comment highlight how RL has been going through a cycle of misunderstanding recently from another one of its popularity booms due to being used to train LLMs
I used to follow advancements in RL pretty closely since 2016; it’s cool to see how far methodologies and algorithms have come to be able to do complex tasks completely offline. Dreamer 4 is another big leap in the Dreamer series
I didn’t see this before, but I work at a non-profit, government hospital system. So increasing profits, although probably a good answer most of the time, is probably not as applicable in this case lol
Along the same lines of missing good junior engineers at work, we occasionally interview stellar engineers that’ve inflated their resume a bit to get an interview, but we end up rejecting them for not having all the specific experiences our manager wants them to have even though they’re generally great and could clearly upskill where necessary. No wonder we can’t grow the team when we’re out here looking for unicorns
Not sure about other hospital systems, but the one I work at is developing CV systems to help fill workforce gaps in places where there isn’t as many trained professionals or even resources to train professionals
This article is pretty good. My current work is transitioning CV models in a large, local hospital system to a more unified deployment system, and much of the content aligns with conversations we have with providers, operations, etc..
I think the part that says models will reduce time to complete tasks and allow providers to focus on other tasks is on point in particular. For one CV task, we’re only saving on average <30min of work per study, so it isn’t massive savings from a provider’s perspective. But scaled across the whole hospital, it’s huge savings
It’s ironic that HN threads, arguably one of the forums where the majority of users should understand a tech company and its tech, about Palantir always devolve into some weird speculative and conspiracy-like discussion. Palantir’s docs are pretty open too - it’s not like it’s a black box that you can only see if you have a contract with them. So one would think the HN crowd would know something and have an interesting discussion on how it compares to what they’ve seen, etc. But it somehow always turns mostly political and less about the tech
Along with burning tokens, how MCP servers are ran and managed is resource wasteful. Running a whole Docker container just to have some model call a single API? Want to call a small CLI utility, people say to run another Docker container for that
Right, this is only power usage. Factoring in labor and all that would make it more expensive for sure. However, it’s not like it’s a complex system to maintain. We use a popular inference server and just run it with some modest rate limits . It’s been hands-off for close to a year at this point
Some anecdotal data, but we recently estimated the cost of running a LLM at $WORK by looking at power usage over a bursty period of requests from our internal users and it was on the order of $10s/mil tokens. And we arent a big place, nor were our servers at max load, so I can see the cost being much lower at scale
I was wondering this myself and made a tool-calling endpoint with OpenAPI to see if it worked well. Turns out it does: https://github.com/theOGognf/toi
It’s funny seeing this blog post again. This is actually a reference I used to make a poker game as a state machine last year: https://github.com/theOGognf/private_poker
It made the development feel a lot safer and it’s nice knowing the poker game state cannot be illegally transitioned with the help of the type system