The harmony format thing is quite interesting. It seems to have been trained to follow certain things in certain "channels". First comes a layer of "analysis" then "commentary" then "analysis" then "final". What's interesting is if this turns out to be helpful for mitigating some injection attacks, where instructions are peppered through data ingested by tool calls. I can see a world where the model is trained to only respond to instructions in the "user" channel, or otherwise ignore any instructions from "tool_response" channel.
Interesting that the license seems to be Apache 2.0. If the model is open as well this is above my expectations. Let's see how this goes.
If I read the details correctly the model will be released with 4bit quants (only?) and the big one should fit in 80GB GPUs while the small one should fit in 24GB consumer GPUs (maybe 16? not sure, depending on context)
Transformers 4.55 New OpenAI GPT OSS · HackerTrans
Interesting that the license seems to be Apache 2.0. If the model is open as well this is above my expectations. Let's see how this goes.
If I read the details correctly the model will be released with 4bit quants (only?) and the big one should fit in 80GB GPUs while the small one should fit in 24GB consumer GPUs (maybe 16? not sure, depending on context)