Show HN: Run Unsloth Dynamic GGUFs using Docker model runner(github.com)
github.com
Show HN: Run Unsloth Dynamic GGUFs using Docker model runner
https://github.com/docker/model-runner
1 コメント
Will be around the next few hours
https://github.com/docker/model-runner
Key features:
Uses docker model run syntax.
Supports Dynamic GGUFs (handling quantization overhead).
Cross-platform support.
Example usage: docker model run ai/gpt-oss:20B
Happy to answer questions about the implementation or the quantization methods used.
https://docs.unsloth.ai/models/how-to-run-llms-with-docker