Selecting the "best" local model usually depends on the task and the hardware.
I created this script as an easy way to test local Ollama models and keep the test output organized.
When you run the script interactively, it asks which model you want to use, what your prompt is, how many times you want to run it, and (optional) the temperature you'd like to set. It can also be scripted with command-line flags.
The output is saved in Markdown/JSON within an organized file structure for easy comparison. Outputs using the same prompt go into a folder together, each output named for the model tested. Timing data and token counts are also recorded.
The tool is intentionally small and dependency-free (standard library only).
Suggestions welcome.
0 comments
—
Show HN: OMT – A simple Python CLI for testing local Ollama models · HackerTrans
I created this script as an easy way to test local Ollama models and keep the test output organized.
When you run the script interactively, it asks which model you want to use, what your prompt is, how many times you want to run it, and (optional) the temperature you'd like to set. It can also be scripted with command-line flags.
The output is saved in Markdown/JSON within an organized file structure for easy comparison. Outputs using the same prompt go into a folder together, each output named for the model tested. Timing data and token counts are also recorded.
The tool is intentionally small and dependency-free (standard library only).
Suggestions welcome.