Hey, glad that you appreciate it! Model version, yes you can either overwrite with the --model and --judge-model you can overwrite at runtime or in the yaml spec.
For the seed: no I haven't added it, and I don't think the harnesses like claude code or Codex supports it nowadays. (You're right OpenAI API exposes a best-effort seed parameter though)
But TBH I was not trying too much to control the LLM itself. I remember reading nice work by thinking machines on it. Here I decided to focus way more on pass@k because I'm focusing on a "classic" usage that is not trying too hard to play with seed or temperature, just the model with sane defaults.
If you feel that could be useful, please file me an issue and I'll consider adding it :)
For the seed: no I haven't added it, and I don't think the harnesses like claude code or Codex supports it nowadays. (You're right OpenAI API exposes a best-effort seed parameter though)
But TBH I was not trying too much to control the LLM itself. I remember reading nice work by thinking machines on it. Here I decided to focus way more on pass@k because I'm focusing on a "classic" usage that is not trying too hard to play with seed or temperature, just the model with sane defaults.
If you feel that could be useful, please file me an issue and I'll consider adding it :)