Funny how if you kept reading before commenting, they addressed that point specifically
> We were cautious to only run after each model’s training cutoff dates for the LLM models. That way we could be sure models couldn’t have memorized market outcomes.
Very cool project! It would be really nice to have support for the other assistants that Microsoft released to use in place of Clippy (I'm particularly fond of the dolphin that was used in the Japanese version of Windows) https://en.wikipedia.org/wiki/Office_Assistant#Assistants
> We were cautious to only run after each model’s training cutoff dates for the LLM models. That way we could be sure models couldn’t have memorized market outcomes.