Yes it just collapses eventually — never stabilizes.
The training process is flawed, I suspect it has to do with the fact that some weights blow up over time, you can see in “weights” tab.
But at around 4K avg score you should see it solve the env almost every time.
Just a demo :) optimized for speed over stability.
Reward structure: Step: -1 Dot: +100 Win: +1000
so ~4k is max theoretical score on 6x6.