Indeed, consistency (with the lengthy dialogue histories, but also the game state as well as game plans) was a huge challenge for us. We spent a lot of time working on techniques for detecting and filtering these kinds of low quality messages. You can see the Supplementary Materials in the paper for full details, but TL;DR: we built a suite of classifiers for detecting common mistakes (training classifiers to discriminate between human messages and counterfactuals), and used these classifiers as an ensemble which acted as a filter on top of message generation.