Only problem is I don't know how likely it would be that the writing is neatly separated by character for mnist to work, and handwriting recognition isn't accurate enough. Maybe some restraints on the inputs will fix that
Would you mind quickly explaining what you guys mean? Sorry, I'd love to improve it but I'm not sure I get it. If you had a bunch of bubbles for each square and piece etc. wouldn't you be taking up much more space than 5 boxes? And it's also a huge notation change for everyone
No, not a clear reason other than wanted to get the product out earlier. We'll work on an opt-in, that does sound the best.
Wouldn't other pieces affect this though? For example, what if the move was e6 and the next e-pawn move was e7. What if a piece was already there, Ne7 (wrongly recognized)? I thought about slowly running through it like this, but it felt like I kept running into problems which are obvious for us to recognize but I wasn't sure how to implement it computationally
Yes, that is the plan. That should heavily increase accuracy. Do you mean handwriting recognition as opposed to character recognition? It wasn't that we tried it and decided it wouldn't work, I just couldn't find previous instances of accurate enough recognition with handwriting. Accuracy was my priority. My sense was also that people would be turned off if they had to still manually input some games, and I had no idea how many manually inputted games it would take to reach good enough accuracy. This was when I hadn't come up with constraining characters and such, but now I can see that (with some attempts at handwriting recognition) that may be possible / is the next step after an app.
We currently don't store scoresheets when uploaded to our website.
Would you be able to send the pictures to us at [email protected]? You could add comments about how it went (or not, since we can just run it and see for ourselves). If you think that might be a hassle I could quickly add a feedback form on the website.
Also, I'd encourage that you try it once before the tourney with the restrictions—we haven't tested on very varied handwriting. Oh, and thanks so much for offering feedback!
P.S. Did you have something in mind for improving the postprocessing / trying out different iterations? Or was it just a challenge :D
The model actually gives us a confidence for each possibility. It might say, N-90%, K-10%, B/R/Q-0% for one of the boxes. Here, we look at how confident our character recognizer is on each character. If the PGN is invalid, then we know some character was recognized incorrectly. We look at the low-confidence characters and change them to the next highest confidence prediction, checking to see which combination of changes delivers a valid PGN. For example, consider the string of moves 1. d4 d2. Black can't play d2 on move 1! But we look at the next most likely predictions, perhaps d8 for white and d5 for black. These are the combinations:
d4 d2
d4 d5
d8 d2
d8 d5
Only the second one is valid, so we choose it and continue ahead.
If you literally wrote "d4 d2," then it's a lot less likely that the correct digit instead of 2 will be in the top 3 predictions.
I think you'll be pleasantly surprised, this is a thing!! I thought about making it but quickly decided computer vision wasn't up to the task yet. It seems I was horribly wrong: https://github.com/maciejczyzewski/neural-chessboard
Haven't tried it myself but it looks amazing
Also, a slight problem with b) is that it must be in the context of the whole game, since incorrect moves may only arise later on. This makes it harder with characters too ofc