Compressing and enhancing hand-written notes (2016)(mzucker.github.io)
mzucker.github.io
Compressing and enhancing hand-written notes (2016)
https://mzucker.github.io/2016/09/20/noteshrink.html
13 comments
Yes. Pandemic professor here, often grading take-home exams. The hardest to look at have uneven illumination, making removing the background dependent on position.
I've thought often of writing something like this code.
For my own writing I've moved to an iPad and an Apple Pencil. Concepts has become my favorite app for algorithmic drawing, though it's not without frustrations. I found it necessary to script converting an infinite canvas to PDF pages: https://github.com/Syzygies/concepts-artboards
Ideally, one wants to write scripts for cleaning up hand drawn notes in an efficient language, and let it run overnight. Better background detection is the tip of the iceberg; one wants to employ machine learning algorithms to infer what you would have drawn with better motor control. This could then be used to translate computer-generated animation into hand-drawn animation, exactly as if there were a thousand of you working for Disney in the time of Fantasia.
I've thought often of writing something like this code.
For my own writing I've moved to an iPad and an Apple Pencil. Concepts has become my favorite app for algorithmic drawing, though it's not without frustrations. I found it necessary to script converting an infinite canvas to PDF pages: https://github.com/Syzygies/concepts-artboards
Ideally, one wants to write scripts for cleaning up hand drawn notes in an efficient language, and let it run overnight. Better background detection is the tip of the iceberg; one wants to employ machine learning algorithms to infer what you would have drawn with better motor control. This could then be used to translate computer-generated animation into hand-drawn animation, exactly as if there were a thousand of you working for Disney in the time of Fantasia.
That sounds very useful.
I see that the scans pick up a lot of writing on the back side of the page. This is usually because the scanner lid has a white background. I accidentally found that to avoid picking up the back-side writing, you should either scan with the lid open in a dark room, or use a lid with a black background.
I've found it really interesting and it works great to also copy documents without altering quality.
About notes, there is one thing that bothers me. It's highlighted text. Colors are clear and are always filtered. Blue and green highlighter are ok, but pink or yellow ones are another story. That's inherent to the method used and as far I know (and tried) can't be fixed with adjusted thresholds.
It's nonetheless a great tool and I've automated its use for every scan I do.
About notes, there is one thing that bothers me. It's highlighted text. Colors are clear and are always filtered. Blue and green highlighter are ok, but pink or yellow ones are another story. That's inherent to the method used and as far I know (and tried) can't be fixed with adjusted thresholds.
It's nonetheless a great tool and I've automated its use for every scan I do.
Yes. Compression and recognition are two sides of the same coin. A serious compression algorithm adapted to hand-drawn notes would evolve a spatial model for the background, and be forced to spawn subprocesses for highlighted background. Normalize, and one gets clean highlighting.
I ran noteshrink in order to get a farewell notebook put together for a colleague with handwritten notes sent in via scan/photo soon after the lockdown started.
you need somewhat decent scans to get this working (but when it works it works amazingly well!) - photographs of a piece of paper are very finicky to get correctly detected - it would often remove other relevant parts or keep too much in... I think with a live preview it would have made my life of working with a broad range of scan methods/lighting/consistency easier... but it's volunteer work and not "live preview" fast computationally, so I wouldn't call it a complaint.
you need somewhat decent scans to get this working (but when it works it works amazingly well!) - photographs of a piece of paper are very finicky to get correctly detected - it would often remove other relevant parts or keep too much in... I think with a live preview it would have made my life of working with a broad range of scan methods/lighting/consistency easier... but it's volunteer work and not "live preview" fast computationally, so I wouldn't call it a complaint.
Thanks a lot for reposting! I remember seeing this on hn year ago. I had searched for it a lot but without a luck. Bookmarked!
When looking at the clustering, I'm asking myself if a density -based clustering algorithm (looking at you DB-Scan) would be also interesting. While loosing the "center color" information, the clusters could be used to replace the color with whatever color one wanted.
This is fantastic, I would only suggest one more thing, yesterday there was article on first page about vectorization, so further compression and cleaning could be done if text would convert to vectors, links is:
https://mzucker.github.io/2018/05/14/maptrace.html
but that is also from same person so I guess they know if it would be improved or not by that technique.
In fact, the vector maps article contains this text:
> (Hmm, reminds me of my noteshrink project…)
...so the author did, in fact, explicitly consider this possibility.
> (Hmm, reminds me of my noteshrink project…)
...so the author did, in fact, explicitly consider this possibility.
So first article is from 2016, and vector one from 2018, 3 years later there is no explicit article combining two, but I get it if you want it, do it on your own.
I did missed that he mentioned it, but it is still quite interesting as I use a lot GoodNotes on iPad.
This made photos much more legible and printable.