QR error correction helps and hinders scanning(huonw.github.io)
huonw.github.io
QR error correction helps and hinders scanning
https://huonw.github.io/blog/2021/09/qr-error-correction/
42 comments
One thing that really annoys me with the use of qr codes (which are more often used due to the pandemic) is that many qr codes don't include the quiet zone required to scan the code. Instead they expect that the app the code is displayed in has a white background. But this doesn't work if you get a qr code via email and have dark mode active. In the first year of the pandemic I often had to fiddle with my mail app because of it, now I just have my app set to light mode. Generate correct QR codes, people!
Well, the very concept of the quiet zone is rather unworkable for mass uses. QR code in particular has a relatively large quiet zone (8/29 = ~28% at the smallest size) in spite of also having multiple large finder patterns. If QR code can't function without that large quiet zone, I think it should have extended timing patterns so that the "quiet" zone actually has some pixels (modules in the QR terminology) set so users are expected not to cut the corner. A similar strategy has been used by EAN-13 where the quiet zone is indicated with a placeholder character `>` underneath.
In practice, I find that the popular ZXing can scan QR Codes very reliably with a 1-module quiet zone, even though the standard requires 4 modules. That said, 0 modules is universally a bad idea; putting a black border immediately around a QR Code is a recipe for failure.
If you are transferring information from one device to another, perhaps a QR code (or barcode) is not the right way to do it to begin with.
I mostly agree, but we can't easily change the fact that people are using two-dimensional barcodes for transferring a small amount of information anyway.
You’re even understating the impact of the quiet zone: at version 1, the QR code itself is only 52% of the code + quiet zone area (21**2 modules out of 29**2). Almost twice as bad! :)
That said, in practice, a quiet zone of one module or even only partial quiet zones (around the position patterns) seems to work: https://qrworld.wordpress.com/2011/08/09/the-quiet-zone/
That said, in practice, a quiet zone of one module or even only partial quiet zones (around the position patterns) seems to work: https://qrworld.wordpress.com/2011/08/09/the-quiet-zone/
Ah, yes, it is even worse when you compare areas. I think the quiet zone is a recommendation and not a strict requirement, so being able to reduce or share the quiet zone is not very surprising. I'm more curious that the valid QR code is bounded with finder patterns so the quiet zone doesn't seem necessary (those patterns are optimized for linear scanning by maintaining 1:1:3:1:1 black-white-b-w-b ratios across all axes, so you don't need any additional quiet zone in principle).
You could write a "QR Quiet" utility which, if given a QR code image with the quiet border missing, generates a new image which has that border. :)
On URL Versions: I find it odd that governments (eg Australia) that mandate QR codes to check-in to places have such complex information and detailed codes. This makes scanning the QR code more difficult due to the smaller dot size. This results in people huddling around a single point at a door for longer and closer proximity.
A little tech love such as using short URL’s with an intermediate backend that redirects to the complex site URL with a dozen query parameters would allow much lower resolution (lower version) QR codes that can be scanned faster and/or from a further distance away resulting in better health outcomes. Or at least minimise the negative health outcome from the scanning process.
A little tech love such as using short URL’s with an intermediate backend that redirects to the complex site URL with a dozen query parameters would allow much lower resolution (lower version) QR codes that can be scanned faster and/or from a further distance away resulting in better health outcomes. Or at least minimise the negative health outcome from the scanning process.
That’s actually exactly the topic of my next post, and the reason I even started looking at this. :)
As the sibling says, there’s some data that is useful to store in every code to enable user-friendly offline check-ins, but they could indeed be much easier to scan and still retain this property.
As the sibling says, there’s some data that is useful to store in every code to enable user-friendly offline check-ins, but they could indeed be much easier to scan and still retain this property.
Have you deciphered the metadata that is associated with the url?
It is odd that we are looking at a paragraph of metadata on a url, however a manual check in is six alphanumeric characters with the Victorian government's system.
We know that none of this data ever changes, these sheets are printed once and are not actively replaced
We know that none of this data ever changes, these sheets are printed once and are not actively replaced
Yeah, the NSW ones contain a base64-encoded JSON blob storing: the id of a business (a short numeric string), the business name (shown in the app even during an offline check-in) and the business address.
I haven’t investigated the Victorian ones deeply, but I would imagine that the name is only used as a confirmation for the user, and a minimal or manual check-in requires only the ID.
I haven’t investigated the Victorian ones deeply, but I would imagine that the name is only used as a confirmation for the user, and a minimal or manual check-in requires only the ID.
Also phones scan CAPS URL’s just fine. So all you need is:
HTTP:// <SHORT DOMAIN> / CODE
Dropping S in HTTPS gives one extra character if you do a 302 redirect. This is fast enough if the server is hosted in same country (which it should be for health data).
6 alphanumeric dropping confusing O0I1 gives 32 chars. 4 digits allows 1M businesses. 5 is 32M. Probably enough.
To fit Version 1 (lowest possible resolution 21x21) using only capital letters including colon and forward slash, you have 25 chars available for low ecc, and 20 chars for medium.
Since the QR code is such low simple resolution we can use Low ECC.
That allows for a domain ending with .state.gov.country (assuming state is 3 chars, country is 2). For example:
HTTP://Q.NSW.GOV.AU/ABCDE
There’s your 32M codes available as a Version 1 QR.
HTTP:// <SHORT DOMAIN> / CODE
Dropping S in HTTPS gives one extra character if you do a 302 redirect. This is fast enough if the server is hosted in same country (which it should be for health data).
6 alphanumeric dropping confusing O0I1 gives 32 chars. 4 digits allows 1M businesses. 5 is 32M. Probably enough.
To fit Version 1 (lowest possible resolution 21x21) using only capital letters including colon and forward slash, you have 25 chars available for low ecc, and 20 chars for medium.
Since the QR code is such low simple resolution we can use Low ECC.
That allows for a domain ending with .state.gov.country (assuming state is 3 chars, country is 2). For example:
HTTP://Q.NSW.GOV.AU/ABCDE
There’s your 32M codes available as a Version 1 QR.
That's a cool exercise! :)
However, it loses some of the niceties for offline check-in (the business name, in particular), without an in-app data source.
And, unfortunately, it low resolution doesn't help that much with error correction: v1 does seem to be slightly easier to scan, but only a little, and the ratios of codes scanned is pretty much the same across all versions:
By this table, the most-scannable encoding under the experimental conditions from the post for that data is ec level H, which requires version 3 (without considering the field-of-view required).
However, it loses some of the niceties for offline check-in (the business name, in particular), without an in-app data source.
And, unfortunately, it low resolution doesn't help that much with error correction: v1 does seem to be slightly easier to scan, but only a little, and the ratios of codes scanned is pretty much the same across all versions:
ec_level L M Q H
version
1 38 46 54 65
2 32 39 49 52
3 32 41 43 50
4 32 40 46 49
5 30 39 42 49
...
39 32 41 46 50
40 31 41 47 51
total 1257 1600 1839 2001
By this table, the most-scannable encoding under the experimental conditions from the post for that data is ec level H, which requires version 3 (without considering the field-of-view required).
Have a look at the ACT's QR code check-in system. It's a simple 8 digit (I think) number that needs to be scanned with the ACT's check in app.
What I would love is poster sized QR codes instead of a small patch of an A4 page. That way you could scan it from afar.
What I would love is poster sized QR codes instead of a small patch of an A4 page. That way you could scan it from afar.
it will not work on devices without internet
What I always hate is: at a bus stop I see a poster for, let's say, a museum which includes a QR code to visit their site:
https://i.imgur.com/SHEmEPn.gif
I already know this is a URL with tracking parameters etc. As it encodes so much data and I am in a hurry, it takes my phone forever to scan.
Meanwhile, this QR code would have led me to their site as well:
https://i.imgur.com/IyxYyZN.gif
https://i.imgur.com/SHEmEPn.gif
I already know this is a URL with tracking parameters etc. As it encodes so much data and I am in a hurry, it takes my phone forever to scan.
Meanwhile, this QR code would have led me to their site as well:
https://i.imgur.com/IyxYyZN.gif
It seems to be obscure that QR codes have an "alphanumeric" character set that includes everything you need to make URLs IF you use only uppercase characters. This increases the data density considerably.
I make URLs that look like
https://gen5.info/$/XQ*42RXF-TLY:$B.8/
(That is the digital twin of an installation on the wall behind me as I type, each square is 8 by 8 inches; the barcodes on the back of the cards are NOT high density, three-sided cards from that generation that link to external sites link directly to low density URLs. New generation three-sided cards use "QRLs" that redirect to external links.)
which are "version 2" QR codes that are larger only than "version 1" QR codes which don't have the little alignment dot in the lower right and don't scan as well. The URL above uses M level error correction, with L level error correction I can make URLs that are a bit longer.
I manage either length of code much like a type 4 UUID and in the later case the space is large enough that I (you and 8 billion of my closest friends) could make every URL unique and be perfectly able to track people at the expense of keeping a database entry for every unique code.
My take is that the degree of compression doesn't matter much in my case because with high density QR codes you can print them large enough that barcode readers have no problem with them.
My experience is that the QR code reading experience on iPhones is night-and-day better than that on cheap Android devices. I think a lot of people are testing barcodes on iOS and have no idea that they scan poorly or not at all for other uses. For one thing, any camera has a minimum depth that it focuses at and if you print a QR code too small you have to hold the camera close to the code to read it and the camera can't focus. Another problem is that most QR code reading programs just don't read white-on-black codes even if the standard allows them. For that matter, many QR code readers require much higher contrast than the standard specifies, and even the standard is conservative compared to what is possible with a generously sized code.
I make URLs that look like
https://gen5.info/$/XQ*42RXF-TLY:$B.8/
(That is the digital twin of an installation on the wall behind me as I type, each square is 8 by 8 inches; the barcodes on the back of the cards are NOT high density, three-sided cards from that generation that link to external sites link directly to low density URLs. New generation three-sided cards use "QRLs" that redirect to external links.)
which are "version 2" QR codes that are larger only than "version 1" QR codes which don't have the little alignment dot in the lower right and don't scan as well. The URL above uses M level error correction, with L level error correction I can make URLs that are a bit longer.
I manage either length of code much like a type 4 UUID and in the later case the space is large enough that I (you and 8 billion of my closest friends) could make every URL unique and be perfectly able to track people at the expense of keeping a database entry for every unique code.
My take is that the degree of compression doesn't matter much in my case because with high density QR codes you can print them large enough that barcode readers have no problem with them.
My experience is that the QR code reading experience on iPhones is night-and-day better than that on cheap Android devices. I think a lot of people are testing barcodes on iOS and have no idea that they scan poorly or not at all for other uses. For one thing, any camera has a minimum depth that it focuses at and if you print a QR code too small you have to hold the camera close to the code to read it and the camera can't focus. Another problem is that most QR code reading programs just don't read white-on-black codes even if the standard allows them. For that matter, many QR code readers require much higher contrast than the standard specifies, and even the standard is conservative compared to what is possible with a generously sized code.
It's even better when the super-complicated QR code is the size of a printed page pasted on the side of a car or truck, gee I wonder how hard those would be to scan when I'm a passenger in a car dozens of feet away, and both me and the truck are moving around.
> I'm a passenger in a car dozens of feet away, and both me and the truck are moving around.
That's the least common use case. They actually work well for their intended use cases, which are (a) a passenger scanning it from a care a few feet away moving at relatively the same speed (such as on the highway), and (b) the code being scanned while waiting at a stop light (maybe dozens of feet away).
That's the least common use case. They actually work well for their intended use cases, which are (a) a passenger scanning it from a care a few feet away moving at relatively the same speed (such as on the highway), and (b) the code being scanned while waiting at a stop light (maybe dozens of feet away).
Another interesting effect: If you have a version 1 code (the smallest size), there is no "alignment" block in the lower-right corner - only the main three positional blocks. Some scanning algorithms (such as the popular zxing library) struggle with this, and it actually works better to use a version 2 code.
Cool, nice tip.
The lowest level EC is the best for when you are building a QR code generator in Minecraft, trust me
Nice post, learned new things about QR codes. I noticed in the wild when doing check-ins, unreliable QR codes have been due to glossy paper and/or their placement, either being too high or low causing distortion. https://datagenetics.com/blog/november12013/index.html illustrates the impact of different destruction cases
I'm surprised I haven't seen NFC tags being used for covid check-ins as a secondary option to QR codes
I'm surprised I haven't seen NFC tags being used for covid check-ins as a secondary option to QR codes
Thanks!
Yeah, glossy paper (or glass) leading to reflections can be bad. The worst one I’ve seen is printed at A5 size (on an A4 page…) and then placed behind glass, outside. Basically useless!
I would imagine that NFC requires too much actual hardware, whereas printing is cheap and common.
Yeah, glossy paper (or glass) leading to reflections can be bad. The worst one I’ve seen is printed at A5 size (on an A4 page…) and then placed behind glass, outside. Basically useless!
I would imagine that NFC requires too much actual hardware, whereas printing is cheap and common.
Has someone made a QR code that is a random dot stereogram and which encodes something useful? Like the source code of the program which generates it or something. :)
The obvious best-tool-for-the-job language here is blazon. Here's a rough POC:
checky argent and sable
5 square voided in saltire palewise argent
12 square inverted sable in dexter chief a square voided fesswise argent
on a plain cross escartelly checky sable and argent 3 square voided fesswise inverted argent
in third quarter 3 square voided in bend sinister argent
in first quarter 3 square voided in bend argent
import qrcode
with open(__file__) as f:
img = qrcode.make(f.read())
img.show()
Something like a quine?
https://www.quaxio.com/qrquine/
https://www.quaxio.com/qrquine/
This is in the same area of exploration, except that my main requirement is that, visually, the QR code is a random-dot stereogram. If you align your eyes the right way while staring at it, a 3D image appears.
Other requirements, like that it decodes to something that is meaningful, are secondary.
Alok's quine work is interesting. The actual quining is trivial because the program pulls out its own image using Location.href. (The trick seems to be that when JS code is embedded in a literal "data:" URL, then the Location.href can pull out that URL itself. I'm not a JS person, though.) It looks like the hard work in this project was developing the QR code encoder using a verbose OOP approach, and then code golfing it through 25 steps to make it small.
Other requirements, like that it decodes to something that is meaningful, are secondary.
Alok's quine work is interesting. The actual quining is trivial because the program pulls out its own image using Location.href. (The trick seems to be that when JS code is embedded in a literal "data:" URL, then the Location.href can pull out that URL itself. I'm not a JS person, though.) It looks like the hard work in this project was developing the QR code encoder using a verbose OOP approach, and then code golfing it through 25 steps to make it small.
Note that there are some areas that may not be damaged or the code is unreadable. The author avoided the big squares that are obvious to the eye and are required for positioning but there are also timing lines and metadata areas that are required for decoding.
The timing lines are not critical; timing can be derived from the data itself. Metadata areas (mask, error correction level, version) are stored twice and protected with error detection bits.
I did not know that there was that much redundancy in the metadata but there is. Still, I think damaging the timing areas causes problems. All the ‘creatively damaged’ QR codes I’ve seen avoid these as well as the positioning areas.
Of course it’s not too useful to speculate about what’s required beyond adherence to the specification, it depends on what the scanners your users use accept.
Of course it’s not too useful to speculate about what’s required beyond adherence to the specification, it depends on what the scanners your users use accept.
TLDR: Assuming a fixed qr code size, more resiliency to noise (mud on image, etc) requires putting more information in the code. To fit this the boxes need to be smaller, so that's harder to read. But if you're in a noisy environment and actually need the error correction it's still worth it.
What also makes scanning harder is people adding logos in the center "using up" some of the error correction.
My company's logo is a small black square.
My company's logo is three black squares, encased in black rectangles, arranged at the first three corners, in reading order, of an imaginary square with four times the height of one of the black rectangles.
Your company's name must be Finder. ;-)