My most memorial bugfix was when I was on a team ,temporary ,that did email encryption/decryption.
They had one customer where some mails could not get decrypted, they had been figthing with this for one year, no one could figure out what was going on.
I told them to do a dump for a week with the good and bad emails.
After one week I was given the dump of files, looked at the count of bad vs good, did some math in my head and said:
"Hmm, it appears that about 1/256 mails is bad.That could indicate that the problem is releated to a specific byte having a specific value in the random 256 bit AES key.
If there is a specific value giving problems it is probaly 0x00 and the position I would guess being at the last or first byte."
I did a check by decoding all SMIME mails to readable text with openssl- sure, all bad emails had 0x00 as the least signicant byte.
Then i looked at asn1 spec and discovered it was a bit vague about if the least significant byte had to be there if it was 0x00.
I inserted a line into the custom written IBM 4764 CCA driver written in c called by JNI.
Then all emails decrypted.
The team dropped their jaws- they had been figthing with it for 1 year and I diagnosed the bug only by looking at the good/bad ratio :)
I might remember some details wrong- but the big picture is correct :)
I did a check by decoding all SMIME mails to readable text with openssl- sure, all bad emails had 0x00 as the least signicant byte. Then i looked at asn1 spec and discovered it was a bit vague about if the least significant byte had to be there if it was 0x00. I inserted a line into the custom written IBM 4764 CCA driver written in c called by JNI. Then all emails decrypted.
The team dropped their jaws- they had been figthing with it for 1 year and I diagnosed the bug only by looking at the good/bad ratio :)
I might remember some details wrong- but the big picture is correct :)