Forget JPEG, How Would a Person Compress a Picture?(spectrum.ieee.org)
spectrum.ieee.org
Forget JPEG, How Would a Person Compress a Picture?
https://spectrum.ieee.org/forget-jpeg-how-would-a-person-compress-a-picture
7 comments
In pondering this as a real world application/solution, in which we humans describe a photo, save the description as text, and then rely on AI to render the image.... I could see this being like a game of telephone. Some people will describe the photo very well (as I believe was done to generate the middle image in the article), whereas some people will be very non-descript with something blunt like "my cat", which will then be rendered veeery differently.
I know image captioning is already a huge field and probably someone has worked on the reverse (though that's harder as the output isn't well constrained). Combining the two you will have a semantic image compressor.
See for instance DALL-E: https://openai.com/blog/dall-e/
I’ve wondered when we’d start having “AI” compression algorithms that take into account the subject matter over what we have now. JPEG, for example, just takes each 16x16 pixel block and throws away high frequency information with no regard for what that information was.
Very fascinating, but as they said cannot be compared because they rely on different requirements.
It only works because the model allready knows what a 'giraffe' is. It's pre-stored. But if one had to describe a giraffe, one needs to rely on other things which we take for granted that humans know (like what hair and eyes look like).
By that standard I can say that a picture of albert einstein only takes like 30 bytes if i assume the recipient knows what he looked like.
It only works because the model allready knows what a 'giraffe' is. It's pre-stored. But if one had to describe a giraffe, one needs to rely on other things which we take for granted that humans know (like what hair and eyes look like).
By that standard I can say that a picture of albert einstein only takes like 30 bytes if i assume the recipient knows what he looked like.
Just like sending a compressed jpeg only works because it assumes the recipient knows the decompression algorithm.
"the describers used text-based messaging and, crucially, could include links to any publicly available image on the internet"
That is a very large algorithm, but technically could be pre-shared knowledge. (And there are lots of ways to improve over keeping a copy of all photos on the internet.)
That is a very large algorithm, but technically could be pre-shared knowledge. (And there are lots of ways to improve over keeping a copy of all photos on the internet.)