No, CMOS image sensor Bayer RAWs are not accurate to what we actually see. Just a few of the many things that need to be addressed to get an even “normal” looking image:
1) They’re overly green because the greens of the RGGB color filter array are more sensitive to light than the other colors. This needs to be corrected with nontrivial auto-white-balance algorithms (not only the green bias needs to be fixed but other scene-dependent factors)
2) The Bayer pattern of the color filter array creates a checkerboard color pattern that needs to be fixed with debayering/demosaicing algorithms - again nontrivial if you don’t want to create artifacts or overblur with simplistic interpolation approaches, and there are even ML algorithms that do this now.
3) Bayer RAWs are linear in photon intensity which is not accurate to how our visual system compresses the high dynamic range. Therefore, various tone mapping algorithms are required to reproduce a natural-looking tone/intensity map of the scene.
4) Small sensors can’t collect enough light and this inhererently results in noisy raw images that need to be denoised. There are a lot of different denoising approaches (including modern that are basically all ML), but care needs to be taken as this is one of the places where it’s easy to generate an overly-processed image.
There are a lot more steps that happen in a typical image processing pipeline, while yes can be tuned in non-ideal ways to produce overly-processed-feeling images, but are at the end of the day necessary if you don’t want something that looks like these: https://images.app.goo.gl/neJCHk5QsVt68XpL7
Can’t this cynical take be used to nullify any and all journalism? I get the need to be suspicious generally but this comment doesn’t add to the conversation in any substantive manner. At least give us some take on how the author and her content would be likely biased given this commercial influence.
Is software that important on the inference side, assuming all the key ops are supported by the compiler? Once the model is quantized and frozen the deployment to alternative chips while somewhat cumbersome hasn’t been too challenging, at least in my experience with Qualcomm NPU deployment (trained on NVIDIA)
From a high-level design standpoint, wouldn’t the general-purposeness of NVIDIA’s GPUs (even if they do have some AI/LLM optimizations) put them generally at a disadvantage compared to more custom/dedicated inference designs? (Disregarding real-world issues like startup execution risks, assume competitors succeed at their engineering goals) Or is there some fundamental architectural reason why NVIDIA can/will always be highly competitive in AI inference? Is the general-purposeness of the GPU not as much of an overhead/disadvantage as it seems?
Also how critical is NVIDIA’s infiniband networking advantage when it comes to inference workloads?
Naive question: how is brute-force cracking still a thing in real-world systems? Aren’t there time-outs/bans for guessing wrong after like 3-5 guesses? How does one get the opportunity to try millions/billions/etc of times?
1) They’re overly green because the greens of the RGGB color filter array are more sensitive to light than the other colors. This needs to be corrected with nontrivial auto-white-balance algorithms (not only the green bias needs to be fixed but other scene-dependent factors)
2) The Bayer pattern of the color filter array creates a checkerboard color pattern that needs to be fixed with debayering/demosaicing algorithms - again nontrivial if you don’t want to create artifacts or overblur with simplistic interpolation approaches, and there are even ML algorithms that do this now.
3) Bayer RAWs are linear in photon intensity which is not accurate to how our visual system compresses the high dynamic range. Therefore, various tone mapping algorithms are required to reproduce a natural-looking tone/intensity map of the scene.
4) Small sensors can’t collect enough light and this inhererently results in noisy raw images that need to be denoised. There are a lot of different denoising approaches (including modern that are basically all ML), but care needs to be taken as this is one of the places where it’s easy to generate an overly-processed image.
There are a lot more steps that happen in a typical image processing pipeline, while yes can be tuned in non-ideal ways to produce overly-processed-feeling images, but are at the end of the day necessary if you don’t want something that looks like these: https://images.app.goo.gl/neJCHk5QsVt68XpL7