No, CMOS image sensor Bayer RAWs are not accurate to what we actually see. Just a few of the many things that need to be addressed to get an even “normal” looking image:
1) They’re overly green because the greens of the RGGB color filter array are more sensitive to light than the other colors. This needs to be corrected with nontrivial auto-white-balance algorithms (not only the green bias needs to be fixed but other scene-dependent factors)
2) The Bayer pattern of the color filter array creates a checkerboard color pattern that needs to be fixed with debayering/demosaicing algorithms - again nontrivial if you don’t want to create artifacts or overblur with simplistic interpolation approaches, and there are even ML algorithms that do this now.
3) Bayer RAWs are linear in photon intensity which is not accurate to how our visual system compresses the high dynamic range. Therefore, various tone mapping algorithms are required to reproduce a natural-looking tone/intensity map of the scene.
4) Small sensors can’t collect enough light and this inhererently results in noisy raw images that need to be denoised. There are a lot of different denoising approaches (including modern that are basically all ML), but care needs to be taken as this is one of the places where it’s easy to generate an overly-processed image.
There are a lot more steps that happen in a typical image processing pipeline, while yes can be tuned in non-ideal ways to produce overly-processed-feeling images, but are at the end of the day necessary if you don’t want something that looks like these: https://images.app.goo.gl/neJCHk5QsVt68XpL7
Can’t this cynical take be used to nullify any and all journalism? I get the need to be suspicious generally but this comment doesn’t add to the conversation in any substantive manner. At least give us some take on how the author and her content would be likely biased given this commercial influence.
Is software that important on the inference side, assuming all the key ops are supported by the compiler? Once the model is quantized and frozen the deployment to alternative chips while somewhat cumbersome hasn’t been too challenging, at least in my experience with Qualcomm NPU deployment (trained on NVIDIA)
From a high-level design standpoint, wouldn’t the general-purposeness of NVIDIA’s GPUs (even if they do have some AI/LLM optimizations) put them generally at a disadvantage compared to more custom/dedicated inference designs? (Disregarding real-world issues like startup execution risks, assume competitors succeed at their engineering goals) Or is there some fundamental architectural reason why NVIDIA can/will always be highly competitive in AI inference? Is the general-purposeness of the GPU not as much of an overhead/disadvantage as it seems?
Also how critical is NVIDIA’s infiniband networking advantage when it comes to inference workloads?
Naive question: how is brute-force cracking still a thing in real-world systems? Aren’t there time-outs/bans for guessing wrong after like 3-5 guesses? How does one get the opportunity to try millions/billions/etc of times?
Just got mine - it did run hot the first day but the next day it was fine (although I did turn off background app refresh after the first day, maybe that had something to do with it)
What’s the fundamental reason why SOTA modems are more challenging than microprocessors for Apple? Intuitively I would’ve guessed the opposite was true.
What are you talking about? He directly addressed it a couple times and ended his remarks on it:
“Moving onto our product roadmap, I've said on a number of these calls that the two technological waves that we're riding are AI in the near term and the metaverse over the longer term.”
“As our investments in AI continue, we remain fully committed to the metaverse vision as well. We've been working on both of these two major priorities for many years in parallel now, and in many ways the two areas are overlapping and complementary. The next big thing on the Reality Labs side is the launch of our Quest 3…”
Source: https://s21.q4cdn.com/399680738/files/doc_financials/2023/q2...
From the press release you linked: “Finally, for Reality Labs, we expect operating losses to increase meaningfully year-over-year due to our ongoing product development efforts in augmented reality/virtual reality and investments to further scale our ecosystem. “ They’re digging in even deeper, despite the incorrect superficial perception they’re done with it.
Thank you for sharing this. I’m a Christian as well and have found the same peace about death. At the same time, I get stressed out easily about work and probably have an existential dread about negative reputation or “wasting my potential” - clearly my work-related identity is an idol (in Christian parlance).
I was watching “Hustle” (Adam Sandler basketball movie) yesterday and strangely enough a line he said resonated with me: “they can’t kill you if you’re already dead.” (The context in the movie was that the youngster protagonist missed his official chance of playing in the NBA by failing the combine, and thus was “dead.” He had nothing to lose when he got a random last-ditch chance to demonstrate his skills in another venue.). As a Christian we believe that our old selves have died and that we are born again in the Spirit, even while we are here on earth. This gives those who believe this the confidence to face any fear or challenge in this lifetime, as we have already died anyway, there is nothing to lose since our new lives are secure in Christ (even after a physical death process).
If the rotation of the craft is simply due to a rotation of the camera, why do the clouds not also rotate? I’ve only skimmed the video but he doesn’t seems to account for this relative rotation between the craft and clouds…
Agreed here. Lots of things are outside our control over reasonably large time spans- macroeconomics, rapid tech & ecosystem disruptions, changes in your own personal life, etc. - but if there’s some underlying deeper “dream” that is flexible in how it can manifest concretely, then it can potentially be robust to these changes and what look to be initial setbacks. Perhaps the same (or similar) thing can be achieved in a different form.
1) They’re overly green because the greens of the RGGB color filter array are more sensitive to light than the other colors. This needs to be corrected with nontrivial auto-white-balance algorithms (not only the green bias needs to be fixed but other scene-dependent factors)
2) The Bayer pattern of the color filter array creates a checkerboard color pattern that needs to be fixed with debayering/demosaicing algorithms - again nontrivial if you don’t want to create artifacts or overblur with simplistic interpolation approaches, and there are even ML algorithms that do this now.
3) Bayer RAWs are linear in photon intensity which is not accurate to how our visual system compresses the high dynamic range. Therefore, various tone mapping algorithms are required to reproduce a natural-looking tone/intensity map of the scene.
4) Small sensors can’t collect enough light and this inhererently results in noisy raw images that need to be denoised. There are a lot of different denoising approaches (including modern that are basically all ML), but care needs to be taken as this is one of the places where it’s easy to generate an overly-processed image.
There are a lot more steps that happen in a typical image processing pipeline, while yes can be tuned in non-ideal ways to produce overly-processed-feeling images, but are at the end of the day necessary if you don’t want something that looks like these: https://images.app.goo.gl/neJCHk5QsVt68XpL7