I've found breathing exercises to be effective for the duration of the exercise, but I'm more interested in the possibility of training myself to adjust my respiration patterns over sustained durations. Would it be beneficial -- or even possible at all -- to adjust my body's default/subconscious breathing patterns to match those mentioned in the article?
Tangentially related, are there any wearable devices that allow for high resolution respiration monitoring? I'm imagining some measurement of lung expansion over time (probably at least 10 Hz) so that I can quantify the deepness/shallowness of my breaths as well as the phase of inhalation/exhalation cycles.
Kind of a backwards take, both in your interpretation of this announcement and the company in general.
The point of this announcement is to draw attention to the fact that the currently hyped topic is what they have been working on since their inception. If anything, it gives off a Schmidhuber-esque 'actually it was me who invented that' vibe. But trying to retroactively claim credit for the hype is nowhere near the same as following the hype.
As for your impression that the company is more generally hype-chasing, I'm really not sure how you would come to that conclusion. At the time of their founding, chatbots were the hype on the product side and model scaling was the hype on the research side -- topics they have largely eschewed. They instead were founded with a focus on evolutionary and collective intelligence and have maintained a fairly cohesive research direction ever since.
I can assure you that a fully deterministic and equally effective claude is possible to build. And yes, that would mean identical prompts would yield 100% identical output 100% of the time. It would still make the occasional logical or factual error, but it would do so deterministically. Would this solve any of the problems with building reliable programs using LLMs?
I see what you're getting at, but determinism isn't the right word either. LLMs are fundamentally deterministic -- they are pure functions which output text as a function of the input text and the network parameters[1]. Depending on your views on free will, it could be effectively argued that humans are deterministic as well.
The concept you're touching on is the idea that LLMs (and humans) are functions which are inscrutable. Their behavior cannot be distilled into a series of logical steps that you can fit in your head, there are no invariants which neatly decompose their complexity into a few interpretable states, and the input and output spaces are unstructured, ambiguous, underspecified, and essentially infinite. This makes them just about impossible to reason about or compose using the same strategies and analysis we apply to traditional programs.
[1] Optionally, they can take in a source of entropy to add nondeterminism, but this is not essential. If LLM providers all fixed their prng seeds to a static value, hardly anyone would notice. I can't imagine there are many workflows which feed an LLM the exact same prompt multiple times and rely on the output having some statistical distribution. In fact, even if you wanted this you may just end up getting a cached response.
The guy writing a thumbnail pipeline isn't getting petabytes (exabytes?) of storage to cache all videos from the past week in their entirety. If this quantity of data is being stored, it's being stored deliberately and at significant cost.
The other explanations here don't explain the long delay between the start of the investigation and the release of the footage.
Yes, storing customer data is what we'd expect from Google and yes, the FBI can coerce Google to provide this data for their investigations.
But it does not take a week for Google to find a file on their servers.
My hunch is that Google initially tried to play dumb to avoid compliance, as to not reveal they do in fact retain customer data.
They had a plausible excuse as well -- the owner had no subscription so they don't store the data -- and took a gamble that this explanation would suffice until the situation resolved itself.
I suspect that authorities initially took Google's excuse at face value, since they parroted this explanation to the public as well.
As pressure mounted on authorities to make some headway on the case, they likely formally exercised whatever legal mechanisms they have at their disposal to force Google's hand, and only then was the footage released.
The implication that OpenAI is a YC company in the same sense as the other listed companies is somewhere between misleading and dishonest. Even more distasteful to show founding teams for all the others, then just Sam for OpenAI.
Of all Schmidhuber's credit-attribution grievances, this is the one I am most sympathetic to. I think if he spent less time remarking on how other people didn't actually invent things (e.g. Hinton and backprop, LeCun and CNNs, etc.) or making tenuous arguments about how modern techniques are really just instances of some idea he briefly explored decades ago (GANs, attention), and instead just focused on how this single line of research (namely, gradient flow and training dynamics in deep neural networks) laid the foundation for modern deep learning, he'd have a much better reputation and probably a Turing award. That said, I do respect the extent to which he continues his credit-attribution crusade even to his own reputational detriment.
My personal experience is that the cost of enduring a negative stimulus is not simply a function of the magnitude of the negative stimulus, but rather the magnitude of the negative stimulus in relation to the magnitude of all other concurrent negative stimuli. This study controls the environment so that a single negative stimulus is isolated and additional external negative stimuli are minimized, but it cannot control for the fact that a depressed person also endures a constant barrage of negative stimuli which are generated internally (hopelessness, exhaustion, fear, self-doubt, etc). The magnitude of these internally generated negative stimuli is likely much larger than that of the aversive external stimulus used in this study, so it seems reasonable that the marginal relief obtained by avoiding the external stimulus may be perceived as relatively negligible, or at least diminished to the point that the cost of avoiding is greater than the cost of enduring.
Tangentially related, are there any wearable devices that allow for high resolution respiration monitoring? I'm imagining some measurement of lung expansion over time (probably at least 10 Hz) so that I can quantify the deepness/shallowness of my breaths as well as the phase of inhalation/exhalation cycles.