It's very clear that the consumer is getting a worse experience than what is technically possible. There is no good phone-slash-laptop, purely because it's less profitable than locking down the devices and selling them separately.
I couldn't see any citations or references in that video or its description. It presents it as him solving the problem himself, but I'm sure other people have written about solving the Game of Life in reverse with SAT solvers prior to this...
> There isn't an idea that "IQ is some innate fixed quality"
Actually yes there is; I have come across many people who believe this, specifically saying that IQ is fixed.
> that intelligence is an innate fixed quality
I would also disagree with this — intelligence can be increased, (e.g. through education, training, and practice), and also decreased, (e.g. by lifestyle / environment).
See also: "Major IQ differences in identical twins linked to schooling, challenging decades of research" [1] [2]
I.e. the idea that IQ is some innate fixed quality has evidence against it. It seems obvious that this is the case, given that people get their children tutors so they can do better at IQ tests to get into schools...
> Any leakiness in BackProp is addressed by researchers who introduce new optimizers
> As a developer, you just pick the best one and find good hparams for it
It would be more correct to say: "As a developer, (not researcher), whose main goal is to get a good model working — just pick a proven architecture, hyperparameters, and training loop for it."
Because just picking the best optimizer isn't enough. Some of the issues in the article come from the model design, e.g. sigmoids, relu, RNNs. And some of the issues need to be addressed in the training loop, e.g. gradient clipping isn't enabled by default in most DL frameworks.
And it should be noted that the article is addressing people on the academic / research side, who would benefit from a deeper understanding.
YouTube has so much questionable content on it that gets millions of views... Parents who've found ways to monetize their kids. Dangerous / unpleasant pranks being pulled on members the public. Conspiracy theories. Fake game shows where the winners being given money are actually friends of the host. Or where the host pretends contestants are doing something dangerous, but actually it's CGI, (misleading young viewers into thinking the dangerous stuff is real / fun). Morons making content that's attractive to kids who don't know better. Etc.
While there is some quality content on there, the amount of terrible content getting vast amount of views is pretty high.
I guess one question is whether TV is much better.. I would say on average it probably is less bad, although there have also been / are questionable unethical tv shows. But at least with TV shows there's more likely to be a few more layers of questioning / analysing / looking at the ethics, with responsible people involved.
Stuff like this feels like some company has managed to monetize an open source object detection model like YOLO [1], creating something that could be cobbled together relatively easily, and then sold it as advance AI capabilities. (You'd hope they've have at least fine-tuned it / have a good training dataset.)
We've got a model out there now that we've just seen has put someone's life at risk... Does anyone apart from that company actually know how accurate it is? What it's been trained on? Its false positive rate? If we are going to start rolling out stuff like this, should it not be mandatory for stats / figures to be published? For us to know more about the model, and what it was trained on?
I agree, the post above you is patently wrong / hasn't read the paper they are dismissing. I also got multiple downvotes for disagreeing, with no actual rebuttal.
It's not a uniform distribution after the first measurement, t_obs. That enables us to update the distribution, and it becomes a decaying one.
I think you mistakenly believe the distribution is still uniform after that measurement.
The best guess, that it will last for as long as it already survived for, is actually the "median" of that distribution. The median isn't the highest point on the probability curve, but the point where half the area under the curve is before it, and half the area under the curve is after it.
> Model state is present only in so-far-generated text
Wrong. There's "model state", (I assume you mean hidden layers), not just in the generated text, but also in the initial prompt given to the model. I.e. the model can start its planning from the moment it's given the instruction, without even having predicted a token yet. That's actually what they show in the paper above...
> It is only after the model has found itself in a poetry generating context and has also selected the first line-ending word, that a rhyme scheme "emerges" as a variable
This is an assertion based on flawed reasoning.
(Also, these ideas should really be backed up by evidence and experimentation before asserting them so definitively.)
Yes that's pretty much it. There will be a decaying probability curve, because given you could fail at any time, you are less likely to survive for N units of time than for just 1 unit of time, etc.
(Just to expand on that, it's true not just the for the first token. There's a lot of computation, including potentially planning ahead, before each token outputted.)
That's why saying "it's just predicting the next word", is a misguided take.
(t_obs is time observed to have survived, t_more how long to survive)
Case 1 (x): It has lasted 1 minute (t_obs=1). The probability of it lasting 1 more minute is: 1 / (1 + 1) = 1/2 = 50%
Case 2: It has lasted 2 minutes (t_obs=2). The probability of it lasting 1 more minute is: 2 / (2 + 1) = 2/3 ≈ 67%
I.e. the curve is a decaying curve, but the shape / height of it changes based on t_obs.
That gets to the whole point of this, which is that the length of time something has survived is useful / provides some information on how long it is likely to survive.
> while this Copernican principle sounds very deep and insightful, it is actually just a pretty trite mathematical observation
It's important to flag that the principle is not trite, and it is useful.
There's been a misunderstanding of the distribution after the measurement of "time taken so far", (illuminated in the other thread), which has lead to this incorrect conclusion.
To bring the core clarification from the other thread here:
The distribution is uniform before you get the measurement of time taken already. But once you get that measurement, it's no longer uniform. There's a decaying curve whose shape is defined by the time taken so far. Such that the estimate `time_left=time_so_far` is useful.