Universe packages are not supported by Ubuntu unless you activate Ubuntu Pro. Thus, if you install ffmpeg on Ubuntu without Pro, it will contain several active vulnerabilities. The full five years only applies packages in the main repo.
> the training cutoff date isn't some absolute date from which they never allowed any new data to be trained on.
It's not a question of whether they are "allowed" to train on new data; the question is whether they have trained it on data containing information about current events. If you know they've implemented a Continuous Integration (CI) system for this, you should link to a source. However, I don't think this is true, as there would be no reason for a cutoff date otherwise.
> Instead there are bits and pieces of newer information captured in the updated models, but it's not a meaningful enough amount to ever rely on.
This seems more like an opinion of the technology's limitations in general, rather than an assessment of the likelihood that new information will be incorporated into its weights and biases.
Sure, there will be some pollution. It's very multivariate and depends on factors like content split, generation quality, and novel information. A scenario in which all of your data is generated by the previous model and you run n training loops is academic.
It's also worth noting that when OpenAI created Whisper, they had to heuristically remove many transcripts from poor ASR systems, and they definitely didn't catch them all.
They are most certainly being fed LLM content. However, I think this "model collapse" narrative is over-subscribed. Here are some things to keep in mind:
(1) Real content is not generated via a synthetic loop: Humans use generative AI in complex ways, intermixing human-generated and AI-generated content. Imagine a person who writes the first draft of an essay, then uses ChatGPT to rewrite parts of it. These are certainly many human additions, modifications, and stylistic flourishes.
(2) The most dramatic effects of model collapse were seen when training multiple generations of AI agents on content generated by the previous agent. This is a very academic scenario.
(3) There is already a lot of junk consumed by these models. RLHF is aimed at eliminating these junk responses. I am not aware of any research that explores how the full training cycle is affected when RLHF is employed.
Also, there is a lot of training material out there that was not used by the original GPT-3 model. The primary limitation is hardware.
I think it should be noted that this enforces grammatical constraints on the model's generated text, but it doesn't do anything to properly align the content. This would be useful if you needed to ensure a server delivered well-formatted JSON, but it I suspect it wont solve a lot of alignment issues with current language generation. For example current iterations of Llama and GPT often do not label markdown code-blocks correctly. Using grammar-based sampling, you could enforce that it labels code blocks but you couldn't enforce correct labeling since this is context-dependent. You also couldn't invent a novel domain-specific language without aligning against that language and expect good output.
The major benefit of tenure is as a protection for pursuing the truth when researching controversial academic subjects. It protects the behavioral genetics research which takes place at UT Austin from left wing attacks and the positions some professors have taken on Texas A&M's annual drag show from right wing attacks.
As a student in the Texas system, I've raised questions which my professors refused to talk about fearing political backlash. Tenure is an important institution. I think you can only go so far to incentivize good teaching through termination.