I think it's pretty reasonable that many users might perceive this as a betrayal of an implicit social contract. People posting things ten years ago, or even a year ago, shouldn't have had to make projections about future modelling advances / how data might be used in the future.
IMO: Renegotiating for "public approval" will always be something computing organizations have to engage with when making advances in tech that's downstream of publicly created records, knowledge, etc.
(That being said, I personally think this announcement is overall a win for healthier data flow, in particular because these kind of deals make the value of data explicit.)
Yeah, that's what I'm basically thinking: for scientists who already like Python because it's readable and mostly matches the mental model of what they care about, being able to catalyze the switch to a different, faster language that still reads nicely could really drive adoption. But it's a stretch for sure!
I'm quite hopeful that language models for code might actually lead to more people getting comfortable with multiple languages and especially some kind of hopping back and forth. I think the prospect of starting off your script in Python (which is a "comfort food" for many, including myself), using LLM to translate over to Julia, probably do some minor debugging, and then taking off from a starting point instead of a blank file could be a nice workflow for people who want to dip their toes in.
I think it's related to Reddit users who posted (very frequently!) on a counting focused subreddit (people literally post "1", "2" , "3" in sequence so usernames appear 50k+ times). Some screenshots and links in this Twitter thread: https://twitter.com/SoC_trilogy/status/1623118034960322560
Basically agree -- feels off-putting, but not technically a wrong detail to add. An additional reason it rubs me the wrong way, however, is that I believe open-source software code is especially critical to ChatGPT family's capabilities. Not just for code-related queries, but for everything! (e.g. see this "lineage-tracing" blog post: https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr...)
Thus, I honestly think firms operating generative AI should be walking on eggshells to avoid placing blame on "open-source". Rather, they really should going out of their way to channel as much positive energy towards it as possible.
Still, agree the charitable interpretation is that this just purely descriptive.
IMO - this kind of tool cuts across a debate that currently involves a lot of people yelling past each other.
If you're an artist, of course you can make changes to your process or your content that modify how it's used. To argue otherwise is analogous to arguing we must always be mindful to stay in the surveillance cameras' view when walking about the streets.
Yes, it's a never-ending back-and-forth game that the obfuscater will probably lose in the long run (though abstractly, an obfuscation technique with >50% adoption could "win" long-term) . And yes, it's important to stay apprised about how effective such tools are.
But in the short term, the existence of these tools provides a critical counter-measure to the current narrative, which is basically that everything that can be scraped will be scraped. Returning to the cameras & streets analogy, obfuscation tools are maps that tell us about routes out of view of the cameras (even though these routes may often be blocked off or inconvenient).
Whether you hate AI art or love it, I honestly believe both sides can get behind understanding obfuscation and poisoning and making tools available: those opposed will use the tools, those who want to improve generative AI can learn from the counter-measures, etc. This kind of thing can be part of a healthy deliberative process around these emerging technologies.
Yes, this doesn't use attribution techniques like influence functions or Shapley values that are popular in machine learning research, but I am pretty convinced that even a nearest neighbors search is better than the current baseline offered by "AI art systems": shrug our shoulders and say nothing about the role of human-created training data in producing the outputs.
As far as I know, nobody is even thinking about doing the very expensive experiments needed to get ground truth data for formal attribution techniques in the generative AI context (for a given prompt, retrain your model so you can see how the output changes when a particular training example or group of examples is omitted or added), so we're nowhere near building true attribution systems for these very large models. Centering the training data will be net good for public discourse on the topic.
That said, I see why people want to push back on some of the language used here.
Appreciate the distinction in the above comment that they are two distinct questions, but also agree the two questions are very connected.
I should've been more specific: I was thinking mainly of the artists v. stable diffusion lawsuit which makes the specific technical claim that the stable diffusion software (which includes a bunch of "weights files") includes compressed copies of the training data. (Line 17, "By training Stable Diffusion on the Training Images, Stability caused those images to be stored at and incorporated into Stable Diffusion as compressed copies", https://stablediffusionlitigation.com/pdf/00201/1-1-stable-d...).
I expect that if the decision hinges on this claim, that could have far reaching implications re: model licensing. I think this along the lines of what you've laid out here!
Not a lawyer, but as I understand the most likely way this question will be answered (for practical purposes in the US) is via the ongoing lawsuits against GitHub Copilot and Stable Diffusion and Midjourney.
I personally agree the creativity is in the source images and the training code, but think that unless it is decided that for legal purposes "AI Artifacts" (the files containing model weights, embedding, etc.) are just transformations of training data and therefore content and subject to the same legal standards as content, I see a lot of value in trying to let people license training and code and models separately. And if models are just transformations of content, I expect we can adjust the norms around licensing to achieve similar outcomes (i.e., trying to balance open sharing with some degree of creator-defined use restriction).
Q: "Are OpenRAILs considered open source licenses according to the Open Source Definition? NO."
A: "THESE ARE NOT OPEN SOURCE LICENSES, based on the definition used by Open Source Initiative, because it has some restrictions on the use of the licensed AI artifact.
That said, we consider OpenRAIL licenses to be “open”. OpenRAIL enables reuse, distribution, commercialization, and adaptation as long as the artifact is not being applied for use-cases that have been restricted.
Our main aim is not to evangelize what is open and what is not but rather to focus on the intersection between open and responsible licensing."
FWIW, there's a lot of active discussion in this space, and it could be the case that e.g. communities settle on releasing code under OSI-approved licenses and models/artifacts under lowercase "open" but use-restricted licenses.
Thanks! Looks like a bunch of changes were made to page structure and CSS (in a way that makes scraping a bit more inconvenient). Filed an issue for myself, but looks like the other suggestions in this thread may be better approaches to avoid this kind of fragility.
One possible suggestion for future iteration: I think it's a bit of a shame that everyone's been sharing ChatGPT outputs almost exclusively through screenshots. I threw together a (very quick and rudimentary) browser extension I've been using to save some of my more exciting transcripts in JSON files. I think something along these lines could be really great for sharing purposes, especially if people want to study these outputs more systematically (e.g. for research, or just a kind of crowd-sourced audit).
There's also a longer discussion to be had about the best way to do this: ideally, we might save some of the formatting information too and/or more metadata (exact timestamps for responses, etc.), but I think the json with plaintext is adequate (at least for personal retrieval use).
Also, based on all the public info about InstructGPT (the closest ChatGPT "family member"), all of StackExchange is definitely in the training via OpenAI's "filtered Common Crawl", if it isn't also included as a special over-weighted training set (English Wikipedia, for instance, was over-weighted in GPT 3 training).
There's a big concern in the long run: if ChatGPT seriously reduces the number of people who visit StackExchange domains, there will be no dataset no for GPT 4 / ChatGPT 2. E.g. what if a brand new programming language gains popularity, or new libraries with very different patterns of use?
It's actually a separate concern from the spam / content bloat described in the linked post, but they complement each other in creating amplified harm for StackExchange-like platforms: some fraction of users may stop visiting the site (because ChatGPT answered without links) and some other fraction of users will submit spam to try and earn points.
It gives a generic answer that it's some proprietary combinations of "books, articles and websites". I'd guess Wikipedia is in there for sure (English and maybe other editions as well), something like "BookCorpus" (https://huggingface.co/datasets/bookcorpus), probably a large scrape of news articles up to 2021. And definitely a full scrape of pretty much the entire academic/scientific literature (just based on poking around). Overall, probably very similar to GPT-3 (which is also a bit mysterious still!)
The official post (https://openai.com/blog/chatgpt/) also describes that some pretty rich human feedback data was collected as well, for the reinforcement learning component. I think this probably the real secret sauce for why this feels so qualitatively different than a lot of the LLMs that came before.
Yeah, I think this is spot on. The version of WMF Enterprise being described here is formalizing / codifying an informal relationship that already existed (Google using Wikipedia content all over the place, and making various one-off donations to WMF), and as you say, formalizing has benefits for both parties! Certainly ways this could have some long-term negative impacts, but WMF is obviously thinking pretty hard about mitigations, it seems.
In practice, I believe the reason is historical: that's how Wikipedia started and it hasn't changed. But there's pretty compelling evidence that it provides Wikipedia some unique benefits relative to an account-locked alternative:
Hill, B. M. and Shaw, A. (2021) ‘The Hidden Costs of Requiring Accounts: Quasi-Experimental Evidence From Peer Production’, Communication Research, 48(6), pp. 771–795. doi: 10.1177/0093650220910345.
Crazy impressive! A question about the training data: anyone familiar with this line of work know what social media platforms the "conversation" data component of the training set came from? There's a datasheet that points to prior work https://arxiv.org/abs/2001.09977, which sounds like it could be reddit, HN, or a similar platform?