Instead of flaming about "open source" vs "free software" why don't we talk about how whichever terminology you prefer, organizations like EleutherAI, Hugging Face, and LAION (the AI orgs on this letter) release software that is licensed in a way you approve of.
No, it's about how the regulations governing OpenAI and Google's commercial models, research by non-profits like EleutherAI and AI2, and finetunes of public models done by hobbyists should not be the same. The current law puts all three groups in the same bucket.
It seems that the user is posting ChatGPT-generated text and not a real summary. It's complete nonsense, with about half of the sentences containing a factual error.
Yes I am (apparently) in the acknowledgments. I was not an author on the paper because I didn’t have time to contribute too much. I also try to err on the side of not being added to papers, as my position (I run EleutherAI) tends to encourage people to be overgenerous with offers. I anticipate having more time this coming month and being on the version that’s submitted for peer review, but we’ll see.
BlinkDL has been working on this project for two-ish years, originally in the EleutherAI discord and then created his own to house the project.
I wasn’t thinking too hard about my exact wording, but yes I was thinking of EleutherAI and its various spin-off servers. EleutherAI doesn’t /run/ any of the other servers, but we all have a close collaborative relationship. I’m sure there’s a lot of duplication of membership (e.g., I’m in all of them) but quickly adding up the membership of each server comes out to around 70,000. EleutherAI and LAION are the largest at 25k each, with the others typically having around 5k each. I would expect at least 30k of those users to be unique though.
Our discord servers (a primary one, a spin-off for RWKV, another spinoff for BioML, etc) have tens of thousands of people between them :) So not quite everyone. But this was a community effort with a public call for contributions
The paper is going to be submitted to EMNLP next month. An early version is being released now to garner feedback and improve the paper before submission.
Our policy is to not comment on timelines for future models, as our ability to meet those timelines is heavily influenced by factors outside of our control and we don’t want to lead people on.
I mean, ultimately there isn’t one. I’m just providing examples of how we fulfill the things that the OP says they want, as they seem unaware of our work.
But I’m confused by the anti non-profit vibes in this comment section. We aren’t saying that becoming a non-profit makes us ethical people, that would be a silly argument. But people do realize that the alternative would be to become a for-profit entity right?
We’re still the same community-driven open collaborative research lab we’ve always been. But incorporating allows us to do things like hire full time staff, enter organizationally binding legal agreements, and protect our members. Between the options of becoming a for-profit and a non-profit, the later seems clearly better suited for our goals.
You're in luck! EleutherAI has trained and released open source weights of several LLMs, including GPT-Neo (2.7B parameters), GPT-J (6B parameters), and GPT-NeoX (20B parameters). This last model is currently tied for second on the list of the largest open source LLMs in the world.
We also developed VQGAN-CLIP and CLIP-Guided Diffusion, techniques for doing text-to-image synthesis that don't require training and can easily be run locally for inference.
We have a number of donors including Hugging Face, Stability AI, Nat Friedman, Lambda Labs, and Canva that make our work possible. We also have some orgs that provide sponsorship for computing resources specifically: Stability AI, CoreWeave, and Google Research.
GPT-NeoX-20B was specifically targeted to fit on A40s, A6000s, and a pair of 3090 Tis. Anything larger than that is going to be a real struggle for people who don’t own computing clusters to use.