> Yeah, just like a turd is a piece of gourmet food if there is no other good food around.
I didn't mean it's on a continuum, as you assumed. Apologies for phrasing it unclearly. I meant that the weights are public. They are open; there is no debate to be had about it. Generally and broadly, that is already considered open-source.
And we all understand what "open-source" means in the context of Llama - it doesn't mean one of the idealized notions of open source, it means open weights.
I understand I was supposed to say “no” and question the open-source label. We’ve heard many arguments that if something can’t be reproduced from scratch, it’s not true open-source.
To me, they sound a bit like “no true Scotsman”. Llama is open source, compared to commercial models with closed weights. Even if it could be more open source.
That’s why I looked at it in a broader sense — what could happen in an open-source world to improve or replace Llama. Much could happen, thanks to Llama’s open nature, actually.
There are many things to be said about open-source projects and, more broadly, the capabilities of the open-source community.
The most capable parts are for-profit organizations that release open-source software for their business imperative, public benefit companies that write open-source software for ideological reasons but still operate as businesses, and a tiny number of public benefit organizations with unstable cash flow. Most other efforts are unorganized and plagued by bickering.
Llama itself is challenging to take over. The weights are public, but the training data and process is not. It could be evolved, but not fully iterated by anyone else. For a full iteration, the training process and inputs would need to be replicated, with improvements there.
But could another open-source model, as capable as Llama, be produced? Yes. Just like Meta, other companies, such as Google and Microsoft, have the incentive to create a moat around their AI business by offering a free model to the public, one that's just barely under their commercial model's capabilities. That way, no competitor can organically emerge. After all, who would pay for their product if it's inferior to the open-source one? It's a classic barrier to entry in the market - a thing highly sought after by monopolistic companies.
Public benefit companies leading in privacy could develop a model to run offline for privacy purposes, to avoid mass consumer data harvesting. A new open-source ideological project without a stable business could also, in theory, pop up in the same pattern as the Linux project. But these are like unicorns - "one in a million years (maybe)."
So, to answer your question, yes, Llama weights could be evolved; no, an entirely new version cannot be made outside of Meta. Yes, someone else could create such a wholly new open-source model from scratch, and different open-source groups have different incentives. The most likely incentive is monopolistic, to my mind.
When I lived in a shared house, Facebook learned about my relation to my housemates without my consent. I didn’t use a single Facebook product at the time and used a VPN on my devices. A few years later, when I signed up to Facebook, my ex-housemates were all “people I may know”.
I was wondering for a long time how it knew. I think it was because some of my ex-housemates shared their contacts with FB and it discovered us as a social group.
It is really an eye-opening experience to sign up to Facebook for the first time in recent years. It already knows so much about you and your interests. It’s as if there was already a profile with your name on it that they were building, without even approaching to ask for consent.
It may be incompetence. Most of big tech today no longer understands why someone would build an app, a business, or a project for reasons other than seeking to extract money from the market. It is ironic, because most of the big tech companies are only as successful because they initially offered products with vision and purpose, rather than as means simply to extract revenue at the lowest cost.
They are not entirely wrong to view the market this way. Most apps and projects are built these days primarily as means to extract revenue from the market at a low cost. Sometimes they are so much this that we call them grifts, and there have been many in tech recently, and certainly on the Play Store. But there are still some companies with better values.
This is a long way to say that unless an app/project/business/website is evidently monetized, it can probably be assumed it's monetized by selling data or ads. Likely, this assumption was made by the reviewer here, there may even be guidelines about it.
To some degree, it would even benefit consumers to do such a reality check too before they buy products and subscribe to services — is this a visionary purpose-driven product, or one that seeks to extract money from me as its primary goal? If latter, how do they do it — in evident ways, or through selling data or ads/influencing? But one must be able to see nuance that not everything in the world is built for the capitalistic objective. Sometimes, cool things are built, art is made, and inventions are made to better the world.
Overall, it is a problem that they don't see this nuance which made Google successful in the first place. It is clear across their business, products and services. But I think it is not unexpected.
An app shot down because it doesn't grift in an evident way, under an assumption it grifts in a hidden way — it's not the most wrong decision in the world. It just does not demonstrate competence either.