There is an understandable gap between the capabilities of closed models and those of open models.
The current difference is primarily expressed in the cost of hardware necessary to sufficiently run a exactly comparable model.
A single higher end graphics card running on your average gaming computer, is capable of running small to medium models that compare with those of their lab-born counterparts in the small-medium range. But the heavyweight models are still outside the realm of possibility for all but the most well-funded individual.
However, I would highly suggest more people experiment with these smaller models. They are incredibly capable in many ways that many people dont realize.
The perceived capabilities of the larger models are also much less the result of the model having more parameters/training cycles, but rather that they are being run through well-made harnesses, something which the open-source community is rapidly approaching with near-peer solutions of their own.
In short, much of the gap between between open-weight models and the larger proprietary models can be considered more of an issue of perception and not an issue of capability. There is a fundamental gap economically, but not so much in capability.
The open source community is rapidly closing the gap on these larger labs, especially thanks to the amazing research being freely given openly by well funded chinese labs.
More signal that the open-weight models should be our destiny as an industry.
These proprietary models are being used to usher in more surveillance and gatekeeping across the industry.
> "To support the claim that coding agents can displace human code review, we survey evidence of agent capability
across three dimensions: benchmark performance on softwareengineering tasks, review-specific capabilities, and developer
productivity with deployed tools"
Not sure I can agree with this premise, especially since there seems to be a complete lack of "real-world results" in this evaluation.
This strikes me as being written by a theorist, who's only experience with Quality Assurance exists in studies or papers.
When someone is expected to be wizened and does not have the knowledge to keep up with the needs of those around them, they in turn become Shamanistic in their practice.
The speed of improvement on these models has been incredible and has outpaced the learning speed of humans and put many experts into these Shamanistic roles.
I think the operative means of addressing this is to recognize that we can only learn so quickly, but we are still called to improve our knowledge and understanding to a higher level.
Since the improvement of these models is neither logorithmic, nor exponential, we currently occupy a space in time in which the models are currently smarter on average than we are as a collective whole.
There are a lot of nutritional blind spots in vegan diets.
It is a diet that requires exceptional planning and intentionality to be at a baseline of health similar to a balanced omnivorous diet.
So indeed, the "it must be veganism" is not an unfounded concern when health complications arise, in a very similar way to "it must be the AI" is a valid concern when software issues arise.
Asian countries have governments that are at least seemingly vested in the interests of their populace. They have significant political and economic safety nets in place that can assure their populations that the government is at least somewhat aligned with the populace.
Instead for many western countries, chiefly The USA, You have a society that very blatantly is restructuring itself to service capital holders and not the population.
These Governments are not aligned with their people, and are instead trying to solidify Stratified Economies where the entire engine of the country moves in service of its rich.
If the promise of AI is to provide intellectual labor in exchange for capital, the population loses its only remaining middle class made up of knowledge-workers which still hold a semblance of political power.
If the middle collapses like this, the only means of social mobility will become high-risk gambles or crime.
This is unlikely to happen in any meaningful fashion for quite some time.
(TLDR; Distributed compute for models will require hardware at a level only really possible with data-centers at the moment.)
Token generation operates at such a scale to demand enough from a single GPU as it will often saturate the bandwidth capabilities of consumer grade interconnects like PCIe. Which fundamentally implies that distributing a model's compute across vast distances is too much of a challenge without significant infrastructure.
To give an example, When we split a model's compute between two seperate cards on a single workstation, this doesnt mean we end up with 2x the compute bandwidth for a model. Instead the increase becomes something small like 20% depending on model, because the inconnects (PCIe on consumer hardware) will quickly become so saturated with data being copied between the two GPUs so as to become a bottleneck. And remember that this is something that happens locally with PCIe, which (depending on generation) will cap out at around 20-35 GB/s depending on the generation of motherboard.
Model performance is very much tied to having the fastest and highest bandwidth single card available so as to keep data transfer operations to a minimum as the sheer volume of data necessary for the model to run is immense.
I simply cant imagine how slow and unusable a model would be if the copy operations necessary for its compute needed to be performed over unreliable network speeds where there will be significant performance loss as network speeds are not reliably distributed across the globe, and their unreliable nature would demand increased overhead due to data verification.
Anthropic is looking to IPO here soon.
A key aspect of this is to prove profitability.
Shifting their focus from Training new models to instead serving inference, they would greatly reduce their spend. In fact this is something being reported on that they are already doing, which is the reason for their first ever profitable quarter.
Its awfully convenient that the company which has greatly reduced its spend on training is now asking for a slow down in this area.
Glad to know the struggle seems to be universal.
Im happy that this really cool and sophisticated tool got invented. But everywhere im seeing it be used is making me sad and frustrated. This software renaissance feels more like the coming dark ages.
There is an obvious shift in sentiment amongst users, at least here in the US.
I feel it myself, even as a proponent of AI tools, the bloviating and language that these companies use in these release articles are starting to wear thin on my patience.
Its possible we might just be witnessing a shift in fashion, where this type of sentimentality was more acceptable when it was novel and new, but now it just appears out of touch.
They make some incredibly outlandish claims over their total addressable market, one can only wonder where $26 trillion dollars in expected AI revenue would even come from, with 22T of that being from "enterprise" when they have no real products yet.
The whole thing looks to be proped up by Starlink which seems to be a genuinely solid business. xAI looks to be costing twice as much as it produces, and we dont even have good numbers for this yet since the deal is so new.
This feels like WeWork but if WeWork also owned a successful coffee shop.
Wealth and Power are linked, but power is the goal for the wealthy, not the wealth itself. The moment the relationship of wealth and power is uncoupled, they will discard it in favor of whatever comes next in their accumulation of power.
It is incredibly naive to think that the way things currently are is the way things will be. There is significant reason to believe that after enough concentration of power, there would be no reason for them to continue to participate in traditional economics as we know them.
On the bright side, history shows us that powerful people tend to concentrate power up to the point in which they start to believe themselves as some sort of god-like being. At which point they are reliably proven they are not. The Sword of Damocles hangs above all of them.
After the many years, there have been insider voices indicating that success was despite Musk in many ways.
Musk bought his way into cutting edge tech, it succeeded despite him due to the already amazing people working in the industries.
The projects that have his actual involvement are pretty regularly seen as mistakes or flops.
I personally hold to the idea that whoever at SpaceX crafted the team used to pre-occupy Musk and keep him entertained while the rest of the company worked, is largely responsible for its success.
AMD does a lot of work to ensure their support for Linux is first-class.
With the kernel now natively supporting their systems, you can expect good support.
It's earned them some good will over Nvidia
which has gotten better recently with the rise of AI, but still requires users to jump through a couple of hoops due to their attempts to protect their IP.
The biggest question on my mind is how the use of Cider V is being affected by the officially ordained Antigravity.
Is the trendline starting to show that its adopting more Antigravity style tooling? or is this causing some sort of rift?
ah yes, the "deep state." The formless, nebulous, rhetorical tool that is always infinitely liquid enough to fit into or over any container necessary that the user can satisfy their immense personal problems disguised as eternal doomerism.
It used to be that the thought process of receiving a portion of sale money before delivering any product allowed the company to pay suppliers and keep afloat as they drove towards the finish line of delivery.
Now it seems the grifting-meta is to make promises around a product with no plans on delivering it, take in pre-order money, and then just park it in an investment account to grow during a bull market.
By the time the grift comes due, your "investment" will have grown to a magnitude where even if you are forced to pay it back, you will have made a tidy profit.
I expect a negative response from markets as this basically means that the party bus just got pulled over.