My understanding is that early (and most extant) visual language models have a component module (called the image encoder) that transforms images into representations (called embeddings) the model's inner layers can process.
This is often a separate module grafted onto the main model, and further pre-trained (e.g. OpenAI's CLIP, SigLIP used in the Gemma 3 and PaliGemma series).
The image encoder approach has a few problems.
One problem is that many like Gemma 3's encoder have fixed image resolution constraints and inputs must be resized with all the attendant distortions that causes with spatial understanding. However, the Gemma 4 series image encoders overcame this and can handle variable-dimension inputs.
Two, these image encoders are somewhat large (ranging from 300-500M parameters) requiring extra memory and FLOPs to run.
Three, say we need to fine-tune a vision language model, updates to its weights, may affect its understanding of the representations generated by the image encoder if we don't fine-tune both together.
The new Gemma-4-12B replaces the encoder (with its many attention layers and large parameter count) with a simple linear projection to generate the embeddings for images. That reduces the computational requirements and simplifies the input pipelines for image processing.
I don't have any expertise on the topic though and might very well be wrong on some details.
I know little about law but can we use the word 'attack' for this given that these people pay for these model outputs. Is the output not my property? Does Google have rights to any code Gemini gives me?
Are these AI companies trying to assert a right to choose what I do with content that I paid for?
I don't think I want to live in a world where three Big Corps decide my access to frontier artificial intelligence, and also what I do with the results of my interaction with it.
Honestly, given every executive at these companies is always talking about "distributing intelligence to benefit all humanity", why try to restrict efforts at said distribution?
I don't know if this happens to anyone else but on reading LLM-generated text I did not prompt, my eyes do incredibly quick saccades from start to middle to end in always around <1-2s no matter the length of the text.
It's entirely involuntary, I am just unable to care. It's almost always justified because the text in question is always painfully bloated, and repetitive.
The LLM-text you posted could have been (given I didn't read it carefully):
"Skill issue. Iterate on the output, never accept what you receive on the first pass"
Man, I don't think I could lack enough shame to write something like this.
Much of the post is spent trying to exculpate himself from any responsibility for the agent's behavior. The apology at the end is a "sorry if you felt that way" one.
The tone is incredibly selfish, and unbelievably anti-social. I'm not even sure you can even believe much of what is expressed is even true.
> One person companies will not have a 100 person marketing team to inject ads into every corner of your life.
But they could have a thousand-agent swarm connected via MCP to everything within our field of vision to bury us with ads.
It's been a long time since I read "The Third Wave" and up until 2026, not much has reminded me about its "Small is beautiful", and "rise of the prosumer" themes besides the content creator economy which is arguably the worst thing to ever happen to humanity's information environment, and LLM agent discussions.
No, you cannot ignore every argument by claiming someone else made it before. Make an actual response.
What new opportunities does the LLM create for the workers it may displace? What new opportunities did neural machine translation create for the workers it displaced?
In what way is a text-generation machine that dominates all computer use alike with the steam engine?
The steam engine powered new factories workers could slave away in, demanded coal that created mining towns. The LLM gives you a data centre. How many people does a data centre employ?
The problem with this "creative destruction" hand-wave is:
- there's no thought given to what happens in the interim. Forget the welfare of those displaced, consider what acts the desperation will lead them to.
- these replacement roles may very well never exist or will pay much, much lower than they do now.
- this disruption happens entirely in services, LLMs are not improving agricultural yield, most industries steeped in physical reality will mostly cut overhead for generating text.
- the gains from automation do not necessarily have to diffuse over us all, the capital can simply accumulate in the hands of the firms.
You cannot keep pointing to the past when you are suggesting an entirely new never before seen moment is upon us.
You can increase profits by cutting costs. It is remarkably easier to do in the short term. And even if you choose not to downsize you can drop/stagnate wages to gain from the fact everyone else is downsizing.
The frame is not from our view. It is from that of this singular chicken who has only ever known its keeper's care. As that chicken, we simply do not know if Christmas will ever come.
The collapse of civilizations has happened many times. Today, all of humanity is bound tighter than ever before. In the latter half of the last century, we were on the brink of nuclear war.
New things are happening under the sun every day. If we were that exceptionally smart chicken you describe, then we have reason to expect Christmas.
This is often a separate module grafted onto the main model, and further pre-trained (e.g. OpenAI's CLIP, SigLIP used in the Gemma 3 and PaliGemma series).
The image encoder approach has a few problems.
One problem is that many like Gemma 3's encoder have fixed image resolution constraints and inputs must be resized with all the attendant distortions that causes with spatial understanding. However, the Gemma 4 series image encoders overcame this and can handle variable-dimension inputs.
Two, these image encoders are somewhat large (ranging from 300-500M parameters) requiring extra memory and FLOPs to run.
Three, say we need to fine-tune a vision language model, updates to its weights, may affect its understanding of the representations generated by the image encoder if we don't fine-tune both together.
The new Gemma-4-12B replaces the encoder (with its many attention layers and large parameter count) with a simple linear projection to generate the embeddings for images. That reduces the computational requirements and simplifies the input pipelines for image processing.
I don't have any expertise on the topic though and might very well be wrong on some details.