Social learning: Collaborative learning with large language models(blog.research.google)
blog.research.google
Social learning: Collaborative learning with large language models
https://blog.research.google/2024/03/social-learning-collaborative-learning.html?m=1
13 comments
No human or LLM is too smart alone. Intelligence is a social process intermediated by language.
They said their focus was PII but I imagine Social Learning is also very useful when dealing with copyrighted content too:
- (current level) use your own model to generate a bunch of solutions to a task
- (teacher level) use RAG, web search or a larger model to solve the same task. But always use multiple sources, never drawing from a single copyrighted source example, we want to integrate information.
- (grading & teaching feedback) analyze the issues of your own model and synthesize a training example to fit the issues found - it might lack some facts, or not have some skills
This would be fair use because it studies the shortcomings of the student model compared to the more empowered teacher models. It can also check for regurgitation of the copyrighted content while formulating completely new text to fit the needs of the student.
I prefer to name the method Machine Study rather than Social Learning, but the social part is there. When LLMs get a legitimate way to process copyrighted content it would help their development.
This method will ensure the new model never sees the original copyrighted content, only the teacher vs. student analysis outputs related to that content.
They said their focus was PII but I imagine Social Learning is also very useful when dealing with copyrighted content too:
- (current level) use your own model to generate a bunch of solutions to a task
- (teacher level) use RAG, web search or a larger model to solve the same task. But always use multiple sources, never drawing from a single copyrighted source example, we want to integrate information.
- (grading & teaching feedback) analyze the issues of your own model and synthesize a training example to fit the issues found - it might lack some facts, or not have some skills
This would be fair use because it studies the shortcomings of the student model compared to the more empowered teacher models. It can also check for regurgitation of the copyrighted content while formulating completely new text to fit the needs of the student.
I prefer to name the method Machine Study rather than Social Learning, but the social part is there. When LLMs get a legitimate way to process copyrighted content it would help their development.
This method will ensure the new model never sees the original copyrighted content, only the teacher vs. student analysis outputs related to that content.
When is the Large Google Machine going to learn to be social and offer me a decline all non collaborative cookies button?
Sounds like a stretch of the concept of social learning, and more like vanilla model distillation.
Social learning exist to transcend the limited processing power and limited training data exposure of individual agents through multimodal transfer of their own individual models (distilled down from an individual's entire worldview, sense-of-self, perspectives, skills, semantic memory etc)
LLMs already exploit the propositional transfer of human models over language, and they abuse their massive compute capacity to compress them all in a giant model to simulate-them-all. For sure internally it does have some notion of distribution - as it at least has to distribute the compute at train time - but this is not an agent level distribution - not to confuse with the weaker metaphor of an "agent" used in model architectures -, and the end product presents itself as a singular "agent" with all of the processing power and all the training data that is infinitely copyable.
> "A teacher model provides instructions or few-shot examples to a student model without sharing its private data."
So the real concern is not utilizing social learning to transcend compute and training data limitations, it is about creating inferior models that can be distributed back into the world without giving up all of the secret sauce.
For sure this could work, one could create inferior "agents" from stronger "agents", but we cannot create an even stronger "agent" through the dialogue of two strong "agent"s, because everything to be shared is already perfectly encoded in the model&architecture and perfectly copyable. Therefore this is not social learning at all.
To abuse back their anthropomorphization, they are trying to create a deliberately stupid kid to send out to the world so that the kid doesn't tell all the things mommy and daddy already knows and could have perfectly taught. Because one can make more money from selling/renting a bundle of differently stupid agents than a singular state-of-the-art one I guess?
Social learning exist to transcend the limited processing power and limited training data exposure of individual agents through multimodal transfer of their own individual models (distilled down from an individual's entire worldview, sense-of-self, perspectives, skills, semantic memory etc)
LLMs already exploit the propositional transfer of human models over language, and they abuse their massive compute capacity to compress them all in a giant model to simulate-them-all. For sure internally it does have some notion of distribution - as it at least has to distribute the compute at train time - but this is not an agent level distribution - not to confuse with the weaker metaphor of an "agent" used in model architectures -, and the end product presents itself as a singular "agent" with all of the processing power and all the training data that is infinitely copyable.
> "A teacher model provides instructions or few-shot examples to a student model without sharing its private data."
So the real concern is not utilizing social learning to transcend compute and training data limitations, it is about creating inferior models that can be distributed back into the world without giving up all of the secret sauce.
For sure this could work, one could create inferior "agents" from stronger "agents", but we cannot create an even stronger "agent" through the dialogue of two strong "agent"s, because everything to be shared is already perfectly encoded in the model&architecture and perfectly copyable. Therefore this is not social learning at all.
To abuse back their anthropomorphization, they are trying to create a deliberately stupid kid to send out to the world so that the kid doesn't tell all the things mommy and daddy already knows and could have perfectly taught. Because one can make more money from selling/renting a bundle of differently stupid agents than a singular state-of-the-art one I guess?
Think about the following scenario: I write a calculus book and the agents of this model just modify every example and every definition and change a little the ordering of the material to teach students. Now they are using my book but it seems they are not using my book. Are they trying to copy without copying?
Really interesting work there, and I particularly liked the gif-based story telling - https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg...
I like that they have them, but I really dislike automatically animated anything when I'm trying to read. I installed an extension that can pause gifs. because Google adds them to so many of their blog posts now.
this is solving engagement problem, not readability
try it with a mindlessly boring topic you can barely get through a paragraph of, not something you're actively trying to read
try it with a mindlessly boring topic you can barely get through a paragraph of, not something you're actively trying to read
It would be more useful as something like a video that plays while actually in view. I was actually reading the first part and when I got to the GIF it was in the middle of the animation.
I wonder where this would be practical. How big would the teacher model need to be?
[deleted]
It reminds me of the general idea of generative adversarial networks where instead of a teacher and learner, you have a one trying to learn real inputs from fake inputs while the other tries to trick the first with fakes, with both training each other and the ultimate goal is to have the latter create realistic fakes.
I'm sure it's been or is being researched, but since I'm new to LLMs, my immediate thought was having code-aware models where one tries to find and fix vulnerabilities in a codebase while another tries to exploit them where the ultimate goal is to secure a codebase.
I'm sure it's been or is being researched, but since I'm new to LLMs, my immediate thought was having code-aware models where one tries to find and fix vulnerabilities in a codebase while another tries to exploit them where the ultimate goal is to secure a codebase.
Edit: misunderstood the paper on first skim, they don’t actually want smaller students, they might actually want bigger ones. So this is more a privacy thing