Isn't the inference cost of running these models at scale challenging? Currently it feels like small LLMs (1B-4B) are able to perform well for simpler agentic workfows. There are definitely some constraints but surely much easier than to pay for big clusters on cloud running for these tasks. I believe it distributes the cost more uniformly
I would love to talk more about it and understand your take. Is there a way I can reach out to you? I have been reading the content in web 3 and form my own opinion on the merits/demerits.
I was thinking along ML. How analytics and ML happens on user data which is stored on data silos. For example, twitter has data about all our posts and interactions and uses that data to build recommendation models which it monetizes as to who comes on top.
In decentralized world, everyone can build their models, host them and train them on user data. But since, I wouldn't want my data to be used for any other purpose, I don't want them to make a copy of that data. So they send their models (after taking my consent) to my device, train it locally and send the aggregated model back.
I believe edge computing is one of the game changing technologies for the decade. I am not sure if it can fall under the purview of Web 3 or not. For example one of the library I implemented was along training ML models on user devices instead of the cloud maintaining privacy and personalization.
That's why not giving them ability to copy data, allow them to use that data only on your device and send aggregated information back. So they cannot copy the data
That's what I am worried about, from its description the intent looks good but being aligned to only one industry or vertical defeats the purpose of ubiquity. People wouldn't trust it if they all just see is a volatile currency
I do care if that data gets leaked or if that data is used by third party companies for things I have bought and I do not want them to know. Why should a bank do my credit scoring based on my what am watching on Amazon or what books am I reading
Exactly if that is the case, it should also include technologies where data ownership lies with the users. After all data itself is a huge resource. One of the biggest reasons corporates could monopolise the platform because as users we only had our own data but no access to others data.
Maybe a consent mechanism to use other's data in privacy safe manner, where anyone can build their analytics models and people can choose which one to follow or choose. Seems like a interesting proposition.
Edge computing for machine learning. Instead of running ML models on the cloud, I train them on user's device, ask these devices to offload computation between each other and give me the best performance out there. One good example is recommendations that work offline for you. Imagine you are listening to spotify in offline mode (with you downloaded playlist) and recommendations adapt accordingly even without internet!
Makes sense though I wonder what would be the original source of that data (someone like google/microsoft must be logging user data and then making some parts of it anonymized and public).
Maybe also look into on-device learning, it can be efficiently hooked up with differential privacy and give more specific results.
How do you train newer models then? From what I read you use public datasets to train your models but what about in future? You would need some kind of data collection mechanism?
Gpt-2 and gpt-3 are great but the datasets they are trained would soon get old.