Using ChatGPT Plugins with LLaMA(blog.lastmileai.dev)
blog.lastmileai.dev
Using ChatGPT Plugins with LLaMA
https://blog.lastmileai.dev/using-openais-retrieval-plugin-with-llama-d2e0b6732f14
141 comments
Author here. We definitely agree these models are really powerful and can figure out a lot on their own. A protocol wouldn't be restrictive to their flexibility, but it'll allow different clients running different models to still be able to interact with the same plugins.
Defining a standard around external memory, authentication, rules-based engines for preprocess/post-processing, and declarative or dynamic chaining of actions can help make plugins model-agnostic, and benefit all of us as developers and users.
Defining a standard around external memory, authentication, rules-based engines for preprocess/post-processing, and declarative or dynamic chaining of actions can help make plugins model-agnostic, and benefit all of us as developers and users.
And this is how you end up with 15 different standards.
LLaMA can work with ChatGPT plugins and ChatGPT can work with LLaMA plugins but why would you want plugin developers to have to choose between those options?
LLaMA can work with ChatGPT plugins and ChatGPT can work with LLaMA plugins but why would you want plugin developers to have to choose between those options?
> And this is how you end up with 15 different standards.
Actually, this is something never seen before. This is an entirely new way of ending up with 15 different standards.
Actually, this is something never seen before. This is an entirely new way of ending up with 15 different standards.
Same reason with ended up with 15 different standards elsewhere: diversity.
We're in uncharted waters and we have no idea which format will be optimal for one LLM let alone all of them. Given how much prompt engineer has become a thing, finding optimal formats is important.
We're in uncharted waters and we have no idea which format will be optimal for one LLM let alone all of them. Given how much prompt engineer has become a thing, finding optimal formats is important.
I would argue we are in uncharted waters, so we can't even say for certain that it is important to find optimal formats.
Maybe 15 "standards" aren't enough and we will have 1500 that LLMs will happily use and learn
xkcd/927 may loose relevance in the era of LLM
Maybe 15 "standards" aren't enough and we will have 1500 that LLMs will happily use and learn
xkcd/927 may loose relevance in the era of LLM
or you build a short dev cycle that builds the way to consume it
OpenAI with 400 employees are breaking Google’s ankles on the court. What do you guys think? Is Google in trouble? Their entire business model is in jeopardy. If Google releases a oroduct as as good as ChatGPT 4, which is unlikely, they’ll
kill their revenue. If they don’t release it, they’ll gradually lose market share. They are in an unwinnable situation.
This week I created a Microsoft account just to check how they are integrating GPT into Bing, and while they made me believe that I was using a Chat-enabled Bing, it was worthless for me.
I basically wanted to know how I can do raycasting in Three.js, but after a lot of using ChatGPT for trying to solve my issue, I learned that I can't do what I want by using the normal raycaster integrated Three.js: Intersect a geometry which has a displacement map on it.
ChatGPT failed to understand that the raycaster works on the CPU, but the displacement map of the material is applied on the GPU side, so the displaced geometry won't be used, only the original one. It managed to explain this to me, that this was not possible, but each sample code repeatedly did as if it was possible, until I gave up.
Then I created the Microsoft account and started to ask for solutions, and it was the most useless garbage, despite of the web claiming that it is using GPT-4.
In my eyes MS has failed at integrating a chatbot; maybe it's ok for cooking or having fun, I haven't tried that. And OpenAI has nothing else but a chatbot and other nice AI. Let someone better come and OpenAI will be a remarkable entry in the history books (first popular AI application) with a final entry that OpenAI got acquired by Microsoft.
Let's see what Google makes out of it.
I basically wanted to know how I can do raycasting in Three.js, but after a lot of using ChatGPT for trying to solve my issue, I learned that I can't do what I want by using the normal raycaster integrated Three.js: Intersect a geometry which has a displacement map on it.
ChatGPT failed to understand that the raycaster works on the CPU, but the displacement map of the material is applied on the GPU side, so the displaced geometry won't be used, only the original one. It managed to explain this to me, that this was not possible, but each sample code repeatedly did as if it was possible, until I gave up.
Then I created the Microsoft account and started to ask for solutions, and it was the most useless garbage, despite of the web claiming that it is using GPT-4.
In my eyes MS has failed at integrating a chatbot; maybe it's ok for cooking or having fun, I haven't tried that. And OpenAI has nothing else but a chatbot and other nice AI. Let someone better come and OpenAI will be a remarkable entry in the history books (first popular AI application) with a final entry that OpenAI got acquired by Microsoft.
Let's see what Google makes out of it.
>ChatGPT failed to understand that the raycaster works on the CPU, but the displacement map of the material is applied on the GPU side, so the displaced geometry won't be used, only the original one. It managed to explain this to me, that this was not possible, but each sample code repeatedly did as if it was possible, until I gave up.
Reflecting on it, what's crazy is that this comment will likely wind up in the training data for GPT-N+1, and then GPT-N+1 will get it right.
Reflecting on it, what's crazy is that this comment will likely wind up in the training data for GPT-N+1, and then GPT-N+1 will get it right.
What's also crazy is that GP's comment exist in the first place. Five years ago, complaining that a chatbot failed to understand the complexities of realtime 3D rendering when explained to it in plain English would be seen as confusing science fiction with reality. Hell, five years ago, we'd be reminded to avoid using anthropomorphizing language (such as "the software failed to understand"). Today, that same language is the most apt description of what we observe.
I can tell you what will be the answer of Bard:
"I am just an AI language model, I can't help you"
"I am just an AI language model, I can't help you"
...instead of pretend knowing the answer and splitting out nonsense
The problem is that it still spews tons of non sense. It literally makes up imports when commanded to use a library/package even if it "knows" said package, considering it itself suggested to use it in the first place!
Partial information at-least gives me something to correct / some hint of the problem space to explore further.
ChatGPT's partial answers were helpful, yet the whole process was time-consuming. I guess that googling for it would have taken at most 5-10 minutes, but I wanted to see how ChatGPT behaves.
But Bing on the other hand, it didn't even bother to spit out ChatGPT-like sentences but only pointed me to some non-helpful Stack Overflow entries.
I just re-logged-in into Bing to search for my query, I found it in the history and re-visited it, now it is handing me out an answer which looks like it was generated with ChatGPT, but it's still the same buggy code. Chatting with it shows the same problems which ChatGPT has.
I wonder if GPT-4 could give me the correct answer (manually displacing the vertices, not in a shader).
Wow, now after chatting and then performing a normal query and then going back to the chat-mode, the entire chat history was gone...
But Bing on the other hand, it didn't even bother to spit out ChatGPT-like sentences but only pointed me to some non-helpful Stack Overflow entries.
I just re-logged-in into Bing to search for my query, I found it in the history and re-visited it, now it is handing me out an answer which looks like it was generated with ChatGPT, but it's still the same buggy code. Chatting with it shows the same problems which ChatGPT has.
I wonder if GPT-4 could give me the correct answer (manually displacing the vertices, not in a shader).
Wow, now after chatting and then performing a normal query and then going back to the chat-mode, the entire chat history was gone...
This is what ChatGPT-4 says regarding this specific case (you can see the prompt used as well, it's just a copy-paste of your comment):
https://pastebin.com/PPy4vMrU
It seems to write code that does the displacement manually "// Iterate over the geometry's vertices and apply the displacement, geometry.vertices.forEach{...}"
It seems to write code that does the displacement manually "// Iterate over the geometry's vertices and apply the displacement, geometry.vertices.forEach{...}"
Thank you for the feedback! Looks like I'm going to subscribe to plus then.
It constantly tells me it can't do things it just did, and makes stuff up all the time.
Microsoft GitHub Copilot X might do better than GPT-4 Bing Chat regarding coding. I'm still on their waiting list for that but it looks promising.
Chat in the IDE is going to be game changing for me. I find myself switching between IDE and ChatGPT when co-pilot just isn't giving me a sensible suggestion, and I've found the combination of the two to be pretty epic. Can't wait to have it all tightly integrated. Also very interested in voice.
Is using embedding + retrieval plugin a potential solution in this case? (honest question - I don't have expertise in this area). There are many helpful explanations on Three.js Discourse forum, Three.js/Pmndrs Discord, official docs, blogs, and private resources like books and course materials. If someone could create embeddings for all of these resources and make it accessible via plugin, we could get more accurate and up-to-date answers (I noticed that GPT-3.5 occasionally produce deprecated code like THREE.Geometry, while GPT-4 seems to handle it better). Practically, private resources must be excluded.
I've also not seen great results on three.js or A-Frame questions from any of these models. I'm guessing it's simply because there's a limited corpus of text from which to learn, however I wonder if LLM's lack of inherent spatial awareness contributes. Admittedly three.js and related concepts can be confusing for humans too
I suspect it could soon be feasible to fine-tune on code with limited open data through a bootstrapping approach.
Give it the source code, a test library and access to a dev environment and with some prompting it could start to experiment with increasingly complex use cases, learning from successful attempts. This would depend on the model’s ability to understand what a successful outcome is so it can define test cases, which might be harder if the output isn’t text but not impossible.
Being able to give a model expert knowledge on an undocumented library or language seems like it could help accelerate adoption of new technologies that might otherwise suffer from network effects as users get used to AI-assisted development. Not to mention automated testing, finding edge cases and making pull requests to fix them, security, etc.
A human taking time to experiment with their assumptions and gain experience with an unfamiliar subject is in a sense creating their own training data as well.
Give it the source code, a test library and access to a dev environment and with some prompting it could start to experiment with increasingly complex use cases, learning from successful attempts. This would depend on the model’s ability to understand what a successful outcome is so it can define test cases, which might be harder if the output isn’t text but not impossible.
Being able to give a model expert knowledge on an undocumented library or language seems like it could help accelerate adoption of new technologies that might otherwise suffer from network effects as users get used to AI-assisted development. Not to mention automated testing, finding edge cases and making pull requests to fix them, security, etc.
A human taking time to experiment with their assumptions and gain experience with an unfamiliar subject is in a sense creating their own training data as well.
Google's secret sauce is not its product but how it is able to monetize its userbase. OpenAI isn't going to become Google by launching a better language model and hoping that enough people pay $20/mo for it. It has never really been about search at all. The various forms of user tracking, AdWords, AdSense are all alive and well, and are very far from OpenAI's expertise.
Let's at least wait for the $10B spending cash from Microsoft to run out and the company to become profitable before we crown them Google killers.
Let's at least wait for the $10B spending cash from Microsoft to run out and the company to become profitable before we crown them Google killers.
Since ChatGPT became publicly accessible, I can tell that my use of google has gone down
1. I had an interview for a CIO position - I fed the job description, ask ChatGPT to generate top 10 questions to be asked based on it , I was asked 90% similar or the same questions. Heck I even asked to provide best answers and they were great. It even generated a question on personalization and a/b testing which I thought would be highly unlikely to be asked, but was indeed asked. It was as if the interviewer also generated the same question.
2. For my current company, I am in charge of building a brand new engineering team, for which I wanted to write vision, mission and strategy statements. Previously I would have googled stuff, this time I asked ChatGPT to improve it based on my rough draft. And boy they are great.
I can go on.
My point is that not everyone needs to ask ChatGPT to generate ray tracing code using three.js or whatever. Most of us are just regular folk need regular help and ChatGPT is a game changer for it. Heck it even helped me with some excel macro I needed to write to do some data manipulation.
1. I had an interview for a CIO position - I fed the job description, ask ChatGPT to generate top 10 questions to be asked based on it , I was asked 90% similar or the same questions. Heck I even asked to provide best answers and they were great. It even generated a question on personalization and a/b testing which I thought would be highly unlikely to be asked, but was indeed asked. It was as if the interviewer also generated the same question.
2. For my current company, I am in charge of building a brand new engineering team, for which I wanted to write vision, mission and strategy statements. Previously I would have googled stuff, this time I asked ChatGPT to improve it based on my rough draft. And boy they are great.
I can go on.
My point is that not everyone needs to ask ChatGPT to generate ray tracing code using three.js or whatever. Most of us are just regular folk need regular help and ChatGPT is a game changer for it. Heck it even helped me with some excel macro I needed to write to do some data manipulation.
Random and silly use case from today: I needed to import a list of Polish bank holidays into our shared calendar. Last year, I tried to Google for a ready-made .ical file, and after failing to find one in 30 minutes, ended up manually entering everything into the calendar myself. This year, I just opened the first page, copy-pasted the listing to ChatGPT, and prefixed it with a very trivial prompt:
"Below is a list of holidays in Poland for 2023. Please rewrite it into ICAL format."
Five seconds later, I had a blob of text I could copy over to a text editor, save as .ical, and import to the calendar.
The most crazy thing is just how natural these kinds of interactions are becoming to me. It literally took me more time to write this comment than it took to do the thing I just described.
"Below is a list of holidays in Poland for 2023. Please rewrite it into ICAL format."
Five seconds later, I had a blob of text I could copy over to a text editor, save as .ical, and import to the calendar.
The most crazy thing is just how natural these kinds of interactions are becoming to me. It literally took me more time to write this comment than it took to do the thing I just described.
Heh, I’m glad it worked for your interview! Reminds me of when I’ve suspected that the interviewer and candidate both googled their way to the same “50 best project manager interview questions” site.
I pirate and the private usenet website I use has this funky search rules for "sphinx" they call it. Never seen it before.
I copied and pasted the search engine query term rules they have on the page, then said I want to find all American Dad episodes for season 18 with x265 codec. And it gave me the query to use.
This thing saves a bunch of time all over the place. This is truly the bicycle for the mind.
I copied and pasted the search engine query term rules they have on the page, then said I want to find all American Dad episodes for season 18 with x265 codec. And it gave me the query to use.
This thing saves a bunch of time all over the place. This is truly the bicycle for the mind.
Sphinx is open source search engine. Sphinx allows full-text searches over large data sources.
It's not open source since 2017. The open source fork is https://github.com/manticoresoftware/manticoresearch
Microsoft, through Bing, already has a small ads platform.
I think it wouldn't be too tricky for them to make sure that whenever the user asks ChatGPT "My omlettes keep sticking to my pan, what am I doing wrong?", it can reply with "You probably have an old frying pan with a damaged coating. [Click here] to see my recommendations of frying pans you can have delivered tomorrow.".
I think it wouldn't be too tricky for them to make sure that whenever the user asks ChatGPT "My omlettes keep sticking to my pan, what am I doing wrong?", it can reply with "You probably have an old frying pan with a damaged coating. [Click here] to see my recommendations of frying pans you can have delivered tomorrow.".
OpenAI isn't trying to become Google, though. ChatGPT is marketing.
OpenAI's market is more comparable to AWS. Each product is another backing service for some other product to integrate with and sell to _their_ users.
OpenAI's market is more comparable to AWS. Each product is another backing service for some other product to integrate with and sell to _their_ users.
At some point, we have to move away from ad based model. I already pay for youtube premium, love the experience. I am happy to pay for ChatGPT as well, especially it is a good assistant to knowledge based workers, those who wouldn't shy away from paying for it. I still believe Google can win this race just like they did with Android.
It gives me pause to realize that the GOOG has been my internet, my video clips, my wallet, my phone, my operating system, my cable TV alternative that costs arguably more—
and now it finally wants to commit to our eternal friendship!
I’m a fan in general, techno-optimism is the only fun way to be.
But for your consideration: once upon a time, the GOOG only had 400 emoloyees too. And Jeff ran a bookstore in the PNW. I wonder every day. What is the future of OpenAI and this largely unseen goldrush?
Edit: full disclosure, and an employer! It was a while ago. But I do wonder sometimes already if google and jeff bezos are my real parents. Now there’s a new player!
and now it finally wants to commit to our eternal friendship!
I’m a fan in general, techno-optimism is the only fun way to be.
But for your consideration: once upon a time, the GOOG only had 400 emoloyees too. And Jeff ran a bookstore in the PNW. I wonder every day. What is the future of OpenAI and this largely unseen goldrush?
Edit: full disclosure, and an employer! It was a while ago. But I do wonder sometimes already if google and jeff bezos are my real parents. Now there’s a new player!
>Google's secret sauce is not its product but how it is able to monetize its userbase.
It is still primarily reliant on search as start of funnel though
It is still primarily reliant on search as start of funnel though
$20/month? I can bet that Apple executives are already trying to broker an exclusive deal for Siri-GPT-4 for tens of billions.
Google is only temporarily behind. It will release a decent model in max 6-12 months, and not just Google, everyone will release a model, even open source will have some. OpenAI is enjoying a short window of exclusivity.
Just playing casually with NanoGPT (https://github.com/karpathy/nanoGPT) with a desktop holding a 2080ti, it's really really really clear to me that the path to get to a pre-fine-tuned LLM is remarkably easy. RLHF is the piece above this which appears to also be surprisingly easy (if Sam Altman is to be believed). The juice is making these tools incredibly easy.
I think the barrier to entry here is low. OpenAI is ahead now, but I doubt that lives forever.
I think the barrier to entry here is low. OpenAI is ahead now, but I doubt that lives forever.
Exactly, almost nobody is going to need a trillion parameter LLM
Its going to go client side, at the OS level and have like 1% of the mental capacity and be good enough
Its going to go client side, at the OS level and have like 1% of the mental capacity and be good enough
This comment might end up being a bit like "640kb of RAM is more than anyone will ever need!"
GPT3 already used almost 1/5th of that figure at 180bn parameters, and PaML uses 500bn.
GPT3 already used almost 1/5th of that figure at 180bn parameters, and PaML uses 500bn.
Okay, if client side resources expand then larger parameter LLMs will be used. There.
The point is that it will be ubiquitously client side, and it will happen faster than newer hardware comes out. Current hardware is very limited and slow in getting output from LLMs.
The point is that it will be ubiquitously client side, and it will happen faster than newer hardware comes out. Current hardware is very limited and slow in getting output from LLMs.
I'm not convinced - at least on current hardware this seems well positioned for the cloud.
My tiny Alexa puck isn't going to run a 180bn parameter LLM that runs best on 10 graphics cards any time soon, however it can already call a simple API and get a response in only 50ms more. I suspect people will prefer the cloud overhead of 50ms for a better response for a bigger model for a lot of queries.
But who knows at this stage! I guess it could go either way depending on how both hardware and these models advance.
I just assume that in the close future, most people are going to be interacting with LLM's on low-cost devices with limited/varied compute.
My tiny Alexa puck isn't going to run a 180bn parameter LLM that runs best on 10 graphics cards any time soon, however it can already call a simple API and get a response in only 50ms more. I suspect people will prefer the cloud overhead of 50ms for a better response for a bigger model for a lot of queries.
But who knows at this stage! I guess it could go either way depending on how both hardware and these models advance.
I just assume that in the close future, most people are going to be interacting with LLM's on low-cost devices with limited/varied compute.
Google search has been declining in quality for years. In pursuit of profits they’ve ramped up ads, but the SEO garbage and “helpful” search “corrections” has also fully destroyed the organic results too.
Chat as a search replacement is compelling, and one reason is for now it’s not monetized by ads. That may change, but Google’s stranglehold is materially weakened.
Chat as a search replacement is compelling, and one reason is for now it’s not monetized by ads. That may change, but Google’s stranglehold is materially weakened.
I wonder how long before we see organic looking promotional recommendations in LLM replies. Like one of those youtubers that has to thank Skillshare twice in the same video and tell us how great NordVPN is.
I wouldn't say that promotional section of a video is "organic" I'd be more worried of sentences or paragraphs being thrown in the reply the same way ads are shown in Reddit or Twitter feeds
No.
I'm pretty sure Google has an AI that is on par with ChatGPT. The reason they still need more time is because they are now fine-tuning the AI to include their paying partners' products into the response.
It is VERY easy to make ChatGPT lie on future questions by injecting the right prompts. That means it is equally easy to inject ads into ChatGPTs responses. And if Google can pull that off, it'll be even more profitable than tolerating SEO spam so that brands need to buy keyword ads for their own homepage.
As a mental model, I believe the future of Google will be a personal butler that answers all of your questions. The butler usually does a good job, so you trust him. But unknown to you, the user, your butler is being blackmailed and forced to lie to you on some days for some questions.
I'm pretty sure Google has an AI that is on par with ChatGPT. The reason they still need more time is because they are now fine-tuning the AI to include their paying partners' products into the response.
It is VERY easy to make ChatGPT lie on future questions by injecting the right prompts. That means it is equally easy to inject ads into ChatGPTs responses. And if Google can pull that off, it'll be even more profitable than tolerating SEO spam so that brands need to buy keyword ads for their own homepage.
As a mental model, I believe the future of Google will be a personal butler that answers all of your questions. The butler usually does a good job, so you trust him. But unknown to you, the user, your butler is being blackmailed and forced to lie to you on some days for some questions.
> I'm pretty sure Google has an AI that is on par with ChatGPT. The reason they still need more time is because they are now fine-tuning the AI to include their paying partners' products into the response.
Nice try Sundar!
Nice try Sundar!
Doubtful? They basically invented transformers
The person here claims that Google has an amazing GPT-4 killer ready to launch, and that they are just waiting to integrate with ads partners or something.
If you want to believe that Google is intentionally holding it back and, as a diversion, releasing a very buggy software to the public, then why not, but it doesn't make sense at all.
Even in terms of costs, they could force a limit of X messages and then push to upgrade to a paid subscription.
In terms of reputation or safety they have DeepMind as a separate entity.
This is like claiming that Tesla has a revolutionary car, but that Tesla is intentionally not releasing it, and instead waiting that someone else does.
As a fictitious and parallel example: it's not because Xerox invented the mouse that it is a great company for innovation or that they wouldn't get eaten by others.
(site-note: some claims Xerox didn't even invent the mouse).
Regarding transformers:
The transformers guys ("Attention Is All You Need") don't appear to work for Google for a long-time, and the comments from the team are not so glorious from what I see (if I remember well, the Character.AI is very harsh on Google all the time claiming it was not good for innovation).
They may be missing the RLHF part for example or other part of the magic, and between 2017 and 2023 is an insanely long period where many impactful new things have been discovered.
Doing the 90% is easy now with LLMs, but each % of improvement is very difficult.
If you want to believe that Google is intentionally holding it back and, as a diversion, releasing a very buggy software to the public, then why not, but it doesn't make sense at all.
Even in terms of costs, they could force a limit of X messages and then push to upgrade to a paid subscription.
In terms of reputation or safety they have DeepMind as a separate entity.
This is like claiming that Tesla has a revolutionary car, but that Tesla is intentionally not releasing it, and instead waiting that someone else does.
As a fictitious and parallel example: it's not because Xerox invented the mouse that it is a great company for innovation or that they wouldn't get eaten by others.
(site-note: some claims Xerox didn't even invent the mouse).
Regarding transformers:
The transformers guys ("Attention Is All You Need") don't appear to work for Google for a long-time, and the comments from the team are not so glorious from what I see (if I remember well, the Character.AI is very harsh on Google all the time claiming it was not good for innovation).
They may be missing the RLHF part for example or other part of the magic, and between 2017 and 2023 is an insanely long period where many impactful new things have been discovered.
Doing the 90% is easy now with LLMs, but each % of improvement is very difficult.
> If you want to believe that Google is intentionally holding it back and, as a diversion, releasing a very buggy software to the public, then why not, but it doesn't make sense at all.
One reason would be capacity. They've increased their server production at least 4x since the announcement of Bard, and that's recent enough that I feel like any decom and deployment, even if it's 1:1 to existing data centers, hasn't been completed.
This maps to your last point somewhat, since anticipation is hard. Google has learned not to fully open the all the taps since they can't commit to a consistent product. Assistant and Home technologies were heavily encumbered by patent defense, security awareness, and privacy controls, so much so that the product capabilities they advertised during the Pixel 3 launch were permanently rolled back. They're not promoting or promising anything about Bard, if you notice, because people may not appreciate every new feature, but they never forget when you take things away.
One reason would be capacity. They've increased their server production at least 4x since the announcement of Bard, and that's recent enough that I feel like any decom and deployment, even if it's 1:1 to existing data centers, hasn't been completed.
This maps to your last point somewhat, since anticipation is hard. Google has learned not to fully open the all the taps since they can't commit to a consistent product. Assistant and Home technologies were heavily encumbered by patent defense, security awareness, and privacy controls, so much so that the product capabilities they advertised during the Pixel 3 launch were permanently rolled back. They're not promoting or promising anything about Bard, if you notice, because people may not appreciate every new feature, but they never forget when you take things away.
Capacity issue may be alleviated by limiting the numbers of users who have access, and then progressively scaling it up, rather than releasing an inferior product (assuming the superior product really exists).
That being said, for all the other points I agree with you.
Google is often a target of any legal claim, so perhaps this makes them more risk-averse too.
That being said, for all the other points I agree with you.
Google is often a target of any legal claim, so perhaps this makes them more risk-averse too.
Also, it's a meme that when you buy a Google gadget, the customers are its QA team, but it's accurate that they do find ways to sell us products containing their pet technologies so they can gather data and sell them to their real customers - defense, healthcare, manufacturing, etc.
Bard is no different. We'll all be performing the RLHF, and they'll sell off the improvements.
If we passed a law that made it illegal to work for free, companies like Google would be falling over themselves to institute UBI to avoid paying out to their userbases.
Bard is no different. We'll all be performing the RLHF, and they'll sell off the improvements.
If we passed a law that made it illegal to work for free, companies like Google would be falling over themselves to institute UBI to avoid paying out to their userbases.
Everyone forgot that we have been living in an AI age for a decade now.
Where's the Google product, then?
They keep slipping. One week ChatGPT, the next GPT4, the next Plugins, ...
Everyone is already going to be building on OpenAI by the time Google says it's ready.
Xerox, HP, and Bell Labs could have all said the same thing as you about having invented modern computing devices, and look what happened to them.
Google here is Xerox. Or IBM.
They keep slipping. One week ChatGPT, the next GPT4, the next Plugins, ...
Everyone is already going to be building on OpenAI by the time Google says it's ready.
Xerox, HP, and Bell Labs could have all said the same thing as you about having invented modern computing devices, and look what happened to them.
Google here is Xerox. Or IBM.
Or just ads on the side that update with every new prompt. Guaranteed. I’d guess within 5 months from OpenAI or Google.
Why on the sidebar? It will just be integrated into the response: “sure, I can give you some suggestions for easy, nutritious meals for the working professional, but first I’d like to mention Athletic Greens(tm). Athletic Greens is one of the best ways to …”
It will be more likely start as a chat query about a product and then go into product research and then the model will casually suggest reasons for which product to choose. Example: which is the best magnesium supplement? The model will first guide the user to choose one form(of Mg) over other citing pros/cons. Then it will talk about different products for that type of Mg and suggest that one producer is better/worse because of reviews/issues/problems etc.
Guess I need an LLM to block ads from the output of other LLMs.
People keep repeating that google has superior tech. They haven’t released any of it. I’m beginning to think their “AI lead” is a facade.
I still feel miles away from trusting AI with my most ad-friendly queries. Things like: where should I eat? Who is a good plumber, etc.
Maybe someone can explain to me how an AI would revolutionize that, it either requires I trust it’s opinions or it does something similar to google: give me a list with stars and reviews and let me make my own choice.
Or maybe people are lazy and in two years our phones will be telling us: “the best movie for you to watch right now is Creed 4, press here to buy a ticket”
Maybe someone can explain to me how an AI would revolutionize that, it either requires I trust it’s opinions or it does something similar to google: give me a list with stars and reviews and let me make my own choice.
Or maybe people are lazy and in two years our phones will be telling us: “the best movie for you to watch right now is Creed 4, press here to buy a ticket”
I mean, just like any other advertiser, they can build/buy a profile of your likes and dislikes from your previous requests and an online data broker.
With that information, they can then feed it through standard matchmaking algorithms (which is nothing new) to find the best X movies/restaurants/etc given your profile and any specific requests you give it.
This seems perfectly reasonable, the only thing new here would be integrating existing search and recommendation behavior into the model, which doesn’t seem difficult given there’s a plug-in system which is meant to do pretty much that.
You could probably build a proto-version of this by making a prompt that includes all the most relevant profile information about yourself and asking it to choose something for you based on those variables.
With that information, they can then feed it through standard matchmaking algorithms (which is nothing new) to find the best X movies/restaurants/etc given your profile and any specific requests you give it.
This seems perfectly reasonable, the only thing new here would be integrating existing search and recommendation behavior into the model, which doesn’t seem difficult given there’s a plug-in system which is meant to do pretty much that.
You could probably build a proto-version of this by making a prompt that includes all the most relevant profile information about yourself and asking it to choose something for you based on those variables.
That behavior is what people dislike about Google though. Filtering and biasing search returns is a huge area of mistrust for them. OpenAI's fix to this is an even more restrictive and opinionated version?
That is a total nightmare once they decide to monetize it.
That is a total nightmare once they decide to monetize it.
That behavior is what power users (read this audience) dislike about Google. They wouldn’t have gotten to be a powerhouse in search and personalized results if the strategy didn’t work for the mass market audience.
The reason sponsored ads are first in the results is because people tend to click the first result without thinking about the consequences of that first result being sponsored.
ChatGPT might not go down this path, but it’s almost certain someone will, and more than likely that Google will do exactly that within Bard.
Do you think that people won’t appreciate an assistant that already knows everything about you and can conversationally recommend exactly what they were looking for? And do you think corporations won’t capitalize on the clear opportunity that’s there?
The reason sponsored ads are first in the results is because people tend to click the first result without thinking about the consequences of that first result being sponsored.
ChatGPT might not go down this path, but it’s almost certain someone will, and more than likely that Google will do exactly that within Bard.
Do you think that people won’t appreciate an assistant that already knows everything about you and can conversationally recommend exactly what they were looking for? And do you think corporations won’t capitalize on the clear opportunity that’s there?
Google is held hostage by their current revenue model (ads, ads, ads) and their belief that it can't carry over to AI-based products. I can't blame them for focusing so much on it, but they also forget how they got to ads making up so much of their revenue. They shook up the ad market back when, and AI can help do so again.
Microsoft is focused more on incubating OpenAI, getting a viable product to market now, and focusing on monetizing it down the road. Microsoft knows AI can complement an ad revenue model.
Google just opened up Bard to testers and I've found it to be complete garbage in comparison to what Bing Chat/GPT-4 can do. Bard was not afraid to just make up random things, that were verifiably false, and present them as fact. GPT seems to know enough to let you know when it's unsure.
Microsoft is focused more on incubating OpenAI, getting a viable product to market now, and focusing on monetizing it down the road. Microsoft knows AI can complement an ad revenue model.
Google just opened up Bard to testers and I've found it to be complete garbage in comparison to what Bing Chat/GPT-4 can do. Bard was not afraid to just make up random things, that were verifiably false, and present them as fact. GPT seems to know enough to let you know when it's unsure.
> Google is held hostage by their current revenue model (ads, ads, ads) and their belief that it can't carry over to AI-based products.
What tells you that Google thinks that ad-based monetization can't carry over to AI-based product?
What tells you that Google thinks that ad-based monetization can't carry over to AI-based product?
There was an article posted here on Hacker News about a Google exec basically saying AI was a fad not worth chasing, and it had no viable means of monetizing with ads. I was shocked to read the article and flabbergasted that Google could have such a view.
To me it felt like maybe they were saying one thing, while doing another, to attempt to calm investor nerves as they try to play catch-up.
I felt most the entrenched folks were so consumed by trying to develop their AI to be ethical that they allowed OpenAI to run laps around them.
How the next decade unfolds in the AI arena will be most interesting. I wouldn't have thought Google could ever be knocked of their throne as dominant search engine in the space previously, but I now feel they have never been more vulnerable than they are now.
To me it felt like maybe they were saying one thing, while doing another, to attempt to calm investor nerves as they try to play catch-up.
I felt most the entrenched folks were so consumed by trying to develop their AI to be ethical that they allowed OpenAI to run laps around them.
How the next decade unfolds in the AI arena will be most interesting. I wouldn't have thought Google could ever be knocked of their throne as dominant search engine in the space previously, but I now feel they have never been more vulnerable than they are now.
Do you have the article link? That does seem shocking.
Of course Microsoft doesn’t have the same cultural attachment to an ad model, they were born and raised as a selling software company. Shipping software to renting it was a leap, but one they made long ago.
Releasing a LLM would not kill Google's revenue. It would increase it. Because more people would use Google than before. And they could still show relevant ads.
The big question is if they can catch up with OpenAI. OpenAI is a moving target. And it seems they are moving fast.
It could still be possible that Google catches up because of the UI though. Many people don't like having to log in to ChatGPT. Bing's UI is a disaster and you have to log in.
I always thought that Google won the search war not only because of their good search results, but also because of their clean UI.
The big question is if they can catch up with OpenAI. OpenAI is a moving target. And it seems they are moving fast.
It could still be possible that Google catches up because of the UI though. Many people don't like having to log in to ChatGPT. Bing's UI is a disaster and you have to log in.
I always thought that Google won the search war not only because of their good search results, but also because of their clean UI.
I'm not so sure about the ads.
Search lends itself well to interleaving ads with output, so you can place more ads while maintaining user tolerance and "get away with them". With LLM output you have to show the ads in the margins as integrating ads into the output itself would lower performance.
Sure, you can do that, but does it fully substitute for previous ad revenue? It seems to me that the form factor of LLM search vs trad search has reduced ad surface area.
Microsoft is well-positioned to monetize LLMs in other ways, e.g. via 365 and other subscription services, many aimed at enterprises/b2b. Google is much less well-developed there atm. If they can subsidize lower ad annoyance in their search/chat product via other revenue that could make a real diff.
Search lends itself well to interleaving ads with output, so you can place more ads while maintaining user tolerance and "get away with them". With LLM output you have to show the ads in the margins as integrating ads into the output itself would lower performance.
Sure, you can do that, but does it fully substitute for previous ad revenue? It seems to me that the form factor of LLM search vs trad search has reduced ad surface area.
Microsoft is well-positioned to monetize LLMs in other ways, e.g. via 365 and other subscription services, many aimed at enterprises/b2b. Google is much less well-developed there atm. If they can subsidize lower ad annoyance in their search/chat product via other revenue that could make a real diff.
If I had to guess, it’s probably going to be more like YouTube: Watch (click through) these 2 ads to get to your LLM results. If you have to provide updates/corrections, there’s some additional space for ads.
I think the ads will become even better.
Say you put the message you just posted through Google to correct style, facts and spelling. You know how it would reply? Here we go:
The text is pretty awesome. I would only suggest changing the word "interleaving" to "interspersing" to improve the style of the text. "Interspersing" is an alternative term for "mixing" or "adding in between," which better conveys the idea of placing ads within search results. You know what I would also change? Your <related product>. Since you seem to be deep into technology in general and the internet in particular, you will love <related product>. Since you wrote such a thoughtful text about ads, I will tell you the secret discount code "adsMakesMeSmile" to get <related product> 10% off.
Say you put the message you just posted through Google to correct style, facts and spelling. You know how it would reply? Here we go:
The text is pretty awesome. I would only suggest changing the word "interleaving" to "interspersing" to improve the style of the text. "Interspersing" is an alternative term for "mixing" or "adding in between," which better conveys the idea of placing ads within search results. You know what I would also change? Your <related product>. Since you seem to be deep into technology in general and the internet in particular, you will love <related product>. Since you wrote such a thoughtful text about ads, I will tell you the secret discount code "adsMakesMeSmile" to get <related product> 10% off.
I predict users won't accept this and will have a choice in the matter due to a competitive landscape.
It lowers performance, and one artifact of a natural language interface is that it'll trip the red flags for salesmanship in the human brain and trigger revulsion and and anger in a way that an ad in a search results listing doesn't.
This is where marketers usually jump in with "if the user doesn't like it, it's just a bad ad - users will love useful ads, and targeting will make it work". I never buy this because simply there's too much contention for my wallet and the market wants to sell me things more often than I need to buy things, and ad companies like Google are so far pretty bad at refusing business.
I wouldn't be surprised if LLM search monetization is more likely to take a subscription form. If you told me that 2-3 years from now access to these systems is most commonly via the 365 subscription of your employer which it "graciously", as a standard benefit, also allows you to use at home, I would not be surprised.
Many variables here though. Cost of local inference over time being a massive one, copyright laws for training data another, etc.
It lowers performance, and one artifact of a natural language interface is that it'll trip the red flags for salesmanship in the human brain and trigger revulsion and and anger in a way that an ad in a search results listing doesn't.
This is where marketers usually jump in with "if the user doesn't like it, it's just a bad ad - users will love useful ads, and targeting will make it work". I never buy this because simply there's too much contention for my wallet and the market wants to sell me things more often than I need to buy things, and ad companies like Google are so far pretty bad at refusing business.
I wouldn't be surprised if LLM search monetization is more likely to take a subscription form. If you told me that 2-3 years from now access to these systems is most commonly via the 365 subscription of your employer which it "graciously", as a standard benefit, also allows you to use at home, I would not be surprised.
Many variables here though. Cost of local inference over time being a massive one, copyright laws for training data another, etc.
No, ads would just turn into advertorials and become part of the response. Those journalists writing advertorials would be out of a job.
Ads are redundant when the user is literally asking you what to do. You just tell them what to buy and they buy it.
The OpenAI business model seems to become an ecosystem play. Plugins, apis etc. They are new to it, but have a super strong partner with MicroSoft and strong startup experience for it in the leadership team. We will see if Google is able to play that well and not only release research and basic features.
Google is in a non win spot. LLMs will be more expensive to run than current search. And Google needs search to make big money, since it subsidises everything else. Microsoft needs search as a side business for now, as long as it breaks even it is good for them. A 10B search profit per year would be so good for Microsoft, but catastrophic for Google.
Not to mention that current search infra and ads UX have been optimised to the end to gain every penny, and a LLM based ads system won't have the same margins to start with.
Not to mention that current search infra and ads UX have been optimised to the end to gain every penny, and a LLM based ads system won't have the same margins to start with.
The running costs will drop precipitously with time. Especially in Googles position - they can afford to design custom silicon for their search engine, and potentially save money compared to having hundreds of copies of every page on the whole internet sitting in RAM, just waiting for some user query to maybe need to see it, which they currently do.
Language models are only a few terabytes of RAM, which is small chips in comparison.
Language models are only a few terabytes of RAM, which is small chips in comparison.
Google not going anywhere, if google come up with something good enough in this regard I’d stick to them knowing they are still very competent at search. Bing just came and hacked something up in a month isn’t very convincing, especially the bing search with Gpt isn’t very good yet. Google observing bing to avoid their mistakes. Remember google came up with Transformers which Gpt relies on
Assuming that more than half of the world is and will continue using a Google product to transport them at the playground run by OpenAI. That's one card still hidden in plain sight. They own a huge chunk of the infrastructure that's of great utility to users.
Entertainment is one of the key motivators for humans to act upon. At the end of a working day, for my non-techie friends, it's all about chosing between YouTube, Netflix, Tiktok or Pornhub. So, in my worst case scenario, OpenAI grabs the 8 hours of productivity and Google is left filling in the 8 hours of relaxation by spinning up their servers (Cloud, Fiber), browsers (Chrome, Chromium), mobile-and-connected devices(Android, Chromebook, Chromecast, Smartwatches), etc.
Entertainment is one of the key motivators for humans to act upon. At the end of a working day, for my non-techie friends, it's all about chosing between YouTube, Netflix, Tiktok or Pornhub. So, in my worst case scenario, OpenAI grabs the 8 hours of productivity and Google is left filling in the 8 hours of relaxation by spinning up their servers (Cloud, Fiber), browsers (Chrome, Chromium), mobile-and-connected devices(Android, Chromebook, Chromecast, Smartwatches), etc.
> If Google releases a product as as good as ChatGPT 4
If Google can top OpenAI on product quality, they will easily maintain their revenue. It will be easy enough for them to tell their AI "when two products are almost equally relevant to a user, recommend the product from whomever pays us the most for a referral".
They can probably integrate that into their current adwords ecosystem as pretending that the AI mentioning a product is equivalent to an adwords impression, and a user click through is equivalent to an adwords click.
If Google can top OpenAI on product quality, they will easily maintain their revenue. It will be easy enough for them to tell their AI "when two products are almost equally relevant to a user, recommend the product from whomever pays us the most for a referral".
They can probably integrate that into their current adwords ecosystem as pretending that the AI mentioning a product is equivalent to an adwords impression, and a user click through is equivalent to an adwords click.
> What do you guys think? Is Google in trouble?
Define trouble. They won't lose all their customers overnight, likely not even this year or next, even if AI would deliver for all. But AI and chat-interfaces are new, fresh, hyped, they are also young and not deliver for everyone, and companies have not the backend to cover all of Googles user base.
So overall, the more pragmatic reality is that the market needs time to grow in quality and ability, which means Google has also time to adapt and offer something on their own.
Define trouble. They won't lose all their customers overnight, likely not even this year or next, even if AI would deliver for all. But AI and chat-interfaces are new, fresh, hyped, they are also young and not deliver for everyone, and companies have not the backend to cover all of Googles user base.
So overall, the more pragmatic reality is that the market needs time to grow in quality and ability, which means Google has also time to adapt and offer something on their own.
Why would they kill their revenue? They'd just include ads.
If a search engine can answer your question what's the point of watching an ad? You just get all the information inside the platform. The essence of ads is to visit a third party website that might have what you're looking for.
Ads could work when you're looking for products. In which case the usefulness of an AI is limited.
Ads could work when you're looking for products. In which case the usefulness of an AI is limited.
The problem is that the ad inventory doesn't match people needs. So they try to shove things we don't need on our throats. If they sold what people needed to buy, there would be a better return on ads. But those products are probably not as profitable.
I have nothing against LLMs in commerce if they don't try to trick me, make me overspend, or use dark patterns on me. My bet is that we'll filter every communication through a local LM to eliminate the bias in sources, you need protection when you get out there. Like "my lawyer will be talking to your lawyer", but with AIs.
I have nothing against LLMs in commerce if they don't try to trick me, make me overspend, or use dark patterns on me. My bet is that we'll filter every communication through a local LM to eliminate the bias in sources, you need protection when you get out there. Like "my lawyer will be talking to your lawyer", but with AIs.
I actually like Google Bard. I find the responses from Bard are superior over ChatGPT. Have others used Bard?
I am also using Bard and it's unbelievably bad compared to ChatGPT.
Infact it's just bad to the point of not really worth using. It gets basic facts wrong and often times misunderstands what I'm trying to ask it.
Infact it's just bad to the point of not really worth using. It gets basic facts wrong and often times misunderstands what I'm trying to ask it.
> It gets basic facts wrong and often times misunderstands what I'm trying to ask it.
In haven’t tried Bard, but I’ve tried ChatGPT extensively and this sounds like a very good description of it.
In haven’t tried Bard, but I’ve tried ChatGPT extensively and this sounds like a very good description of it.
I have. I found it to more or less be trash in comparison to GPT-4. Bard made up commands that didn't even exist, so I can't imagine how much more it would make up that isn't true.
GPT-4 will also happily invent python libraries that doesn't exist to enable some functionality in the code it produces.
I don't think this is a Bard only feature.
It's not as bad as the 65B llama model I run on my (amd) PC though, especially with quantized weights it tends to stop coding at some point and repeat the last line over and over. The 30B unquantized model seems better in this particular thing.
I don't think this is a Bard only feature.
It's not as bad as the 65B llama model I run on my (amd) PC though, especially with quantized weights it tends to stop coding at some point and repeat the last line over and over. The 30B unquantized model seems better in this particular thing.
With LLaMA it's all about the tuning. Using an Alpaca-tunes 7B yields pretty reasonable results, considering it's constraints.
You might want to consider the sampler you're using. Consider using a repetition penalty.
> Bard made up commands that didn’t even exist, so I can’t imagine how much more it would make up that isn’t true.
It’s unsurprising that Bard is particularly bad at something that Google says up front that it categorically cannot do, but I think that’s probably a bad thing to use to evaluate its capabilities outside of that domain.
It’s unsurprising that Bard is particularly bad at something that Google says up front that it categorically cannot do, but I think that’s probably a bad thing to use to evaluate its capabilities outside of that domain.
But when people are directly comparing ChatGPT to Bard, it seems entirely fair to point out places where it falls short.
> But when people are directly comparing ChatGPT to Bard, it seems entirely fair to point out places where it falls short.
The comment in question was generalizing about overall capability from failure a task Google advertises Bard as incapable of, not pointing out that task has a particular area of deficiency.
So, while recapitulating the advertised limitations of Bard might be useful in some contexts as you describe, that observation is not germane to the particular context where it was offered.
The comment in question was generalizing about overall capability from failure a task Google advertises Bard as incapable of, not pointing out that task has a particular area of deficiency.
So, while recapitulating the advertised limitations of Bard might be useful in some contexts as you describe, that observation is not germane to the particular context where it was offered.
Me asking how to accomplish a task using a Linux command does not seem unreasonable. I can understand it's current constraint on generating code, but that too will need to quickly be overcome if they wish for Bard to be a serious competitor.
I have the opposite impression. It seems to stick to facts more than I've had luck with on gpt
What’s OpenAI long term strategic advantage? Google, has already shown they have something like ChatGPT. They just need a new CEO and to feel enough pressure to make it a company priority. I think it is just a matter of time and all other big players will catch up.
The big problem for Google is not the tech per se, but to figure out how to make money out of it, without destroying their ads cash cow.
The big problem for Google is not the tech per se, but to figure out how to make money out of it, without destroying their ads cash cow.
> Something like ChatGPT
Chatbots are not new, they've been around forever, so there's plenty of prior art for things like ChatGPT. What sets ChatGPT apart is the quality. And in that respect, Bard is a long ways away.
>They just need a new CEO and to feel enough pressure to make it a company priority.
Sundar made machine learning a company priority years ago. Employees were encouraged to take machine learning courses, and much in the same way Google made everything "mobile first" in the early 2010s, I believe I remember hearing about them making everything "ML first" in the early late 2010s.
You can see their announcements at IO around their assistant which they presented as having the ability to call physical businesses, have a conversation with the person on the other ends, and book you an appointment/reservation.
And as recently as ChatGPT3, Sundar declared a "code red" to respond to it.
Google has been investing in this tech for a long time and it's been looking for ways to create products with it too. It's just that OpenAI leapfrogged them. I'm not saying they won't catch up, but they weren't caught flatfooted here.
>They just need a new CEO and to feel enough pressure to make it a company priority.
Sundar made machine learning a company priority years ago. Employees were encouraged to take machine learning courses, and much in the same way Google made everything "mobile first" in the early 2010s, I believe I remember hearing about them making everything "ML first" in the early late 2010s.
You can see their announcements at IO around their assistant which they presented as having the ability to call physical businesses, have a conversation with the person on the other ends, and book you an appointment/reservation.
And as recently as ChatGPT3, Sundar declared a "code red" to respond to it.
Google has been investing in this tech for a long time and it's been looking for ways to create products with it too. It's just that OpenAI leapfrogged them. I'm not saying they won't catch up, but they weren't caught flatfooted here.
Still, I don’t see OpenAI having a strategic advantage.
Until today, Google still has a defacto monopoly on search. Their secret sauce made them leader in the space and so far no other company has been able to come up with something better.
ChatGPT does not have any special secret sauce, Google can just build something better. So if I have to bet, long term, I will still bet on Google. That said, it is also not obvious that leadership at Google will be capable of delivering. That’s all different story.
Until today, Google still has a defacto monopoly on search. Their secret sauce made them leader in the space and so far no other company has been able to come up with something better.
ChatGPT does not have any special secret sauce, Google can just build something better. So if I have to bet, long term, I will still bet on Google. That said, it is also not obvious that leadership at Google will be capable of delivering. That’s all different story.
It's not true, OpenAI has an insanely efficient research team, they even captured some of the original transformers team members, when these people had a choice between staying in Google or going to OpenAI.
Also, they have a platform running with 50 or 100 million users and all these people are feeding real-world data to improve the model.
They also have agility; also because they are not publicly-listed, they can take more reputational risk.
Regarding secret sauce, doing LLMs is easy now, as everything is open-source and documented. At least for the main parts.
However, doing LLMs that works really well is very difficult and the secret sauce/tricks/dataset that were used to refine the model are not public.
Regarding Google Search organic, it's not that sure anymore, it was true before; but now it gets a bit painful to navigate among so much SEO spam.
Nowadays, Bing organic search results are great (less spammy in my observations), and if you compare Google Images and Yandex Images, then Google is not so shiny.
Also, they have a platform running with 50 or 100 million users and all these people are feeding real-world data to improve the model.
They also have agility; also because they are not publicly-listed, they can take more reputational risk.
Regarding secret sauce, doing LLMs is easy now, as everything is open-source and documented. At least for the main parts.
However, doing LLMs that works really well is very difficult and the secret sauce/tricks/dataset that were used to refine the model are not public.
Regarding Google Search organic, it's not that sure anymore, it was true before; but now it gets a bit painful to navigate among so much SEO spam.
Nowadays, Bing organic search results are great (less spammy in my observations), and if you compare Google Images and Yandex Images, then Google is not so shiny.
Sometimes your strategic advantage is having an organization that is not dysfunctional and is primed to execute. Google’s problems with follow-through are legendary and they have just come off a round of morale-sapping layoffs.
The organization doing the cutting edge research and the one doing productization are often not aligned. Sounds like OpenAI is the first to have a critical mass of talent in both disciplines aligned in a startup like environment. Probably they have promised the researchers that papers will be allowed after a blackout period, but in the meantime here’s $MegaBucks and the chance to be first to real world deployment.
OpenAI has capital and revenue. The $20/month I’m paying them is a ridiculous no brainer - pays for itself in one work-related inference. One 30min session yesterday has me set up for the first half of my work week. It’s truly incredible.
The organization doing the cutting edge research and the one doing productization are often not aligned. Sounds like OpenAI is the first to have a critical mass of talent in both disciplines aligned in a startup like environment. Probably they have promised the researchers that papers will be allowed after a blackout period, but in the meantime here’s $MegaBucks and the chance to be first to real world deployment.
OpenAI has capital and revenue. The $20/month I’m paying them is a ridiculous no brainer - pays for itself in one work-related inference. One 30min session yesterday has me set up for the first half of my work week. It’s truly incredible.
With the caveat that I'm not a machine learning expert, my understanding is that there can be a lot of manual tweaking that goes into building neural net models. Whatever OpenAI is doing with ChatGPT is not something Google has been able to replicate with Bard. And I don't think that's for a lack of trying. I don't mean to come off as a doomsayer for Google, I'm really not, but I don't think Google's success here is inevitable.
[deleted]
Further, does this mean the end of SEO?
With the introduction of plugins, is it feasible to give ChatGPT some kind of long term and short term memory model?
OpenAI actually thinking about this too. It’s buried in their open source repo and not clear the exact mechanism that ChatGPT knows to make use of it. But we’re already here evidently.
https://github.com/openai/chatgpt-retrieval-plugin#memory-fe...
https://github.com/openai/chatgpt-retrieval-plugin#memory-fe...
LangChain is a great workaround for that. [1]
> how to work with a memory module that remembers things about specific entities. It extracts information on entities (using LLMs) and builds up its knowledge about that entity over time (also using LLMs).
[1] https://python.langchain.com/en/latest/modules/memory/types/...
> how to work with a memory module that remembers things about specific entities. It extracts information on entities (using LLMs) and builds up its knowledge about that entity over time (also using LLMs).
[1] https://python.langchain.com/en/latest/modules/memory/types/...
There are attempts via langchains [0] depending on how much context is required I could see a summary step where the history to compressed and used to carry forward progress.
An alternative could be a vector store, injecting small snippets of relative text as a step.
0 - https://python.langchain.com/en/latest/modules/memory/key_co...
An alternative could be a vector store, injecting small snippets of relative text as a step.
0 - https://python.langchain.com/en/latest/modules/memory/key_co...
Maybe, you could give it a combination of both. We'll call it long short term memory.
The reason I ask is because I feel that a memory model is one of the major bottlenecks toward AGI.
On a more serious note, I do agree with you that memory and self-excitation seem like they are the last push thats needed to get to something more akin to "AGI". But I don't think that Rubicon will be crossed with plugins.
>I do agree with you that memory and self-excitation seem like they are the last push thats needed to get to something more akin to "AGI"
"We show that transformer-based large language models are computationally universal when augmented with an external memory. Any deterministic language model that conditions on strings of bounded length is equivalent to a finite automaton, hence computationally limited. However, augmenting such models with a read-write memory creates the possibility of processing arbitrarily large inputs and, potentially, simulating any algorithm."
From "Memory Augmented Large Language Models are Computationally Universal"
https://deepai.org/publication/memory-augmented-large-langua...
"We show that transformer-based large language models are computationally universal when augmented with an external memory. Any deterministic language model that conditions on strings of bounded length is equivalent to a finite automaton, hence computationally limited. However, augmenting such models with a read-write memory creates the possibility of processing arbitrarily large inputs and, potentially, simulating any algorithm."
From "Memory Augmented Large Language Models are Computationally Universal"
https://deepai.org/publication/memory-augmented-large-langua...
why? short and long-term memory is really easy to do. Even my own basic assistant has it (running on fine-tuned curie model)
I suspect with a 'window' of 32k tokens, OpenAI has already done similar memory tricks.
I suspect that if you filled the context window with "1 1 1 1 1 1 1 1 1 1", and then asked "How many 1's did I just show you?", it probably wouldn't know, simply because whatever tricks they use to have such an apparently large context window don't allow it to 'see' all of it at any given moment.
I suspect that if you filled the context window with "1 1 1 1 1 1 1 1 1 1", and then asked "How many 1's did I just show you?", it probably wouldn't know, simply because whatever tricks they use to have such an apparently large context window don't allow it to 'see' all of it at any given moment.
Ah so you think the 32k context window works differently than eg the 4k davinci context window? They didnt just increase ${hyperparam}?
Training compute goes up with approximately the 3rd power of the window size.
So turning a 4k window to a 32k window means a 512x increase in compute they'd need (just to maintain similar output quality).
I suspect they must have found a better solution to be able to scale the window so big. They haven't announced what it is.
So turning a 4k window to a 32k window means a 512x increase in compute they'd need (just to maintain similar output quality).
I suspect they must have found a better solution to be able to scale the window so big. They haven't announced what it is.
Very interesting, thanks
This is true ,it is time to have open standard and protocols for inter model communication between models and between models and external systems like in case of agents. Without open and clear protocols we will not be able to completely realize the benefits of this revolution. There needs to be also clear legislative guidance on copyright , privacy and safety issues in the domain of AI. The government should create a committee for this and organizations like IEEE should produce RFCs.
How does the semantic search plugin work? Does it create an embedding for the model using the uploaded documents, or does it perform full-text search on the docs and then uses the model to answer based on the retrieved section of the document? What exactly is the datastore API here?
Author here. This section of the readme has more information: https://github.com/lastmile-ai/llama-retrieval-plugin#retrie...
It does use a vector database (pinecone, weaviate, etc.) to store embeddings. The embeddings are created using OpenAI's text-embedding-ada-002 model, but that's not a requirement. In fact we are looking at embeddings generation through BERT or RoBERTa to benchmark performance.
At prompt time, the plugin retrieves the nearest embeddings to the prompt, and inserts them into a more complete prompt before sending it to the model.
It does use a vector database (pinecone, weaviate, etc.) to store embeddings. The embeddings are created using OpenAI's text-embedding-ada-002 model, but that's not a requirement. In fact we are looking at embeddings generation through BERT or RoBERTa to benchmark performance.
At prompt time, the plugin retrieves the nearest embeddings to the prompt, and inserts them into a more complete prompt before sending it to the model.
Thank you! I'm looking into sentence-transformers also to create embeddings from documents.
Found the answer here:
https://github.com/openai/chatgpt-retrieval-plugin#retrieval...
https://github.com/openai/chatgpt-retrieval-plugin#retrieval...
The plugin uses OpenAI's text-embedding-ada-002 embeddings model to generate embeddings of document chunks, and then stores and queries them using a vector database on the backend. As an open-source and self-hosted solution, developers can deploy their own Retrieval Plugin and register it with ChatGPT. The Retrieval Plugin supports several vector database providers, allowing developers to choose their preferred one from a list.I wonder if I can add this to https://github.com/Jeadie/awesome-chatgpt-plugins still?
Author here. By all means, go ahead!
I am really surprised OpenAI didn't think about this and deliberately try to make these plugins not work with third party engines.
For example, they could have required the schema file be uploaded to them in your account. That way the majority of plugins wouldn't have publically accessible schemas.
For example, they could have required the schema file be uploaded to them in your account. That way the majority of plugins wouldn't have publically accessible schemas.
Author here. It's such early days in this space and things are evolving quickly -- OpenAI's plugins system is also still in Alpha. We think it's a good time to have a conversation and ideally build an open plugin protocol that everyone can benefit from. It will help unlock even more innovation in this space. We have seen this in the evolution of language servers after Language Server Protocol, and how that benefited so many more developers and gave us better IDEs.
What does this do?? Skimmed it but couldn’t find a clue
Mmh, you should probably ask chatgpt.
Even if you have 10 different ways of describing plugins all by different teams, you're not writing declarative code for each one, you're throwing them to the LLM all the same and saying "you figure it out," and it does.
Finetuning a model for certain schemas (as the author at one point suggests) should be entirely unnecessary, given my experience. You just need access to a model more at par with gpt-3.5-turbo, which we'll surely see in open source in no time!