The authors claim that LLM are reducing public knowledge sharing and that the effect is not merely displacing duplicate, low-quality, or beginner-level content.
However their claim is weak and the effect is not quite as sensational as they make it sound.
First, they only present Figure 3 and not regression results for their suggested tests of LLMs being substitutes of bad quality posts. In contrast, they report tests for their random qualification by user experience (where someone is experienced if they posted 10 times). Now, why would they omit tests by post quality but show results by a random bucketing of user “experience”?
Second, their own Figure 3 “shows” a change in trends for good and neutral questions. Good questions were downtrending and now they are flat, and neutral questions (arguably the noise) went from an uptrend to flat. Bad question continue to go down, no visible change in the trend. This suggests the opposite, ie that LLMs are in fact substituting bad quality content.
I feel the conclusion needed a stronger statement and research doesn’t reward meticulous but unsurprising results. Hence the sensational title and the somewhat redacted results.
If the inference is drawn from the number of claims the first question that comes to mind is, how easy is it to file a claim with Waymo?
Also, do people behave differently when involved in an accident with a driverless car?
For example, at McDonalds, the automatic checkouts don’t simply substitute expensive workforce but actually boost sales of addons because people are less inhibited by a screen than by a human.
So, do people just drive off if it was a minor bump?
Finally, does Waymo initiate claims? If it were me, I wouldn’t. Id simply partner with a mechanic and fix issues internally as part of a fleet maintenance system.
If that’s the case, I’d roughly half the stats before doing any additional inference.
I agree there is another side of the medal, if that’s what you mean. In fact, if we accept that innovation is also the recombination of existing knowledge, then AI will help up discover entire fractals of new stuff.
Good candidates for what I am looking are OpenAI Playground (once it supports versioning and variables for chat), PromptLayer (once it supports chat prompts), EveryPrompt (chat prompts missing). More dev oriented, LangChain https://js.langchain.com/docs/getting-started/guide-chat which can then be easily versioned.
Thanks for pointing towards the right direction. I'll edit the original question.
To rephrase, I am looking for a tool to do model lifecycle management https://github.com/kelvins/awesome-mlops#model-lifecycle and wonder if there is any one in particular that you'd think is better suited for prompts, i.e. an array of objects with templated text
However their claim is weak and the effect is not quite as sensational as they make it sound.
First, they only present Figure 3 and not regression results for their suggested tests of LLMs being substitutes of bad quality posts. In contrast, they report tests for their random qualification by user experience (where someone is experienced if they posted 10 times). Now, why would they omit tests by post quality but show results by a random bucketing of user “experience”?
Second, their own Figure 3 “shows” a change in trends for good and neutral questions. Good questions were downtrending and now they are flat, and neutral questions (arguably the noise) went from an uptrend to flat. Bad question continue to go down, no visible change in the trend. This suggests the opposite, ie that LLMs are in fact substituting bad quality content.
I feel the conclusion needed a stronger statement and research doesn’t reward meticulous but unsurprising results. Hence the sensational title and the somewhat redacted results.