As prediction markets already show, forecasts can influence the outcomes they are trying to predict.
What happens when these models become extremely accurate and widely trusted? A forecast like “Will there be a war between countries A and B?” may itself affect whether the war happens.
If the model says there is a 1% chance of war, little changes. But if it says 90%, governments, markets, militaries, and the public may react: capital flees, troops mobilize, diplomatic trust collapses, and each side starts preparing for the other side’s preparation. The prediction helps make itself true.
The same feedback loop could apply to bank runs, market crashes, civil unrest, elections, and corporate failures.
At some point, the most accurate forecaster may become less like an observer and more like an actor with enormous power over the system it predicts.
Is it possible to tell slop from non slop if you were not there when the tokens get emitted? Somebody can just lie and pretend that they were not generated
I must admit I don't really understand what the point of the post-install script concern is.
Usually, you run the actual packaged dependency code at some point anyway, and usually with the same permissions as the install process.
So all of these setup scripts (good or bad) can just move their entrypoint from npm to wherever the `import` or `require` happens.
It seems to me that this is a small stumbling block at best, unless the whole ecosystem moves to a deno-like sandboxed environment. Maybe that is the plan?
I don't know, maybe something about backwards compatibility, maybe nobody can agree on how to do it correctly. It hasn't happened for decades, so I'm not going to hold my breath.
Unfortunately, real apps and native tech stacks can not only write data to your SSD, they can usually write data to the user directory however they want and they can read it as well!
1. in order to run LLMs, especially the best ones, you need complicated devices which are expensive
2. if you buy one for your personal use, you are probably not going to utilize it all the time and it will be idle a lot
It seems to me that it will always be more economical that the LLM-running devices are in a datacenter where it is easier to make sure they are always utilized
The problem is that often the program runs into some edge case that requires interpretation, at which point one is tempted to let the LLM deal with the edge case, at which point one is tempted to let the LLM deal with the whole loop and let it do the tool calls
As prediction markets already show, forecasts can influence the outcomes they are trying to predict.
What happens when these models become extremely accurate and widely trusted? A forecast like “Will there be a war between countries A and B?” may itself affect whether the war happens.
If the model says there is a 1% chance of war, little changes. But if it says 90%, governments, markets, militaries, and the public may react: capital flees, troops mobilize, diplomatic trust collapses, and each side starts preparing for the other side’s preparation. The prediction helps make itself true.
The same feedback loop could apply to bank runs, market crashes, civil unrest, elections, and corporate failures.
At some point, the most accurate forecaster may become less like an observer and more like an actor with enormous power over the system it predicts.