According to the theory, an attention model. Here's artificial consciousness in three steps:
1. Have a robot build perception models of its environment and itself
2. Have the robot allocate computational resources and sensory bandwidth to the models using attention
3. Have the robot control attention using model predictive control
Because the attention model is less detailed than its actual attention, by virtue of being a model, it doesn't represent the mechanisms of attention or modeling accurately. Instead, it uses non-physical concepts such as "mental possession" to model itself or other agents paying attention to things, or "qualia" to denote the recursion that occurs when percepts we attend to are summarized by the attention model (which in turn can be attended to, and so on).
It's not a comforting story, but I think it can provide meaning. It does for me at least. While we are not gods, we are different from all other animals - we are where "the fallen angel meets the rising ape" as Terry Pratchett wrote.
While we might not have free will in an absolute, metaphysical sense, we can self-reflect, practice self-control, shape our environment, and even change our nature. What will we do with this power?
There is no eternal afterlife to go to, but we now understand what actual life is - and it's no longer so nasty and brutish. Perhaps soon it won't be so short either. We are certainly capable of extending it in principle, we just need to get our shit together.
Comforting stories are cozy, but we are growing up. We've become smart enough to cause a whole lot of trouble for ourselves, and are not yet wise enough to fix it. We're confused, can't make sense of things and constantly whine about it. But that's how growth works. Humanity might just be in its awkward emo teenage phase.
Yes, it's correct to say that I have knowledge of your conjecture, and in the same way that physicists have knowledge of QM and GR regardless of their truth status, but beyond just having knowledge of the theory, they also have knowledge of the reality that the theory describes.
>Were NASA physicists indefensible to use Newtonian mechanics to send a person to the moon because Newtonian mechanics are "wrong"?
No, it was defensible, and that's exactly my point. Even though they didn't believe in the content of the theory (and ignoring the fact that they know a better theory), they do have knowledge of reality through it.
I don't think instrumentalism makes sense for reasons unrelated to this discussion. A scientist can hold instrumentalist views without being a worse scientist for it, it's a philosophical position. Basically, I think it's bad metaphysics. If you refuse to believe that the objects described by a well-established theory really exist, but you don't have any concrete experiment that falsifies it or a better theory, then to me it seems like sheer refusal to accept reality. I think people find instrumentalism appealing because they expect that any theory could be replaced by a new one that could turn out very different, and then they see it as foolish to have believed the old one, so they straight up refuse to believe or care what any theory says about reality. But you always believe something, whether you are aware of it or not, and the question is whether your beliefs are supported by evidence and logic.
My point is that QM and GR make very different claims about what exists. Perhaps it's possible to unify the descriptions. But more likely there will be a new theory with a completely different description of reality.
On small scales, GR and Newtonian mechanics make almost the same predictions, but make completely different claims about what exists in reality. In my view, if the theories made equally good predictions, but still differed so fundamentally about what exists, then that matters, and implies that at least one of the theories is wrong. This is more a realist, than an instrumentalist position, which perhaps is what you subscribe to, but tbh instrumentalism always seemed indefensible to me.
You've called J and T into question, so let's do B as well. Physicists know that QM and relativity can't be true, so it's fair to say that they don't believe in these theories, in a naive sense at least. In general anyone who takes Box' maxim that all models are wrong (but some are useful) to heart, doesn't fully believe in any straightforward sense. But clearly we'd say physicists do have knowledge.
Didn't know about these, thanks for the pointer! Do you have a good resource for learning about these (specifically about the hardness of sampling from posterior distributions)?
I think a much closer analogy to function inversion is MCMC (or Bayesian inference in general), where we can easily compute the density p(x) of any point x, but finding the x given a p(x) is intractable. Strictly speaking it's about finding a set of x's that are distributed as p(X), not finding the x given any single density p(x), but it's close.
Relatedly, probabilistic programming was originally imagined pretty much like your second quote: you define a model, get some data, run them both through the built-in inference engine, and you get the parameters of the model likely to have produced the data. In practice though, there's no universal inference engine that works for everything (some people disagree, but they're NUTS ;)
I guess pretty much for the same reason P is probably not equal to NP.
Even simple inference problems are NP-hard (k means for example). I think what matters is that we have decent average case performance (and sample complexity). Most people can find a pretty good solution to travelings salesman problems in 2D. Not sure if that should be chalked up to myriad shortcuts or domain specialization.. Maybe there's no difference. What do you have in mind re Latin?
The argument in the paper (that AGI through ML is intractable because the perfect-vs-chance problem is intractable) sounds similar to the uncomputability of Solomonoff induction (and AIXI, and the no free lunch theorem). Nobody thinks AGI is equivalent to Solomonoff induction. This paper is silly.
Fair enough, but for the sake of this conversation, if we say 'good' values are those that keep things from staying the same, aren't the values of smart people just as likely to evolve towards 'bad' ones? For example, I'm sure most people know at least one smart person who only plays video games; it does seem that we'll keep inventing forms of entertainment that wirehead people more and more effectively, which seems in line with the Brave New World scenario.
>it's what capable and smart people value and pursue that makes all the difference.
How do you know capable and smart people will keep having good values? Seems to me that it's true until it isn't - populism takes over politics, ideology takes over the humanities, science gets Goodharted to death, etc. Values are highly circular - we value what high-status people in our (sub)culture value, and you become high-status by getting what people value. This holds for smart people as well.
Do you still see such dynamics in the coefficients if you have an order of magnitude more data or fewer dimensions? 100 points in 6D is not much even for linear regression, the model might just be too high variance to interpolate monotonically between the two populations.
"Rives sees ESM3’s generation of new proteins by iterating through various sequences as analogous to evolution."
Except for the part where a sequence is actually deemed more fit, ie natural selection? And the part where mutations are random, instead of sampled from the training data manifold, so much more constrained?
...so really it's a worse version of random search?
I imagine that the problem with getting LLMs to do it is 1)hallucination and 2)a lack of training examples of teaching through text. We rarely teach purely through text messages, I think this might explain why LLMs almost never ask clarifying questions or use the Socratic method. But it might be possible to RLHF it into doing that.
Can you say something about what your approach is?
1. Have a robot build perception models of its environment and itself
2. Have the robot allocate computational resources and sensory bandwidth to the models using attention
3. Have the robot control attention using model predictive control
Because the attention model is less detailed than its actual attention, by virtue of being a model, it doesn't represent the mechanisms of attention or modeling accurately. Instead, it uses non-physical concepts such as "mental possession" to model itself or other agents paying attention to things, or "qualia" to denote the recursion that occurs when percepts we attend to are summarized by the attention model (which in turn can be attended to, and so on).