Because if there are obvious edge and corner cases, like overflow scenarios, a professional system will either ensure that expectations are lived up to, or flatly denied as errors.
That’s just you projecting your impression of agency onto a puppet, based on prior observation of actual animals.
But make no mistake. It is a puppet. It’s a multicore processing circuit with stack pointers, instruction pointers and little else going for it.
It’s your laptop strapped to some motors. It’s not sentient, and has no agency. It’s a guided missile at best. A step above cruise control.
It lacks authority to define where it goes or form a need for continuing to stand. Thus it lacks true agency.
We can ascribe happy/sad to stand/fall, as crude, fundamental binary “emotions” but robots like BigDog are less complicated than amoeboid life found in pond scum.
Consider whether traffic lights are happy or sad, based on whether traffic obeys their signalling. Now consider traffic cameras. Now consider whether an automated ticket for running a red light on camera is an expression of emotion.
Part of your statement is true, and part of it is false.
An agent would need to model the behavior of peers, yes.
But communicate? No. Solve goals together? No.
To coalesce civilization or society? Maybe, maybe not. Socialization among peers is not a prerequisite for agency. Not by a mile.
And certainly not amid a state of nature. Not at all would communication or collaboration become a necessity.
Emotion might become an aspect of investment in hypothetical experiments performed by an agent. Hope that equilibrium might be achieved with less work through communication and collaboration.
But put it this way. A caveman grunts at a wild boar standing on top of a hill. The caveman wishes to discern if the silohette atop the hill is potential food by provoking movement, or an inert object offering the illusory shape of a backlit animal. The boar notices and experiences fear. The boar freezes, hoping the grunt was not directed toward it intentionally.
Is neither an agent? Does the conflict of interests preclude emotion?
The boar models the adversary, and experiences emotion to preserve the equilibrium of staying alive.
The caveman experiences hunger as a loss of equilibrium, which provokes a mixture of anxiety, and unhappiness which may cascade into a malaise or depression as weakness progresses with starvation. The aggression of the hunt is not anger, although anger may arrive incidentally.
Is the grunt communication? Perhaps as much as any tactic might be. Deceptive comminication (bird calls, immitating a female in heat to draw male prey) might still be communication, after all.
But to model nature, there must have been a period of where some agents seemingly existed without peers. But those agents likely experienced emotion before cognizant sentience and a rich awareness of the potential for sentience within peers, which most likely precedes a capacity to communicate.
But all that’s needed are are a handful of rules, to provide for a system that emotes. You’d probably dismiss it as an inauthentic toy, but emotions actually aren’t the core aspect of agency.
Anyway, the rules just need to assemble a goal, a threshold for equilibium, and reactions for deviation from that equilibrium.
Bonus points if you account for radiant measurements of equilibrium. What I mean by that is anticipation of adjacent conditions that signal a probable loss of equilibrium, such that the system doesn’t just react to an unbalanced circumstance, but also things that could lead to an undesired imbalance.
Examples:
A. If the cup is disturbed so that the milk spills, then a negative experience ensues.
B. If a balloon, inflated with ordinary compressed air, sinks onto the grass and pops, a negative experience ensues.
C. Ambulate through an environment obstructed by complex obstacles, and negotiate each obstacle without falling onto the ground. Falling onto the ground will result in a negative experience.
Each of these three goals represents a targeted state of equilibrium: don’t spill the milk, keep the balloon safe, don’t fall down go boom.
Now, layer an array of reactions on top of the branched set of possible outcomes. You can also buld up variations on top of each branch.
Positive branches are indicated in moments of success at achieving the goal. Negative branches are indicated upon equilibrium being defeated.
So the computer or robot can externalize its inner state with a happy face or a sad face, but we’re missing some of the emotional range. When would anger display? When the machine can assign blame and consider revenge, of course.
So if an entity (preferably a rival robot, since we wouldn’t want the robot to exact revenge on a person) knocks over the milk, pops the balloon, tackles the robot, the obvious motive is to make sure that never happens again, the root cause is the rival entity. Stand back up, destroy the entity, and acquire more milk, another balloon, and try to achieve equilibrium, and thus happiness again.
Prior to reacquiring its happy state, the machine can externalize an angry face if it can assign blame to a detected responsible entity, in all other cases, it would simply be sad, until it can stand back up, inflate another balloon and pour itself another glass of milk to protect. If it cannot set things back in order, as desired, then it is simply permanently sad (no balloon, no milk, unable to stand or walk), forever.
See how that works? It’s actually not much more complicated than that.
No surprises.