A couple weeks back a Google Research post mentioned some fascinating research into “emotional agents”, i.e. intelligent software agents that can autonomously rewrite their own behaviours, the rewriting being triggered by charged emotional events.
They reported that, for agents created by Manish Mehta using the ABL agent language originally developed by Michael Mateas and Andrew Stern for the Facade project (emphasis added):
“… using two agents playing tag, Mehta was able to make them revise their behaviors based on how well they played the game. However, it had an unintended consequence: one of the agents decided it was less stressed out when it was "IT". When it was "IT", it wasn't being chased and wasn't feeling stressed, so it just let itself get tagged, enabling it to "chase" the other player at a leisurely pace.
… Mehta realized this had exposed one more property of human behavioral change: emotional events can prompt change, but we also have a self image - a model of what we think our personality should be like. As our personalities change, we audit the changes to make sure they fit our self image and apply further corrections. For a computer game character, that "personality" is really the designer's intent, so Mehta augmented his emotional learning system to check potential behavioral changes and make sure that they did not violate the original intended design. With this change, both characters were able to learn from their good and bad experiences playing tag, and neither "quit playing the game" just because they didn't want the stress of being chased.”
I love that effectively they had to program in a “personality” (conscientiousness?) in order to get the intelligent agent to keep playing tag.
And that, without that personality check, the agent decided that the intelligent thing to do in order to minimise stress is to let yourself get caught so that you can then take your time ambling along chasing the other player as slowly as you like!
See, this proves what my friends say: I’m not being lazy. I’m just being efficient.