Autonomous Adaptive Agent with Intrinsic Motivation for Sustainable HAI*

Takayuki Nozawa, Toshiyuki Kondo

.

DOI: 10.4236/jilsa.2010.24020                

Abstract

For most applications of human-agent interaction (HAI) research, maintaining the user’s interest and continuation of interaction are the issues of primary importance. To achieve sustainable HAI, we proposed a new model of intrinsically motivated adaptive agent, which learns about the human partner and behaves to satisfy its intrinsic motivation. Simulation of interaction with several types of other agents demonstrated how the model seeks new relationships with the partner and avoids situations which are not learnable. To investigate effectiveness of the model, we conducted a comparative HAI experiment with a simple interaction setting. The results showed that the model was effective in inducing subjective impressions of higher enjoyability, charm, and sustainability. Information theoretic analysis of the interaction suggested that a balanced information transfer between the agent and human partner would be important. The participants’ brain activity measured by functional near-infrared spectroscopy (fNIRS) indicated higher variability of activity at the dorsolateral prefrontal cortex during the interaction with the proposed agent. These results suggest that the intrinsically motivated adaptive agent successfully maintained the participants’ interest, by affecting their attention level.

Keywords

Human-Agent Interaction (HAI), Intrinsic Motivation, Reinforcement Learning, Functional Near-Infrared Spectroscopy (Fnirs)

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	T. Fong, I. Nourbakhsh and K. Dautenhahn, “A Survey of Socially Interactive Robots,” Robotics and Autonomous Systems, Vol. 42, No. 1, 2003, pp. 143-166.
[2]	T. Nakata, T.-S. Ko, T. Mori and T. Sato, “Informational Analysis on Impression of Human Robot Interaction,” Journal of Robotics Society of Japan, Vol. 19, 2001, pp. 667-675.
[3]	T. Nakata, “Information Transmission and Impression Formulation in Human-Robot Interaction,” 2004. http:// staff.aist.go.jp/toru-nakata/hri/HRI-Info.html
[4]	T. Kondo, D. Hirakawa and T. Nozawa, “Sustainability and Predictability in a Lasting Human-Agent Interaction,” Proceedings of 8th International Conference on Intelli- gent Virtual Agents; Springer, Lecture Notes in Computer Science, Japan, 2008, pp. 505-506.
[5]	E. L. Deci and R. M. Ryan, “Intrinsic Motivation and Self-Determination in Human Behavior,” Plenum, New York, 1985.
[6]	A. Barto, S. Singh and N. Chentanez, “Intrinsically Motivated Learning of Hierarchical Collections of Skills,” Proceedings of 3rd International Conference on Developmental Learning, UCSD Institute for Neural Computation, La Jolla, CA, 2004, pp. 112-119.
[7]	P. Oudeyer, F. Kaplan and V. V. Hafner, “Intrinsic Motivation Systems for Autonomous Mental Develop- ment,” IEEE Transactions on Evolutionary Computation, Vol. 11, No. 2, 2007, pp. 265-286.
[8]	R. S. Sutton and A. G. Barto, “Reinforcement Learning: An Introduction,” MIT Press, Cambridge, MA, 1998.
[9]	T. Schreiber, “Measuring Information Transfer,” Physical. Review Letters, Vol. 85, No. 2, 2000, pp. 461-464.
[10]	N. Bertschinger, E. Olbrich, N. Ay and J. Jost, “Autonomy: An Information Theoretic Perspective,” BioSystems, Vol. 91, No. 2, 2008, pp. 331-345.
[11]	T. Nozawa and T. Kondo, “A Comparison of Artifact Reduction Methods for Real-Time Analysis of Fnirs Data,” Proceedings of 13th International Conference on Human-Computer Interaction, Lecture Notes in Computer Science, Springer, Heidelberg, Vol. 5618, 2009, pp. 413-422.
[12]	E. K. Miller and J. D. Cohen, “An Integrative Theory of Prefrontal Cortex Function,” Annual Review of Neuroscience, Vol. 24, No. 1, 2001, pp. 167-202.
[13]	Y. Hoshi, “Functional Near-Infrared Spectroscopy: Current Status and Future Prospects,” Journal of Biomedical Optics, Vol. 12, No. 6, 2007, pp. 062106-1-9.