I was fortunate to engage in numerous meetings, brainstorming sessions, and discussions with a variety of colleagues at Aalto University. I wish to express my gratitude to Jyri Kivinen, Vikas Verma, Mert Celikok, Tomi Peltola, and all the members of the Probabilistic Machine Learning (PML) group for their insightful and valuable contributions to these discussions. Furthermore, I had the pleasure of participating in many enriching conversations with Professor Antti Oulasvirta and the members of the Computational Behavior Lab at Aalto, led by Professor Oulasvirta.I am also grateful to Isak Westerlund, who has worked alongside me during my PhD project, taking on a significant amount of responsibility in coding the applications and training the various AI models. My thanks also go to Ariel Kwiatkowski, who contributed to our second paper.I am deeply grateful to the pre-examiners, Professor Anna Rafferty and Dr. Xiaoyang Wang, for their essential insights and constructive feedback on my thesis. In addition, I am thankful to Dr. John Williamson for willingly taking on the role of the opponent I want to thank Bitville Oy CEO Jouko Ranta, as well as all my colleagues at Bitville Oy, for making it possible for me to concentrate practically fulltime on the research projects during these years.I had the unique opportunity to share my research journey with my son, Oskar, who has now embarked on his own PhD journey at Aalto University as I write this preface. Thank you for the hundreds of hours of brainstorming, discussion, and reflection over nearly the last 10 years, and for your invaluable contribution as a member of the research team. Our discussions have been of the utmost importance to me, whether they were about mathematical details or the extensive influence of AI on society.Last but not least, I want to extend my heartfelt thanks to my daughter, Erika, and my wife, Sofia, for their encouragement and patience. I am aware that discussions about AI have prominently featured in our family conversations over the last few years. It is evident that this project could not have reached its conclusion without your unwavering support.

No abstract

No abstract

The cooperation among AI systems, and between AI systems and humans is becoming increasingly important. In various realworld tasks, an agent needs to cooperate with unknown partner agent types. This requires the agent to assess the behaviour of the partner agent during a cooperative task and to adjust its own policy to support the cooperation. Deep reinforcement learning models can be trained to deliver the required functionality but are known to suffer from sample inefficiency and slow learning. However, adapting to a partner agent behaviour during the ongoing task requires ability to assess the partner agent type quickly. We suggest a method, where we synthetically produce populations of agents with different behavioural patterns together with ground truth data of their behaviour, and use this data for training a meta-learner. We additionally suggest an agent architecture, which can efficiently use the generated data and gain the meta-learning capability. When an agent is equipped with such a meta-learner, it is capable of quickly adapting to cooperation with unknown partner agent types in new situations. This method can be used to automatically form a task distribution for meta-training from emerging behaviours that arise, for example, through self-play.

The ability of an AI agent to assist other agents, such as humans, is an important and challenging goal, which requires the assisting agent to reason about the behavior and infer the goals of the assisted agent. Training such an ability by using reinforcement learning usually requires large amounts of online training, which is difficult and costly. On the other hand, offline data about the behavior of the assisted agent might be available, but is non-trivial to take advantage of by methods such as offline reinforcement learning. We introduce methods where the capability to create a representation of the behavior is first pre-trained with offline data, after which only a small amount of interaction data is needed to learn an assisting policy. We test the setting in a gridworld where the helper agent has the capability to manipulate the environment of the assisted artificial agents, and introduce three different scenarios where the assistance considerably improves the performance of the assisted agents.