Create a machine learning agent to facilitate interactions between participants in virtual and augmented reality environments, such as the Metaverse, to help them achieve their goals and resolve conflicting situations. This is the main objective of GuestXR, the European project under the coordination of Professor Mel Slater, from the Faculty of Phycology and the Institute of Neurosciences of the University of Barcelona (UBNeuro).
As part of the GuestXR (A Machine Learning Agent for Social Harmony in eXtended Reality) project, coordinated by the Eurecat technology center, a machine learning system called guest will be designed, and it will work as an agent capable of examining the individual and group behavior of participants drawing on existing models from the perspective of neuroscience and social psychology. Additionally, Deep Reinforcement Learning will train the guest to learn over time – both in simulations and real meetings – and be more effective in social interactions.
In these lines, the agent “will participate in the conversation in different ways, performing a range of multi-sensory actions through music or space changes, among others”, notes the director of Eurecat Audiovisual Technologies, Adan Garriga.
“Every meeting in virtual and augmented reality has a purpose, whether it is explicit (for example, reaching an agreement on a subject) or implicit (such as fun). The basic idea of the guest is to help the group of people achieve this goal. This involves exciting research with a strong interdisciplinary element, which opens up new avenues and hopefully contributes to the success of virtual and augmented reality,” notes Professor Mel Slater.
In order to modulate this social interaction, the guest will perform a series of multi-sensory actions, “through, for example, visual and auditory characteristics to create specific states of mind and stimulate a relaxed environment when identifying a conflict” , underlines the researcher Umut Dire, of Eurecat Audiovisual Technologies.
This project “is designed to help provide solutions to existing challenges in online social interaction environments such as online harassment, for which there is no specific European legislation”, explains the Eurecat GuestXR project coordinator , Aurora Sesé.
An immersive online social space thanks to extended reality
The intervention of the machine learning system will be tested initially in conflict resolution situations, interactions with hearing impaired participants and contexts that lead to polarized debates such as climate change.
Two open calls to include other innovative use cases to test the effectiveness of the system are expected. One will address society in general — including businesses and associations, among others — while the other will address the arts.
The GuestXR technology will be produced within the framework of the Ethics by design approach, which consists of considering the potential ethical issues arising from the use of artificial intelligence (respect for human autonomy, prevention of damage, fairness, etc. ).
The GuestXR consortium, coordinated by UB and the Eurecat technology center, is formed by eight organizations from six countries with multidisciplinary teams from the field of extended reality, machine learning, artificial intelligence, social psychology , neuroscience of emotions, multisensory integration, research ethics and technology transfer. Collaborators include Maastricht University (Netherlands), University of Warsaw (Poland), Reichman University (Israel), National Institute for Research in Digital Science and Technology (INRIA, France) and Virtual Bodyworks and G.tec medical engineering GMBH.
Warning: AAAS and EurekAlert! are not responsible for the accuracy of press releases posted on EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.