Affective Intelligence for Humanoid Service Robotics

Project Aim:

The Affective Intelligence for Humanoid Service Robotics Project aims to:

  1. research automatic recognition of taste-liking from facial videos by comparatively training and evaluating models with engineered features and state-of-the-art deep learning architectures;
  2. analyse the classification results along the aspects of facilitator type, and the gender, ethnicity, and personality of the participants;
  3. utilize a new beverage tasting dataset acquired under different conditions (human vs. robot facilitator and priming vs. non-priming facilitation).

Studies Conducted:

A summary of studies with human participants (as of June 2023):

  1. Automatic Analysis of Facilitated Taste-liking [ICMI 2020]
  2. Investigating Taste-liking with a Humanoid Robot Facilitator [RO-MAN 2020]
  3. Learning Socially Appropriate Robo-waiter Behaviours through Real-time User Feedback [HRI 2022]

Major Findings:

Major findings (as of June 2023):

  • - The experimental results show that: (i) The deep spatiotemporal architectures provide better classification results than the engineered feature models; (ii) the classification results for all three classes of liking, neutral and disliking reach F1 scores in the range of 71% - 91%; (iii) the personality-aware network that fuses participants' personality information with that of facial reaction features provides improved classification performance; and (iv) classification results vary across participant gender, but not across facilitator type and participant ethnicity
  • - We found that the facilitator type and facilitation style had no significant influence on cognitive taste-liking. However, in robot facilitator scenarios, people were more willing to follow the instruction and felt more comfortable when facilitated with priming
  • - Our results show that both explicit and implicit adaptation mechanisms enabled the adaptive robo-waiter to be rated as more enjoyable and sociable, and its positioning relative to the participants as more appropriate compared to using the pre-trained model or a randomised control implementation

Project Team:

  • - Prof Hatice Gunes (PI, Apr 2019-present)
  • - Nikhil Churamani (Research Assistant, Oct 2022 – Jan 2023) – now a postdoc at AFAR Lab on another project
  • - Yifan Chen (University of Cambridge)
  • - Zhuoni Jie (MPhil Student, University of Cambridge, Oct 2018 - Jun 2019)
  • - Emily McQuillin (MPhil Student, University of Cambridge, Oct 2020 - Jun 2021)
--> Designed by BootstrapMade