Project Aim:

The Affective Intelligence for Humanoid Service Robotics Project aims to:

1. research automatic recognition of taste-liking from facial videos by comparatively training and evaluating models with engineered features and state-of-the-art deep learning architectures;
2. analyse the classification results along the aspects of facilitator type, and the gender, ethnicity, and personality of the participants;
3. utilize a new beverage tasting dataset acquired under different conditions (human vs. robot facilitator and priming vs. non-priming facilitation).

Studies Conducted:

A summary of studies with human participants (as of June 2023):

1. Automatic Analysis of Facilitated Taste-liking [ICMI 2020]
2. Investigating Taste-liking with a Humanoid Robot Facilitator [RO-MAN 2020]
3. Learning Socially Appropriate Robo-waiter Behaviours through Real-time User Feedback [HRI 2022]

Major Findings:

Major findings (as of June 2023):

• - The experimental results show that: (i) The deep spatiotemporal architectures provide better classification results than the engineered feature models; (ii) the classification results for all three classes of liking, neutral and disliking reach F1 scores in the range of 71% - 91%; (iii) the personality-aware network that fuses participants' personality information with that of facial reaction features provides improved classification performance; and (iv) classification results vary across participant gender, but not across facilitator type and participant ethnicity
• - We found that the facilitator type and facilitation style had no significant influence on cognitive taste-liking. However, in robot facilitator scenarios, people were more willing to follow the instruction and felt more comfortable when facilitated with priming
• - Our results show that both explicit and implicit adaptation mechanisms enabled the adaptive robo-waiter to be rated as more enjoyable and sociable, and its positioning relative to the participants as more appropriate compared to using the pre-trained model or a randomised control implementation

Project Team:

• - Prof Hatice Gunes (PI, Apr 2019-present)
• - Nikhil Churamani (Research Assistant, Oct 2022 – Jan 2023) – now a postdoc at AFAR Lab on another project
• - Yifan Chen (University of Cambridge)
• - Zhuoni Jie (MPhil Student, University of Cambridge, Oct 2018 - Jun 2019)
• - Emily McQuillin (MPhil Student, University of Cambridge, Oct 2020 - Jun 2021)