Project Aim:

The Continual Learning for Affective Computing and Robotics Project aims to:

1. analyse, understand, and appropriately respond to human nonverbal behaviour and affective states;
2. address the problem of conventional machine learning approaches not scaling well to the dynamic nature of real-world interactions by learning incrementally through continual learning;
3. create fully adaptive and affective robots through continual learning.

Studies Conducted:

A summary of studies with human participants (as of June 2023):

1. Towards Causal Replay for Knowledge Rehearsal in Continual Learning [PMLR 2023]
2. Latent Generative Replay for Resource-Efficient Continual Learning of Facial Expressions [IEEE 2023]
3. Continual Learning for Affective Robotics: A Proof of Concept for Wellbeing [IEEE 2022]
4. Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition [IEEE 2022]
5. Towards Fair Affective Robotics: Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition [ACM/IEEE 2021]
6. Continual Learning for Affective Robotics: Why, What and How? [IEEE 2020]
7. CLIFER: Continual Learning with Imagination for Facial Expression Recognition [IEEE 2020]

Major Findings:

Major findings (as of June 2023):

• - Our results demonstrate a clear preference in the participants for CL-based continual personalisation with significant improvements observed in the robot's anthropomorphism, animacy and likeability ratings as well as the interactions being rated significantly higher for warmth and comfort as the robot is rated as significantly better at understanding how the participants feel
• - Our experimental results show that CL-based methods, on average, outperform other popular bias mitigation techniques on both accuracy and fairness metrics
• - Our experiments show that CL-based methods, on average, outperform popular bias mitigation techniques, strengthening the need for further investigation into CL for the development of fairer FER algorithms
• - The framework is evaluated on its ability to remember previously seen classes as well as on generalising to yet unseen classes, resulting in high F1-scores for multiple FER datasets: RAVDESS (episodic: F1=0.98 ± 0.01, semantic: F1=0.75 ± 0.01), MMI (episodic: F1=0.75 ± 0.07, semantic: F1=0.46 ± 0.04) and BAUM-I (episodic: F1=0.87 ± 0.05, semantic: F1=0.51 ± 0.04)

Project Team:

• - Prof Hatice Gunes (PI, Apr 2019-present)
• - Nikhil Churamani (Research Assistant, Oct 2022 – Jan 2023) – now a postdoc at AFAR Lab on another project
• - Sinan Kalkan (Visiting Academic, Oct 2019 - Sep 2020)
• - Samuil Stoychev (M.Phil Student, Oct 2020 - Jun 2021)