Towards a Holistic Assessment of Animal Welfare using Emotion and Deep Learning

AIM: Inspired by holistic human observational processes, realise a machine learning approach to recognise animal emotional states from video.
The project targets Topic 1 of the project call by EUPAH&W - Novel Technologies for Prevention, Detection, Assessment and Management of Animal Health and Welfare

Background: Developments in AI, particularly in relation to image / video captioning and for activity and behaviour recognition in humans, together with recent work on using a holistic (whole animal) approach for detecting emotional state, provide a timely technological driver aimed at creating significantly more capable tools for assessing animal welfare (Topic 1 of the call). An opportunity now exists to explore the evidence base for developing new products and tools for animal welfare monitoring in terms of emotional wellbeing with a potential for near-term impact. As such, this project addresses the call Priority Areas in Surveillance and Monitoring and Risk Assessment and Alert Communication in the context of enhancing animal health and welfare.

The project builds on the successful collaboration between machine vision and learning experts at UWE and animal behaviour and welfare experts at SRUC. It is expanded here to include partners from across Europe who are interested in use of AI technologies for animal welfare assessment, particularly recognising affective state.

The work at SRUC will involve providing video data to generate labelled images for machine vision and learning model development (at UWE's Centre for Machine Vision). We propose a 3Rs approach to use existing experimental work and footage to generate a media library of individual pigs that have been exposed to different stimuli in order that they express a range of affective states/emotions. These videos will facilitate Qualitative Behavioural Assessment (QBA) as well as capture of image data for development of deep learning models by UWE's Centre for Machine Learning. QBA footage will be shown to recruited human observers and these observers will themselves be filmed for development of deep learning models to investigate the following hypotheses and objectives.

RESEARCH HYPOTHESES AND OBJECTIVES
Our first hypothesis is that: a machine can use a holistic human-like interpretation model to detect animal emotion as a location plotted in a two-dimensional emotional space, for example of valance (positive or negative) and arousal (high or low).
Our second hypothesis is that: human expert domain knowledge for recognising emotion may be captured and used to inform feature extraction in machine learning, enabling the detection of animal emotional states against the context of limited quantities of training data.
Our third hypothesis is that: both human and machine holistic learning does not require the use of all available information across all available time, but rather the sampling of partial, possibly fragmented, yet highly focused and salient pieces of information able to generate an affective profile, and over time a trajectory, for an animal’s emotion.

We use three linked objectives to investigate our hypotheses:
OBJ 1 - Human-like machine models: Inspired by work in QBA we develop and test four holistic spatiotemporal machine models to recognise a range of animal emotional states and compare against benchmarks.
OBJ 2 - Learning: we explore a novel form of supervised learning to capture human visual search behaviour for identifying salient features in detecting animal emotional states to inform a machine learning process for a reduced dataset.
OBJ 3 - Evaluation: we evaluate animal emotion classification accuracy of holistic human-like artificial assessment models as a tool in a real-world application using pigs in their farm settings with our EU partners. Using emotion-generating contexts, observable as body language / behaviour as well as changes in physiology (heart and respiration rate, temperature, etc) and neural markers as ground truth, we identify, compare, and contrast feature spaces used in human and machine animal assessment.

OBJ 1 and 2 will produce novel advances in the capture and interpretation of holistic emotional cues across the disciplines of machine vision, human vision, and machine learning. OBJ 3 will demonstrate how these advances can be integrated and deployed in real farm applications for monitoring animal wellbeing.