EmotiPig, Investigating automatic detection of emotion in biometrically identified pig faces using machine learning

Early identification and resolution of pig health issues results in reduced production costs and improved animal wellbeing. Current manual visual inspection offers only intermittent and subjective information often at a group level. A capacity for continuous automated monitoring of individual pigs allows on-going learning about individuals, and consequently allows early detection of altered health and welfare, so permitting more timely and cost effective remedial interventions. Going beyond that goal would be the development and harnessing of technology capable of assessing animal affective state thus offering a truly insightful, animal-centric welfare assessment tool.

This research is particularly novel and timely as it uses highly innovative technologies to develop an animal-centric assessment of welfare, understanding that the sentience of animals is something of great importance to society and policy makers. By focussing on the highly individual measure of facial expression we can deliver a welfare assessment technique that goes beyond basic monitoring to actually inferring something about the importance the animals themselves place on particular experiences. Whilst the absence of negative affective state is and should be a priority we will also include particularly novel work to detect positive affect in facial expression, thus moving closer to the ultimate goal of measuring whether animals are experiencing "a good life". The project is therefore relevant to the BBSRC strategic priorities: welfare of managed animals, sustainably enhancing agricultural production, animal health and technology development for the biosciences.

Machine vision offers the potential to realise a low-cost, non-intrusive and practical means to both biometrically identify individual animals and then assess and record their condition continuously each day using only the face. By employing state-of-the-art machine learning techniques, such a system would offer the capacity for on-going learning about individuals, and consequently allow for early detection of altered health/welfare, personalised thresholds for intervention, and tailored treatment approaches. Such individualised data recording can also be used in a wider precision farming context by association with other measurable parameters, such as individual food and water intake, treatment history, growth and weight gain, in order to better optimise farm production efficiency. The most innovative application, however, is the potential to use a non-intrusive technique to infer affective state, allowing insight into both short-term emotional reactions and longer-term individual "moods" of animals under human care. The project delivers clear benefits to multiple stakeholders within the livestock sector and therefore it has attracted the interest and support of influential industry and technology companies keen to be involved, from the start, in the development and application of such an innovative approach to animal welfare assessment.


Technical Summary

We propose a highly novel approach of using advances in machine vision and machine learning to automatically detect and monitor key affective states in individually identified pigs and measure performance traits using only the face. The basis for this project is our successful proof-of-concept work showing pig' faces are reliable biometrics.

Our project is phased: Phase 1 involves a series of controlled experimental studies to ground-truth the technology by generating facial images of animals in negative and positive affective states. Images will optimise our existing animal identification algorithms and establish and validate specific facial characteristics relating to affective states. Phase 2 applies this knowledge in a commercial context and investigates facial correlates of animal performance (i.e. weight). Negative affect will be investigated by using established models of pain (naturally occurring lameness) and stress (social defeat). Positive affect will be induced by temporary amelioration of chronic hunger (i.e. removal of a negative state). We have shown that deep learning achieves 97% accuracy in identifying pigs using conventional 2D images. The dataset size and composition will significantly increase to establish scalability limits and quantify the benefits of adding 2.5D and 3D features for more robust biometric recognition. Changes in facial expression can be subtle so 2D, 2.5D, 3D and infrared data will all be explored for automated expression recognition. To detect deviations in individual facial expressions and recognise emotional valence different machine learning techniques will be used (x2 established feature-based approaches, x1 novel, state-of-the-art Convolutional Neural Network). Finally a refined prototype system will be tested on a large commercial farm and additional data taken to investigate facial correlates of weight. This phase determines the feasibility of our innovative, animal-centric welfare assessment tool for the end-user.


Planned Impact

Welfare of managed animals: Machine vision and machine learning offer potential to realise low-cost, non-intrusive and practical means to both biometrically identify individual animals and assess and record their condition daily using only the face. This would facilitate on-going learning about individuals and allows for early detection of altered health/welfare, personalised thresholds for intervention, and tailored treatment. Whilst the absence of negative affective state is and should be a priority such technology provides further opportunities to identify positive affective state thus moving closer to the ultimate goal of measuring whether animals experience "a good life".

Pig Producers, veterinarians, breeding companies: Our consortium has strong industry links, with academic partners demonstrating track records in engaging with industry to develop precision livestock technology for commercial uptake. Our industry veterinary partners say tools that can identify animals requiring attention "earlier than the most skilled stockperson" would be highly valued especially in larger operations with increased mechanisation where the ratio of pigs to stockpersons is increasing. Breeding companies are interested in technology that could help incorporate new traits into breeding programmes and select the most robust animals for clients. An ability to detect animals that might be particularly pain and/or stress susceptible at a pre-selection phase would be highly valued.

Welfare accreditation schemes, retailers, supply chain: Welfare assessment measures undertaken on farm for assurance schemes (RSPCA Assured, Red Tractor, Retailer-specific - e.g. "Taste the Difference") rely heavily on systems-based approaches by inspectors observing the farm intermittently at a group level. This project offers a highly animal-centric health and welfare assessment tool that would operate continuously to assess the affective state of individual animals on-farm.

Environmental: Benefits accrue from a more efficient, sustainable pig industry. Higher health and welfare status for breeding animals will lead to longevity in herds, will reduce environmental impact and economic losses for the industry and help improve feed conversion efficiency per kg of meat produced, boosting food security. The individualised data recording in our automated system can also be used in a wider precision farming context by association with other measurable parameters, eg individual food and water intake, treatment history, growth/weight gain, to better optimise farm production efficiency.

Society: Consumers are increasingly aware of and concerned for farm animal welfare. They have historically stimulated legislation to protect animal welfare and recent heated discussions between policy makers and citizens over animal sentience confirms the public's significant and enduring demand that welfare of farmed animals is assured by government. The potential for improved health, welfare and quality standards in the pig sector that this technology presents will lead to an enhanced public perception of industry as well as, via improved animal health and reduced pig farming costs, providing consumers with greater access to lower-cost, higher quality pork products.

Economic: The UK farm industry has a global reputation for livestock production under high welfare standards and transparency of such a reputation will enhance the integrity of the UK pig industry and provide a USP for pig meat produced in this way, giving the UK an added advantage and economic competitiveness. Such an endeavour is particularly timely as the UK agricultural sector aims to position itself in trade negotiations for a post-Brexit era. There will be more immediate economic impacts via exploitation by project commercial partners and pig farmers who will be provided with a precision tool for automatically and objectively monitoring individual pig health and wellbeing, thus improving herd efficiency.