Emotion Detection through Facial Feature Analysis: Towards Digital Support for Children with Autism

Date of Award


Degree Name

Master of Science in Engineering


Electrical and Computer Engineering

First Advisor

Dr. Ikhlas Abdel-Qadar

Second Advisor

Dr. Janos Grantner

Third Advisor

Dr. Michelle Suarez


Autism Spectrum Disorder, facial expression recognition, digital image processing, computer vision

Access Setting

Masters Thesis-Campus Only

Restricted to Campus until



The human desire for a better life for all has caused a tremendous growth in human computer interaction technology. Facial Expression Recognition (FER) has been acknowledged as an emerging research thrust due to its wide range of applications specifically in biometrics, and emotion analysis. Facial expressions are a natural form of non-verbal communication. However, many children with Autism Spectrum Disorder (ASD) have difficulty interpreting the emotions of others and expressing their own emotions. Solutions such as FER systems may help as a tool designed to better understanding their emotions and ultimately better their communication abilities.

The goal of this investigative work is to design and implement a FER system that detects basic facial expressions using image processing and Machine Learning (ML) algorithms. This proposed FER system is divided into three significant components: 1) a face detection component where the face is detected using a Viola-Jones object detector, 2) a feature extraction and selection component where facial features are extracted using Gabor Filter Banks with the most significant features selected using an Adaboost algorithm, and 3) an emotion classification using Strong Adaboost Classifiers. For system validation, JAFFE dataset is used, and the experimental results show that the system achieves 84.12% successful recognition. This system can be custom designed for ASD children to help them learn social cues of their surroundings, and a tool for clinician to better understand the child’s ability to express emotions.

This document is currently not available here.