MultiMediate: Multi-modal Group Behaviour Analysis for Artificial Mediation

Eye Movement Analysis for Activity Recognition

Andreas Bulling, Jamie A. Ward, Hans Gellersen, Gerhard Tröster

Proc. ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), pp. 41-50, 2009.


In this work we investigate eye movement analysis as a new modality for recognising human activity. We devise 90 different features based on the main eye movement characteristics: saccades, fixations and blinks. The features are derived from eye movement data recorded using a wearable electrooculographic (EOG) system. We describe a recognition methodology that combines minimum redundancy maximum relevance feature selection (mRMR) with a support vector machine (SVM) classifier. We validate the method in an eight participant study in an office environment using five activity classes: copying a text, reading a printed paper, taking hand-written notes, watching a video and browsing the web. In addition, we include periods with no specific activity. Using a person-independent (leave-one-out) training scheme, we obtain an average precision of 76.1% and recall of 70.5% over all classes and participants. We discuss the most relevant features and show that eye movement analysis is a rich and thus promising modality for activity recognition.



@inproceedings{bulling09_ubicomp, author = {Bulling, Andreas and Ward, Jamie A. and Gellersen, Hans and Tr{\"{o}}ster, Gerhard}, keywords = {Activity Recognition, Electrooculography (EOG), Eye Movement Analysis, Recognition of Office Activities}, title = {Eye Movement Analysis for Activity Recognition}, booktitle = {Proc. ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp)}, year = {2009}, pages = {41-50}, doi = {10.1145/1620545.1620552} }