CATER: Combined Animal Tracking & Environment Reconstruction

1Institute for Geoinformatics & Institute for Computer Science, University of Münster
2Department of Computer Science, University of Sheffield
3Research Centre on Animal Cognition, Centre for Integrative Biology, CNRS, Université Paul Sabatier, Toulouse
4School of Informatics, University of Edinburgh

These authors contributed equally to this work

*Corresponding author:


Quantifying the behavior of small animals traversing long distances in complex environments is one of the most difficult tracking scenarios for computer vision. Tiny and low-contrast foreground objects have to be localized in cluttered and dynamic scenes as well as trajectories compensated for camera motion and drift in multiple lengthy recordings. We introduce CATER, a novel methodology combining an unsupervised probabilistic detection mechanism with a globally optimized environment reconstruction pipeline enabling precision behavioral quantification in natural environments. Implemented as an easy to use and highly parallelized tool, we show its application to recover fine-scale motion trajectories, registered to a high-resolution image mosaic reconstruction, of naturally foraging desert ants from unconstrained field recordings. By bridging the gap between laboratory and field experiments, we gain previously unknown insights into ant navigation with respect to motivational states, previous experience, and current environments and provide an appearance-agnostic method applicable to study the behavior of a wide range of terrestrial species under realistic conditions.