A system for long-term high-resolution 3D tracking of movement kinematics in freely behaving animals

PROJECT SUMMARY The aim of this proposal is to deliver an innovative and easy-to-use experimental platform for measuring and quantifying naturalistic behaviors of mammalian animal models used for biomedical research, including rodents and monkeys, across a range of spatial and temporal scales. This will require developing

a method for tracking movements freely behaving animals with far higher spatiotemporal resolution and more kinematic detail than currently possible. To overcome the limitations of current technologies, a new solution is proposed that synergistically combines two methods - marker based motion capture and a video-

based machine learning approach. First, using marker-based motion capture, the gold standard for 3D tracking in humans, the position of experimental subjects' head, trunk, and limbs will be tracked in 3D with submillimeter precision. An innovative marker design, placement strategy, and post-processing pipeline

will ensure an unprecedentedly detailed description of rodent behavior over a large range of timescales. To make the system more efficient, robust, affordable and better suited for high-throughput longitudinal studies, the unprecedentedly rich and large 3D datasets generated by the motion capture experiments will

be leveraged to train a deep neural network to predict pose and appendage positions from a set of 1-6 normal video cameras. To best capitalize on the large training datasets, the latest advances in convolutional neural networks for image analysis will be incorporated. Together, these advances will promote generalization of

the high-resolution 3D tracking system to a variety of animals and environments, thus establishing a cheap, flexible, and easy-to use kinematic tracking method that can easily be scaled up and adopted by other labs. The large ground-truth datasets will allow the system to be benchmarked and compared against state-of-the

art technologies in quantitative and rigorous ways. Preliminary studies have been very positive and suggest large improvements over current methods both when it comes to the range of behaviors that can be tracked and the precision with which they can be measured. Importantly, all new technology will be readily shared

with the scientific community, thereby leveraging from this single grant the potential for numerous investigators to dramatically improve the efficiency of their research programs requiring rigorous quantitative descriptions of animal behavior.