Skip navigation
Please use this identifier to cite or link to this item:
Title: Fast animal pose estimation using deep neural networks
Contributors: Pereira, Talmo D.
Aldarondo, Diego E.
Willmore, Lindsay
Kislin, Mikhail
Wang, Samuel S.-H.
Murthy, Mala
Shaevitz, Joshua W.
Issue Date: 30-May-2018
Related Publication: Pereira, Talmo, Diego Aldarondo, Lindsay Willmore, Mikhail Kislin, Samuel S. Wang, Mala Murthy, and Joshua W. Shaevitz. "Fast animal pose estimation using deep neural networks." bioRxiv (2018): 331181.
Abstract: Recent work quantifying postural dynamics has attempted to define the repertoire of behaviors performed by an animal. However, a major drawback to these techniques has been their reliance on dimensionality reduction of images which destroys information about which parts of the body are used in each behavior. To address this issue, we introduce a deep learning-based method for pose estimation, LEAP (LEAP Estimates Animal Pose). LEAP automatically predicts the positions of animal body parts using a deep convolutional neural network with as little as 10 frames of labeled data for training. This framework consists of a graphical interface for interactive labeling of body parts and software for training the network and fast prediction on new data (1 hr to train, 185 Hz predictions). We validate LEAP using videos of freely behaving fruit flies (Drosophila melanogaster) and track 32 distinct points on the body to fully describe the pose of the head, body, wings, and legs with an error rate of <3% of the animal's body length. We recapitulate a number of reported findings on insect gait dynamics and show LEAP's applicability as the first step in unsupervised behavioral classification. Finally, we extend the method to more challenging imaging situations (pairs of flies moving on a mesh-like background) and movies from freely moving mice (Mus musculus) where we track the full conformation of the head, body, and limbs.
Description: This dataset contains videos of freely moving fruit flies, as well as trained networks and body position estimates for all ~21 million frames. Download the README.txt file for a detailed description of this dataset's content. See the code repository ( for usage examples of these files.
Referenced By:
Appears in Collections:Research Data Sets

Files in This Item:
File Description SizeFormat 
README.txt821 BTextView/Download
expts_072212_163153.h52.77 GBUnknownView/Download
expts_01.tar13.19 GBUnknownView/Download
expts_02.tar13.99 GBUnknownView/Download
expts_03.tar12.31 GBUnknownView/Download
expts_04.tar11.89 GBUnknownView/Download
expts_05.tar12.98 GBUnknownView/Download
expts_06.tar13.06 GBUnknownView/Download
expts_07.tar13.26 GBUnknownView/Download
expts_08.tar13.8 GBUnknownView/Download
expts_09.tar14.27 GBUnknownView/Download
expts_10.tar13.85 GBUnknownView/Download
expts_11.tar13.63 GBUnknownView/Download
expts_12.tar7.78 GBUnknownView/Download
dsets_2018-05-03_cluster-sampled.k=10,n=150.h570.43 MBUnknownView/Download
dsets_2018-05-03_cluster-sampled.k=10,n=150.labels.mat291.79 kBUnknownView/Download
models_FlyAging-DiegoCNN_v1.0_filters=64_rot=15_lrfactor=0.1_lrmindelta=1e-05_03.tar.gz1.44 GBUnknownView/Download
models_hourglass.tar.gz1.67 GBUnknownView/Download
models_rotate_angle_sweep.tar.gz1.47 GBUnknownView/Download
models_sample_size_sweep.tar.gz2.21 GBUnknownView/Download
models_stacked_hourglass.tar.gz3.22 GBUnknownView/Download
preds_FlyAging-DiegoCNN_v1.0_filters=64_rot=15_lrfactor=0.1_lrmindelta=1e-05_03.tar1.44 GBUnknownView/Download

Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.