Leveraging Deep Learning to Provide Automated Assistance for Physical Therapy Exercises

Abstract

Demand for physical therapy (PT) services has skyrocketed over recent decades. However, due to the strained national supply of physical therapists, the financial burden of PT episodes, the uneven distribution of PT clinics, and mobility barriers for patients, millions still struggle to adhere to their PT regimens. We aim to boost patient adherence to PT in two primary ways. First, we release the first critique-based labeling of a PT video dataset. These labels provide specific, relevant critiques for a dataset of squat videos (KIMORE), identifying seven common mistakes that beginners make when performing the exercise. These data was generated manually in consultation with a professional physical therapist, and is intended to spur the development of statistical/machine learning models that can make similar critiques.

Second, we develop several deep learning (DL) models that assess and critique exercise videos of PT patients. These models fall into two categories: score-based models, which assign “quality scores” to videos of various exercises, and rule-based models, which assign specific critiques to squat videos. We train and evaluate these models on the KIMORE dataset (as well as our own label set), and are able to predict ground-truth quality scores and critiques to a high degree of accuracy. We assess the current state of the virtual PT space, and suggest possible avenues of future work that we believe are promising.