A Machine Learning System for Editing Stand-up Comedy Specials

Abstract

In this thesis, we contribute a holistic editing model and an associated Jupyter notebook-based interface for editing stand-up comedy specials using facial detection signals and other features. Our paper's system capably produces fully-edited output videos comprising entire comedy performances, and also provides utilities for exporting generated edit decision sequences into existing external editing software such as Adobe Premiere Pro. We implement this as an end-to-end editor designed with a Jupyter notebook as its interface.

This editing program is capable of generating fully-edited comedy specials using the selected input video files, and can also output the sequence of edit decisions in a format usable by external non-linear editing programs, allowing it to compliment robust existing video editing solutions. Our model also supports composition, allowing users to apply different editing algorithms to specific portions of the overall video sequence and assemble them together. Additionally, we generate a variety of short edited segments drawn from a demonstration comedy special recorded for this thesis, and collect viewer feedback on aspects such as viewing experience quality, the perceived expressiveness and logical nature of an edit, and various other factors.

Finally, our model is evaluated on its various algorithm outputs, with positive feedback and ratings accruing to our standard editing paradigms, while the baseline random editing model is viewed more negatively. We discuss at length the need for more established datasets within the comedy special analysis domain, and detail potential routes for expanding on this paper's approach.