ML-Based Misinformation Detection in Podcasts

Abstract

In recent years, the ubiquity of misinformation has spurred a lot of research around using machine learning to classify text-based misinformation in news articles and social media posts. However, there has been little prior work on classifying audio-based misinformation such as podcasts, despite the large quantity of misinformation they facilitate. Using the Spotify Podcast Dataset, I compile a new dataset (PAWcast) containing transcript snippets, their misinformation labels, and other metadata. Using this PAWcast dataset, in addition to five other texual misinformation datasets, I train two state-of-the-art classifiers, one based on LIWC features and one us- ing the BERT model. I then design a new ML classifier (TIGER) that finds a balance between training on the combined datasets and training on a single dataset. The TIGER model achieves a 74% F1 score on the PAWcast dataset (both with and without podcast- specific features). It also achieves a 75% average F1 score across all the datasets, which matches or exceeds the existing state-of-the-art models on each dataset.