Skip navigation
Please use this identifier to cite or link to this item:
Title: Low-dimensional Representations of Semantic Context in Language and the Brain
Authors: Vodrahalli, Kiran N
Advisors: Arora, Sanjeev
Norman, Ken
Contributors: Dvir, Zeev
Department: Mathematics
Class Year: 2016
Abstract: We study the problem of finding low-dimensional shared representations of meaning for natural language and brain response modalities for multiple-subject narrative story datasets (a portion of an episode of the Sherlock television program and a chapter of a Harry Potter book). These datasets have paired fMRI responses and textual descriptions. Our first goal is to determine if any fMRI space can be learned across subjects that correlates well with semantic context vectors derived from recent, unsupervised methods in natural language understanding for embedding word meaning in Rn. Can distributed, low-dimensional representations of narrative context predict voxels? Our second goal is to determine if a shared space between the fMRI voxels and the semantic word embeddings exists which can be purposed to decode brain states into coherent textual representations of thought. First, we were able to construct a fine-grained 300-dimensional embedding of the semantic context induced by a scene annotation dataset for Sherlock. Our primary positive result in this thesis is that the multi-view Shared Response Model produces a semantically relevant 20-dimensional space using views of multiple subjects watching Sherlock. This lowdimensional shared fMRI space is able to match fMRI responses to scenes with performance considerably above chance. Using the fMRI shared space over individual fMRI responses brings a large improvement in reconstructing voxels from semantic vectors, and suggests that other recent work in this area may benefit from applying the Shared Response Model
Extent: 90 pages
Type of Material: Princeton University Senior Theses
Language: en_US
Appears in Collections:Mathematics, 1934-2020

Files in This Item:
File SizeFormat 
VODRAHALLI_Kiran_thesis.pdf1.76 MBAdobe PDF    Request a copy

Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.