Justin Gross (Director, Data Analytics & Computational Social Science)
Friday, May 1, 12-1:30pm
Lederle Graduate Research Center (LGRC), Room A112
AI-Enabled Thematic Analysis:
Core Value Detection & Validation in the Ideological Books Corpus
Abstract:
Rapid advances in generative AI have vastly expanded the possibilities for large-scale text analysis in situations that require successful decoding of subtle differences in meaning and rhetoric. Topic models and conventional NLP tools have provided limited assistance to the researcher who seeks to better understand diversity within right-wing or left-wing parties and movements, shifts of value framing among party stalwarts after a dramatic change in leadership, or competing definitions of contested concepts such as liberty, equality, or security across partisan divides, national settings, or epochs. However, off-the-shelf LLMs now do extraordinarily well on this and other related tasks.
The current project begins with instrument development, resulting in modules of prompts to an AI research assistant tasked with the detection of salient political values by treating corpus texts as pseudo-surveys of their authors. The endeavor raises interesting questions about the nature of validity and reliability in the AI age, which we describe and address.
Work by: Justin Gross, Venkat Dasari, and Sumedha Sumedha.
Bio:
Justin Gross is an Associate Professor and Director of the Data Analytics & Computational Social Science program in the School of Public Policy at UMass Amherst. His research focuses on ideology and political communication, particularly how opinion leaders frame public policy issues in terms of competing core values and beliefs. He earned a Ph.D. in Statistics and Public Policy from Carnegie Mellon University.
