The Linguistics Department presents Dr. J. Elliott Casal discussing: “Its important to consult” a linguist?: Two perspectives on ChatGPT-generated text

Abstract:  Considerable recent discussion (in research papers, faculty meetings, and public discourse) has emphasized the potential dangers, affordances, and curiosities of ChatGPT and other LLM-powered chatbots that can generate lucid text on demand. This talk draws from two recent studies - Casal and Kessler (2023) and Casal, Stewart, and Windsor (in press). The two studies are summarized and illustrated with examples, and implications for ethics, pedagogy, and research are discussed. Using a specific text-generation protocol, Casal and Kessler (2023) investigated the extent to which 72 editorial board members from top Applied Linguistics journals could distinguish AI-generated from human-written research article abstracts and what criterion formed the basis of their decisions. Findings suggest that despite employing multiple rationales to judge texts, reviewers were largely unsuccessful in identifying AI versus human writing in this context, with an overall positive identification rate of only 38.9%. Casal, Stewart, and Windsor (in press) adopt a usage-based linguistics perspective to examine the use of verb-argument constructions and the verbs within these schemas in human and ChatGPT produced advice in medical and financial domains (approximately 5,200 question-answer pairs). ChatGPT responses to all queries were generated by GPT 3.5 web, GPT API with default settings, 4 web, and 4o web, to allow for comparisons across models. While the overall verb-argument constructions and frequency ranks were notably consistent across human and ChatGPT corpora, ChatGPT over relies on prototypical patterns in general and the sharp differences in verbs within analyzed schemas highlight important divergences away from broad conventions of human-produced advice over time.

Dr. J. Elliott Casal is an Assistant Professor of Applied Linguistics in the Department of English and Affiliate faculty member of the Institute for Intelligent Systems at the University of Memphis. Casal is also an alumnus of the  M.A. Applied Linguistics program  '14.

Via Microsoft Teams 

Join the meeting now

Meeting ID: 219 960 959 435

Passcode: rq9bs6am