Abstract
BACKGROUND. Interactivity has been defined in various ways: in instructed SLA, as distinguishing dialogic tasks from monologic ones (Ellis et al., 1994); in game design, as involving agency and user control. We propose a bidimensional model of interactivity in conversational AI, structured around user control and bi-directional adaptivity. This study addresses the question of how these different dimensions of interactivity are operationalized in conversational AI for language learning, i.e., when learners interact with a chatbot to practice an L2 (Bibauw et al., 2019; 2022), and how much they influence the learning experience.
METHODS. We conducted an experimental study with 215 teenage learners of French, who interacted with two versions of a serious game designed around guided conversations with AI-based talking characters. The two versions differed in interactivity, one being entirely free while the other replicated a dialogue completion task. We measured learners’ perceptions, engagement, and learning effects on vocabulary knowledge.
RESULTS. Contrary to our expectations, the free dialogue system was not perceived as significantly different compared to the dialogue completion task. However, there were significant differences in the perception of a pilot version of the system devoid of scaffolding and feedback mechanisms. On the other hand, the interactivity of the dialogue system increased behavioral engagement and production through trials-and-errors incentivized by the system feedback. Vocabulary learning shows similar effects for both conditions in receptive (d = 1.16) and productive knowledge (d = 0.59).
DISCUSSION. Beyond certain limitations proper to the study design, we hypothesize that interactivity, understood as a set of game-like motivational characteristics such as agency, might have less impact on perceptions and effectiveness than expected. Our results refocus our understanding of the benefits of interactivity towards differences in engagement, feedback, and scaffolding, and concretely realized in conversational AI through negotiation of form and meaning.