Serge Bibauw

Dialogue-based computer-assisted language learning (CALL) can be understood as “meaningful conversational interactions” with “automated agents” (Bibauw et al., 2019, p. 827) for second language (L2) learning. Grounded in the interactionist approach to SLA, it offers learners unique opportunities for “pushed output” in authentic contexts. The emergence of Generative AI tools since 2022 has revolutionised this field by dramatically broadening access to chatbots and enhancing the naturalness and pedagogical potential of interactions, warranting a revisit to the conclusions of previous meta-analyses on dialogue-based CALL (i.e., Bibauw et al., 2022; Hou & Min, 2025). These earlier works did not compare Generative AI with previous generations of chatbots, nor did they examine the conditions (e.g., human interlocutors, traditional classroom instruction) against which chatbots were found to be effective. Therefore, the proposed meta-analysis attempts to synthesize evidence on the effects of meaningful interaction with bots (e.g., chatbots, robots, and non-player agents in XR environments) on language learning outcomes. Through an exhaustive search of multiple databases in both English and Chinese that incorporated previously overlooked terms related to robots and extended reality, 67 studies published between 2011 and 2025 were retained, yielding 237 effect sizes based on 4,544 participants across 23 L1 backgrounds. From this pool, we reconceptualize dialogue-based CALL as a continuum and propose a typology based on the freedom in learners’ spontaneous output, ranging from non-verbal responses, repetition, limited verbal output, to free speech, offering a simplified and practical alternative to the earlier framework (i.e., Bibauw et al., 2019) that would remain valid amid the evolving technological landscape. A three-level random-effects model yielded a positive, medium-sized overall effect (g = 0.61, 95% CI [0.48, 0.74], p < .001), confirming that meaningful interaction with dialogue systems improves L2 performance. Moderator analyses revealed that age group significantly influenced outcomes, with teenagers showing the largest gains (g = 1.05), followed by adults (g = 0.54) and children (g = 0.49). Outcome type also emerged as a significant moderator: dialogue systems were most beneficial for productive skills (g = 0.65) and language knowledge (g = 0.60)—particularly vocabulary (g = 0.92)—while their impact on comprehension remained non-significant. A notable finding was that matched test–practice modality conditions produced effect sizes approximately 2.5 times larger than mismatched ones, and delayed posttests (g = 0.92) outperformed immediate ones (g = 0.58), suggesting sustained learning benefits. While the underlying technology was not a significant moderator, recent LLM-powered chatbots showed the largest estimates (g = 0.64). Positive effects were observed across both in-class and out-of-class settings. In sum, this study contributes to updating the conceptual framework of dialogue-based CALL, facilitating the transfer of empirical results into real practices through more consistent terminology, proposing an evaluation framework of chatbot implementations and guidelines for future effectiveness studies, and indicating directions for the design of chatbots and pedagogical strategies.

The development of specialized vocabulary knowledge has been widely recognized as a key factor supporting learners’ ability to access disciplinary content (Coxhead, 2018; Nation, 2013). While ESP vocabulary is often learned incidentally through specialized reading, a combination of intentional and contextualized learning, with repeated practice of target words in fill-in-the-blanks and in-context multiple-choice exercises, could provide an optimal blend of intentional and contextualized learning. Background. While previous research has shown that decontextualized learning can be as effective as contextualized exercises (Webb, 2007) and that fill-in-the-blanks exercises might lack efficiency against decontextualized items (Webb et al., 2021), the contexts provided around the target words also contribute to reinforcing knowledge of other words and might provide incidental learning opportunities (Kodama & Shirahata, 2021). Context has also been demonstrated to be more beneficial for intermediate and advanced learners (Griffin, 1992; Prince, 1996) and to have a greater effect on the acquisition of words with no synonyms (Webb, 2007), common in ESP. Generative AI provides new opportunities for such learning, as it allows teachers and learners to create contextualized, proficiency-level-adjusted input and exercises on demand (Drackert et al., 2025; Fincham & Alvarez, 2024). In this project, we developed and tested an AI-enhanced tool to generate contextualized, repeated-practice ESP vocabulary exercises in multiple-choice and fill-in-the-blank formats, tailored to students’ global proficiency levels. Research question. The present study aims to answer the question: How do vocabulary learning gains from repeated, intentional practice with contextualized exercises compare with those from autonomous learning from a word list, in terms of form recall and form recognition, for both target (intentional) and non-target (incidental) ESP vocabulary? Methods. A controlled experiment with N = 164 intermediate and advanced (B1-B2) students in English for sciences was conducted with two cohorts of students over one semester, to measure the effectiveness of the AI-tailored exercises on ESP vocabulary acquisition. Each cohort was tested at the beginning and end of the semester on their form recognition and form recall of 120 items, consisting of target and non-target academic vocabulary. Both groups had to study the target words as part of the word list of the course. The experimental group (n = 105) had access to the in-context vocabulary exercises in their usual learning environment, Moodle, throughout the term, while the control group (n = 59) had access only to the word list. The experiment was conducted in an ecologically valid manner by integrating it into the natural flow of the course and aligning it with its learning objectives. Using the AI-enhanced tool, 3178 exercises were generated and manually validated for students to practice with. Results. Preliminary results show that word learning is impacted by the number of occurrences of the target word in response options and gapfill (OR = 1.24, p < .001), much more than in questions. Each additional occurrence of the target word in a student’s own responses increases post-test odds by ~24%. Productive engagement (typing/producing the word) predicts learning; passive exposure in question text does not (p = .64).

L’agentivité de l’apprenant en formation médiatisée articule divers plans, notamment ce que le dispositif rend possible (affordances), ce que l’apprenant en fait et ce qu’il en perçoit (Eteläpelto et al., 2013). L’étude examine ces trois plans dans un système de dialogue pour l’apprentissage des langues, manipulant l’interactivité du dialogue afin d’en influencer l’agentivité. Une expérience randomisée auprès de 215 adolescents apprenant le français en Flandre compare deux versions du même jeu : un dialogue interactif libre et une tâche de complétion. Les résultats portent sur les perceptions du système (interactivité, utilité, facilité d’utilisation perçues, attitude, comme prédicteurs TAM de la motivation à utiliser le dispositif) et l’engagement comportemental et cognitif des participants. Contrairement aux attentes, les conditions ne diffèrent pas significativement en perceptions, mais le dialogue interactif suscite un engagement comportemental et cognitif nettement plus élevé. Les apprenants sont plus engagés, sans être plus motivés : un décalage entre affordances, exercice et perception de l’agentivité, central pour la conception des dispositifs numériques d’apprentissage.