Dialogue-based computer-assisted language learning (CALL) systems allow a learner to practice meaningfully a foreign language (L2) with an automated agent, whether orally or through text chat. With the aim of establishing the impact of such systems on L2 proficiency development, we conducted a meta-analysis of empirical research on dialogue-based CALL. Twenty-nine effect sizes from effectiveness studies on various systems were included into a multilevel model, with moderator variables allowing for fine-grained analysis. Our findings establish a significant effect ($d = .64$) of dialogue-based CALL on general proficiency development. The comparisons of system characteristics, testing conditions and language learning outcome measurements allow to draw conclusions and recommendations for future system and research design on the topic.