Deliverables

Upon completion of UNITE, our team will deliver the following.

Project

Deliverables

Review

Systematic review (PRISMA 2020) of LLM-based chatbots in/for language learning

A systematic review of existing dialogue systems that can be used as pedagogical tools in EFL contexts,  outlining those potentially more inclusive.

 

The review comprises 43 peer-reviewed journal articles published from 2018 onwards, all written in English. It focuses on meaning-oriented practice (Bibauw et al. 2022) and features both written and spoken conversational tasks. 

 

The dimension of inclusion of all learners, including learners with disabilities and learning disorders, is evaluated post-hoc based on accessibility criteria (cf. CAST 2018).  

Corpus

Corpus

A richly annotated corpus of written interactions between learners and LLM-based chatbots, available both as a corpus proper and as a standalone dataset.

 

The UNITE corpus follows in the wake of other initiatives aiming to provide corpus resources representative of a range of written outputs by English language learners (Fernández and Davis 2020),  with a specific focus on the Italian higher education context and on pedagogical applications.

 

It is one of the few learner corpora to feature instances of learner-machine interaction, which differ according to such variables as the task performed, the technology involved and the student’s profile. Its annotation layers not only include contextual and linguistic information, but also information on normative discourses and errors in chatbot and learner production, respectively.

Tagset

Tagset

A specific and innovative tagset (DIS-TAG) connecting chatbots' linguistic outputs and exclusionary practices for the analysis of ableist discourses.

 

DIS-TAG includes semantic and pragmatic tags that either implicitly or explicitly refer to multiple dimensions:

  • the pragmatic tagset marks features like implicatures, metaphors, analogies, humour and irony, all relating to cognition;
  • the semantic tagset marks both linguistic and extra-linguistic triggers of motor and sensory abilities.

Teaching Materials

Teaching Materials

Teacher-oriented guidelines and accompanying teaching materials for teacher-led and autonomous language practice using chatbots.

 

  • The guidelines feature a comprehensive overview of available chatbots, with clear indications of their level of inclusivity.
  • The teaching materials consist of a series of topics and tasks, with suggestions for teachers on how to overcome potentially limiting issues, as well as methods to effectively collect feedback from students and monitor their progress.

 

To broaden UNITE's audience, the materials will also be produced as independent guides for learners available as Open Educational Resources in written and/or video form.