Project

The specific objectives of UNITE include: 

The UNITE project aims to develop evidence-based teaching and learning materials that will facilitate the integration and even incorporation of Dialogue Systems in Italian universities.  

Detailed descriptions of each individual objective can be found by clicking on the designated sections on the right side of this page. 

Providing an overview of available DS which could be integrated in EFL courses. An outline of existing DS to potentially use in EFL teaching is a critical first step in establishing a solid research foundation. Only publicly available tools that use English as communication language will be considered and evaluated. The review process will focus on features related to engagement, representation, actions and expression. Furthermore, DS technologies will be analysed in terms of their potential usability, prioritising DS in the written medium, with the ultimate goal of identifying a diverse range of Dialogue Systems.

Assembling empirical evidence of how Italian university students interact with DS. After determining the most promising DS among those analysed, test sessions will be conducted to investigate the Dialogue Systems' features as well as the user-perceived efficacy of interactions between Italian English learners and DS. 

Compiling corpora/datasets of DS-learners written interactions. Preparing a corpus is a preliminary step that must be carried out appropriately before further analyses. The final result to be achieved is a dialogic corpus of interactions in English, which will be used to explore the capabilities of chatbots and their performance.

Investigating normative discourses in the interactions between DS and users which may generate asymmetries of opportunities and social exclusion.  Due to the distinctive features of dialogue systems, trained on LLMs and capable of entertaining conversations with human users, one of the main issues to be addressed is the potential presence of discriminations, prejudices and biases, which may interfere with students’ learning and overall experience. In order for dialogue systems to be suitable tools for language learning, it is indeed mandatory to tackle these issues – if present – to guarantee equitable, respectful access to language practice to everyone.

Analysing errors in learners’ production which might lead to failure of the interactions and point to the need for further linguistic reinforcement on the part of lecturers. We all learn from our mistakes: learners’ errors are a valuable source of information about both students’ proficiency and chatbots’ capabilities, and tell us the extent to which dialogue systems are able to detect and manage inaccuracies in an efficient manner, so that students can engage in successful interactions.

Creating guidelines and teaching materials for successfully integrating DS in English language courses and for autonomous learning. The results of our study will be crucial to develop materials to support both learners and language teaching professionals in the use of dialogue systems. These resources will be beneficial for the purpose of language learning, while supporting autonomous practice and self-confidence in the use of the foreign language. 

Upon completion of UNITE, our team will deliver the following.

Review

A systematic review of existing dialogue systems that can be used as pedagogical tools in an EFL context. The technologies will be reviewed based on several parameters: type, modality, features and scope. 

An innovative aspect of UNITE is the formulation of a model specifically designed to evaluate the inclusivity of DS, owing to three main principles:

(1) providing multiple means of engagement;

(2) providing multiple means of representation;

(3) providing multiple means of action and expression.

Corpus

A richly annotated corpus of written interactions between learners and dialogue systems, available both as a corpus proper and as a standalone dataset.

The UNITE corpus follows in the wake of other initiatives aiming to provide corpus resources representative of a diverse range of written production of English language learners. Unlike most learner corpora, it will be made up of dialogic data and will constitute one of the very first learner corpora of human-machine interaction. In its XML format, the corpus will also be reusable as a dataset for Natural Language Processing to train data-driven DS.

Tagset

A specific and innovative tagset (DIS-TAG) aiming at connecting dialogue systems' linguistic outputs and normative discourses for the analysis of ableist discourses.

 DIS-TAG will include semantic and pragmatic tags that refer to different abilities or could implicitly be connected to them. The pragmatic tagset will encompass tags allowing the marking of pragmatic features like implicatures, metaphors, analogies, humour and irony that pertain to the field of cognition. The semantic tagset will include tags allowing the marking of references to linguistic and extra-linguistic instances referring to mobility (“click”, “push”, “tap” etc.) and senses (“watch”, “listen”, “spell”, etc.), including spatial lexical repertoires in the form of verbs, adpositions and adverbs, since not all learners are grounded on the same visual modality.

Teaching Materials

Teacher-oriented guidelines and accompanying teaching materials for teacher-led and autonomous language practice using dialogue systems.

Guidelines will feature the comprehensive overview of available DS, with clear indications on their level of inclusivity.

The teaching materials will consist of a series of topics and tasks which might make the focus of the interactions between learners and DS, with suggestions for teachers on potentially problematic points (related to syntax, grammar, lexis, etc.) that might need explicit reinforcement to favour successful interactions, as well as methods to effectively collect feedback from students on the activities performed and monitor their progress.

To broaden the potential audience of this project result, the materials will also be produced as independent guides for learners, to be distributed as soon as they are available as Open Educational Resources in written and/or video form.