Modeling and Evaluation for Perspectivist NLP

Relatore: Prof. Valerio Basile

  • Data: 06 GIUGNO 2025  dalle 11:00 alle 13:00

  • Luogo: Laboratorio 76, via Cartoleria 5, Bologna - Evento in presenza e online

Abstract

A recent line of research in NLP proposes to never aggregate human annotations [1], but rather to leverage the worth of knowledge found in label variation for building models [2] and evaluating them [3]. This approach is particularly relevant when dealing with highly subjective aspects of natural language such as irony or undesirable language.
In this talk, I will present the perspectivist paradigm in NLP, and the results of recent and ongoing research focusing on building perspective-aware predictive models and automatically extract human perspectives from annotated data. I will present language resources developed under the perspectivist paradigm and their application for training or few-shotting models. Finally, I will explore the new set of challenges related to evaluating perspectivist models and how to build  benchmarking metrics and tools.

 

Literature

 

Link Webinar e registrazione

https://events.teams.microsoft.com/event/8d7bf4c4-777f-4bea-bd8f-e5fe978356eb@e99647dc-1b08-454a-bf8c-699181b389ab

Bio

Valerio Basile is an Associate Professor at the Computer Science Department of the University of Turin, Italy, member of the Content-centered Computing group. His work spans across several areas such as: formal representations of meaning, linguistic annotation, natural language generation, commonsense knowledge, semantic parsing, sentiment analysis, and hate speech detection, perspectives and bias in supervised machine learning, from data creation to system evaluation. He is the main proponent of the Perspectivist Data Manifesto.