Language, Speech and Interaction | Doctoral Program - Information Engineering and Computer Science

Language, Speech and Interaction

Our main areas of interest are speech, vision and language processing, machine learning and interaction.

The purpose of our research is to investigate how humans process speech, language and vision, and to design mathematical models for automatic processing that can be used with communicating machines.

Our studies rely on computational models from batch or on-line examples and provide predictive models for data classification, automatic sequence labelling and structure inference.

Another research area the group is involved in is human-centered communication systems. We investigate models of interactions in any ICT scenario, such as speech-to-speech, speech-to-web and multimodal interactions. Our focus is both on the computational aspect of the interaction models as well as on the usability of user interfaces.

Finally, we also devote our attention to collaborative systems and web architecture. We are active in the design of software architecture for advanced web and multimedia collaborative systems.

 

Publications

24 publications for 16 currently enrolled students

Are LLMs Robust for Spoken Dialogues?
Mousavi, Seyed Mahed; Roccabruna, Gabriel; Alghisi, Simone; Rizzoli, Massimo; Ravanelli, Mirco; Riccardi, Giuseppe in Proceedings of the 14th International Workshop on Spoken Dialogue Systems Technology, sapporo, Japan: 14th International Workshop on Spoken Dialogue Systems Technology, 2024. Proceedings of: IWSDS2024, JAPAN, 04/03/2024

FedEdge: Federated Learning with Docker and Kubernetes for Scalable and Efficient Edge Computing
Hassan, Mir; Leonardo, Lucio; Yildirim, Sinan; Iacca, Giovanni in 1st International Workshop on Machine Learning for Autonomic System Operation in the Device-Edge-Cloud Continuum (MLSysOps 2023), New York, NY, USA: ACM, 2023. Proceedings of: The International Conference on Embedded Wireless Systems and Networks (EWSN 2023), Rende, Italy, 25th September-27th September 2023
[other topics: Machine Learning and Intelligent Optimization

PRODIGy: a PROfile-based DIalogue Generation dataset
Occhipinti, Daniela; Tekiroglu, Serra Sinem; Guerini, Marco 2023

Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpus
Piergentili, Andrea; Savoldi, Beatrice; Fucci, Dennis; Negri, Matteo; Bentivogli, Luisa in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore: Association for Computational Linguistics, 2023. Proceedings of: Empirical Methods in Natural Language Processing, Singapore, 6 –10, December, 2023. - DOI: 10.18653/v1/2023.emnlp-main.873

Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection
Fucci, Dennis; Gaido, Marco; Papi, Sara; Cettolo, Mauro; Negri, Matteo; Bentivogli, Luisa in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore: Association for Computational Linguistics, 2023. Proceedings of: EMNLP 2023, Singapore, December 2023. - DOI: 10.18653/v1/2023.emnlp-main.705

Countering Misinformation via Emotional Response Generation
Russo, Daniel; Kaszefski-Yaschuk, Shane; Staiano, Jacopo; Guerini, Marco in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 209 N. Eighth Street, Stroudsburg PA 18360, USA: Association for Computational Linguistics, 2023, p. 11476-11492. Proceedings of: The 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, 6th-10th December 2023. - Publication URL . - DOI: 10.18653/v1/2023.emnlp-main.703

Weigh Your Own Words: Improving Hate Speech Counter Narrative Generation via Attention Regularization
Bonaldi, Helena; Attanasio, Giuseppe; Nozza, Debora; Guerini, Marco in Proceedings of the 1st Workshop on CounterSpeech for Online Abuse, 317 Sidney Baker Street S., Suite 400-134, Kerrville, TX 78028, USA: Association for Computational Linguistics, 2023. Proceedings of: 1st Workshop on CounterSpeech for Online Abuse (CS4OA), Prague, Czechia, 11th Septemper 2023. - Publication URL

PoliticIT at EVALITA 2023: Overview of the Political Ideology Detection in Italian Texts Task
Russo, D.; Jimenez-Zafra, S. M.; Garcia-Diaz, J. A.; Caselli, T.; Guerini, M.; Alfonso Urena-Lopez, L.; Valencia-Garcia, R. in CEUR Workshop Proceedings, Aachen: CEUR-WS, 2023. - (CEUR WORKSHOP PROCEEDINGS). Proceedings of: 8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop, EVALITA 2023, Parma, Italy, 7–8 September 2023. - Publication URL

Benchmarking the Generation of Fact Checking Explanations
Russo, Daniel; Tekiroğlu, Serra Sinem; Guerini, Marco in TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, v. 11, (2023), p. 1250-1264. - Publication URL . - DOI: 10.1162/tacl_a_00601

Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challenges
Piergentili, Andrea; Fucci, Dennis; Savoldi, Beatrice; Bentivogli, Luisa; Negri, Matteo in Proceedings of the first Workshop on Gender-Inclusive Translation Technologies, Tampere: ACL anthology, 2023. Proceedings of: GITT, Tampere, 15/06/2023. - Publication URL

Understanding Emotion Valence is a Joint Deep Learning Task
Roccabruna, Gabriel; Mousavi, Seyed Mahed; Riccardi, Giuseppe in Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, Toronto, Canada: Association for Computational Linguistics, 2023, p. 85-95. - ISBN: 978-1-959429-87-6. Proceedings of: ACL, Toronto, Canada, 9th-14th July 2023. - Publication URL

Let's Give a Voice to Conversational Agents in Virtual Reality
Yin, Michele; Roccabruna, Gabriel; Azad, Abhinav; Riccardi, Giuseppe in Proc. INTERSPEECH 2023, Dublin, Ireland: International Speech Communication Association ISCA, 2023, p. 5247-5248. Proceedings of: Interspeech 2023, Dublin, Ireland, 20th August-24th August 2022. - Publication URL

What{'}s New? Identifying the Unfolding of New Events in a Narrative
Mousavi, Seyed Mahed; Tanaka, Shohei; Roccabruna, Gabriel; Yoshino, Koichiro; Nakamura, Satoshi; Riccardi, Giuseppe in Proceedings of the The 5th Workshop on Narrative Understanding, Association for Computational Linguistics: Association for Computational Linguistics, 2023, p. 1-10. Proceedings of: 5th Workshop on Narrative Understanding, toronto canada, July 14, 2023. - Publication URL . - DOI: 10.18653/v1/2023.wnu-1.1

Can Emotion Carriers Explain Automatic Sentiment Prediction? A Study on Personal Narratives
Mousavi, M.; Roccabruna, G.; Tammewar, A.; Azzolin, Steve; Riccardi, G. in Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, Dublin: ACL, 2022. Proceedings of: Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, Dublin, 23rd - 27th May 2022. - Publication URL
[other topics: Deep and Structured Machine Learning

Evaluation of Response Generation Models: Shouldn't It Be Shareable and Replicable?
Mousavi, Seyed Mahed; Roccabruna, Gabriel; Lorandi, Michela; Caldarella, Simone; Riccardi, Giuseppe in Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), Association for Computational Linguistics: Association for Computational Linguistics, 2022, p. 136-147. Proceedings of: EMNLP 22, abu dhabi, 5 december 2022. - Publication URL

Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering
Bonaldi, Helena; Dellantonio, Sara; Tekiroglu, Serra Sinem; Guerini, Marco in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 209 N. Eighth Street, Stroudsburg PA 18360, USA: Association for Computational Linguistics, 2022, p. 8031-8049. Proceedings of: The Conference on Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, United Arab Emirates, 7th-11th December 2022. - Publication URL

Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Study
Tekiroglu, Serra Sinem; Bonaldi, Helena; Fanton, Margherita; Guerini, Marco in Findings of the Association for Computational Linguistics: ACL 2022, 209 N. Eighth Street, Stroudsburg PA 18360, USA: Association for Computational Linguistics, 2022, p. 3099-3114. - ISBN: 978-1-955917-25-4. Proceedings of: Annual Meeting of the Association for Computational Linguistics (ACL), Dublin, Ireland, 22nd-27th May, 2022. - Publication URL . - DOI: 10.18653/v1/2022.findings-acl.245

Efficient yet Competitive Speech Translation: FBK@IWSLT2022
Gaido, Marco; Papi, Sara; Fucci, Dennis; Fiameni, Giuseppe; Negri, Matteo; Turchi, Marco in Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), Dublin, Ireland (in-person and online): Association for Computational Linguistics (ACL), 2022, p. 177-189. - ISBN: 978-1-955917-41-4. Proceedings of: 19th International Conference on Spoken Language Translation, IWSLT 2022, Dublin, Ireland (in-person and online), 26-27, May 2022. - Publication URL . - DOI: 10.18653/v1/2022.iwslt-1.13

What can Speech and Language Tell us About the Working Alliance in Psychotherapy
Peter Bayer, Sebastian; Roccabruna, Gabriel; Chowdhury, Shammur Absar; Ciulli, Tommaso; Danieli, Morena; Riedhammer, Korbinian; Riccardi, Giuseppe in Proceedings Interspeech 2022, BAIXAS, FRANCE: International Speech Communication Association, 2022, p. 2443-2447. Proceedings of: Interspeech 2022, Incheon, Korea, 18th September-22nd September 2022. - Publication URL . - DOI: 10.21437/Interspeech.2022-347

Annotation of Valence Unfolding in Spoken Personal Narratives
Tammewar, Aniruddha Uttam; Braun, Franziska; Roccabruna, Gabriel; Bayerl, Sebastian; Riedhammer, Korbinian; Riccardi, Giuseppe in Proceedings of the Thirteenth Language Resources and Evaluation Conference, Marsiglia, Francia: European Language Resources Association, 2022, p. 7004-7013. Proceedings of: LREC, Marsiglia, Francia, 20th-25th June 2022. - Publication URL

Multi-source Multi-domain Sentiment Analysis with BERT-based Models
Roccabruna, G.; Azzolin, Steve; Riccardi, G. in European Language Resources Association, Parigi: European Language Resources Association, 2022, p. 581-589. - ISBN: 979-10-95546-72-6. Proceedings of: LREC, Marsiglia, 20th June - 25th June 2022. - Publication URL
[other topics: Deep and Structured Machine Learning

Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech
Fanton, Margherita; Bonaldi, Helena; Tekiroglu, Serra Sinem; Guerini, Marco in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 209 N. Eighth Street, Stroudsburg PA 18360, USA: Association for Computational Linguistics, 2021, p. 3226-3240. Proceedings of: Annual Meeting of the Association for Computational Linguistics (ACL), online, 2nd -5th August 2021. - Publication URL . - DOI: 10.18653/v1/2021.acl-long.250

Multifunctional ISO standard Dialogue Act tagging in Italian
Roccabruna, G.; Cervone, A.; Riccardi, G. in Seventh Italian Conference on Computational Linguistics, Aachen: CEUR Workshop Proceedings, 2021. Proceedings of: Clicit, Bologna, 1-3 March 2021

Multifunctional ISO standard Dialogue Act tagging in Italian
Roccabruna, G.; Cervone, A.; Riccardi, G. in Seventh Italian Conference on Computational Linguistics, Aachen: CEUR Workshop Proceedings, 2020. - (CEUR WORKSHOP PROCEEDINGS). Proceedings of: Clicit, Bologna, 1-3 March 2021. - Publication URL

 

Students

Alghisi, Simones.alghisi [at] unitn.itwebpage
Bonaldi, Helenahelena.bonaldi [at] unitn.itwebpage
Fong, Mei Yue Seraphinameiyueseraphina.fong [at] unitn.itwebpage
Fucci, Dennisdennis.fucci [at] unitn.itwebpage
Gonella, Giacomogiacomo.gonella [at] unitn.itwebpage
Hannan, Abdulabdul.hannan [at] unitn.itwebpage
Hassan, Mirmir.hassan [at] unitn.itwebpage
Kostadinov, Samuelsamuel.kostadinov [at] unitn.itwebpage
Loginova, Olgaolga.loginova [at] unitn.itwebpage
Occhipinti, Danieladaniela.occhipinti [at] unitn.itwebpage
Penzo, Nicolònicolo.penzo [at] unitn.itwebpage
Piergentili, Andreaandrea.piergentili [at] unitn.itwebpage
Rizzoli, Massimomassimo.rizzoli [at] unitn.itwebpage
Roccabruna, Gabrielgabriel.roccabruna [at] unitn.itwebpage
Russo, Danieldaniel.russo [at] unitn.itwebpage
Varella Conti, Linalina.varellaconti [at] unitn.itwebpage