Language, Speech and Interaction | Doctoral Program - Information Engineering and Computer Science

Language, Speech and Interaction

Our main areas of interest are speech, vision and language processing, machine learning and interaction.

The purpose of our research is to investigate how humans process speech, language and vision, and to design mathematical models for automatic processing that can be used with communicating machines.

Our studies rely on computational models from batch or on-line examples and provide predictive models for data classification, automatic sequence labelling and structure inference.

Another research area the group is involved in is human-centered communication systems. We investigate models of interactions in any ICT scenario, such as speech-to-speech, speech-to-web and multimodal interactions. Our focus is both on the computational aspect of the interaction models as well as on the usability of user interfaces.

Finally, we also devote our attention to collaborative systems and web architecture. We are active in the design of software architecture for advanced web and multimedia collaborative systems.

 

Publications

39 publications for 14 currently enrolled students

Are LLMs Robust for Spoken Dialogues?
Mousavi, Seyed Mahed; Roccabruna, Gabriel; Alghisi, Simone; Rizzoli, Massimo; Ravanelli, Mirco; Riccardi, Giuseppe in Proceedings of the 14th International Workshop on Spoken Dialogue Systems Technology, sapporo, Japan: 14th International Workshop on Spoken Dialogue Systems Technology, 2024. Proceedings of: IWSDS2024, JAPAN, 04/03/2024

Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challenges
Piergentili, Andrea; Fucci, Dennis; Savoldi, Beatrice; Bentivogli, Luisa; Negri, Matteo in Proceedings of the first Workshop on Gender-Inclusive Translation Technologies, Tampere: ACL anthology, 2023. Proceedings of: GITT, Tampere, 15/06/2023. - Publication URL

Self-building Neural Networks
Ferigo, Andrea; Iacca, Giovanni in GECCO '23 Companion: Proceedings of the Companion Conference on Genetic and Evolutionary Computation, New York: ACM, 2023, p. 643-646. - ISBN: 979-8-4007-0120-7. Proceedings of: GECCO '23, Lisbon, 15th-19th July. - Publication URL . - DOI: 10.1145/3583133.3590531
[other topics: Machine Learning and Intelligent Optimization

A Population-Based Approach for Multi-Agent Interpretable Reinforcement Learning
Crespi, Marco; Ferigo, Andrea; Custode, Leonardo Lucio; Iacca, Giovanni in APPLIED SOFT COMPUTING, v. 2023, 147, (2023), p. 11075801-11075812. - Publication URL . - DOI: 10.1016/j.asoc.2023.110758
[other topics: Machine Learning and Intelligent Optimization

Interactive Neural Painting
Peruzzo, E. lia; Menapace, Willi; Goel, Vidit; Arrigoni, Federica; Tang, Hao; Xu, Xingqian; Chopikyan, Arman; Orlov, Nikita; Hu, Yuxiao; Shi, Humphrey; Sebe, Nicu; Ricci, Elisa in COMPUTER VISION AND IMAGE UNDERSTANDING, v. 235, (2023), p. 10377801-10377810. - Publication URL . - DOI: 10.1016/j.cviu.2023.103778
[other topics: Machine Learning and Intelligent Optimization

Understanding Emotion Valence is a Joint Deep Learning Task
Roccabruna, Gabriel; Mousavi, Seyed Mahed; Riccardi, Giuseppe in Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, Toronto, Canada: Association for Computational Linguistics, 2023, p. 85-95. - ISBN: 978-1-959429-87-6. Proceedings of: ACL, Toronto, Canada, 9th-14th July 2023. - Publication URL

Let's Give a Voice to Conversational Agents in Virtual Reality
Yin, Michele; Roccabruna, Gabriel; Azad, Abhinav; Riccardi, Giuseppe in Proc. INTERSPEECH 2023, Dublin, Ireland: International Speech Communication Association ISCA, 2023, p. 5247-5248. Proceedings of: Interspeech 2023, Dublin, Ireland, 20th August-24th August 2022. - Publication URL

What{'}s New? Identifying the Unfolding of New Events in a Narrative
Mousavi, Seyed Mahed; Tanaka, Shohei; Roccabruna, Gabriel; Yoshino, Koichiro; Nakamura, Satoshi; Riccardi, Giuseppe in Proceedings of the The 5th Workshop on Narrative Understanding, Association for Computational Linguistics: Association for Computational Linguistics, 2023, p. 1-10. Proceedings of: 5th Workshop on Narrative Understanding, toronto canada, July 14, 2023. - Publication URL . - DOI: 10.18653/v1/2023.wnu-1.1

DH-FBK at HODI: Multi-Task Learning with Classifier Ensemble Agreement, Oversampling and Synthetic Data
Leonardelli, E.; Casula, C. in CEUR Workshop Proceedings, Parma: CEUR-WS, 2023. - (CEUR WORKSHOP PROCEEDINGS). Proceedings of: 8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop, EVALITA 2023, ita, 2023
[other topics: Deep and Structured Machine Learning

Quality–diversity optimization of decision trees for interpretable reinforcement learning
Ferigo, Andrea; Custode, Leonardo Lucio; Iacca, Giovanni in NEURAL COMPUTING & APPLICATIONS, v. 2023, (2023). - Publication URL . - DOI: 10.1007/s00521-023-09124-5
[other topics: Machine Learning and Intelligent Optimization

Benchmarking the Generation of Fact Checking Explanations
Russo, Daniel; Tekiroğlu, Serra Sinem; Guerini, Marco in TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, v. 11, (2023), p. 1250-1264. - Publication URL . - DOI: 10.1162/tacl_a_00601

PoliticIT at EVALITA 2023: Overview of the Political Ideology Detection in Italian Texts Task
Russo, D.; Jimenez-Zafra, S. M.; Garcia-Diaz, J. A.; Caselli, T.; Guerini, M.; Alfonso Urena-Lopez, L.; Valencia-Garcia, R. in CEUR Workshop Proceedings, Aachen: CEUR-WS, 2023. - (CEUR WORKSHOP PROCEEDINGS). Proceedings of: 8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop, EVALITA 2023, Parma, Italy, 7–8 September 2023. - Publication URL

Weigh Your Own Words: Improving Hate Speech Counter Narrative Generation via Attention Regularization
Bonaldi, Helena; Attanasio, Giuseppe; Nozza, Debora; Guerini, Marco in Proceedings of the 1st Workshop on CounterSpeech for Online Abuse, 317 Sidney Baker Street S., Suite 400-134, Kerrville, TX 78028, USA: Association for Computational Linguistics, 2023. Proceedings of: 1st Workshop on CounterSpeech for Online Abuse (CS4OA), Prague, Czechia, 11th Septemper 2023. - Publication URL

Countering Misinformation via Emotional Response Generation
Russo, Daniel; Kaszefski-Yaschuk, Shane; Staiano, Jacopo; Guerini, Marco in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 209 N. Eighth Street, Stroudsburg PA 18360, USA: Association for Computational Linguistics, 2023, p. 11476-11492. Proceedings of: The 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, 6th-10th December 2023. - Publication URL . - DOI: 10.18653/v1/2023.emnlp-main.703

Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection
Fucci, Dennis; Gaido, Marco; Papi, Sara; Cettolo, Mauro; Negri, Matteo; Bentivogli, Luisa in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore: Association for Computational Linguistics, 2023. Proceedings of: EMNLP 2023, Singapore, December 2023. - DOI: 10.18653/v1/2023.emnlp-main.705
[other topics: Deep and Structured Machine Learning

Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpus
Piergentili, Andrea; Savoldi, Beatrice; Fucci, Dennis; Negri, Matteo; Bentivogli, Luisa in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore: Association for Computational Linguistics, 2023. Proceedings of: Empirical Methods in Natural Language Processing, Singapore, 6 –10, December, 2023. - DOI: 10.18653/v1/2023.emnlp-main.873

An Investigation of the Combination of Rehearsal and Knowledge Distillation in Continual Learning for Spoken Language Understanding
Cappellazzo, Umberto; Falavigna, Daniele; Brutti, Alessio in Interspeech, Dublin: ISCA, 2023. Proceedings of: Interspeech, Dublin, 20th August-24th August

Sequence-Level Knowledge Distillation for Class-Incremental End-to-End Spoken Language Understanding
Cappellazzo, Umberto; Yang, Muqiao; Falavigna, Daniele; Brutti, Alessio in Interspeech, Dublin: ISCA, 2023. Proceedings of: Interspeech, Dublin, 20th August-24th August. - DOI: 10.21437/Interspeech.2023-242

PRODIGy: a PROfile-based DIalogue Generation dataset
Occhipinti, Daniela; Tekiroglu, Serra Sinem; Guerini, Marco 2023

FedEdge: Federated Learning with Docker and Kubernetes for Scalable and Efficient Edge Computing
Hassan, Mir; Leonardo, Lucio; Yildirim, Sinan; Iacca, Giovanni in 1st International Workshop on Machine Learning for Autonomic System Operation in the Device-Edge-Cloud Continuum (MLSysOps 2023), New York, NY, USA: ACM, 2023. Proceedings of: The International Conference on Embedded Wireless Systems and Networks (EWSN 2023), Rende, Italy, 25th September-27th September 2023
[other topics: Machine Learning and Intelligent Optimization

Quality Diversity Evolutionary Learning of Decision Trees
Ferigo, Andrea; Custode, Leonardo Lucio; Iacca, Giovanni in SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing, New York: ACM, 2023. Proceedings of: SAC, Tallin, 27th March-31st March 2023. - DOI: 10.1145/3555776.3577591
[other topics: Machine Learning and Intelligent Optimization

Evaluation of Response Generation Models: Shouldn't It Be Shareable and Replicable?
Mousavi, Seyed Mahed; Roccabruna, Gabriel; Lorandi, Michela; Caldarella, Simone; Riccardi, Giuseppe in Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), Association for Computational Linguistics: Association for Computational Linguistics, 2022, p. 136-147. Proceedings of: EMNLP 22, abu dhabi, 5 december 2022. - Publication URL

Optimizing the Sensory Apparatus of Voxel-Based Soft Robots Through Evolution and Babbling
Ferigo, Andrea; Medvet, Eric; Iacca, Giovanni in SN COMPUTER SCIENCE, v. 3, n. 2 (2022), p. 10901-10917. - Publication URL . - DOI: 10.1007/s42979-021-00987-w
[other topics: Machine Learning and Intelligent Optimization

Evolving Hebbian Learning Rules in Voxel-based Soft Robots
Ferigo, Andrea; Iacca, Giovanni; Medvet, Eric; Pigozzi, Federico in IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS, v. 2022, (2022), p. 1-11. - Publication URL . - DOI: 10.1109/TCDS.2022.3226556
[other topics: Machine Learning and Intelligent Optimization

Evolutionary Optimization of Spiking Neural P Systems for Remaining Useful Life Prediction
Custode, Leonardo Lucio; Mo, Hyunho; Ferigo, Andrea; Iacca, Giovanni in ALGORITHMS, v. 15, n. 3 (2022), p. 9801-9818. - Publication URL . - DOI: 10.3390/a15030098
[other topics: Machine Learning and Intelligent Optimization

Can Emotion Carriers Explain Automatic Sentiment Prediction? A Study on Personal Narratives
Mousavi, M.; Roccabruna, G.; Tammewar, A.; Azzolin, Steve; Riccardi, G. in Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, Dublin: ACL, 2022. Proceedings of: Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, Dublin, 23rd - 27th May 2022. - Publication URL

Multi-source Multi-domain Sentiment Analysis with BERT-based Models
Roccabruna, G.; Azzolin, Steve; Riccardi, G. in European Language Resources Association, Parigi: European Language Resources Association, 2022, p. 581-589. - ISBN: 979-10-95546-72-6. Proceedings of: LREC, Marsiglia, 20th June - 25th June 2022. - Publication URL

On the Entanglement between Evolvability and Fitness: an Experimental Study on Voxel-based Soft Robots
Ferigo, Andrea; Soros, L. B.; Medvet, Eric; Iacca, Giovanni in Proceedings of the ALIFE 2022: The 2022 Conference on Artificial Life, Online: ASME, 2022, p. [1-10]. - (ALIFE: PROCEEDINGS OF THE ARTIFICIAL LIFE CONFERENCE). Proceedings of: ALIFE 2022, Online, 18th July-22nd July 2022. - Publication URL . - DOI: 10.1162/isal_a_00493
[other topics: Machine Learning and Intelligent Optimization

Annotation of Valence Unfolding in Spoken Personal Narratives
Tammewar, Aniruddha Uttam; Braun, Franziska; Roccabruna, Gabriel; Bayerl, Sebastian; Riedhammer, Korbinian; Riccardi, Giuseppe in Proceedings of the Thirteenth Language Resources and Evaluation Conference, Marsiglia, Francia: European Language Resources Association, 2022, p. 7004-7013. Proceedings of: LREC, Marsiglia, Francia, 20th-25th June 2022. - Publication URL

What can Speech and Language Tell us About the Working Alliance in Psychotherapy
Peter Bayer, Sebastian; Roccabruna, Gabriel; Chowdhury, Shammur Absar; Ciulli, Tommaso; Danieli, Morena; Riedhammer, Korbinian; Riccardi, Giuseppe in Proceedings Interspeech 2022, BAIXAS, FRANCE: International Speech Communication Association, 2022, p. 2443-2447. Proceedings of: Interspeech 2022, Incheon, Korea, 18th September-22nd September 2022. - Publication URL . - DOI: 10.21437/Interspeech.2022-347

Efficient yet Competitive Speech Translation: FBK@IWSLT2022
Gaido, Marco; Papi, Sara; Fucci, Dennis; Fiameni, Giuseppe; Negri, Matteo; Turchi, Marco in Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), Dublin, Ireland (in-person and online): Association for Computational Linguistics (ACL), 2022, p. 177-189. - ISBN: 978-1-955917-41-4. Proceedings of: 19th International Conference on Spoken Language Translation, IWSLT 2022, Dublin, Ireland (in-person and online), 26-27, May 2022. - Publication URL . - DOI: 10.18653/v1/2022.iwslt-1.13
[other topics: Deep and Structured Machine Learning

Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Study
Tekiroglu, Serra Sinem; Bonaldi, Helena; Fanton, Margherita; Guerini, Marco in Findings of the Association for Computational Linguistics: ACL 2022, 209 N. Eighth Street, Stroudsburg PA 18360, USA: Association for Computational Linguistics, 2022, p. 3099-3114. - ISBN: 978-1-955917-25-4. Proceedings of: Annual Meeting of the Association for Computational Linguistics (ACL), Dublin, Ireland, 22nd-27th May, 2022. - Publication URL . - DOI: 10.18653/v1/2022.findings-acl.245

Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering
Bonaldi, Helena; Dellantonio, Sara; Tekiroglu, Serra Sinem; Guerini, Marco in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 209 N. Eighth Street, Stroudsburg PA 18360, USA: Association for Computational Linguistics, 2022, p. 8031-8049. Proceedings of: The Conference on Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, United Arab Emirates, 7th-11th December 2022. - Publication URL

Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech
Fanton, Margherita; Bonaldi, Helena; Tekiroglu, Serra Sinem; Guerini, Marco in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 209 N. Eighth Street, Stroudsburg PA 18360, USA: Association for Computational Linguistics, 2021, p. 3226-3240. Proceedings of: Annual Meeting of the Association for Computational Linguistics (ACL), online, 2nd -5th August 2021. - Publication URL . - DOI: 10.18653/v1/2021.acl-long.250

Exploiting contextualized word representations to profile haters on Twitter
Ceron, T.; Casula, C. in CEUR Workshop Proceedings, Aachen: CEUR-WS, 2021, p. 1871-1882. - (CEUR WORKSHOP PROCEEDINGS). Proceedings of: 2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021, Online, Bucharest, 21st Sept- 24th Sept 2021. - Publication URL
[other topics: Deep and Structured Machine Learning

Beyond Body Shape and Brain: Evolving the Sensory Apparatus of Voxel-Based Soft Robots
Ferigo, A.; Iacca, G.; Medvet, E. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Verlag: Springer Science and Business Media Deutschland GmbH, 2021, p. 210-226. - (LECTURE NOTES IN ARTIFICIAL INTELLIGENCE). - ISBN: 978-3-030-72698-0. Proceedings of: 24th International Conference on the Applications of Evolutionary Computation, EvoApplications 2021 held as Part of EvoStar 2021, online, 2021. - Publication URL . - DOI: 10.1007/978-3-030-72699-7_14
[other topics: Machine Learning and Intelligent Optimization

Hate speech detection with machine-translated data: The role of annotation scheme, class imbalance and undersampling
Casula, C.; Tonelli, S. in CEUR Workshop Proceedings, Bologna: CEUR-WS, 2020. - (CEUR WORKSHOP PROCEEDINGS). Proceedings of: 7th Italian Conference on Computational Linguistics, CLiC-it 2020, Bologna, 1 March - 3 March 2021
[other topics: Deep and Structured Machine Learning

A GPU-enabled compact genetic algorithm for very large-scale optimization problems
Ferigo, A.; Iacca, G. in MATHEMATICS, v. 8, n. 5 (2020), p. 758. - DOI: 10.3390/MATH8050758
[other topics: Machine Learning and Intelligent Optimization

Multifunctional ISO standard Dialogue Act tagging in Italian
Roccabruna, G.; Cervone, A.; Riccardi, G. in Seventh Italian Conference on Computational Linguistics, Aachen: CEUR Workshop Proceedings, 2020. - (CEUR WORKSHOP PROCEEDINGS). Proceedings of: Clicit, Bologna, 1-3 March 2021. - Publication URL

 

Students

Bonaldi, Helenahelena.bonaldi [at] unitn.itwebpage
Cappellazzo, Umbertoumberto.cappellazzo [at] unitn.itwebpage
Casula, Camillacamilla.casula [at] unitn.itwebpage
Ferigo, Andreaandrea.ferigo [at] unitn.itwebpage
Fucci, Dennisdennis.fucci [at] unitn.itwebpage
Hannan, Abdulabdul.hannan [at] unitn.itwebpage
Hassan, Mirmir.hassan [at] unitn.itwebpage
Loginova, Olgaolga.loginova [at] unitn.itwebpage
Occhipinti, Danieladaniela.occhipinti [at] unitn.itwebpage
Penzo, Nicolònicolo.penzo [at] unitn.itwebpage
Peruzzo, Eliaelia.peruzzo [at] unitn.itwebpage
Piergentili, Andreaandrea.piergentili [at] unitn.itwebpage
Roccabruna, Gabrielgabriel.roccabruna [at] unitn.itwebpage
Russo, Danieldaniel.russo [at] unitn.itwebpage