Advanced Multimodal Learning

• Overview of Multimodal Machine Learning
  - Key challenges: representation, alignment, fusion, reasoning, and generation
  - Review of recent trends and research directions
• Multimodal Applications & Research
  - Overview of prominent applications: vision-language models, audio-visual systems and robotics
• Fusion of representations, co-learning and alignment
  - Cross-modal Interactions and Multimodal Fusion
  - Co-learning of representations
  - Multimodal Alignment and attention mechanisms
• Distillation and learning with missing modalities
• Multimodal Generative Models

A minimum of 75% attendance is required.

Teaching methods

Lectures, readings and practical sessions.

Assessment methods

Presentation and discussion of a recent state-of-the-art paper on multimodal learning

Bibliography

https://cmu-multicomp-lab.github.io/mmml-course/fall2023/
https://cmu-multicomp-lab.github.io/mmml-tutorial/icml2023/