An Introduction to Identifiability in Representation Learning and Causality

• Representations in deep learning; problems in representation learning—the cocktail-party problem, the independent-listeners problem, and interpretability.
• Independent component analysis: linear (identifiability and estimation) and nonlinear (non-identifiability, auxiliary variables, and restricted function classes).
• Identifiability in disentanglement and causal representation learning.
• Representational similarity.
• Identifiability in causal inference and causal discovery.
• Open problems in identifiable representation learning.

A minimum of 75% attendance is required

Teaching methods

Lectures with presentations and in-class discussions, possibly complemented by short exercises.

Assessment methods

Final assessment is based on an individual or small-group project with a short in-class presentation on a topic related to identifiability in representation learning or causal inference.

Bibliography

• Independent Component Analysis, Aapo Hyvärinen, Juha Karhunen, Erkki Oja, Wiley, 2001.
• Nonlinear Independent Component Analysis: Existence and Uniqueness Results, Aapo Hyvärinen and Petteri Pajunen, Neural Networks, 1999.
• Identifiability of latent-variable and structural-equation models: from linear to nonlinear, Aapo Hyvärinen, Ilyes Khemakhem, Ricardo Monti, Annals of the Institute of Statistical Mathematics, 2024.
• Learning Identifiable Representations: Independent Influences and Multiple Views, Luigi Gresele, PhD thesis, University of Tübingen, 2023.
• Identifying latent structures in data, Sébastien Lachapelle, PhD thesis, Université de Montréal, 2024