Summary

The current state of the art in Machine Learning (ML) and Statistical Inference heavily relies on principles which defy traditional statistical thinking, such as high overparameterization in deep learning and non-convex optimization on high-dimensional landscapes using stochastic descent. This brings the necessity of investigating previously unexplored statistical regimes where classical results do not apply to better understand these novel and impactful practices. As a result, innovative theoretical approaches have emerged, notably through interdisciplinary collaborations, among which those based on high-dimensional statistics, random matrix theory and mathematical physics figure prominently. This thematic trimester therefore aims at fostering the interaction of researchers from several communities working in this fast-developing and emerging area, as well as providing young researchers and students with opportunities for acquiring knowledge on these new theoretical approaches of ML and statistical inference which will play a major role in the future of these disciplines.

Opening colloquium

An opening colloquium about recent developments and challenges of high-dimensional statistical inference and machine learning.

September, 11th 2024, from 3pm to 5pm (+ cocktail)

Speakers : Andrea Montanari (Stanford U., USA) and Lenka Zdeborová (EPFL, Switzerland) 

More information: link

Thematic school: Optimization & algorithms for high-dimensional machine learning and inference

This thematic school features mini-courses on the study of high-dimensional (random) optimization landscapes, on the dynamics of optimization algorithms in high dimensions, on approximate message passing algorithms, and related topics.

October, 7th to 11th 2024

List of speakers and subjects:

• François Malgouyres - Properties of the Landscape in Neural Network Optimization
• Edouard Pauwels - An Introduction to Optimization for Deep Learning
• Valentina Ros - High-dimensional Optimization Landscapes
• Cynthia Rush - An Introduction to Approximate Message Passing Algorithms
• Cédric Févotte - Majorization-Minimization for Non-Negative Matrix Factorization

Slides: link

Thematic school: Models & methods for high-dimensional machine learning and Inference

This thematic school features mini-courses on tools and techniques for the analysis of high-dimensional models in statistical inference and machine learning.

October, 14th to 18th 2024

List of speakers and subjects:

• Florent Krzakala - Replica Method
• Marc Lelarge - Statistical Physics and Inference
• Zhenyu Liao - Random Matrix Theory Tools for Machine Learning and Inference
• Alexander Wein - Mini-course on Random Tensor Models

 Slides: link

Workshop: Recent developments beyond classical regimes in statistical learning

This workshop is focused on recent results on high-dimensional (supervised and unsupervised) machine learning and statistical inference. It will in particular involve a round-table debate with top experts on this domain about the major open problems on the field and some promising trends and recent developments.

*This is a joint workshop with the CIMI Thematic Semester "Stochastic control and learning for complex networks"

 Slides: link 

November 4th 2024 (9:30 am to 4:30 pm)

•  Bruno Loureiro (ENS, France) - Learning Features with Two-layers Neural Networks, One Step at Time
• Pascal Maillard (IMT, France) - Probing the Transition from Polynomial to Exponential Complexity in Spin Glasses via N-particle Branching Brownian Motions
• Nicolas Macris (EPFL, Switzerland) - Sampling Diffusion Process
• Malik Tiomoko (Huawei) - Enhancing Time Series Forecasting with Random Matrix Theory

November 5th 2024 (9:30 am to 7 pm)

• Christos Thrampoulidis (British Columbia U., Canada) - On the implicit Geometry of Word and Context Embeddings in Next-token Prediction
• Jon Keating (Oxford U., UK) - Some Connexions between Random Matrix Theory and Machine Learning
• Charles Bordenave (IMM, France) - Freeness for Tensors
• Aukosh Jagannath (Waterloo U., Canada) - Effective Dynamics and Spectral Alignment

Short talks : 

• Jad Hamdan (Oxford U., UK) - Graph Expansion of Deep Neural Networks and their Universal Scaling Limits
• Mustapha Maimouni (Mohamed V U., Morocco) - A Hybrid Approach Inspired by Artificial Neural Networks for RFID Network Planning
• O.Duranthon (EPFL, Switzerland) - Generalization in Single-layer Graph Convolutional Network

+ Social cocktail

November 6th 2024 (9:30 am to 4:30 pm)

• Jean Barbier (ICTP, Italy) - Information and Algorithmic Limits in Structured Principal Components Analysis
• Cosme Louart (Hong Kong U., China) - Concentration of the Measure for Machine Learning: an Exploration through Linear Regression
• Subhabrata Sen (Harvard U., USA) - Causal Effect Estimation under Inference Using Mean Field Methods
• Bertrand Lacroix-A-Chez-Toine (KIng's college, UK) - Random Landscape Built by Superposition of Plan Waves in High Dimension

Novembert 7th 2024 (9:30 am to 4:30 pm)

• Inbar Seroussi (Tel-Aviv U., Israel) - Exact Dynamics of Stochastic and Adaptative Optimization in High Dimension with Structured Data
• Giulio Biroli (ENS, France) - Generative AI and Diffusion Models - A Statistical Physics Analysis
• Pierfrancesco Urbani (IPHT, France) - Statistical Physics of Learning in High-dimensional Chaotic Systems
• Aurélien Decelle (UCM, Spain) - How Phase Transitions Shape the Learning of Complex Data in the Restricted Boltzmann Machine 

November 8th 2024 (9:30 am to 3 pm)

• Ludovic Stefan (ENSAI, France) - A Non-backtracking Method for Long Matrix and Tensor Completion
• Franck Iutzeler (IMT, France) - What is the Long-run Distribution of Stochastic Gradient Descent ? A Large Deviations Analysis
• Marc Lelarge (ENS, France) - Combinatorial Optimization with Graph Neural Network: Chaining to Learn the Graph Alignment Problem

Organisation staff: Henrique Goulart (IRIT/Toulouse INP), Vanessa Kientz (CEA List), Xiaoyi Mai (IMT/UT2J),

Mohamed Tamaazousti (CEA List)