PhD Position F/M Theory for Discrete Diffusion Models

Context

The PhD position will be in the framework of the ERC Starting Grant DYNASTY (Dynamics-Aware Theory of Deep Learning).
The position might include traveling to conferences for paper presentation. Travel expenses will be covered within the limits of the scale in force.

Assignment

Discrete diffusion models have become a leading alternative to autoregressive models for generating text and other categorical data [1, 2]. They generate samples by reversing a Markov chain that progressively corrupts the data, using a score function learned by a neural network [1]. Despite their empirical success, their theoretical understanding remains far less developed than that of their continuous counterparts.
The objective of this PhD is to study the theoretical foundations of discrete diffusion models, and in particular how well they learn and generalize when the score is estimated by a neural network. Building on recent statistical analyses [3, 4] and on generalization bounds for diffusion models [5], the project will investigate how statistical, approximation and optimization errors affect the quality of the generated samples, and will validate the theory through numerical experiments.
References
[1] A. Lou, C. Meng, S. Ermon. Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution. ICML, 2024.
[2] S. S. Sahoo, M. Arriola, Y. Schiff, A. Gokaslan, E. Marroquin, J. T. Chiu, A. Rush, V. Kuleshov. Simple and Effective Masked Diffusion Language Models. NeurIPS, 2024.
[3] Z. Zhang, H. Fu, Z. Yang, M. Wang, T. Zhao, M. Chen. Generalization Bounds for Discrete Diffusion: Statistical Advantage of Masking. ICML, 2026.
[4] L. Blunda, T. Farghly, U. Şimşekli, A. Durmus. Discrete Diffusion Models are Minimax Optimal Under Sparsity. arXiv:2610.19020, 2026.
[5] B. Dupuis, D. Shariatian, M. Haddouche, A. Durmus, U. Şimşekli. Algorithm- and Data-Dependent Generalization Bounds for Diffusion Models. arXiv:2506.03849, 2026.

Main activities

Conduct theoretical research
Conduct experiments for empirical verification
Write scientific articlesDisseminate the scientific work in appropriate venues.

Skills

Technical skills and level required :
Languages : High-level of professional/academic English
Coding skills : Good level of coding in Python and related deep learning libraries