Skip to yearly menu bar Skip to main content


Multi-Source Diffusion Models for Simultaneous Music Generation and Separation

Giorgio Mariani · Irene Tallini · Emilian Postolache · Michele Mancusi · Luca Cosmo · Emanuele RodolĂ 

Halle A 8 - 9
[ ] [ Visit Oral 6A ]
Thu 9 May 6:45 a.m. — 7 a.m. PDT


In this work, we define a diffusion-based generative model capable of both music generation and source separation by learning the score of the joint probability density of sources sharing a context. Alongside the classic total inference tasks (i.e., generating a mixture, separating the sources), we also introduce and experiment on the partial generation task of source imputation, where we generate a subset of the sources given the others (e.g., play a piano track that goes well with the drums). Additionally, we introduce a novel inference method for the separation task based on Dirac likelihood functions. We train our model on Slakh2100, a standard dataset for musical source separation, provide qualitative results in the generation settings, and showcase competitive quantitative results in the source separation setting. Our method is the first example of a single model that can handle both generation and separation tasks, thus representing a step toward general audio models.

Chat is not available.