Diffusion models have emerged as a promising choice for learning robot skills from demonstrations. However, they face three problems: diffusion models are not sample-efficient, data is expensive to collect in robotics, and the space of tasks is combinatorially large. The established method to train diffusion models on skill demonstrations borrow from the literature on image generation, and results in a conditional distribution of robot actions given the visual, proprioceptive and other observations. However, they have little room to accommodate solutions for the aforementioned challenges, in addition to scaling the model size and paired observation-action data. In this work, we propose a novel method for training diffusion models termed ‘Composable Diffusion Guidance’ CoDiG to compositionally learn diffusion policies for robot skills. CoDiG decouples the observation modalities allowing the residual learning of one modality with respect to another. While presenting a more intuitive modeling paradigm, CoDiG also enables the scaling of modalities such as robot motions or visual observations independently. Our preliminary results show that visual CoDiG with motion-priors outperforms the conventional way of learning visuomotor policies using diffusion models on skills where the motion of the robot is heavily dependent on proprioceptive observations. Further experimentation is needed to evaluate the performance and robustness of CoDiG for different observation modalities, and on different classes of skills, such as long-horizon and precise manipulation. More work is also needed to understand how observational priors such as visual or motion pre-training affect the performance of CoDiG in comparison to the existing way of training diffusion models on robot demonstrations.