WHAT TO SHOW AND HOW TO SHOW: STRUCTURED FAILURE-GUIDED SUPERVISION FOR T2I ALIGNMENT

Feedback-driven alignment has emerged as a primary mechanism for improving the controllability and reliability of generative models. While existing approaches focus on improving behavioral alignment through preference optimization, relatively little attention has been paid to how supervision design shapes the underlying representations that give rise to aligned behavior. This challenge is particularly pronounced in diffusion-based text-to-image (T2I) models, where errors are spatially distributed and arise from entangled visual representations, making it difficult to provide targeted corrective feedback analogous to language-based supervision. In this work, we propose F-GPS (Failure-Guided Preference Supervision), a framework that leverages structured and recurring failure patterns of diffusion models to construct targeted supervision signals for alignment. F-GPS identifies systematic failure modes and atomizes them into composable visual programs that synthesize controlled failure instances while preserving semantic content. By exposing failure-relevant ambiguities during training, the proposed approach encourages representations that better separate human-relevant visual distinctions without requiring large-scale human annotation or uncontrolled model sampling. We evaluate F-GPS on visual text rendering, a setting where failures are systematic and symbol-sensitive, and demonstrate that failure-guided supervision leads to improved prompt adherence and stronger alignment in text-sensitive image generation. Our results suggest that explicitly modeling structured failures provides an effective pathway for improving representational alignment in diffusion models through supervision design.