Today's text-to-image generation technologies have revolutionized the creation of realistic and high-quality images, but they often struggle with the ambiguities in user prompts. To address this, we introduce TDRI: Two-Phase Dialogue Refinement and Co-Adaptation for Interactive Image Generation, a framework designed to enhance iterative image generation through multi-turn dialogues. The system operates in two phases: an initial generation phase that processes user prompts to create base images, and an interactive refinement phase that adapts and optimizes images based on user feedback. Our framework ensures generated outputs continuously adapt to user preferences through iterative dialogue and optimization. Experiments validate that TDRI improves user experience and efficiency, generating high-quality images with fewer iterations, thus streamlining the creative process in various design applications.