This study explores a data-driven approach that combines flood forcing factors from observation and reanalysis datasets, antecedent flood extent maps, and deep learning to forecast daily flood extents in Rwanda. We extend the architecture used in ClimaX (transformer weather and climate foundation model), investigate its pretrained representations for flood forecasting, and compare performance against a U-Net baseline. Our results demonstrate that a ClimaX variant trained from scratch with a linear projection decoder outperforms the U-Net and other ClimaX variants, highlighting its potential as an effective tool for flood extent forecasting. This work underscores the potential of data-driven deep learning models for flood extent forecasting with implications for improving disaster preparedness and flood risk assessment in vulnerable regions.