Abstract: Control of chaotic partial differential equations is challenging but valuable, with far-reaching applications in energy systems, economics, fluid dynamics and many other domains. Realistic engineering applications often only admit partial observations of the state and the controller must learn to steer the system towards a desired state using incomplete information. We introduce CHAROT, an attention-based memory architecture designed to augment actor-critic reinforcement learning algorithms and improve their performance in controlling chaotic PDEs using only partial observations. We present numerical experiments for control of the Kuramoto-Sivashinsky equation in chaotic ($0.005 \leq \nu \leq 0.05$) and partially observable regimes. In the most chaotic regime considered, our method outperforms a memoryless controller by $150 $% and an LSTM-augmented controller by $206$%.