Sub-seasonal to seasonal (S2S) weather forecasts are an important decision-making tool that informs economical and logistical planning in agriculture,energy management, and disaster mitigation. They are issued on time scalesof weeks to months and differ from short-term weather forecasts in twoimportant ways: (i) the dynamics of the atmosphere on these timescales canbe described only statistically and (ii) these dynamics are characterized bylarge-scale phenomena in both space and time. While deep learning (DL)has shown promising results in short-term weather forecasting, DL-basedS2S forecasts are challenged by comparatively small volumes of availabletraining data and large fluctuations in predictability due to atmosphericconditions. In order to develop more reliable S2S predictions that leveragecurrent advances in DL, we propose to utilize the masked auto-encoder(MAE) framework to learn generic representations of large-scale atmosphericphenomena from high resolution global data. Besides exploring the suitabilityof the learned representations for S2S forecasting, we will also examinewhether they account for climatic phenomena (e.g., the Madden-JulianOscillation) that are known to increase predictability on S2S timescales.