Recently, Neural Fields (NeFs) have emerged as a powerful modelling paradigm to represent discretely-sampled continuous signals. As such, novel work has explored the use of Conditional NeFs to model PDEs, by learning continuous flows in the latent space of the Conditional NeF. Although this approach benefits from favourable properties of neural fields such as grid-agnosticity and space-time-continuous dynamics modelling, it does not make use of important geometric information about the domain of the PDE being modelled -- such as information on symmetries of the PDE -- in favour of modelling flexibility.Instead, we propose a NeF parameterization that preserves geometric information in the latent space of the Conditional NeF: \textit{Equivariant Neural Fields}. Using this representation, we construct a framework for space-time continuous PDE modelling that preserves known symmetries of the PDE. We experimentally validate our model and show it readily generalizes to arbitrary locations, as well as geometric transformations of the initial conditions - where other NeF-based PDE forecasting methods fail.