Bayesian Experimental Design (BED) is a robust model-based framework for optimising experiments but faces significant computational barriers, especially in the setting of inverse problems for partial differential equations (PDEs). In this paper, we propose a novel approach, modelling the joint posterior distribution with an energy-based model, trained on simulation data. Unlike existing simulation-based inference approaches, we leverage implicit neural representations to learn a functional representation of parameters and data. This is used as a resolution-independent plug-and-play surrogate for the posterior, which can be conditioned over any set of design-points, permitting an efficient approach to BED.