Harnessing program induction, coupling robustness, and expressivity, by combining some form of symbolic and procedural knowledge appears to be a promising direction towards more open-ended innovation, and extrapolative behavior. Building upon DreamCoder framework (Ellis et al., 2021), we present an unsupervised diversity-oriented neurosymbolic learner: Open-Ended Dreamer (OED). Balancing environmental, language and novelty pressures, OED aims to learn novel, and useful programmatic abstractions. As a first test-bed we experiment with a tower building environment, where we analyze the benefits of library learning, neural guidance, innate priors, or environmental pressures to guide the formation of symbolic knowledge and open-ended program discovery.