Complex systems are very hard to predict and control (chaotic dynamics and open-ended outcomes), but the understanding and harnessing of their underlying mechanisms holds great promises to revolutionize many areas of science. Whereas considerable progress has been made at the bench for manipulating/measuring the system activity down to the lowest-level, there is a fundamental gap between knowledge at the low-level and control of the resulting properties at the global scale. Modern tools from machine learning hold great promises to assist humans in mapping and navigating the space of possible outcomes toward novel or hard-to-reach morphological/functional targets. However current methods are largely restricting and biasing the boundary of events that the AI can measure and try to affect. In this talk, I will present how recent computational models of intrinsically motivated learning and exploration can be used to design more open-ended forms of AI "discovery assistant". In particular I will discuss how meta-diversity search, curriculum learning and external guidance (environmental or preference-based) can be key ingredients for shaping the search process. I will show how those ingredients, when implemented in practice, can help solving challenging problems in science. This includes the search of interesting patterns in continuous cellular automata, the study of the origins of sensorimotor agency and the design of novel forms of collective intelligence for AI and biology.