As AI systems become more powerful, it is increasingly important that developers be able to strictly enforce desired policies for the systems. Unfortunately, via techniques such as adversarial attacks, it has traditionally been possible to circumvent model policies, allowing bad actors to manipulate LLMs for unintended and potentially harmful purposes. In this talk, I will highlight several recent directions of work that are making progress in addressing these challenges, including methods for robustness to jailbreaks, safety pre-training, and methods for preventing undesirable model distillation. I will additionally highlight some of the areas I believe to be most crucial for future work in the field.

As AI systems become more powerful, it is increasingly important that developers be able to strictly enforce desired policies for the systems. Unfortunately, via techniques such as adversarial attacks, it has traditionally been possible to circumvent model policies, allowing bad actors to manipulate LLMs for unintended and potentially harmful purposes. In this talk, I will highlight several recent directions of work that are making progress in addressing these challenges, including methods for robustness to jailbreaks, safety pre-training, and methods for preventing undesirable model distillation. I will additionally highlight some of the areas I believe to be most crucial for future work in the field.

In this talk, I will present a set of work done at the Beijing Institute of General Artificial Intelligence (BIGAI) and Peking University on AGI, which is also called TongAI as the Chinese character ‘Tong’ means ‘general’ and contains the letters ‘A’, ‘G’ and ‘I’. I will start with introducing a digital agent --- a little girl who lives and learns continuously in simulated diverse physics-realistic environments with multi-physics and social interactions. The little girl with a nickname ‘TongTong’ is driven by her own value system with desires and goals which generates plans and actions. Then I will reveal the framework underneath this self-conscious agent with three interconnected components: Cognitive architecture (C), the potential functions (U) representing skills, and the value functions (V). Then we define various AGI systems as points in this joint (C,U,V) –space. This framework represents a paradigm shift from the popular “data-driven” "large data for small tasks" statistical paradigm which we pioneered at Harvard, Brown and UCLA since the early 1990s, to the “value-driven” "small data for large tasks" paradigm which I have been advocating since 2010. Then I will introduce TongTest as new criteria, benchmarks and test platform for measuring the general intelligence of various …

In this talk, I will present a set of work done at the Beijing Institute of General Artificial Intelligence (BIGAI) and Peking University on AGI, which is also called TongAI as the Chinese character ‘Tong’ means ‘general’ and contains the letters ‘A’, ‘G’ and ‘I’. I will start with introducing a digital agent --- a little girl who lives and learns continuously in simulated diverse physics-realistic environments with multi-physics and social interactions. The little girl with a nickname ‘TongTong’ is driven by her own value system with desires and goals which generates plans and actions. Then I will reveal the framework underneath this self-conscious agent with three interconnected components: Cognitive architecture (C), the potential functions (U) representing skills, and the value functions (V). Then we define various AGI systems as points in this joint (C,U,V) –space. This framework represents a paradigm shift from the popular “data-driven” "large data for small tasks" statistical paradigm which we pioneered at Harvard, Brown and UCLA since the early 1990s, to the “value-driven” "small data for large tasks" paradigm which I have been advocating since 2010. Then I will introduce TongTest as new criteria, benchmarks and test platform for measuring the general intelligence of various …

In this talk, we will try to clarify different levels and mechanisms of intelligence from historical, scientific, mathematical, and computational perspective. From the evolution of intelligence in nature, from phylogenetic, to ontogenetic, to societal, and to artificial intelligence, we will try to shed light on how to understand the true nature of the seemingly dramatic advancements in the technologies of machine intelligence in the past decade. We achieve this goal by developing a principled mathematical framework to explain the practice of deep learning from the perspective of compressive data encoding and decoding. This framework not only reveals true nature hence limitations of the current practice and but also provides principled guidelines to develop more complete and more efficient learning architectures and systems. Eventually, we will clarify the difference and relationship between Knowledge and Intelligence, which may guide us to pursue the goal of developing systems with true intelligence.

In this talk, we will try to clarify different levels and mechanisms of intelligence from historical, scientific, mathematical, and computational perspective. From the evolution of intelligence in nature, from phylogenetic, to ontogenetic, to societal, and to artificial intelligence, we will try to shed light on how to understand the true nature of the seemingly dramatic advancements in the technologies of machine intelligence in the past decade. We achieve this goal by developing a principled mathematical framework to explain the practice of deep learning from the perspective of compressive data encoding and decoding. This framework not only reveals true nature hence limitations of the current practice and but also provides principled guidelines to develop more complete and more efficient learning architectures and systems. Eventually, we will clarify the difference and relationship between Knowledge and Intelligence, which may guide us to pursue the goal of developing systems with true intelligence.

Recent advancements in AI and LLM agents have unlocked powerful new capabilities across a wide range of applications. However, these advancements also bring significant risks that must be addressed. In this talk, I will explore the various risks associated with building and deploying AI and LLM agents and discuss approaches to mitigate them. I will also examine how frontier AI and LLM agents could be misused, particularly in cyber security attacks, and how they may reshape the cyber security landscape. Ensuring a safe AI future demands a sociotechnical approach. I will outline our recent proposal for a science- and evidence-based AI policy, highlighting key priorities to deepen our understanding of AI risks, develop effective mitigation approaches, and guide the development of robust AI policies.

Training large language models has become a defining pursuit in modern machine learning—one that is almost entirely led by industry, fueled by massive computational resources and guided by scaling laws that reward ever-larger models and datasets. For academic researchers, participating in this space can feel out of reach. The barriers—limited compute, infrastructure, and access to proprietary data—are real and growing. Still, I believe academia has an essential role to play. Even with constraints, there are important scientific questions and meaningful opportunities that academic research is uniquely positioned to tackle. By engaging with the training process itself, we can deepen our understanding of language models and develop novel and efficient approaches that complement large-scale efforts. In this talk, I’ll share my lab’s research efforts over the past two years in both pre-training and post-training of language models under an academic budget. Our work has aimed to better understand training dynamics, innovate within limitations, and release artifacts that benefit the broader research community. I’ll also highlight three areas where academic researchers can make significant contributions: (1) developing small but capable models, (2) understanding and improving training data, and (3) advancing post-training methods on top of open-weight models. My hope is to encourage …

Training large language models has become a defining pursuit in modern machine learning—one that is almost entirely led by industry, fueled by massive computational resources and guided by scaling laws that reward ever-larger models and datasets. For academic researchers, participating in this space can feel out of reach. The barriers—limited compute, infrastructure, and access to proprietary data—are real and growing. Still, I believe academia has an essential role to play. Even with constraints, there are important scientific questions and meaningful opportunities that academic research is uniquely positioned to tackle. By engaging with the training process itself, we can deepen our understanding of language models and develop novel and efficient approaches that complement large-scale efforts. In this talk, I’ll share my lab’s research efforts over the past two years in both pre-training and post-training of language models under an academic budget. Our work has aimed to better understand training dynamics, innovate within limitations, and release artifacts that benefit the broader research community. I’ll also highlight three areas where academic researchers can make significant contributions: (1) developing small but capable models, (2) understanding and improving training data, and (3) advancing post-training methods on top of open-weight models. My hope is to encourage …

The pursuit of Artificial Superintelligence (ASI) requires a shift from narrow objective optimization towards embracing Open-Endedness—a research paradigm, pioneered in AI by Stanley, Lehman and Clune, that is focused on systems that generate endless sequences of novel but learnable artifacts. In this talk, I will present our work on large-scale foundation world models that can generate a wide variety of diverse environments that can in turn be used to train more general and robust agents. Furthermore, I will argue that the connection between Open-Endedness and Foundation Models points towards automating innovation itself. This convergence is already yielding practical results, enabling self-referential self-improvement loops for automated prompt engineering, automated red-teaming, and AI debate in Large Language Models, and it hints at a future where AI drives its own discoveries.

The pursuit of Artificial Superintelligence (ASI) requires a shift from narrow objective optimization towards embracing Open-Endedness—a research paradigm, pioneered in AI by Stanley, Lehman and Clune, that is focused on systems that generate endless sequences of novel but learnable artifacts. In this talk, I will present our work on large-scale foundation world models that can generate a wide variety of diverse environments that can in turn be used to train more general and robust agents. Furthermore, I will argue that the connection between Open-Endedness and Foundation Models points towards automating innovation itself. This convergence is already yielding practical results, enabling self-referential self-improvement loops for automated prompt engineering, automated red-teaming, and AI debate in Large Language Models, and it hints at a future where AI drives its own discoveries.