Position: AI Development Should Prioritize Cognitive Security
Abstract
Generative AI systems designed to influence human beliefs and actions are becoming increasingly pervasive, raising concerns about \textit{cognitive security}--the protection of human cognitive processes from hazardous influence. Recent advances have only amplified the cognitive risks of AI technologies, leading institutions worldwide to identify cognitive security as an emerging and urgent governance challenge. However, research on such cognitive impacts remains ad hoc and fragmented across the literature, and the field lacks a framework to address them. In this paper, we argue that generative AI research and development should prioritize cognitive security. First, we track and categorize state-of-the-art capabilities for cognitive influence in generative AI systems. Then, we outline a roadmap for advancing cognitive security research within AI research by (1) proposing attack and defense mechanisms to formalize threat models, expose vulnerabilities, and evaluate countermeasures and (2) defining metrics to measure cognitive impacts.