With the rise of large language models (LLMs), increasing research has recognizedtheir risk of leaking personally identifiable information (PII) under maliciousattacks. Although efforts have been made to protect PII in LLMs, existing methodsstruggle to balance privacy protection with maintaining model utility. In this paper,inspired by studies of amnesia in cognitive science, we propose a novel approach,Proactive Privacy Amnesia (PPA), to safeguard PII in LLMs while preserving theirutility. This mechanism works by actively identifying and forgetting key memoriesmost closely associated with PII in sequences, followed by a memory implantingusing suitable substitute memories to maintain the LLM’s functionality. We conductevaluations across multiple models to protect common PII, such as phone numbersand physical addresses, against prevalent PII-targeted attacks, demonstrating thesuperiority of our method compared with other existing defensive techniques. Theresults show that our PPA method completely eliminates the risk of phone numberexposure by 100% and significantly reduces the risk of physical address exposureby 9.8% – 87.6%, all while maintaining comparable model utility performance.