Steering Fine Tuning with Targeted Concept Ablation

Models often learn unintended behaviors during fine-tuning, such as adopting spurious correlations present in training data. We present a novel technique for controlling what models learn during fine-tuning by identifying and ablating specific sparse autoencoder latents that represent undesired concepts. Our approach steers models toward intended generalizations even in cases where multiple policies correctly fit the training data. We evaluate our method on two tasks: a gender bias task containing spurious correlations and a multiple choice task where models must learn to focus on intended questions while ignoring others. Our technique successfully guides models to learn the intended generalization on 13 out of 17 cases, significantly outperforming random ablation baselines. The results demonstrate a practical application of interpretability techniques for ensuring the safe and reliable deployment of AI models.