On Trojan Signatures in Large Language Models of Code

Trojan signatures, as described by Fields et al. (2021), are noticeable differencesin the distribution of the trojaned class parameters (weights) and the non-trojanedclass parameters of the trojaned model, that can be used to detect the trojaned model.Fields et al. (2021) found trojan signatures in computer vision classification taskswith image models, such as, Resnet, WideResnet, Densenet, and VGG. In thispaper, we investigate such signatures in the classifier layer parameters of largelanguage models of source code. Our results suggest that trojan signatures couldnot generalize to LLMs of code. We found that trojaned code models are stubborn,even when the models were poisoned under more explicit settings (finetuned withpre-trained weights frozen). We analyzed nine trojaned models for two binaryclassification tasks: clone and defect detection. To the best of our knowledge, thisis the first work to examine weight-based trojan signature revelation techniquesfor large-language models of code and furthermore to demonstrate that detectingtrojans only from the weights in such models is a hard problem.