BackFed: A Standardized and Efficient Benchmark Framework for Backdoor Attacks in Federated Learning

Research on backdoor attacks in Federated Learning (FL) has accelerated in recent years, with new attacks and defenses continually proposed in an escalating arms race. However, the evaluation of these methods remains neither standardized nor reliable. First, there are severe inconsistencies in the experimental settings across studies, and many rely on unrealistic threat models. Second, our code review uncovers semantic bugs in the official codebases of several attacks that artificially inflate their reported performance. These issues raise fundamental questions about whether the proposed attacks are truly effective and realistic. We introduce BackFed, a benchmark designed to standardize FL backdoor evaluation by unifying attacks and defenses under a common evaluation framework. Our benchmark reveals critical limitations of existing attacks and defenses. For example, most backdoor attacks require excessive training time and computation, making them vulnerable to server-enforced time constraints. Meanwhile, some defenses suffer from severe accuracy degradation or aggregation overhead that limits their applicability. Under standardized settings, several popular methods achieve limited performance, which challenges their previous efficacy claims. We establish BackFed as a comprehensive benchmark framework that promotes a more reliable evaluation of backdoor attacks in FL.