Platonic Transformers: A Solid Choice for Equivariance

Transformers lack inductive biases for geometric symmetries common across scientific domains. Existing equivariant methods often sacrifice the efficiency and flexibility that make Transformers so scalable through complex, computationally intensive designs. We introduce the Platonic Transformer to resolve this long- standing trade-off. By defining attention relative to reference frames from Platonic solid symmetry groups, our method induces a principled weight-sharing scheme. This enables combined E(3) equivariance to continuous translations and Platonic symmetries, while preserving the exact architecture and computational cost of a standard Transformer. Platonic Transformers achieve highly competitive results on molecular property prediction and unconditional generation tasks (QM9, OMol25), leveraging geometric constraints at no additional cost.