Secure aggregation is a popular protocol for privacy-aware model aggregation in federated learning. However, due to its large communication overhead, users with scarce wireless resources are unable to participate in the protocol as much as users with better wireless conditions, which can lead to significant bias against users from underserved communities. Towards addressing this challenge, in this work we propose a communication-efficient gradient sparsification technique for secure aggregation, where the server learns the aggregate of sparsified local gradients from a large number of users, without having access to the individual local gradients. Through large-scale distributed experiments with up to 100 users, we demonstrate up to 27x reduction in the communication overhead, and up to 8x speed up in the wall clock training time, compared to conventional secure aggregation.