Pretraining deep learning models on a large corpus of unlabeled data using selfsupervised learning approaches can be an efficient a mitigation strategy to dealwith the lack of annotated data. We proposed to use a siamese framework forthe pretraining of a convolutional neural network on the Computing in Cardiology2021 dataset therefore making it invariant to ECG lead configuration changes. The obtained representation was then trained and tested on a heartbeat classification task on the MIT BIH Arrhythmia database, and on an external independent set, namely the INCART database. The proposed model reached a median F1 score of 0.89 on the MIT BIH Arrhythmia database comparable to the 0.90 F1 score obtained without pretraining. However, the pretrained model obtained a median F1 score of 0.74 on average over the different leads, compared to 0.53 the model without pretraining. The proposed pretraining approach, leveraging the availability of relatively large database of un-(or weakly)annotated ECG data, allows for the training of more generalizable, lead-agnostic, heartbeat classification models. Such an approach would ensure avoiding overfitting complex deep learning models on the small MIT-BIH arrhythmia database.