Abstract
Auscultation of the heart and the electrocardiogram (ECG) are two central components of the cardiac exam. Recent innovations of the stethoscope have enabled the simultaneous acquisition of a high-quality digital acoustic signal and ECG. We present foundation models trained on phonocardiogram (PCG) and ECG data collected from digital stethoscopes during routine clinical practice. We show that these foundation models that are pre-trained on large unlabeled datasets in a self-supervised manner can be fine-tuned for a variety of cardiovascular disease detection tasks. This is the first study that builds foundation models specifically for synchronously captured PCG and ECG data. Our approach is based on the recently developed masked autoencoder framework which we extend to handle multiple signals that are synchronously captured. This paradigm makes it possible to use large capacity models leading to superior performance even though the size of datasets with medical label annotations may be limited.