Poster
in
Workshop: Workshop on Learning from Time Series for Health
Multi-Modal Contrastive Learning for Online Clinical Time-Series Applications
Fabian Baldenweg · Manuel Burger · Gunnar Ratsch · Rita Kuznetsova
Keywords: [ multi-modal ] [ contrastive learning ] [ zero-shot ] [ Health-Care ] [ Clinical Time-Series ] [ Intensive Care ] [ Online Predictions ] [ Time-Series ]
Electronic Health Record (EHR) datasets from Intensive Care Units (ICU) contain a diverse set of data modalities. While prior works have successfully leveraged multiple modalities in supervised settings, we apply advanced self-supervised multi-modal contrastive learning techniques to ICU data, specifically focusing on clinical notes and time-series for clinically relevant online prediction tasks. We introduce a loss function Multi-Modal Neighborhood Contrastive Loss (MM-NCL), a soft neighborhood function, and showcase the excellent linear probe and zero-shot performance of our approach.