Intelligent medical systems capable of capturing and interpreting sensor data and providing context-aware assistance promise to revolutionize interventional healthcare. However, flaws in common practice as well as a lack of standardization in the field of medical image analysis substantially impede successful adoption of modern ML research into clinical use. Drawing from research within my own group as well as large international expert consortia, I will discuss pervasive shortcomings in current medical imaging procedures -- specifically focusing on the three core aspects of image acquisition, image analysis, and algorithm validation – as well as present possible solutions. My talk will showcase the importance of systematically professionalizing every aspect of the medical imaging pipeline to the end of readying intelligent imaging systems for clinical use.

Explainability of machine learning methods is of fundamental importance in healthcare to calibrate trust. A large branch of explainable machine learning uses tools linked to the Shapley value, which have nonetheless been found difficult to interpret and potentially misleading. Taking multiclass classification as a refer- ence task, we argue that a critical issue in these methods is that they disregard the structure of the model outputs. We develop the Categorical Shapley value as a theoretically-grounded method to explain the output of multiclass classifiers, in terms of transition (or flipping) probabilities across classes. We demonstrate on a case study composed of three example scenarios for pneumonia detection and subtyping using X-ray images.

Rule-based surrogate models are an effective and interpretable way to approximate a Deep Neural Network's (DNN) decision boundaries, allowing humans to easily understand deep learning models. Current state-of-the-art decompositional methods, which are those that consider the DNN's latent space to extract more exact rule sets, manage to derive rule sets at high accuracy. However, they a) do not guarantee that the surrogate model has learned from the same variables as the DNN (alignment), b) only allow to optimise for a single objective, such as accuracy, which can result in excessively large rule sets (complexity), and c) use decision tree algorithms as intermediate models, which can result in different explanations for the same DNN (stability). This paper introduces the CGX (Column Generation eXplainer) to address these limitations - a decompositional method using dual linear programming to extract rules from the hidden representations of the DNN. This approach allows to optimise for any number of objectives and empowers users to tweak the explanation model to their needs. We evaluate our results on a wide variety of tasks and show that CGX meets all three criteria, by having exact reproducibility of the explanation model that guarantees stability and reduces the rule set size by >80% (complexity) at equivalent or improved accuracy and fidelity across tasks (alignment).

When analyzing robustness of predictive models under distribution shift, many works focus on tackling generalization in the presence of spurious correlations. In this case, one typically makes use of covariates or environment indicators to enforce independencies in learned models to guarantee generalization under various distribution shifts. In this work, we analyze a class of distribution shifts, where such independencies are not desirable, as there is a causal association between covariates and outcomes of interest. This case is common in the health space where covariates can be causally, as opposed to spuriously, related to outcomes of interest. We formalize this setting and relate it to common distribution shift settings from the literature. We theoretically show why standard supervised learning and invariant learning will not yield robust predictors in this case, while including the causal covariates into the prediction model can recover robustness. We demonstrate our theoretical findings in experiments on both synthetic and real data.

Why can deep learning predictors trained on Whole Slide Images fail to generalize? It is a common theme in Computational Pathology to see a high performing model developed in a research setting experience a large drop in performance when it is deployed to a new clinical environment. One of the major reasons for this is the batch effect that is introduced during the creation of Whole Slide Images resulting in a domain shift. Computational Pathology pipelines try to reduce this effect via stain normalization techniques. However, in this paper, we provide empirical evidence that stain normalization methods do not result in any significant reduction of the batch effect. This is done via clustering analysis of the dataset as well as training weakly-supervised models to predict source sites. This study aims to open up avenues for further research for effective handling of batch effects for improving trustworthiness and generalization of predictive modelling in the Computational Pathology domain.

While deep learning has achieved excellent performance in many various tasks, because of its black-box nature, it is still a long way from being widely used in the safety-critical task like healthcare tasks. For example, it suffers from poor explainability problem and is vulnerable to be attacked both in the training and testing time. Yet, existing works mainly for local explanations lack global knowledge to show class-wise explanations in the whole training procedure. In this talk, I will introduce our effort on visualizing a global explanation in the input space for every class learned in the training procedure. Our solution finds a representation set that could demonstrate the learned knowledge for each class, which could provide analyse on the model knowledge in different training procedures. We also show that the generated explanations could lend insights into diagnosing model failures, such as revealing triggers in a backdoored model.

During the past decade, deep learning has achieved great success in healthcare. However, most existing methods aim at model performance in terms of higher accuracy, which lacks the information reflecting the reliability of the prediction. It cannot be trustworthy for diagnosis making and even is disastrous for safety-critical clinical applications. How to build a reliable and robust healthcare system has become a focal topic in both academia and industry. In the talk, I will introduce our recent works for trustworthy AI in healthcare. Moreover, I also discuss some open challenges for trustworthy learning.

Saliency methods provide visual explainability for deep image processing models by highlighting informative regions in the input images based on feature-wise (pixels) importance scores. These methods have been adopted to the time series domain, aiming to highlight important temporal regions in a sequence. This paper identifies, for the first time, the systematic failure of such methods in the time series domain when underlying patterns (e.g., dominant frequency or trend) are based on latent information rather than temporal regions. The latent feature importance postulation is highly relevant for the medical domain as many medical signals, such as EEG signals or sensor data for gate analysis, are commonly assumed to be related to the frequency domain. To the best of our knowledge, no existing post-hoc explainability method can highlight influential latent information for a classification problem. Hence, in this paper, we frame and analyze the problem of latent feature saliency detection. We first assess the explainability quality of multiple state-of-the-art saliency methods (Integrated Gradients, DeepLift, Kernel SHAP, Lime) on top of various classification methods (LSTM, CNN, LSTM and CNN trained via saliency guided training) using simulated time series data with underlying temporal or latent space patterns. In conclusion, we identify that Integrated Gradients and DeepLift, if redesigned, could be potential candidates for latent saliency scores.

Surgical action triplet recognition provides a better understanding of the surgical scene. This task is of high relevance as it provides the surgeon with context-aware support and safety. The current go-to strategy for improving performance is the development of new network mechanisms. However, the performance of current state-of-the-art techniques is substantially lower than other surgical tasks. Why is this happening? This is the question that we address in this work. We present the first study to understand the failure of existing deep learning models through the lens of robustness and explainability. Firstly, we study current existing models under weak and strong $\delta-$perturbations via an adversarial optimisation scheme. We then analyse the failure modes via feature based explanations. Our study reveals that the key to improving performance and increasing reliability is in the core and spurious attributes. Our work opens the door to more trustworthy and reliable deep learning models in surgical data science.

Estimating uncertainty in image-to-image recovery networks is an important task, particularly as such networks are being increasingly deployed in the biological and medical imaging realms. A recent conformal prediction technique derives per-pixel uncertainty intervals, guaranteed to contain the true value with a user-specified probability. Yet, these intervals are hard to comprehend and fail to express uncertainty at a conceptual level. In this paper, we introduce a new approach for uncertainty quantification and visualization, based on masking. The proposed technique produces interpretable image masks with rigorous statistical guarantees for image regression problems. Given an image recovery model, our approach computes a mask such that a desired divergence between the masked reconstructed image and the masked true image is guaranteed to be less than a specified risk level, with high probability. The mask thus identifies reliable regions of the predicted image while highlighting areas of high uncertainty. Our approach is agnostic to the underlying recovery model and the true unknown data distribution. We evaluate the proposed approach on image colorization, image completion, and super-resolution tasks, attaining high quality performance on each.

Heart attack remain one of the greatest contributors to mortality in the United States and globally. Patients admitted to the intensive care unit (ICU) with diagnosed heart attack (myocardial infarction or MI) are more likely to suffer a secondary episode of MI and are at higher risk of death. In this study, we use two retrospective cohorts extracted from the eICU and MIMIC-IV databases, to develop a pseudo-dynamic machine learning framework for mortality and recurrent heart attack prediction in the ICU with interpretability and clinical risk analysis. The method provides accurate prediction of both outcomes for ICU patients up to 24 hours before the event and provide time-resolved interpretability results based on dynamic extraction of features from noisy and sparse time-series signals. The performance of the framework relying on extreme gradient boosting was evaluated on a held-out test set from eICU, and externally validated on the MIMIC-IV cohort using the most important features identified by time-resolved Shapley values achieving AUCs of 91.0 (balanced accuracy of 82.3) and 85.6 (balanced accuracy of 74.5) for 6-hour prediction of mortality and recurrent heart attack respectively. We show that our framework successfully leverages time-series physiological measurements by translating them into stacked static prediction problems to be robustly predictive through time in the ICU stay and can offer clinical insight from time-resolved interpretability.

The use of Convolutional Neural Networks (CNNs) has brought significant benefits to the healthcare industry, enabling the successful execution of challenging tasks such as disease diagnosis and drug discovery. However, CNNs are vulnerable to various types of noise and attacks, including transmission noise, noisy mediums, truncated operations, and intentional poisoning attacks. To address these challenges, this paper proposes a robust recovery method that removes noise from potentially contaminated CNNs and offers an exact recovery guarantee for one-hidden-layer non-overlapping CNNs with the rectified linear unit (ReLU) activation function. The proposed method can recover both the weights and biases of the CNNs precisely, given some mild assumptions and an overparameterization setting. Our experimental results on synthetic data and the Wisconsin Diagnostic Breast Cancer (WDBC) dataset validate the efficacy of the proposed method. Additionally, we extend the method to eliminate poisoning attacks and demonstrate that it can be used as a defense strategy against malicious model poisoning.

There are numerous efforts on technical development and translation of AI/ML in the healthcare domain. In addition to some of the classic challenges such as small datasets, limited annotations, imbalanced classes, how to gain and enhance trust from the users and practitioners of medical AI/ML is an emerging topic and key for successful applications of AI to patient care. In this talk, the speaker will elaborate what are the important pillars in developing trustworthy medical AI tools, how to marry medical intelligence and AI to enhance trust from clinicians, and showcase a range of applications of AI/ML in medical imaging.

Medical imaging plays a vital role in diagnosing and treating various health conditions, but it also raises significant privacy concerns as sensitive personal information can be contained within these images. Differential privacy, a privacy-preserving artificial intelligence technique, offers a solution to these challenges and enable the secure analysis of medical images while protecting patient privacy.

In this talk, we will focus on the potential of differential privacy in medical imaging. We will explore its various applications, including disease detection, diagnosis, and treatment planning, and discuss its ethical implications. We will also examine the technical aspects of differential privacy, including its implementation in machine learning algorithms, such as deep learning, and its limitations and challenges.

Furthermore, we will highlight some of our ongoing research and development efforts in this area, including recent advancements in differentially private deep learning for medical imaging. We will discuss the trade-offs between privacy and utility in these applications and provide insights on how to achieve a balance between the two.

Attendees will gain a deeper understanding of the potential and challenges of differential privacy in medical imaging and its implications for healthcare.

Accurate time prediction of patients' critical events is crucial in urgent scenarios where timely decision-making is important. Though many studies have proposed automatic prediction methods using Electronic Health Records (EHR), their coarse-grained time resolutions limit their practical usage in urgent environments such as the emergency department (ED) and intensive care unit (ICU). Therefore, in this study, we propose an hourly prediction method based on self-supervised predictive coding and multi-modal fusion for two critical tasks: mortality and vasopressor need prediction. Through extensive experiments, we prove significant performance gains from both multi-modal fusion and self-supervised predictive regularization, most notably in far-future prediction, which becomes especially important in practice. Our uni-modal/bi-modal/bi-modal self-supervision scored 0.846/0.877/0.897 (0.824/0.855/0.886) and 0.817/0.820/0.858 (0.807/0.81/0.855) with mortality (far-future mortality) and with vasopressor need (far-future vasopressor need) prediction data in AUROC, respectively.

In drug development, early phase dose-finding clinical trials are carried out to identify an optimal dose to administer to patients in larger confirmatory clinical trials. Standard trial procedures do not optimize for participant benefit and do not consider participant heterogeneity, despite consequences to the health of participants and downstream impacts to under-represented population subgroups. Additionally, many newly investigated drugs do not obey modelling assumptions made in common dose-finding procedures. We present Safe Allocation for Exploration of Treatments (SAFE-T), a procedure for adaptive dose-finding that works well with small samples sizes and improves the utility for heterogeneous participants while adhering to safety constraints for treatment arm allocation. SAFE-T flexibly learns models for drug toxicity and efficacy without requiring strong prior assumptions and provides final recommendations for optimal dose by participant subgroup. We provide a preliminary evaluation of SAFE-T on a comprehensive set of realistic synthetic dose-finding scenarios, illustrating the improved performance of SAFE-T with respect to safety, utility, and dose recommendation accuracy across heterogeneous participants against a comparable baseline method.

In this study, we introduce ExBEHRT, an extended version of BEHRT (BERT applied to electronic health record data) and applied various algorithms to interpret its results. While BEHRT only considers diagnoses and patient age, we extend the feature space to several multi-modal records, namely demographics, clinical characteristics, vital signs, smoking status, diagnoses, procedures, medications and lab tests by applying a novel method to unify the frequencies and temporal dimensions of the different features. We show that additional features significantly improve model performance for various down-stream tasks in different diseases. To ensure robustness, we interpret the model predictions using an adaption of expected gradients, which has not been applied to transformers with EHR data so far and provides more granular interpretations than previous approaches such as feature and token importances. Furthermore, by clustering the models' representations of oncology patients, we show that the model has implicit understanding of the disease and is able to classify patients with same cancer type into different risk groups. Given the additional features and interpretability, ExBEHRT can help making informed decisions about disease progressions, diagnoses and risk factors of various diseases.

Machine Learning (ML) has recently shown tremendous success in modeling various healthcare prediction tasks, ranging from disease diagnosis and prognosis to patient treatment. Due to the sensitive nature of medical data, privacy must be considered along the entire ML pipeline, from model training to inference. In this paper, we conduct a review of recent literature concerning Privacy-Preserving Machine Learning (PPML) for healthcare. We primarily focus on privacy-preserving training and inference-as-a-service, and perform a comprehensive review of existing trends, identify challenges, and discuss opportunities for future research directions. The aim of this review is to guide the development of private and efficient ML models in healthcare, with the prospects of translating research efforts into real-world settings.

Federated learning (FL) is a trending framework to enable multi-institutional collaboration in machine learning without sharing raw data. This presentation will discuss our ongoing progress in designing FL algorithms that embrace the data heterogeneity properties for distributed medical data analysis in the FL setting. First, I will present our work on theoretically understanding FL training convergence and generalization using a neural tangent kernel, called FL-NTK. Then, I will present our algorithms for tackling data heterogeneity (on features and labels) and device heterogeneity, motivated by our previous theoretical foundation. Lastly, I will also show the promising results of applying our FL algorithms in healthcare applications.

Simplified ultrasound scanning protocols (sweeps) have been developed to reduce the high skill required to perform a regular obstetric ultrasound examination. However, without automated quality assessment of the video, the utility of such protocols in clinical practice is limited. An automated quality assessment algorithm is proposed that applies an object detector to detect fetal anatomies within ultrasound videos. Kernel density estimation is applied to the bounding box annotations to estimate a probability density function of certain bounding box properties such as the spatial and temporal position during the sweeps. This allows quantifying how well the spatio-temporal position of anatomies in a sweep agrees with previously seen data as a quality metric. The new quality metric is compared to other metrics of quality such as the confidence of the object detector model.

The annotation of domain experts is important for some medical applications where the objective ground truth is ambiguous to define, e.g., the rehabilitation for some chronic diseases, and the prescreening of some musculoskeletal abnormalities without further medical examinations. However, improper uses of the annotations may hinder developing reliable models. On one hand, forcing the use of a single ground truth generated from multiple annotations is less informative for the modeling. On the other hand, feeding the model with all the annotations without proper regularization is noisy given existing disagreements. For such issues, we propose a novel Learning to Agreement (Learn2Agree) framework to tackle the challenge of learning from multiple annotators without objective ground truth. The framework has two streams, with one stream fitting with the multiple annotators and the other stream learning agreement information between annotators. In particular, the agreement learning stream produces regularization information to the classifier stream, tuning its decision to be better in line with the agreement between annotators. The proposed method can be easily added to existing backbones, with experiments on two medical datasets showed better agreement levels with annotators.

Coronary artery segmentation has great significance in providing morphological information and treatment guidance in clinics. However, the complex structures with tiny and narrow branches of the coronary artery bring it a great challenge. Limited by the low resolution and poor contrast of medical images, voxel-based segmentation methods could potentially lead to fragmentations of segmented vessels and surface voids are commonly found in the reconstructed mesh. Therefore, we propose a geometry-based end-to-end segmentation method for the coronary artery in computed tomography angiography. A U-shaped network is applied to extract image features, which are projected to mesh space, driving the geometry-based network to deform the mesh. Integrating the ability of geometric deformation, the proposed network could output mesh results of the coronary artery directly. Besides, the centerline-based approach is utilized to produce the ground truth of the mesh instead of the traditional marching cube method. Extensive experiments on our collected dataset CCA-520 demonstrate the feasibility and robustness of our method. Quantitatively, our model achieves Dice of 0.779 and HD of 0.299, exceeding other methods in our dataset. Especially, our geometry-based model generates an accurate, intact and smooth mesh of the coronary artery, devoid of any fragmentations of segmented vessels.

Microscopy images from different imaging conditions, organs, and tissues often have numerous cells with various shapes on a range of backgrounds. As a result, designing a deep learning model to count cells in a source domain becomes precarious when transferring them to a new target domain. To address this issue, manual annotation costs are typically the norm when training deep learning-based cell counting models across different domains. In this paper, we propose a cross-domain cell counting approach that requires only weak human annotation efforts. Initially, we implement a cell counting network that disentangles domain-specific knowledge from domain-agnostic knowledge in cell images, where they pertain to the creation of domain style images and cell density maps, respectively. We then devise an image synthesis technique capable of generating massive synthetic images founded on a few target-domain images that have been labeled. Finally, we use a public dataset consisting of synthetic cells as the source domain, where no manual annotation cost is present, to train our cell counting network; subsequently, we transfer only the domain-agnostic knowledge to a new target domain of real cell images. By progressively refining the trained model using synthesized target-domain images and several real annotated ones, our proposed cross-domain cell counting method achieves good performance compared to state-of-the-art techniques that rely on fully annotated training images in the target domain. We evaluated the efficacy of our cross-domain approach on two target domain datasets of actual microscopy cells, demonstrating the feasibility of requiring annotations on only a few images in a new domain.

We present on-going work toward building Stasis, a suite of reinforcement learning (RL) environments that aim to maintain realism for human-centric agents operating in real-world settings. Through representation learning and alignment with real-world offline data, Stasis allows for the evaluation of RL algorithms in offline environments with adjustable characteristics, such as observability, heterogeneity and levels of missing data. We aim to introduce environments the encourage training RL agents that are capable of maintaining a level of performance and robustness comparable to agents trained in real-world online environments, while avoiding the high cost and risks associated with making mistakes during online training. We provide examples of two environments that will be part of Stasis and discuss its implications for the deployment of RL-based systems in sensitive and high-risk areas of application.