Modeling the Evolution of Chronic Diseases from Heterogeneous, Multimodal Data
Chronic conditions such as congenital heart disease, Alzheimer's disease and osteoarthritis affect a significant segment of the population. Today, an estimated 133 million Americans – nearly half the population – suffer from at least one chronic illness. Longitudinal studies that monitor subjects over extended periods of time help determine the relationships between risk factors and disease evolution, which is essential in quantifying the effectiveness of treatment and palliative care. The studies comprise multimodal data such as demographics, time series, medical images, and genetic information. All are collected across multiple institutions, multiple patient populations and multiple visits. The collection process induces heterogeneity at all levels: there is high irregularity, inter-subject variability, and potentially changing collection protocols. Reliable disease trajectory models, constructed through retrospective statistical analysis of this multimodal longitudinal data, are necessary to inform patients and facilitate clinical decisions.
We address the methodological gap by tightly integrating multimodal data and leveraging the different sources of information, including domain expertise, to extract salient features. In the Information Fusion Lab, we develop hybrid models that optimize multi-component objectives, specialized to the task and for the available data. Our models include hybrid layers, designed to cope with multiple inputs of distinct types, such as attributes encoded as discrete features provided together with their associated images. In the talk, I will present mechanisms to conditionally route samples through the neural networks depending on their cross-modal characteristics, models that leverage the intrinsic frequency of signals to learn sparse forecasting models from multivariate time series and weakly supervised deep learning architectures incorporating domain-specific heuristics. These techniques have enabled us to efficiently construct representations of images that adhere to specific patterns, such as medical images of different organs. In the talk, I will demonstrate the performance of our models in attaining state of the art results on tasks such as Alzheimer's disease forecasting, detecting heart conditions and in-hospital mortality prediction. Finally, I will describe how multimodal and multiresolution networks can be used for weather modeling.
Madalina Fiterau is a Computer Scientist, working as an Assistant Professor in the College of Information and Computer Sciences at UMass Amherst, leading the Information Fusion Lab. Previously, she was a postdoc at Stanford University, having completed a PhD in Machine Learning from Carnegie Mellon University. Dr. Fiterau current research is on hybrid models and on the development of new deep learning methodology to obtain salient representations from multimodal biomedical data, including time series, text and images. Dr. Fiterau was awarded the Marr Prize for Best Paper at ICCV 2015, the Star Research Award at the Annual Congress of the Society of Critical Care Medicine 2016, the Manning IALS Research Award in 2019, an IALS Midigrant in 2022, an Institute of Diversity Sciences Seed Grant in 2023 and an R03 from the NIH in 2023. Dr. Fiterau is keenly interested in applying my ML research towards the advancement of healthcare, having previously co-organized several editions of the NeurIPS workshop on Machine Learning in Healthcare and the Machine Learning in Healthcare Conference.
