Computer Science Colloquium – Ina Fiterau, UMass Amherst

Modeling multimodal data is particularly important in healthcare applications. Chronic conditions such as osteoarthritis, congenital heart disease and Alzheimer’s disease affect a significant segment of the population. According to the CDC, almost 45% of all Americans suffer from at least one chronic condition. Longitudinal studies that monitor subjects over extended periods of time help determine the relationships between risk factors and disease evolution, key to quantifying the effectiveness of treatment and palliative care. The studies comprise multimodal data such as demographics, time series, medical images and genetic information. All are collected across multiple institutions, multiple patient populations and multiple visits. Essentially, the collection process induces heterogeneity at all levels: there is high irregularity, inter-subject variability and potentially changing collection protocols. Reliable disease trajectory models, constructed through retrospective statistical analysis of this multimodal longitudinal data, are necessary to inform patients and facilitate clinical decisions.

We address the methodological gap by tightly integrating multimodal data and leveraging the different sources of information, including domain expertise, to extract salient features. Our hybrid models optimize multi-component objectives, specialized to the task and for the available data. Moreover, they include hybrid layers, built with `neurons’ that are designed to cope with multiple inputs of distinct types, such as attributes encoded as discrete features provided together with their associated images. We also use various mechanisms to conditionally route samples through the neural networks depending on their cross modal characteristics. We use generative models to enable weak supervision through domain-specific heuristics. We also introduce structured sparsity and manifold learning in normalizing flows, a class of deep generative models that achieve both feature learning and tractable marginal likelihood estimation. This allows us to efficiently construct representations of images that adhere to specific patterns, such as medical images of different organs. I’ll demonstrate the performance of our models in attaining state of the art results on tasks such as Alzheimer’s disease forecasting, detecting heart conditions and in-hospital mortality prediction.

Ina Fiterau has completed a PhD in Machine Learning at Carnegie Mellon University in Fall 2015, where she was a member of the Auton Lab. Between Fall 2015 and Fall 2018, she was a Postdoctoral Fellow in the Mobilize Center at Stanford University. She joined the College of Information and Computer Sciences at UMass in September 2018. Ina is the recipient of the Marr Prize for Best Paper at ICCV 2015 and of Star Research Award at the Annual Congress of the Society of Critical Care Medicine 2016. She was also the recipient of the Manning IALS Research Award in 2019.