Integrating Data Correlations into Modern Predictive Modeling
Wednesday, August 2, 2023 at 1 PM (Pacific Time)
125 Li Ka Shing (LKS), UC Berkeley
Seminar title:
Integrating Data Correlations into Modern Predictive Modeling
Speaker:
Prof. Saharon Rosset, Department of Statistics and Operations Research, Tel Aviv University
Abstract:
Correlations are ubiquitous in many domains, and can stem from spatial or temporal structure, repeated measures or otherwise clustered observations. Properly taking the correlations into account and using them can be important in model building, model evaluation, model selection, etc. I will survey two separate lines of work that explore this area, offer solutions and demonstrate their efficacy: 1. Cross validation for correlated data : taking into account correlations in model evaluation and selection using cross validation. We make explicit the conditions under which regular cross-validation can still be applied, and derive corrections when these conditions don't hold. 2. Integrating random effects into deep learning: taking into account correlations in model building using deep learning approaches, by describing the correlations in a random effects or random field framework and changing the learning approach (in particular, the network loss function) to account for the correlations. Both lines of work are demonstrated to be highly effective in simulations and real data analysis in biological and other domains. This talk is based on the PhD research of Assaf Rabinowicz and Giora Simchoni.
