Directed by Drs. Abhijit Chowdhary & Elizabeth Newman

Project Description: Machine learning models are often constructed by solving a challenging stochastic optimization problem, a process commonly known as training.  Among the training challenges include the high-dimensionality of the solution (optimization of potentially millions or billions of weights), a lack of convergence guarantees, and time- and resource-consuming hyperparameter tuning.  In this project, students will explore efficient optimization strategies that train lightweight models with practical structure to reduce the training burden.  The particular structures that this project will explore are linearly separable models, which have a feature extraction phase followed by a linear mapping.  An effective training strategy that leverages this separable structure is called variable projection (VarPro), which optimizes the linear mapping throughout the training process.  Depending on student interest and experience, this project will explore training techniques that use variable projection combined with efficient randomized numerical linear algebra, scalable high-performance computing implementations, practical physics-informed training paradigms, and beyond.  

Desired background: Students interested in this project must have completed a calculus sequence course,  a linear algebra and/or a differential equation course, and have knowledge of a computer programming language, e.g., Python or Matlab.