Abstract:Deep Learning (DL) has been incredibly successful, due to its ability to automatically acquire useful representations from raw data by a joint optimization process of all layers.
Deep Learning (DL) has been incredibly successful, due to its ability to automatically acquire useful representations from raw data by a joint optimization process of all layers. However, current DL practice still requires substantial manual efforts to define the right neural architecture and training hyperparameters to optimally learn these representations for the data at hand. The next logical step is to jointly optimize these components as well, based on a meta-level of learning and optimization. I predict that this will allow the next generation of DL systems to simply accept data and user objectives to optimize for (which can, e.g., include fairness, robustness, uncertainty calibration, interpretability, etc) and to thereby provide a clean interface between domain experts (who best know the data and the relevant objectives for the application at hand, but do not need to be machine learning experts) on the one hand and the next-generation DL system on the other hand. In this talk, I will discuss several advances towards this goal, focussing on (1) joint optimization of several meta-choices in the DL pipeline, (2) efficiency of this meta-optimization, and (3) optimization of uncertainty estimates and robustness to data shift.
Frank Hutter is a Full Professor for Machine Learning at the Computer Science Department of the University of Freiburg (Germany), as well as Chief Expert AutoML at the Bosch Center for Artificial Intelligence. Frank holds a PhD from the University of British Columbia (UBC, 2009) and a Diplom (eq. MSc) from TU Darmstadt (2004). He received the 2010 CAIAC doctoral dissertation award for the best thesis in AI in Canada, and with his coauthors, several best paper awards and prizes in international competitions on machine learning, SAT solving, and AI planning. He received a 2016 ERC Starting Grant, a 2018 Google Faculty Research Award, a 2020 ERC PoC Award, and he is a Fellow of ELLIS. Frank is best known for his research on automated machine learning (AutoML), including neural architecture search and efficient hyperparameter optimization. He co-organized the ICML workshop series on AutoML every year since its inception in 2014, co-authored the prominent AutoML tools Auto-WEKA, Auto-sklearn and Auto-PyTorch, won the first two AutoML challenges with his team, co-authored the first book on AutoML, and gave a NeurIPS 2018 tutorial on AutoML with over 3000 attendees.
(Wednesday) 9:15 AM - 10:00 AM