Abstract:  Bayesian optimization is a powerful tool for optimizing time-consuming-to-evaluate non-convex derivative-free objective functions.  While BayesOpt has historically been deployed as a black-box optimizer, recent advances show considerable gains by “peeking inside the box”.  For example, when tuning hyperparameters in deep neural networks to minimize validation error, state-of-the-art BayesOpt tuning methods leverage the ability to stop training early, restart previously paused training, perform training and testing on a strict subset of the available data, and warm-start from previously tuned network architectures.  We describe new “grey box” Bayesian optimization methods that selectively exploit problem structure to deliver state-of-the-art performance.  We then describe applications of these methods to tuning deep neural networks, inverse reinforcement learning and calibrating physics-based simulators to observational data.

Bio:  Peter Frazier is an Associate Professor in the School of Operations Research and Information Engineering at Cornell University and a Staff Data Scientist at Uber. He received a Ph.D. in Operations Research and Financial Engineering from Princeton University in 2009. His academic research is on the optimal collection of information, including Bayesian optimization, incentive design for social learning and multi-armed bandits, with applications in applications in e-commerce, the sharing economy and materials design. At Uber, he managed UberPool’s data science group, helped design Uber’s pricing system and the driver destination feature, and currently works on allocation of rider promotions and driver incentives. He is an associate editor for Operations Research, ACM TOMACS, and IISE Transactions, and is the recipient of an AFOSR Young Investigator Award and an NSF CAREER Award.