Scalable Deep Learning

Ameet Talwalkar Photo
Speaker Name: 
Ameet Talwalkar
Speaker Title: 
Associate Professor
Speaker Organization: 
Carnegie Mellon University
Start Time: 
Wednesday, November 15, 2017 - 2:00pm
End Time: 
Wednesday, November 15, 2017 - 3:00pm
UCSC, E2-599
Data Science Santa Cruz

Abstract: Although deep learning has received much acclaim due to its widespread empirical success, fundamental bottlenecks exist when attempting to develop deep learning applications at scale. One bottleneck involves exploring the design space of a model family, which typically requires training tens to thousands of models with different hyperparameters. Model training itself is a second major bottleneck, as classical learning algorithms are often infeasible for the petabyte datasets that are fast becoming the norm. In this talk, I present my research addressing these two core bottlenecks. I first introduce Hyperband, a novel algorithm for hyperparameter optimization that is simple, flexible, theoretically sound, and an order of magnitude faster than leading competitors. I then present work aimed at understanding the underlying landscape of training deep learning models in parallel and distributed environments. Specifically, I introduce an analytical performance model called Paleo, which can quickly and accurately model the expected scalability and performance of putative parallel and distributed deep learning systems.

Bio: Ameet Talwalkar is an assistant professor (starting January 2018) in the Machine Learning Department at Carnegie Mellon University, and also co-founder and Chief Scientist at Determined AI. His research addresses scalability and ease-of-use issues in the field of statistical machine learning, with applications in computational genomics. He led the initial development of the MLlib project in Apache Spark, is a co-author of the graduate-level textbook 'Foundations of Machine Learning' (2012, MIT Press), and teaches an award-winning MOOC on edX called 'Distributed Machine Learning with Apache Spark.' Prior to CMU, he was an assistant professor at UCLA.