Tuo Zhao H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Speaker's Website Thu, 10/20/2022 - 4:00pm 204 Caldwell Hall Approximation and Statistical Properties of Deep Neural Networks on Structured Data Abstract Deep neural networks have demonstrated remarkable generalization performance in high dimensional problems, e.g., image classification, where each image contains a large number of pixels. Such appealing performance contradicts a fundamental theoretical challenge – curse of data dimensionality. To explain this huge gap, we take the data intrinsic geometric structures into consideration, and model high-dimensional data as samples on a low-dimensional manifold. We show that neural networks can efficiently approximate functions supported on a low-dimensional manifold. The network size scales exponentially in the approximation error, with an exponent depending on the data intrinsic dimension. As an application of our function approximation theory in statistics, we show that deep neural networks can circumvent the curse of data ambient dimensionality by capturing unknown data intrinsic structures and attain fast statistical convergence in regression and distribution estimation. Biography Dr. Tuo Zhao is an Assistant Professor in the H. Milton Stewart School of Industrial and Systems Engineering at Georgia Tech. He received his Ph.D. degree in Computer Science at Johns Hopkins University in 2016. He was a visiting scholar in the Department of Biostatistics at Johns Hopkins Bloomberg School of Public Health from 2010 to 2012, and the Department of Operations Research and Financial Engineering at Princeton University from 2014 to 2016. His research focuses on developing principled methodologies, nonconvex optimization algorithms and practical theories for machine learning (especially deep learning). He is also interested in natural language processing and actively contributing to open source software development for scientific computing.
