Some Mathematical Aspects of Deep Learning and Stochastic Gradient Descent 【2023.5.22 2:00pm, 腾讯会议】 
【大 中 小】【打印】【关闭】 
2023515 Colloquia & Seminars Speaker  Prof. Lexing Ying, Stanford University  Title  Some Mathematical Aspects of Deep Learning and Stochastic Gradient Descent  Time  2023年5月22日 14：00  Venue  腾讯会议:287933194  Abstract  This talk concerns several mathematical aspects of deep learning and stochastic gradient descent. The first aspect is why deep neural networks trained with stochastic gradient descent often generalize. We will make a connection between the generalization and the stochastic stability of the stochastic gradient descent dynamics. The second aspect is to understand the training process of stochastic gradient descent. Here, we use several simple mathematical examples to explain several key empirical observations, including the edge of stability, exploration of flat minimum, and learning rate decay. Based on joint work with Chao Ma.  Affiliation  Lexing Ying is a professor of mathematics at Stanford University. He received B.S. from Shanghai Jiaotong University in 1998 and Ph.D. from New York University in 2004. Before joining Stanford in 2012, he was a postdoc at Caltech and a professor at UT Austin. He received a Sloan Fellowship in 2007, an NSF Career Award in 2009, the Fengkang Prize in 2011, the James H. Wilkinson Prize in 2013, and the Silver Morningside Medal in 2016. He was an invited speaker of ICM 2022.  
