网站地图 | 联系我们 | English | 意见反馈 | 主任信箱
 
首页 中心概况 新闻动态 科研进展 交流合作 人才培养 研究队伍 人才招聘 政策规章 数学交叉科学传播
学术报告
现在位置:首页 > 学术报告

Some Mathematical Aspects of Deep Learning and Stochastic Gradient Descent
【2023.5.22 2:00pm, 腾讯会议】

【打印】【关闭】

   2023-5-15 

  Colloquia & Seminars 

  

  Speaker

Prof. Lexing Ying, Stanford University

  Title

Some Mathematical Aspects of Deep Learning and Stochastic Gradient Descent

  Time

2023年5月22日 14:00

  Venue

腾讯会议:287-933-194

  Abstract

   This talk concerns several mathematical aspects of deep learning and stochastic gradient descent. The first aspect is why deep neural networks trained with stochastic gradient descent often generalize. We will make a connection between the generalization and the stochastic stability of the stochastic gradient descent dynamics. The second aspect is to understand the training process of stochastic gradient descent. Here, we use several simple mathematical examples to explain several key empirical observations, including the edge of stability, exploration of flat minimum, and learning rate decay. Based on joint work with Chao Ma.  

  Affiliation

  Lexing Ying is a professor of mathematics at Stanford University. He received B.S. from Shanghai Jiaotong University in 1998 and Ph.D. from New York University in 2004. Before joining Stanford in 2012, he was a post-doc at Caltech and a professor at UT Austin. He received a Sloan Fellowship in 2007, an NSF Career Award in 2009, the Fengkang Prize in 2011, the James H. Wilkinson Prize in 2013, and the Silver Morningside Medal in 2016. He was an invited speaker of ICM 2022.

  

  

欢迎访问国家数学与交叉科学中心 
地址:北京海淀区中关村东路55号 邮编:100190 电话: 86-10-62613242 Fax: 86-10-62616840 邮箱: ncmis@amss.ac.cn