Speaker

　　Title

　　Time

　　Venue

N109

　　Abstract

　　In this talk, I will present our recent work on the convergence/generalization analysis for the popular optimizers in deep learning. (1) We establish the convergence for Adam under (L0,L1 ) smoothness condition and argue that Adam can adapt to the local smoothness condition while SGD cannot. (2) We study the implicit regularization of DL optimizers. For adaptive optimizers, we prove that the convergent direction of RMSProp is the same with GD, while that of AdaGrad depends on the conditioner; for momentum acceleration, we prove that gradient descent with momentum converges to the L2 max-margin solution, which is the same as vanilla gradient descent. 

　　Affiliation

　　陈薇，中国科学院计算技术研究所研究员、博士生导师,中国科学技术大学数学科学学院兼职博士生导师.曾任微软亚洲研究院计算学习理论组负责人、理论中心共同主任。研究领域为机器学习基础理论与方法，研究兴趣包括深度学习理论、因果机器学习、分布式机器学习等。入选中科院人才引进计划、中科院基础研究领域青年团队计划，福布斯中国科技女性榜。在机器学习和人工智能顶级国际会议/期刊发表学术论文60余篇，出版学术专著1本，担任ICML、NeurIPS等机器学习国际会议领域主席。