|
Abstract |
This talk starts with a brief review of recent advances in artificial intelligence, including the break-through in speech recognition, image classification, natural language understanding, and games playing. One natural question behind these exciting moments is what is the true technical driving force that makes them happen. Regarding this question, I would like to point out that the underlying technologies (deep learning and reinforcement learning) are not really new; what makes them different today is the availability of big training data and the big computational power that allows us to leverage these data to train very deep models. I will then introduce the recent technical innovations made by Microsoft Research Asia on how to train very deep neural networks through effective and efficient distribute learning. First, I will introduce how the residual neural networks (ResNet) can help resolve the problem of gradient vanishing and enable the training of neural networks with thousands of layers, and then make some discussions on the optimal depth for neural networks from a theoretical perspective. Second, I will introduce how to appropriately use asynchronous parallelization to distribute deep learning onto multiple machines, in order to achieve linear speed up while maintaining the accuracy. At the end of the talk, I will give an overview of other related research in MSR Asia and discuss several promising future research directions. |
|
Affiliation |
Tie-Yan Liu is a principal researcher of Microsoft Research Asia, leading the research on artificial intelligence and machine learning. He is very well known for his pioneer work on learning to rank and computational advertising, and his recent research interests include deep learning and distributed machine learning. As a researcher in an industrial lab, Tie-Yan Liu is making his unique contributions to the world. On one hand, many of his technologies have been transferred to Microsoft’s products and online services, such as Bing, Microsoft Advertising, and Azure. He has received many recognitions and awards in Microsoft for his significant product impacts. On the other hand, he has been actively contributing to academic communities. He is an adjunct professor at Carnegie Mellon University (CMU) and several other universities. His top ten papers have been cited for about 4000 times in refereed conferences and journals. He has won quite a few awards, including the best student paper award at SIGIR (2008), the most cited paper award at Journal of Visual Communications and Image Representation (2004-2006), and the research break-through award at Microsoft Research (2012). He has been invited to serve as general chair, PC chair, or area chair for a dozen of top conferences including SIGIR, WWW, KDD, NIPS, IJCAI, AAAI, ICTIR, as well as associate editor/editorial board member of ACM Transactions on Information Systems, ACM Transactions on the Web, Information Retrieval Journal, and Foundations and Trends in Information Retrieval. Tie-Yan Liu and his works have been reported by many International media, including National Public Radio, CNET, MIT Technology Review, and PCTech Magazine. |
