| Abstract | The primary task of many applications is approximating/estimating a function through samples drawn from a probability distribution on the input space. The deep approximation is to approximate a function by compositions of many layers of simple functions, that can be viewed as a series of nested feature extractors. The key idea of deep learning network is to convert layers of compositions to layers of tuneable parameters that can be adjusted through a learning process, so that it achieves a good approximation with respect to the input data. In this talk, we shall discuss mathematical theory behind this new approach and approximation rate of deep network; we will also show how this new approach differs from the classic approximation theory, and how this new theory can be used to understand and design deep learning networks. |
| Affiliation | Prof Shen received his PhD in 1991 from the University of Alberta, Canada and completed his postdoctoral training at University of Wisconsin-Madison. He joined NUS in 1993, was promoted to full professor in 2002, and Distinguished Professor in 2009. Prof Shen was Tan Chin Tuan Centennial Professor from 2013 to 2021. A renowned mathematician, Professor Shen is well-known for his fundamental contributions in mathematical foundations of data science, especially in the areas of approximation and wavelet theory, image processing and compressed sensing, computer vision and machine learning. Together with his collaborators, he has several signature theorems and algorithms that include developing a duality analysis that leads to three mathematical principles: the duality principle, the unitary extension principle and the oblique extension principle in approximation and wavelet theory; sparsity based balanced model and algorithms by using redundant systems in image processing; and the singular value thresholding algorithm in compressed sensing. His recent research interests focus on approximation theory of deep neural networks. Prof Shen is a prominent researcher in his various fields of research. He sits on several editorial boards of top journals and has been invited to speak at many international conferences and congresses, including the International Congress of Mathematicians (ICM) in 2010 and the International Congress of Industrial and Applied Mathematics (ICIAM) in 2015. Both ICM and ICIAM, which are held every four years, are the most reputable congresses in mathematics and applied mathematics, and being an invited speaker at these events is testament to his expertise and leadership in these fields. Prof Shen has received numerous awards and honours, including the NUS Outstanding University Researcher Award (2008 and 1997), Wavelet Pioneer Award from the Society of Photographic Instrumentation Engineers, US (2012), and the National Science Award of Singapore (1998). He has been elected as Fellow of the World Academy of Sciences (2020), Fellow of the Society for Industrial and Applied Mathematics, US (2019), Fellow of the American Mathematical Society, US (2017), and inaugural Fellow of the Singapore National Academy of Science (2011). |
