Statistical Machine Learning has matured into a rich and quickly growing discipline whose methods can have many practical applications. In this tutorial we will briefly sketch some of the most often used supervised learning techniques, with a view towards learning conditional probabilistic models of data that may not be i.i.d.. The tutorial will start with a brief introduction to theoretical concepts of capacity and overfitting, and follow with a description and user-based discussion of many algorithms. The starting point will be the logistic classifier (a.k.a. maximum entropy classifier). A large portion of the tutorial will then be devoted to multi-layer neural networks for learning conditional probabilities, which can be seen as extensions of logistic regression. The presentation will then briefly present the principles and limits of Support Vector Machines (which can also be seen as non-linear extensions of logistic regression), and conclude with boosting and ensemble methods.

Yoshua Bengio is full professor at the Department of Computer Science and Operations Research at University of Montreal, and he is the Canada Research Chair in Statistical Learning Algorithms. He obtained his B.Eng. (computer engineering), M.Sc. and Ph.D. (both in computer science) from McGill University. Yoshua Bengio completed two post-doctorates: the first at MIT and the second at AT&T Bell Labs. He is action editor for the Journal of Machine Learning Research and associate editor for the IEEE Transactions on Neural Networks. Yoshua Bengio specializes in improving statistical learning algorithms and their applications to high-dimensional data.