Introduction to Machine Learning
ETH Zurich, Prof. Andreas Krause, Autumn Semester 2013
Course Description
Machine learning algorithms are data analysis methods which search data sets for patterns and characteristic structures. Typical tasks are the classification of data, automatic regression and unsupervised model fitting. Machine learning has emerged mainly from computer science and artificial intelligence, and draws on methods from a variety of related subjects including statistics, applied mathematics and more specialized fields, such as pattern recognition and neural computation. Applications are, for example, image and speech analysis, medical imaging, bioinformatics and exploratory data analysis in natural science and engineering:
This course is intended as an introduction to machine learning. It will review the necessary statistical preliminaries and provide an overview of commonly used machine learning methods. Further and more advanced topics will be discussed in the course Statistical Learning Theory, held in the spring semester by Prof. Buhmann.
News
Date  What? 

30.01.14 

26.01.14 

15.01.14 

14.01.14 

09.01.14 

08.01.14 

06.01.14 

06.01.14 

9.12 

3.12 

3.12 

25.11 

18.11 

30.10 

20.10 

02.10 

01.10 

20.09 

19.09 

27.08 

General Information
VVZ Information: See here.Time and Place
 Lectures
Mon 1415 CAB G11 Tue 1012 CAB G11  Tutorials
Wed 1517 CAB G61 Last Names AE Thu 1517 CAB G59 Last Names FK Fr 0810 CAB G52 Last Names LR Fr 1315 CHN G46 Last Names SZ
* All tutorial sessions are identical, please only attend one session.
Syllabus
Calendar Week  Lecture Topics  Lecture Slides  Tutorial Slides  Exercise Sheets & Solutions  

38  Introduction to Machine Learning  [pdf]  
39  Regression  [pdf]  
40  Crossvalidation Perceptron 
[pdf]  
41  SVMs  [pdf]  
42  Nonlinear SVMs Kernels kNN 
[pdf]  
43  Kernels; kernelized linear regression; Feature selection 
[pdf]  
44  Sparsity, multiclass, structured prediction 
[pdf]  
45  Probabilistic modeling Logistic regression 
[pdf]  
46  Bayesian learning, Gaussian processes 
[pdf] 

47  Ensemble methods  [pdf][videos] 

48  Discriminative vs. Generative models 
[pdf]  
49  Clustering kMeans,GMMs 
[pdf]  
50  No class  
51  Dimension reduction (K)PCA,LLE,MDS 
[pdf]  
*  Q & A 
Some of the material is password protected, send an email to iml2013@inf.ethz.ch to obtain it.
Exercises
The exercise problems will include theoretical and programming problems. Please note that it is not mandatory to submit solutions, a Testat is not required in order to participate in the exam. We will publish exercise solutions after one week.
If you choose to submit: Send a soft copy of the exercise to iml2013@inf.ethz.ch. This can be latex, but also a simple scan or even a picture of a handwritten solution.
 Please do not submit hard copies of your solutions.
Project
Part of the coursework will be a project, carried out in groups of up to 3 students. The goal of this project is to get handson experience in machine learning tasks. The project grade will constitute 30% of the total grade. More details on the project will be given in the tutorials.
Exam
The mode of examination is written, 120 minutes length. The language of examination is English. As written aids, you can bring two A4 pages (i.e. one A4 sheet of paper), either handwritten or 11 point minimum font size. The written exam will constitute 70% of the total grade.
Resources
Java Applets
MATLAB Demos
GPO Videos from Lecture 10
Old Exams
Important: Don't expect the exam to cover the same topics as previous exams. Exam material is everything that has been discussed in the lecture and tutorials, the lecture and tutorial slides and the exercises.Text Books
 C. Bishop. Pattern Recognition and Machine Learning. Springer 2007.
This is an excellent introduction to machine learning that covers most topics which will be treated in the lecture. Contains lots of exercises, some with exemplary solutions. Available from ETHHDB and ETHINFK libraries.  R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley & Sons, second edition, 2001.
The classic introduction to the field. An early edition is available online for students attending this class, the second edition is available from ETHBIB and ETHINFK libraries.  T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2001.
Another comprehensive text, written by three Stanford statisticians. Covers additive models and boosting in great detail. Available from ETHBIB and ETHINFK libraries.
A free PDF version (second edition) is available online  L. Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2004.
This book is a compact treatment of statistics that facilitates a deeper understanding of machine learning methods. Available from ETHBIB and ETHINFK libraries.
Matlab
 The official Matlab documentation is available online at the Mathworks website.
 If you have trouble accessing Matlab's builtin help function, you can use the online function reference on that page or use the commandline version (type help <function> at the prompt). There are several primers and tutorials on the web, a later edition of this one became the book Matlab Primer by T. Davis and K. Sigmon, CRC Press, 2005.
Discussion Forum
We maintain a discussion board at the VIS inforum. Use it to ask questions of general interest and interact with other students of this class. We regularly visit the board to provide answers.
Contact
 Instructor: Prof. Andreas Krause
 Head Assistant: Gabriel Krummenacher
 Assistants: Alexey Gronskiy, Dmitry Laptev, Dr. Dwarikanath Mahapatra, Mario Lučić, Hemant Tyagi, Marcela Zuluaga, Ekaterina Lomakina