Introduction to Machine Learning

ETH Zurich, Prof. Andreas Krause, Autumn Semester 2013

Course Description

Machine learning algorithms are data analysis methods which search data sets for patterns and characteristic structures. Typical tasks are the classification of data, automatic regression and unsupervised model fitting. Machine learning has emerged mainly from computer science and artificial intelligence, and draws on methods from a variety of related subjects including statistics, applied mathematics and more specialized fields, such as pattern recognition and neural computation. Applications are, for example, image and speech analysis, medical imaging, bioinformatics and exploratory data analysis in natural science and engineering:

This course is intended as an introduction to machine learning. It will review the necessary statistical preliminaries and provide an overview of commonly used machine learning methods. Further and more advanced topics will be discussed in the course Statistical Learning Theory, held in the spring semester by Prof. Buhmann.

News

Date What?
30.01.14
  • Small change to problem 2 in Homework 4: c < 0.5 in the cost function.
26.01.14
15.01.14
  • Fixed typo in solution to Problem 3 of Homework 6: indexing in denominators.
14.01.14
09.01.14
  • Added note about stochastic gradient descent to Solution 4.
08.01.14
  • Q&A session on Friday, January 24th from 14:00 - 16:00 in NO C60.
  • Send in your questions to iml2013@inf.ethz.ch until Jan. 20th
06.01.14
06.01.14
  • Fixed typo in solution to Problem 1 of Homework 4: Derivative is w.r.t. wT.
9.12
3.12
3.12
  • No class on 9.12. and 10.12. Tutorials will take place as usual.
25.11
18.11
  • Added proofs for some closure properties of Multivariate Normal Distributions: Exercise 4*.
30.10
20.10
02.10
  • The group registration web site is up and running: http://project.las.ethz.ch/register
  • You need to be in the ETH network to access the site.
  • Please register in groups of 3 and use your main ETH E-Mail address for registering (username@student.ethz.ch for BSc/MSc students).
  • To find other team members, you can use the forum.
01.10
  • The topic of the tutorial this week is "efficient Matlab programming". If possible, please bring a laptop with Matlab already installed.
20.09
  • New information about the grading: The total grade will be determined by the exam grade (70%) and the project grade (30%).
19.09
  • New tutorial slot: Friday 13-15 in CHN G46
27.08
  1. The first lecture will take place on Tuesday 17.09
  2. The tutorial classes will start as of the second week of the semester. Please note that all three sessions are identical.

General Information

VVZ Information: See here.

Time and Place

* All tutorial sessions are identical, please only attend one session.

Syllabus

Calendar Week Lecture Topics Lecture Slides Tutorial Slides Exercise Sheets & Solutions
38 Introduction to Machine Learning [pdf]
39 Regression [pdf]
40 Cross-validation
Perceptron		 [pdf]
41 SVMs [pdf]
42 Nonlinear SVMs
Kernels
k-NN 		[pdf]
43 Kernels; kernelized
linear regression;
Feature selection 		[pdf]
44 Sparsity,
multiclass,
structured prediction		 [pdf]
45 Probabilistic
modeling
Logistic regression		 [pdf]
46 Bayesian learning,
Gaussian processes		 [pdf]
47 Ensemble methods [pdf][videos]
48 Discriminative vs.
Generative models		 [pdf]
49 Clustering
k-Means,GMMs
 [pdf]
50 No class
51 Dimension reduction
(K)-PCA,LLE,MDS 		[pdf]
* Q & A

Some of the material is password protected, send an email to iml2013@inf.ethz.ch to obtain it.

Exercises

The exercise problems will include theoretical and programming problems. Please note that it is not mandatory to submit solutions, a Testat is not required in order to participate in the exam. We will publish exercise solutions after one week.

If you choose to submit:
  • Send a soft copy of the exercise to iml2013@inf.ethz.ch. This can be latex, but also a simple scan or even a picture of a hand-written solution.
  • Please do not submit hard copies of your solutions.

Project

Part of the coursework will be a project, carried out in groups of up to 3 students. The goal of this project is to get hands-on experience in machine learning tasks. The project grade will constitute 30% of the total grade. More details on the project will be given in the tutorials.

Exam

The mode of examination is written, 120 minutes length. The language of examination is English. As written aids, you can bring two A4 pages (i.e. one A4 sheet of paper), either handwritten or 11 point minimum font size. The written exam will constitute 70% of the total grade.

Resources

Java Applets

MATLAB Demos

GPO Videos from Lecture 10

Old Exams

Important: Don't expect the exam to cover the same topics as previous exams. Exam material is everything that has been discussed in the lecture and tutorials, the lecture and tutorial slides and the exercises.

Text Books

  • C. Bishop. Pattern Recognition and Machine Learning. Springer 2007.
    This is an excellent introduction to machine learning that covers most topics which will be treated in the lecture. Contains lots of exercises, some with exemplary solutions. Available from ETH-HDB and ETH-INFK libraries.
  • R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley & Sons, second edition, 2001.
    The classic introduction to the field. An early edition is available online for students attending this class, the second edition is available from ETH-BIB and ETH-INFK libraries.
  • T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2001.
    Another comprehensive text, written by three Stanford statisticians. Covers additive models and boosting in great detail. Available from ETH-BIB and ETH-INFK libraries.
    A free PDF version (second edition) is available online
  • L. Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2004.
    This book is a compact treatment of statistics that facilitates a deeper understanding of machine learning methods. Available from ETH-BIB and ETH-INFK libraries.

Matlab

  • The official Matlab documentation is available online at the Mathworks website.
  • If you have trouble accessing Matlab's built-in help function, you can use the online function reference on that page or use the command-line version (type help <function> at the prompt). There are several primers and tutorials on the web, a later edition of this one became the book Matlab Primer by T. Davis and K. Sigmon, CRC Press, 2005.

Discussion Forum

We maintain a discussion board at the VIS inforum. Use it to ask questions of general interest and interact with other students of this class. We regularly visit the board to provide answers.

Contact