IntroductionThe course will introduce the foundations of learning and making predictions from data. We will study basic concepts such as trading goodness of fit and model complexity. We will discuss important machine learning algorithms used in practice, and provide hands-on experience in a series of course projects. VVZ Information is available here.
- [14.03.2023] For students who do not want to solve projects on their personal laptops, check out this guide by Vukasin Bozic explaining how to use Euler, the scientific computer clusters of ETH. More information regarding Euler is available here.
- [07.03.2023] The Projects and FAQ section have been updated. Importantly there will be a project introduction session on each Wednesday the project is released. And a project solution session on the Wednesday the week after the project deadline. Both to be held at the Q&A session 17(sharp)-18.
- [02.03.2023] Project 0 is online. It is ungraded and aims to help you familiarize the project workflow.
- [27.02.2023] Regarding the Q&A on March 1st: Introduction to Python for Data Science (jupyter, numpy, pandas, seaborn, matplotlib). Please check the README before the tutorial. This is for installing all the necessary libraries so you can follow along during the tutorial.
- [22.02.2023] During the Q&A session on March 1st, Olga Mineeva and Stefan Stark will present a Python libraries introduction covering Numpy and Pandas.
- [10.02.2023] Welcome to the course Introduction to Machine Learning!
- Q: This course is compulsory for my program, but I cannot register. What should I do?
A: If this course is compulsory for your study program (Kernfach), you are able to register irrespective of the waiting list. Please allow some time for the transfer from the waiting list.
- Q: Is physical attendance to the didactic activities mandatory?
A: Physical attendance at lectures, tutorials and Q&As is not mandatory but strongly encouraged.
- Q: How can I access the materials on the website?
A: Lecture slides, exercises, lecture notes, and recordings of the Q&A are password protected. To obtain the password you need to be inside the ETH network or use the ETH VPN and click here. Check here to learn how to establish a VPN connection.
- Q: What programming knowledge is required for this course?
A: For the programming background, we recommend knowing Python. For those without experience in it, check out this Python tutorial.
- Q: Will the projects contain boilerplate codes?
A: Yes. There will be some code to guide you through the steps.
- Q: What are the most useful libraries to learn for the projects?
A: numpy, sklearn, pandas, torch.
- Q: Is there a preferred library for the deep learning section?
A: We will give a tutorial on pytorch and use pytorch in the boilerplate code for the projects. However, the use of other libraries is not disallowed.
- Q: Am I eligible to take the exam if I fail some of the projects?
A: You must receive an average grade of 4 or above to be eligible to take the exam. Failure in a project (not passing an easy baseline) means a grade of 2. However, as long as the average is above 4 (you can achieve this by getting a 6 in other projects) you can take the exam. If you do not do so we will ask you to deregister from the exam.
- Q: I did the projects last year (spring semester) and didn’t take or failed the exam in the spring or following autumn semester (or both). Do they count towards this year’s projects?
A: No, they do not. Projects can be submitted only in the spring semester and they make you eligible to take the exam in the same (spring) semester and the autumn semester following. If you do not take the exam or fail the exam in these two sessions, you have to enroll in the class next year and redo them again.
- Q: I am an attendance-only doctoral student what do I need to do?
A: You need to enroll in the class and complete the projects to get your attendance certified. Your department decides how many credits you get for just attendance. Most departments require you to take the exam as well to be eligible for credits . Please contact your department study administrator for details.
- Q: I am a Ph.D. student, can I not do the projects or the exam but still get a grade?
A: Most Ph.D. students that would like to get credits for the class need to do the projects and the exam. A minority of departments allow you to get credits without taking an exam. See the question above.
- Q: Is distance examination allowed?
A: Distance examination is allowed, but you need to file an official request via study administration. We do not handle these requests.
LecturesLectures will be on Tuesdays and Wednesdays, 14:15 to 16:00, in ETA F5 with a simultaneous video screening in ETF E1. You can ask questions in person or via the course channel on EduApp.
All lectures and tutorials will be recorded, and the recording will be made available within a day after the lecture in the ETH Video Portal and below.
The first lecture will take place on Tuesday, 21 February 2023.
|Tue 21.02.||Introduction||Preliminary [Final]||Recording|
|Wed 22.02.||Linear Regression||Preliminary [Final]||Recording|
|Tue 28.02.||Optimization||Preliminary [Final]||Recording|
|Wed 01.03.||Optimization & Nonlinear Features||Preliminary [Final]||Recording|
|Tue 07.03.||Model Selection||Preliminary [Final]||Recording|
|Wed 08.03.||Bias-Variance Tradeoff & Regularization||Preliminary [Final]||Recording|
|Tue 14.03.||Classification||Preliminary [Final]||Recording|
|Wed 15.03.||Classification II||Preliminary [Final]||Recording|
|Tue 21.03.||Classification & Kernel Methods||Preliminary [Final]||Recording|
|Wed 22.03.||Kernel & Other Methods||Preliminary [Final]||Recording|
|Tue 28.03.||Neural Networks I||Preliminary [Final]||Recording|
|Wed 29.03.||Neural Networks II||Preliminary [Final]||Recording|
|Tue 04.04.||Neural Networks III||Preliminary [Final]||Recording|
|Wed 05.04.||Neural Networks IV||Preliminary [Final]||Recording|
Lecture NotesWe provide a detailed manuscript that contains the most important mathematical background needed for understanding the course. These notes will also serve as a reference for the lectures and set up the notation and needed theorems, definitions, and concepts. The manuscript is not final and will be updated and expanded with notes for some of the lectures as the course progresses, so please check back regularly.
|Table of Notations||Last version: 29 June 2022|
|Preliminaries, Linear Regression, Optimization, and parts of Classification||Last version: 29 June 2022|
TutorialsTutorials will be held on Fridays, 14:15 to 16:00 ETA F5 with a simultaneous video screening in ETF E1. Similar to the lectures, you will be able to ask questions via the course channel on the EduApp.
The tutorial will be recorded and the recording will be made available after the tutorial on ETH Video Portal.
The first tutorial will take place on Friday, 24 February 2023.
|Fri 24.02.||Math Recap||Notes||Recording||–|
|Fri 03.03.||Linear Regression & Optimization||Materials||Recording||Homework 1|
|Fri 10.03.||Review of Homework 1||Slides||Recording||Solution 1|
|Fri 17.03.||Classification||Materials||Recording||Homework 2|
|Fri 24.03.||-||-||Recording||Solution 2|
Q&A SessionsQ&A sessions (virtual office hours) will be held on Wednesdays, 17:00 to 18:00 virtually on Zoom. The Q&A sessions are an informal opportunity to ask questions about the course. We may use some Q&A sessions for giving more information about the projects. You will be able to ask questions via the native Zoom chat or by speaking out, if requested to do so. It is not mandatory to attend the Q&A sessions.
The Q&A session will be recorded and the recording will be made available after the Q&A session.
The first Q&A session will take place on Wednesday, 22 February.
|Wed 01.03.||Python Tutorial||Recording|
|Wed 08.03.||Linear Regression & Optimization||Recording|
|Wed 15.03.||Project 1 Introduction||Recording|
|Wed 22.03.||Classification & Kernel Methods||Recording|
|Instructors||Prof. Andreas Krause and Prof. Fanny Yang|
|Head TA||Andisheh Amrollahi|
|Pragnya Alatur, Parnian Kassraie, Lars Lorch, Lenart Treven, Bhavya Sukhija, David Lindner, Yarden As, Scott Sussex, Hugo Yeche, Vignesh Ram, Vukasin Bozic, Charlotte Bunne, Cynthia Chen, Sonali Adnan, Mojmir Mutny, Zhengrong Lang, Gavrilopoulos Georgios, Phillip Scherer, Xinyu Sun, Zhiyuan Hu, Zhenru Jia, Rajesh Sharma, Giorgia Racca, Angeline Pouget, Yuhao Mao, Thomas Out, Javier Abad Martinez, Piersilvio De Bartolomeis, Alexandru Tifrea, Viacheslav Borovitskiy, Jannis Bolick, Stefan Stark, Olga Mineeva, Harun Mustafa, Daniel Yang|
|Please use Moodle for questions regarding course material, organization and projects. If you need to contact the Head TA or the lecturer directly, please send an email to firstname.lastname@example.org. Please think twice before you send an email though and make sure you read all information here carefully.|
|Tue 14-16||ETA F5||ETF E1 (via video)|
|Wed 14-16||ETA F5||ETF E1 (via video)|
|Fri 14-16||ETA F5||ETF E1 (via video)|
Questions & Answers
Code ProjectsThe code projects will require solving machine learning problems with methods taught within the course. Projects will require handing in the solution code as well as a short report. You are allowed to work in groups of 1 – 3 students, but it is your responsibility to find a group. You can search for teammates by posting on Moodle.
In particular, there will be 5 code projects. The first project is ungraded and will allow you to become familiar with our code submission workflow. The remaining projects are graded (pass/fail) and mandatory for passing the course.
Following is a timetable of the projects. Details regarding the projects can be found here. If you are having technical issues please send an email to email@example.com.
|Project||Release Date||End Date||Weight on Project Grade|
|Project 0 (dummy)||Wed, 01.03.2023 08:00||-||0|
|Project 1a&b||Wed, 15.03.2023 08:00||Wed, 29.03.2023 12:00||0.25 (0.125 each)|
|Project 2||Wed, 29.03.2023 08:00||Wed, 26.04.2023 12:00||0.25|
|Project 3||Wed, 26.04.2023 08:00||Wed, 10.05.2023 12:00||0.25|
|Project 4||Wed, 10.05.2023 08:00||Wed, 31.05.2023 12:00||0.25|
There will be a presentation introducing the project in the Q&A session on the same Wednesday each project gets released. Also, there will be a project solution session in the Q&A session the week after the deadline of a project. Hence there are 8 sessions for all 4 graded projects in total.
DemosThe demos are hosted at GitLab. Demos are based on jupyter notebook (with Python 3.9). Please look at this intro for installing and running instructions. We recommend that you create a conda environment to maintain the code base. You should have been automatically added to the repository as a Reporter. This gives you the opportunity to clone the project on your machine and run the demos. If you still cannot access the repository, please send an email to firstname.lastname@example.org with the subject “[IML 2023] GitLab access” and include your nethz in the email.
Performance Assessment70% session examination, 30% code project; the final grade will be calculated as weighted average of both these elements. As a compulsory continuous performance assessment task, the project must be passed on its own. The coding projects are an integral part (60 hours of work, 2 credits) of the course. Participation is mandatory. To be eligible for the examination of Introduction to Machine Learning (252-0220-00L), you need to pass the code projects, i.e., attain an overall project grade of 4 or higher. Students who do not pass the project are required to de-register from the exam and will otherwise be treated as a “no show”.
For the final exam, you can bring two A4-pages (that is, one A4-sheet of paper), either handwritten or 11 point minimum font size. A simple non-programmable calculator is allowed during the exam. The exam will be multiple choice. Here you can find an example of the question types as well as how to fill out the answer sheet to guarantee successful automatic grading.
Previous Exams: Exam 2015, Exam 2016, Exam 2017, Exam 2018, Exam 2019, Exam 2020, Sol 2020, Exam 2021, Sol 2021
- Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong Mathematics for Machine Learning. Cambridge University Press, 2020.
- K. Murphy. Machine Learning: a Probabilistic Perspective. MIT Press, 2012.
- C. Bishop. Pattern Recognition and Machine Learning. Springer, 2007. (optional)
- T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2001.
- L. Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2004.
- G. James., D. Witten and et al. An Introduction to Statistical Learning. Springer, 2021.