Introduction to Machine Learning (2022)

Introduction to Machine Learning

The course will introduce the foundations of learning and making predictions from data. We will study basic concepts such as trading goodness of fit and model complexity. We will discuss important machine learning algorithms used in practice, and provide hands-on experience in a series of course projects. VVZ Information is available here.
News
  • 16.05.2022 On Wednesday 18.05. the Q&A session will be about the project tasks 2 and 3. If you have specific problems or questions about your code/approach, the best place to ask is on Moodle. There, you can include your code, so that the project team, as well as other students, can help you more efficiently. We have already made two posts, describing a possible solution for task 2 (here) and task 3 (here) on Moodle. You can follow-up further questions there as well.
  • 13.04.2022 There will be no classes in the Easter break (from Friday 15.04. till Monday 25.04., both inclusive). The tutorial on 15.04. will not take place. Instead, we upload a recorded session, reviewing homework 3. Moreover, on Tuesday 26.04. there will be a PyTorch tutorial at the time of the lecture (in the same course hall).
  • 30.03.2022 The project information has been updated (CET has changed to CEST). Moreover, on 30.03.2022, the Q&A session will be about the Task 1 of the projects.
  • 08.03.2022 All students are added to the demos Gitlab repository. See the “Demos” section for more information.
  • 02.03.2022 On the week of 7.3.22, instead of the lectures, there will be a probability theory review on Tuesday and an in-class Q&A on Wednesday. The place is the same as the lectures. On the Friday of that week, there will be a Python Tutorial, and there will be no Zoom Q&A.
  • 01.03.2022 Homework 1 is available. You can download the PDF below (in Tutorials section). There is an accompanying Moodle quiz, where you can test your solutions and get immediate feedback. The quizzes and the homework are not graded.
  • 23.02.2022 Project information is updated.
  • 21.02.2022 Project information is updated.
  • 17.02.2022 As per new ETH masterplan, wearing a mask is obligatory if you attend the lectures in-person.
  • 13.02.2022 Due to the large number of registrations for this course, the in-person attendance (in the main course hall) is limited. Students with an odd Legi number can attend the odd weeks, and students with even Legi numbers can attend the even weeks, both for the lectures (main hall) and tutorials. Attendance to the extra room (ETF E1) is possible for those with odd Legi on even weeks and vice versa (only for lectures, there is no extra room for tutorials). The priority is with those who are already enrolled, and Students in the waiting list are kindly requested to follow the lectures online.
  • 10.02.2022 In the first week’s tutorial session (25 February), we will offer a review session of required background material for the course. This will include a short recap of linear algebra, multivariate analysis and probability theory.
  • 10.02.2022 Due to ETH Covid-19 regulations, only students with a valid Covid certificate (COVID Certificate app, valid ETH CovMass test, or foreign vaccination certificates accepted by ETH) and ETH student card are allowed to enter the classroom, both for lectures and tutorials. Please bring them to class.
  • 10.02.2022 The first lecture is on 22 February and the first tutorial is on 25 February.
  • 10.02.2022 Welcome to the course Introduction to Machine Learning!

FAQ
  1. If this course is compulsory for your study program (Kernfach), you are able to register irrespective of the waiting list. Please allow some time for the transfer from the waiting list.
  2. The project is a mandatory part of the examination. Without achieving a passing grade (4), you are not allowed to do the final examination.
  3. PhD students attending the class and willing to get a grade, should do the projects and the exam. We do not provide a special course this semester.
  4. Distance examination is allowed but you need to file an official request via study administration. We do not handle these requests.
  5. Attendance at tutorials, Q&As, and lectures is not mandatory.
  6. For programming background, we recommend knowing Python. For those without experience in it, check out this Python tutorial.
  7. Lecture slides, exercises, lecture notes, Q&A Zoom calls and the recordings of the Q&A are password protected. To obtain the password you need to be inside the ETH network or use the ETH VPN and click here. Check here to learn how to establish a VPN connection.

Lectures
Lectures will be on Tuesdays and Wednesdays, 2 pm to 4 pm, in ETA F5 with a simultaneous video screening in ETF E1. Link to the live stream is also available, allowing following the lectures from home. If you follow the live stream, you can ask questions via the course channel on EduApp.
All lectures will be recorded, and the recording will be made available within a day after the lecture in the ETH Video Portal and below.
The first lecture will take place on Tuesday, 22 February 2022, and the last one will be on Wednesday, 1 June 2022. It is not mandatory to attend the lectures. However, if you want to attend the lectures physically, you are required to provide a Covid-19 certificate per ETH policy. In addition to providing a certificate, ETH requires you to wear a mask during the class.
Date Topic Preliminary[Final] Recording
Week 1
Tue 22.2. Introduction Preliminary [Final] Recording
Wed 23.2. Linear Regression Preliminary [Final] Recording
Week 2
Tue 01.3. Optimization Preliminary [Final] Recording
Wed 02.3. Optimization2 Preliminary [Final] Recording
Week 2¾
Tue 08.3. Probability Theory Review Final Recording
Wed 09.3. In-class Q&A - Recording
Week 3
Tue 15.3. Model selection Preliminary [Final] Recording
Wed 16.3. Regularization Preliminary [Final] Recording
Week 4
Tue 22.3. Classification Preliminary [Final] Recording
Wed 23.3. Classification 2 Preliminary [Final] Recording
Week 5
Tue 29.3. Kernels Preliminary [Final] Recording
Wed 30.3. Kernels 2 Preliminary [Final] Recording
Week 6
Tue 05.4. Neural Networks Preliminary [Final] Recording
Wed 06.4. Neural Networks 2 Preliminary [Final] Recording
Week 7
Tue 12.4. Neural Networks 3 Preliminary [Final] Recording
Wed 13.4. Convolutional Networks Preliminary [Final] Recording
Easter Break
🐰
Week 8
Tue 26.4. PyTorch Tutorial JN1, JN2, JN3, Slides Recording
Wed 27.4. Clustering Preliminary [Final] Recording
Week 9
Tue 03.5. Dimensionality Reduction I Preliminary [Final] Recording
Wed 04.5. Dimensionality Reduction II Preliminary [Final] Recording
Week 10
Tue 10.5. Statistical Perspective Preliminary [Final] Recording
Wed 11.5. Statistical Perspective II Preliminary [Final] Recording
Week 11
Tue 17.5. Bayesian Decision Preliminary [Final] Recording
Wed 18.5. Generative Modeling Preliminary [Final] Recording

Lecture Notes
New in this semester, we provide a detailed manuscript that contains the most important mathematical background needed for understanding the course. These notes will also serve as a reference for the lectures and set up the notation and needed theorems, definitions, and concepts. The manuscript is not final and will be updated and expanded with notes for some of the lectures as the course progresses, so please check back regularly.
Table of Notations Last version: 23 Feb 2022
Preliminaries Part Last version: 23 Feb 2022
Tutorials
Tutorials will be held on Fridays, 2 pm to 4 pm in ML D28. Live streaming is also available and you can follow the tutorials from home. Similar to the lectures, you will be able to ask questions via the course channel on the EduApp. The tutorial will be recorded and the recording will be made available after the tutorial on ETH Video Portal. The first tutorial will take place on Friday, 25 February 2022, and the last tutorial will take place on Friday, 3 June 2022. It is not mandatory to attend the tutorials. Similar Covid-related regulations apply to the tutorials.
Date Topic Material Recording Homework/Solution
Fri 25.2. Math Recap Notes Recording* [Part 1, Part 2] Homework 1
Fri 04.3. Regression and Optimization Notes, Code Recording
Fri 11.3. Data Analysis with Python Notes, Code Recording Solution 1
Fri 18.3. Review of Homework 1 Notes Recording Homework 2
Fri 25.3. Classification and Model Selection Notes1,Notes2 Recording
Fri 01.4. Review of Homework 2 Notes Recording Solution 2
Fri 08.4. Neural Networks Notes, Code Recording Homework 3
Fri 15.4. No Class Solution 3
Homework 4
Fri 29.4. ConvNets and Clustering Notes1,Notes2, Code Recording Solution 4
Fri 06.5. ConvNets and Clustering Notes3,Notes4, Notes5 Recording Homework 5
Fri 13.4. Statistical Notes Recording Solution 5
* This is a re-iteration of the tutorial, recorded at home. Due to technical problems, the first tutorial was not properly recorded.
Q&A Sessions
Q&A sessions (virtual office hours) will be held on Wednesdays, 5 pm to 6 pm virtually on Zoom. The Q&A sessions are an informal opportunity to ask questions about the course. We may use some Q&A sessions for giving more information about the projects. The Q&A session will be recorded and the recording will be made available after the Q&A session. You will be able to ask questions via the native Zoom chat or by speaking out, if requested to do so. The first Q&A session will take place on Wednesday, 23 February, and the last Q&A session will take place on Wednesday, 1 June. It is not mandatory to attend the Q&A sessions. To obtain the password for joining Q&A see FAQ point 7.
Date Topic Recording
Wed 23.2. Administration Recording [Slides]
Wed 02.3. - Recording
Wed 16.3. - Recording
Wed 23.3. - Recording
Wed 30.3. Project Task 1 Recording [Part I, Part II]
Wed 06.4. - Recording
Wed 13.4. - – (nothing happened)
Wed 27.4. - Recording
Wed 04.5. - Recording
Wed 11.5. - Recording
Contact
Instructors Prof. Andreas Krause and Prof. Fanny Yang
Head TA Mohammad Reza Karimi
Assistants Alexander Stutz, Alexandru Tifrea, Anastasia Makarova, Andisheh Amrollahi, Anver Khusainov, Arjun Bhardwaj, Chris Wendler, David Lindner, Eugene Bykovetc, Gizem Yüce, Hugo Yèche, Jacob Clarysse, Jonas Rothfuss, João de Borges Sá Carvalho, Kai Lion, Lars Lorch, Lenart Treven, Luca Corinzia, Matteo Turchetta, Mojmir Mutný, Olga Mineeva, Oliver Holl, Panagiotis Misiakos, Parnian Kassraie, Riccardo Zuliani, Robin Geyer, Scott Sussex, Sebastian Curi, Seyedmorteza Sadat, Shyam Sundhar Ramesh, Stefan Stark, Vignesh Ram Somnath, Xia Li, Xinyu Sun, Ya-Ping Hsieh
Mailing List Please use Moodle for questions regarding course material, organization and projects. If you need to contact the Head TA or the lecturer directly, please send an email to introml22-info@inf.ethz.ch. Please think twice before you send an email though and make sure you read all information here carefully.
Lectures
Tue 14-16 ETA F5 ETF E1 (via video) and Live stream
Wed 14-16 ETA F5 ETF E1 (via video) and Live stream
Tutorials
Fri 14-16 ML D28 Live stream
Questions & Answers
Wed 17-18 Online Zoom

Code Projects
The code projects will require solving machine learning problems with methods taught within the course. You are allowed to work in groups of 1 – 3 students, but it is your responsibility to find a group. You can search for teammates by posting on Moodle. Assignments will require handing in the solution code as well as a short report. In particular, there will be 6 code assignments. The first assignment is ungraded and will allow you to become familiar with our code submission workflow. The remaining projects are graded (pass/fail) and mandatory for passing the Introduction to Machine Learning course. Details regarding the projects can be found here.
Demos
Demos are based on jupyter notebook (with Python 3). Please look at this intro for installing and running instructions. We recommend that you create a conda environment to maintain the code base. The demos are hosted at GitLab. You should have been automatically added to the repository as a Reporter. This gives you the opportunity to clone the project on your machine and run the demos. If you sill cannot access the repository, please send an email to introml22-info@inf.ethz.ch with the subject “[IML 2022] Gitlab access” and include your nethz in the email.
Performance Assessment
70% session examination, 30% code project; the final grade will be calculated as weighted average of both these elements. As a compulsory continuous performance assessment task, the project must be passed on its own. The coding projects are an integral part (60 hours of work, 2 credits) of the course. Participation is mandatory. To be eligible for the examination of Introduction to Machine Learning (252-0220-00L), you need to pass the code projects, i.e., attain an overall project grade of 4 or higher. Students who do not pass the project are required to de-register from the exam and will otherwise be treated as a “no show”.
For the final exam, you can bring two A4-pages (that is, one A4-sheet of paper), either handwritten or 11 point minimum font size. A simple non-programmable calculator is allowed during the exam. The exam will be multiple choice. Here you can find an example of the question types as well as how to fill out the answer sheet to guarantee successful automatic grading.


Previous Exams:
Exam 2015, Exam 2016, Exam 2017, Exam 2018, Exam 2019, Exam 2020, Sol 2020, Exam 2021, Sol 2021
Other Resources