Probabilistic Artificial Intelligence (2020)

Probabilistic Artificial Intelligence (Fall ’20)

How can we build systems that perform well in uncertain environments and unforeseen situations? How can we develop systems that exhibit “intelligent” behavior, without prescribing explicit rules? How can we build systems that learn from experience in order to improve their performance? We will study core modeling techniques and algorithms from statistics, optimization, planning, and control and study applications in areas such as sensor networks, robotics, and the Internet. The course is designed for upper-level undergraduate and graduate students. VVZ information is available here.
News
  • For the students taking the summer exam: a video recording of the exam review session is available below.
  • The exam review session will be held at 29 Jan at 10AM. There will be a video recording for those of you who still can not attend.
  • There will be no lectures on December 18th, tutorial session will be held however.
  • As the lectures got a bit asynchronized with the project’s topic, we updated the deadlines. Note, that the deadline for project 2 was prolonged by a week, and is now November 12, 12.00.
  • All lectures will be held online as of now, via Zoom. Here is the link to the webinar. To obtain the lecture password you need to be inside the ETH network and click here.
  • Project 1 is re-opened till Oct 27, 12:00, please use this opportunity to submit your solutions and reports.
  • Dates for the projects are updated. You may find them on the Project information handout.
  • Zoom tutorials are recorded. The videos are password protected; to obtain the tutorials password you need to be inside the ETH network and click here.
  • The link to Zoom classroom for tutorial sessions is updated.
  • The first lecture starts on 18.09, and the exercise sessions begin on 24.09.
  • During the lecture, for questions from the remote audience, we’ll use the ETH EduApp. There is a course channel for PAI 2020, where you can post questions (also anonymous if preferred), and the TA present in class will moderate the incoming questions.
  • The tutorial will be once a week and online only. As no physical office hours are allowed, one extra hour after the tutorials will be added for these purposes.
  • The lectures will mostly be given in a lecture hall with limited attendance (at most 50% of lecture hall capacity). It will be possible to join remotely via zoom with acccess to slides, whiteboard, and speaker camera. Students can interact, e.g. ask questions, physically as well as digitally. The lectures will be recorded via zoom’s recording functionality.
  • The files are password protected. To obtain the password you need to be inside the ETH network and click here. To establish a VPN connection click here
  • The lecture video recordings will available at ETH Videoportal.
Syllabus
Date Topic References & Readings Tutorial Homework Solution
Sep 18 Introduction & Probability [Unannotated Slides] AI A Modern Approach: Ch. 1 & Ch. 13.1-5 • Mathematics for ML: Ch. 6 , 8 , 9 - Hw1 Solution1
Sep 25 Bayesian Linear Regression [Unannotated Slides] AI A Modern Approach: Ch. 14.1 , 14.4 • GPML: Ch. 2 to 2.1.1 On Gaussians [Recording] - –
Oct 2 Gaussian Processes [Unannotated Slides] GPML Ch. 2: 2.1.1-2.3 On Hw1 [Recording] Hw2 Solution2
Oct 9 Gaussian Processes II [Unannotated Slides] GPML Ch. 2: 2.1.1-2.3 , Ch. 4: up to 4.2 • A Unifying View of Sparse Approximate Gaussian Process Regression • Random Features for Large-Scale Kernel Machines On HW2 [Recording] - –
Oct 16 Variational Inference [Unannotated Slides] GPML Ch. 3 (3.1-3.4) • Black Box Variational Inference • Stochastic Variational Inference using reparametrization • Scalable Variational Gaussian Process Classification • Survey on Monte Carlo Gradient Estimation Sparse Gaussian Processes & Intro to VI [Recording part 1, Recording part 2] Hw3 Solution3
Oct 23 Markov Chain Monte Carlo [Unannotated Slides] Bishop: Ch. 11 up to 11.3 • Bayesian Learning via Stochastic Gradient Langevin Dynamics • Sampling can be faster than optimization • Consistency and Fluctuations For Stochastic Gradient Langevin Dynamics • Stochastic Gradient Hamiltonian Monte Carlo On Hw3 [Recording] - –
Oct 30 Bayesian Deep Learning [Unannotated Slides] Recording: Part I, Part II Guo et al. On Calibration of Modern Neural Networks • Blundell et al. Weight Uncertainty in Neural Networks • Kendall & Gal. • Gal & Ghahramani Dropout as a Bayesian Approximation Sampling and Markov Chains [Class Notes][Recording] Hw4 Solution4
Nov 6 Active Learning [Unannotated Slides] Recording: Part I , Part II Srinivas et al. Gaussian Process Optimization in the Bandit Setting Sampling, Bayesian Networks [Recording] - –
Nov 13 Markov Decision Processes [Unannotated Slides] Recording: Part I, Part II Russell & Norvig: Ch. 17.1-17.2, 17.4 • Sutton & Barto: Ch. 3, 4-4.4. Bayesian Learning & Active Optimization [Recording] Hw5 Solution5
Nov 20 Reinforcement Learning [Unannotated Slides] Recording: Part I , Part II Russell & Norvig: Ch. 22-22.3 • Sutton & Barto: Ch. 6.1-6.3, 6.5 MDPs and Hw5 [Recording] - –
Nov 27 Reinforcement Learning II [Unannotated Slides] Recording: Part I, Part II Sutton & Barto Ch. 9, 13.1-13.4 • Russell & Norvig: Ch. 22.1-22.5 • Szepesvari: Ch. 2.2, 3.3 • Mnih et al. Human Level Control through Deep Reinforcement Learning • Van Hasselt et al. Deep reinforcement learning with double Q-Learning RL [Recording] Hw6 Solution6
Dec 4 Reinforcement Learning III [Unannotated Slides] Recording: Part I, Part II Sutton & Barto Ch. 13 • Szepesvari Ch. 3.4 • A3C: Mnih et al. Asynchronous Methods for Deep Reinforcement Learning • TRPO: Schulman et al. Trust Region Policy Optimization • PPO: Schulman et al. Proximal Policy Optimization Algorithms • DDPG: Lillicrap et al. Continuous Control with Deep Reinforcement Learning • SAC: Haarnoja et al. Soft Actor Critic: Off-Policy Maximum Entropy Reinforcement Learning with a Stochastic Actor Function Approximation and Policy Gradient [Recording] - –
Dec 11 Model Based Deep RL [Unannotated Slides] Recording: Part I, Part II Chua et al. Deep Reinforcement Learning in a Handful of Trials, NeurIPS 2018 • Curi et al. Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning, NeurIPS 2020 • Deisenroth, Rasmussen. PILCO: A Model-Based and Data-Efficient Approach to Policy Search, ICML 2011 • Berkenkamp et al. Safe Model-Based Reinforcement Learning with Stability Guarantees, NeurIPS 2017 • Koller et al. Learning-Based Model Predictive Control for Safe Exploration, CDC 2018 Maximum Entropy and Soft Actor Critic , Optimization and Gradient Estimation for Reinforcement Learning [Recording] Hw7 Solution7
Dec 18 - - [Recording] - –
Jan 29 - - Exam Review I, II, III [Recording] - –
Contact
Instructor Prof. Andreas Krause
Head TA Anastasia Makarova
Assistants Andisheh Amrollahi, Ilija Bogunovic, Zalán Borsos, Charlotte Bunne, Sebastian Curi, Gideon Dresdner, Vincent Fortuin, Carl Johann Simon Gabriel, Johannes Kirschner, Matthias Hüser, Mojmír Mutný, Mohammad Reza Karimi, Max Paulus , Jonas Rothfuss, Stefan Stark, Olga Mineeva, Hugo Yeche, Amir Joudaki, Luka Rimanic, Laura Manduchi, Zhao Zhikuan, Immer Alexander, Ya-Ping Hsieh, Noman Ahmed Sheikh, Parnian Kassraie, David Lindner, Scott Sussex, Cristina Pinneri
Piazza If you have any questions, please use the Piazza Course Forum.
Mailing List Please use the Piazza Forum for questions regrading course material, organisation and projects. If this does not work for your request, please use the tutorials to ask questions.
Lectures
Friday 10-12 13-14 Online Zoom
Tutorials
Thursday 16-18 Online Zoom
When entering the webinar, please use your nethz email address (i.e. [name]@ethz.ch or [name]@student.ethz.ch).
Exam
The exam is 120 minutes long. It might take place at a computer. The language of examination is English. As written aids, you can bring one A4 sheet of paper (you can write on both sides), either handwritten or 11 point minimum font size. Please bring your Legi (ID card) for the exam. Please do not use cellphones / tablets in the exam. Simple non-programmable calculators are allowed in the exam. You can find previous exams here: [2019], [2018], [2017], [2016], [2015], [2014], [2013], [2012].
Projects
Code projects will require solving machine learning problems with methods taught within the course. You are allowed to work in groups of 1 – 3 students, but it is your responsibility to find a group. You can search for teammates by posting on Piazza. Assignments will require handing in the solution code as well as a short report. In particular, there will be 5 code assignments. The first project is ungraded and will allow students to become familiar with our code submission workflow. The remaining projects are graded (pass/fail) and mandatory for passing the PAI course. Out of the 4 code projects, we construct the overall grade as follows: project grade = 6 – number of failed projects. For passing the course and being allowed to write the exam, students are required to pass at least 2 out of the 4 assignments. Overall, the projects grade counts 30% towards the final grade in the course. The code projects will be released throughout the semester. You can find the tentative project schedule and further details in the project information sheet [pdf]. The projects can be accessed and submitted on our project sever https://project.las.ethz.ch/. You will need to be in the ETH network or use the VPN to access the server.
Homework
There will be a different homework fortnightly. These will not be corrected and do not count for the final grade. However, we recommend to do them as they will help preparing for the final exam and understanding the concepts.
Demos
You can find the demos as well as a README.md and DOCKERFILE in this [zip file]. We will update this zip file as more demo’s are out! So keep posted. You can also find the demos in this GitLab repository.
Performance Assessment
70% session examination, 30% project; the final grade will be calculated as weighted average of both these elements. As a compulsory continuous performance assessment task, the project must be passed on its own. The exam might take place at a computer. More updates are coming soon.
Text Books
  • S. Russell, P. Norvig. Artificial Intelligence: A Modern Approach (4th edition).
  • C. E. Rasmussen, C. K. I. Williams Gaussian Processes for Machine Learning.
  • Christopher M. Bishop. Pattern Recognition and Machine Learning. [optional]
  • Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction.