Introduction to Machine Learning

Weekly outline

• General
  

  General

  Welcome to the machine learning class!

  Machine learning and data analysis are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analyzed, compared, and implemented.
  
  

  Instructor

  Prof. Pascal Fua
  Computer Vision Laboratory (CVLAB)
  BC 310
  E-mail: pascal.fua@epfl.ch

  Course Times and Locations

  Lectures: Monday 16:15 - 18:00 in SG1
  

  Exercises: Tuesday 17:15 - 19:00 in INF1, INJ218, INM10, INM200, INM202
  

  Office Hours

  If you have any questions please post them in the discussion forum and one of the TAs will answer you.
  

  Final Exam
  

  The final exam will be closed book and will represent 90% of your final grade. You will be allowed ONE hand-written A4 page of notes.
  

  Exam date: to be determined

  Exam room: to be determined
  

  Graded Exercises

  Not all practical sessions will be graded. However, two of them will be, one in the middle of the term and another towards the end. This will count for the remaining 10% of your grade. The dates of the graded exercise sessions are shown in the schedule below.
  

  • GRADED EXERCISE 2 File
  • GRADED EXERCISE #2 SUBMISSION LINK Assignment
  • News forum
  • Discussion Forum
  • Graded Exercise 2 - Solutions File
• Class Schedule
  

  Class Schedule

  
  

  18.02.2019	Course
  19.02.2019	Exercise Session
  25.02.2019	Course
  26.02.2019	Exercise Session
  04.03.2019	Course
  05.03.2019	Exercise Session
  11.03.2019	Course
  12.03.2019	Exercise Session
  18.03.2019	Course
  19.03.2019	Exercise Session
  25.03.2019	Course
  26.03.2019	Exercise Session
  01.04.2019	Course
  02.04.2019	Graded Exercise Session
  08.04.2019	Course
  09.04.2019	Exercise Session
  15.04.2019	Course
  16.04.2019	Exercise Session
  22.04.2019	No Class (Holiday)
  23.04.2019	No Class (Holiday)
  29.04.2019	Course
  30.04.2019	Exercise Session
  06.05.2019	Course
  07.05.2019	Exercise Session
  13.05.2019	Course
  14.05.2019	Graded Exercise Session
  20.05.2019	Course
  21.05.2019	Exercise Session
  27.05.2019	Course
  28.05.2019	Exercise Session
• Grades
  

  Grades

  • 1st Graded Exercise - Results File

    The results of the first graded session which took place on 2.4.2019. The total number of points is 15.

  • Graded Exercise 1 Solutions File

    The solutions of the first graded exercise

  • 2nd Graded Exercise - Results File
• Reference Text Books
  

  Reference Text Books

  C.M. Bishop, Pattern Recognition and Machine Learning, Springer. 2006.

  
  
• Introduction
  

  Introduction

  Introduction to the class and prerequisites

  • Course Introduction File
  • Naive Bayes File
  • Images as vectors File
  • Python and NumPy Primer File
  • Exercise Session 1 File
  • Solutions File
• Nearest Neighbors
  

  Nearest Neighbors

  One of the simplest machine learning techniques.

  • K nearest neighbors File
  • Bishop: 2.5.2 Nearest-neighbour methods URL
  • Exercise Session 2 File
  • Solutions (jupyter notebook) File
  • Solutions (html) File
• Linear Classification
  

  Linear Classification

  Perceptrons and logistic regression.

  • Logistic regression File
  • Bishop: 4.1.7 The Perceptron Algorithm URL
  • Bishop: 4.3.2 Logistic regression URL
  • Optimization File
  • Triggs: Bundle Adjustment — A Modern Synthesis URL
  • Exercise Session 3 File
  • Solutions (jupyter notebook) File
  • Solutions (html) File
  • Exercise Session 4 File
  • Solutions (jupyter notebook) File
  • Solutions (html) File
• From Linear to Non-Linear Classification
  

  From Linear to Non-Linear Classification

  From basic perceptrons to Support Vector Machines and AdaBoost.

  • High-Dimensional Features and Kernels File
  • Proof of Cover's theorem URL
  • Bishop: 7.1 Maximum margin classifiers URL
  • Support Vector Machine vs Logistic Regression URL
  • Boosting File
  • Bishop: 14.3 Boosting URL
  • Trees and Forests File
  • Bishop: 14.4 Tree-based models URL
  • Exercise Session 5 File
  • Solution(jupyter notebook) File
  • Solution(html) File
  • Exercise Session 6 File
  • Solution(notebook) File
  • Solution(html) File
• Neural Nets
  

  Neural Nets

  From simple perceptrons to sophisticated convolutional networks. 

  • Multi-layer perceptrons File
  • Bishop: 5 Neural networks URL
  • Nielsen: Deep learning tutorial URL
  • Kingma: Adaptive Moment Estimation URL
  • Telgarsky: Benefits of depth URL
  • Convolutional networks File
  • AlphaGo File
  • Exercise Session 7 File
  • solution (notebook) File
  • solution (html) File
  • Exercise Session 8 File
  • Exercise Session 8 - WINDOWS File
  • solution (html) File
  • Solutions (notebook) File
• Regression
  

  Regression

  Linear and non-linear regression.

  • Linear regression and least squares File
  • Bishop: 3.1 Linear basis function models URL
  • Non-linear least squares File
  • Fua: Constrained least squares URL
  • Gaussian processes File
  • Bishop: 6.4 Gaussian processes URL
  • Regression Trees File
  • Caruana: An empirical evaluation of supervised learning in high dimensions URL
  • Neural Nets File
  • Exercise Session 9 File
  • Exercise 9 Solution (HTML) File
  • Exercise 9 Solution File
  • Exercise session 10 File
  • Exercise 10 Solution Folder
• Modeling over time
  

  Modeling over time

  Taking the temporal dimension into account

  • Recurrent networks File
• Human Brains
  

  Human Brains

  The Human Brain as a Source of Inspiration

  • Brain networks File
  • Gerstner: Spiking Neuron Models URL
• Summary
  

  Summary

  • Conclusion File