Foundations of Data Science
Weekly outline
-
SWITCHtube Channel
We will not make new recording this year. However you can access the videos from last year. The content is largely the same.
Summary
We discuss a set of topics that are important for the understanding of modern data science but that are typically not taught in an introductory ML course. In particular we discuss fundamental ideas and techniques that come from probability, information theory as well as signal processing.
Content
This class presents basic concepts of Information Theory and Signal Processing and their relevance to emerging problems in Data Science and Machine Learning.
A tentative list of topics covered is:
- Information Measures
- Signal Representations
- Detection and Estimation
- Multi-arm Bandits
- Distribution Estimation, Property Testing, and Property Estimation
- Exponential Families
- Compression and Dimensionality Reduction
- Information Measures and Generalization Error
Materials
- Lecture Notes (Version August 23, 2022) Note: Check for updates on a semi-regular basis.
Additional Material:- T. M. Cover and J. A. Thomas, Elements of Information Theory (Click to get access to the full PDF via the EPFL library). New York: Wiley. Second Edition, 2006.
- T. Lattimore and C. Szepesvari, Bandit Algorithms
Schedule
Classes:
Exercise:- Friday 10:15-12:00 (INM200)
Grading
- If you do not hand in your final exam your overall grade will be NA.
- Otherwise, if we can hold the Midterm Exam (on Thursday, November 17, 2022, 17:15-19:00), your grade will be determined based on the following weighted average: 10% for the Homework, 30% for the Midterm Exam, 60% for the Final Exam.
- If we cannot hold the Midterm Exam, your grade will be determined based on the following weighted average: 10% for the Homework, 90% for the Final Exam.
The Final Exam will take place on Friday, February 3 at 9:15 - 12:15 in SG 0211
-
Sept 22 : General Introduction ; Probability Review
Sept 23 : Information Measures -
Sept 29 : Information Measures
Sept 30 : Information Measures ; Signal Representations -
Oct 6 : Signal Representations
Oct 7 : Signal Representations -
Oct 13 : Signal Representations
Oct 14 : Signal Representations -
Oct 20 : Multi-arm Bandits explore-then-exploit
Oct 21 : Multi-arm Bandits : UCB -
Oct 27 : Multi-arm Bandits
Oct 28 : Multi-arm Bandits ; Distribution Estimation -
Nov 3 : Distribution Estimation
Nov 4 : Distribution Estimation -
Nov 10 : Distribution Estimation and Property Testing
Nov 11 : No class -
Nov 17 : Midterm Exam
Nov 18 : Detection and Estimation -
Nov 24 : Detection and Estimation
Nov 25 : Exponential Families -
Dec 1 : Exponential Families
Dec 2 : Exponential Families -
Dec 8 : Compression
Dec 9 : Compression -
Dec 15 : Compression
Dec 16 : Exploration Bias and Generalization Bounds -
Dec 22 : Generalization Bounds
Dec 23 : Review Session
