Theory and Methods for Reinforcement Learning

Weekly outline

• General

  Collapse all Expand all
  • 

    Announcements Forum
  • 

    Logistics File

    Teaching Assistants:
    

    • Luca Viano (Head TA)
    • Ali Kavis
    • Leello Dadi
    • Thomas Pethick
    • Fatih Sahin
    • Pedro Abranches
      
      Should you have any question, please send a mail to the head TA (Luca Viano, email = luca.viano@epfl.ch) or post a question on the Moodle discussion board.

    
    

• 21 February - 27 February
  • 

    Lecture 1 File

    An overview of the course

• 28 February - 6 March
  • 

    Lecture 2: Dynamic Programming 1 File

    MDPs; value and Q functions; value iteration, policy iteration; operator perspectives

• 7 March - 13 March
  • 

    Lecture 3 File

    Model-free policy-based and value-based methods; Monte Carlo (MC) method and temporal difference (TD) learning.

• 14 March - 20 March
  • 

    Project Proposal Assignment

    A brief description of the project (2-3 pages including references) which includes the following:

    1. the names of the project team members

    2. motivation of the projects

    3. formal description of the problem and the goal

    4. references

    5. software and computational resources you will use

  • 

    Lecture 4 File

    Primal and Dual LP, ALP, ALP with constraint sampling, primal dual methods, REPS.

• 21 March - 27 March
  • 

    Lecture 5 File

    Policy gradient methods

• 28 March - April 3
  • 

    Lecture 6 File

    Policy gradient II : rates, gradient dominance property, distributions mismatch coefficients, natural policy gradient.

• 4 April - 10 April
  • 

    Lecture 7 File

    Policy gradient III: Natural policy gradient convergence bounds

• 11 April - 17 April
  • 

    Lecture 8 File

    Behavioral cloning, imitation learning, inverse reinforcement learning.

• 18 April - 24 April
  • 

    Lecture 9 File

• 2 May - 8 May
  • 

    Lecture 10 File

    Markov Games 2: nonlinear programming, policy gradient

• 9 May - 15 May
  • 

    Lecture 11 File

    Deep RL, Actor Critic methods, DQN

• 16 May - 22 May
  • 

    Lecture 12 File

    Robust RL

• 23 May - 29 May
  • 

    Final Project report Assignment

    Dear all,
    
    please upload for your final report by Friday, Jun 10th at 11:59 PM.

    Please double-check the submission instructions that we uploaded on Moodle during the first week https://moodlearchive.epfl.ch/2021-2022/pluginfile.php/3075502/mod_assign/intro/syllabus-2022.pdf (page 4) 

    In particular, we expect between 6 and 8 pages in the NeurIPS template https://neurips.cc/Conferences/2022/PaperInformation/StyleFiles

    The required structure is 

    1. Abstract
    2. Introduction
    3. Related Work
    4. Approach
    5. Results
    6. Conclusion
    7. References
       

    If you ran experiments, please attach your code as supplementary material, uploading a single zip file containing the main report in pdf format and a folder named supplementary for the attached files.
    It is also possible to upload an Appendix in a separate pdf including it in the same zip file.
    
    PS: Due to bank holiday, there will be no class this week. The final class is on June 2nd when you will be giving a 15 minutes presentation of your project. There is no need to submit the slides you will use at this stage.