Reinforcement Learning (RL), Fall 2024

CSCI 4160/6963, ECSE 4965/6965

Personnel

Class Time and Location

Course Description

This is an introductory course on the theory and practice of reinforcement learning (RL). We will start with an introduction to linear algebra and probability and will quickly go over standard supervised learning techniques, such as linear regression. After that, we will derive the RL framework, starting from Markov chains and Markov reward processes and building up to Markov decision processes. We will then cover classic RL approaches such as dynamic programming, Monte Carlo methods, Q-learning and policy gradients. In the last part of the course, we will cover deep learning and deep RL.

Textbook

There is no required book for the course since there is no individual book that covers all the course material. All of the necessary material will be included in the lecture slides. I will suggest additional reading material along with each lecture. We will follow some of the material in the following books, (most of) which are available for free online:

Grading

Students will be graded on 10 homework assignments. Each assignment is worth 10% of the final grade.

Late submission rule: You can use up to 2 extensions – you don’t need to explain why you submitted late. Each extension is for 3 days, midnight to midnight. If your submission is more than 3 days late (or more than 1 minute late in case you have used up your extensions), you will receive a score of 0. You cannot use both extensions on the same assignment.

Useful Resources

Announcements

It is the student's responsibility to be aware of and understand all announcements made in the lectures.

Assignments

Please check the tentative schedule for a tentative homework deadline schedule. Unless otherwise noted, each assignment will be due at midnight Thursday. I will post each assignment after the previous one is completed. Email announcements will be sent accordingly.

All assignments will be posted on and must be submitted through LMS. You are expected to work alone on all assignments, unless specifically noted otherwise -- please check the syllabus for a clarification of what constitutes academic dishonesty.

Lectures