CSCI 699: Robot Learning

Fall 2024-2025, Class: Fri 1:00-4:20pm, KAP 158


Syllabus | Piazza | Gradescope


Description:

Robot learning is an interdisciplinary field at the intersection of robotics, machine learning, cognitive science, and control theory, aiming to create intelligent and adaptable robotic systems capable of learning from their environment and experience. With rapid advances in artificial intelligence and computing power, as well as the possibility of having larger datasets, robot learning has the potential to revolutionize a wide range of applications, from manufacturing and healthcare to transportation and personal assistance. However, developing learning algorithms for real-world robotic systems poses unique challenges due to the complexities of the physical world, safety concerns, and the need for efficient and robust learning methods.

This course provides a comprehensive introduction to the fundamentals of robot learning, covering topics such as reinforcement learning, computer vision, meta-learning, sim-to-real transfer, and multi-agent learning. Students will explore cutting-edge techniques in imitation learning, inverse reinforcement learning, representation learning, and safe and robust learning, while also discussing the real-world applications and challenges of robot learning. The course is designed to be accessible to PhD students in robotics, control theory, machine learning, artificial intelligence, optimization, and related fields; with an emphasis on both theoretical foundations and practical applications.

In addition to lectures, the course features a series of student-led presentations on recent research papers and a course project, allowing students to gain hands-on experience with the latest advances in robot learning and explore emerging research topics. Through a combination of lectures, homework assignments, presentations, and project work, students will develop a deep understanding of robot learning techniques and their potential to transform the way we interact with and utilize robots in our everyday lives.


Prerequisites:

Students are recommended to have familiarity with fundamental concepts in machine learning. [CSCI 467: Introduction to Machine Learning AND (CSCI 445L: Introduction to Robotics OR CSCI 545: Robotics)] are recommended but not required.


Staff

Erdem Bıyık

Erdem Bıyık

Instructor

Office Hours: Fri 11:00AM-12:00PM
Location: PHE 214
biyik [at] usc [dot] edu
Webpage
Course TA

Ishika Singh

Teaching Assistant

Office Hours: Thu 3:30PM-4:30PM
Location: RTH 4th Floor Lounge
ishikasi [at] usc [dot] edu
Webpage


Timeline

Date Lecture Readings / Deadlines Notes
Week 1
Fri, Aug 30
General course information
Lecture Basics of robotics
Lecture Fundamentals of machine learning
Please checkout our Course Policies. Slides
Week 2
Fri, Sep 6
Lecture Basics of computer vision for robotics
Lecture Representation learning
Homework 1
Slides
Week 3
Fri, Sep 13
Presentation Representation learning
Lecture Reinforcement learning
Slides
Week 4
Fri, Sep 20
Lecture Reinforcement learning
Presentation Reinforcement learning
Due Homework #1
Slides
Week 5
Fri, Sep 27
Presentation Reinforcement learning
Lecture Imitation learning
Homework 2
Slides
Week 6
Fri, Oct 4
Presentation Imitation learning
Lecture Learning from human feedback
Slides
Week 7
Fri, Oct 11
Fall Recess: No Lecture Homework 3
Week 8
Fri, Oct 18
Lecture Learning from human feedback
Presentation Learning from human feedback
Due Homework #2
Due Project Proposal
Week 9
Fri, Oct 25
Lecture Sim-to-real transfer
Presentation Sim-to-real transfer
Slides
Week 10
Fri, Nov 1
Lecture Meta & Multi-task learning
Presentation Meta & Multi-task learning
Due Homework #3
Slides
Week 11
Fri, Nov 8
Guest Lecture Prof. Heather Culbertson
Guest Lecture Dr. Aaquib Tabrez
TBD
Multimodal Explanation-based Reward Coaching and Decision Support to Improve Human-Robot Teaming
Week 12
Fri, Nov 15
Presentation Safe and robust learning
Lecture Multi-agent learning
Due Project Milestone Report Slides
Week 13
Fri, Nov 22
Presentation Multi-agent learning
Presentation Robot learning using natural language
Week 14
Fri, Nov 29
Thanksgiving Break: No Lecture
Week 15
Fri, Dec 6
Project Project Presentations Due Final Project Report
Finals Week
Fri, Dec 13
No lecture. No final exam. Due Peer Review



Grading Metrics

Component Contribution to Grade
Homework 45%
Class Presentations 15%
Course Project 40%
Total 100%

Project Grading

Component Contribution to Grade
Project Proposal Report 5%
Project Milestone Report 5%
Project Presentation (Possibly with Demo) 10%
Final Project Report 15%
Peer Review 5%
Total 40%

Grading Policies


Homework (15%): Students will be assigned three homework sets that consist of both report questions and programming questions (in Python). Each student will select one of the three homework sets to complete. Report questions will require students to work on problems related to past lectures with pen and paper. Programming questions will require students to implement some of the methods covered in the lectures, occasionally with further improvements, and experiment them on simulated robot environments and/or machine learning tasks.

Class Presentation (45%): Students will present three research papers from literature. Presentations will be followed by open discussions. Students will be graded based on their presentations.

Course Project (40%): Students will be required to work on a course project in groups of 2-3. The projects must have both robotics and machine learning components. They can be, for example, application-dependent improvements over an existing robot learning method, a novel robot learning related application of an existing technique, or a completely new method that may have potential benefits. Students will write a 2-page project proposal, present their findings in an oral presentation, write a conference paper-style 6-8 pages project report, and write an anonymous peer review (max 1 page) for the project report of another group. There will be a 2-page project milestone along the way to guide progress. Instructor and teaching assistant(s) will provide feedback on the project milestone.




    © Erdem Bıyık 2024