Logistics
Learning Resources
Readings
Reinforcement Learning
Robotics
- Lynch and Park, Modern Robotics: Mechanics, Planning, and Control
- Murray and Li and Sastry, A Mathematical Introduction to Robotic Manipulation
Online Courses
Reinforcement Learning
- Katerina Fragkiadaki’s class: Deep RL
- David Silver’s class: Reinforcement Learning
- Sergey Levine’s class: Deep Reinforcement Learning
Control
Robotics
- Russ Tedrake’s class: Underactuated Robotics
- Russ Tedrake’s class: Robotic Manipulation
- Kris Hauser’s class: Robotic Systems
- Abishek Gupta’s class: Probabilistic Robotics
- Pieter Abbeel’s class: Advanced Robotics:
Communication
Discord is intended for all announcements, general questions about the course, clarifications about assignments, student questions to each other, discussions about material, and so on.
Grading
This class requirements include 2 homework assignments (30% of the grade), a final project presentation and participation (70% of the grade). Each assignment takes 15% of the grade. The requirements of the final project include a project proposal presentation (10% of the grade), in-person meeting with the instructor (2 * 10% of the grade), and a poster presentation (40% of the grade). In-person participation of these presentations are required.
Assignments
For the first assignment, students need to team up to (1) assemble a real inverted pendulum robot, (2) implementing a simple PID controller, (3) implementing LQR, iLQR and MPC controllers, (4) train RL controllers such as DQN, PPO and SAC. All controllers will be implemented and tested in simulation, then deployed directly in the real world. Therefore, system identification is needed to bridge the sim-real gap.
For the second assignment, students need to work individually on in-hand orientation in simulation. Students will have to (1) set up the simulator environment, (2) train RL policies for basic in-hand orientation, (3) enhance the policy’s OOD generation to novel objects, unseen physical properties and unexpected environmental perturbations.
Late Policy of Homework Assignments
Throughout the whole semester, students should submit their assignments before the deadline. The score of the assignment will be multiplied by 0.9 for each additional day of delay. The submission deadline is based on Taiwan’s time zone
Grading policy of final project presentations
The proposal and rehearsal presentations will be scored by the instructor and TAs:
$\mathrm{score} = \frac{1}{N+1} \mathrm{score_{tw}} + \frac{1}{N+1} \sum_{i}^{N} \mathrm{score_{TA, i}}$
The poster presentations will be scored by the instructor, TAs and the students:
$\mathrm{score} = \frac{1}{N+2} \mathrm{score_{tw}} + \frac{1}{N+2} \sum_{i}^{N} \mathrm{score_{TA, i}} + \frac{1}{N+2} \frac{1}{M} \sum_{j}^{M} \mathrm{score_{student, j}}$
Homework Assignments
You are encouraged to discuss with others, but do not share your codes with them! If you wrote the same code as others, you may waive the penalty by refactoring your code in-person within limited time, or otherwise, you’ll get 15% total grade penalty (for each assignment).
Please list your collaborator in the appendix of each assignment
You are allowed to use AIs at your own risk. You are responsible for refactoring the code snippets generated by AIs. You’ll get the penalty as long as your submitted codes are the same as others.
Final Project Presentations
You can form a team of 4-5 members. If you really want to work alone, come and chat with us.
You will submit a proposal, describing the topic, experimental setup, todos and expected contribution of each member for the final project.
All members need to code. You are encouraged to create a github repo, keeping track the contributions of each member. If you found piggybackers on your team, come to discuss with us along with your github repo.