Course Information
Term: Fall 2025
Class Level: Graduate
Activity Type: Lecture
Days & Times: Monday & Wednesday 1:00 PM – 2:15 PM
Location: GR 3.420
Instructor: Prof. Yu Xiang
Office Location: ECSS 4.702
Office Hours: Monday & Wednesday 3:00PM – 4:00 PM
Teaching Assistant: Luis Felipe Casas Murillo
Office Location: ECSS 4.222
Office Hours: Tuesday & Thursday 2:00PM – 3:00 PM
Course Description
Theory and practice of robotics. Provides in-depth overview of robot manipulation and robot navigation, including kinematics, statics, and dynamics of robot manipulators, motion planning, state estimation, environment mapping and robot control.
Textbooks
Kevin M. Lynch and Frank C. Park. Modern Robotics: Mechanics, Planning, and Control. 1st Edition. (PDF)
ISBN-13: 978-1107156302
ISBN-10: 1107156300
Grading Policy
- Homework (50%)
- Assignment 1 (10%)
- Assignment 2 (10%)
- Assignment 3 (10%)
- Assignment 4 (10%)
- Assignment 5 (10%)
- Team Project (45%)
- Project proposal (10%)
- Project mid-term report (10%)
- Project presentation (15%)
- Project final report (10%)
- In-class Activity (5%)
Project
- Project proposal description (PDF)
- Project mid-term report requirement (PDF)
- Project presentation and final report requirement (PDF)
Homework
- Assignment 1 (PDF)
- Assignment 2 (PDF, programming)
- Assignment 3 (PDF, programming)
- Assignment 4 (PDF, programming)
- Assignment 5 (PDF, programming)
Guest Lecturers
Dr. Ankit Goyal from NVIDIA will talk about robot manipulation on 10/27/2025.
Title: Perspectives on Designing Vision-Language-Action Models
Abstract:
Vision-Language-Action (VLA) models hold immense promise for creating generalist robots, but the best way to build them remains an open question. This talk first provides an overview of the current design landscape, covering common VLA families and their strategies. We then introduce two novel design perspectives that challenge current conventions. The first, Hierarchical VLAs, decouples high-level task reasoning from low-level motion control, a design we find is highly effective for generalizing from off-domain data. The second, VLA-0, investigates the surprisingly potent and simple strategy of representing actions directly as text, eliminating the need for complex architectural modifications. Together, these two designs present potent, alternative perspectives for building the next generation of capable, generalist VLAs.
Bio:
Dr. Ankit Goyal is a Research Scientist at the NVIDIA Robotics Research Lab. His research focuses on foundation models for robotics and exploring the connection between 3D vision and robotics. He completed his Ph.D. in Computer Science at Princeton University and his M.S. in Computer Science and Engineering from the University of Michigan. He also holds a B.Tech. in Electrical Engineering from IIT Kanpur. Dr. Goyal is a recipient of many awards, including the RSS Pioneers Award , the Qualcomm Innovation Fellowship , and the NeurIPS Scholar Award.
Dr. Kuan Fang from Cornell University will talk about robot manipulation on 12/3/2025.
Title: Physically Grounded Reasoning for Open-World Robot Dexterity
Abstract: Generalist robots must seamlessly integrate semantic and physical understanding to act robustly in unstructured environments. While recent multimodal foundation models offer unprecedented capabilities for semantic reasoning, leveraging these models for real-world robotic control remains deeply challenging due to their limited knowledge of physical interactions in the real world. In this talk, I will present a series of works that combine foundation models with physically grounded representations to enable broad generalization across environments, objects, behaviors, and instructions. First, I will introduce a point-based affordance representation that allows pretrained foundation models to perform zero-shot and few-shot manipulation across novel tasks. Next, I will show how structured action representations can be extended to whole-body control, enabling flexible interlimb coordination specified by multimodal instructions. Finally, I will discuss a method that closes the loop between high-level reasoning and low-level execution by optimizing over language decompositions, enabling efficient adaptation to long-horizon tasks from only a handful of demonstrations.
Bio: Kuan Fang is an Assistant Professor of Computer Science at Cornell University. His research develops scalable learning-based methods that enable robots to perform diverse and complex tasks in unstructured environments. He received his Ph.D. and M.S. in Electrical Engineering from Stanford University and his bachelor’s degree from Tsinghua University. Before joining Cornell, he was a postdoctoral researcher at UC Berkeley and a researcher at the Robotics and AI Institute. His work has been recognized with a Computing Innovation Fellowship and an Amazon Research Award.
Lectures
| Date | Topic |
| Week 1, 8/25, Lecture 1 | Introduction to Robotics (slides) |
| Week 1, 8/27, Lecture 2 | Configuration Space (slides) |
| Week 2, 9/1 | Labor Day |
| Week 2, 9/3, Lecture 3 | Task Space, Workspace and Introduction to ROS Installation of ROS in Docker (slides, docker) |
| Week 3, 9/8, Lecture 4 | Course Project Description (slides) |
| Week 3, 9/10, Lecture 5 | SO-101 Building Session (slides) |
| Week 4, 9/15, Lecture 6 | Rigid-Body Motions and Rotation Matrices (slides) |
| Week 4, 9/17, Lecture 7 | Homogeneous Transformation Matrices (slides) |
| Week 5, 9/22, Lecture 8 | Forward Kinematics: Denavit-Hartenberg Parameters (slides) |
| Week 5, 9/24, Lecture 9 | Velocity Kinematics: Angular Velocity and Linear Velocity (slides) |
| Week 6, 9/29, Lecture 10 | Velocity Kinematics: Exponential Coordinates of Rigid-Body Motions and Twists (slides) |
| Week 6, 10/1, Lecture 11 | Forward Kinematics: Product of Exponentials Formula (slides) |
| Week 7, 10/6, Lecture 12 | Velocity Kinematics: Jacobian (slides) |
| Week 7, 10/8, Lecture 13 | Jacobian and Inverse Kinematics (slides) |
| Week 8, 10/13, Lecture 14 | Dynamics of a Single Rigid Body (slides) |
| Week 8, 10/15, Lecture 15 | Dynamics of Open Chains (slides) |
| Week 9, 10/20, Lecture 16 | Robot Control: Overview (slides) |
| Week 9, 10/22, Lecture 17 | Robot Control: Motion Control with Velocity Inputs (slides) |
| Week 10, 10/27, Lecture 18 | Guest Lecture: Dr. Ankit Goyal Perspectives on Designing Vision-Language-Action Models (slides) |
| Week 10, 10/29, Lecture 19 | Robot Control: Motion Control with Torque or Force Inputs (slides) |
| Week 11, 11/3, Lecture 20 | Motion Planning: Overview and Path Planning (slides) |
| Week 11, 11/5, Lecture 21 | Motion Planning: Algorithms (slides) |
| Week 12, 11/10, Lecture 22 | Grasp Planning (slides) |
| Week 12, 11/12, Lecture 23 | Reinforcement Learning: Overview and Foundations (slides) |
| Week 13, 11/17, Lecture 24 | Reinforcement Learning: Policy Optimization (slides) |
| Week 13, 11/19, Lecture 25 | Reinforcement Learning: Actor-Critic (slides) |
| Week 14, 11/24 | Fall Break |
| Week 14, 11/26 | Fall Break |
| Week 15, 12/1, Lecture 26 | Imitation Learning |
| Week 15, 12/3 | Guest Lecture: Dr. Kuan Fang |
| Week 16, 12/8 | Project Presentation I |
| Week 16, 12/10 | Project Presentation II |