Fall 2022: CS 6301 Special Topics in Computer Science: Introduction to Robot Manipulation and Navigation

Course Information

Term: Fall 2022
Class Level: Graduate
Activity Type: Lecture
Days & Times: Monday & Wednesday 1:00 PM – 2:15 PM
Location: ECSW 3.210

Instructor: Prof. Yu Xiang
Office Location: ECSS 4.702
Office Hours: Monday & Wednesday 3:30PM – 4:30 PM

Teaching Assistant: Ninad Arun Khargonkar
Office Hours: Tuesday 1:00PM – 2:00 PM

Syllabus

All the course materials can be found here.

Course Description

Theory and practice of robotics. Provides in-depth overview of robot manipulation and robot navigation, including kinematics, statics, and dynamics of robot manipulators, motion planning, state estimation, environment mapping and robot control.

Textbooks

Kevin M. Lynch and Frank C. Park. Modern Robotics: Mechanics, Planning, and Control. 1st Edition. (PDF)
ISBN-13: 978-1107156302
ISBN-10: 1107156300

Grading Policy

  • Homework (50%)
    • Assignment 1 (10%)
    • Assignment 2 (10%)
    • Assignment 3 (10%)
    • Assignment 4 (10%)
    • Assignment 5 (10%)
  • Team Project (45%)
    • Project proposal (5%)
    • Project mid-term report (10%)
    • Project presentation (15%)
    • Project final report (15%)
  • In-class Activity (5%)

Project

  • Project proposal description (pdf)
  • Project mid-term report requirement (pdf)
  • Project presentation and final report requirement (pdf)

Homework

Guest Lecturer

Dr. David Held from CMU talked about robot manipulation on 11/30/2022.
Title: Relational Affordance Learning for Robot Manipulation
Abstract
Robots today are typically confined to interact with rigid, opaque objects with known object models. However, the objects in our daily lives are often non-rigid, can be transparent or reflective, and are diverse in shape and appearance. I argue that, to enhance the capabilities of robots, we should develop perception methods that consider what robots need to know to interact with the world. Specifically, I will present novel perception methods that reason about object relational affordances; these estimated relational affordances can enable robots to perform complex tasks such as manipulating cloth, articulated objects, and grasping transparent and reflective objects. We also show how we can use such relational affordances to generalize to unseen objects in a category from a small number of demonstrations by learning to focus on the important parts of the objects in the demonstration. By reasoning about relational affordances, we can significantly improve our progress on difficult robots tasks.

Lectures

DateTopic
Week 1, 8/22, Lecture 1Introduction to Robotics (slides)
Week 1, 8/24, Lecture 2Configuration Space (slides)
Week 2, 8/29, Lecture 3Task Space, Workspace and Introduction to ROS (slides)
Installation of ROS in Docker (pdf)
Week 2, 8/31, Lecture 42D Rigid-Body Motions and Rotation Matrices (slides)
Week 3, 9/5Labor Day
Week 3, 9/7, Lecture 5Course Project Description (slides)
Week 4, 9/12, Lecture 6Angular Velocities and Exponential Coordinates of Rotations (slides)
Week 4, 9/14, Lecture 7Matrix Logarithm of Rotations and Homogeneous Transformation Matrices (slides)
Week 5, 9/19, Lecture 8Twists and Screw Axes (slides)
Week 5, 9/21, Lecture 9Exponential Coordinates of Rigid-Body Motions and Wrenches (slides)
Week 6, 9/26, Lecture 10Forward Kinematics and Denavit-Hartenberg Parameters (slides)
Week 6, 9/28, Lecture 11Forward Kinematics and Product of Exponentials Formula (slides)
Week 7, 10/3, Lecture 12Velocity Kinematics (slides)
Week 7, 10/5, Lecture 13Inverse Kinematics (slides)
Week 8, 10/10, Lecture 14Dynamics of Open Chains: Lagrangian Formulation (slides)
Week 8, 10/12, Lecture 15Dynamics of Open Chains: A Single Rigid Body (slides)
Week 9, 10/17, Lecture 16Dynamics of Open Chains: Newton-Euler Formulation (slides)
Week 9, 10/19, Lecture 17Robot Control: Motion Control with Velocities (slides)
Week 10, 10/24, Lecture 18Robot Control: Motion Control with Forces or Torques (slides)
Week 10, 10/26, Lecture 19Robot Control: Force Control, Hybrid Motion-Force Control, Impedance Control (slides)
Week 11, 10/31, Lecture 20Motion Planning: Overview and Foundations (slides)
Week 11, 11/2, Lecture 21Motion Planning: Algorithms (slides)
Week 12, 11/7, Lecture 22Wheeled Mobile Robots (slides)
Week 12, 11/9, Lecture 23Grasp Planning (slides)
by Ninad Arun Khargonkar
Week 13, 11/14, Lecture 24Reinforcement Learning: Overview and Foundations (slides)
Week 13, 11/16, Lecture 25Reinforcement Learning: Algorithms (slides)
Week 14, 11/21Fall Break
Week 14, 11/23Fall Break
Week 15, 11/28, Lecture 26IRVL Visit (video)
Week 15, 11/30, Lecture 27Guest Lecture: Dr. David Held
Relational Affordance Learning for Robot Manipulation (slides)
Week 16, 12/5Project Presentation I
Group 1: Goal-driven Autonomous Exploration (slides, demo)
Group 2: Obstacle avoidance using ROS on Gazebo (slides, demo)
Group 3: 6D Model-Free Mug Grasp Planning (slides, demo)
Group 4: Target-Driven Robot Navigation using Deep Reinforcement Learning for Mapless Navigation (slides, demo)
Group 6: Produce Segmentation Bot (slides, demo)
Group 7: Reinforcement Learning for Obstacle Navigation in Low Control Situations (slides, demo)
Group 9: Snack Helper – Getting Your Food So You Don’t Have To! (slides, demo)
Group 10: Target-driven Navigation (slides, demo)
Group 11: AskGrasp: Is this grasp suitable for performing a task on a particular object? (slides, demo)
Week 16, 12/7Project Presentation II
Group 5: Grasping Manipulation for Throwing a Ball under Constraints (slides, demo)
Group 12: Screws, Nuts, and Bolts Sorting with Model Based Top down Grasping (slides, demo)
Group 13: Audio embodied Indoor navigation (slides, demo)
Group 14: Indoor Navigation for Office Spaces (slides, demo)
Group 15: Navigation and Object Detection using a Custom-built Robot (slides, demo)
Group 16: Home Bot (slides, demo)
Group 17: Target Based Navigation using ROS Navigation Stack (slides, demo)
Group 18: Target Driven Visual Navigation in Outdoor Scenes (slides, demo)
Group 20: Target Driven Visual Navigation: PyTorch Implementation (slides, demo)