Interdisciplinary Exploration of Generalizable Manipulation Policy Learning:
Paradigms and Debates

RSS 2023 Workshop
July 10, 2023
Daegu, Republic of Korea


Acquiring generalizable manipulation policies through data-driven, learning-based approaches is a fundamental problem in robotics. Despite advancements in this field, there are many unresolved questions that have resulted in a variety of, sometimes conflicting, paradigms. This workshop is designed to bring together researchers and practitioners from a diverse range of disciplines, robotics, vision, graphics, machine learning to share their insights and perspectives, with the goal of advancing interdisciplinary collaboration and promoting the development of innovative, new paradigms in generalizable manipulation policy learning. More specifically, we aim to discuss questions that are important for the future development of this field, including but not limited to:

Our targeted participants are researchers and engineers who are interested in the field of robotics and embodied AI, and have a background in machine learning, computer vision, computer graphics, natural language processing or related areas. The workshop will also be of interest to practitioners in industry who are working on the development of intelligent robots and other related systems.

In affiliation to this workshop, we are also organizing the ManiSkill2 Challenge, which is a large-scale robotic manipulation challenge that seeks to benchmark generalizable robotic agents that can interact with the 3D world. We will announce winners and host winner presentations in this workshop.

Workshop Schedule

Start Time (KST, UTC+9) Event Chair
9:00 Welcome and introduction Rui Chen
9:15 Invited talks:
Andy Zeng: From words to actions
Xingyu Lin: Generalizable Manipulation with Large Internet Data and Small Robot Data
Mini discussion
Rui Chen
10:15 Coffee break & Poster session
10:50 Invited talks:
Shan Luo: Empowering Robotic Manipulation: Bridging Sim-to-Real with Tactile Representations
Yuzhe Qin: Learning Generalizable Dexterous Manipulation from Vision, Touch and Human Demonstration
Mini discussion
Rui Chen
11:50 Lunch break
13:30 Invited talks:
Georgia Chalvatzaki: Real robots learn with structure
Joseph J. Lim: Skill-based Robot Learning
Mini discussion
Jiayuan Gu
14:30 Coffee break
15:00 Contributed talks:
Siddhant Haldar: Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations
Moritz Reuss: Goal-Conditioned Imitation Learning using Score-based Diffusion Policies
Pietro Mazzaglia: World Models for Robotic Manipulation
Jiayuan Gu
15:30 Award ceremony of the ManiSkill2 challenge Jiayuan Gu
15:40 Winner presentation: Xuetao Li Jiayuan Gu
16:00 Panel discussion Jiayuan Gu
16:50 Closing remark


Not Only Domain Randomization: Universal Policy with Embedding System Identification [PDF][Poster]

Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations [PDF][Poster]

MoVie: Visual Model-Based Policy Adaptation for View Generalization[PDF][Poster]

Learning Autonomous Ultrasound via Latent Task Representation and Robotic Skills Adaptation[PDF][Poster]

H-InDex: Visual Reinforcement Learning with Hand-Informed Representations for Dexterous Manipulation[PDF][Poster]

SE(3)-Diffusion Policy: Online target adaptation for dynamic robotic manipulation[PDF][Poster]

AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System[PDF][Poster]

Uncertainty-driven Affordance Discovery for Efficient Robotics Manipulation[PDF][Poster]

Towards General Food Acquisition with Human-Informed Actions[PDF][Poster]

FOCUS: Object-Centric World Models for Robotics Manipulation[PDF][Poster]

SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks[PDF][Poster]

A Two-stage Fine-tuning Strategy for Generalizable Manipulation Skill of Embodied AI[PDF][Poster]

The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learning[PDF][Poster]

A Curriculum-Based Approach for Imitating Versatile Skills[PDF][Poster]

Goal-Conditioned Imitation Learning using Score-based Diffusion Policies[PDF][Poster]

Plug-And-Play Object-Centric Representations From "What" and "Where" Foundation Models[PDF][Poster]

ManiSkill2 Challenge

1st Prize: Team GXU-LIPE, Guangxi University

2nd Prize: Team baochen, Shanghai Jiaotong University

3rd Prize: Team dee, Hong Kong Polytechnic University

Please refer to the ManiSkill2 Challenge website for details.
© RSS 2023 Workshop: Generalizable Manipulation Policy Learning