The first workshop on

Generalizable Policy Learning in the Physical World

April 29, ICLR 2022 Workshop (Virtual)
Attend our workshop here , ICLR registration needed.


Generalization is particularly important when learning policies to interact with the physical world. The spectrum of such policies is broad: the policies can be high-level, such as action plans that concern temporal dependencies and casualties of environment states; or low-level, such as object manipulation skills to transform objects that are rigid, articulated, soft, or even fluid. In the physical world, an embodied agent can face a number of changing factors such as physical parameters, action spaces, tasks, visual appearances of the scenes, geometry and topology of the objects, etc. And many important real-world tasks involving generalizable policy learning, e.g., visual navigation, object manipulation, and autonomous driving. Therefore, learning generalizable policies is crucial to developing intelligent embodied agents in the real world.

Learning generalizable policies in the physical world requires deep synergistic efforts across fields of vision, learning, and robotics, and poses many interesting research problems. This workshop is designed to foster progress in generalizable policy learning, in particular, with a focus on the tasks in the physical world, such as visual navigation, object manipulation, and autonomous driving, because these real-world tasks require complex reasoning involving visual appearance, geometry, and physics. Technically, we expect to stimulate improvement in new directions such as:

Our main targeted participants are researchers interested in applying learning methods to develop intelligent embodied agents in the physical world. More specifically, target communities include, but are not limited to: robotics, reinforcement learning, learning from demonstrations, offline reinforcement learning, meta-learning, multi-task learning, 3D vision, computer vision, computer graphics, and physical simulation.

In affiliation to this workshop, we are also organizing the ManiSkill Challenge, which focuses on learning to manipulate unseen objects in simulation with 3D visual inputs. We will announce winners and host winner presentations in this workshop.

Call for Papers

We invite submission to the Generalizable Policy Learning in the Physical World workshop, hosted at ICLR 2022.

Paper topics

A non-exhaustive list of relevant topics:

Submission Guidelines

Review and Selection

Timeline (11:59 PM Pacific Standard Time)

Jan 17, 2022Announcement and call for submissions
Feb 25 Mar 2, 2022Paper submission deadline
Mar 25, 2022Review decisions announced
Apr 8 Apr 15, 2022Camera ready and poster uploading deadline


Please refer to the ManiSkill Challenge website for details.

Workshop Schedule

Please attend our workshop via our ICLR workshop virtual website.
Start Time (PDT) End Time (PDT) Event
8:00:00 AM 8:10:00 AM Intro and Opening Remark
8:10:00 AM 8:40:00 AM Invited Talk (Danica Kragic): Learning for contact rich tasks
8:40:00 AM 9:10:00 AM Invited Talk (Peter Stone): Grounded Simulation Learning for Sim2Real
9:10:00 AM 9:20:00 AM Break
9:20:00 AM 10:15:00 AM Poster Session 1
10:15:00 AM 11:15:00 AM Live Panel Discussion (password: bluefew)
11:15:00 AM 11:23:00 AM Challenge Winner Presentation (Zhutian & Aidan)
11:23:00 AM 11:31:00 AM Challenge Winner Presentation (Fattonny)
11:31:00 AM 1:00:00 PM Lunch Break
1:00:00 PM 1:10:00 PM Contributed Talk (Sim-to-Lab-to-Real: Safe RL with Shielding and Generalization Guarantees)
1:10:00 PM 1:40:00 PM Invited Talk (Shuran Song): Iterative Residual Policy for Generalizable Dynamic Manipulation of Deformable Objects  
1:40:00 PM 2:10:00 PM Invited Talk (Nadia Figueroa): Towards Safe and Efficient Learning and Control for Physical Human Robot Interaction
2:10:00 PM 2:18:00 PM Challenge Winner Presentation (EPIC lab)
2:18:00 PM 2:30:00 PM Break
2:30:00 PM 2:40:00 PM Contributed Talk (Know Thyself: Transferable Visual Control Policies Through Robot-Awareness)
2:40:00 PM 3:10:00 PM Invited Talk (Mrinal Kalakrishnan): Robot Learning & Generalization in the Real World
3:10:00 PM 3:40:00 PM Invited Talk (Xiaolong Wang): Generalizing Dexterous Manipulation by Learning from Humans
3:40:00 PM 3:48:00 PM Challenge Winner Presentation (Silver-Bullet-3D)
3:48:00 PM 3:50:00 PM Break
3:50:00 PM 4:45:00 PM Poster Session 2
4:45:00 PM 5:30:00 PM Challenge Award Ceremony
5:30:00 PM 5:35:00 PM Closing Remarks

Invited Speakers

listed alphabetically


listed alphabetically


listed alphabetically

Program Committee

We would like to thank the following people for their effort in providing feedback for submissions!

Poster Sessions

Poster session assignments are posted below. The session will be held at

Session A (9:20-10:15 PDT) Session B (15:50-16:45 PDT)
Poster Paper Name Poster Paper Name
A0 A Minimalist Ensemble Method for Generalizable Offline Deep Reinforcement Learning. Kun Wu, Yinuo Zhao, Zhiyuan Xu, Zhen Zhao, Pei Ren, Zhengping Che, Chi Harold Liu, Feifei Feng, Jian Tang A1 Continuous Control on Time. Tianwei Ni, Eric Jang
A2 Learning Generalizable Dexterous Manipulation from Human Grasp Affordance. Yueh-Hua Wu, Jiashun Wang, Xiaolong Wang A3 Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning. Denis Yarats, David Brandfonbrener, Hao Liu, Michael Laskin, Pieter Abbeel, Alessandro Lazaric, Lerrel Pinto
B0 An Empirical Study and Analysis of Learning Generalizable Manipulation Skill in the SAPIEN Simulator. Liu Kun, Huiyuan Fu, Zheng Zhang, Huanpu Yin B2 Let’s Handle It: Generalizable Manipulation of Articulated Objects. Zhutian Yang, Aidan Curtis
B1 Revisiting Model-based Value Expansion. Daniel Palenicek, Michael Lutter, Jan Peters B3 Versatile Offline Imitation Learning via State-Occupancy Matching. Yecheng Jason Ma, Andrew Shen, Dinesh Jayaraman, Osbert Bastani
C1 Control of Two-way Coupled Fluid Systems with Differentiable Solvers. Brener Ramos, Felix Trost, Nils Thuerey C0 One-Shot Imitation with Skill Chaining using a Goal-Conditioned Policy in Long-Horizon Control. Hayato Watahiki, Yoshimasa Tsuruoka
I2 PAnDR: Fast Adaptation to New Environments from Offline Experiences via Decoupling Policy and Environment Representations. Sang Tong, Hongyao Tang, Jianye HAO, YAN ZHENG, Zhaopeng Meng, Boyan Li, Zhen Wang C2 Density Estimation For Conservative Q-Learning. Paul Daoudi, Ludovic Dos Santos, Merwan Barlier, Aladin Virmaux
J0 Know Thyself: Transferable Visual Control Policies Through Robot-Awareness. Edward S. Hu, Kun Huang, Oleh Rybkin, Dinesh Jayaraman E1 Silver-Bullet-3D at ManiSkill 2021: Learning-from-Demonstrations and Heuristic Rule-based Methods for Object Manipulation. Yingwei Pan, Yehao Li, Yiheng Zhang, Qi Cai, Fuchen Long, Zhaofan Qiu, Ting Yao, Tao Mei
C3 Compositional Multi-Object Reinforcement Learning with Linear Relation Networks. Davide Mambelli, Frederik Träuble, Stefan Bauer, Bernhard Schölkopf, Francesco Locatello E2 Learning Robust Task Context with Hypothetical Analogy-Making. Shinyoung Joo, Sang Wan Lee
D0 Prompts and Pre-Trained Language Models for Offline Reinforcement Learning. Denis Tarasov, Vladislav Kurenkov, Sergey Kolesnikov F1 Improving performance on the ManiSkill Challenge via Super-convergence and Multi-Task Learning. Fabian Dubois, Eric Platon, Tom Sonoda
D1 A Probabilistic Perspective on Reinforcement Learning via Supervised Learning. Alexandre Piché, Rafael Pardinas, David Vazquez, Christopher Pal F2 ShiftNorm: On Data Efficiency in Reinforcement Learning with Shift Normalization. Sicong Liu, Xi Sheryl Zhang, Yushuo Li, Yifan Zhang, Jian Cheng
D3 Reinforcement Learning for Location-Aware Warehouse Scheduling. Stelios Andrew Stavroulakis, Biswa Sengupta G0 Separating the World and Ego Models for Self-Driving. Vlad Sobal, Alfredo Canziani, Nicolas Carion, Kyunghyun Cho, Yann LeCun
E0 Zero-Shot Reward Specification via Grounded Natural Language. Parsa Mahmoudieh, Deepak Pathak, Trevor Darrell G3 Using Deep Learning to Bootstrap Abstractions for Robot Planning. Naman Shah, Siddharth Srivastava
E3 Deep Sequenced Linear Dynamical Systems for Manipulation Policy Learning. Mohammad Nomaan Qureshi, Ben Eisner, David Held H1 Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space. Kuan Fang, Patrick Yin, Ashvin Nair, Sergey Levine
F0 Multi-task Reinforcement Learning with Task Representation Method. Myungsik Cho, Whiyoung Jung, Youngchul Sung H2 A Study of Off-Policy Learning in Environments with Procedural Content Generation. Andrew Ehrenberg, Robert Kirk, Minqi Jiang, Edward Grefenstette, Tim Rocktäschel
F3 Multi-objective evolution for Generalizable Policy Gradient Algorithms. Juan Jose Garau-Luis, Yingjie Miao, John D Co-Reyes, Aaron Parisi, Jie Tan, Esteban Real, Aleksandra Faust H3 Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations. Hao Shen, Weikang Wan, He Wang
G1 Safer Autonomous Driving in a Stochastic, Partially-Observable Environment by Hierarchical Contingency Planning. Ugo Lecerf, Christelle Yemdji-Tchassi, Pietro Michiardi I0 FlexiBiT: Flexible Inference in Sequential Decision Problems via Bidirectional Transformers. Micah Carroll, Jessy Lin, Orr Paradise, Raluca Georgescu, Mingfei Sun, David Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin
G2 Don't Freeze Your Embedding: Lessons from Policy Finetuning in Environment Transfer. Victoria Dean, Daniel Kenji Toyama, Doina Precup I1 Imitation Learning for Generalizable Self-driving Policy with Sim-to-real Transfer. Zoltán Lőrincz, Márton Szemenyei, Robert Moni
H0 Learning Transferable Policies By Inferring Agent Morphology. Brandon Trabucco, Mariano Phielipp, Glen Berseth I3 Sim-to-Lab-to-Real: Safe RL with Shielding and Generalization Guarantees. Kai-Chieh Hsu, Allen Z. Ren, Duy Phuong Nguyen, Anirudha Majumdar, Jaime Fernández Fisac


For any questions, you may contact us at
© 2022 Generalizable Policy Learning in the Physical World