publications

2024

1. 

   Causal Robot Learning for Manipulation

   Tabitha Edith Lee

   Carnegie Mellon University, Ph.D. Thesis, 2024

   Abs HTML PDF

   Two decades into the third age of artificial intelligence, the rise of deep learning has yielded two seemingly disparate realities. In one, massive accomplishments have been achieved in deep reinforcement learning, protein folding, and large language models. Yet, in the other, the promises of deep learning to empower robots that operate robustly in real-world environments have yet remained unfulfilled. Vast diversity of objects, distribution shifts, long-tailed phenomena: outside of laboratories, real-world environments challenge modern statistical learning assumptions of the data.
   
   Although such environments have generally been referred to as "unstructured," this terminology belies their nature. Real-world environments are not "unstructured," but arise because of structure: the underlying causal processes that generate the observed data. In this view, robots should not only reason and learn with respect to data, but also the data generating processes. Such processes can be formalized by the language of causality. Therefore, to learn and leverage the structure of these "open-world" environments, new causal-based robot learning algorithms are needed.
   
   Towards this goal, this thesis explores a diversity of robot learning problems, from perception to control. First, we explore how perception models can be learned using sim-to-real transfer from synthetic data (DREAM, FormNet). In these works, relevant features were learned through domain randomization, leading to insights into how structure can be learned more directly using causality. To this end, we introduce structural sim-to-real transfer, where simulation can serve as a causal reasoning engine for the robot to select the relevant features for a control policy (CREST) or skill (SCALE). By construction, these policies are robust to irrelevant distribution shifts that would otherwise stymie correlation-based deep learning. Next, the rich interplay between control, dynamical systems, and causality is explored through the Learning By Doing (LBD) competition and LMeshNet, a methodology for constructing hybrid causal world models that integrate both latent and semantic information. Lastly, we examine curriculum learning. In ACL, we explore the commonalities and differences between human and agent curriculum learning. Then, we employ these lessons learned for CURATE: how agents can manipulate the sequencing of training data to efficiently learn a control policy to solve a desired, difficult task.
   
   The advantages of adopting the principles of causal inference have been witnessed to date in fields such as biomedical sciences, economics, and genomics. In the machine learning community, it has recently been argued that such principles should be integrated to harness deep learning, towards causal learning of representations. Analogously, this thesis forwards that the robot learning community stands to gain by leveraging the principles of causality. In so doing, this new paradigm holds promise for robots to learn and leverage structure within the open world through causal robot learning for manipulation.

2. 

   Causal-HRI: Causal Learning for Human-Robot Interaction

   Jiaee Cheong, Nikhil Churamani, Luke Guerdan, Tabitha Edith Lee, Zhao Han, and Hatice Gunes

   2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2024

   Abs DOI Website

   Real-world Human-Robot Interaction (HRI) requires robots to adeptly perceive and understand the dynamic human-centred environments in which they operate. Recent decades have seen remarkable advancements that have endowed robots with exceptional perception capabilities. The first workshop on "Causal-HRI: Causal Learning for Human-Robot Interaction" aims to bring together research perspectives from Causal Discovery and Inference and Causal Learning, in general, to real-world HRI applications. The objective of this workshop is to explore strategies that will not only embed robots with capabilities to discover cause-and-effect relationships from observations, allowing them to generalise to unseen interaction settings, but also to enable users to understand robot behaviours, moving beyond the ’black-box’ models used by these robots. This workshop aims to facilitate an exchange of views through invited keynote presentations, contributed talks, group discussions and poster sessions, encouraging collaborations across diverse scientific communities. The theme of HRI 2024, "HRI in the real world," will inform the overarching theme of this workshop, encouraging discussions on HRI theories, methods, designs and studies focused on leveraging Causal Learning for enhancing real-world HRI.

3. Fail-Active Autonomy in Unknown-Unknown Situations for Deep Space Missions

   Christian Debrunner, Eric Dixon, Benjamin Hockman, Saman Fahandezhsaadi, Daniel Kolosa, Tabitha Edith Lee, Shruti Mahadevan, and John Steinbis

   2024 IEEE Aerospace Conference, 2024

   Abs DOI

   This paper discusses the environment anomalies that the CARL/COLDArm system was subjected to utilizing the OceanWATERS simulator to demonstrate "fail-active" autonomy. The geotechnical task used throughout our tests is the pressure sinkage test performed at various locations to assess the surface hardness using its end effector. As the lander is performing the pressure sinkage tests, environmental anomalies are injected into the simulation, including fiducial obscuration due to shadowing and unexpected ice build-up on the COLDArm end effector. Adjustments to the controller’s available safe actions and variations in causal model knowledge that informs controller actions were assessed, using both learned and expert-developed causal models. Through these tests and demonstrations in the OceanWATERS simulator, CARL uses its causal model to infer that the fail-state is triggered by an explicit cause. Based on the inference from the causal model, CARL selects from various possible actions to resolve the fail-state from several possible actions, including COLDArm pose adjustment, camera integration time change, light activation, and heater power adjustment. This allows the lander to proceed with completing its pressure sinkage tests.

2023

1. 

   From Child’s Play to AI: Insights into Automated Causal Curriculum Learning

   Annya Dahmani^*, Eunice Yiu^*, Tabitha Edith Lee, Nan Rosemary Ke, Oliver Kroemer, and Alison Gopnik

   Intrinsically Motivated Open-ended Learning Workshop at the Thirty-seventh Conference on Neural Information Processing Systems (IMOL@NeurIPS), 2023

   Abs HTML PDF

   We study how reinforcement learning algorithms and children develop a causal curriculum to achieve a challenging goal that is not solvable at first. Adopting the Procgen environments that include various challenging tasks, we found that 5- to 7-year-old children actively used their current level competence to determine their next step in the curriculum and made improvements to their performance during this process as a result. This suggests that children treat their level competence as an intrinsic reward, and are motivated to master easier levels in order to do better at the more difficult one, even without explicit reward. To evaluate RL agents, we exposed them to the same demanding Procgen environments as children and employed several curriculum learning methodologies. Our results demonstrate that RL agents that emulate children by incorporating level competence as an additional reward signal exhibit greater stability and are more likely to converge during training, compared to RL agents that are solely reliant on extrinsic reward signals for game-solving. Curriculum learning may also offer a significant reduction in the number of frames needed to solve a target environment. Taken together, our human-inspired findings suggest a potential path forward for addressing catastrophic forgetting or domain shift during curriculum learning in RL agents.

2. 

   SCALE: Causal Learning and Discovery of Robot Manipulation Skills using Simulation

   Tabitha Edith Lee^*, Shivam Vats^*, Siddharth Girdhar, and Oliver Kroemer

   7th Conference on Robot Learning (CoRL), 2023

   Abs HTML PDF Video Poster Website

   We propose SCALE, an approach for discovering and learning a diverse set of interpretable robot skills from a limited dataset. Rather than learning a single skill which may fail to capture all the modes in the data, we first identify the different modes via causal reasoning and learn a separate skill for each of them. Our main insight is to associate each mode with a unique set of causally relevant context variables that are discovered by performing causal interventions in simulation. This enables data partitioning based on the causal processes that generated the data, and then compressed skills that ignore the irrelevant variables can be trained. We model each robot skill as a Regional Compressed Option, which extends the options framework by associating a causal process and its relevant variables with the option. Modeled as the skill Data Generating Region, each causal process is local in nature and hence valid over only a subset of the context space. We demonstrate our approach for two representative manipulation tasks: block stacking and peg-in-hole insertion under uncertainty. Our experiments show that our approach yields diverse skills that are compact, robust to domain shifts, and suitable for sim-to-real transfer.

2021

1. 

   Learning by Doing: Controlling a Dynamical System using Causality, Control, and Reinforcement Learning

   Sebastian Weichwald, Søren Wengel Mogensen, Tabitha Edith Lee, Dominik Baumann, Oliver Kroemer, Isabelle Guyon, Sebastian Trimpe, Jonas Peters, and Niklas Pfister

   NeurIPS 2021 Competitions and Demonstrations Track, 2021

   Abs HTML PDF Supp Code Website

   Questions in causality, control, and reinforcement learning go beyond the classical machine learning task of prediction under i.i.d. observations. Instead, these fields consider the problem of learning how to actively perturb a system to achieve a certain effect on a response variable. Arguably, they have complementary views on the problem: In control, one usually aims to first identify the system by excitation strategies to then apply model-based design techniques to control the system. In (non-model-based) reinforcement learning, one directly optimizes a reward. In causality, one focus is on identifiability of causal structure. We believe that combining the different views might create synergies and this competition is meant as a first step toward such synergies. The participants had access to observational and (offline) interventional data generated by dynamical systems. Track CHEM considers an open-loop problem in which a single impulse at the beginning of the dynamics can be set, while Track ROBO considers a closed-loop problem in which control variables can be set at each time step. The goal in both tracks is to infer controls that drive the system to a desired state. Code is open-sourced (https://github.com/LearningByDoingCompetition/learningbydoing-comp) to reproduce the winning solutions of the competition and to facilitate trying out new methods on the competition tasks.

2. 

   Visual Identification of Articulated Object Parts

   Vicky Zeng, Tabitha Edith Lee^*, Jacky Liang^*, and Oliver Kroemer

   2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021

   Abs DOI HTML PDF Website

   As autonomous robots interact and navigate around real-world environments such as homes, it is useful to reliably identify and manipulate articulated objects, such as doors and cabinets. Many prior works in object articulation identification require manipulation of the object, either by the robot or a human. While recent works have addressed predicting articulation types from visual observations alone, they often assume prior knowledge of category-level kinematic motion models or sequence of observations where the articulated parts are moving according to their kinematic constraints. In this work, we propose FormNet, a neural network that identifies the articulation mechanisms between pairs of object parts from a single frame of an RGB-D image and segmentation masks. The network is trained on 100k synthetic images of 149 articulated objects from 6 categories. Synthetic images are rendered via a photorealistic simulator with domain randomization. Our proposed model predicts motion residual flows of object parts, and these flows are used to determine the articulation type and parameters. The network achieves an articulation type classification accuracy of 82.5% on novel object instances in trained categories. Experiments also show how this method enables generalization to novel categories and can be applied to real-world images without fine-tuning.

3. 

   Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies

   Tabitha Edith Lee, Jialiang (Alan) Zhao, Amrita S. Sawhney, Siddharth Girdhar, and Oliver Kroemer

   2021 IEEE International Conference on Robotics and Automation (ICRA), 2021

   Abs DOI HTML PDF Video Website

   We present CREST, an approach for causal reasoning in simulation to learn the relevant state space for a robot manipulation policy. Our approach conducts interventions using internal models, which are simulations with approximate dynamics and simplified assumptions. These interventions elicit the structure between the state and action spaces, enabling construction of neural network policies with only relevant states as input. These policies are pretrained using the internal model with domain randomization over the relevant states. The policy network weights are then transferred to the target domain (e.g., the real world) for fine tuning. We perform extensive policy transfer experiments in simulation for two representative manipulation tasks: block stacking and crate opening. Our policies are shown to be more robust to domain shifts, more sample efficient to learn, and scale to more complex settings with larger state spaces. We also show improved zero-shot sim- to-real transfer of our policies for the block stacking task.

2020

1. 

   Camera-to-Robot Pose Estimation from a Single Image

   T. E. Lee, Jonathan Tremblay, Thang To, Jia Cheng, Terry Mosier, Oliver Kroemer, Dieter Fox, and Stan Birchfield

   2020 IEEE International Conference on Robotics and Automation (ICRA), 2020

   Abs DOI HTML PDF Blog Code Website

   We present an approach for estimating the pose of an external camera with respect to a robot using a single RGB image of the robot. The image is processed by a deep neural network to detect 2D projections of keypoints (such as joints) associated with the robot. The network is trained entirely on simulated data using domain randomization to bridge the reality gap. Perspective-n-point (PnP) is then used to recover the camera extrinsics, assuming that the camera intrinsics and joint configuration of the robot manipulator are known. Unlike classic hand-eye calibration systems, our method does not require an off-line calibration step. Rather, it is capable of computing the camera extrinsics from a single frame, thus opening the possibility of on-line calibration. We show experimental results for three different robots and camera sensors, demonstrating that our approach is able to achieve accuracy with a single frame that is comparable to that of classic off-line hand-eye calibration using multiple frames. With additional frames from a static pose, accuracy improves even further. Code, datasets, and pretrained models for three widely-used robot manipulators are made available.

2017

1. 

   State Estimation and Localization for ROV-Based Reactor Pressure Vessel Inspection

   T. E. Lee and Nathan Michael

   11th International Conference on Field and Service Robotics (FSR), 2017

   Abs DOI HTML PDF

   A vision-based extended Kalman filter is proposed to estimate the state of a remotely operated vehicle (ROV) used for inspection of a nuclear reactor pressure vessel. The state estimation framework employs an overhead, pan-tilt-zoom (PTZ) camera as the primary sensing modality. In addition to the camera state, a map of the nuclear reactor vessel is also estimated from a prior. We conduct experiments to validate the framework in terms of accuracy and robustness to environmental image degradation due to speckling and color attenuation. Subscale mockup experiments highlight estimate consistency as compared to ground truth despite visually degraded operating conditions. Full-scale platform experiments are conducted using the actual inspection system in a dry setting. In this case, the ROV achieves a lower state uncertainty as compared to subscale mockup evaluation. For both subscale and full-scale experiments, the state uncertainty was robust to environmental image degradation effects.

2. 

   State Estimation and Localization for ROV-Based Reactor Pressure Vessel Inspection Using a Pan-Tilt-Zoom Camera

   T. E. Lee

   Carnegie Mellon University, M.S. Thesis, 2017

   Abs HTML PDF

   A vision-based state estimation framework is proposed to localize a remotely operated vehicle (ROV) that is used to inspect a nuclear reactor pressure vessel. Using an extended Kalman filter, the framework leverages an external, pan-tilt-zoom (PTZ) camera as the primary sensing modality. In addition to the ROV and camera states, the framework also estimates a map of the reactor lower core from a prior. Both simulation and camera experiments are conducted to establish the correctness of the framework. Camera experiments (consisting of subscale and platform tests) validate the framework in terms of accuracy and robustness to environmental image degradation caused by speckling and color attenuation. Subscale mockup experiments highlight estimation consistency as compared to ground truth despite visually degraded operated conditions. Full-scale platform experiments are conducted using the actual inspection system in a dry setting. In this case, the ROV achieves a lower state uncertainty as compared to subscale mockup evaluation. For both subscale and full-scale experiments, the state uncertainty is robust to environmental image degradation effects. To further increase the efficiency and utility of the framework, we formulate and evaluate an online initialization methodology that bootstraps the framework \textitin situ by calibrating the camera from the reactor structure. A Monte Carlo simulation shows that the initialization methodology is not sensitive to camera mounting position and orientation if the camera is placed near the center of the vessel. This framework sets the foundation for improving not only ROV-based inspection through automation, but also visual inspection with the PTZ camera through mosaics and dense reconstructions of the reactor lower core.

2016

1. Autonomous Inspection of Nuclear Reactor Pressure Vessels via a Remotely Operated Vehicle

   T. E. Lee, Nathan Michael, and Lyman J. Petrosky

   American Nuclear Society Decommissioning and Remote Systems 2016 (D&RS), 2016

   HTML Website

2013

1. Automated, Integrated UAV Health Monitoring System: Flight Test Case Study

   T. E. Lee, Mary Jayne Cribbs, and John C. Krainski

   American Helicopter Society Specialists’ Meeting on Unmanned Rotorcraft and Network Centric Operations, 2013

   HTML