GELLO is a general, low-cost, and intuitive teleoperation framework for robot manipulators. This repository contains all the software components for GELLO.

For additional resources:

• Project Website
• Hardware Repository - STL files and build instructions
• ROS 2 Support

git clone https://github.com/wuphilipp/gello_software.git
cd gello_software

First, install uv if you don't have it:

curl -LsSf https://astral.sh/uv/install.sh | sh

Create and activate a virtual environment:

uv venv --python 3.11
source .venv/bin/activate  # Run this every time you open a new shell
git submodule init
git submodule update
uv pip install -r requirements.txt
uv pip install -e .
uv pip install -e third_party/DynamixelSDK/python

Install Docker, then:

docker build . -t gello:latest
python scripts/launch.py

Note: GELLO also supports ROS 2 Humble for the Franka FR3 robot. See the ROS 2-specific README in the ros2 directory.

The recommended setup for GELLO is with the I2RT YAM robot arm, using the YAML-based configuration system. This provides the most features and is the best-supported configuration.

For the I2RT YAM robot, you can automatically generate your configuration files. This process calibrates the joint offsets and creates configuration files for both simulation and real hardware.

1. Update Motor IDs: Before generating the config, ensure each Dynamixel motor has a unique ID. Install the Dynamixel Wizard and follow these steps:

   1. Connect a single motor to the U2D2 controller.
   2. Open Dynamixel Wizard and scan to detect the motor.
   3. Change the ID to a unique number (e.g., 1 through 7).
   4. Repeat for each motor, ensuring they are in order from base to gripper.

2. Run the Generation Script: With the YAM arm in its default build position (see image below), run the script:

   python scripts/generate_yam_config.py

   Follow the prompts in the terminal. This will create configs/yam_auto_generated.yaml for the real robot and configs/yam_auto_generated_sim.yaml for the simulation.

You can now skip to the Usage section.

GELLO uses YAML files in configs/ for configuration. This allows for flexible setup of different robots, environments, and teleoperation parameters. If you have automatically generated your .yaml config files with scripts/generate_yam_config.py, you probably will not need to modify these confings manually.

Sample configs for the YAM arm and the xarm can be found in configs.

• Robot Config: Defines robot type, communication parameters, and physical settings.
• Agent Config: Defines GELLO device settings, joint mappings, and calibration.
• DynamixelRobotConfig: Motor-specific settings including IDs, offsets, signs, and gripper.
• Control Parameters: Update rates (hz), step limits (max_steps), and safety settings.

• Most widely supported across different arms
• Located in gello/agents/gello_agent.py
• Uses PORT_CONFIG_MAP dictionary
• Maps USB serial ports to robot configurations

• Used for ROS 2 packages
• Runtime configuration loading
• Located in ros2/src/franka_gello_state_publisher/config/gello_config.yaml

To integrate a new robot to the Python configs:

1. Check Compatibility: Ensure your GELLO kinematics match the target robot
2. Implement Robot Interface: Create a new class implementing the Robot protocol from gello/robots/robot.py
3. Add Configuration: Update the configuration system with your robot's parameters

See existing implementations in gello/robots/ for reference:

• panda.py - Franka Panda robot
• ur.py - Universal Robots
• xarm_robot.py - xArm robots
• yam.py - YAM robot

Set your GELLO and robot arm to a known, matching configuration (see images below) and run the offset detection script.

Command examples:

UR Robot:

python scripts/gello_get_offset.py \
    --start-joints 0 -1.57 1.57 -1.57 -1.57 0 \
    --joint-signs 1 1 -1 1 1 1 \
    --port /dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FT7WBG6

Franka FR3:

python scripts/gello_get_offset.py \
    --start-joints 0 0 0 -1.57 0 1.57 0 \
    --joint-signs 1 1 1 1 1 -1 1 \
    --port /dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FT7WBG6

I2RT YAM:

python scripts/gello_get_offset.py \
    --start-joints 0 0 0 0 0 0 \
    --joint-signs 1 -1 -1 -1 1 1 \
    --port /dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FTAAMLV6-if00-port0

Joint Signs Reference:

• UR: 1 1 -1 1 1 1
• Panda: 1 -1 1 1 1 -1 1
• FR3: 1 1 1 1 1 -1 1
• xArm: 1 1 1 1 1 1 1
• YAM: 1 -1 -1 -1 1 1

Add the generated joint offsets to gello/agents/gello_agent.py in the PORT_CONFIG_MAP.

1. Copy an existing config from configs/ as a template (e.g., yam_passive.yaml).
2. Modify the robot _target_ and parameters for your setup:
   • For hardware: gello.robots.ur.URRobot, gello.robots.panda.PandaRobot, etc.
   • For simulation: gello.robots.sim_robot.MujocoRobotServer
3. Update the agent configuration with your GELLO device settings:
   • port: Your U2D2 device path
   • joint_offsets: From the offset detection script
   • joint_signs: Based on your robot type
   • start_joints: Your GELLO's starting position

The recommended way to launch GELLO is with a YAML configuration file.

Robot arms such as the YAM use a CAN bus to communicate with your machine. If your arm uses a CAN bus, you will need to configure udev rules. First, get your CAN bus ID:

udevadm info -a -p /sys/class/net/can* | grep -i serial

Then open your CAN bus rules using your text editor of choice.

sudo nano /etc/udev/rules.d/90-can.rules

If you only have one arm, add this line:

SUBSYSTEM=="net", ACTION=="add", ATTRS{serial}=="<your-CAN-id>", NAME="can_left"

If you have two arms (a bimanual setup), you will need a second line for your right arm. Your bimanual CAN rules file should contain:

SUBSYSTEM=="net", ACTION=="add", ATTRS{serial}=="<left-CAN-id>", NAME="can_left"
SUBSYSTEM=="net", ACTION=="add", ATTRS{serial}=="<right-CAN-id>", NAME="can_right"

After updating your udev rules, run the following and then unplug and reconnect your CAN devices.

sudo udevadm control --reload-rules && sudo systemctl restart systemd-udevd && sudo udevadm trigger

At this point, your CAN devices are correctly configured. If you encounter CAN connctivity issues after this point run sh scripts/reset_all_can.sh to reset your CAN buses.

First, install the YAM-specific dependency:

• YAM: I2RT
• uv pip install -e third_party/i2rt

Testing in Simulation: Launch the simulation with the auto-generated sim config file:

python experiments/launch_yaml.py --left-config-path configs/yam_auto_generated_sim.yaml

Real Robot Operation: Launch the real robot with the auto-generated hardware config file:

python experiments/launch_yaml.py --left-config-path configs/yam_auto_generated.yaml

For other robots or if not using a YAML configuration, you must launch the robot and controller nodes in separate terminals.

First, install robot-specific dependencies:

• UR: ur_rtde
• Panda: polymetis
• xArm: xArm Python SDK

1. Launch the robot node:

# For simulation
python experiments/launch_nodes.py --robot <sim_ur|sim_panda|sim_xarm>

# For real hardware
python experiments/launch_nodes.py --robot <ur|panda|xarm>

2. Launch GELLO controller:

python experiments/run_env.py --agent=gello

If, when you run generate_yam_config.py, you get an error detecting offsets, you may need to add your user to the dialout user group. To do so, run: sudo usermod -aG dialout $USER And then log out and log back in or restart your computer.s

If some joints in your arm are not behaving as expected, you may need to modify the joint signs of your configuration. Simply invert the affected joint sign(s) in your .yaml or gello_agent.py or physically reverse the installation of the servo.

Use --start-joints to specify GELLO's starting configuration for automatic robot reset:

python experiments/run_env.py --agent=gello --start-joints <joint_angles>

Collect teleoperation demonstrations with keyboard controls.

For the YAM arm launched with launch_yaml.py, you can append the flag --use-save-interface to enable data saving. This is the recommended method.

python experiments/launch_yaml.py --left-config-path configs/yam_passive.yaml --use-save-interface

After launching, you can begin saving with s and stop saving with q. Data saved will be in the data directory in the root of the project.

For non-YAM setups, use the following:

python experiments/run_env.py --agent=gello --use-save-interface

Process collected data:

python gello/data_utils/demo_to_gdict.py --source-dir=<source_dir>

The recommended way to use bimanual mode is with launch_yaml.py. Pass a config file for the right arm to --right-config-path.

python experiments/launch_yaml.py --left-config-path configs/gello_1.yaml --right-config-path configs/gello_2.yaml

For non-YAM setups, use:

python experiments/launch_nodes.py --robot=bimanual_ur
python experiments/run_env.py --agent=gello --bimanual

├── scripts/             # Utility scripts
├── experiments/         # Entry points and launch scripts
├── gello/               # Core GELLO package
│   ├── agents/          # Teleoperation agents
│   ├── cameras/         # Camera interfaces
│   ├── data_utils/      # Data processing utilities
│   ├── dm_control_tasks/# MuJoCo environment utilities
│   ├── dynamixel/       # Dynamixel hardware interface
│   ├── robots/          # Robot-specific interfaces
│   ├── utils/           # Shared launch and control utilities
│   └── zmq_core/        # ZMQ multiprocessing utilities

Install development dependencies and set up pre-commit hooks to ensure code quality before contributing:

uv pip install -r requirements_dev.txt
uv pip install pre-commit
pre-commit install

The codebase uses isort and black for code formatting.

We welcome contributions! Submit pull requests to help make teleoperation more accessible and higher quality.

@misc{wu2023gello,
    title={GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators},
    author={Philipp Wu and Yide Shentu and Zhongke Yi and Xingyu Lin and Pieter Abbeel},
    year={2023},
}

This project is licensed under the MIT License (see LICENSE file).

• google-deepmind/mujoco_menagerie: Robot models for MuJoCo
• brentyi/tyro: Argument parsing and configuration
• ZMQ: Multiprocessing communication framework