Trossen AI Configuration

In order to use the Trossen AI Kits with LeRobot, you need to first configure the arms with the necessary specifications. The specifications include the IP address, model_name, and camera serial numbers. The following steps will guide you on how to configure the Trossen AI Kits with LeRobot.

This is an example of a configuration file for the Trossen AI Kits with LeRobot, you can find this in lerobot/common/robot_devices/robots/configs.py:

Trossen AI StationaryTrossen AI MobileTrossen AI Solo

@RobotConfig.register_subclass("trossen_ai_stationary")
@dataclass
class TrossenAIStationaryRobotConfig(ManipulatorRobotConfig):
    # /!\ FOR SAFETY, READ THIS /!\
    # `max_relative_target` limits the magnitude of the relative positional target vector for safety purposes.
    # Set this to a positive scalar to have the same value for all motors, or a list that is the same length as
    # the number of motors in your follower arms.
    # For Trossen AI Arms, for every goal position request, motor rotations are capped at 5 degrees by default.
    # When you feel more confident with teleoperation or running the policy, you can extend
    # this safety limit and even removing it by setting it to `null`.
    # Also, everything is expected to work safely out-of-the-box, but we highly advise to
    # first try to teleoperate the grippers only (by commenting out the rest of the motors in this yaml),
    # then to gradually add more motors (by uncommenting), until you can teleoperate both arms fully
    max_relative_target: int | None = 5

    # Gain applied to external efforts sensed on the follower arm and transmitted to the leader arm.
    # This enables the user to feel external forces (e.g., contact with objects) through force feedback.
    # A value of 0.0 disables force feedback. A good starting value for a responsive experience is 0.1.
    force_feedback_gain: float = 0.0

    # Set this according to the camera interface you want to use.
    # "intel_realsense" is the default and recommended option.
    # "opencv" is a fallback option that uses OpenCV to access the cameras.
    camera_interface: Literal["intel_realsense", "opencv"] = "intel_realsense"

    leader_arms: dict[str, MotorsBusConfig] = field(
        default_factory=lambda: {
            "left": TrossenArmDriverConfig(
                # wxai
                ip="192.168.1.3",
                model="V0_LEADER",
            ),
            "right": TrossenArmDriverConfig(
                # wxai
                ip="192.168.1.2",
                model="V0_LEADER",
            ),
        }
    )

    follower_arms: dict[str, MotorsBusConfig] = field(
        default_factory=lambda: {
            "left": TrossenArmDriverConfig(
                ip="192.168.1.5",
                model="V0_FOLLOWER",
            ),
            "right": TrossenArmDriverConfig(
                ip="192.168.1.4",
                model="V0_FOLLOWER",
            ),
        }
    )

    if camera_interface == "opencv":
        print("Using OpenCV camera interface")
        cameras: dict[str, CameraConfig] = field(
            default_factory=lambda: {
                "cam_high": OpenCVCameraConfig(
                    camera_index=26,
                    fps=30,
                    width=640,
                    height=480,
                ),
                "cam_low": OpenCVCameraConfig(
                    camera_index=14,
                    fps=30,
                    width=640,
                    height=480,
                ),
                "cam_left_wrist": OpenCVCameraConfig(
                    camera_index=8,
                    fps=30,
                    width=640,
                    height=480,
                ),
                "cam_right_wrist": OpenCVCameraConfig(
                    camera_index=20,
                    fps=30,
                    width=640,
                    height=480,
                ),
            }
        )
    elif camera_interface == "intel_realsense":
        # Troubleshooting: If one of your IntelRealSense cameras freeze during
        # data recording due to bandwidth limit, you might need to plug the camera
        # on another USB hub or PCIe card.
        cameras: dict[str, CameraConfig] = field(
            default_factory=lambda: {
                "cam_high": IntelRealSenseCameraConfig(
                    serial_number=218622270304,
                    fps=30,
                    width=640,
                    height=480,
                ),
                "cam_low": IntelRealSenseCameraConfig(
                    serial_number=130322272628,
                    fps=30,
                    width=640,
                    height=480,
                ),
                "cam_left_wrist": IntelRealSenseCameraConfig(
                    serial_number=218622274938,
                    fps=30,
                    width=640,
                    height=480,
                ),
                "cam_right_wrist": IntelRealSenseCameraConfig(
                    serial_number=128422271347,
                    fps=30,
                    width=640,
                    height=480,
                ),
            }
        )
    else:
        raise ValueError(
            f"Unknown camera interface: {camera_interface}. Supported values are 'opencv' and 'intel_realsense'."
        )

    mock: bool = False

Setup IP Address

Note

By default, the IP address for a Trossen AI arm is set to 192.168.1.2. Make sure to change the IP addresses of your Trossen AI arms to match the ones in the configuration file.

To set up the IP address for the Trossen AI Arms, you must first ensure that the arms are connected to the same network as your computer. Refer to Ethernet Setup for correct connection instructions. Once connected, you can find or configure the IP address using the configure_cleanup demo or the set_ip_method and set_manual_ip demos.

Camera Serial Number

There are two ways to set up the camera serial numbers for the Trossen AI Kits with LeRobot: using the Intel RealSense interface or the OpenCV interface. Based on the camera interface you choose, follow the appropriate steps below to set up the camera serial numbers. You can setup both interfaces at the same time, and use the --robot.camera_interface argument to switch between them. By default, the camera interface is set to intel_realsense. We will look at this in more detail in the next sections.

Intel RealSense InterfaceOpenCV Interface

1. Open realsense-viewer

   realsense-viewer
   

   Note

   If realsense-viewer is not already installed on your machine, follow these steps on the librealsense GitHub repository to install librealsense2-utils.

2. Plug in a single camera and check the sidebar for its entry. If it does not show up in the side bar, click Add Source and find the Intel RealSense D405 in the drop down.

3. Click on Info for the camera, find the Serial Number, and copy it.

   

4. Put the camera serial number in the appropriate config entry at lerobot/common/robot_devices/robots/configs.py.

5. Repeat for the rest of the cameras.