Record Episodes

Once you’re familiar with teleoperation, you can record your first dataset with Trossen AI.

Logging into Hugging Face

If you want to use Hugging Face Hub features for uploading your dataset and you haven’t done so before, make sure you’ve logged in using a write-access token, which can be generated from the Hugging Face settings.

huggingface-cli login --token ${HUGGINGFACE_TOKEN} --add-to-git-credential

Store your Hugging Face repository name in a variable:

HF_USER=$(huggingface-cli whoami | head -n 1)
echo $HF_USER

Recording and Uploading a Dataset

Record 2 episodes and upload your dataset to the Hugging Face Hub:

Trossen AI StationaryTrossen AI MobileTrossen AI Solo

python lerobot/scripts/control_robot.py \
--robot.type=trossen_ai_stationary \
--robot.max_relative_target=null \
--control.type=record \
--control.fps=30 \
--control.single_task="Test recording episode using Trossen AI Stationary." \
--control.repo_id=${HF_USER}/trossen_ai_stationary_test \
--control.tags='["tutorial"]' \
--control.warmup_time_s=5 \
--control.episode_time_s=30 \
--control.reset_time_s=30 \
--control.num_episodes=2 \
--control.push_to_hub=true

Note

The units for each joint are as follows:

• Joints 0-5: Radians

• Joint 6 (Gripper): Millimeters (mm)

Warning

We have introdced a new change in how the values are saved in the dataset. The values are now saved in the dataset as radians for all joints and no scaling is applied for the gripper. If you are using a previous version of the dataset, the values for joints 0-5 will be in degrees and a scaling of 10000 will be applied to gripper. Check Trossen v1.0 Dataset Format before using datasets from previous versions.

Note

The video encoding process can be resource-intensive. This can cause longer wait times between episodes, especially if you are recording at longer episode lengths. To mitigate this, consider adjusting the save_interval parameter to save data less frequently or at the end of the entire session.

Handling Camera FPS Issues

Note: If the camera FPS is unstable, consider increasing the number of image writers per thread or disable camera display.

Trossen AI StationaryTrossen AI MobileTrossen AI Solo

python lerobot/scripts/control_robot.py \
--robot.type=trossen_ai_stationary\
--robot.max_relative_target=null \
--control.type=record \
--control.fps=30 \
--control.single_task="Test recording episode using Trossen AI Stationary." \
--control.repo_id=${HF_USER}/trossen_ai_stationary_test \
--control.tags='["tutorial"]' \
--control.warmup_time_s=5 \
--control.episode_time_s=30 \
--control.reset_time_s=30 \
--control.num_episodes=2 \
--control.push_to_hub=true \
--control.num_image_writer_threads_per_camera=8 \
--control.display_cameras=false

If using IntelRealsense camera interface is causing fps issues, you can try using the OpenCV interface instead. Make sure that you have configured the cameras correctly as described in Camera Serial Number.

Trossen AI StationaryTrossen AI MobileTrossen AI Solo

python lerobot/scripts/control_robot.py \
--robot.type=trossen_ai_stationary \
--robot.max_relative_target=null \
--control.type=record \
--control.fps=30 \
--control.single_task="Test recording episode using Trossen AI Stationary." \
--control.repo_id=${HF_USER}/trossen_ai_stationary_test \
--control.tags='["tutorial"]' \
--control.warmup_time_s=5 \
--control.episode_time_s=30 \
--control.reset_time_s=30 \
--control.num_episodes=2 \
--control.push_to_hub=true \
--control.num_image_writer_threads_per_camera=8 \
--control.display_cameras=false \
--camera.interface_type=opencv

Recording Configuration

When recording a dataset, you can specify command line arguments to customize the behavior:

• --control.fps (int): The number of frames per second to record. By default, uses the policy fps.

• --control.single_task (str): The task description for the episode (e.g. “Pick the Lego block and drop it in the box on the right.”).

• --control.repo_id (str): The repository ID to upload the dataset to. By convention, it should match ‘{hf_username}/{dataset_name}’ (e.g. lerobot/test).

• --control.tags (list[str]): The tags to add to the dataset.

• --control.warmup_time_s (int | float): The duration of the warm-up phase in seconds. It allows the robot devices to warm up and synchronize.

• --control.episode_time_s (int | float): The duration of the episode in seconds.

• --control.reset_time_s (int | float): The duration of the reset phase in seconds.

• --control.num_episodes (int): The number of episodes to record.

• --control.push_to_hub (bool): Flag to upload the dataset to the Hugging Face Hub.

• --control.num_image_writer_threads_per_camera (int): The number of image writer threads per camera. Too many threads might cause unstable teleoperation fps due to the main thread being blocked. Not enough threads might cause low camera fps.

• --control.root (str | Path | None): The root directory to save the dataset to (e.g. ‘dataset/path’).

• --control.device (str | None): The device to use for computation (e.g. ‘cuda’, ‘cpu’, ‘mps’).

• --control.use_amp (bool | None): Flag to use Automatic Mixed Precision (AMP) for training and evaluation.

• --control.video (bool): Flag to encode frames in the dataset into video.

• --control.run_compute_stats (bool): Flag to run the computation of the data statistics at the end of data collection.

• --control.private (bool): Flag to upload the dataset to a private repository on the Hugging Face Hub.

• --control.num_image_writer_processes (int): The number of subprocesses handling the saving of frames as PNGs.

• --control.display_cameras (bool): Flag to display all cameras on screen.

• --control.play_sounds (bool): Flag to use vocal synthesis to read events.

• --control.resume (bool): Flag to resume recording on an existing dataset.

• --control.local_files_only (bool): Flag to use local files only.

• --control.save_interval (int): Interval in episodes at which to encode images to video and save data to disk.

  For example, if set to 5, data will be saved every 5 episodes. Default is 1 (save after every episode). Setting save interval to -1, 0, or > total number of episodes will only save data at the end of the entire data collection session.

Experimental Feature

Note

The following features are experimental and may not be fully stable. Use them at your own discretion.

1. GPU Acceleration for Video Encoding

   The application supports GPU acceleration for video encoding using NVIDIA GPUs with CUDA support. To enable this feature, ensure that you have the necessary NVIDIA drivers and CUDA toolkit installed on your system. You can enable GPU acceleration by setting the LEROBOT_GPU_ENCODING environment variable.

   export LEROBOT_GPU_ENCODING=1
   

   You can also choose the GPU device by setting the LEROBOT_GPU_ID environment variable to the desired GPU index (default is 0).

   export LEROBOT_GPU_ID=0
   

   The GOP (Group of Pictures) size for video encoding can be adjusted by setting the LEROBOT_GOP_SIZE environment variable (default is 30). A smaller GOP size can lead to better video quality but may increase file size. GPU encoding requires the GOP size to be a multiple of the FPS.

   export LEROBOT_GOP_SIZE=30