Warning

You are viewing the documentation for a version of the Interbotix ALOHA stack that is no longer supported. Please upgrade to keep up to date with the latest features. See the v2.0 documentation for more information.

Training and Evaluation Guide

This guide covers the training and evaluation workflows for ACT (Aloha Stationary) and ACT++ (Aloha Mobile).

Setup and Environment

You can use either Virtual Environment (VENV) or Conda for containerization based on your preference. Follow the steps below to set up your environment:

Virtual Environment Setup

1. Install the virtual environment package:

   sudo apt-get install python3-venv
   

2. Create and activate a virtual environment:

   python3 -m venv ~/act
   source ~/act/bin/activate
   

Conda Setup

1. Create and activate a Conda virtual environment:

   conda create -n aloha python=3.8.10 -y
   conda activate aloha
   

ACT Workflow (Aloha Stationary)

Install Dependencies (ACT)

Install the necessary dependencies inside your containerized environment:

$ pip install dm_control==1.0.14
$ pip install einops
$ pip install h5py
$ pip install ipython
$ pip install matplotlib
$ pip install mujoco==2.3.7
$ pip install opencv-python
$ pip install packaging
$ pip install pexpect
$ pip install pyquaternion
$ pip install pyyaml
$ pip install rospkg
$ pip install torch
$ pip install torchvision

Clone ACT Repository

$ cd ~
$ git clone https://github.com/Interbotix/act.git act_training_evaluation

Build and Install ACT Models

 ├── act
 │   ├── assets
 │   ├── constants.py
 │   ├── detr
 │   ├── ee_sim_env.py
 │   ├── imitate_episodes.py
 │   ├── __init__.py
 │   ├── policy.py
 │   ├── record_sim_episodes.py
 │   ├── scripted_policy.py
 │   ├── sim_env.py
 │   ├── utils.py
 │   └── visualize_episodes.py
 ├── COLCON_IGNORE
 ├── conda_env.yaml
 ├── LICENSE
 └── README.md

Navigate to the detr directory inside the repository and install the detr module which contains the model definitions using the below command:

$ cd /path/to/act/detr && pip install -e .

Training (ACT)

To start the training, follow the steps below:

1. Sanity Check:

   Ensure you have all the hdf5 episodes located in the correct folder after following the data collection steps Task Creation.

2. Source ROS Environment:

   $ source /opt/ros/humble/setup.bash
   $ source interbotix_ws/install/setup.bash
   

3. Activate Virtual Environment:

   $ source act/bin/activate
   

4. Start Training

   $ cd /path/to/act/repository/
   $ python3 imitate_episodes.py \
     --task_name aloha_stationary_dummy \
     --ckpt_dir <ckpt dir> \
     --policy_class ACT \
     --kl_weight 10 \
     --chunk_size 100 \
     --hidden_dim 512 \
     --batch_size 8 \
     --dim_feedforward 3200 \
     --num_epochs 2000 \
     --lr 1e-5 \
     --seed 0
   

Note

• task_name argument should match one of the task names in the TASK_CONFIGS, as configured in the Task Creation section.
• ckpt_dir: The relative location where the checkpoints and best policy will be stored.
• policy_class: Determines the choice of policy ‘ACT’/’CNNMLP’.
• kl_weight: Controls the balance between exploration and exploitation.
• chunk_size: Determines the length of the action sequence. K=1 is no action chunking and K=episode length is full open loop control.
• batch_size: Low batch size leads to better generalization and high batch size results in slower convergence but faster training time.
• num_epochs: Too many epochs lead to overfitting; too few epochs may not allow the model to learn.
• lr: Higher learning rate can lead to faster convergence but may overshoot the optima, while lower learning rate might lead to slower but stable optimization.

Tip

We recommend the following parameters:

Parameter	Value
Policy Class	ACT
KL Weight	10
Chunk Size	100
Batch Size	2
Num of Epochs	3000
Learning Rate	1e-5

Evaluation (ACT)

To evaluate a trained model, follow the steps below:

1. Bringup the ALOHA

2. Configure the environment

   $ source /opt/ros/humble/setup.bash  # Configure ROS system install environment
   $ source interbotix_ws/install/setup.bash  # Configure ROS workspace environment
   $ source /<path_to_aloha_venv>/bin/activate  # Configure ALOHA Python environment
   $ cd ~/<act_repository>/act/
   

3. Run the evaluation script

    python3 imitate_episodes.py \
     --task_name aloha_stationary_dummy \
     --ckpt_dir <ckpt dir> \
     --policy_class ACT \
     --kl_weight 10 \
     --chunk_size 100 \
     --hidden_dim 512 \
     --batch_size 8 \
     --dim_feedforward 3200 \
     --num_epochs 2000 \
     --lr 1e-5 \
     --seed 0 \
     --eval \
     --temporal_agg
   

Note

• The task_name argument should match one of the task names in the TASK_CONFIGS, as configured in the Task Creation section.
• The ckpt_dir argument should match the correct relative directory location of the trained policy.
• The eval flag will set the script into evaluation mode.
• The temporal_agg is not required, but helps to smoothen the trajectory of the robots.

ACT++ Workflow (Aloha Mobile)

Install Dependencies (ACT++)

Install the necessary dependencies inside your containerized environment:

$ pip install dm_control==1.0.14
$ pip install einops
$ pip install h5py
$ pip install ipython
$ pip install matplotlib
$ pip install modern_robotics
$ pip install mujoco==2.3.7
$ pip install opencv-python
$ pip install packaging
$ pip install pexpect
$ pip install pyquaternion
$ pip install pyyaml
$ pip install rospkg
$ pip install torch
$ pip install torchvision
$ pip install transforms3d
$ pip install wandb

Clone Robomimic

$ cd ~
$ git clone -b r2d2 https://github.com/ARISE-Initiative/robomimic.git
$ cd robomimic
$ pip install -e .

Clone ACT++ Repository

$ cd ~
$ git clone https://github.com/Interbotix/act_plus_plus.git act_training_evaluation

Build and Install ACT++ Models

 ├── act
 │   ├── assets
 │   ├── constants.py
 │   ├── detr
 │   ├── ee_sim_env.py
 │   ├── imitate_episodes.py
 │   ├── __init__.py
 │   ├── policy.py
 │   ├── record_sim_episodes.py
 │   ├── scripted_policy.py
 │   ├── sim_env.py
 │   ├── utils.py
 │   └── visualize_episodes.py
 ├── COLCON_IGNORE
 ├── conda_env.yaml
 ├── LICENSE
 └── README.md

Navigate to the act_plus_plus directory inside the repository and install the required modules for model definitions by running the following command:

$ cd /path/to/act_plus_plus && pip install -e .

Training (ACT++)

To start the training, follow the steps below:

1. Sanity Check:

   Ensure you have all the hdf5 episodes located in the correct folder after following the data collection steps Task Creation.

2. Source ROS Environment:

   $ source /opt/ros/humble/setup.bash
   $ source interbotix_ws/install/setup.bash
   

3. Activate Virtual Environment:

   $ source act/bin/activate
   

4. Start Training

   $ cd /path/to/act/repository/
   $ python3 imitate_episodes.py \
     --task_name aloha_mobile_dummy \
     --ckpt_dir <ckpt dir> \
     --policy_class ACT \
     --kl_weight 10 \
     --chunk_size 100 \
     --hidden_dim 512 \
     --batch_size 8 \
     --dim_feedforward 3200 \
     --num_steps 2000 \
     --lr 1e-5 \
     --seed 0
   

Note

• task_name argument should match one of the task names in the TASK_CONFIGS, as configured in the Task Creation section.
• ckpt_dir: The relative location where the checkpoints and best policy will be stored.
• policy_class: Determines the choice of policy ‘ACT’/’CNNMLP’.
• kl_weight: Controls the balance between exploration and exploitation.
• chunk_size: Determines the length of the action sequence. K=1 is no action chunking and K=episode length is full open loop control.
• batch_size: Low batch size leads to better generalization and high batch size results in slower convergence but faster training time.
• num_steps: Too many steps lead to overfitting; too few steps may not allow the model to learn.
• lr: Higher learning rate can lead to faster convergence but may overshoot the optima, while lower learning rate might lead to slower but stable optimization.

Tip

We recommend the following parameters:

Parameter	Value
Policy Class	ACT
KL Weight	10
Chunk Size	100
Batch Size	2
Num of Steps	3000
Learning Rate	1e-5

Evaluation (ACT++)

To evaluate a trained model, follow the steps below:

1. Bringup the ALOHA

2. Configure the environment

   $ source /opt/ros/humble/setup.bash  # Configure ROS system install environment
   $ source interbotix_ws/install/setup.bash  # Configure ROS workspace environment
   $ source /<path_to_aloha_venv>/bin/activate  # Configure ALOHA Python environment
   $ cd ~/<act_repository>/act_plus_plus/
   

3. Run the evaluation script

    python3 imitate_episodes.py \
     --task_name aloha_stationary_dummy \
     --ckpt_dir <ckpt dir> \
     --policy_class ACT \
     --kl_weight 10 \
     --chunk_size 100 \
     --hidden_dim 512 \
     --batch_size 8 \
     --dim_feedforward 3200 \
     --num_steps 2000 \
     --lr 1e-5 \
     --seed 0 \
     --eval \
     --temporal_agg
   

Note

• The task_name argument should match one of the task names in the TASK_CONFIGS, as configured in the Task Creation section.
• The ckpt_dir argument should match the correct relative directory location of the trained policy.
• The eval flag will set the script into evaluation mode.
• The temporal_agg is not required, but helps to smoothen the trajectory of the robots.

Major Differences Between ACT and ACT++

• Repository: ACT uses act, while ACT++ uses act_plus_plus.
• Robomimic Dependency: Required for ACT++ but not for ACT.
• Training Argument: ACT uses num_epochs, while ACT++ uses num_steps.
• Installation Directory: For ACT, pip install is run inside detr, while for ACT++ it’s in the root directory.