Data Collection
This page describes how to record a data collection session and the best practices that keep your SLAM success rate high. Recording quality is the single biggest factor in how much of your data is usable, so it is worth following these steps carefully.
Sessions and Datasets
A dataset consists of one or more sessions. A session is a distinct unit of data collection, all originating from the same location. Each session includes three key components:
A SLAM mapping video — used to build a 3D map of the environment.
A gripper calibration video — used to calibrate gripper finger tag detection.
A series of task demonstration videos — the demonstrations themselves.
Record these in order for each session.
Markers and Identifiers
TRumi uses two kinds of visual markers, both included in the kit:
Mapping marker — a printed ArUco target placed in the scene to anchor the SLAM map and set metric scale.
Gripper finger identifiers — multicolor identifiers embedded in the finger mounts, based on the ArUco tags listed below (IDs 0, 1, 6, 7), used to measure gripper width.
Printable PDFs of every marker are available in the table below, in case you ever need a replacement.
Important
Whenever you do print a marker, the physical printed size must match the table below exactly. Incorrect sizes produce inaccurate poses and gripper widths.
Marker |
Dictionary |
ID |
Size |
|
|---|---|---|---|---|
Mapping |
DICT_4X4_50 |
13 |
0.16 m |
|
Gripper 0 left |
DICT_4X4_50 |
0 |
0.016 m |
|
Gripper 0 right |
DICT_4X4_50 |
1 |
0.016 m |
|
Gripper 1 left |
DICT_4X4_50 |
6 |
0.016 m |
|
Gripper 1 right |
DICT_4X4_50 |
7 |
0.016 m |
Step 1: Mapping Video
The mapping video is used by ORB-SLAM3 to build a 3D map of the environment. SLAM success rate is highly sensitive to the scene and to how you record this video.
Scene selection tips:
Prefer environments with enough visual texture.
Avoid large plain surfaces (white walls, bare ceilings, empty corners).
Place the printed mapping marker on the work surface where you will record your demonstrations. To record the mapping video, hold the gripper and move slowly and smoothly through the workspace, viewing the scene from a range of positions, heights, and angles so SLAM captures enough of the environment. Keep the mapping marker in frame, avoid fast motion and motion blur, and revisit areas you have already covered so SLAM can close loops. Correct marker placement and thorough coverage are critical for SLAM success rate.
For reference, see the example mapping video from the example dataset.
Step 2: Gripper Calibration Video
Record a short video of opening and closing the gripper 5 times. This is used to calibrate gripper finger tag detection.
For reference, see the example gripper calibration video from the example dataset.
Step 3: Demonstrations
Record N demonstration videos.
For reference, the example dataset includes two sample demonstration videos: episode 1 and episode 2.
Best Practices
Texture matters. SLAM relies on visual features. Richly textured scenes track far better than plain ones.
Keep the mapping marker visible and correctly placed during the mapping step. It anchors the map and sets scale.
Move smoothly. Avoid rapid jerks and motion blur, which degrade both SLAM and ArUco detection.
Keep lighting consistent. Avoid strong glare and heavy shadows on the markers.
Verify your printed marker sizes before collecting a large batch of data.
For bimanual collection, confirm timecode sync (see timecode sync) before recording.
If your dataset has a low SLAM success rate, revisit these practices — most low-yield sessions trace back to a poor mapping video or an incorrectly sized/placed marker.
Once you have recorded a session, continue to Dataset Generation Pipeline to process it.