Rakuda2
Hardware-verified
Rakuda2 has 17 joints: 3 torso/head joints, plus 6 arm joints and 1 gripper joint on each of the left and right sides. Joint values are raw Dynamixel position counts, and current is in mA.
The follower is type: rakuda2_follower and the operator input, in the leader role, is the Rakuda-mini arm (type: rakuda_mini). The leader and follower use separate Dynamixel buses, each owned exclusively by its node. Create the configuration with roboloom init rakuda2.
Supported controllers
| Controller | Options | Status |
|---|---|---|
Leader (joint) |
none | Hardware-verified |
| Gravity-compensated leader | gravity |
Hardware-verified |
| Policy | policy |
Hardware-verified |
| Policy with interventions | policy, policy_teleop |
Hardware-verified |
| Pika Sense and IK | pika |
In development |
Configuration examples are in Rakuda2 controllers.
Supported sensors
| Sensor | Node name | How to enable | Status |
|---|---|---|---|
| RealSense | camera |
on by default | Hardware-verified |
| DIGIT | digit_left, digit_right |
option digit |
Hardware-verified |
| Audio | audio |
choose with --sensors |
In development |
Combinations and how to choose are in Rakuda2 sensors.
Observation and action
The recorded dataset and the policy's inputs and outputs share the same order and units.
| Name | Entries | Unit |
|---|---|---|
observation.state, action |
17 (follower.torso_yaw, follower.head_yaw, follower.head_pitch, the right arm follower.r_arm_sh_pitch1 … follower.r_arm_grip, the left arm follower.l_arm_sh_pitch1 … follower.l_arm_grip) |
Dynamixel position counts |
observation.effort |
17 (same order; record_effort: true in the template) |
mA (current) |
observation.leader |
17 (the leader's joint positions, same order; names without a prefix, torso_yaw … l_arm_grip; not recorded with option policy, which has no leader; recorded with policy_teleop) |
Dynamixel position counts |
observation.images.camera |
480×640×3 (RGB) | storage |
observation.images.digit_left, observation.images.digit_right |
320×240×3 (RGB; option digit) |
storage |
observation.meta.<node>.valid, capture_age_ms, ready_age_ms |
1 each (float32), for every recorded node: leader, follower, camera, and with digit also digit_left, digit_right |
valid 0/1, ages in ms |
observation.meta.<node>.seq, capture_ns, ready_ns, receive_ns |
1 each (int64), same nodes | sequence number, monotonic-clock ns |
action.valid, action.clamped |
1 each (float32) | 0/1 |
action.meta.follower.valid, clamped, seq, sent_ns |
1 each (valid, clamped float32; seq, sent_ns int64) |
0/1, command number, monotonic-clock ns |
action.intervention |
1 (float32; only with option policy_teleop) |
1 in rows a person drove with the leader |
observation.audio.audio |
only when the sensor audio is selected; shape in Audio |
mel spectrogram |
- What
observation.leader,observation.meta.<node>.*,action.valid,action.clampedandaction.meta.follower.*mean, how missing samples are recorded, and whatactionholds in rows without a command are described in Dataset format; they are the same for every robot. - The policy (option
policy) observes joint state andcameraby default. Select sensors withobserveinnodes/policy.yaml([]for joint state only).inferreturns targets for the 17 joints in counts: one step[17]or a chunk[H, 17]. max_speedis in counts/s ({follower: 1024}in the template).storage: videoimages are H.264 by default;video_codec: av1innodes/recorder.yamlor--video-codec av1records AV1 (Video codec).- The template's default policy
roboloom.policy:holdkeeps the arms in place. How to write a policy is in Driving with a policy.
Launch options and combinations
There is one configuration, .roboloom/rakuda2/config.yaml. With no option enabled it uses the standard leader, the joint controller, and the RealSense. The template has the following launch options (all disabled by default). See CLI for choosing them and "Launch options" in Configuration for the rules.
| Option | What it does | Requires / conflicts | Dataset name |
|---|---|---|---|
digit |
Records the left and right DIGIT | Appends -digit |
|
gravity |
Gravity-compensates the leader's left and right arms (nodes/leader_gravity.yaml) |
Unchanged | |
bilateral |
Force feedback (not verified on hardware); replaces the leader, controller, and robot and adds the estimator effort |
Requires gravity |
Appends -bilateral |
policy |
A learned policy drives the follower; removes the leader and controller and adds controllers.policy; roboloom infer enables it automatically |
Conflicts with gravity |
Appends -eval |
policy_teleop |
With policy, the leader can drive before and during an episode (teleoperation and interventions); brings the leader back and uses nodes/policy_teleop.yaml |
Requires policy |
Appends -teleop |
pika |
A Pika Sense per arm through IK instead of the leader (in development, mock only) | conflicts with gravity, policy |
appends -pika |
| Enabled options | Leader | Sensors | repo_id |
|---|---|---|---|
| none | Standard | RealSense | local/rakuda2 |
digit |
Standard | RealSense + left/right DIGIT | local/rakuda2-digit |
gravity |
With gravity compensation | RealSense | local/rakuda2 |
gravity, digit |
With gravity compensation | RealSense + left/right DIGIT | local/rakuda2-digit |
gravity, bilateral |
With gravity compensation + force feedback (not verified on hardware) | RealSense | local/rakuda2-bilateral |
policy |
None (policy) | RealSense | local/rakuda2-eval |
digit, policy |
None (policy) | RealSense + left/right DIGIT | local/rakuda2-digit-eval |
policy, policy_teleop |
Normal (before an episode and for interventions) | RealSense | local/rakuda2-eval-teleop |
Limitations:
- Use
gravityonly after the measurement steps of the Rakuda2: gravity compensation (calibrate-joints,measure-travel,identify-gravity,tune-gravity). Loading is refused while measured values are stillSET_.... - Enabling or disabling
gravitychanges the leader's Operating Mode. On a hardware launch from a terminal, the CLI shows the EEPROM differences and writes them after confirmation. policyworks without DIGIT. To pass DIGIT images to the policy, enabledigitand changeobserveinnodes/policy.yamlto[camera, digit_left, digit_right]. Enablingdigitalone does not change the policy inputs. Match the inputs used for training. Loading a LeRobot checkpoint that needs DIGIT without DIGIT is refused before starting.- For a policy using joint state only, set
observe: []. To avoid opening sensors, also remove their nodes fromsensorsinconfig.yaml. nodes/policy_teleop.yamlofpolicy_teleopisnodes/policy.yamlplusteleop: [controller.yaml]; keep the policy settings of both the same.policydoes not use the leader, so it cannot be combined withgravity, nor withbilateral, which requiresgravity.gravitydoes not change the dataset name because it records the same features as the standard leader; episodes with the standard and the gravity-compensated leader go into the same dataset.- Because the image features change,
digitseparates the dataset; DIGIT episodes are not appended to an existing RealSense-only dataset.
Bilateral (not verified on hardware)
The option bilateral is the combination that returns the external force on the follower to the leader. It has not been verified on hardware; only the mock tests pass.
| Node | Configuration | Contents |
|---|---|---|
| robot | nodes/follower_bilateral.yaml |
Runs the arms in current-based position control (Operating Mode 5, current_position). goal_current_mA is the force limit on contact. Choose SET_FOLLOWER_ARM_GOAL_CURRENT_mA and write the Mode and Current Limit with roboloom commission |
| leader | nodes/leader_bilateral.yaml |
The gravity-compensated leader with feedback: added to the arm groups |
| controller | nodes/controller_bilateral.yaml |
feedback: names the estimator and the joints that receive feedback (initially the right elbow only, with a small gain) |
| estimator | nodes/effort.yaml |
Estimator of the follower's external effort (model_residual). Uses gravity models identified on the follower, calibration/effort_gravity_<side>.yaml. The contact thresholds are provisional values, not measured |
Limitations:
- Start is refused unless
goal_current_mAis at most the EEPROMcurrent_limit. - After the motors are rewritten to Mode 5, the standard
nodes/follower.yaml(Mode 3) is refused by the startup check. To go back, write the values back withcommission follower; on a hardware launch from a terminal, the CLI shows the differences before launching and writes them back after confirmation. - Every command with
bilateralenabled (such asroboloom commission follower --with bilateral) loads the whole configuration, so it stops while the follower gravity models (calibration/effort_gravity_<side>.yaml) are missing. Identification works before the Mode is changed (still Mode 3), so identify first, thencommission. - The configuration keys are under "Force feedback" in Configuration.
Checks before launch
Verify ports, motor IDs, models, EEPROM, joint ranges, and speed/current limits on the hardware. The template has torque set to true, so if you want to check the pipeline the first time without moving the follower, change nodes/follower.yaml to torque: false before launch. Leader-side control is independent; the trigger and gravity compensation are energized.
The control rate is 120 Hz in the templates. For its conditions and how to lower it, see Control rate.
For the steps from setup to data collection, see Rakuda2: configuration and read check; for the meaning of the control values, see Gravity tuning.