Policies and inference
A type: policy controller drives the followers with a learned policy. Policies you write yourself and LeRobot checkpoints are loaded the same way.
What has been checked
The path for your own policies is used with the Rakuda2 configuration and with mock devices. LeRobot policies (roboloom.policy.lerobot) have been checked only with the mock PiPER + Pika configuration: an ACT trained briefly on CPU with lerobot-train on mock recordings ran the whole path from recording to inference. They have not been checked on hardware or with a well-trained policy.
How it fits together
flowchart LR
obs["observation<br/>(state, images, inputs)"] --> policy["policy.infer()"]
policy --> chunk["action chunk [H, n]"]
chunk --> interp["interpolation<br/>rate_hz × N"]
interp --> follower["follower<br/>(max_speed, FollowerSafety)"]
follower --> obs
leader["leader (teleop)"] -. intervention .-> follower
follower --> recorder["recorder → LeRobotDataset"]
- The policy node hands the policy an observation: the followers' joint state and the inputs and sensors listed in
observe(Your own policy). - The policy returns one step
[n]or a chunk[H, n]. The policy node takes one action every1/rate_hzand interpolates between actions, so commands go out close to the followers' control rate (Asynchronous inference and interpolation). - Targets are cut to
max_speed, andFollowerSafetychecks the joint limits and the deadline, as in teleoperation. - With
teleop:, a person can take over with the leader during an episode and hand back to the policy (Teleoperation and interventions). - The recorder writes the commands actually sent to the followers to
action(Dataset format).
Pages
| Page | Content | Status |
|---|---|---|
| Running a policy | nodes/policy.yaml, roboloom infer, rate and steps, home, Python, sensor selection |
Mock only (sensor selection), In development (PiPER + Pika) |
| Your own policy | The policy's shape, bind, choosing observations, failures, history and hold, your own model |
— |
| LeRobot policies | Loading a checkpoint, training with lerobot-train |
Mock only |
| Policy server | Running the policy in another process or environment (serve-policy) |
Mock only |
| Asynchronous inference and interpolation | inference: async, action interpolation, pchip |
Mock only (async, pchip) |
| Teleoperation and interventions | teleop:, interventions, what is recorded |
— |
| Evaluation and replay | Recording verdicts, replaying a recording | Mock only (replay) |
The labels are defined in Status labels. The keys are listed in Policy configuration and the commands in the CLI.
Limitations
- LeRobot's RTC is not supported. Asynchronous inference replaces the queue without blending the chunks.
- ACT's temporal ensembling (
temporal_ensemble_coeff) works only withmode: step.