Policy
| Module | Description |
|---|---|
roboloom.policy |
roboloom.policy defines what the policy node runs and the functions that load a policy. How to use them is in the Policies and inference chapter (in particular Custom policies); this page holds the descriptions generated from the docstrings. |
The policy contract
The policy contract for inference in the dataflow (see controllers.policy).
A policy is any object with reset() and infer(observation). The observation uses the
dataset's feature names, so a policy sees what it was trained on:
{"observation.state": float32[n], "observation.images.camera": uint8[h, w, 3],
..., "task": "pick up the cup"}
infer returns the next joint targets in the robot's unit: one step [n] or a chunk
[H, n]. The node runs it in its own process and loads it from a "module:callable"
reference that returns the policy, e.g. my_lab.policies:load_act with kwargs.
Modules are looked up in the installed packages first, then in the robot's directory
(.roboloom/<robot>/), so a policy kept next to the configuration needs no PYTHONPATH.
A policy may also have bind(spec: PolicySpec). The node calls it once after loading,
before any episode, with the observation's names, dtypes and shapes and the action's
names. A policy checks there that it gets what it was trained on and raises otherwise,
so a mismatch refuses the start instead of failing mid-episode. Policies without bind
work as before. roboloom.policy.lerobot:load is such a policy for LeRobot checkpoints.
What a model needs beyond the latest frame is asked from the controller, not rebuilt in
every policy: observation_history: N adds observation["history"] (each observation
stacked [N, ...], oldest first, the current frame last, plus "action": the actions
executed so far) and hold: names joints the policy leaves where they are. A policy may
state these itself with options() (a dict with observation_history, hold,
action_horizon), which wins over the YAML, so swapping the model needs no edited
configuration. Optional hooks() -> Sequence[Hook] (roboloom.hooks) is called after
bind. Writing contributions require an explicit from: policy slot and write limit.
Normalization, resizing and decoding the model's own action form into
joint targets stay in the policy.
roboloom.policy.Policy
Bases: Protocol
What the policy node runs: any object with reset() and infer(observation).
observation maps the dataset's feature names (observation.state,
observation.images.<camera>, ...) to the latest values, plus task (and
history with observation_history). infer returns joint targets in the
order of action and the followers' units: one step [n] or a chunk [H, n].
Optional methods: hooks() -> Sequence[Hook] (from roboloom.hooks),
called after bind to contribute local steps or lifecycle handlers;
bind(spec: PolicySpec), called once after loading to refuse a
configuration the policy was not trained for, and options(), returning
observation_history, hold or action_horizon to use instead of the YAML's.
reset()
Forget the previous episode (called when an episode starts).
infer(observation)
Joint targets: shape [n] or [H, n].
roboloom.policy.PolicySpec
dataclass
What the configuration gives a policy (bind).
observation: dataset feature name -> {"dtype", "shape", "names"}, the keys and
shapes of the observation infer receives (besides task); images are [h, w, 3].
action: the names of the action's entries, <follower>.<joint> in order; units
their units. history and hold are the controller's observation_history and
hold, inference its inference (see controllers.policy).
roboloom.policy.load_policy(reference, kwargs=None, search=())
Call the factory module:callable with kwargs; the result must be a Policy.
search: directories to import from after the installed packages (appended to
sys.path, so a local file never shadows an installed module).
A policy that holds
roboloom.policy.hold(horizon=1)
roboloom.policy:hold, the template's default policy: keep the arms where they
are, horizon steps per call.
roboloom.policy.Hold
Stays where the robot is: a policy for checking the inference path safely.
reset()
Nothing to forget.
infer(observation)
horizon copies of the current observation.state.
LeRobot
roboloom.policy.lerobot
LeRobot checkpoints as roboloom policies (roboloom.policy.lerobot:load).
type: policy
factory: roboloom.policy.lerobot:load
kwargs:
path: outputs/train/act_piper/checkpoints/last/pretrained_model # or a hub repo id
device: auto # cuda when available, else cpu
mode: step # step: select_action, one step per call (action_horizon: 1)
# chunk: predict_action_chunk, [H, n] per call
rename: {} # {checkpoint feature: roboloom feature}
Loading follows LeRobot 0.7.0's own rollout: the config and weights from the checkpoint,
then its saved pre- and post-processors, which hold the normalization statistics.
bind checks the checkpoint's input and output features against what the configuration
observes and drives, so a policy trained on other cameras or joints is refused at load.
mode: step lets the LeRobot policy keep its own action queue and observation history,
as LeRobot's rollout does; it is right for every policy type (Diffusion needs the
history). mode: chunk returns the whole predicted chunk for roboloom's
action_horizon; it skips the history, so use it only for single-frame policies (ACT).
LeRobotPolicy
A LeRobot checkpoint with its pre- and post-processors, as a roboloom policy
(made by load; see the module's description of mode and rename).
inputs()
The checkpoint's input features the policy reads -> (type, shape).
LeRobot's training lists every observation.* of the dataset as an input
(observation.meta.*, observation.effort, a Sense's pose, ...), but its
policies read only the robot state, the images and the environment state.
bind(spec)
Refuse a configuration whose observation or action does not match the checkpoint's input and output features.
reset()
Clear the model's action queue and the processors' state.
infer(observation)
Joint targets from the checkpoint: one step [n] (mode: step) or the
predicted chunk [H, n] (mode: chunk).
load(path, **kwargs)
roboloom.policy.lerobot:load: a LeRobot checkpoint (local directory or hub id).
Another process
Mock only
roboloom.policy.remote
A policy that runs in another process, e.g. another Python environment or a GPU machine.
The policy node loads roboloom.policy.remote:connect, which forwards bind, reset
and infer, negotiates protocol-2 lifecycle events, and refuses server step hooks.
Steps always run locally. Lifecycle calls share the inference call lock. A server that holds the real policy:
# where the model lives (its own environment; only numpy and roboloom are needed)
export ROBOLOOM_POLICY_AUTHKEY=... # any shared secret
python -m roboloom.policy.remote my_lab.policies:load --kwargs '{"checkpoint": "..."}'
# nodes/policy.yaml
factory: roboloom.policy.remote:connect
kwargs: {address: "127.0.0.1:18765"}
This is for models whose dependencies do not fit the environment roboloom runs in (a different torch) or that should run on another machine. The server loads the model once and serves one policy node at a time, across episodes and sessions; a second node that connects meanwhile is told so instead of being left waiting.
Every call carries a number the answer repeats. An answer that arrives after its call timed out is dropped, so it is never taken for the answer to the next call.
Wire format: multiprocessing.connection (the key authenticates both ends with HMAC;
nothing is encrypted, so keep it on the loopback interface or a trusted network). A
message is a JSON header followed by the raw bytes of its arrays: no pickle, so a peer
cannot run code here and numpy versions may differ.
RemotePolicy
The Policy the node loads; every call is answered by the server's policy.
hooks()
Only lifecycle forwarding; per-step hooks always run in the local node.
options()
The served policy's options() (empty when it has none).
bind(spec)
Forward bind to the served policy.
reset()
Forward reset to the served policy.
infer(observation)
The served policy's answer to observation.
close()
Close the connection to the server.
RemotePolicyError
Bases: RuntimeError
The policy on the server raised; the message carries its type and text.
connect(address=ADDRESS, authkey_env=AUTHKEY_ENV, timeout_s=30.0, connect_timeout_s=30.0)
roboloom.policy.remote:connect: the policy served at address (host:port).
timeout_s bounds one call (an infer), connect_timeout_s the wait for the
server to come up; the secret is read from the environment variable authkey_env.
serve(policy, address=ADDRESS, authkey=None, kwargs=None, authkey_env=AUTHKEY_ENV, clients=None, ready=None)
Answer policy nodes at address with policy (an object with reset and
infer, or a "module:callable" factory called with kwargs), one at a time, until
interrupted (or after clients connections). ready, if given, is called once the
server listens.
Replay
Mock only
roboloom.policy.replay
Replay recorded actions as a policy (roboloom.policy.replay:load).
factory: roboloom.policy.replay:load
kwargs:
path: datasets/rakuda2 # a recorded LeRobotDataset, or a .npy file [T, n]
episode: 0 # which episode of the dataset
stride: 1 # every stride-th row: rate_hz = dataset fps / stride
action_horizon: 8
The recorded action rows go to the followers in order, whatever is observed (open
loop). It checks the path from a recording to the robot without a model: the same
controller, speed limit, interpolation and safety checks as a learned policy, so the
recorded observation.state of the replay can be compared with the original. A .npy
file replays actions from elsewhere, e.g. to check the joint order and units of data
recorded with another tool: move the robot to the episode's first pose first (home)
and keep max_speed low.
After the last action the policy keeps returning it (the followers stay there).
inference: async is refused: a replay's chunks follow one another, they are not
computed from the observation's time, so replacing a queued chunk would skip rows.
Replay
Recorded actions [T, n] returned horizon rows per call, open loop (made by
load).
options()
action_horizon: the rows returned per call.
bind(spec)
Refuse a configuration whose action size (and, for a recording, names
and rate) differ from the recorded ones, or that uses inference: async.
reset()
Start again from the first row.
infer(observation)
The next horizon rows, whatever is observed (the last row once done).
load(path, episode=0, horizon=8, stride=1)
roboloom.policy.replay:load: the actions of one recorded episode, or of a
.npy file [T, n] (then stride only thins the rows and nothing is checked
against the configuration but the action's size).
Hooks
Mock only
roboloom.hooks.Requirements
dataclass
Resources a hook needs. Future API additions have defaults.
roboloom.hooks.HookSpec
dataclass
The node's action layout and features of its observed sources only.
roboloom.hooks.Hook
Optional methods: requirements, bind, step, on_event and close.
A step implementation accepts (action: np.ndarray, ctx: StepContext) and
returns an array or None (unchanged). Event handlers run on a per-hook worker.
bind(spec)
Raise ConfigError to refuse the node's configuration.
on_event(event)
Route an event to the optional on_<kind>(event) method.
close()
Release hook-owned resources after draining the event queue.
roboloom.hooks.StepContext
dataclass
One step invocation; snapshots and observations are built only when read.
previous is the last command sent, held the follower's current target,
and state its measured position. Arrays and observations are read-only.
note accepts up to 16 small scalar fields, retained in this hook's stats.
roboloom.hooks.ChunkPosition
dataclass
roboloom.hooks.EpisodeStarted
dataclass
roboloom.hooks.EpisodeEnded
dataclass
roboloom.hooks.ControlChanged
dataclass
roboloom.hooks.PolicyRestarted
dataclass
roboloom.hooks.ObservationStale
dataclass
roboloom.hooks.HookChain
Run bound hooks in order; step and event methods of one hook never overlap.
start_timeout_s is injectable for tests. Event errors are logged; a barrier
timeout uses on_error. No synchronous step can be interrupted by a time budget.
run(action, ctx)
One period. Each hook sees the preceding output and only its observation.
emit(event, *, rate_hz=1.0, settle=None)
Queue for every event hook; barriers wait only for hooks with a step.
close(timeout=DRAIN_TIMEOUT_S)
Drain workers, then close. On timeout the worker closes after its queue.