コンテンツにスキップ

Policy

Module 説明
roboloom.policy roboloom.policy は,方策nodeが動かす方策の形と,方策を読み込む関数です.使い方は 方策と推論 の章(特に 自作方策)にあり,このページは docstring から生成した説明(英語)を載せます.

The policy contract

The policy contract for inference in the dataflow (see controllers.policy).

A policy is any object with reset() and infer(observation). The observation uses the dataset's feature names, so a policy sees what it was trained on:

{"observation.state": float32[n], "observation.images.camera": uint8[h, w, 3],
 ..., "task": "pick up the cup"}

infer returns the next joint targets in the robot's unit: one step [n] or a chunk [H, n]. The node runs it in its own process and loads it from a "module:callable" reference that returns the policy, e.g. my_lab.policies:load_act with kwargs. Modules are looked up in the installed packages first, then in the robot's directory (.roboloom/<robot>/), so a policy kept next to the configuration needs no PYTHONPATH.

A policy may also have bind(spec: PolicySpec). The node calls it once after loading, before any episode, with the observation's names, dtypes and shapes and the action's names. A policy checks there that it gets what it was trained on and raises otherwise, so a mismatch refuses the start instead of failing mid-episode. Policies without bind work as before. roboloom.policy.lerobot:load is such a policy for LeRobot checkpoints.

What a model needs beyond the latest frame is asked from the controller, not rebuilt in every policy: observation_history: N adds observation["history"] (each observation stacked [N, ...], oldest first, the current frame last, plus "action": the actions executed so far) and hold: names joints the policy leaves where they are. A policy may state these itself with options() (a dict with observation_history, hold, action_horizon), which wins over the YAML, so swapping the model needs no edited configuration. Optional hooks() -> Sequence[Hook] (roboloom.hooks) is called after bind. Writing contributions require an explicit from: policy slot and write limit. Normalization, resizing and decoding the model's own action form into joint targets stay in the policy.

roboloom.policy.Policy

Bases: Protocol

What the policy node runs: any object with reset() and infer(observation).

observation maps the dataset's feature names (observation.state, observation.images.<camera>, ...) to the latest values, plus task (and history with observation_history). infer returns joint targets in the order of action and the followers' units: one step [n] or a chunk [H, n]. Optional methods: hooks() -> Sequence[Hook] (from roboloom.hooks), called after bind to contribute local steps or lifecycle handlers; bind(spec: PolicySpec), called once after loading to refuse a configuration the policy was not trained for, and options(), returning observation_history, hold or action_horizon to use instead of the YAML's.

reset()

Forget the previous episode (called when an episode starts).

infer(observation)

Joint targets: shape [n] or [H, n].

roboloom.policy.PolicySpec dataclass

What the configuration gives a policy (bind).

observation: dataset feature name -> {"dtype", "shape", "names"}, the keys and shapes of the observation infer receives (besides task); images are [h, w, 3]. action: the names of the action's entries, <follower>.<joint> in order; units their units. history and hold are the controller's observation_history and hold, inference its inference (see controllers.policy).

roboloom.policy.load_policy(reference, kwargs=None, search=())

Call the factory module:callable with kwargs; the result must be a Policy.

search: directories to import from after the installed packages (appended to sys.path, so a local file never shadows an installed module).

A policy that holds

roboloom.policy.hold(horizon=1)

roboloom.policy:hold, the template's default policy: keep the arms where they are, horizon steps per call.

roboloom.policy.Hold

Stays where the robot is: a policy for checking the inference path safely.

reset()

Nothing to forget.

infer(observation)

horizon copies of the current observation.state.

LeRobot

roboloom.policy.lerobot

LeRobot checkpoints as roboloom policies (roboloom.policy.lerobot:load).

type: policy
factory: roboloom.policy.lerobot:load
kwargs:
  path: outputs/train/act_piper/checkpoints/last/pretrained_model  # or a hub repo id
  device: auto          # cuda when available, else cpu
  mode: step            # step: select_action, one step per call (action_horizon: 1)
                        # chunk: predict_action_chunk, [H, n] per call
  rename: {}            # {checkpoint feature: roboloom feature}

Loading follows LeRobot 0.7.0's own rollout: the config and weights from the checkpoint, then its saved pre- and post-processors, which hold the normalization statistics. bind checks the checkpoint's input and output features against what the configuration observes and drives, so a policy trained on other cameras or joints is refused at load.

mode: step lets the LeRobot policy keep its own action queue and observation history, as LeRobot's rollout does; it is right for every policy type (Diffusion needs the history). mode: chunk returns the whole predicted chunk for roboloom's action_horizon; it skips the history, so use it only for single-frame policies (ACT).

LeRobotPolicy

A LeRobot checkpoint with its pre- and post-processors, as a roboloom policy (made by load; see the module's description of mode and rename).

inputs()

The checkpoint's input features the policy reads -> (type, shape).

LeRobot's training lists every observation.* of the dataset as an input (observation.meta.*, observation.effort, a Sense's pose, ...), but its policies read only the robot state, the images and the environment state.

bind(spec)

Refuse a configuration whose observation or action does not match the checkpoint's input and output features.

reset()

Clear the model's action queue and the processors' state.

infer(observation)

Joint targets from the checkpoint: one step [n] (mode: step) or the predicted chunk [H, n] (mode: chunk).

load(path, **kwargs)

roboloom.policy.lerobot:load: a LeRobot checkpoint (local directory or hub id).

Another process

模擬のみ

roboloom.policy.remote

A policy that runs in another process, e.g. another Python environment or a GPU machine.

The policy node loads roboloom.policy.remote:connect, which forwards bind, reset and infer, negotiates protocol-2 lifecycle events, and refuses server step hooks. Steps always run locally. Lifecycle calls share the inference call lock. A server that holds the real policy:

# where the model lives (its own environment; only numpy and roboloom are needed)
export ROBOLOOM_POLICY_AUTHKEY=...            # any shared secret
python -m roboloom.policy.remote my_lab.policies:load --kwargs '{"checkpoint": "..."}'

# nodes/policy.yaml
factory: roboloom.policy.remote:connect
kwargs: {address: "127.0.0.1:18765"}

This is for models whose dependencies do not fit the environment roboloom runs in (a different torch) or that should run on another machine. The server loads the model once and serves one policy node at a time, across episodes and sessions; a second node that connects meanwhile is told so instead of being left waiting.

Every call carries a number the answer repeats. An answer that arrives after its call timed out is dropped, so it is never taken for the answer to the next call.

Wire format: multiprocessing.connection (the key authenticates both ends with HMAC; nothing is encrypted, so keep it on the loopback interface or a trusted network). A message is a JSON header followed by the raw bytes of its arrays: no pickle, so a peer cannot run code here and numpy versions may differ.

RemotePolicy

The Policy the node loads; every call is answered by the server's policy.

hooks()

Only lifecycle forwarding; per-step hooks always run in the local node.

options()

The served policy's options() (empty when it has none).

bind(spec)

Forward bind to the served policy.

reset()

Forward reset to the served policy.

infer(observation)

The served policy's answer to observation.

close()

Close the connection to the server.

RemotePolicyError

Bases: RuntimeError

The policy on the server raised; the message carries its type and text.

connect(address=ADDRESS, authkey_env=AUTHKEY_ENV, timeout_s=30.0, connect_timeout_s=30.0)

roboloom.policy.remote:connect: the policy served at address (host:port). timeout_s bounds one call (an infer), connect_timeout_s the wait for the server to come up; the secret is read from the environment variable authkey_env.

serve(policy, address=ADDRESS, authkey=None, kwargs=None, authkey_env=AUTHKEY_ENV, clients=None, ready=None)

Answer policy nodes at address with policy (an object with reset and infer, or a "module:callable" factory called with kwargs), one at a time, until interrupted (or after clients connections). ready, if given, is called once the server listens.

Replay

模擬のみ

roboloom.policy.replay

Replay recorded actions as a policy (roboloom.policy.replay:load).

factory: roboloom.policy.replay:load
kwargs:
  path: datasets/rakuda2     # a recorded LeRobotDataset, or a .npy file [T, n]
  episode: 0                 # which episode of the dataset
  stride: 1                  # every stride-th row: rate_hz = dataset fps / stride
action_horizon: 8

The recorded action rows go to the followers in order, whatever is observed (open loop). It checks the path from a recording to the robot without a model: the same controller, speed limit, interpolation and safety checks as a learned policy, so the recorded observation.state of the replay can be compared with the original. A .npy file replays actions from elsewhere, e.g. to check the joint order and units of data recorded with another tool: move the robot to the episode's first pose first (home) and keep max_speed low.

After the last action the policy keeps returning it (the followers stay there). inference: async is refused: a replay's chunks follow one another, they are not computed from the observation's time, so replacing a queued chunk would skip rows.

Replay

Recorded actions [T, n] returned horizon rows per call, open loop (made by load).

options()

action_horizon: the rows returned per call.

bind(spec)

Refuse a configuration whose action size (and, for a recording, names and rate) differ from the recorded ones, or that uses inference: async.

reset()

Start again from the first row.

infer(observation)

The next horizon rows, whatever is observed (the last row once done).

load(path, episode=0, horizon=8, stride=1)

roboloom.policy.replay:load: the actions of one recorded episode, or of a .npy file [T, n] (then stride only thins the rows and nothing is checked against the configuration but the action's size).

Hooks

Mock only

roboloom.hooks.Requirements dataclass

Resources a hook needs. Future API additions have defaults.

roboloom.hooks.HookSpec dataclass

The node's action layout and features of its observed sources only.

roboloom.hooks.Hook

Optional methods: requirements, bind, step, on_event and close.

A step implementation accepts (action: np.ndarray, ctx: StepContext) and returns an array or None (unchanged). Event handlers run on a per-hook worker.

bind(spec)

Raise ConfigError to refuse the node's configuration.

on_event(event)

Route an event to the optional on_<kind>(event) method.

close()

Release hook-owned resources after draining the event queue.

roboloom.hooks.StepContext dataclass

One step invocation; snapshots and observations are built only when read.

previous is the last command sent, held the follower's current target, and state its measured position. Arrays and observations are read-only. note accepts up to 16 small scalar fields, retained in this hook's stats.

roboloom.hooks.ChunkPosition dataclass

roboloom.hooks.EpisodeStarted dataclass

roboloom.hooks.EpisodeEnded dataclass

roboloom.hooks.ControlChanged dataclass

roboloom.hooks.PolicyRestarted dataclass

roboloom.hooks.ObservationStale dataclass

roboloom.hooks.HookChain

Run bound hooks in order; step and event methods of one hook never overlap.

start_timeout_s is injectable for tests. Event errors are logged; a barrier timeout uses on_error. No synchronous step can be interrupted by a time budget.

run(action, ctx)

One period. Each hook sees the preceding output and only its observation.

emit(event, *, rate_hz=1.0, settle=None)

Queue for every event hook; barriers wait only for hooks with a step.

close(timeout=DRAIN_TIMEOUT_S)

Drain workers, then close. On timeout the worker closes after its queue.