Skip to content

Session

The collection and inference API at the top of the roboloom package. Usage examples come first, followed by the descriptions generated from the docstrings.

Structure

The robot learning systems we examined (franka_ros2, DROID, deoxys, diffusion_policy, OpenPI, etc.) all separated the following three parts:

  • A fast loop that drives the motors
  • Teleoperation and recording
  • The policy

roboloom takes the same structure.

  • dora dataflow: each of the following runs in its own process.
  • The follower nodes and the leader node (control loops)
  • Sensors
  • The recorder
  • The policy node
  • Session: launches the dataflow and sends events. Events are start, stop, save, task name, and so on. It does not run the control loop.
  • Therefore, even if the user's script does heavy processing or stalls, the control period is not disturbed.

The settings of one experiment (the policy, the options, the dataset) are passed with Session.launch(experiment=...): a name (configs/roboloom/<name>.yaml), a file path or a mapping with the same keys (experiment files).

Loading a configuration

A configuration can be specified either by file path or by its robot name under .roboloom/ (rakuda2 is .roboloom/rakuda2/config.yaml).

Launch options are chosen with options=: the listed options and the options they require are enabled. With None (the default), the options with default: true are enabled. Unlike the CLI's --with / --without, options= is the whole set to enable (it is not added to the default options). mock=True runs mock devices and mock=False real ones; None follows the YAML's mode (default hardware).

from roboloom import load_config

cfg = load_config("rakuda2", options=["gravity", "digit"])
print(cfg.options, cfg.repo_id)  # ('digit', 'gravity') local/rakuda2-digit

load_config(..., video_codec="av1") and Session.launch(..., video_codec="av1") record storage: video images as AV1 ("h264" for H.264, None for the recorder YAML's choice, else H.264). With a loaded RobotConfig, choose it in load_config. For appending with a codec that differs from the existing dataset, see Video codec.

Recording episodes

from roboloom import Session

with Session.launch("rakuda2", options=["gravity"]) as s:
    s.wait_ready()  # wait until the leader is aligned with the follower
    result = s.record("pick up the cup", seconds=30)
    print(result.index, result.frames, result.warnings)
  • launch waits until every node has started reporting its status.
  • Every start is checked: the follower must be within alignment_tolerance of the leader, not only at the first alignment. record waits for this (ready_timeout, default 60 s) before each episode; start raises SessionError when the arms differ.
  • Real hardware is the default. Pass mock=True to run mock devices.
  • launch uses options and mock only when the configuration is given as a path or robot name. When passing a loaded RobotConfig, choose them with load_config(..., options=..., mock=...) first.
  • Each node process receives the enabled options in ROBOLOOM_OPTIONS and the mode in ROBOLOOM_MODE.
  • Leaving the with block does the following:
  • Discards any in-progress episode.
  • If an operator is configured, asks you to support the arms. Enter releases the follower too; typing hold keeps the follower holding.
  • Stops the dataflow.
  • The leader goes limp. The follower's torque goes off only when you chose to release it; otherwise it keeps its torque and last target. A follower that does not hold when left alone (unattended other than holds) does not keep holding; see Behaviour without torque and when left alone.
  • To decide from a script, call s.close(release=True) (release) or s.close(release=False) (hold).
  • Logs are kept in logs/<dataset name>-<time>/ next to the configuration file (for example logs/rakuda2-digit-20261102-101500/).
  • dora.log
  • The leader trace
  • The location is available as s.log_dir.

Status screen and exit confirmation

  • Session.launch(..., screen="auto") is the default. When stdout is a terminal, a background thread updates the Rich node-status table and messages panel at 10 Hz. Interactive CLI commands instead use the Textual operator screen.
  • Set screen=False to disable the display. It is hidden by default when stdout is not a terminal, such as in a pipe.
  • Your script's print() output appears above the screen. The screen reads no keys, so you can use input() and API calls to operate the session. It does not add the keyboard controls of CLI record.
  • Save results and quality warnings also appear in the messages panel.
  • operator="auto" is the default. When stdin is a terminal, the session asks you to support the leader and follower arms before exiting.
  • Support the arms and press Enter to end with the follower torque off as well.
  • Type hold to end with the follower still holding.
  • When the follower has torque: false, only the leader confirmation is shown.
  • The confirmation is built from the devices' traits: it names what to support (falls) and what to empty (lets_go), and when no follower holds, hold means quitting without releasing (those followers may not hold). s.release_note() returns what to do before release as a string; release(confirm=False) raises HardwareError with it.
  • With operator=None, or when stdin is not a terminal, this confirmation is skipped and the follower keeps holding. Setting only screen=False does not disable it.
  • If the follower is left holding, support its arms or lower them to a safe pose before switching off its power.

Automatic task naming

For task, you can pass either a string or a function that takes an EpisodeInfo and returns a string.

EpisodeInfo field Contents
index Episode number in the dataset (number of saved episodes)
number Number recorded in this Session
dataset The dataset's name (the part of repo_id after /, with the enabled options' names; e.g. rakuda2-digit)
previous List of tasks saved in this Session
import itertools

objects = itertools.cycle(["cup", "bottle", "sponge"])

with Session.launch("rakuda2") as s:
    s.wait_ready()
    for _ in range(9):
        s.record(lambda info: f"pick up the {next(objects)}", seconds=20)
# numbered, ask the operator each time, etc.
s.record(lambda info: f"trial {info.index:03d}", seconds=20)
s.record(lambda info: input(f"task for episode {info.index}: "), seconds=20)
  • The resolved task string is saved in task of each frame of the LeRobotDataset.
  • It can be used directly as the language instruction during training.

Fine-grained control

  • You can call start(task), stop(), save(), discard(), estop(), and release(confirm=True) one at a time.
  • An episode save() returned for stays readable even if the process stops afterwards. When the write fails, save() raises SessionError and drops that episode (the earlier ones stay and the next can be recorded; Saved episodes).
  • Example: when start and end are decided by an external signal.
  • Inspecting state:
  • status(): latest status of each node.
  • reports(): unread reports, such as robot mode changes.
  • When a command is rejected (e.g. the leader is not aligned), a SessionError is raised.
  • start is also rejected while the previous episode is being saved, is stopped but neither saved nor discarded, or is still recording. When the recorder could not take a start, the followers and the policy are stopped too (no episode runs that nobody records).
  • record discards the episode and raises EpisodeFailed (a SessionError) when a controller fails during it (for example the policy raised) or a follower latches FAULT or ESTOP. The policy can be used for the next episode as it is.
  • estop() goes out even after a node has failed (other commands raise SessionError once they see a failed node).
  • When the process that called Session.launch ends, however it ends (killed, an exception, its terminal closed), the dataflow discards the episode in progress, lets the followers hold and ends. A robot never keeps following a policy after its script is gone, with or without close(). Child processes made with multiprocessing and the like count as the process still being there.
  • home(timeout=None): move the leader and follower slowly to the initial pose chosen in the controller and wait until both arrive. Raises SessionError if they do not arrive or a node refuses. record(..., home=True) does this before every episode. See "Move to an initial pose" in Leader.
  • teleop(), policy(), stop_teleop(): with teleop: in the policy, hand the followers to the leader (before an episode, or during one as an intervention), hand back to the policy, and end teleoperation outside an episode. See teleoperation and interventions.
s.start("wipe the table")
while not task_done():  # user's own check
    print(s.status()["follower"]["fields"]["mode"])
    time.sleep(0.5)
stopped = s.stop()  # not saved yet; metrics and warnings can be inspected
s.save() if stopped.ok else s.discard()

Policies, evaluation and choosing episodes

Sensor selection

load_config(..., sensors=[...], observe=[...]) and Session.launch(..., sensors=[...], observe=[...]) select the sensors to start and record independently from the policy's observations. None keeps the configuration's defaults; [] selects none. Follower state is always observed. When passing a RobotConfig, select in load_config (Session.launch rejects additional selection arguments).

with Session.launch(
    "rakuda2",
    options=["policy"],
    sensors=["camera", "audio"],
    observe=["camera"],
    policy="my_lab.policies:load",
) as session:
    session.wait_ready()
    session.record("pick", seconds=20)
  • Unselected sensor YAML files are not loaded. Observing a sensor that is not started fails.
  • Sensors introduced by options can also be selected individually, including either DIGIT. Options apply first.
  • An optional sensor uses {file: nodes/audio.yaml, default: false}; string paths remain enabled by default.
  • Recording includes every started sensor, even those omitted from observe.

The CLI selection is in Sensor selection in the CLI and the rules in Sensor selection. See also Audio.

Running in one process: InProcessSession

  • InProcessSession performs robot bus access, sensor reads, and controller calls all on the caller's thread.
  • Use cases:
  • Tests
  • Simulation
  • Simple robots with few sensors
  • It drives one follower. Run configurations with several followers with Session.
  • A policy can be passed directly to step(policy=...).
  • However, if the policy or sensors are slow, the control period is delayed too.
  • Run the real Rakuda-2 with Session.
  • from_yaml rejects configurations that use a motor-controlling leader (trigger spring, gravity compensation).
  • from_yaml also rejects configurations that use a policy controller.
  • In InProcessSession.from_yaml(path, *, options=None, mock=None, sensors=None, dataset=None), options and mock mean the same as in load_config.

This mock example uses no devices or sample files from the repository. The record extra is required. Do not reuse its joint ranges or speeds on hardware.

from roboloom import InProcessSession
from roboloom.drivers.mock import MockBus

with InProcessSession.from_components(
    robot="my-mock",
    bus=MockBus(dimension=2),
    joint_names=["j1", "j2"],
    unit="count",
    limits=[
        {"name": "j1", "min": 0, "max": 4095, "max_speed": 3000, "max_jump": 400},
        {"name": "j2", "min": 0, "max": 4095, "max_speed": 3000, "max_jump": 400},
    ],
    controller=lambda obs: [2048, 2048],
    output="datasets/my-mock",
    repo_id="local/my-mock",
) as session:
    session.start_episode("handover")
    session.run(3.0)
    session.stop_episode()
    session.save_episode()

For the data collection procedure on real hardware, see Rakuda2: teleoperated collection.

API

roboloom.Session

A robot's running dora dataflow, directed from this process.

The control loops, sensors, recorder and policy run in their own processes; a Session only starts the dataflow and sends it events (start, stop, save, the task), so a slow or stopped script never delays a control loop. Create one with Session.launch and use it as a context manager: leaving the with block discards an unfinished episode, asks the operator to support the arms (when operator is set) and stops the dataflow (close).

alive property

True while the dataflow process is running.

actuated property

Inputs whose motors run control laws; they go limp when the session ends.

torqued property

Followers that run with torque, with their safety traits.

held property

Followers that keep holding after the session unless released: torque on and unattended: holds (core.traits).

limp_at_end property

What goes, or may go, limp when the session ends without release: actuated inputs, and followers that do not hold unattended (or are not known to).

episodes property

Episodes in the dataset, as the recorder last reported.

launch(config=None, *, experiment=None, options=None, mock=None, sensors=None, observe=None, video_codec=None, policy=None, policy_kwargs=None, log_dir=None, timeout=60.0, screen='auto', operator='auto') classmethod

Validate and start the dataflow; returns once every node reports status.

  • options: the launch options to enable (None: their defaults); mock: True runs mock devices, False real ones (None: the YAML's mode, default hardware). Both apply only when config is a path or name.
  • experiment: an experiment's name (configs/roboloom/<name>.yaml), its file, a mapping with the same keys (e.g. what Hydra composed) or an Experiment. It gives the robot (when config is left out), the options (when options is left out), what is laid over the nodes and where the episodes go. Its task, seconds or steps are yours to pass to record.
  • sensors, observe, video_codec: as in load_config (None: the YAML); with a RobotConfig, choose them in load_config instead.
  • policy ("module:callable") and policy_kwargs replace the policy of a type: policy controller.
  • screen: the node-status screen, drawn on a thread ("auto": when stdout is a terminal; see roboloom.ui.screen).
  • operator: asked to support the arms before closing makes an actuated leader go limp ("auto": a TerminalOperator when stdin is a terminal; None: no question).

release_note()

What the operator does before release turns torque off on the actuated inputs and the followers.

close(timeout=15.0, *, release=None)

Discard an unfinished episode, stop every node and wait for the dataflow to exit.

An actuated leader goes limp. The follower torque goes off when release is true; when it is None, the operator is asked (Enter releases, hold keeps the follower holding) and without an operator the follower keeps holding.

status()

Latest status of every node: {name: {"role", "level", "fields", "t_ns"}}.

reports()

Reports not yet returned by a wait (robot mode changes, warnings, ...).

listen(listener)

Call listener with every report and session note (the screen uses this).

poll()

Read new reports from the dataflow; safe from any thread.

note(text, level='info')

A message from this side (e.g. quality warnings) for the listeners.

node_errors()

Error lines of failed nodes in dora.log (a node can die while dora keeps running).

wait_ready(timeout=30.0)

Until every controller may start an episode (e.g. the leader is aligned).

send(command, task='', **extra)

Send one command to every node; returns its id (echoed in the reports). extra: what else the command carries (save: label, note).

wait_report(ident, timeout=10.0)

The report answering command ident; raises SessionError if it failed.

start(task)

Start recording an episode of task and let the controllers drive.

Raises SessionError when a controller refuses (e.g. the leader is not aligned with the follower, alignment_tolerance) or while the previous episode is recording, stopped but not saved, or being saved. When the recorder could not take the start, the followers and the policy are stopped too, so no episode runs that nobody records.

stop()

End the episode's recording; the result (not yet saved) has its metrics and quality warnings, to decide between save and discard.

save(timeout=300.0, *, label=None, note=None)

Write the stopped episode (video encoding can take a while). When the write fails the episode is dropped (SessionError); the dataset keeps the episodes saved before it and the next one can be recorded.

label ("success" or "failure") and note are the verdict of an evaluation episode. They go into the dataset's meta/roboloom_episodes.jsonl with the task, the metrics and what the controllers reported; roboloom label sets the verdict afterwards.

discard()

Stop the robot and drop the episode in progress.

home(timeout=None)

Move every leader-follower pair with a home pose there, slowly, and wait until both arrived (at most timeout s, default the YAML's timeout_s). Leaders go back to their normal laws a second after arriving. Raises SessionError when a node refused or did not arrive.

begin_home()

Send home; returns (send time in monotonic ns, the YAML's timeout in s).

home_waiting(sent_ns)

Nodes still moving home since sent_ns (empty: arrived); raises SessionError when one refused.

teleop()

Drive the followers with the leader: before an episode (nothing is recorded) or during one (an intervention, recorded as action.intervention). Each joint holds until the leader reaches it, then follows; the policy's status lists the joints still to reach (pickup).

policy()

Hand the followers back to the policy during an episode (it restarts from where they are, with a fresh history).

stop_teleop()

Stop teleoperating outside an episode; the followers hold where they are.

estop()

Latch the follower in ESTOP (stays until the dataflow restarts). Written straight to the FIFO: it must also go out when a node has failed, which every other command refuses.

release(*, confirm=False)

Torque off the actuated inputs and the followers after stopping; first do what release_note() says (support what falls, empty what lets go).

next_info()

What a task-naming function is given for the next episode.

record(task, seconds, *, keep=None, on_tick=None, tick_s=0.2, ready_timeout=60.0, home=False, label=None, note=None, steps=None)

One episode: wait until it may start (leader aligned with the follower, at most ready_timeout s), start, record for seconds, stop, then save it unless keep rejects the stopped episode (default: keep every one; quality warnings are in the result either way).

steps: end as soon as the policy has executed this many actions (steps_done); seconds is then the longest the episode may take (steps_timeout).

task is a string or a function of EpisodeInfo. on_tick(status) is called every tick_s while recording. An exception or Ctrl-C discards the episode, and so does a controller that fails or a follower that latches FAULT or ESTOP while it runs (EpisodeFailed): an episode that broke off is not saved. home=True first moves to the saved home pose (home), so every episode starts from the same pose. label is the verdict saved with the episode ("success" / "failure"), or a function of the stopped episode returning it (None: no verdict); note is free text (save).

roboloom.EpisodeInfo dataclass

What a task-naming function knows about the episode it names.

Attributes:

Name Type Description
index int

The episode's index in the dataset (the episodes already saved).

number int

The episodes this session has recorded so far.

dataset str

The dataset's name, e.g. rakuda2-digit (it tells the options).

previous list[str]

The tasks this session has saved, oldest first.

roboloom.EpisodeResult dataclass

A stopped episode, saved or not (Session.stop, save, record).

Attributes:

Name Type Description
task str

The episode's task text.

saved bool

Whether the episode was written to the dataset.

frames int

The rows recorded.

index int | None

The episode's index in the dataset once saved, else None.

metrics dict

The recorder's quality metrics (missing rates, the share of valid actions, the control interval distribution).

warnings list[str]

The quality limits (quality: of the recorder YAML) it exceeded.

label str | None

The verdict it was saved with, "success" or "failure", else None.

ok property

True when no quality limit was exceeded (warnings is empty).

roboloom.EpisodeFailed

Bases: SessionError

The episode could not go on (a controller failed, a follower latched FAULT or ESTOP); it was discarded. A failed policy is ready again for the next episode.

roboloom.SessionError

Bases: RoboloomError

The dataflow refused a command or stopped. report is the node's answer, when one caused it.

roboloom.InProcessSession

Collect with a controller or run a policy using the same checked robot bus.

controller takes the current observation dict and returns a target vector or None. sensors maps source names to callables returning Sample or None. For YAML configurations, from_yaml supplies the configured controller by default.

from_yaml(path, *, options=None, mock=None, sensors=None, dataset=None) classmethod

options: launch options to enable (None: their defaults); mock: override the YAML's mode (None: as written, default hardware).

from_components(*, robot, bus, joint_names, limits, controller=None, sensors=None, image_shapes=None, audio_sample_rate=None, fps=10, control_hz=30, output='dataset', repo_id='local/in-process', unit, dataset=None) classmethod

A one-follower session built from parts, without a configuration file.

  • bus: the follower's joint bus (e.g. roboloom.drivers.mock.MockBus); joint_names, unit and limits (one mapping per joint: name, min, max, max_speed, max_jump) describe its joints.
  • controller: observation dict -> targets (or None), as in the constructor.
  • sensors: source name -> callable returning a Sample or None. A source named leader is recorded as a joint input; the others need an entry in image_shapes (HWC RGB) or are the audio stream (audio_sample_rate).
  • fps: the dataset rate; control_hz (at least fps): the control rate.
  • output, repo_id: where and under which id the dataset is written.

start_episode(task)

Start recording an episode of task and arm the follower.

start_inference()

Arm checked one-step control without creating an episode.

stop_inference()

Stop commanding the follower (it holds); the counterpart of start_inference.

step(*, policy=None)

One checked control tick; policy overrides the configured controller.

run(seconds, *, policy=None)

Call step at the control rate for seconds; returns each step's result.

stop_episode()

Stop the follower and end the episode's recording (save or discard it next).

save_episode(label=None, note=None)

Save the stopped episode; label ("success" / "failure") and note go into the dataset's record of it (recording.episodes).

discard_episode()

Stop the follower and drop the episode in progress.

estop()

Latch the follower in ESTOP.

release()

Torque off after stop; support the arm first. Latched until the robot is reopened.

close()

Close the recorder and the bus without releasing torque: what the follower does next is its traits().unattended (holds, goes limp, or not known).

roboloom.load_config(config=None, root=PROJECT_ROOT, *, options=None, mock=None, sensors=None, observe=None, video_codec=None, experiment=None)

Load and validate a configuration file, or one named under the project root.

experiment: an Experiment (load_experiment) or its settings(); see read_config. Its options are not applied here: pass them as options.

sensors: active recorded sensors (None: defaults, []: none). observe: policy inputs (None: YAML, []: follower state only). video_codec: codec of storage: video images, "av1" (alias "libsvtav1") or "h264" (None: the recorder YAML's video_codec, else the saved videos' codec when appending, else H.264). options: the launch options to enable (None: their defaults). mock: override the YAML's mode (True: mock devices, False: hardware).

roboloom.resolve_config(arg=None, root=PROJECT_ROOT)

A configuration file, or a robot name under the project root: rakuda2 is .roboloom/rakuda2/config.yaml. Without a name, the only robot under the root.

roboloom.init_project(robot, root=PROJECT_ROOT, template=None)

Create root/robot from the shipped template of robot, or from template (a package holding template/, or a directory).

roboloom.Sample dataclass

One reading of a source.

Attributes:

Name Type Description
value ndarray

The reading (joint positions, an HWC RGB image, PCM audio, ...).

capture_ns int

When it was captured, in time.monotonic_ns() time.

Exceptions

roboloom.RoboloomError

Bases: Exception

Base of every error caused by configuration, setup or the robot, not by a bug.

roboloom.ConfigError

Bases: RoboloomError, ValueError

A configuration or calibration file is invalid or still has SET_ values.

roboloom.HardwareError

Bases: RoboloomError, RuntimeError

The devices are not in the state a command needs (torque on, wrong mode, ...).

roboloom.SetupError

Bases: RoboloomError

A measurement or fit cannot proceed, or its result is not good enough to save.

roboloom.errors.Cancelled

Bases: RoboloomError

The operator declined a confirmation; nothing was written.