Session
The collection and inference API at the top of the roboloom package. Usage examples come first, followed by the descriptions generated from the docstrings.
Structure
The robot learning systems we examined (franka_ros2, DROID, deoxys, diffusion_policy, OpenPI, etc.) all separated the following three parts:
- A fast loop that drives the motors
- Teleoperation and recording
- The policy
roboloom takes the same structure.
- dora dataflow: each of the following runs in its own process.
- The follower nodes and the leader node (control loops)
- Sensors
- The recorder
- The policy node
Session: launches the dataflow and sends events. Events are start, stop, save, task name, and so on. It does not run the control loop.- Therefore, even if the user's script does heavy processing or stalls, the control period is not disturbed.
The settings of one experiment (the policy, the options, the dataset) are passed with Session.launch(experiment=...): a name (configs/roboloom/<name>.yaml), a file path or a mapping with the same keys (experiment files).
Loading a configuration
A configuration can be specified either by file path or by its robot name under .roboloom/ (rakuda2 is .roboloom/rakuda2/config.yaml).
Launch options are chosen with options=: the listed options and the options they require are enabled. With None (the default), the options with default: true are enabled. Unlike the CLI's --with / --without, options= is the whole set to enable (it is not added to the default options). mock=True runs mock devices and mock=False real ones; None follows the YAML's mode (default hardware).
from roboloom import load_config
cfg = load_config("rakuda2", options=["gravity", "digit"])
print(cfg.options, cfg.repo_id) # ('digit', 'gravity') local/rakuda2-digit
load_config(..., video_codec="av1") and Session.launch(..., video_codec="av1") record storage: video images as AV1 ("h264" for H.264, None for the recorder YAML's choice, else H.264). With a loaded RobotConfig, choose it in load_config. For appending with a codec that differs from the existing dataset, see Video codec.
Recording episodes
from roboloom import Session
with Session.launch("rakuda2", options=["gravity"]) as s:
s.wait_ready() # wait until the leader is aligned with the follower
result = s.record("pick up the cup", seconds=30)
print(result.index, result.frames, result.warnings)
launchwaits until every node has started reporting its status.- Every
startis checked: the follower must be withinalignment_toleranceof the leader, not only at the first alignment.recordwaits for this (ready_timeout, default 60 s) before each episode;startraisesSessionErrorwhen the arms differ. - Real hardware is the default. Pass
mock=Trueto run mock devices. launchusesoptionsandmockonly when the configuration is given as a path or robot name. When passing a loadedRobotConfig, choose them withload_config(..., options=..., mock=...)first.- Each node process receives the enabled options in
ROBOLOOM_OPTIONSand the mode inROBOLOOM_MODE. - Leaving the
withblock does the following: - Discards any in-progress episode.
- If an
operatoris configured, asks you to support the arms. Enter releases the follower too; typingholdkeeps the follower holding. - Stops the dataflow.
- The leader goes limp. The follower's torque goes off only when you chose to release it; otherwise it keeps its torque and last target. A follower that does not hold when left alone (
unattendedother thanholds) does not keep holding; see Behaviour without torque and when left alone. - To decide from a script, call
s.close(release=True)(release) ors.close(release=False)(hold). - Logs are kept in
logs/<dataset name>-<time>/next to the configuration file (for examplelogs/rakuda2-digit-20261102-101500/). dora.log- The leader trace
- The location is available as
s.log_dir.
Status screen and exit confirmation
Session.launch(..., screen="auto")is the default. When stdout is a terminal, a background thread updates the Rich node-status table and messages panel at 10 Hz. Interactive CLI commands instead use the Textual operator screen.- Set
screen=Falseto disable the display. It is hidden by default when stdout is not a terminal, such as in a pipe. - Your script's
print()output appears above the screen. The screen reads no keys, so you can useinput()and API calls to operate the session. It does not add the keyboard controls of CLIrecord. - Save results and quality warnings also appear in the messages panel.
operator="auto"is the default. When stdin is a terminal, the session asks you to support the leader and follower arms before exiting.- Support the arms and press Enter to end with the follower torque off as well.
- Type
holdto end with the follower still holding. - When the follower has
torque: false, only the leader confirmation is shown. - The confirmation is built from the devices' traits: it names what to support (
falls) and what to empty (lets_go), and when no follower holds,holdmeans quitting without releasing (those followers may not hold).s.release_note()returns what to do beforereleaseas a string;release(confirm=False)raisesHardwareErrorwith it. - With
operator=None, or when stdin is not a terminal, this confirmation is skipped and the follower keeps holding. Setting onlyscreen=Falsedoes not disable it. - If the follower is left holding, support its arms or lower them to a safe pose before switching off its power.
Automatic task naming
For task, you can pass either a string or a function that takes an EpisodeInfo and returns a string.
EpisodeInfo field |
Contents |
|---|---|
index |
Episode number in the dataset (number of saved episodes) |
number |
Number recorded in this Session |
dataset |
The dataset's name (the part of repo_id after /, with the enabled options' names; e.g. rakuda2-digit) |
previous |
List of tasks saved in this Session |
import itertools
objects = itertools.cycle(["cup", "bottle", "sponge"])
with Session.launch("rakuda2") as s:
s.wait_ready()
for _ in range(9):
s.record(lambda info: f"pick up the {next(objects)}", seconds=20)
# numbered, ask the operator each time, etc.
s.record(lambda info: f"trial {info.index:03d}", seconds=20)
s.record(lambda info: input(f"task for episode {info.index}: "), seconds=20)
- The resolved task string is saved in
taskof each frame of the LeRobotDataset. - It can be used directly as the language instruction during training.
Fine-grained control
- You can call
start(task),stop(),save(),discard(),estop(), andrelease(confirm=True)one at a time. - An episode
save()returned for stays readable even if the process stops afterwards. When the write fails,save()raisesSessionErrorand drops that episode (the earlier ones stay and the next can be recorded; Saved episodes). - Example: when start and end are decided by an external signal.
- Inspecting state:
status(): latest status of each node.reports(): unread reports, such as robot mode changes.- When a command is rejected (e.g. the leader is not aligned), a
SessionErroris raised. startis also rejected while the previous episode is being saved, is stopped but neither saved nor discarded, or is still recording. When the recorder could not take astart, the followers and the policy are stopped too (no episode runs that nobody records).recorddiscards the episode and raisesEpisodeFailed(aSessionError) when a controller fails during it (for example the policy raised) or a follower latches FAULT or ESTOP. The policy can be used for the next episode as it is.estop()goes out even after a node has failed (other commands raiseSessionErroronce they see a failed node).- When the process that called
Session.launchends, however it ends (killed, an exception, its terminal closed), the dataflow discards the episode in progress, lets the followers hold and ends. A robot never keeps following a policy after its script is gone, with or withoutclose(). Child processes made withmultiprocessingand the like count as the process still being there. home(timeout=None): move the leader and follower slowly to the initial pose chosen in the controller and wait until both arrive. RaisesSessionErrorif they do not arrive or a node refuses.record(..., home=True)does this before every episode. See "Move to an initial pose" in Leader.teleop(),policy(),stop_teleop(): withteleop:in the policy, hand the followers to the leader (before an episode, or during one as an intervention), hand back to the policy, and end teleoperation outside an episode. See teleoperation and interventions.
s.start("wipe the table")
while not task_done(): # user's own check
print(s.status()["follower"]["fields"]["mode"])
time.sleep(0.5)
stopped = s.stop() # not saved yet; metrics and warnings can be inspected
s.save() if stopped.ok else s.discard()
Policies, evaluation and choosing episodes
- Running a policy: Running a policy
- Recording an evaluation's verdicts: Evaluation verdicts
- Choosing which episodes to keep (
keep): Choosing episodes
Sensor selection
load_config(..., sensors=[...], observe=[...]) and Session.launch(..., sensors=[...], observe=[...]) select the sensors to start and record independently from the policy's observations. None keeps the configuration's defaults; [] selects none. Follower state is always observed. When passing a RobotConfig, select in load_config (Session.launch rejects additional selection arguments).
with Session.launch(
"rakuda2",
options=["policy"],
sensors=["camera", "audio"],
observe=["camera"],
policy="my_lab.policies:load",
) as session:
session.wait_ready()
session.record("pick", seconds=20)
- Unselected sensor YAML files are not loaded. Observing a sensor that is not started fails.
- Sensors introduced by options can also be selected individually, including either DIGIT. Options apply first.
- An optional sensor uses
{file: nodes/audio.yaml, default: false}; string paths remain enabled by default. - Recording includes every started sensor, even those omitted from
observe.
The CLI selection is in Sensor selection in the CLI and the rules in Sensor selection. See also Audio.
Running in one process: InProcessSession
InProcessSessionperforms robot bus access, sensor reads, and controller calls all on the caller's thread.- Use cases:
- Tests
- Simulation
- Simple robots with few sensors
- It drives one follower. Run configurations with several followers with
Session. - A policy can be passed directly to
step(policy=...). - However, if the policy or sensors are slow, the control period is delayed too.
- Run the real Rakuda-2 with
Session. from_yamlrejects configurations that use a motor-controlling leader (trigger spring, gravity compensation).from_yamlalso rejects configurations that use a policy controller.- In
InProcessSession.from_yaml(path, *, options=None, mock=None, sensors=None, dataset=None),optionsandmockmean the same as inload_config.
This mock example uses no devices or sample files from the repository. The record extra is required. Do not reuse its joint ranges or speeds on hardware.
from roboloom import InProcessSession
from roboloom.drivers.mock import MockBus
with InProcessSession.from_components(
robot="my-mock",
bus=MockBus(dimension=2),
joint_names=["j1", "j2"],
unit="count",
limits=[
{"name": "j1", "min": 0, "max": 4095, "max_speed": 3000, "max_jump": 400},
{"name": "j2", "min": 0, "max": 4095, "max_speed": 3000, "max_jump": 400},
],
controller=lambda obs: [2048, 2048],
output="datasets/my-mock",
repo_id="local/my-mock",
) as session:
session.start_episode("handover")
session.run(3.0)
session.stop_episode()
session.save_episode()
For the data collection procedure on real hardware, see Rakuda2: teleoperated collection.
API
roboloom.Session
A robot's running dora dataflow, directed from this process.
The control loops, sensors, recorder and policy run in their own processes; a
Session only starts the dataflow and sends it events (start, stop, save, the
task), so a slow or stopped script never delays a control loop. Create one with
Session.launch and use it as a context manager: leaving the with block discards
an unfinished episode, asks the operator to support the arms (when operator is
set) and stops the dataflow (close).
alive
property
True while the dataflow process is running.
actuated
property
Inputs whose motors run control laws; they go limp when the session ends.
torqued
property
Followers that run with torque, with their safety traits.
held
property
Followers that keep holding after the session unless released: torque on and
unattended: holds (core.traits).
limp_at_end
property
What goes, or may go, limp when the session ends without release: actuated inputs, and followers that do not hold unattended (or are not known to).
episodes
property
Episodes in the dataset, as the recorder last reported.
launch(config=None, *, experiment=None, options=None, mock=None, sensors=None, observe=None, video_codec=None, policy=None, policy_kwargs=None, log_dir=None, timeout=60.0, screen='auto', operator='auto')
classmethod
Validate and start the dataflow; returns once every node reports status.
options: the launch options to enable (None: their defaults);mock: True runs mock devices, False real ones (None: the YAML'smode, default hardware). Both apply only whenconfigis a path or name.experiment: an experiment's name (configs/roboloom/<name>.yaml), its file, a mapping with the same keys (e.g. what Hydra composed) or anExperiment. It gives the robot (whenconfigis left out), the options (whenoptionsis left out), what is laid over the nodes and where the episodes go. Its task, seconds or steps are yours to pass torecord.sensors,observe,video_codec: as inload_config(None: the YAML); with aRobotConfig, choose them inload_configinstead.policy("module:callable") andpolicy_kwargsreplace the policy of atype: policycontroller.screen: the node-status screen, drawn on a thread ("auto": when stdout is a terminal; seeroboloom.ui.screen).operator: asked to support the arms before closing makes an actuated leader go limp ("auto": a TerminalOperator when stdin is a terminal; None: no question).
release_note()
What the operator does before release turns torque off on the actuated inputs
and the followers.
close(timeout=15.0, *, release=None)
Discard an unfinished episode, stop every node and wait for the dataflow to exit.
An actuated leader goes limp. The follower torque goes off when release is
true; when it is None, the operator is asked (Enter releases, hold keeps the
follower holding) and without an operator the follower keeps holding.
status()
Latest status of every node: {name: {"role", "level", "fields", "t_ns"}}.
reports()
Reports not yet returned by a wait (robot mode changes, warnings, ...).
listen(listener)
Call listener with every report and session note (the screen uses this).
poll()
Read new reports from the dataflow; safe from any thread.
note(text, level='info')
A message from this side (e.g. quality warnings) for the listeners.
node_errors()
Error lines of failed nodes in dora.log (a node can die while dora keeps running).
wait_ready(timeout=30.0)
Until every controller may start an episode (e.g. the leader is aligned).
send(command, task='', **extra)
Send one command to every node; returns its id (echoed in the reports).
extra: what else the command carries (save: label, note).
wait_report(ident, timeout=10.0)
The report answering command ident; raises SessionError if it failed.
start(task)
Start recording an episode of task and let the controllers drive.
Raises SessionError when a controller refuses (e.g. the leader is not aligned
with the follower, alignment_tolerance) or while the previous episode is
recording, stopped but not saved, or being saved. When the recorder could not take the
start, the followers and the policy are stopped too, so no episode runs that
nobody records.
stop()
End the episode's recording; the result (not yet saved) has its metrics and
quality warnings, to decide between save and discard.
save(timeout=300.0, *, label=None, note=None)
Write the stopped episode (video encoding can take a while). When the write fails the episode is dropped (SessionError); the dataset keeps the episodes saved before it and the next one can be recorded.
label ("success" or "failure") and note are the verdict of an evaluation
episode. They go into the dataset's meta/roboloom_episodes.jsonl with the
task, the metrics and what the controllers reported; roboloom label sets
the verdict afterwards.
discard()
Stop the robot and drop the episode in progress.
home(timeout=None)
Move every leader-follower pair with a home pose there, slowly, and wait until
both arrived (at most timeout s, default the YAML's timeout_s). Leaders go back to
their normal laws a second after arriving. Raises SessionError when a node refused or did not
arrive.
begin_home()
Send home; returns (send time in monotonic ns, the YAML's timeout in s).
home_waiting(sent_ns)
Nodes still moving home since sent_ns (empty: arrived); raises SessionError
when one refused.
teleop()
Drive the followers with the leader: before an episode (nothing is recorded)
or during one (an intervention, recorded as action.intervention). Each joint
holds until the leader reaches it, then follows; the policy's status lists the
joints still to reach (pickup).
policy()
Hand the followers back to the policy during an episode (it restarts from where they are, with a fresh history).
stop_teleop()
Stop teleoperating outside an episode; the followers hold where they are.
estop()
Latch the follower in ESTOP (stays until the dataflow restarts). Written straight to the FIFO: it must also go out when a node has failed, which every other command refuses.
release(*, confirm=False)
Torque off the actuated inputs and the followers after stopping; first do what
release_note() says (support what falls, empty what lets go).
next_info()
What a task-naming function is given for the next episode.
record(task, seconds, *, keep=None, on_tick=None, tick_s=0.2, ready_timeout=60.0, home=False, label=None, note=None, steps=None)
One episode: wait until it may start (leader aligned with the follower, at most
ready_timeout s), start, record for seconds, stop, then save it unless keep
rejects the stopped episode (default: keep every one; quality warnings are in the
result either way).
steps: end as soon as the policy has executed this many actions (steps_done);
seconds is then the longest the episode may take (steps_timeout).
task is a string or a function of EpisodeInfo. on_tick(status) is called
every tick_s while recording. An exception or Ctrl-C discards the episode, and
so does a controller that fails or a follower that latches FAULT or ESTOP while
it runs (EpisodeFailed): an episode that broke off is not saved.
home=True first moves to the saved home pose (home), so every episode starts
from the same pose.
label is the verdict saved with the episode ("success" / "failure"), or a
function of the stopped episode returning it (None: no verdict); note is
free text (save).
roboloom.EpisodeInfo
dataclass
What a task-naming function knows about the episode it names.
Attributes:
| Name | Type | Description |
|---|---|---|
index |
int
|
The episode's index in the dataset (the episodes already saved). |
number |
int
|
The episodes this session has recorded so far. |
dataset |
str
|
The dataset's name, e.g. |
previous |
list[str]
|
The tasks this session has saved, oldest first. |
roboloom.EpisodeResult
dataclass
A stopped episode, saved or not (Session.stop, save, record).
Attributes:
| Name | Type | Description |
|---|---|---|
task |
str
|
The episode's task text. |
saved |
bool
|
Whether the episode was written to the dataset. |
frames |
int
|
The rows recorded. |
index |
int | None
|
The episode's index in the dataset once saved, else None. |
metrics |
dict
|
The recorder's quality metrics (missing rates, the share of valid actions, the control interval distribution). |
warnings |
list[str]
|
The quality limits ( |
label |
str | None
|
The verdict it was saved with, |
ok
property
True when no quality limit was exceeded (warnings is empty).
roboloom.EpisodeFailed
Bases: SessionError
The episode could not go on (a controller failed, a follower latched FAULT or ESTOP); it was discarded. A failed policy is ready again for the next episode.
roboloom.SessionError
Bases: RoboloomError
The dataflow refused a command or stopped. report is the node's answer, when
one caused it.
roboloom.InProcessSession
Collect with a controller or run a policy using the same checked robot bus.
controller takes the current observation dict and returns a target vector or
None. sensors maps source names to callables returning Sample or None.
For YAML configurations, from_yaml supplies the configured controller by default.
from_yaml(path, *, options=None, mock=None, sensors=None, dataset=None)
classmethod
options: launch options to enable (None: their defaults); mock: override
the YAML's mode (None: as written, default hardware).
from_components(*, robot, bus, joint_names, limits, controller=None, sensors=None, image_shapes=None, audio_sample_rate=None, fps=10, control_hz=30, output='dataset', repo_id='local/in-process', unit, dataset=None)
classmethod
A one-follower session built from parts, without a configuration file.
bus: the follower's joint bus (e.g.roboloom.drivers.mock.MockBus);joint_names,unitandlimits(one mapping per joint:name,min,max,max_speed,max_jump) describe its joints.controller: observation dict -> targets (or None), as in the constructor.sensors: source name -> callable returning aSampleor None. A source namedleaderis recorded as a joint input; the others need an entry inimage_shapes(HWC RGB) or are theaudiostream (audio_sample_rate).fps: the dataset rate;control_hz(at leastfps): the control rate.output,repo_id: where and under which id the dataset is written.
start_episode(task)
Start recording an episode of task and arm the follower.
start_inference()
Arm checked one-step control without creating an episode.
stop_inference()
Stop commanding the follower (it holds); the counterpart of start_inference.
step(*, policy=None)
One checked control tick; policy overrides the configured controller.
run(seconds, *, policy=None)
Call step at the control rate for seconds; returns each step's result.
stop_episode()
Stop the follower and end the episode's recording (save or discard it next).
save_episode(label=None, note=None)
Save the stopped episode; label ("success" / "failure") and note go
into the dataset's record of it (recording.episodes).
discard_episode()
Stop the follower and drop the episode in progress.
estop()
Latch the follower in ESTOP.
release()
Torque off after stop; support the arm first. Latched until the robot is reopened.
close()
Close the recorder and the bus without releasing torque: what the follower does
next is its traits().unattended (holds, goes limp, or not known).
roboloom.load_config(config=None, root=PROJECT_ROOT, *, options=None, mock=None, sensors=None, observe=None, video_codec=None, experiment=None)
Load and validate a configuration file, or one named under the project root.
experiment: an Experiment (load_experiment) or its settings(); see
read_config. Its options are not applied here: pass them as options.
sensors: active recorded sensors (None: defaults, []: none).
observe: policy inputs (None: YAML, []: follower state only).
video_codec: codec of storage: video images, "av1" (alias "libsvtav1") or
"h264" (None: the recorder YAML's video_codec, else the saved videos' codec when
appending, else H.264).
options: the launch options to enable (None: their defaults). mock: override the
YAML's mode (True: mock devices, False: hardware).
roboloom.resolve_config(arg=None, root=PROJECT_ROOT)
A configuration file, or a robot name under the project root: rakuda2 is
.roboloom/rakuda2/config.yaml. Without a name, the only robot under the root.
roboloom.init_project(robot, root=PROJECT_ROOT, template=None)
Create root/robot from the shipped template of robot, or from template (a
package holding template/, or a directory).
roboloom.Sample
dataclass
One reading of a source.
Attributes:
| Name | Type | Description |
|---|---|---|
value |
ndarray
|
The reading (joint positions, an HWC RGB image, PCM audio, ...). |
capture_ns |
int
|
When it was captured, in |
Exceptions
roboloom.RoboloomError
Bases: Exception
Base of every error caused by configuration, setup or the robot, not by a bug.
roboloom.ConfigError
Bases: RoboloomError, ValueError
A configuration or calibration file is invalid or still has SET_ values.
roboloom.HardwareError
Bases: RoboloomError, RuntimeError
The devices are not in the state a command needs (torque on, wrong mode, ...).
roboloom.SetupError
Bases: RoboloomError
A measurement or fit cannot proceed, or its result is not good enough to save.
roboloom.errors.Cancelled
Bases: RoboloomError
The operator declined a confirmation; nothing was written.