Session
roboloom packageの直下にある,収集と推論のための API です.使い方の例のあとに,docstring から生成した説明(英語)を載せます.
構成
調べたrobot学習の系(franka_ros2,DROID,deoxys,diffusion_policy,OpenPI など)は,どれも次の 3 つを分けていました.
- motorを動かす速いloop
- 遠隔操作と記録
- 方策
roboloom も同じ構成をとります.
- dora の dataflow:次のものが,それぞれ別のprocessで動きます.
- followerのnodeと leader node(制御のloop)
- sensor
- recorder
- 方策node
Session:dataflow を起動し,出来事を送ります.出来事とは,開始,停止,保存,task 名などです.制御のloopは回しません.- そのため,利用者のscriptが重い処理をしたり,止まったりしても,制御の周期は乱れません.
実験ごとの設定(方策,option,保存先)は,Session.launch(experiment=...) で渡せます.名前(configs/roboloom/<名前>.yaml),fileのpath,同じkeyのmappingのどれかです(実験file).
設定を読む
設定は,fileのpathでも,.roboloom/ の下の機体名(rakuda2 は .roboloom/rakuda2/config.yaml)でも指定できます.
起動optionは options= で選びます.指定したoptionと,それが必要とするoptionが有効になります.None(既定)なら default: true のoptionが有効です.CLIの --with/--without と違い,options= は有効にするものの全体を表します(既定のoptionに足すのではありません).mock=True はmock device,mock=False は実機で動かします.None ならYAMLの mode(既定 hardware)に従います.
from roboloom import load_config
cfg = load_config("rakuda2", options=["gravity", "digit"])
print(cfg.options, cfg.repo_id) # ('digit', 'gravity') local/rakuda2-digit
load_config(..., video_codec="av1") と Session.launch(..., video_codec="av1") で,storage: video の画像を AV1 で記録します("h264" は H.264,None は recorder の YAML の指定.どちらもなければ H.264).構成済みの RobotConfig を渡すときは load_config で選びます.既存のdatasetと codec が違うときの扱いは Video codec を参照してください.
episodeを録る
from roboloom import Session
with Session.launch("rakuda2", options=["gravity"]) as s:
s.wait_ready() # leader が follower に揃うまで待つ
result = s.record("pick up the cup", seconds=30)
print(result.index, result.frames, result.warnings)
launchは,すべてのnodeが状態を報告し始めるまで待ちます.- 毎回の
startの前に,followerがleaderからalignment_tolerance以内にあるかを確かめます(最初の位置合わせだけではありません).recordはepisodeごとに,揃うまで待ちます(ready_timeout,既定 60 秒).startは,ずれているとSessionErrorを出します. - 既定は実機です.mock deviceで動かすときは
mock=Trueを渡します. launchのoptions・mockは,設定をpathか機体名で渡したときだけ使われます.読み込み済みのRobotConfigを渡す場合は,load_config(..., options=..., mock=...)で選んでおきます.- 各nodeのprocessには,有効なoptionが
ROBOLOOM_OPTIONS,modeがROBOLOOM_MODEで渡ります. withを抜けると,次のことをします.- 途中のepisodeがあれば破棄する.
operatorが設定されていれば,腕を支えるように求める.Enter で follower も脱力し,holdと入力すると follower は保持を続ける.- dataflow を止める.
- leader は脱力する.follower は,脱力を選んだときだけtorqueを切り,それ以外はtorqueと最後の目標を保持する.ただし,放置したときに保持しない(
unattendedがholdsでない)follower は保持しない.性質は 脱力したときと放置したときの性質 にある. - scriptから決めるときは
s.close(release=True)(脱力)またはs.close(release=False)(保持)を呼ぶ. - logは,設定fileの隣の
logs/<dataset name>-<time>/に残ります(例:logs/rakuda2-digit-20261102-101500/). dora.log- leader のtrace
- 場所は
s.log_dirで分かります.
状態画面と終了時の確認
Session.launch(..., screen="auto")が既定です.標準出力が端末なら,Richのnode状態表とmessage欄を別threadで10 Hzで更新します.対話CLIは別の Textual操作画面 を使います.screen=Falseで表示を切れます.pipeなど標準出力が端末でない場合,既定では表示しません.- scriptの
print()は画面の上に出ます.画面はkeyを読まないので,input()や API 呼び出しで操作できます.CLI のrecordのようなkey操作は追加しません. - 保存結果と品質の警告もmessage欄に出ます.
operator="auto"が既定です.標準入力が端末なら,終了前に leader と follower の腕を支える確認を出します.- 腕を支えてから Enter を押すと,follower のtorqueも切って終了します.
holdと入力すると,follower は保持を続けたまま終了します.- follower が
torque: falseのときは,leader の確認だけを出します. - 確認文は機器の性質から作ります.支える機器(
falls)と物を外す機器(lets_go)を名前で示し,保持できる follower がなければ,holdは脱力せずに終了することを示します(その follower は保持しないかもしれません).s.release_note()で,releaseの前にすることを文字列で得られます.release(confirm=False)は,これを添えたHardwareErrorを出します. operator=None,または標準入力が端末でない場合は,この確認を出さず,follower は保持を続けます.screen=Falseだけでは確認は無効になりません.- 保持を続けた follower は,電源を切る前に腕を支えるか安全な姿勢に下ろしてください.
task名の自動命名
task には,文字列のほか,EpisodeInfo を受け取って文字列を返す関数を渡せます.
EpisodeInfo の項目 |
中身 |
|---|---|
index |
datasetでのepisode番号(保存済みのepisode数) |
number |
この Session で録った数 |
dataset |
datasetの名前(repo_id の / より後ろ.有効なoptionの名前が付く.例:rakuda2-digit) |
previous |
この Session で保存した task の一覧 |
import itertools
objects = itertools.cycle(["cup", "bottle", "sponge"])
with Session.launch("rakuda2") as s:
s.wait_ready()
for _ in range(9):
s.record(lambda info: f"pick up the {next(objects)}", seconds=20)
# 番号つき,操作者に毎回たずねる,など
s.record(lambda info: f"trial {info.index:03d}", seconds=20)
s.record(lambda info: input(f"task for episode {info.index}: "), seconds=20)
- 決まった task の文字列は,LeRobotDataset の各frameの
taskに保存されます. - 学習のときは,そのまま言語指示として使えます.
細かく操作する
start(task),stop(),save(),discard(),estop(),release(confirm=True)を,1 つずつ呼べます.save()が返ったepisodeは,そのあとprocessが止まっても読めます.書き込みに失敗するとSessionErrorを上げ,そのepisodeを破棄します(それまでのepisodeは残り,次を録れます.保存したepisode).- 例:外部の合図で開始と終了を決めるとき.
- 状態を見る:
status():各nodeの最新の状態.reports():未読の報告.robot のmodeの変化など.- commandが拒否されたとき(leader が揃っていないなど)は,
SessionErrorになります. startは,前のepisodeが保存中・停止したまま未保存・記録中のときも拒否されます.recorderがstartを受けられなかったときは,followerと方策も止めます(記録されないepisodeが走り続けません).recordは,episodeの途中でcontrollerが失敗したとき(方策の例外など)と,followerがFAULT・ESTOPになったとき,そのepisodeを破棄してEpisodeFailed(SessionErrorの一種)を上げます.方策は次のepisodeでそのまま使えます.estop()は,nodeが落ちたあとでも送れます(ほかのcommandは,nodeの障害を見つけるとSessionErrorになります).Session.launchを呼んだprocessが終わると,どう終わった場合でも(強制終了,例外,端末を閉じた),dataflowは記録中のepisodeを破棄し,followerを保持させて終わります.close()を呼ばずに放置しても,robotが方策で動き続けることはありません.multiprocessingなどで作った子processが残っている間は,終わったと見なされません.home(timeout=None):leaderとfollowerを,controllerで選んだ初期姿勢へゆっくり動かし,両方が着くまで待ちます.着かないときや拒否されたときはSessionErrorです.record(..., home=True)は毎回のepisodeの前にこれを行います.詳しくは Leader の「初期姿勢へ動かす」を参照してください.teleop(),policy(),stop_teleop():方策にteleop:があるとき,leaderに切り替え(episodeの前,または途中の介入),方策に戻し,episodeの外のteleopを終えます.詳しくは teleopと介入 を参照してください.
s.start("wipe the table")
while not task_done(): # 利用者の判定
print(s.status()["follower"]["fields"]["mode"])
time.sleep(0.5)
stopped = s.stop() # まだ保存していない.metrics と warnings を見られる
s.save() if stopped.ok else s.discard()
方策・評価・episodeの選別
- 方策で動かす:方策で動かす
- 評価の成否を残す:評価の成否
- 残すepisodeを選ぶ(
keep):残すepisodeを選ぶ
sensor選択
load_config(..., sensors=[...], observe=[...]) と Session.launch(..., sensors=[...], observe=[...]) で,起動・記録するsensorと方策の観測を独立に指定します.None は設定の既定,[] は選択なしです.followerの状態は常に渡します.RobotConfig を渡す場合は load_config で選択を確定してください(Session.launch は追加の選択の引数を拒否します).
with Session.launch(
"rakuda2",
options=["policy"],
sensors=["camera", "audio"],
observe=["camera"],
policy="my_lab.policies:load",
) as session:
session.wait_ready()
session.record("pick", seconds=20)
- 未選択のsensorの YAML は読み込みません.起動していないsensorの観測を指定するとerrorです.
- 左右の DIGIT を含め,optionで追加されるsensorも個別に選べます.選択はoptionを適用したあとで解決します.
- 任意のsensorは
{file: nodes/audio.yaml, default: false}と宣言します.従来の文字列のpathは既定で有効です. observeに含めないsensorも,起動すれば記録します.
CLIでの指定は CLI のsensor選択 に,規則は sensor選択 にあります.音声 も参照してください.
1つのprocessで動かす:InProcessSession
InProcessSessionは,robotのbus,sensor,controller の呼び出しを,すべて呼び出し側のthreadで行います.- 使いどころ:
- 試験
- 模擬
- sensorの少ない簡単な機体
- 動かせるfollowerは1台です.followerが複数ある設定は
Sessionで動かします. - 方策を
step(policy=...)に直接渡せます. - ただし,方策やsensorが遅いと,制御の周期も遅れます.
- 実機の Rakuda-2 は,
Sessionで動かします. - motorを制御する leader(triggerのばね,重力補償)を使う設定では,
from_yamlを拒否します. - 方策の controller を使う設定でも,
from_yamlを拒否します. InProcessSession.from_yaml(path, *, options=None, mock=None, sensors=None, dataset=None)のoptions・mockはload_configと同じ意味です.
deviceやrepository内のsample fileを使わない模擬例です.record extraが必要です.この例の関節範囲・速度を実機へ流用しないでください.
from roboloom import InProcessSession
from roboloom.drivers.mock import MockBus
with InProcessSession.from_components(
robot="my-mock",
bus=MockBus(dimension=2),
joint_names=["j1", "j2"],
unit="count",
limits=[
{"name": "j1", "min": 0, "max": 4095, "max_speed": 3000, "max_jump": 400},
{"name": "j2", "min": 0, "max": 4095, "max_speed": 3000, "max_jump": 400},
],
controller=lambda obs: [2048, 2048],
output="datasets/my-mock",
repo_id="local/my-mock",
) as session:
session.start_episode("handover")
session.run(3.0)
session.stop_episode()
session.save_episode()
実機の収集手順は Rakuda2:teleopで収集 を参照してください.
API
roboloom.Session
A robot's running dora dataflow, directed from this process.
The control loops, sensors, recorder and policy run in their own processes; a
Session only starts the dataflow and sends it events (start, stop, save, the
task), so a slow or stopped script never delays a control loop. Create one with
Session.launch and use it as a context manager: leaving the with block discards
an unfinished episode, asks the operator to support the arms (when operator is
set) and stops the dataflow (close).
alive
property
True while the dataflow process is running.
actuated
property
Inputs whose motors run control laws; they go limp when the session ends.
torqued
property
Followers that run with torque, with their safety traits.
held
property
Followers that keep holding after the session unless released: torque on and
unattended: holds (core.traits).
limp_at_end
property
What goes, or may go, limp when the session ends without release: actuated inputs, and followers that do not hold unattended (or are not known to).
episodes
property
Episodes in the dataset, as the recorder last reported.
launch(config=None, *, experiment=None, options=None, mock=None, sensors=None, observe=None, video_codec=None, policy=None, policy_kwargs=None, log_dir=None, timeout=60.0, screen='auto', operator='auto')
classmethod
Validate and start the dataflow; returns once every node reports status.
options: the launch options to enable (None: their defaults);mock: True runs mock devices, False real ones (None: the YAML'smode, default hardware). Both apply only whenconfigis a path or name.experiment: an experiment's name (configs/roboloom/<name>.yaml), its file, a mapping with the same keys (e.g. what Hydra composed) or anExperiment. It gives the robot (whenconfigis left out), the options (whenoptionsis left out), what is laid over the nodes and where the episodes go. Its task, seconds or steps are yours to pass torecord.sensors,observe,video_codec: as inload_config(None: the YAML); with aRobotConfig, choose them inload_configinstead.policy("module:callable") andpolicy_kwargsreplace the policy of atype: policycontroller.screen: the node-status screen, drawn on a thread ("auto": when stdout is a terminal; seeroboloom.ui.screen).operator: asked to support the arms before closing makes an actuated leader go limp ("auto": a TerminalOperator when stdin is a terminal; None: no question).
release_note()
What the operator does before release turns torque off on the actuated inputs
and the followers.
close(timeout=15.0, *, release=None)
Discard an unfinished episode, stop every node and wait for the dataflow to exit.
An actuated leader goes limp. The follower torque goes off when release is
true; when it is None, the operator is asked (Enter releases, hold keeps the
follower holding) and without an operator the follower keeps holding.
status()
Latest status of every node: {name: {"role", "level", "fields", "t_ns"}}.
reports()
Reports not yet returned by a wait (robot mode changes, warnings, ...).
listen(listener)
Call listener with every report and session note (the screen uses this).
poll()
Read new reports from the dataflow; safe from any thread.
note(text, level='info')
A message from this side (e.g. quality warnings) for the listeners.
node_errors()
Error lines of failed nodes in dora.log (a node can die while dora keeps running).
wait_ready(timeout=30.0)
Until every controller may start an episode (e.g. the leader is aligned).
send(command, task='', **extra)
Send one command to every node; returns its id (echoed in the reports).
extra: what else the command carries (save: label, note).
wait_report(ident, timeout=10.0)
The report answering command ident; raises SessionError if it failed.
start(task)
Start recording an episode of task and let the controllers drive.
Raises SessionError when a controller refuses (e.g. the leader is not aligned
with the follower, alignment_tolerance) or while the previous episode is
recording, stopped but not saved, or being saved. When the recorder could not take the
start, the followers and the policy are stopped too, so no episode runs that
nobody records.
stop()
End the episode's recording; the result (not yet saved) has its metrics and
quality warnings, to decide between save and discard.
save(timeout=300.0, *, label=None, note=None)
Write the stopped episode (video encoding can take a while). When the write fails the episode is dropped (SessionError); the dataset keeps the episodes saved before it and the next one can be recorded.
label ("success" or "failure") and note are the verdict of an evaluation
episode. They go into the dataset's meta/roboloom_episodes.jsonl with the
task, the metrics and what the controllers reported; roboloom label sets
the verdict afterwards.
discard()
Stop the robot and drop the episode in progress.
home(timeout=None)
Move every leader-follower pair with a home pose there, slowly, and wait until
both arrived (at most timeout s, default the YAML's timeout_s). Leaders go back to
their normal laws a second after arriving. Raises SessionError when a node refused or did not
arrive.
begin_home()
Send home; returns (send time in monotonic ns, the YAML's timeout in s).
home_waiting(sent_ns)
Nodes still moving home since sent_ns (empty: arrived); raises SessionError
when one refused.
teleop()
Drive the followers with the leader: before an episode (nothing is recorded)
or during one (an intervention, recorded as action.intervention). Each joint
holds until the leader reaches it, then follows; the policy's status lists the
joints still to reach (pickup).
policy()
Hand the followers back to the policy during an episode (it restarts from where they are, with a fresh history).
stop_teleop()
Stop teleoperating outside an episode; the followers hold where they are.
estop()
Latch the follower in ESTOP (stays until the dataflow restarts). Written straight to the FIFO: it must also go out when a node has failed, which every other command refuses.
release(*, confirm=False)
Torque off the actuated inputs and the followers after stopping; first do what
release_note() says (support what falls, empty what lets go).
next_info()
What a task-naming function is given for the next episode.
record(task, seconds, *, keep=None, on_tick=None, tick_s=0.2, ready_timeout=60.0, home=False, label=None, note=None, steps=None)
One episode: wait until it may start (leader aligned with the follower, at most
ready_timeout s), start, record for seconds, stop, then save it unless keep
rejects the stopped episode (default: keep every one; quality warnings are in the
result either way).
steps: end as soon as the policy has executed this many actions (steps_done);
seconds is then the longest the episode may take (steps_timeout).
task is a string or a function of EpisodeInfo. on_tick(status) is called
every tick_s while recording. An exception or Ctrl-C discards the episode, and
so does a controller that fails or a follower that latches FAULT or ESTOP while
it runs (EpisodeFailed): an episode that broke off is not saved.
home=True first moves to the saved home pose (home), so every episode starts
from the same pose.
label is the verdict saved with the episode ("success" / "failure"), or a
function of the stopped episode returning it (None: no verdict); note is
free text (save).
roboloom.EpisodeInfo
dataclass
What a task-naming function knows about the episode it names.
Attributes:
| Name | Type | Description |
|---|---|---|
index |
int
|
The episode's index in the dataset (the episodes already saved). |
number |
int
|
The episodes this session has recorded so far. |
dataset |
str
|
The dataset's name, e.g. |
previous |
list[str]
|
The tasks this session has saved, oldest first. |
roboloom.EpisodeResult
dataclass
A stopped episode, saved or not (Session.stop, save, record).
Attributes:
| Name | Type | Description |
|---|---|---|
task |
str
|
The episode's task text. |
saved |
bool
|
Whether the episode was written to the dataset. |
frames |
int
|
The rows recorded. |
index |
int | None
|
The episode's index in the dataset once saved, else None. |
metrics |
dict
|
The recorder's quality metrics (missing rates, the share of valid actions, the control interval distribution). |
warnings |
list[str]
|
The quality limits ( |
label |
str | None
|
The verdict it was saved with, |
ok
property
True when no quality limit was exceeded (warnings is empty).
roboloom.EpisodeFailed
Bases: SessionError
The episode could not go on (a controller failed, a follower latched FAULT or ESTOP); it was discarded. A failed policy is ready again for the next episode.
roboloom.SessionError
Bases: RoboloomError
The dataflow refused a command or stopped. report is the node's answer, when
one caused it.
roboloom.InProcessSession
Collect with a controller or run a policy using the same checked robot bus.
controller takes the current observation dict and returns a target vector or
None. sensors maps source names to callables returning Sample or None.
For YAML configurations, from_yaml supplies the configured controller by default.
from_yaml(path, *, options=None, mock=None, sensors=None, dataset=None)
classmethod
options: launch options to enable (None: their defaults); mock: override
the YAML's mode (None: as written, default hardware).
from_components(*, robot, bus, joint_names, limits, controller=None, sensors=None, image_shapes=None, audio_sample_rate=None, fps=10, control_hz=30, output='dataset', repo_id='local/in-process', unit, dataset=None)
classmethod
A one-follower session built from parts, without a configuration file.
bus: the follower's joint bus (e.g.roboloom.drivers.mock.MockBus);joint_names,unitandlimits(one mapping per joint:name,min,max,max_speed,max_jump) describe its joints.controller: observation dict -> targets (or None), as in the constructor.sensors: source name -> callable returning aSampleor None. A source namedleaderis recorded as a joint input; the others need an entry inimage_shapes(HWC RGB) or are theaudiostream (audio_sample_rate).fps: the dataset rate;control_hz(at leastfps): the control rate.output,repo_id: where and under which id the dataset is written.
start_episode(task)
Start recording an episode of task and arm the follower.
start_inference()
Arm checked one-step control without creating an episode.
stop_inference()
Stop commanding the follower (it holds); the counterpart of start_inference.
step(*, policy=None)
One checked control tick; policy overrides the configured controller.
run(seconds, *, policy=None)
Call step at the control rate for seconds; returns each step's result.
stop_episode()
Stop the follower and end the episode's recording (save or discard it next).
save_episode(label=None, note=None)
Save the stopped episode; label ("success" / "failure") and note go
into the dataset's record of it (recording.episodes).
discard_episode()
Stop the follower and drop the episode in progress.
estop()
Latch the follower in ESTOP.
release()
Torque off after stop; support the arm first. Latched until the robot is reopened.
close()
Close the recorder and the bus without releasing torque: what the follower does
next is its traits().unattended (holds, goes limp, or not known).
roboloom.load_config(config=None, root=PROJECT_ROOT, *, options=None, mock=None, sensors=None, observe=None, video_codec=None, experiment=None)
Load and validate a configuration file, or one named under the project root.
experiment: an Experiment (load_experiment) or its settings(); see
read_config. Its options are not applied here: pass them as options.
sensors: active recorded sensors (None: defaults, []: none).
observe: policy inputs (None: YAML, []: follower state only).
video_codec: codec of storage: video images, "av1" (alias "libsvtav1") or
"h264" (None: the recorder YAML's video_codec, else the saved videos' codec when
appending, else H.264).
options: the launch options to enable (None: their defaults). mock: override the
YAML's mode (True: mock devices, False: hardware).
roboloom.resolve_config(arg=None, root=PROJECT_ROOT)
A configuration file, or a robot name under the project root: rakuda2 is
.roboloom/rakuda2/config.yaml. Without a name, the only robot under the root.
roboloom.init_project(robot, root=PROJECT_ROOT, template=None)
Create root/robot from the shipped template of robot, or from template (a
package holding template/, or a directory).
roboloom.Sample
dataclass
One reading of a source.
Attributes:
| Name | Type | Description |
|---|---|---|
value |
ndarray
|
The reading (joint positions, an HWC RGB image, PCM audio, ...). |
capture_ns |
int
|
When it was captured, in |
例外
roboloom.RoboloomError
Bases: Exception
Base of every error caused by configuration, setup or the robot, not by a bug.
roboloom.ConfigError
Bases: RoboloomError, ValueError
A configuration or calibration file is invalid or still has SET_ values.
roboloom.HardwareError
Bases: RoboloomError, RuntimeError
The devices are not in the state a command needs (torque on, wrong mode, ...).
roboloom.SetupError
Bases: RoboloomError
A measurement or fit cannot proceed, or its result is not good enough to save.
roboloom.errors.Cancelled
Bases: RoboloomError
The operator declined a confirmation; nothing was written.