runs
Training runs — submitted here, executed by an executor: a self-hosted worker (M7-RL) or Modal (modal, not built yet)
GET /v1/runs
List training runs
Runs visible to the caller (created by them or owned by an org they belong to), newest first. A self-hosted runner discovers its work here with status=queued&executor=worker (M7-RL).
Parameters
| Name | In | Type | Description |
|---|---|---|---|
status | query | JobStatus | |
executor | query | RunExecutor | Filter by executor ( |
owner | query | Handle | Filter by billing owner handle (username or org slug). |
kind | query | RunKind | UI2 — filter by kind: |
robot | query | string | UI2 — filter by the robot the recipe names ( |
gated | query | boolean | UI2 — |
parent_run_id | query | Id | UI2 — the direct forks of one run (its lineage children). |
facets | query | boolean | UI2 — |
limit | query | integer | Page size. |
cursor | query | string | Opaque cursor from the previous page's |
Responses
POST /v1/runs
Create a training run
Queues a training run from a recipe. The API validates the recipe envelope and the dataset ref URI format (lucen://, hf://, github://); external ref resolution happens in the executor. owner defaults to the caller; pass an org slug to bill an org. parent_run_id records lineage (fork / retrain attribution) and is what the coach's resolved-config diff runs against.
Executor (M7-RL). recipe.executor picks the backend; absent means worker. A worker run is recorded as queued and stays there until a self-hosted runner (lucen worker run, holding a train key for this owner) claims it — the hub does not track connected runners, so a run with no runner simply waits. A modal run (W13) is recorded queued and the hub then spawns the template's deployed Modal function on the tier's GPU (modal_call_id / dispatched_at on the run); the container claims and reports like a runner. On a deployment without MODAL_APP_NAME the submit answers 503 naming it rather than recording a run that would never start; a spawn that fails after the commit leaves the run failed with the reason, never queued forever.
Idempotency. Send an Idempotency-Key header to make retries safe: per credential, the same key with the same body within 24 h returns the run created the first time (200, same id); the same key with a different body is 422.
GPU-hour quota (M7-RL, L0). The GPU hours an account may reserve on priced executors (modal) are capped: the sum of max_hours (service default 4) over the priced runs the user created in the last 24 h that are not canceled — through ANY credential, a session, any key, an agent session's keys — plus this run, must stay within the deployment's RUN_DAILY_GPU_HOUR_QUOTA (default 24 h), else 429 with Retry-After, X-GPU-Quota-Limit / -Used and a problem body naming the numbers. An API key's own daily_gpu_hour_quota, when set, narrows that further for the runs the key created. worker runs are unpriced and never count.
Planned templates (M7-RL ③). A template that the hardware planner knows but no executor implements yet (pi0_lora, groot_lora, openvla_lora) is 503 naming the missing template, on every executor, and nothing is recorded — POST /v1/runs/recommend still answers for it. Ask recommend before create: it picks the GPU tier from the model family, the method and the dataset's shape.
Isaac Lab on Modal (C1). ppo_isaac with executor: modal runs the owner's Isaac Lab task on a hosted L40S (the default when recipe.gpu is null) or L4 / A10G. NVIDIA's Isaac Sim licence covers a company training its own robots, not a service training other people's, so the hosted template is 403 for any owner not in the deployment's ISAAC_HOSTED_OWNERS (default lucen), naming the reason; a worker run of ppo_isaac (the owner's own box) is never restricted. Isaac Sim renders through RTX, so gpu: a100 | h100 is 422 for it. recipe.parent_checkpoint_step starts a child from a parent's published checkpoint (422 if the parent does not list that step). A modal run that Modal preempts is re-claimed by its own call and resumes from its last published checkpoint (claimRun).
Hosted execution (L0). There is no payment system, so executor: modal runs only for owners in the deployment's HOSTED_OWNERS (default lucen; * = everyone): anyone else is 403 with errors[].type: forbidden.hosted_owner, a sentence naming the setting and the self-hosted way, and nothing recorded or dispatched. A worker run is never restricted, but a recipe.gate whose rollouts would be hosted is asked the same question. On a deployment with no Modal app the 503 answers instead.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
Idempotency-Key | header | string | Client-chosen key that makes a create retry-safe. Scoped to the credential (API key, or session user): within 24 h the same key with the same body returns the original resource with |
Request body
| Field | Type | Description |
|---|---|---|
recipe | Recipe | |
owner | Handle | null | Billing owner handle; defaults to the caller's username. |
parent_run_id | string | null | Lineage — the run this one forks/retrains from. |
Responses
| Status | Description | Body |
|---|---|---|
200 | Replay — this | Run |
201 | Run queued. | Run |
401 | Missing or invalid credentials. | Problem |
403 | Authenticated but not allowed (visibility, membership or scope). | Problem |
422 | Request failed validation. | Problem |
429 | The account's (or the key's own) daily GPU-hour quota would be exceeded (priced executors only). | Problem |
503 | The requested executor cannot run on this deployment ( | Problem |
POST /v1/runs/import
Import a training run trained off-platform
Records a run that trained somewhere else (kind: external) so the Training Coach can read it and so it appears on /runs beside cloud runs. The three artifacts — resolved config, metrics summary, scorecard — are sent inline and stored as objects; parent_run_id records lineage, and is what makes the coach's resolved-config diff (its first piece of attribution evidence) possible. Importing spends no GPU dollars, so this is key:write, not key:train. Imported runs have no live logs: logs_key is always null.
Request body
| Field | Type | Description |
|---|---|---|
artifacts | RunImportArtifacts | |
owner | Handle | null | Owner handle; defaults to the caller's username. |
parent_run_id | Id | null | Lineage. Set it whenever one exists — the coach's first piece of attribution evidence is this run's resolved config diffed against the parent's. |
template | string | null | What produced the run, free text. Defaults to |
dataset | array of DatasetRef | Dataset refs the run trained on, if any. A simulator-only RL run legitimately has none. |
robot | string | null | |
status | JobStatus | |
hyperparams | object | null |
Responses
| Status | Description | Body |
|---|---|---|
201 | Run imported. | Run |
401 | Missing or invalid credentials. | Problem |
403 | Authenticated but not allowed (visibility, membership or scope). | Problem |
413 | An artifact is over the per-artifact size limit, or the owner's storage quota would be exceeded (L0: imports count toward it). | Problem |
422 | Request failed validation. | Problem |
POST /v1/runs/estimate
Price a training run before submitting it
The same body as createRun, validated the same way (owner, dataset refs, parent), and nothing is created: the answer is the most the run can cost — the tier's billed hourly rate × the wall-clock cap (recipe.max_hours, else the service default of 4 h), then capped by recipe.budget_usd when one is set. Rates are the ones GET /v1/hardware_tiers publishes. A worker run executes on the owner's own machine and costs the hub nothing, so its estimate is cost_usd_max: 0, priced: false — the same rule the coach applies to an unknown model id. A modal estimate is priced whether or not this deployment can run it: the price list exists on every deployment, the backend only where MODAL_APP_NAME is set. (M7-RL, W13) C1: the rate is the tier's plus host_rate_usd_per_hour (the CPU cores and memory ppo_isaac's function reserves), and a hosted ppo_isaac estimate for an owner outside ISAAC_HOSTED_OWNERS is the same 403 createRun answers — a price for something the owner may not buy would be a false answer.
Hosted execution (L0). A modal estimate for an owner outside HOSTED_OWNERS is the same 403 (forbidden.hosted_owner) createRun answers.
Request body
| Field | Type | Description |
|---|---|---|
recipe | Recipe | |
owner | Handle | null | Billing owner handle; defaults to the caller's username. |
parent_run_id | string | null | Lineage — the run this one forks/retrains from. |
Responses
| Status | Description | Body |
|---|---|---|
200 | The estimate. | RunEstimate |
401 | Missing or invalid credentials. | Problem |
403 | Authenticated but not allowed (visibility, membership or scope). | Problem |
422 | Request failed validation. | Problem |
POST /v1/runs/recommend
Recommend a GPU tier for a training run (deterministic planner)
The same body as createRun, validated the same way, and nothing is created: the answer is which hardware_tiers entry the run needs. Deterministic — no model is in this path. The hub looks the recipe's template up in its training_policies registry (model family, parameter count, method lora | full | rl, control class, per-sample activation cost, published requirements — each row carries the URL it was read from), reads the first lucen:// dataset ref's meta/info.json with the episode viewer's parser (cameras and their resolution, fps, episodes, frames, state and action width), and computes vram_estimate_gb = runtime floor + params × bytes × (lora ? 1.6 : 6) + samples × activations(cameras, resolution, action chunk), where samples is hyperparams.batch_size for imitation learning and hyperparams.num_envs for RL. A published floor (GR00T: "40 GB+ VRAM") raises the estimate when the arithmetic lands below it. The smallest tier whose VRAM covers the estimate plus 20 % headroom is tier; every tier that does is fits_on; est_hours comes from the registry's reference throughput scaled by the tier's relative BF16 tensor throughput, and cost_usd_est / cost_usd_max from the tier's billed rate (a worker recipe is unpriced, and the tier then names the GPU class the owner's own machine needs). Measured beats modelled: every succeeded reportRun that carried peak_vram_gb / samples_per_s updates a rolling value per (template, tier, batch size), and when one exists for this recipe's batch size it replaces the arithmetic — basis says which was used, and why[] shows the sums either way. A template registered for planning only (implemented: false) is answered here and refused by createRun with 503. An unknown template is 422 naming the registered ones; a dataset ref the caller cannot read, or that is not a LeRobotDataset, is a warning and the registry's defaults are used — never a 404 (a planner must not be an existence oracle). explanation is reserved for M16 and is always null today. (M7-RL ③)
Request body
| Field | Type | Description |
|---|---|---|
recipe | Recipe | |
owner | Handle | null | Billing owner handle; defaults to the caller's username. |
parent_run_id | string | null | Lineage — the run this one forks/retrains from. |
Responses
| Status | Description | Body |
|---|---|---|
200 | The recommendation, with its arithmetic. | RunRecommendation |
401 | Missing or invalid credentials. | Problem |
403 | Authenticated but not allowed (visibility, membership or scope). | Problem |
422 | Request failed validation. | Problem |
GET /v1/runs/{run_id}
Get a training run
Parameters
| Name | In | Type | Description |
|---|---|---|---|
run_id | path | Id | Training run id ( |
Responses
GET /v1/runs/{run_id}/logs
Stream run logs (SSE)
Server-Sent Events stream (text/event-stream). Each event is event: log with a JSON LogEvent payload ({ts, stream, line}). Replays existing logs from tail; with follow=true the stream stays open while the run is active, then closes with event: end.
What is on the wire (M7-RL). The log is the sequence of chunks a runner sent through reportRun, one object per report under logs_key; the stream emits one log event per line, ts being the chunk's arrival time (a runner batches ~5 s of output per report, so lines inside one chunk share a stamp) and stream being stdout for the runner's output and stderr for a notice the hub wrote into the log itself (a lease re-queue, a cancel request). The same stream also carries event: metric — one MetricEvent per metrics.jsonl line, all of them, unaffected by tail — so a client draws the loss curve from the same connection. event: end always closes the stream, with the run's status at that moment as its payload; with follow=true it comes once the run is terminal, otherwise right after the replay. An imported (external) run has no log and answers end at once. Same visibility as getRun: a run you may not see is 404.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
run_id | path | Id | Training run id ( |
follow | query | boolean | Keep the stream open while the run is queued/running. |
tail | query | integer | Replay only the last N log lines (default = all). Metric events are never tailed. |
strip_ansi | query | boolean | L1 — remove ANSI escape sequences (colour, bold, cursor control) from every log line before it is sent. Off by default: the log is the trainer's bytes, and a terminal pane renders rsl_rl's bold header from them. An agent reading text sets it (the MCP |
Responses
GET /v1/runs/{run_id}/logs/download
Download a run's whole log as text
L1 — every log chunk of the run concatenated, in arrival order, as one text/plain; charset=utf-8 body (the Terminal tab's Download button, or curl … > run.log): exactly the bytes the runner reported and the hub's own [lucen <iso>] … notices, nothing added between chunks, in the order streamRunLogs replays them. ANSI SGR sequences (rsl_rl's bold ESC[1m header) are kept unless strip_ansi=true. Sent with Content-Disposition: attachment; filename="<run_id>.log" and streamed chunk by chunk, so a long run's log never sits in the API's memory. Same visibility as getRun (404 when invisible); a run with no log yet (queued, or imported) answers an empty body.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
run_id | path | Id | Training run id ( |
strip_ansi | query | boolean | L1 — remove ANSI escape sequences (colour, bold, cursor control) from every log line before it is sent. Off by default: the log is the trainer's bytes, and a terminal pane renders rsl_rl's bold header from them. An agent reading text sets it (the MCP |
Responses
POST /v1/runs/{run_id}/cancel
Cancel a training run
Requests cancellation. Queued runs cancel immediately; running jobs transition to canceled once the worker acknowledges (billing stops within 60 s). Terminal runs return 409.
Mechanism (M7-RL). A queued run becomes canceled in this request (nothing ran, nothing is billed, no ledger row). A running worker run stays running — there is no process to kill on the hub — and gets cancel_requested_at set; the runner sees that field on its next reportRun response, kills the template and reports status: canceled, which is the terminal report that writes the ledger (cost 0 for a worker run). Until then the response is 202 with the run still running; asking again is idempotent. Anyone who can see the run may cancel it (the owner, an org member, or the runner's own credential). A modal run answers 501 until that backend exists. Writes a run.cancel audit row and a stderr notice into the run's log.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
run_id | path | Id | Training run id ( |
Responses
| Status | Description | Body |
|---|---|---|
202 | Cancellation requested; current run snapshot returned. | Run |
401 | Missing or invalid credentials. | Problem |
403 | Authenticated but not allowed (visibility, membership or scope). | Problem |
404 | Resource not found (or hidden from the caller). | Problem |
409 | State conflict (duplicate handle/slug, wrong repo kind, terminal job, ...). | Problem |
501 | The run's executor cannot cancel on this deployment ( | Problem |
GET /v1/runs/{run_id}/config
A run's resolved config, flattened, and its diff against the parent
UI2 — what the run page's Config tab and its "Config against the parent" panel read. The resolved config artifact (config_key) is flattened to leaf paths, and when the run declares a parent_run_id whose config the caller can also see, the two are diffed exactly as the coach's prompt builder diffs them (M14b's attribution evidence #1): changed columns listed, unchanged columns counted. Same visibility as getRun (404 when invisible).
L1 — grouped by RL structure, and readable while the run trains. A runner uploads config.yaml on its first progress report after the template wrote it (ppo_isaac: as soon as Isaac Lab dumped params/env.yaml + params/agent.yaml, before runner.learn), so this answers mid-run. groups[] sorts every key and every diff entry into Actions · Contract · Observations · Plant · Rewards · Network · Algorithm · Domain randomization · Curriculum · Terminations · Commands · Sim · Bookkeeping · Other by one rule table (docs/API.md "Config groups") — the same implementation Round.variables and the MCP get_run_config tool count with. When one side is the importer's manifest shape (contract.* / plant.* / lineage.*) and the other Isaac Lab's resolved env.* / agent.*, comparison is cross_shape: only the keys that map one to one (pairs[]) are compared, and the rest of each group is counted as parent_has_no_record / child_has_no_record — never as unchanged.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
run_id | path | Id | Training run id ( |
Responses
GET /v1/runs/{run_id}/checkpoints
List a run's checkpoints
Checkpoints are ordinary repo files under checkpoints/step_{N}/ in the run's output model repo. This lists them grouped by step, read from that repo's own file rows — there is no separate checkpoint store, and a checkpoint downloads through presignDownload on the repo like any other file. A run with no output repo yet (queued, or running and not past its first checkpoint) answers an empty list with repo: null. (M7-RL) Listed while the run trains (C1): the runner publishes every checkpoints/step_{N}/ the template writes as soon as it is complete — for an ONNX-shaped template it holds exactly one *.onnx stamped with lucen_manifest beside the framework checkpoint (model_{N}.pt for rsl_rl) — and the progress report that first publishes sets output_model_repo, so this list grows during training. Steps appear in order and are never rewritten (content addressing dedupes unchanged bytes). AG1: published_at — when the step's last file landed.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
run_id | path | Id | Training run id ( |
Responses
| Status | Description | Body |
|---|---|---|
200 | Checkpoints by step, oldest first. | CheckpointList |
401 | Missing or invalid credentials. | Problem |
403 | Authenticated but not allowed (visibility, membership or scope). | Problem |
404 | Resource not found (or hidden from the caller). | Problem |
POST /v1/runs/{run_id}/claim
Claim a queued worker run (self-hosted runner protocol)
Compare-and-set queued → running on a run whose executor is worker. Exactly one caller wins; a run that is already claimed, is terminal, or is not a worker run answers 409. Records claimed_by, claimed_at, started_at and the first heartbeat_at, writes an audit row, and returns the run — its recipe, parent_run_id and owner are what a runner needs to assemble the template's inputs. key:train because a runner spends the owner's compute on the owner's behalf; the run must be visible to the credential (own, or an org it belongs to) and an invisible run is 404. The claim records the claiming credential: only that credential may reportRun afterwards (409 for anyone else). Lease: every report renews heartbeat_at; a running run whose heartbeat is older than the hub's lease (RUN_LEASE_SECONDS, default 300 s) is re-queued by a periodic sweep — back to queued, claim fields cleared, partial metrics dropped, a stderr notice in its log — so a dead runner never strands a run, and the dead runner's next report is 409. (M7-RL) Re-claim after preemption (C1). Modal restarts a preempted input in a new container under the same function call. A claim on a running modal run is accepted — not 409 — when modal_call_id equals the run's own and the credential is the one that claimed it; the new claimer's name replaces claimed_by, the heartbeat is renewed, started_at (and so billing) is unchanged, resume_step (a checkpoint step this run already published, or null) is where the template restarts, the lost attempt's metrics past it are dropped, a stderr notice lands in the log and run.reclaim is audited. Every other claim on a non-queued run is still 409.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
run_id | path | Id | Training run id ( |
Request body
| Field | Type | Description |
|---|---|---|
worker | string | A name for the runner — its hostname, or anything the operator will recognise on the run page. Recorded as |
modal_call_id | string | null | C1. The Modal function call the claiming container is running ( |
resume_step | integer | null | C1, re-claim only. The checkpoint step the restarted container will resume from — one of the steps |
Responses
| Status | Description | Body |
|---|---|---|
200 | Claimed; the run as it now stands. | Run |
401 | Missing or invalid credentials. | Problem |
403 | Authenticated but not allowed (visibility, membership or scope). | Problem |
404 | Resource not found (or hidden from the caller). | Problem |
409 | State conflict (duplicate handle/slug, wrong repo kind, terminal job, ...). | Problem |
422 | Request failed validation. | Problem |
POST /v1/runs/{run_id}/report
Report progress or a terminal state for a claimed run
The runner's one write path after claimRun. The run must be running (409 otherwise). Every field is optional and additive: log_chunk (≤ 256 KiB of UTF-8) is stored as one object under runs/{run_id}/logs/ and logs_key becomes that prefix; metrics lines are appended to runs/{run_id}/metrics.jsonl and the last one is echoed on the run as latest_metrics; config_yaml (the resolved config) and scorecard_json land at config_key / scorecard_key exactly where importRun puts them, so the coach reads an executed run and an imported one the same way; output_model_repo names the model repo the runner published — it must be a model repo the caller can write, else 422; since C1 the runner sends it on the progress report that first publishes a checkpoint, not only on the terminal one. status may only move running → succeeded | failed, or running → canceled after the hub set cancel_requested_at (a canceled nobody asked for is 422); a terminal report sets finished_at, records cost_usd (0 for a worker run — the hub billed nothing), writes the run's ledger entry and an audit row. Every report refreshes heartbeat_at (the lease). The caller must be the credential that claimed the run — another key, or a runner whose run was re-queued after a missed lease, gets 409. Read cancel_requested_at on the returned run: when it is set, stop the template and report canceled. (M7-RL) Measurements (M7-RL ③): peak_vram_gb, samples_per_s and gpu_tier may ride on any report and are echoed on the run; on a succeeded terminal report they also update the hub's rolling measured value for (template, tier, batch size), which is what makes recommendRun prefer measured over modelled next time.
Parameters
| Name | In | Type | Description |
|---|---|---|---|
run_id | path | Id | Training run id ( |
Request body
| Field | Type | Description |
|---|---|---|
status | RunReportStatus | null | Omit (or |
metrics | array of object | Metrics lines to append to |
log_chunk | string | null | Raw stdout/stderr since the last report, ≤ 256 KiB of UTF-8. |
error | string | null | Failure summary; expected with |
config_yaml | string | null | The template's RESOLVED config, stored at |
scorecard_json | object | null | Acceptance scorecard, stored at |
output_model_repo | string | null |
|
peak_vram_gb | number | null | Peak GPU memory the template used, in GB (M7-RL ③). Echoed on the run; on a |
samples_per_s | number | null | Training throughput, samples per second (M7-RL ③). Same handling as |
gpu_tier | string | null | Which tier class the GPU that ran the template belongs to (M7-RL ③). Absent, the run's own |
progress | RunReportProgress | null | L1 — where the run stands, sent on progress reports; each report's object replaces the last and is echoed as |
Responses
| Status | Description | Body |
|---|---|---|
200 | Recorded; the run as it now stands. | Run |
401 | Missing or invalid credentials. | Problem |
403 | Authenticated but not allowed (visibility, membership or scope). | Problem |
404 | Resource not found (or hidden from the caller). | Problem |
409 | State conflict (duplicate handle/slug, wrong repo kind, terminal job, ...). | Problem |
413 | A log chunk, metrics batch or artifact is over its size limit, or what the report would store puts the run owner past their storage quota (L0); nothing was stored. | Problem |
422 | Request failed validation. | Problem |
GET /v1/hardware_tiers
GPU tiers and what an hour on each costs
The tiers Recipe.gpu accepts, each with the hourly rate a modal run is billed at. Rates are what you pay, all-in — the hub's margin is already inside them, and estimateRun multiplies exactly these by the wall-clock cap. worker runs are never billed and no tier applies to them. Public, because a price list that needs a key to read is not a price list. (M7-RL) C1 adds l40s, and one template whose function reserves a large CPU host (ppo_isaac) pays that host on top of the tier — estimateRun names it as host_rate_usd_per_hour. L0: hosting says who this deployment runs modal work for at all (HOSTED_OWNERS).
Responses
| Status | Description | Body |
|---|---|---|
200 | The tier table. | HardwareTierList |
401 | Missing or invalid credentials. | Problem |
Schemas (48)
The schemas these operations reach before any other tag’s do. A type that links elsewhere is rendered on that tag’s page.
RunExecutor
Where a run executes (M7-RL). worker — a self-hosted runner (lucen worker run) on the owner's own machine claims it; the hub orchestrates and bills nothing. modal — the hub's GPU backend (W13): the run is recorded queued, the hub spawns the template's deployed function on the tier's GPU, and the container claims, reports and publishes through the same routes a runner uses; billed elapsed seconds × tier rate. On a deployment without MODAL_APP_NAME (a laptop, CI) a modal submit answers 503 naming it and records nothing. From M15 the same two names are an inference endpoint's backend: worker is lucen serve on the owner's box (unpriced), modal is a hub-spawned container priced by the tier.
RunKind
cloud — submitted through POST /v1/runs and executed by an executor; Run.executor says which one (a self-hosted worker, or modal). external — trained off-platform and imported via POST /v1/runs/import; it carries artifacts and never live logs. Added in M14b; absent means cloud. orchestration (M16) — a planner run: no trainer, no GPU; its config_key holds the plan's inputs and its metrics_key the timeline, so parent_run_id lineage and the coach's config diff work on it like on any other run.
RunPage
RunCreate
Run
| Field | Type | Description |
|---|---|---|
id | Id | |
kind | RunKind | |
status | JobStatus | |
recipe | Recipe | |
recipe_hash | Sha256 | |
owner | RepoOwner | |
created_by | UserPublic | null | |
gpu_type | string | null | Resolved GPU type the run is billed on. |
cost_usd | number | null | Accrued cost in USD (4 decimal places). |
metrics_key | StorageKey | null | Storage key of the metrics: |
logs_key | StorageKey | null | Storage prefix of the log chunks, |
config_key | StorageKey | null | Storage key of the resolved training config (M14b). The coach's attribution evidence #1 is the diff of this against the parent's. |
scorecard_key | StorageKey | null | Storage key of the acceptance scorecard JSON (M14b). |
gate | RunGate | null | UI2 — the scorecard's verdict as two numbers, on every read: how many of its PASS/FAIL rows pass (the same reader the coach uses, |
gates | array of RunGateRow | null | UI2 — every PASS/FAIL row of the scorecard, |
output_model_repo | RepoRef | null | Model repo produced on success. |
parent_run_id | string | null | Lineage — the run this one forks/retrains from. |
executor | RunExecutor | null | Which backend runs it (M7-RL). Null for an |
claimed_by | string | null | The runner name given to |
claimed_at | string (date-time) | null (date-time) | |
heartbeat_at | string (date-time) | null (date-time) | Last time the runner claimed or reported (M7-RL). |
cancel_requested_at | string (date-time) | null (date-time) | Set by |
latest_metrics | object | null | The last metrics line the runner reported (M7-RL). |
peak_vram_gb | number | null | Peak GPU memory the template used, in GB, as the runner last reported it (M7-RL ③). Null until a report carries one. |
samples_per_s | number | null | Training throughput the runner last reported, samples per second (M7-RL ③). Null until a report carries one. |
modal_call_id | string | null | The Modal function call executing a |
dispatched_at | string (date-time) | null (date-time) | When the hub spawned the |
error | string | null | Failure summary when status is |
created_at | string (date-time) | |
started_at | string (date-time) | null (date-time) | |
finished_at | string (date-time) | null (date-time) | |
display_name | string | L1 — what to call the run before (and after) its output repo exists: the name of the campaign round that owns it, else |
display_name_source | string | |
progress | RunProgress | null | L1 — iteration, total, iterations/s and ETA as the runner last reported them; null until a report carries progress (a queued run, an imported one, a template that declares nothing). |
spend | RunSpend | null | L1 — what the run has cost so far and what is left of its budget, computed on every read by the function that bills it at the end. |
round | RunRound | null | L1 — the campaign round that owns this run (C3), when there is one the caller can see; null otherwise. |
sparkline | RunSparkline | null | AG1 — the reward curve decimated to at most 64 points, kept by the hub as the runner reports metrics; null until a metrics line carries a reward. |
RunImport
| Field | Type | Description |
|---|---|---|
artifacts | RunImportArtifacts | |
owner | Handle | null | Owner handle; defaults to the caller's username. |
parent_run_id | Id | null | Lineage. Set it whenever one exists — the coach's first piece of attribution evidence is this run's resolved config diffed against the parent's. |
template | string | null | What produced the run, free text. Defaults to |
dataset | array of DatasetRef | Dataset refs the run trained on, if any. A simulator-only RL run legitimately has none. |
robot | string | null | |
status | JobStatus | |
hyperparams | object | null |
RunEstimate
The most a run can cost, computed before anything is created.
| Field | Type | Description |
|---|---|---|
executor | RunExecutor | |
tier | string | null | Resolved GPU tier ( |
rate_usd_per_hour | number | Billed hourly rate (4 decimal places): the tier's, plus |
host_rate_usd_per_hour | number | C1. The part of |
max_hours | number |
|
cost_usd_max | number |
|
budget_usd | number | null | The recipe's hard cap, echoed. |
capped_by_budget | boolean | True when |
priced | boolean | False when the executor is not billed by the hub ( |
RunRecommendation
| Field | Type | Description |
|---|---|---|
template | string | |
model_family | string | |
control_class | ControlClass | |
method | TrainingMethod | |
implemented | boolean | False for a planning-only row — |
executor | RunExecutor | |
tier | string | null | The smallest tier that fits with headroom; null when none does. |
requested_tier | string | null |
|
vram_estimate_gb | number | The estimate itself (modelled or measured), before headroom. |
vram_required_gb | number |
|
headroom | number | The safety margin applied (0.2 unless the deployment changed it). |
fits_on | array of string | Every tier whose VRAM covers |
est_hours | number | null | Expected wall clock on |
rate_usd_per_hour | number | The tier's billed rate plus the template's host rate (C1, |
cost_usd_est | number | null |
|
cost_usd_max | number |
|
priced | boolean | |
basis | RecommendationBasis | |
measured | RecommendationMeasured | null | |
inputs | RecommendationInputs | |
dataset | RecommendationDataset | null | |
why | array of string | The arithmetic, one step per line, with the source URL of every published number. |
warnings | array of string | |
explanation | string | null | Reserved for M16 (a model's prose over this plan). Always null in v0. |
RunConfig
UI2 — a run's resolved config, flattened, and its diff against the parent when there is one.
| Field | Type | Description |
|---|---|---|
run_id | Id | |
parent_run_id | string | null | |
keys | array of RunConfigKey | Every leaf of this run's resolved config, sorted by path. Empty when the run has no readable config. |
total | integer |
|
diff | RunConfigDiff | null | Null when there is no parent, or one side has no parseable config — |
reason | string | null | Why |
shape | string | L1 — this run's config shape: |
parent_shape | string | null | The parent's config shape; null without a visible parent. |
comparison | string | null | How the diff was computed ( |
groups | array of RunConfigGroup | L1 — the keys and the diff by RL structure, in the rule table's order, every group listed (a group with nothing in it on either side has |
pairs | array of RunConfigPair |
|
declared | array of RunConfigDeclared | The owning round's declared keys and what the diff says about each; |
CheckpointList
| Field | Type | Description |
|---|---|---|
items | array of RunCheckpoint | |
repo | RepoRef | null | The output model repo the checkpoints live in; null when none exists yet. |
not_fine_tunable | string | null | WA — why this run cannot be a fine-tune's parent, in one sentence: set when the run has ended (or was imported) and published no |
RunClaim
| Field | Type | Description |
|---|---|---|
worker | string | A name for the runner — its hostname, or anything the operator will recognise on the run page. Recorded as |
modal_call_id | string | null | C1. The Modal function call the claiming container is running ( |
resume_step | integer | null | C1, re-claim only. The checkpoint step the restarted container will resume from — one of the steps |
RunReport
One progress or terminal report from a runner. Every field is optional.
| Field | Type | Description |
|---|---|---|
status | RunReportStatus | null | Omit (or |
metrics | array of object | Metrics lines to append to |
log_chunk | string | null | Raw stdout/stderr since the last report, ≤ 256 KiB of UTF-8. |
error | string | null | Failure summary; expected with |
config_yaml | string | null | The template's RESOLVED config, stored at |
scorecard_json | object | null | Acceptance scorecard, stored at |
output_model_repo | string | null |
|
peak_vram_gb | number | null | Peak GPU memory the template used, in GB (M7-RL ③). Echoed on the run; on a |
samples_per_s | number | null | Training throughput, samples per second (M7-RL ③). Same handling as |
gpu_tier | string | null | Which tier class the GPU that ran the template belongs to (M7-RL ③). Absent, the run's own |
progress | RunReportProgress | null | L1 — where the run stands, sent on progress reports; each report's object replaces the last and is echoed as |
HardwareTierList
| Field | Type | Description |
|---|---|---|
items | array of HardwareTier | |
hosting | HostedExecution |
RunFacets
UI2 — how the caller's whole visible set splits (the request's own filters are NOT applied, so a rail's counts hold still while you click). Buckets with a count of zero are omitted, except gated, which always carries both true and false.
| Field | Type | Description |
|---|---|---|
total | integer | Every run the caller can see. |
kind | array of RunFacetBucket |
|
executor | array of RunFacetBucket | |
status | array of RunFacetBucket | |
robot | array of RunFacetBucket | By |
gated | array of RunFacetBucket |
|
Recipe
Training recipe envelope. template names a training template; dataset lists dataset ref URIs; hyperparams passes through to the template untouched. executor and budget_usd were added in M7-RL, additively.
| Field | Type | Description |
|---|---|---|
template | string | Which trainer runs. Free-form so an imported run can name whatever produced it ( |
dataset | array of DatasetRef | |
robot | string | null | The robot this run trains for, as the |
hyperparams | object | null | Template-specific hyperparameters (opaque in v0). |
gpu | string | null | GPU type; service default applies when null. |
max_hours | number | null | Hard wall-clock cap; the run is stopped at the limit (a |
executor | RunExecutor | null | Which backend executes the run (M7-RL). Null means |
budget_usd | number | null | Hard spend cap in USD (M7-RL). |
gate | CheckpointGateSpec | null | Gate checkpoints in simulation while the run trains, and stop it on a tripwire (C2). Needs |
parent_checkpoint_step | integer | null | C1. Start this run from the parent run's checkpoint at this step, not from scratch: the runner downloads the parent's |
RunGate
UI2 — a scorecard's PASS/FAIL rows, counted.
| Field | Type | Description |
|---|---|---|
passed | integer | |
total | integer | Zero when the scorecard states no PASS/FAIL row the reader recognises. |
RunGateRow
UI2 — one PASS/FAIL row of a run's scorecard, as coach/gates.py reads it.
| Field | Type | Description |
|---|---|---|
name | string | |
status | GateVerdict | |
detail | string | null | The row's own measurement text ( |
RunProgress
L1 — how far a run has got. The TEMPLATE declares where it starts and where it will stop (progress.json in its output directory — docs/API.md "The template contract"; ppo_isaac: the resume or parent step and that plus max_iterations); the RUNNER reads the newest iteration off metrics.jsonl (the template's step key — rsl_rl's iter) and measures iterations per second over the trailing minute of metrics lines, by their own clock (t) where they carry one; the HUB stores what the last report said and computes the ETA at read time.
| Field | Type | Description |
|---|---|---|
iteration | integer | null | The newest iteration the metrics show, in the trainer's own numbering. |
total_iterations | integer | null | The number the trainer counts to, as it prints it — rsl_rl's |
final_iteration | integer | null | The last iteration the trainer will log: |
start_iteration | integer | 0 from scratch; the parent's checkpoint step on a resume from a parent (C1), the resumed step after a preemption. |
iterations_per_s | number | null | Measured over the last minute of metrics lines; null until two lines exist. |
fraction | number | null |
|
eta_s | number | null | Seconds left at this read: |
eta_at | string (date-time) | null (date-time) |
|
step_key | string | null | The metrics key the iteration was read from ( |
as_of | string (date-time) | When the hub received the report this was read from. |
RunSpend
L1 — the run's cost so far, on every read. A priced (modal) run accrues elapsed since the claim × rate_usd_per_hour, where the rate is the tier's plus the template's CPU host (C1) — the same function (executors.modal.spend_usd) the terminal report bills with and the budget stop compares against, so the live number and the final bill cannot disagree. Once the run is terminal final is true and accrued_usd is cost_usd. An unpriced (worker, imported) run answers priced: false with the money fields null.
| Field | Type | Description |
|---|---|---|
priced | boolean | |
accrued_usd | number | null | USD to 4 decimal places. |
rate_usd_per_hour | number | null | The billed rate, tier plus host ( |
budget_usd | number | null |
|
remaining_usd | number | null |
|
final | boolean | True once the run is terminal and |
as_of | string (date-time) |
RunRound
L1 — which campaign round a run belongs to, for "round 4 of Omni walk" and the Shortlist link.
| Field | Type | Description |
|---|---|---|
campaign_id | Id | |
campaign_slug | string | |
campaign_name | string | |
campaign_owner | string | null | The campaign owner's handle. |
n | integer | The round's index in its campaign. |
name | string | The round's name. |
RunSparkline
AG1 — a run's reward curve decimated by the hub to at most 64 points, so a run card needs no stream.
| Field | Type | Description |
|---|---|---|
metric | string | The metrics key it follows ( |
points | array of array of number |
|
RunImportArtifacts
The three documents the coach reads, sent inline. Each is capped at COACH_MAX_ARTIFACT_BYTES (4 MiB by default) — a resolved config and a scorecard are kilobytes, and anything larger belongs in a repo.
| Field | Type | Description |
|---|---|---|
config_yaml | string | The RESOLVED training config, not the source template. |
metrics_summary | string | Metrics as text (CSV, JSONL tail, or a table). |
scorecard_json | object | null | Acceptance results; PASS rows become standing constraints. |
DatasetRef
Dataset reference URI. Schemes: lucen://{owner}/{repo} (this hub), hf://{namespace}/{name} (Hugging Face), github://{owner}/{repo} with optional @ref and /subpath. v0 validates the format only; workers resolve hf/github refs at training time.
TrainingMethod
lora — adapters on a frozen base; full — every weight; rl — a simulator drives a small policy.
RecommendationBasis
modelled — the arithmetic in why[]; measured — a rolling value from succeeded runs of this template at this batch size replaced the arithmetic (measured says which run count and tier).
RecommendationMeasured
The measured value the planner used, when basis is measured.
| Field | Type | Description |
|---|---|---|
tier | string | Tier the measurements came from ( |
batch_size | integer | |
peak_vram_gb | number | null | Highest peak reported so far. |
samples_per_s | number | null | Running mean of the reported throughput. |
n_runs | integer |
RecommendationInputs
The numbers the arithmetic multiplied, after defaults were applied.
| Field | Type | Description |
|---|---|---|
batch_size | integer |
|
cameras | integer | |
image_px | integer | The side of the square image the model actually sees (VLAs resize their inputs). |
chunk_size | integer | |
steps | integer |
RecommendationDataset
What the planner read out of the dataset's meta/info.json, when it could.
| Field | Type | Description |
|---|---|---|
ref | DatasetRef | |
cameras | array of object | |
key | string | |
width | integer | null | |
height | integer | null | |
fps | number | null | |
episodes | integer | null | |
frames | integer | null | |
state_dim | integer | null | |
action_dim | integer | null | |
robot_type | string | null |
RunConfigKey
| Field | Type | Description |
|---|---|---|
path | string | A flattened leaf path ( |
value | string | The leaf's value, rendered as text and elided past 200 characters. |
group | RunConfigGroupId | WA — on |
RunConfigDiff
UI2 — the coach's attribution evidence #1 (M14b, coach/config_diff.py), exposed to a page: the resolved config against the parent's, flat and path-addressed. Only the columns that moved are listed; the zero-variance columns are counted (unchanged_keys), never listed.
| Field | Type | Description |
|---|---|---|
entries | array of RunConfigDiffEntry | |
changed_keys | integer | |
unchanged_keys | integer | |
truncated | integer | Changed keys beyond the first 120, not listed. |
comparison | string | L1. |
parent_has_no_record | integer |
|
child_has_no_record | integer |
|
RunConfigGroup
L1 — one RL structure's slice of the config and of the diff. total counts this run's keys in the group; compared the keys that could be compared with the parent (the same path both sides, or a mapped pair on a cross_shape comparison), split into changed and unchanged; parent_has_no_record / child_has_no_record the rest — which is not "unchanged" and is never shown as such. A bookkeeping group is listed and counted but never counted as a change (counted: false).
| Field | Type | Description |
|---|---|---|
id | RunConfigGroupId | |
name | string | How a page titles it: |
counted | boolean | False for |
total | integer | |
parent_total | integer | null | The parent's keys in this group; null when there is no diff. |
compared | integer | |
changed | integer | |
unchanged | integer | |
parent_has_no_record | integer | |
child_has_no_record | integer | |
entries | array of RunConfigDiffEntry | The group's changed keys (same shape; added and removed keys included), in path order; |
unchanged_first | string | null | The first unchanged key by path, for a "first … last" summary line. |
unchanged_last | string | null | |
own_keys | array of RunConfigKey | P1 — this run's own keys and values in the group that were NOT compared with the parent: the keys the parent has no record of ( |
own_truncated | integer | P1 — |
summary | string | null | P1 — |
unchanged_summary | string | null | P1 — the group's unchanged keys as one line, by the same rule; on a |
RunConfigPair
L1 — one key pair a cross_shape comparison compared: the parent's manifest key and this run's Isaac Lab key that mean the same thing (action scale, clip and offset; observation terms; actuator gains, armature, friction and limits; control rate and decimation; lat_ff_gain). The list is the rule table in docs/API.md, derived from the founder's real resolved config.
| Field | Type | Description |
|---|---|---|
group | RunConfigGroupId | |
parent_path | string | |
child_path | string | |
from | string | null | The parent's value, rendered; null when the parent lacks the key. |
to | string | null | This run's value, rendered (after the pair's conversion, e.g. |
changed | boolean | null | Null when either side lacks the key (the pair could not be compared). |
rule | string | null | How the two were made comparable, in one phrase. |
RunConfigDeclared
L1 — one key of the owning round's declared change (C3), checked against the parent. moved — a counted key under it changed; not_moved — nothing under it changed; cannot_compare — the parent's config holds no record of it (a cross_shape comparison, or a key the parent never carried), so it neither passes nor fails.
| Field | Type | Description |
|---|---|---|
key | string | |
status | string | |
group | RunConfigGroupId | null | P1 — the RL structure the key falls in by the one rule table, so a declared key the parent never recorded still sits under its group (a command mode under |
summary | string | null | P1 — this run's value(s) under the key, one line by the summary rule (docs/API.md "Config summaries"); null when this run's config holds nothing under it. |
RunCheckpoint
| Field | Type | Description |
|---|---|---|
step | integer | |
path | string | The directory, e.g. |
files | array of RepoFileEntry | |
bytes | integer | |
published_at | string (date-time) | null (date-time) | AG1 — when the step's last file landed (the ONNX is pushed last, so this is when the checkpoint became complete). |
RunReportStatus
The only transitions a runner may make — running → succeeded | failed, and running → canceled once the hub has set cancel_requested_at on the run (a canceled the hub did not ask for is 422).
RunReportProgress
L1 — what a runner knows about a run's progress: the template's declared range (progress.json) and what the metrics show. Every field is optional; send what is known.
| Field | Type | Description |
|---|---|---|
iteration | integer | null | The newest iteration in |
total_iterations | integer | null | |
final_iteration | integer | null | |
start_iteration | integer | null | |
iterations_per_s | number | null | |
step_key | string | null |
HardwareTier
| Field | Type | Description |
|---|---|---|
tier | string | |
rate_usd_per_hour | number | USD per GPU-hour as billed — the hub's margin is already included; there is no cheaper number. A template whose Modal function reserves a large CPU host beside the GPU ( |
description | string |
HostedExecution
L0 — who this deployment runs executor: modal work for (runs, endpoints, rollouts, scans, gates, robot checks). There is no payment system, so it is a list of owners (HOSTED_OWNERS) plus, for every other account, a small monthly allowance for CPU work (K2, allowance); anything else is 403 (forbidden.hosted_owner) and runs on the owner's own machine. A client reads this to offer modal only where it would be accepted. Every hosted job acts as its owner: the hub mints a key for that one job (its owner's, scoped to the job's own operations, dead once the job ends) and hands it to the container (K2).
| Field | Type | Description |
|---|---|---|
allowance | HostedAllowance | null | K2 — the monthly hosted allowance every account outside |
configured | boolean | A Modal app is configured ( |
everyone | boolean |
|
owners | array of string | The owner handles (users or orgs) the hub hosts for, lower-case; empty when |
RunFacetBucket
| Field | Type | Description |
|---|---|---|
value | string | The filter value that selects this bucket ( |
count | integer |
CheckpointGateSpec
Gate checkpoints while the run trains (C2, the founder's watch_ckpt.py). While the run is running, the hub looks for new checkpoints/step_<N>/ directories in its output model repo (one .onnx each) about once a minute and queues a rollout of the first checkpoint past every every_steps boundary — battery (default watch), seeds (default 3), at delay (default 2) — on recipe.robot. tripwires are evaluated on the gated results; one that fires asks the run to stop (a notice in its log names the tripwire, the step and the rollout), exactly as the budget stop does.
| Field | Type | Description |
|---|---|---|
every_steps | integer | Gate the first checkpoint at or past each multiple of this. |
battery | string | |
seeds | integer | |
delay | integer | Control steps of actuation delay ( |
executor | RolloutExecutor | null | Null means the run's own executor. |
tripwires | array of GateTripwire | |
video | VideoMode | null | L1 — what each gate rollout renders. Null keeps C2's behaviour ( |
horizon_s | number | null | P1 — the seconds each seed runs, as the battery defines it for the run's robot ( |
cell_count | integer | null | P1 — how many cells the battery runs on the run's robot ( |
GateVerdict
A cell's verdict under the harness rule.
RunConfigGroupId
L1 — the RL structure a resolved-config key belongs to (docs/API.md "Config groups": one rule table, first match wins, for both the Isaac Lab and the importer's manifest shape).
RunConfigDiffEntry
| Field | Type | Description |
|---|---|---|
path | string | |
kind | string | |
from | string | null | The parent's value; null when the key is absent there. |
to | string | null | This run's value; null when the key is absent here. |
in_declared_change | boolean | null | C3 — for a campaign round's run: |
group | string | null | L1 — the |
parent_path | string | null | L1 — on a |
HostedAllowance
K2 — what an account outside HOSTED_OWNERS may spend on the hub's Modal account per calendar month (UTC), CPU work only: hosted robot and model checks, ppo_mujoco runs, CPU sim rollouts (single, scans, gates). Every GPU tier and endpoint stays HOSTED_OWNERS-only (L0's 403). A job reserves its most possible cost before it is queued (max_hours x rate, capped by budget_usd, for a run; the function's timeout x its host rate for a rollout or check) and settles from the ledger when it ends; a job that does not fit is 403 with errors[].type: forbidden.hosted_allowance.
| Field | Type | Description |
|---|---|---|
limit_usd | number | |
cpu_only | boolean | |
work | array of string | What the allowance covers, by name. |
viewer | HostedAllowanceUsage | null | The caller's own account this month; null when anonymous, or when the caller is in |
RolloutExecutor
Where a rollout runs (C2). worker (the default) — it waits queued for a lucen rollout worker the owner runs. modal — on a deployment that names a deployed Modal app (MODAL_APP_NAME), the hub spawns the app's rollout function (CPU, EGL, ffmpeg) after the commit and stores modal_call_id; the container claims and reports like any worker. Unset, modal is 503 naming the variable and nothing is recorded. A CPU rollout is unpriced either way.
GateTripwire
Stop the run when a gate metric crosses a line (C2): fires when the last consecutive gated checkpoints at or past from_step all satisfy metric op threshold. pass_rate = PASS cells / cells, alive_rate = seeds alive / seeds run, gates_passed = PASS cells. An early stop on success is a tripwire too: pass_rate >= 1.
| Field | Type | Description |
|---|---|---|
metric | string | |
op | string | |
threshold | number | |
from_step | integer | |
consecutive | integer |
VideoMode
L1 — what the worker renders of each cell's seed 0. full (the default, and what every rollout rendered before L1) — C2's three 640×480 views (side, front, feet) as mp4s, ~75 s of CPU per 20 s cell. poster — ONE still per cell: the side view, seed 0, at the episode's last control step (the end of the horizon, or the step the robot fell), 320×240 PNG — enough to scan a checkpoint row without playing anything. none — physics only. The measurements, the trajectory and the verdict are identical in all three: rendering never touches the physics. A gate rollout's videos can be rendered afterwards with renderRollout.
HostedAllowanceUsage
| Field | Type | Description |
|---|---|---|
account | string | |
used_usd | number | Settled spend this month (ledger rows of finished hosted jobs). |
reserved_usd | number | The most the account's queued and running hosted jobs can still cost. |
remaining_usd | number | |
period_start | string (date-time) | |
resets_at | string (date-time) |