this derivation will be built: /nix/store/1gx06f5n8y3nvqkc8lva86j8ww91fk6n-treefmt-check.drv building '/nix/store/1gx06f5n8y3nvqkc8lva86j8ww91fk6n-treefmt-check.drv' on 'ssh-ng://customer@mac02' building '/nix/store/1gx06f5n8y3nvqkc8lva86j8ww91fk6n-treefmt-check.drv' treefmt-check> treefmt v2.5.0traversed 295 files treefmt-check> emitted 246 files for processing treefmt-check> formatted 246 files (1 changed) in 1.536s treefmt-check> M PLAN-radicle.md treefmt-check> diff --git a/PLAN-radicle.md b/PLAN-radicle.md treefmt-check> index 7af7117..2b1ad28 100644 treefmt-check> --- a/PLAN-radicle.md treefmt-check> +++ b/PLAN-radicle.md treefmt-check> @@ -2,200 +2,189 @@ treefmt-check> treefmt-check> Add Radicle as a fourth forge next to GitHub, Gitea and GitLab. treefmt-check> treefmt-check> -Radicle has no HTTP API, no webhooks and no commit-status endpoint. treefmt-check> -Everything runs against a local `radicle-node` instead: treefmt-check> +Radicle has no HTTP API, no webhooks and no commit-status endpoint. Everything treefmt-check> +runs against a local `radicle-node` instead: treefmt-check> treefmt-check> - **Change source**: subscribe to the node's event stream. treefmt-check> - **Fetching**: repos already sit on disk in the node's storage. treefmt-check> -- **Reporting**: write "job" collaborative objects (COBs) that gossip treefmt-check> - through the network. treefmt-check> +- **Reporting**: write "job" collaborative objects (COBs) that gossip through treefmt-check> + the network. treefmt-check> treefmt-check> -radicle-desktop renders job COBs in its commit view, patch review and treefmt-check> -source header, so users see a ✓/✗ per commit linking to nixbot's web treefmt-check> -frontend. treefmt-check> +radicle-desktop renders job COBs in its commit view, patch review and source treefmt-check> +header, so users see a ✓/✗ per commit linking to nixbot's web frontend. treefmt-check> treefmt-check> ## Why not a ci-broker adapter? treefmt-check> treefmt-check> -Radicle's CI broker (`radicle-ci-broker`) handles triggering, filtering treefmt-check> -and COB writing, and dispatches to adapters — nixbot could be one. But treefmt-check> -its one-shot "run adapter, get verdict" model cannot express treefmt-check> -per-attribute builds, rerun-from-UI or effects, and it would stack a treefmt-check> -second queue and state machine in front of nixbot's scheduler. nixbot treefmt-check> -talks to the node directly; operators who want the broker's report treefmt-check> -pages can run it separately. treefmt-check> +Radicle's CI broker (`radicle-ci-broker`) handles triggering, filtering and COB treefmt-check> +writing, and dispatches to adapters — nixbot could be one. But its one-shot "run treefmt-check> +adapter, get verdict" model cannot express per-attribute builds, rerun-from-UI treefmt-check> +or effects, and it would stack a second queue and state machine in front of treefmt-check> +nixbot's scheduler. nixbot talks to the node directly; operators who want the treefmt-check> +broker's report pages can run it separately. treefmt-check> treefmt-check> ## 1. Tests first (red before green) treefmt-check> treefmt-check> Write these tests first; §2–§6 make them pass. treefmt-check> treefmt-check> - **Harness**: a pytest fixture building a real radicle setup in a temp treefmt-check> - `RAD_HOME` (`rad auth --stdin`, `rad init`, `radicle-node`), like the treefmt-check> - nixpkgs radicle tests. All fixtures come from it, no hand-written treefmt-check> - JSON. Adds `radicle-node` and `radicle-job` to the dev shell and treefmt-check> - flake test inputs; tests skip when the binaries are missing. treefmt-check> -- **Unit, events** (§2): event JSON → change mapping; patch-COB treefmt-check> - revision extraction (head, base, author DID) from a real patch. treefmt-check> -- **Unit, poster** (§4): status transitions → expected `rad-job` treefmt-check> - invocations; `StatusState` mapping. treefmt-check> -- **Idempotency** (§4): re-posting a terminal state after a nixbot treefmt-check> - restart must not duplicate runs. treefmt-check> -- **Integration** (§2–§5): spawn `radicle-node` in a temp `RAD_HOME`, treefmt-check> - `rad init` a flake repo, open a patch; assert nixbot builds it, treefmt-check> - `rad-job list` shows the runs, the central clone borrows objects via treefmt-check> - alternates (near-empty objects dir), and clone corruption after a treefmt-check> - storage prune recovers via re-clone. treefmt-check> -- **NixOS VM test** (§6): three nodes (alice, a seed, nixbot), modeled treefmt-check> - on `nixos/tests/radicle-ci-broker.nix`. treefmt-check> + `RAD_HOME` (`rad auth --stdin`, `rad init`, `radicle-node`), like the nixpkgs treefmt-check> + radicle tests. All fixtures come from it, no hand-written JSON. Adds treefmt-check> + `radicle-node` and `radicle-job` to the dev shell and flake test inputs; tests treefmt-check> + skip when the binaries are missing. treefmt-check> +- **Unit, events** (§2): event JSON → change mapping; patch-COB revision treefmt-check> + extraction (head, base, author DID) from a real patch. treefmt-check> +- **Unit, poster** (§4): status transitions → expected `rad-job` invocations; treefmt-check> + `StatusState` mapping. treefmt-check> +- **Idempotency** (§4): re-posting a terminal state after a nixbot restart must treefmt-check> + not duplicate runs. treefmt-check> +- **Integration** (§2–§5): spawn `radicle-node` in a temp `RAD_HOME`, `rad init` treefmt-check> + a flake repo, open a patch; assert nixbot builds it, `rad-job list` shows the treefmt-check> + runs, the central clone borrows objects via alternates (near-empty objects treefmt-check> + dir), and clone corruption after a storage prune recovers via re-clone. treefmt-check> +- **NixOS VM test** (§6): three nodes (alice, a seed, nixbot), modeled on treefmt-check> + `nixos/tests/radicle-ci-broker.nix`. treefmt-check> treefmt-check> ## 2. Change source: the node event stream treefmt-check> treefmt-check> -New module `nixbot/nixbot/forge/radicle.py`, enabled by a `radicle` treefmt-check> -config section (`RadicleConfig`): treefmt-check> +New module `nixbot/nixbot/forge/radicle.py`, enabled by a `radicle` config treefmt-check> +section (`RadicleConfig`): treefmt-check> treefmt-check> -| option | default | meaning | treefmt-check> -|-----------------|--------------|------------------------------------------| treefmt-check> -| `radicle_home` | `~/.radicle` | control socket, storage, keys | treefmt-check> -| `rid_allowlist` | unset | repos to build (default: all seeded) | treefmt-check> -| `follow_nodes` | unset | NIDs whose announced repos are auto-seeded and enabled | treefmt-check> +| option | default | meaning | treefmt-check> +| --------------- | ------------ | ---------------------------------------------------------- | treefmt-check> +| `radicle_home` | `~/.radicle` | control socket, storage, keys | treefmt-check> +| `rid_allowlist` | unset | repos to build (default: all seeded) | treefmt-check> +| `follow_nodes` | unset | NIDs whose announced repos are auto-seeded and enabled | treefmt-check> | `patch_trust` | `delegates` | whose patches to build: `delegates` \| `followed` \| `all` | treefmt-check> treefmt-check> **Subscribing.** Connect to `node/control.sock` and send the JSON line treefmt-check> -`{"command": "subscribe"}`; the node streams events as JSON lines (the treefmt-check> -stream `rad node events` prints). Subscriptions have no server-side treefmt-check> -expiry — the stream ends only on socket EOF or error; then reconnect treefmt-check> -with backoff. treefmt-check> +`{"command": "subscribe"}`; the node streams events as JSON lines (the stream treefmt-check> +`rad node events` prints). Subscriptions have no server-side expiry — the stream treefmt-check> +ends only on socket EOF or error; then reconnect with backoff. treefmt-check> treefmt-check> **Catching up.** Events missed while disconnected are gone. On every treefmt-check> -(re)connect, compare sigrefs of all seeded repos against known heads treefmt-check> -and process whatever moved. For patches, build only the latest revision treefmt-check> -per patch; revisions superseded while offline are skipped, like treefmt-check> -force-pushed-away commits on other forges. Patch close/merge does not treefmt-check> -cancel in-flight builds. treefmt-check> +(re)connect, compare sigrefs of all seeded repos against known heads and process treefmt-check> +whatever moved. For patches, build only the latest revision per patch; revisions treefmt-check> +superseded while offline are skipped, like force-pushed-away commits on other treefmt-check> +forges. Patch close/merge does not cancel in-flight builds. treefmt-check> treefmt-check> **Events consumed:** treefmt-check> treefmt-check> -- `RefsFetched` — refs arrived from a peer. Its `updated` list uses treefmt-check> - namespaced names (`refs/namespaces//refs/...`): treefmt-check> - - delegate namespace, `refs/heads/` moved → default-branch treefmt-check> - build (`head_changed`); treefmt-check> - - `refs/cobs/xyz.radicle.patch/*` moved (any namespace) → reload the treefmt-check> - patch COB and build its latest revision. The revision records head treefmt-check> - and base, so the change event carries `commit_sha=`, treefmt-check> - `base_sha=`, `branch=`, `pr_author=` and treefmt-check> - the existing merge-head-into-base pipeline works unchanged. treefmt-check> -- `LocalRefsAnnounced` — a local push; carries only a sigrefs pointer, treefmt-check> - so run the catch-up for that repo. treefmt-check> -- `SeedDiscovered` / `InventoryAnnounced` — repo lists from peers. When treefmt-check> - the announcing NID is in `follow_nodes`, seed the RID (`rad seed`, treefmt-check> - scope `followed`) and register the project. Radicle has no "seed treefmt-check> - repos of followed users" policy (following only scopes ref fetching; treefmt-check> - a node-wide allow policy would seed the whole network), so this treefmt-check> - subscription lives in nixbot. treefmt-check> - treefmt-check> -**Discovery.** `rad ls` has no JSON output, so enumerate the treefmt-check> -`storage/` directories and read each identity payload with treefmt-check> -`rad inspect --payload ` (name, description, default branch, treefmt-check> -visibility) into `DiscoveredRepo`: `forge="radicle"`, treefmt-check> -`forge_repo_id=`, `clone_url=`. A private RID treefmt-check> -becomes a private project under the existing `visibility.py` rules. treefmt-check> +- `RefsFetched` — refs arrived from a peer. Its `updated` list uses namespaced treefmt-check> + names (`refs/namespaces//refs/...`): treefmt-check> + - delegate namespace, `refs/heads/` moved → default-branch build treefmt-check> + (`head_changed`); treefmt-check> + - `refs/cobs/xyz.radicle.patch/*` moved (any namespace) → reload the patch COB treefmt-check> + and build its latest revision. The revision records head and base, so the treefmt-check> + change event carries `commit_sha=`, `base_sha=`, treefmt-check> + `branch=`, `pr_author=` and the existing treefmt-check> + merge-head-into-base pipeline works unchanged. treefmt-check> +- `LocalRefsAnnounced` — a local push; carries only a sigrefs pointer, so run treefmt-check> + the catch-up for that repo. treefmt-check> +- `SeedDiscovered` / `InventoryAnnounced` — repo lists from peers. When the treefmt-check> + announcing NID is in `follow_nodes`, seed the RID (`rad seed`, scope treefmt-check> + `followed`) and register the project. Radicle has no "seed repos of followed treefmt-check> + users" policy (following only scopes ref fetching; a node-wide allow policy treefmt-check> + would seed the whole network), so this subscription lives in nixbot. treefmt-check> + treefmt-check> +**Discovery.** `rad ls` has no JSON output, so enumerate the `storage/` treefmt-check> +directories and read each identity payload with `rad inspect --payload ` treefmt-check> +(name, description, default branch, visibility) into `DiscoveredRepo`: treefmt-check> +`forge="radicle"`, `forge_repo_id=`, `clone_url=`. A private treefmt-check> +RID becomes a private project under the existing `visibility.py` rules. treefmt-check> treefmt-check> ## 3. Fetching treefmt-check> treefmt-check> Nothing to authenticate: the clone URL is the local bare repo under treefmt-check> `storage/`; the credentials provider returns empty credentials and treefmt-check> -`git fetch` uses the path. The default branch comes from the identity treefmt-check> -payload, not HEAD guessing. treefmt-check> +`git fetch` uses the path. The default branch comes from the identity payload, treefmt-check> +not HEAD guessing. treefmt-check> treefmt-check> **Storage sharing via alternates.** Create the central clone with treefmt-check> -`git clone --bare --reference storage/`: it borrows objects from treefmt-check> -the node's storage, so fetches only update refs and the clone stays treefmt-check> -near-empty. Build merge commits land in nixbot's clone; the node's treefmt-check> -storage is never written to, and worktrees work unchanged. If the node treefmt-check> -prunes borrowed objects the clone corrupts — covered by gitrepo.py's treefmt-check> -existing delete-and-re-clone recovery. Do not point worktrees at treefmt-check> -`storage/` directly: merges and fetch locks would fight the node. treefmt-check> +`git clone --bare --reference storage/`: it borrows objects from the node's treefmt-check> +storage, so fetches only update refs and the clone stays near-empty. Build merge treefmt-check> +commits land in nixbot's clone; the node's storage is never written to, and treefmt-check> +worktrees work unchanged. If the node prunes borrowed objects the clone corrupts treefmt-check> +— covered by gitrepo.py's existing delete-and-re-clone recovery. Do not point treefmt-check> +worktrees at `storage/` directly: merges and fetch locks would fight the treefmt-check> +node. treefmt-check> treefmt-check> ## 4. Reporting: job COBs treefmt-check> treefmt-check> -`RadicleStatusPoster` implements the existing `CommitStatusPoster` treefmt-check> -protocol: treefmt-check> +`RadicleStatusPoster` implements the existing `CommitStatusPoster` protocol: treefmt-check> treefmt-check> -- **pending** → create (or find) the job COB for the commit, attach a treefmt-check> - `Started` run with the `log` URL = nixbot's build page; treefmt-check> +- **pending** → create (or find) the job COB for the commit, attach a `Started` treefmt-check> + run with the `log` URL = nixbot's build page; treefmt-check> - **success** → mark the run `Succeeded`; treefmt-check> -- **failure / error** → mark it `Failed` (`StatusState` has no treefmt-check> - cancelled member; cancellation arrives as error); treefmt-check> +- **failure / error** → mark it `Failed` (`StatusState` has no cancelled member; treefmt-check> + cancellation arrives as error); treefmt-check> - announce after each write so seeds pick it up. treefmt-check> treefmt-check> -COB writes shell out to the `rad-job` CLI (`new`, `run`, `succeeded`, treefmt-check> -`failed`, `list`), which signs with the profile key in `radicle_home`. treefmt-check> -`rad-job run` generates the run UUID and prints it; `succeeded`/`failed` treefmt-check> -take commit oid + UUID, so run UUIDs are persisted in the DB the way treefmt-check> -`CheckRunStore` persists GitHub check-run ids. treefmt-check> +COB writes shell out to the `rad-job` CLI (`new`, `run`, `succeeded`, `failed`, treefmt-check> +`list`), which signs with the profile key in `radicle_home`. `rad-job run` treefmt-check> +generates the run UUID and prints it; `succeeded`/`failed` take commit oid + treefmt-check> +UUID, so run UUIDs are persisted in the DB the way `CheckRunStore` persists treefmt-check> +GitHub check-run ids. treefmt-check> treefmt-check> -**Granularity.** One run per commit: an aggregate verdict plus a link, treefmt-check> -like the single "nixbot" status context on Gitea/GitLab. One run per treefmt-check> -check attribute is deferred: runs have no name field and desktop labels treefmt-check> -rows by the log URL, so dozens of near-identical labels may be treefmt-check> -unreadable. Validate in desktop before adopting. treefmt-check> +**Granularity.** One run per commit: an aggregate verdict plus a link, like the treefmt-check> +single "nixbot" status context on Gitea/GitLab. One run per check attribute is treefmt-check> +deferred: runs have no name field and desktop labels rows by the log URL, so treefmt-check> +dozens of near-identical labels may be unreadable. Validate in desktop before treefmt-check> +adopting. treefmt-check> treefmt-check> ## 5. Wiring into the service treefmt-check> treefmt-check> -- `service.py` builds the event source, credentials provider and status treefmt-check> - poster when `config.radicle` is set, and registers `"radicle"` in the treefmt-check> - forge dispatch tables. treefmt-check> +- `service.py` builds the event source, credentials provider and status poster treefmt-check> + when `config.radicle` is set, and registers `"radicle"` in the forge dispatch treefmt-check> + tables. treefmt-check> - Projects are stored with `forge="radicle"`, `forge_repo_id=`. treefmt-check> -- **Patch numbers.** The builds schema types `pr_number` as `int`, but treefmt-check> - radicle patch ids are git OIDs. A `radicle_patches(project_id, treefmt-check> - patch_oid, number)` table hands out a per-project sequential number treefmt-check> - on first sight; schema and queries stay untouched, users get short treefmt-check> - patch numbers, the full OID appears on the build page. treefmt-check> -- **Permissions.** No forge login exists; restart/cancel follows the treefmt-check> - existing non-forge login rules (`visibility.py`). Which patches get treefmt-check> - built is `patch_trust` (global and per repo): `delegates` (default), treefmt-check> - `followed` (authors the node follows), `all` (rely on the sandbox). treefmt-check> +- **Patch numbers.** The builds schema types `pr_number` as `int`, but radicle treefmt-check> + patch ids are git OIDs. A `radicle_patches(project_id, treefmt-check> + patch_oid, number)` treefmt-check> + table hands out a per-project sequential number on first sight; schema and treefmt-check> + queries stay untouched, users get short patch numbers, the full OID appears on treefmt-check> + the build page. treefmt-check> +- **Permissions.** No forge login exists; restart/cancel follows the existing treefmt-check> + non-forge login rules (`visibility.py`). Which patches get built is treefmt-check> + `patch_trust` (global and per repo): `delegates` (default), `followed` treefmt-check> + (authors the node follows), `all` (rely on the sandbox). treefmt-check> treefmt-check> ## 6. NixOS module treefmt-check> treefmt-check> `services.nixbot.radicle.enable` plus options mirroring `RadicleConfig`. treefmt-check> treefmt-check> -nixbot runs its **own dedicated radicle-node** with its own identity treefmt-check> -(alias "nixbot") instead of sharing an operator's seed node: job-COB treefmt-check> -runs are attributed per node identity, so results show up under the treefmt-check> -nixbot alias in desktop, and a nixbot compromise never touches the seed treefmt-check> -node's identity. Costs: a second node process and storage copy (largely treefmt-check> -neutralized by §3 alternates), and keeping "enabled in nixbot" in sync treefmt-check> -with "seeded by the node" — the module seeds the allowlist; discovery treefmt-check> -seeds the rest. treefmt-check> - treefmt-check> -The node is **publicly reachable by default** (listen address, opened treefmt-check> -firewall port, configurable `externalAddresses`), so seeds and users treefmt-check> -fetch nixbot's job COBs directly. Users must allow/follow the nixbot treefmt-check> -node identity for its COBs to replicate to them; document this in the treefmt-check> -setup guide. treefmt-check> +nixbot runs its **own dedicated radicle-node** with its own identity (alias treefmt-check> +"nixbot") instead of sharing an operator's seed node: job-COB runs are treefmt-check> +attributed per node identity, so results show up under the nixbot alias in treefmt-check> +desktop, and a nixbot compromise never touches the seed node's identity. Costs: treefmt-check> +a second node process and storage copy (largely neutralized by §3 alternates), treefmt-check> +and keeping "enabled in nixbot" in sync with "seeded by the node" — the module treefmt-check> +seeds the allowlist; discovery seeds the rest. treefmt-check> + treefmt-check> +The node is **publicly reachable by default** (listen address, opened firewall treefmt-check> +port, configurable `externalAddresses`), so seeds and users fetch nixbot's job treefmt-check> +COBs directly. Users must allow/follow the nixbot node identity for its COBs to treefmt-check> +replicate to them; document this in the setup guide. treefmt-check> treefmt-check> ## Version floor treefmt-check> treefmt-check> -radicle-job 0.6 pins the `radicle` crate at 0.24; current heartwood treefmt-check> -ships 0.25.1 (`rad` release 1.10.x). Require nixpkgs radicle-node treefmt-check> -≥ 1.10. The `Event` enum is `#[non_exhaustive]`: the JSON parser must treefmt-check> -ignore unknown event types and fields. treefmt-check> +radicle-job 0.6 pins the `radicle` crate at 0.24; current heartwood ships 0.25.1 treefmt-check> +(`rad` release 1.10.x). Require nixpkgs radicle-node ≥ 1.10. The `Event` enum is treefmt-check> +`#[non_exhaustive]`: the JSON parser must ignore unknown event types and fields. treefmt-check> treefmt-check> ## Out of scope treefmt-check> treefmt-check> - Per-attribute runs in the job COB (§4). treefmt-check> -- Patch comments: radicle-explorer cannot render job COBs yet, so treefmt-check> - web-only users see no status. An edit-in-place summary comment per treefmt-check> - patch revision (`rad patch comment`) is a follow-up. treefmt-check> +- Patch comments: radicle-explorer cannot render job COBs yet, so web-only users treefmt-check> + see no status. An edit-in-place summary comment per patch revision treefmt-check> + (`rad patch comment`) is a follow-up. treefmt-check> - Radicle-based login in the web UI. treefmt-check> - Merge gating — radicle has no required-checks concept. treefmt-check> treefmt-check> ## Risks treefmt-check> treefmt-check> -- The job-COB schema may still move (radicle-job 0.6.x); all COB I/O treefmt-check> - goes through the `rad-job` CLI, so only the CLI needs updating. treefmt-check> +- The job-COB schema may still move (radicle-job 0.6.x); all COB I/O goes treefmt-check> + through the `rad-job` CLI, so only the CLI needs updating. treefmt-check> - No event delivery guarantee across restarts; the catch-up in §2 is treefmt-check> load-bearing, not an optimization. treefmt-check> -- Patch builds evaluate attacker-controlled Nix code — same threat treefmt-check> - model as PRs, mitigated by the `delegates` default (§5) and treefmt-check> - sandbox-only execution. treefmt-check> +- Patch builds evaluate attacker-controlled Nix code — same threat model as PRs, treefmt-check> + mitigated by the `delegates` default (§5) and sandbox-only execution. error: build of '/nix/store/1gx06f5n8y3nvqkc8lva86j8ww91fk6n-treefmt-check.drv' on 'ssh-ng://customer@mac02' failed: Cannot build '/nix/store/1gx06f5n8y3nvqkc8lva86j8ww91fk6n-treefmt-check.drv'. Reason: builder failed with exit code 1. Output paths: /nix/store/23cy1dgy92w7f02i7c2gaxxzdpgp88bm-treefmt-check Last 25 log lines: > ## Out of scope > > - Per-attribute runs in the job COB (§4). > -- Patch comments: radicle-explorer cannot render job COBs yet, so > - web-only users see no status. An edit-in-place summary comment per > - patch revision (`rad patch comment`) is a follow-up. > +- Patch comments: radicle-explorer cannot render job COBs yet, so web-only users > + see no status. An edit-in-place summary comment per patch revision > + (`rad patch comment`) is a follow-up. > - Radicle-based login in the web UI. > - Merge gating — radicle has no required-checks concept. > > ## Risks > > -- The job-COB schema may still move (radicle-job 0.6.x); all COB I/O > - goes through the `rad-job` CLI, so only the CLI needs updating. > +- The job-COB schema may still move (radicle-job 0.6.x); all COB I/O goes > + through the `rad-job` CLI, so only the CLI needs updating. > - No event delivery guarantee across restarts; the catch-up in §2 is > load-bearing, not an optimization. > -- Patch builds evaluate attacker-controlled Nix code — same threat > - model as PRs, mitigated by the `delegates` default (§5) and > - sandbox-only execution. > +- Patch builds evaluate attacker-controlled Nix code — same threat model as PRs, > + mitigated by the `delegates` default (§5) and sandbox-only execution. For full logs, run: nix log /nix/store/1gx06f5n8y3nvqkc8lva86j8ww91fk6n-treefmt-check.drv error: Cannot build '/nix/store/1gx06f5n8y3nvqkc8lva86j8ww91fk6n-treefmt-check.drv'. Reason: builder failed with exit code 1. Output paths: /nix/store/23cy1dgy92w7f02i7c2gaxxzdpgp88bm-treefmt-check