tribuchet: building on jamie treefmt v2.5.0traversed 295 files emitted 246 files for processing formatted 246 files (1 changed) in 8.371s M PLAN-radicle.md diff --git a/PLAN-radicle.md b/PLAN-radicle.md index 7af7117..2b1ad28 100644 --- a/PLAN-radicle.md +++ b/PLAN-radicle.md @@ -2,200 +2,189 @@ Add Radicle as a fourth forge next to GitHub, Gitea and GitLab. -Radicle has no HTTP API, no webhooks and no commit-status endpoint. -Everything runs against a local `radicle-node` instead: +Radicle has no HTTP API, no webhooks and no commit-status endpoint. Everything +runs against a local `radicle-node` instead: - **Change source**: subscribe to the node's event stream. - **Fetching**: repos already sit on disk in the node's storage. -- **Reporting**: write "job" collaborative objects (COBs) that gossip - through the network. +- **Reporting**: write "job" collaborative objects (COBs) that gossip through + the network. -radicle-desktop renders job COBs in its commit view, patch review and -source header, so users see a ✓/✗ per commit linking to nixbot's web -frontend. +radicle-desktop renders job COBs in its commit view, patch review and source +header, so users see a ✓/✗ per commit linking to nixbot's web frontend. ## Why not a ci-broker adapter? -Radicle's CI broker (`radicle-ci-broker`) handles triggering, filtering -and COB writing, and dispatches to adapters — nixbot could be one. But -its one-shot "run adapter, get verdict" model cannot express -per-attribute builds, rerun-from-UI or effects, and it would stack a -second queue and state machine in front of nixbot's scheduler. nixbot -talks to the node directly; operators who want the broker's report -pages can run it separately. +Radicle's CI broker (`radicle-ci-broker`) handles triggering, filtering and COB +writing, and dispatches to adapters — nixbot could be one. But its one-shot "run +adapter, get verdict" model cannot express per-attribute builds, rerun-from-UI +or effects, and it would stack a second queue and state machine in front of +nixbot's scheduler. nixbot talks to the node directly; operators who want the +broker's report pages can run it separately. ## 1. Tests first (red before green) Write these tests first; §2–§6 make them pass. - **Harness**: a pytest fixture building a real radicle setup in a temp - `RAD_HOME` (`rad auth --stdin`, `rad init`, `radicle-node`), like the - nixpkgs radicle tests. All fixtures come from it, no hand-written - JSON. Adds `radicle-node` and `radicle-job` to the dev shell and - flake test inputs; tests skip when the binaries are missing. -- **Unit, events** (§2): event JSON → change mapping; patch-COB - revision extraction (head, base, author DID) from a real patch. -- **Unit, poster** (§4): status transitions → expected `rad-job` - invocations; `StatusState` mapping. -- **Idempotency** (§4): re-posting a terminal state after a nixbot - restart must not duplicate runs. -- **Integration** (§2–§5): spawn `radicle-node` in a temp `RAD_HOME`, - `rad init` a flake repo, open a patch; assert nixbot builds it, - `rad-job list` shows the runs, the central clone borrows objects via - alternates (near-empty objects dir), and clone corruption after a - storage prune recovers via re-clone. -- **NixOS VM test** (§6): three nodes (alice, a seed, nixbot), modeled - on `nixos/tests/radicle-ci-broker.nix`. + `RAD_HOME` (`rad auth --stdin`, `rad init`, `radicle-node`), like the nixpkgs + radicle tests. All fixtures come from it, no hand-written JSON. Adds + `radicle-node` and `radicle-job` to the dev shell and flake test inputs; tests + skip when the binaries are missing. +- **Unit, events** (§2): event JSON → change mapping; patch-COB revision + extraction (head, base, author DID) from a real patch. +- **Unit, poster** (§4): status transitions → expected `rad-job` invocations; + `StatusState` mapping. +- **Idempotency** (§4): re-posting a terminal state after a nixbot restart must + not duplicate runs. +- **Integration** (§2–§5): spawn `radicle-node` in a temp `RAD_HOME`, `rad init` + a flake repo, open a patch; assert nixbot builds it, `rad-job list` shows the + runs, the central clone borrows objects via alternates (near-empty objects + dir), and clone corruption after a storage prune recovers via re-clone. +- **NixOS VM test** (§6): three nodes (alice, a seed, nixbot), modeled on + `nixos/tests/radicle-ci-broker.nix`. ## 2. Change source: the node event stream -New module `nixbot/nixbot/forge/radicle.py`, enabled by a `radicle` -config section (`RadicleConfig`): +New module `nixbot/nixbot/forge/radicle.py`, enabled by a `radicle` config +section (`RadicleConfig`): -| option | default | meaning | -|-----------------|--------------|------------------------------------------| -| `radicle_home` | `~/.radicle` | control socket, storage, keys | -| `rid_allowlist` | unset | repos to build (default: all seeded) | -| `follow_nodes` | unset | NIDs whose announced repos are auto-seeded and enabled | +| option | default | meaning | +| --------------- | ------------ | ---------------------------------------------------------- | +| `radicle_home` | `~/.radicle` | control socket, storage, keys | +| `rid_allowlist` | unset | repos to build (default: all seeded) | +| `follow_nodes` | unset | NIDs whose announced repos are auto-seeded and enabled | | `patch_trust` | `delegates` | whose patches to build: `delegates` \| `followed` \| `all` | **Subscribing.** Connect to `node/control.sock` and send the JSON line -`{"command": "subscribe"}`; the node streams events as JSON lines (the -stream `rad node events` prints). Subscriptions have no server-side -expiry — the stream ends only on socket EOF or error; then reconnect -with backoff. +`{"command": "subscribe"}`; the node streams events as JSON lines (the stream +`rad node events` prints). Subscriptions have no server-side expiry — the stream +ends only on socket EOF or error; then reconnect with backoff. **Catching up.** Events missed while disconnected are gone. On every -(re)connect, compare sigrefs of all seeded repos against known heads -and process whatever moved. For patches, build only the latest revision -per patch; revisions superseded while offline are skipped, like -force-pushed-away commits on other forges. Patch close/merge does not -cancel in-flight builds. +(re)connect, compare sigrefs of all seeded repos against known heads and process +whatever moved. For patches, build only the latest revision per patch; revisions +superseded while offline are skipped, like force-pushed-away commits on other +forges. Patch close/merge does not cancel in-flight builds. **Events consumed:** -- `RefsFetched` — refs arrived from a peer. Its `updated` list uses - namespaced names (`refs/namespaces//refs/...`): - - delegate namespace, `refs/heads/` moved → default-branch - build (`head_changed`); - - `refs/cobs/xyz.radicle.patch/*` moved (any namespace) → reload the - patch COB and build its latest revision. The revision records head - and base, so the change event carries `commit_sha=`, - `base_sha=`, `branch=`, `pr_author=` and - the existing merge-head-into-base pipeline works unchanged. -- `LocalRefsAnnounced` — a local push; carries only a sigrefs pointer, - so run the catch-up for that repo. -- `SeedDiscovered` / `InventoryAnnounced` — repo lists from peers. When - the announcing NID is in `follow_nodes`, seed the RID (`rad seed`, - scope `followed`) and register the project. Radicle has no "seed - repos of followed users" policy (following only scopes ref fetching; - a node-wide allow policy would seed the whole network), so this - subscription lives in nixbot. - -**Discovery.** `rad ls` has no JSON output, so enumerate the -`storage/` directories and read each identity payload with -`rad inspect --payload ` (name, description, default branch, -visibility) into `DiscoveredRepo`: `forge="radicle"`, -`forge_repo_id=`, `clone_url=`. A private RID -becomes a private project under the existing `visibility.py` rules. +- `RefsFetched` — refs arrived from a peer. Its `updated` list uses namespaced + names (`refs/namespaces//refs/...`): + - delegate namespace, `refs/heads/` moved → default-branch build + (`head_changed`); + - `refs/cobs/xyz.radicle.patch/*` moved (any namespace) → reload the patch COB + and build its latest revision. The revision records head and base, so the + change event carries `commit_sha=`, `base_sha=`, + `branch=`, `pr_author=` and the existing + merge-head-into-base pipeline works unchanged. +- `LocalRefsAnnounced` — a local push; carries only a sigrefs pointer, so run + the catch-up for that repo. +- `SeedDiscovered` / `InventoryAnnounced` — repo lists from peers. When the + announcing NID is in `follow_nodes`, seed the RID (`rad seed`, scope + `followed`) and register the project. Radicle has no "seed repos of followed + users" policy (following only scopes ref fetching; a node-wide allow policy + would seed the whole network), so this subscription lives in nixbot. + +**Discovery.** `rad ls` has no JSON output, so enumerate the `storage/` +directories and read each identity payload with `rad inspect --payload ` +(name, description, default branch, visibility) into `DiscoveredRepo`: +`forge="radicle"`, `forge_repo_id=`, `clone_url=`. A private +RID becomes a private project under the existing `visibility.py` rules. ## 3. Fetching Nothing to authenticate: the clone URL is the local bare repo under `storage/`; the credentials provider returns empty credentials and -`git fetch` uses the path. The default branch comes from the identity -payload, not HEAD guessing. +`git fetch` uses the path. The default branch comes from the identity payload, +not HEAD guessing. **Storage sharing via alternates.** Create the central clone with -`git clone --bare --reference storage/`: it borrows objects from -the node's storage, so fetches only update refs and the clone stays -near-empty. Build merge commits land in nixbot's clone; the node's -storage is never written to, and worktrees work unchanged. If the node -prunes borrowed objects the clone corrupts — covered by gitrepo.py's -existing delete-and-re-clone recovery. Do not point worktrees at -`storage/` directly: merges and fetch locks would fight the node. +`git clone --bare --reference storage/`: it borrows objects from the node's +storage, so fetches only update refs and the clone stays near-empty. Build merge +commits land in nixbot's clone; the node's storage is never written to, and +worktrees work unchanged. If the node prunes borrowed objects the clone corrupts +— covered by gitrepo.py's existing delete-and-re-clone recovery. Do not point +worktrees at `storage/` directly: merges and fetch locks would fight the +node. ## 4. Reporting: job COBs -`RadicleStatusPoster` implements the existing `CommitStatusPoster` -protocol: +`RadicleStatusPoster` implements the existing `CommitStatusPoster` protocol: -- **pending** → create (or find) the job COB for the commit, attach a - `Started` run with the `log` URL = nixbot's build page; +- **pending** → create (or find) the job COB for the commit, attach a `Started` + run with the `log` URL = nixbot's build page; - **success** → mark the run `Succeeded`; -- **failure / error** → mark it `Failed` (`StatusState` has no - cancelled member; cancellation arrives as error); +- **failure / error** → mark it `Failed` (`StatusState` has no cancelled member; + cancellation arrives as error); - announce after each write so seeds pick it up. -COB writes shell out to the `rad-job` CLI (`new`, `run`, `succeeded`, -`failed`, `list`), which signs with the profile key in `radicle_home`. -`rad-job run` generates the run UUID and prints it; `succeeded`/`failed` -take commit oid + UUID, so run UUIDs are persisted in the DB the way -`CheckRunStore` persists GitHub check-run ids. +COB writes shell out to the `rad-job` CLI (`new`, `run`, `succeeded`, `failed`, +`list`), which signs with the profile key in `radicle_home`. `rad-job run` +generates the run UUID and prints it; `succeeded`/`failed` take commit oid + +UUID, so run UUIDs are persisted in the DB the way `CheckRunStore` persists +GitHub check-run ids. -**Granularity.** One run per commit: an aggregate verdict plus a link, -like the single "nixbot" status context on Gitea/GitLab. One run per -check attribute is deferred: runs have no name field and desktop labels -rows by the log URL, so dozens of near-identical labels may be -unreadable. Validate in desktop before adopting. +**Granularity.** One run per commit: an aggregate verdict plus a link, like the +single "nixbot" status context on Gitea/GitLab. One run per check attribute is +deferred: runs have no name field and desktop labels rows by the log URL, so +dozens of near-identical labels may be unreadable. Validate in desktop before +adopting. ## 5. Wiring into the service -- `service.py` builds the event source, credentials provider and status - poster when `config.radicle` is set, and registers `"radicle"` in the - forge dispatch tables. +- `service.py` builds the event source, credentials provider and status poster + when `config.radicle` is set, and registers `"radicle"` in the forge dispatch + tables. - Projects are stored with `forge="radicle"`, `forge_repo_id=`. -- **Patch numbers.** The builds schema types `pr_number` as `int`, but - radicle patch ids are git OIDs. A `radicle_patches(project_id, - patch_oid, number)` table hands out a per-project sequential number - on first sight; schema and queries stay untouched, users get short - patch numbers, the full OID appears on the build page. -- **Permissions.** No forge login exists; restart/cancel follows the - existing non-forge login rules (`visibility.py`). Which patches get - built is `patch_trust` (global and per repo): `delegates` (default), - `followed` (authors the node follows), `all` (rely on the sandbox). +- **Patch numbers.** The builds schema types `pr_number` as `int`, but radicle + patch ids are git OIDs. A `radicle_patches(project_id, + patch_oid, number)` + table hands out a per-project sequential number on first sight; schema and + queries stay untouched, users get short patch numbers, the full OID appears on + the build page. +- **Permissions.** No forge login exists; restart/cancel follows the existing + non-forge login rules (`visibility.py`). Which patches get built is + `patch_trust` (global and per repo): `delegates` (default), `followed` + (authors the node follows), `all` (rely on the sandbox). ## 6. NixOS module `services.nixbot.radicle.enable` plus options mirroring `RadicleConfig`. -nixbot runs its **own dedicated radicle-node** with its own identity -(alias "nixbot") instead of sharing an operator's seed node: job-COB -runs are attributed per node identity, so results show up under the -nixbot alias in desktop, and a nixbot compromise never touches the seed -node's identity. Costs: a second node process and storage copy (largely -neutralized by §3 alternates), and keeping "enabled in nixbot" in sync -with "seeded by the node" — the module seeds the allowlist; discovery -seeds the rest. - -The node is **publicly reachable by default** (listen address, opened -firewall port, configurable `externalAddresses`), so seeds and users -fetch nixbot's job COBs directly. Users must allow/follow the nixbot -node identity for its COBs to replicate to them; document this in the -setup guide. +nixbot runs its **own dedicated radicle-node** with its own identity (alias +"nixbot") instead of sharing an operator's seed node: job-COB runs are +attributed per node identity, so results show up under the nixbot alias in +desktop, and a nixbot compromise never touches the seed node's identity. Costs: +a second node process and storage copy (largely neutralized by §3 alternates), +and keeping "enabled in nixbot" in sync with "seeded by the node" — the module +seeds the allowlist; discovery seeds the rest. + +The node is **publicly reachable by default** (listen address, opened firewall +port, configurable `externalAddresses`), so seeds and users fetch nixbot's job +COBs directly. Users must allow/follow the nixbot node identity for its COBs to +replicate to them; document this in the setup guide. ## Version floor -radicle-job 0.6 pins the `radicle` crate at 0.24; current heartwood -ships 0.25.1 (`rad` release 1.10.x). Require nixpkgs radicle-node -≥ 1.10. The `Event` enum is `#[non_exhaustive]`: the JSON parser must -ignore unknown event types and fields. +radicle-job 0.6 pins the `radicle` crate at 0.24; current heartwood ships 0.25.1 +(`rad` release 1.10.x). Require nixpkgs radicle-node ≥ 1.10. The `Event` enum is +`#[non_exhaustive]`: the JSON parser must ignore unknown event types and fields. ## Out of scope - Per-attribute runs in the job COB (§4). -- Patch comments: radicle-explorer cannot render job COBs yet, so - web-only users see no status. An edit-in-place summary comment per - patch revision (`rad patch comment`) is a follow-up. +- Patch comments: radicle-explorer cannot render job COBs yet, so web-only users + see no status. An edit-in-place summary comment per patch revision + (`rad patch comment`) is a follow-up. - Radicle-based login in the web UI. - Merge gating — radicle has no required-checks concept. ## Risks -- The job-COB schema may still move (radicle-job 0.6.x); all COB I/O - goes through the `rad-job` CLI, so only the CLI needs updating. +- The job-COB schema may still move (radicle-job 0.6.x); all COB I/O goes + through the `rad-job` CLI, so only the CLI needs updating. - No event delivery guarantee across restarts; the catch-up in §2 is load-bearing, not an optimization. -- Patch builds evaluate attacker-controlled Nix code — same threat - model as PRs, mitigated by the `delegates` default (§5) and - sandbox-only execution. +- Patch builds evaluate attacker-controlled Nix code — same threat model as PRs, + mitigated by the `delegates` default (§5) and sandbox-only execution.