tribuchet: building on jamie
Machine state will be reset. To keep it, pass --keep-machine-state
start all VLans
(finished: start all VLans, in 0.00 seconds)
Initializing machine ID from random generator.
Test will time out and terminate in 3600.0 seconds
run the VM test script
start all VMs
agent: systemd-nspawn running (pid 52)
relay: systemd-nspawn running (pid 54)
client: systemd-nspawn running (pid 53)
agent: Waiting for journal at /build/vm-state-agent/var/log/journal...
relay: Waiting for journal at /build/vm-state-relay/var/log/journal...
client: Waiting for journal at /build/vm-state-client/var/log/journal...
(finished: start all VMs, in 0.00 seconds)
relay: waiting for unit flakelet-relay.service
nixos-nspawn(relay): TAP vde-tap1 not found; container will be isolated from VDE
nixos-nspawn(relay): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.
nixos-nspawn(client): TAP vde-tap1 not found; container will be isolated from VDE
nixos-nspawn(client): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.
nixos-nspawn(agent): TAP vde-tap1 not found; container will be isolated from VDE
nixos-nspawn(agent): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.
Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.
Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.
░ Spawning container relay on /build/vm-state-relay.
Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.
░ Spawning container agent on /build/vm-state-agent.
░ Spawning container client on /build/vm-state-client.
relay # No journal files were found.
relay # No journal boot entry found for the specified boot (+0).
agent # No journal files were found.
agent # No journal boot entry found for the specified boot (+0).
client # No journal files were found.
client # No journal boot entry found for the specified boot (+0).
relay # [ 9602.610118] relay systemd-journald[57]: Journal started
relay # [ 9602.610158] relay systemd-journald[57]: Runtime Journal (/run/log/journal/248dcf733e5e4021bfbb9db19395b461) is 8M, max 4G, 3.9G free.
relay # [ 9602.621758] relay systemd[1]: Finished Apply Kernel Variables.
relay # [ 9602.633442] relay systemd[1]: Finished Create Static Device Nodes in /dev gracefully.
relay # [ 9602.647141] relay systemd[1]: Starting Flush Journal to Persistent Storage...
relay # [ 9602.647948] relay systemd[1]: Starting Create Static Device Nodes in /dev...
relay # [ 9602.687314] relay systemd-journald[57]: Time spent on flushing to /var/log/journal/248dcf733e5e4021bfbb9db19395b461 is 1.365ms for 6 entries.
agent # [ 9602.616034] agent systemd-journald[57]: Journal started
relay # [ 9602.687314] relay systemd-journald[57]: System Journal (/var/log/journal/248dcf733e5e4021bfbb9db19395b461) is 512B, max 4G, 3.9G free.
agent # [ 9602.616075] agent systemd-journald[57]: Runtime Journal (/run/log/journal/76045c452adf408887329d35b7f2c388) is 8M, max 4G, 3.9G free.
relay # [ 9602.695495] relay systemd[1]: Finished Create Static Device Nodes in /dev.
agent # [ 9602.626408] agent systemd[1]: Finished Apply Kernel Variables.
client # [ 9602.610676] client systemd-journald[57]: Journal started
agent # [ 9602.638157] agent systemd[1]: Finished Create Static Device Nodes in /dev gracefully.
client # [ 9602.610716] client systemd-journald[57]: Runtime Journal (/run/log/journal/6fda3615315145978b328a9b7132a79c) is 8M, max 4G, 3.9G free.
agent # [ 9602.681533] agent systemd[1]: Starting Flush Journal to Persistent Storage...
client # [ 9602.621023] client systemd[1]: Finished Apply Kernel Variables.
agent # [ 9602.682264] agent systemd[1]: Starting Create Static Device Nodes in /dev...
client # [ 9602.632673] client systemd[1]: Finished Create Static Device Nodes in /dev gracefully.
relay # [ 9602.696239] relay systemd[1]: Reached target Preparation for Local File Systems.
client # [ 9602.646406] client systemd[1]: Starting Flush Journal to Persistent Storage...
relay # [ 9602.696353] relay systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys
client # [ 9602.647113] client systemd[1]: Starting Create Static Device Nodes in /dev...
agent # [ 9602.690919] agent systemd-journald[57]: Time spent on flushing to /var/log/journal/76045c452adf408887329d35b7f2c388 is 2.576ms for 6 entries.
relay # [ 9602.710804] relay systemd[1]: Finished Flush Journal to Persistent Storage.
agent # [ 9602.690919] agent systemd-journald[57]: System Journal (/var/log/journal/76045c452adf408887329d35b7f2c388) is 512B, max 4G, 3.9G free.
client # [ 9602.686559] client systemd-journald[57]: Time spent on flushing to /var/log/journal/6fda3615315145978b328a9b7132a79c is 1.352ms for 6 entries.
relay # [ 9602.781445] relay systemd[1]: Finished Firewall.
client # [ 9602.686559] client systemd-journald[57]: System Journal (/var/log/journal/6fda3615315145978b328a9b7132a79c) is 512B, max 4G, 3.9G free.
relay # [ 9603.609591] relay systemd[1]: Mounting /run/wrappers...
agent # [ 9602.699822] agent systemd[1]: Finished Create Static Device Nodes in /dev.
client # [ 9602.695742] client systemd[1]: Finished Create Static Device Nodes in /dev.
agent # [ 9602.700505] agent systemd[1]: Reached target Preparation for Local File Systems.
client # [ 9602.696369] client systemd[1]: Reached target Preparation for Local File Systems.
relay # [ 9603.670252] relay systemd[1]: Mounted /run/wrappers.
client # [ 9602.696469] client systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys
agent # [ 9602.700602] agent systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys
relay # [ 9603.671301] relay systemd[1]: Reached target Local File Systems.
agent # [ 9602.717240] agent systemd[1]: Finished Flush Journal to Persistent Storage.
client # [ 9602.707708] client systemd[1]: Finished Flush Journal to Persistent Storage.
relay # [ 9603.672565] relay systemd[1]: Listening on Boot Loader Control Service Socket.
client # [ 9602.762678] client systemd[1]: Finished Firewall.
agent # [ 9602.779593] agent systemd[1]: Finished Firewall.
relay # [ 9603.674153] relay systemd[1]: Starting Create SUID/SGID Wrappers...
agent # [ 9603.638734] agent systemd[1]: Mounting /run/wrappers...
relay # [ 9603.674200] relay systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container
agent # [ 9603.669815] agent systemd[1]: Mounted /run/wrappers.
client # [ 9603.638746] client systemd[1]: Mounting /run/wrappers...
relay # [ 9603.675419] relay systemd[1]: Starting Save Transient machine-id to Disk...
client # [ 9603.670111] client systemd[1]: Mounted /run/wrappers.
agent # [ 9603.670856] agent systemd[1]: Reached target Local File Systems.
relay # [ 9603.676549] relay systemd[1]: Starting Create System Files and Directories...
agent # [ 9603.672150] agent systemd[1]: Listening on Boot Loader Control Service Socket.
client # [ 9603.671131] client systemd[1]: Reached target Local File Systems.
agent # [ 9603.673765] agent systemd[1]: Starting Create SUID/SGID Wrappers...
client # [ 9603.672384] client systemd[1]: Listening on Boot Loader Control Service Socket.
agent # [ 9603.673813] agent systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container
client # [ 9603.673660] client systemd[1]: Starting Create SUID/SGID Wrappers...
agent # [ 9603.674997] agent systemd[1]: Starting Save Transient machine-id to Disk...
client # [ 9603.673705] client systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container
agent # [ 9603.676423] agent systemd[1]: Starting Create System Files and Directories...
client # [ 9603.675232] client systemd[1]: Starting Save Transient machine-id to Disk...
client # [ 9603.676108] client systemd[1]: Starting Create System Files and Directories...
client # [ 9603.694199] client systemd-tmpfiles[161]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted
client # [ 9603.694651] client systemd-tmpfiles[161]: fchmod() of /var/log/journal failed: Operation not permitted
client # [ 9603.694986] client systemd-tmpfiles[161]: fchmod() of /var/log/journal/6fda3615315145978b328a9b7132a79c failed: Operation not permitted
client # [ 9603.695348] client systemd-tmpfiles[161]: fchmod() of /run/log/journal failed: Operation not permitted
relay # [ 9603.695481] relay systemd-tmpfiles[169]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted
relay # [ 9603.695901] relay systemd-tmpfiles[169]: fchmod() of /var/log/journal failed: Operation not permitted
agent # [ 9603.695681] agent systemd-tmpfiles[161]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted
relay # [ 9603.696231] relay systemd-tmpfiles[169]: fchmod() of /var/log/journal/248dcf733e5e4021bfbb9db19395b461 failed: Operation not permitted
agent # [ 9603.696105] agent systemd-tmpfiles[161]: fchmod() of /var/log/journal failed: Operation not permitted
agent # [ 9603.696427] agent systemd-tmpfiles[161]: fchmod() of /var/log/journal/76045c452adf408887329d35b7f2c388 failed: Operation not permitted
relay # [ 9603.696558] relay systemd-tmpfiles[169]: fchmod() of /run/log/journal failed: Operation not permitted
agent # [ 9603.696748] agent systemd-tmpfiles[161]: fchmod() of /run/log/journal failed: Operation not permitted
relay # [ 9603.697704] relay suid-sgid-wrappers-start[176]: chmod: changing permissions of '/run/wrappers/wrappers.Pr2sEjtRRY/chsh': Operation not permitted
relay # [ 9603.703128] relay systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE
relay # [ 9603.703168] relay systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.
relay # [ 9603.703228] relay systemd[1]: Failed to start Create SUID/SGID Wrappers.
relay # [ 9603.733739] relay systemd[1]: Finished Create System Files and Directories.
relay # [ 9603.761395] relay systemd[1]: etc-machine\x2did.mount: Deactivated successfully.
relay # [ 9603.774161] relay systemd[1]: Finished Save Transient machine-id to Disk.
relay # [ 9603.777209] relay systemd[1]: Starting Rebuild Journal Catalog...
relay # [ 9603.778032] relay systemd[1]: Starting Record System Boot/Shutdown in UTMP...
relay # [ 9603.818682] relay systemd[1]: Finished Record System Boot/Shutdown in UTMP.
relay # [ 9603.826188] relay systemd[1]: Finished Rebuild Journal Catalog.
relay # [ 9603.827248] relay systemd[1]: Starting Update is Completed...
relay # [ 9603.835825] relay systemd[1]: Finished Update is Completed.
relay # [ 9603.835902] relay systemd[1]: Reached target System Initialization.
relay # [ 9603.835962] relay systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container
relay # [ 9603.835981] relay systemd[1]: Started Daily Cleanup of Temporary Directories.
relay # [ 9603.835995] relay systemd[1]: Reached target Timer Units.
relay # [ 9603.836093] relay systemd[1]: Listening on D-Bus System Message Bus Socket.
relay # [ 9603.836270] relay systemd[1]: Listening on Nix Daemon Socket.
relay # [ 9603.836360] relay systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.
relay # [ 9603.836374] relay systemd[1]: Reached target Socket Units.
relay # [ 9603.836399] relay systemd[1]: Reached target Basic System.
relay # [ 9603.837388] relay systemd[1]: Started device-mock.service.
relay # [ 9603.838161] relay systemd[1]: Starting Re-link flakelet services at boot...
relay # [ 9603.838809] relay systemd[1]: Starting Import lastlog data into lastlog2 database...
relay # [ 9603.839587] relay systemd[1]: Starting Name Service Cache Daemon (nsncd)...
relay # [ 9603.840186] relay systemd[1]: Starting resolvconf update...
relay # [ 9603.849180] relay systemd[1]: Finished Re-link flakelet services at boot.
relay # [ 9603.850595] relay systemd[1]: Starting Reconcile flakelet services with the host configuration...
relay # [ 9603.856414] relay systemd[1]: Finished Import lastlog data into lastlog2 database.
agent # [ 9603.699231] agent suid-sgid-wrappers-start[168]: chmod: changing permissions of '/run/wrappers/wrappers.coaPTLOMMz/chsh': Operation not permitted
agent # [ 9603.701061] agent systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE
agent # [ 9603.701102] agent systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.
agent # [ 9603.701147] agent systemd[1]: Failed to start Create SUID/SGID Wrappers.
agent # [ 9603.733476] agent systemd[1]: Finished Create System Files and Directories.
agent # [ 9603.761836] agent systemd[1]: etc-machine\x2did.mount: Deactivated successfully.
agent # [ 9603.774383] agent systemd[1]: Finished Save Transient machine-id to Disk.
agent # [ 9603.777902] agent systemd[1]: Starting Rebuild Journal Catalog...
agent # [ 9603.778529] agent systemd[1]: Starting Record System Boot/Shutdown in UTMP...
agent # [ 9603.819061] agent systemd[1]: Finished Record System Boot/Shutdown in UTMP.
agent # [ 9603.826320] agent systemd[1]: Finished Rebuild Journal Catalog.
agent # [ 9603.827375] agent systemd[1]: Starting Update is Completed...
agent # [ 9603.837757] agent systemd[1]: Finished Update is Completed.
agent # [ 9603.837896] agent systemd[1]: Reached target System Initialization.
agent # [ 9603.837966] agent systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container
agent # [ 9603.837988] agent systemd[1]: Started Daily Cleanup of Temporary Directories.
agent # [ 9603.838010] agent systemd[1]: Reached target Timer Units.
agent # [ 9603.838103] agent systemd[1]: Listening on D-Bus System Message Bus Socket.
agent # [ 9603.838310] agent systemd[1]: Listening on Nix Daemon Socket.
agent # [ 9603.838413] agent systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.
agent # [ 9603.838426] agent systemd[1]: Reached target Socket Units.
agent # [ 9603.838454] agent systemd[1]: Reached target Basic System.
agent # [ 9603.839565] agent systemd[1]: Starting Re-link flakelet services at boot...
agent # [ 9603.840397] agent systemd[1]: Starting Import lastlog data into lastlog2 database...
agent # [ 9603.841140] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...
agent # [ 9603.841735] agent systemd[1]: Starting resolvconf update...
agent # [ 9603.852176] agent systemd[1]: Finished Re-link flakelet services at boot.
agent # [ 9603.853133] agent systemd[1]: Starting Reconcile flakelet services with the host configuration...
agent # [ 9603.858059] agent systemd[1]: Finished Import lastlog data into lastlog2 database.
agent # [ 9603.866420] agent systemd[1]: Finished Reconcile flakelet services with the host configuration.
client # [ 9603.695737] client suid-sgid-wrappers-start[168]: chmod: changing permissions of '/run/wrappers/wrappers.ZPV4x77r3M/chsh': Operation not permitted
client # [ 9603.700455] client systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE
client # [ 9603.700552] client systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.
client # [ 9603.700744] client systemd[1]: Failed to start Create SUID/SGID Wrappers.
client # [ 9603.733175] client systemd[1]: Finished Create System Files and Directories.
client # [ 9603.761410] client systemd[1]: etc-machine\x2did.mount: Deactivated successfully.
client # [ 9603.774150] client systemd[1]: Finished Save Transient machine-id to Disk.
client # [ 9603.776955] client systemd[1]: Starting Rebuild Journal Catalog...
client # [ 9603.777831] client systemd[1]: Starting Record System Boot/Shutdown in UTMP...
client # [ 9603.819023] client systemd[1]: Finished Record System Boot/Shutdown in UTMP.
client # [ 9603.825110] client systemd[1]: Finished Rebuild Journal Catalog.
client # [ 9603.826080] client systemd[1]: Starting Update is Completed...
client # [ 9603.834600] client systemd[1]: Finished Update is Completed.
client # [ 9603.834669] client systemd[1]: Reached target System Initialization.
client # [ 9603.834733] client systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container
client # [ 9603.834751] client systemd[1]: Started Daily Cleanup of Temporary Directories.
client # [ 9603.834764] client systemd[1]: Reached target Timer Units.
client # [ 9603.834853] client systemd[1]: Listening on D-Bus System Message Bus Socket.
client # [ 9603.835057] client systemd[1]: Listening on Nix Daemon Socket.
client # [ 9603.835146] client systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.
client # [ 9603.835157] client systemd[1]: Reached target Socket Units.
client # [ 9603.835183] client systemd[1]: Reached target Basic System.
client # [ 9603.836399] client systemd[1]: Starting Import lastlog data into lastlog2 database...
client # [ 9603.837198] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...
client # [ 9603.837970] client systemd[1]: Starting resolvconf update...
client # [ 9603.858944] client systemd[1]: Finished Import lastlog data into lastlog2 database.
relay # [ 9603.862834] relay systemd[1]: Finished Reconcile flakelet services with the host configuration.
relay # [ 9603.921846] relay systemd[1]: nscd.service: Deactivated successfully.
relay # [ 9603.921964] relay systemd[1]: Stopped Name Service Cache Daemon (nsncd).
relay # [ 9603.924357] relay systemd[1]: Starting Name Service Cache Daemon (nsncd)...
relay # [ 9603.959227] relay systemd[1]: Finished resolvconf update.
relay # [ 9603.959403] relay systemd[1]: Reached target Preparation for Network.
relay # [ 9603.960448] relay systemd[1]: Starting Address configuration of eth1...
relay # [ 9603.961373] relay systemd[1]: Starting Extra networking commands....
relay # [ 9603.979995] relay network-addresses-eth1-start[251]: adding address 192.168.1.3/24... done
relay # [ 9603.982660] relay network-addresses-eth1-start[251]: adding address 2001:db8:1::3/64... done
relay # [ 9603.986796] relay systemd[1]: Finished Address configuration of eth1.
relay # [ 9604.027856] relay systemd[1]: Finished Extra networking commands..
relay # [ 9604.027939] relay systemd[1]: Reached target Network.
relay # [ 9604.029306] relay systemd[1]: Starting Dnsmasq Daemon...
relay # [ 9604.030139] relay systemd[1]: Starting Nginx Web Server...
relay # [ 9604.030885] relay systemd[1]: Starting Update flakelet service flakelet-relay...
relay # [ 9604.079313] relay flakelet[311]: flakelet-relay: using prebuilt artifact /nix/store/j272yq4kj97x5q0nggal747653288wg4-flakelet-flakelet-relay
relay # [ 9604.079957] relay flakelet[311]: flakelet-relay: activating generation 1
relay # [ 9604.084159] relay systemd[1]: Reload requested from client PID 316 ('systemctl') (unit flakelet-flakelet-relay.service)...
relay # [ 9604.084205] relay systemd[1]: Reloading...
agent # [ 9603.918072] agent systemd[1]: nscd.service: Deactivated successfully.
agent # [ 9603.921628] agent systemd[1]: Stopped Name Service Cache Daemon (nsncd).
agent # [ 9603.924183] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...
agent # [ 9603.959220] agent systemd[1]: Finished resolvconf update.
agent # [ 9603.959470] agent systemd[1]: Reached target Preparation for Network.
agent # [ 9603.960517] agent systemd[1]: Starting Address configuration of eth1...
agent # [ 9603.961320] agent systemd[1]: Starting Extra networking commands....
agent # [ 9603.982950] agent network-addresses-eth1-start[242]: adding address 192.168.1.1/24... done
agent # [ 9603.986513] agent network-addresses-eth1-start[242]: adding address 2001:db8:1::1/64... done
agent # [ 9603.991163] agent systemd[1]: Finished Address configuration of eth1.
agent # [ 9604.071372] agent systemd[1]: Finished Extra networking commands..
agent # [ 9604.071633] agent systemd[1]: nscd.service: Deactivated successfully.
agent # [ 9604.071778] agent systemd[1]: Stopped Name Service Cache Daemon (nsncd).
agent # [ 9604.073975] agent systemd[1]: Reached target Network.
agent # [ 9604.075158] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...
agent # [ 9604.075967] agent systemd[1]: Starting Update flakelet service app...
agent # [ 9604.076917] agent systemd[1]: Starting Update flakelet service flakelet-agent...
agent # [ 9604.077742] agent systemd[1]: Starting Update flakelet service other...
agent # [ 9604.088011] agent flakelet[308]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1
agent # [ 9604.088315] agent flakelet[309]: flakelet-agent: using prebuilt artifact /nix/store/j44k16n69mi2k45ccgnkdvbqapv6qb5n-flakelet-flakelet-agent
agent # [ 9604.088620] agent flakelet[308]: app: activating generation 1
agent # [ 9604.088783] agent flakelet[309]: flakelet-agent: activating generation 1
agent # [ 9604.088806] agent flakelet[310]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1
agent # [ 9604.089319] agent flakelet[310]: other: activating generation 1
agent # [ 9604.093500] agent systemd[1]: Reload requested from client PID 312 ('systemctl') (unit flakelet-app.service)...
agent # [ 9604.093537] agent systemd[1]: Reloading...
client # [ 9603.930999] client systemd[1]: nscd.service: Deactivated successfully.
client # [ 9603.959218] client systemd[1]: Stopped Name Service Cache Daemon (nsncd).
client # [ 9603.960684] client systemd[1]: Finished resolvconf update.
client # [ 9603.961371] client systemd[1]: Reached target Preparation for Network.
client # [ 9603.962516] client systemd[1]: Starting Address configuration of eth1...
client # [ 9603.963264] client systemd[1]: Starting Extra networking commands....
client # [ 9603.964066] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...
client # [ 9603.983228] client network-addresses-eth1-start[236]: adding address 192.168.1.2/24... done
client # [ 9603.986035] client network-addresses-eth1-start[236]: adding address 2001:db8:1::2/64... done
client # [ 9603.989430] client systemd[1]: Finished Address configuration of eth1.
client # [ 9604.071362] client systemd[1]: Finished Extra networking commands..
client # [ 9604.071623] client systemd[1]: nscd.service: Deactivated successfully.
client # [ 9604.071754] client systemd[1]: Stopped Name Service Cache Daemon (nsncd).
client # [ 9604.073787] client systemd[1]: Reached target Network.
client # [ 9604.074912] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...
relay # [ 9604.414299] relay nsncd[246]: Sep 06 09:44:42.376 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"
relay # [ 9604.510585] relay systemd[1]: Reloading finished in 425 ms.
relay # [ 9604.635671] relay systemd[1]: Started Name Service Cache Daemon (nsncd).
relay # [ 9604.636133] relay systemd[1]: Reached target Host and Network Name Lookups.
relay # [ 9604.636219] relay systemd[1]: Reached target User and Group Name Lookups.
relay # [ 9604.637577] relay systemd[1]: Starting User Login Management...
relay # [ 9604.638410] relay systemd[1]: Starting Permit User Sessions...
relay # [ 9604.656364] relay systemd[1]: Reload requested from client PID 357 ('systemctl') (unit flakelet-flakelet-relay.service)...
relay # [ 9604.656428] relay systemd[1]: Reloading...
agent # [ 9604.418848] agent nsncd[307]: Sep 06 09:44:42.380 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"
agent # [ 9604.506421] agent systemd[1]: Reloading finished in 412 ms.
agent # [ 9604.638600] agent systemd[1]: Started Name Service Cache Daemon (nsncd).
agent # [ 9604.639166] agent systemd[1]: Reload requested from client PID 313 ('systemctl') (unit flakelet-flakelet-agent.service)...
agent # [ 9604.639225] agent systemd[1]: Reloading...
client # [ 9604.397042] client nsncd[303]: Sep 06 09:44:42.358 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"
client # [ 9604.397746] client systemd[1]: Started Name Service Cache Daemon (nsncd).
client # [ 9604.397836] client systemd[1]: Reached target Host and Network Name Lookups.
client # [ 9604.398231] client systemd[1]: Reached target User and Group Name Lookups.
client # [ 9604.408141] client systemd[1]: Starting User Login Management...
client # [ 9604.411657] client systemd[1]: Starting Permit User Sessions...
client # [ 9604.451043] client systemd[1]: Finished Permit User Sessions.
client # [ 9604.452721] client systemd[1]: Started Console Getty.
client # [ 9604.452764] client systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0
client # [ 9604.452784] client systemd[1]: Reached target Login Prompts.
relay # [ 9604.682213] relay dnsmasq-pre-start[366]: dnsmasq: syntax check OK.
agent # [ 9605.003751] agent systemd[1]: Reloading finished in 364 ms.
agent # [ 9605.134102] agent systemd[1]: Reload requested from client PID 314 ('systemctl') (unit flakelet-other.service)...
agent # [ 9605.134155] agent systemd[1]: Reloading...
relay # [ 9604.998725] relay systemd[1]: Reloading finished in 341 ms.
relay # [ 9605.131965] relay systemd[1]: Finished Permit User Sessions.
relay # [ 9605.136396] relay systemd[1]: Started Console Getty.
relay # [ 9605.136459] relay systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0
relay # [ 9605.136528] relay systemd[1]: Reached target Login Prompts.
relay # [ 9605.153269] relay systemd[1]: Starting flakelet-relay...
agent # [ 9605.488225] agent systemd[1]: Reloading finished in 353 ms.
agent # [ 9605.569219] agent systemd[1]: Reached target Host and Network Name Lookups.
agent # [ 9605.569356] agent systemd[1]: Reached target User and Group Name Lookups.
agent # [ 9605.570721] agent systemd[1]: Starting User Login Management...
agent # [ 9605.571816] agent systemd[1]: Starting Permit User Sessions...
agent # [ 9605.575832] agent systemd[1]: Reload requested from client PID 406 ('systemctl') (unit flakelet-app.service)...
agent # [ 9605.575862] agent systemd[1]: Reloading...
relay # [ 9605.591220] relay nginx-pre-start[402]: nginx: the configuration file /nix/store/01avjj26wcihp926xgasszcj5s0k7cbs-nginx.conf syntax is ok
relay # [ 9605.591964] relay nginx-pre-start[402]: nginx: configuration file /nix/store/01avjj26wcihp926xgasszcj5s0k7cbs-nginx.conf test is successful
relay # [ 9605.609707] relay systemd[1]: Started Nginx Web Server.
relay # [ 9605.808784] relay systemd[1]: Starting D-Bus System Message Bus...
client # [ 9605.744854] client systemd-logind[315]: New seat seat0.
client # [ 9605.745350] client systemd[1]: Started User Login Management.
client # [ 9605.747595] client systemd[1]: Starting D-Bus System Message Bus...
client # [ 9605.748889] client systemd[1]: Starting linger-users.service...
client # [ 9605.792153] client systemd[1]: linger-users.service: Deactivated successfully.
client # [ 9605.792507] client systemd[1]: Finished linger-users.service.
client # [ 9605.792844] client systemd[1]: Reached target Multi-User System.
agent # [ 9605.968678] agent systemd[1]: Reloading finished in 392 ms.
agent # [ 9606.043375] agent systemd[1]: Finished Permit User Sessions.
agent # [ 9606.049232] agent systemd[1]: Started Console Getty.
agent # [ 9606.049346] agent systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0
agent # [ 9606.049499] agent systemd[1]: Reached target Login Prompts.
agent # [ 9606.051850] agent systemd[1]: Reload requested from client PID 408 ('systemctl') (unit flakelet-flakelet-agent.service)...
agent # [ 9606.051881] agent systemd[1]: Reloading...
relay # [ 9606.063697] relay systemd-logind[354]: New seat seat0.
relay # [ 9606.064125] relay systemd[1]: Started User Login Management.
relay # [ 9606.066634] relay systemd[1]: Starting linger-users.service...
relay # [ 9606.118124] relay systemd[1]: linger-users.service: Deactivated successfully.
relay # [ 9606.118323] relay systemd[1]: Finished linger-users.service.
client # [ 9606.299528] client dbus-broker-launch[325]: Looking up NSS user entry for 'systemd-timesync'...
client # [ 9606.300887] client dbus-broker-launch[325]: NSS returned no entry for 'systemd-timesync'
client # [ 9606.300887] client dbus-broker-launch[325]: Invalid user-name in /nix/store/5cfppnr9iv53mjjvcafddv4813rd7x9c-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"
client # [ 9606.301809] client systemd[1]: Started D-Bus System Message Bus.
client # [ 9606.302355] client systemd[1]: Startup finished in 4.119s.
client # [ 9606.308524] client dbus-broker-launch[325]: Ready
relay # [ 9606.324608] relay dbus-broker-launch[408]: Looking up NSS user entry for 'systemd-timesync'...
relay # [ 9606.325816] relay dbus-broker-launch[408]: NSS returned no entry for 'systemd-timesync'
relay # [ 9606.325816] relay dbus-broker-launch[408]: Invalid user-name in /nix/store/27h4g2jdw18vrabfh6p1l0n8ss5dhbp5-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"
relay # [ 9606.326871] relay systemd[1]: Started D-Bus System Message Bus.
relay # [ 9606.332785] relay dbus-broker-launch[408]: Ready
relay # [ 9606.336320] relay systemd[1]: Started Dnsmasq Daemon.
relay # [ 9606.337903] relay dnsmasq[394]: started, version 2.93 cachesize 150
relay # [ 9606.337910] relay dnsmasq[394]: compile time options: IPv6 GNU-getopt DBus no-UBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset nftset auth DNSSEC loop-detect inotify dumpfile
relay # [ 9606.337916] relay dnsmasq[394]: DBus support enabled: connected to system bus
relay # [ 9606.337923] relay dnsmasq[394]: no upstream servers configured - please set them from DBus
relay # [ 9606.338012] relay dnsmasq[394]: read /etc/hosts - 9 names
agent # [ 9606.399636] agent systemd[1]: Reloading finished in 347 ms.
agent # [ 9606.485175] agent systemd[1]: Reload requested from client PID 444 ('systemctl') (unit flakelet-other.service)...
agent # [ 9606.485219] agent systemd[1]: Reloading...
agent # [ 9606.830030] agent systemd[1]: Reloading finished in 344 ms.
agent # [ 9606.915740] agent systemd[1]: Starting app v1...
agent # [ 9606.925467] agent systemd[1]: Starting other v1...
agent # [ 9606.948163] agent systemd[1]: Starting flakelet-agent...
agent # [ 9606.951460] agent systemd[1]: Started app v1.
agent # [ 9606.951673] agent systemd[1]: Started other v1.
agent # [ 9606.979964] agent flakelet[308]: app: updated to generation 1
agent # [ 9606.980550] agent systemd[1]: Finished Update flakelet service app.
agent # [ 9606.984056] agent flakelet[310]: other: updated to generation 1
agent # [ 9606.985840] agent systemd[1]: Finished Update flakelet service other.
agent # [ 9606.992233] agent systemd-logind[405]: New seat seat0.
agent # [ 9606.993154] agent systemd[1]: Starting D-Bus System Message Bus...
agent # [ 9606.993272] agent systemd[1]: Started User Login Management.
agent # [ 9606.995284] agent systemd[1]: Starting linger-users.service...
agent # [ 9607.041629] agent systemd[1]: Started flakelet-agent.
agent # [ 9607.042369] agent systemd[1]: linger-users.service: Deactivated successfully.
agent # [ 9607.042492] agent systemd[1]: Finished linger-users.service.
agent # [ 9607.045449] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:45.007143Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}
agent # [ 9607.058546] agent flakelet[309]: flakelet-agent: updated to generation 1
agent # [ 9607.058998] agent systemd[1]: Finished Update flakelet service flakelet-agent.
agent # [ 9607.059921] agent systemd[1]: Reached target flakelet managed services.
agent # [ 9607.059990] agent systemd[1]: Reached target Multi-User System.
relay # [ 9607.228364] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:45.190000Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}
relay # [ 9607.228879] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:45.190638Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}
relay # [ 9607.228982] relay systemd[1]: Started flakelet-relay.
relay # [ 9607.238917] relay systemd[1]: Starting health for flakelet-relay...
agent # [ 9607.463313] agent dbus-broker-launch[898]: Looking up NSS user entry for 'systemd-timesync'...
agent # [ 9607.464808] agent dbus-broker-launch[898]: NSS returned no entry for 'systemd-timesync'
agent # [ 9607.464808] agent dbus-broker-launch[898]: Invalid user-name in /nix/store/q3c2gxfm178wqgzv8m87na4aa72wxc04-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"
agent # [ 9607.465462] agent systemd[1]: Started D-Bus System Message Bus.
agent # [ 9607.465764] agent systemd[1]: Startup finished in 5.282s.
agent # [ 9607.477586] agent dbus-broker-launch[898]: Ready
relay: (finished: waiting for unit flakelet-relay.service, in 6.16 seconds)
relay: waiting for unit nginx.service
relay: (finished: waiting for unit nginx.service, in 0.01 seconds)
agent: waiting for unit flakelet.target
agent: (finished: waiting for unit flakelet.target, in 0.02 seconds)
agent: waiting for unit flakelet-agent.service
agent: (finished: waiting for unit flakelet-agent.service, in 0.02 seconds)
client: waiting for unit multi-user.target
client: (finished: waiting for unit multi-user.target, in 0.02 seconds)
relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1'
relay # [ 9607.506852] relay systemd[1]: flakelet-relay-health.service: Deactivated successfully.
relay # [ 9607.507146] relay systemd[1]: Finished health for flakelet-relay.
relay # [ 9607.530809] relay flakelet[311]: flakelet-relay: updated to generation 1
relay # [ 9607.532646] relay systemd[1]: Finished Update flakelet service flakelet-relay.
relay # [ 9607.532912] relay systemd[1]: Reached target flakelet managed services.
relay # [ 9607.532997] relay systemd[1]: Reached target Multi-User System.
relay # [ 9607.533120] relay systemd[1]: Startup finished in 5.353s.
relay # [ 9609.064591] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:47.026321Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(1), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}
agent # [ 9609.064140] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:47.025915Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}
relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1', in 2.05 seconds)
agent: must succeed: systemctl is-active app.service other.service
agent: (finished: must succeed: systemctl is-active app.service other.service, in 0.01 seconds)
subtest: deploy with client certificate, relay found via SRV
agent: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
client: must succeed: push --relay-srv test --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app 2>&1
agent # [ 9610.009483] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:47.971202Z","level":"INFO","message":"start","id":"a545a9efc6d4f2130cea8d5a1427f0d9","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9610.010481] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:47.972297Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"a545a9efc6d4f2130cea8d5a1427f0d9\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9610.077740] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9610.117211] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9610.121513] agent flakelet[959]: app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2
agent # [ 9610.122207] agent flakelet[959]: app: activating generation 2
agent # [ 9610.134848] agent systemd[1]: Stopping app v1...
agent # [ 9610.136553] agent systemd[1]: app.service: Deactivated successfully.
agent # [ 9610.136883] agent systemd[1]: Stopped app v1.
agent # [ 9610.154574] agent systemd[1]: Reload requested from client PID 966 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9610.154642] agent systemd[1]: Reloading...
relay # [ 9610.008511] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:47.970238Z","level":"INFO","message":"deploy accepted","job":"3aa35c37e59eaf074982997d80ea382f","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}
agent # [ 9610.495591] agent systemd[1]: Reloading finished in 340 ms.
agent # [ 9610.587252] agent systemd[1]: Reload requested from client PID 996 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9610.587291] agent systemd[1]: Reloading...
agent # [ 9610.909313] agent systemd[1]: Reloading finished in 321 ms.
agent # [ 9610.992510] agent systemd[1]: Reload requested from client PID 1025 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9610.992551] agent systemd[1]: Reloading...
agent # [ 9611.320156] agent systemd[1]: Reloading finished in 327 ms.
agent # [ 9611.414321] agent systemd[1]: Reload requested from client PID 1059 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9611.414359] agent systemd[1]: Reloading...
agent # [ 9611.736285] agent systemd[1]: Reloading finished in 321 ms.
agent # [ 9611.806286] agent systemd[1]: Starting app v2...
agent # [ 9611.835070] agent systemd[1]: Started app v2.
agent # [ 9611.845482] agent flakelet[959]: app: updated to generation 2
agent # [ 9611.846458] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
client: (finished: must succeed: push --relay-srv test --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app 2>&1, in 3.00 seconds)
agent: must succeed: systemctl show app.service -p Description | grep -q 'app v2'
agent: (finished: must succeed: systemctl show app.service -p Description | grep -q 'app v2', in 0.02 seconds)
agent: must succeed: journalctl -u flakelet-relay-job-app.service | grep -q 'generation 2'
agent: (finished: must succeed: journalctl -u flakelet-relay-job-app.service | grep -q 'generation 2', in 0.01 seconds)
(finished: subtest: deploy with client certificate, relay found via SRV, in 3.04 seconds)
subtest: deploy with OIDC token, sub principal
agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
client: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)
client: (finished: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.05 seconds)
client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5MCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3ODkxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJBMzc1VkNBR0VVUEk2PT09IiwibmJmIjoxNzg4Njg3ODkxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.acsV-fWTCs2atdlnUplgoKj2zgGgpbzGmjEMx7QVbS038c-ChDSSg_q2kAr2-OUjsxn2hXuVsRn-CRI96qkmDQ' push --relay https://relay:7443 deploy agent/app 2>&1
agent # [ 9612.976475] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:50.938214Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(2)","target":"flakelet_relay::agent::jobs"}
agent # [ 9613.106090] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:51.067843Z","level":"INFO","message":"start","id":"55e95ed5d514288d49c36b7bd1becf61","flakelet":"app","rule":"ci","relay":"relay","caller_name":"Some(\"repo:github:example/app:ref:refs/heads/main\")","target":"flakelet_relay::agent::conn"}
agent # [ 9613.106376] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:51.068186Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"55e95ed5d514288d49c36b7bd1becf61\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9613.149642] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9613.185081] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9613.190260] agent flakelet[1160]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1
agent # [ 9613.190984] agent flakelet[1160]: app: activating generation 3
agent # [ 9613.202961] agent systemd[1]: Stopping app v2...
agent # [ 9613.204302] agent systemd[1]: app.service: Deactivated successfully.
agent # [ 9613.204522] agent systemd[1]: Stopped app v2.
agent # [ 9613.217802] agent systemd[1]: Reload requested from client PID 1167 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9613.217860] agent systemd[1]: Reloading...
relay # [ 9612.978083] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:50.939756Z","level":"INFO","message":"deploy finished","job":"3aa35c37e59eaf074982997d80ea382f","ok":true,"target":"flakelet_relay::relay::api"}
relay # [ 9613.105746] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:51.067515Z","level":"INFO","message":"deploy accepted","job":"8680abbd92bba6531427a6116622e760","caller":"repo:github:example/app:ref:refs/heads/main","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}
agent # [ 9613.550665] agent systemd[1]: Reloading finished in 332 ms.
agent # [ 9613.633247] agent systemd[1]: Reload requested from client PID 1197 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9613.633286] agent systemd[1]: Reloading...
agent # [ 9613.959766] agent systemd[1]: Reloading finished in 326 ms.
agent # [ 9614.052531] agent systemd[1]: Reload requested from client PID 1226 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9614.052571] agent systemd[1]: Reloading...
agent # [ 9614.374874] agent systemd[1]: Reloading finished in 321 ms.
agent # [ 9614.442758] agent systemd[1]: Reload requested from client PID 1257 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9614.442794] agent systemd[1]: Reloading...
agent # [ 9614.766894] agent systemd[1]: Reloading finished in 323 ms.
agent # [ 9614.863875] agent systemd[1]: Starting app v1...
agent # [ 9614.893024] agent systemd[1]: Started app v1.
agent # [ 9614.907825] agent flakelet[1160]: app: updated to generation 3
agent # [ 9614.909900] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5MCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3ODkxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJBMzc1VkNBR0VVUEk2PT09IiwibmJmIjoxNzg4Njg3ODkxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.acsV-fWTCs2atdlnUplgoKj2zgGgpbzGmjEMx7QVbS038c-ChDSSg_q2kAr2-OUjsxn2hXuVsRn-CRI96qkmDQ' push --relay https://relay:7443 deploy agent/app 2>&1, in 2.93 seconds)
(finished: subtest: deploy with OIDC token, sub principal, in 3.00 seconds)
subtest: policy denies uncovered target
client: must fail: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5MCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3ODkxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJBMzc1VkNBR0VVUEk2PT09IiwibmJmIjoxNzg4Njg3ODkxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.acsV-fWTCs2atdlnUplgoKj2zgGgpbzGmjEMx7QVbS038c-ChDSSg_q2kAr2-OUjsxn2hXuVsRn-CRI96qkmDQ' push --relay https://relay:7443 deploy agent/other 2>&1
client: (finished: must fail: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5MCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3ODkxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJBMzc1VkNBR0VVUEk2PT09IiwibmJmIjoxNzg4Njg3ODkxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.acsV-fWTCs2atdlnUplgoKj2zgGgpbzGmjEMx7QVbS038c-ChDSSg_q2kAr2-OUjsxn2hXuVsRn-CRI96qkmDQ' push --relay https://relay:7443 deploy agent/other 2>&1, in 0.02 seconds)
(finished: subtest: policy denies uncovered target, in 0.02 seconds)
subtest: claim principal from groups, unchanged counts as success
client: must succeed: echo '{"groups": ["wheel"]}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'someone' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)
client: (finished: must succeed: echo '{"groups": ["wheel"]}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'someone' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.05 seconds)
client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5NCwiZ3JvdXBzIjpbIndoZWVsIl0sImlhdCI6MTc4ODY4Nzg5NCwiaXNzIjoiaHR0cHM6Ly9yZWxheTo4NDQzIiwianRpIjoiWk1TVTZQMjJDRDVNTz09PSIsIm5iZiI6MTc4ODY4Nzg5NCwic3ViIjoic29tZW9uZSJ9.NdYpb78dD4bfhM4pgUNGi03z8pT9nGHtXny379QZyz_QTPBFZDzS0ekmZGStbbPKc52-ZtMFt3NgC64JICEtCA' push --relay https://relay:7443 deploy agent/other 2>&1
agent # [ 9616.003999] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:53.965744Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}
agent # [ 9616.102883] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:54.064676Z","level":"INFO","message":"start","id":"ae0de93b75e827ff3c5fcc67ba222394","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"someone\")","target":"flakelet_relay::agent::conn"}
agent # [ 9616.103455] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:54.065281Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"ae0de93b75e827ff3c5fcc67ba222394\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9616.172598] agent systemd[1]: Starting flakelet update other (via relay)...
agent # [ 9616.209026] agent systemd[1]: Started flakelet update other (via relay).
agent # [ 9616.213515] agent flakelet[1338]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1
agent # [ 9616.214083] agent flakelet[1338]: other: up to date
agent # [ 9616.216485] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.
relay # [ 9616.005594] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:53.967331Z","level":"INFO","message":"deploy finished","job":"8680abbd92bba6531427a6116622e760","ok":true,"target":"flakelet_relay::relay::api"}
relay # [ 9616.102615] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:54.064414Z","level":"INFO","message":"deploy accepted","job":"02c5ac753196a07ebb8cf94b7f603650","caller":"someone","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}
client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5NCwiZ3JvdXBzIjpbIndoZWVsIl0sImlhdCI6MTc4ODY4Nzg5NCwiaXNzIjoiaHR0cHM6Ly9yZWxheTo4NDQzIiwianRpIjoiWk1TVTZQMjJDRDVNTz09PSIsIm5iZiI6MTc4ODY4Nzg5NCwic3ViIjoic29tZW9uZSJ9.NdYpb78dD4bfhM4pgUNGi03z8pT9nGHtXny379QZyz_QTPBFZDzS0ekmZGStbbPKc52-ZtMFt3NgC64JICEtCA' push --relay https://relay:7443 deploy agent/other 2>&1, in 0.70 seconds)
(finished: subtest: claim principal from groups, unchanged counts as success, in 0.75 seconds)
subtest: push login via device flow, cached token used afterwards
client: must succeed: push login --issuer https://relay:8443 --client-id flakelet-relay 2>&1
relay # [ 9616.783592] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:54.745333Z","level":"INFO","message":"deploy finished","job":"02c5ac753196a07ebb8cf94b7f603650","ok":true,"target":"flakelet_relay::relay::api"}
relay # [ 9616.804817] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:44:54] "POST /device HTTP/1.1" 200 -
agent # [ 9616.782308] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:54.744063Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}
relay # [ 9617.810168] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:44:55] "POST /token HTTP/1.1" 400 -
client: (finished: must succeed: push login --issuer https://relay:8443 --client-id flakelet-relay 2>&1, in 2.08 seconds)
client: must succeed: test "$(stat -c %a /root/.local/state/flakelet-push/token.json)" = 600
client: (finished: must succeed: test "$(stat -c %a /root/.local/state/flakelet-push/token.json)" = 600, in 0.01 seconds)
client: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1
relay # [ 9618.860156] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:44:56] "POST /token HTTP/1.1" 200 -
relay # [ 9618.888991] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:56.850781Z","level":"INFO","message":"deploy accepted","job":"a72041d7fe18aade8f793775c6d74383","caller":"dev@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}
agent # [ 9618.889318] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:56.851096Z","level":"INFO","message":"start","id":"2df89cb357d1e6e88e4b1a680393be0c","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"dev@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9618.890118] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:56.851737Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"2df89cb357d1e6e88e4b1a680393be0c\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9618.953429] agent systemd[1]: Starting flakelet update other (via relay)...
agent # [ 9618.990231] agent systemd[1]: Started flakelet update other (via relay).
agent # [ 9618.993857] agent flakelet[1388]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1
agent # [ 9618.994400] agent flakelet[1388]: other: up to date
agent # [ 9618.997311] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.
client: (finished: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1, in 0.69 seconds)
client: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1
relay # [ 9619.564891] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:57.526629Z","level":"INFO","message":"deploy finished","job":"a72041d7fe18aade8f793775c6d74383","ok":true,"target":"flakelet_relay::relay::api"}
relay # [ 9619.584235] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:57.546039Z","level":"INFO","message":"deploy accepted","job":"dd20960cf993670ab011469990176db9","caller":"dev@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}
agent # [ 9619.563462] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:57.525203Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}
agent # [ 9619.584496] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:57.546304Z","level":"INFO","message":"start","id":"ff913cd8361cf0eb83cf85bd406f9c8f","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"dev@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9619.584871] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:57.546693Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"ff913cd8361cf0eb83cf85bd406f9c8f\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9619.648521] agent systemd[1]: Starting flakelet update other (via relay)...
agent # [ 9619.686129] agent systemd[1]: Started flakelet update other (via relay).
agent # [ 9619.690105] agent flakelet[1437]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1
agent # [ 9619.690659] agent flakelet[1437]: other: up to date
agent # [ 9619.691556] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.
client: (finished: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1, in 0.70 seconds)
(finished: subtest: push login via device flow, cached token used afterwards, in 3.48 seconds)
subtest: waves: failing first wave skips the second
agent: must succeed: jq '.services.app.prebuilt="/nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave agent/other 2>&1
agent # [ 9620.258461] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:58.220204Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}
agent # [ 9620.292319] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:58.254116Z","level":"INFO","message":"start","id":"d9b8e1683093e5b08b9ee45cc2a8c931","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9620.292716] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:58.254538Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"d9b8e1683093e5b08b9ee45cc2a8c931\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9620.349701] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9620.388120] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9620.393073] agent flakelet[1477]: app: using prebuilt artifact /nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken
agent # [ 9620.393740] agent flakelet[1477]: app: activating generation 4
agent # [ 9620.404788] agent systemd[1]: Stopping app v1...
agent # [ 9620.406140] agent systemd[1]: app.service: Deactivated successfully.
agent # [ 9620.406361] agent systemd[1]: Stopped app v1.
agent # [ 9620.420648] agent systemd[1]: Reload requested from client PID 1484 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9620.420707] agent systemd[1]: Reloading...
relay # [ 9620.259864] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:58.221589Z","level":"INFO","message":"deploy finished","job":"dd20960cf993670ab011469990176db9","ok":true,"target":"flakelet_relay::relay::api"}
relay # [ 9620.292064] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:58.253872Z","level":"INFO","message":"deploy accepted","job":"56aec3139ffb9895cad78fd920f943b6","caller":"admin@example.org","targets":"[\"agent/app\", \"agent/other\"]","target":"flakelet_relay::relay::api"}
agent # [ 9620.752554] agent systemd[1]: Reloading finished in 331 ms.
agent # [ 9620.819850] agent systemd[1]: Reload requested from client PID 1514 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9620.819886] agent systemd[1]: Reloading...
agent # [ 9621.147008] agent systemd[1]: Reloading finished in 326 ms.
agent # [ 9621.227745] agent systemd[1]: Reload requested from client PID 1545 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9621.227782] agent systemd[1]: Reloading...
agent # [ 9621.552874] agent systemd[1]: Reloading finished in 324 ms.
agent # [ 9621.632640] agent systemd[1]: Reload requested from client PID 1579 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9621.632676] agent systemd[1]: Reloading...
agent # [ 9621.958440] agent systemd[1]: Reloading finished in 325 ms.
agent # [ 9622.040561] agent systemd[1]: Starting app broken...
agent # [ 9622.077380] agent (nonexistent)[1611]: app.service: Unable to locate executable '/nonexistent': No such file or directory
agent # [ 9622.077405] agent (nonexistent)[1611]: app.service: Failed at step EXEC spawning /nonexistent: No such file or directory
agent # [ 9622.079402] agent systemd[1]: app.service: Main process exited, code=exited, status=203/EXEC
agent # [ 9622.079499] agent systemd[1]: app.service: Failed with result 'exit-code'.
agent # [ 9622.079671] agent systemd[1]: Failed to start app broken.
agent # [ 9622.108583] agent systemd[1]: Reload requested from client PID 1625 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9622.108620] agent systemd[1]: Reloading...
agent # [ 9622.437760] agent systemd[1]: Reloading finished in 328 ms.
agent # [ 9622.501574] agent systemd[1]: Reload requested from client PID 1655 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9622.501610] agent systemd[1]: Reloading...
agent # [ 9622.825726] agent systemd[1]: Reloading finished in 323 ms.
agent # [ 9622.909167] agent systemd[1]: Reload requested from client PID 1685 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9622.909202] agent systemd[1]: Reloading...
agent # [ 9623.231471] agent systemd[1]: Reloading finished in 321 ms.
agent # [ 9623.320332] agent systemd[1]: Reload requested from client PID 1718 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9623.320367] agent systemd[1]: Reloading...
agent # [ 9623.642862] agent systemd[1]: Reloading finished in 322 ms.
agent # [ 9623.709178] agent systemd[1]: Starting app v1...
agent # [ 9623.738040] agent systemd[1]: Started app v1.
agent # [ 9623.739267] agent flakelet[1477]: app: deploy failed, rolled back: systemctl start app.service failed:
agent # [ 9623.739267] agent flakelet[1477]: failed units: app.service. See journalctl -u app.service
agent # [ 9623.740852] agent systemd[1]: flakelet-relay-job-app.service: Main process exited, code=exited, status=1/FAILURE
agent # [ 9623.740946] agent systemd[1]: flakelet-relay-job-app.service: Failed with result 'exit-code'.
client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave agent/other 2>&1, in 4.21 seconds)
agent: must succeed: systemctl show app.service -p Description | grep -q 'app v1'
agent: (finished: must succeed: systemctl show app.service -p Description | grep -q 'app v1', in 0.02 seconds)
(finished: subtest: waves: failing first wave skips the second, in 4.24 seconds)
subtest: host patterns expand on the relay
agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/app' 2>&1
agent # [ 9624.480143] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:02.441884Z","level":"INFO","message":"update finished","flakelet":"app","status":"RolledBack","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}
agent # [ 9624.536527] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:02.498319Z","level":"INFO","message":"start","id":"c084fbf4d01d441195874e85788fd123","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9624.536974] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:02.498792Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"c084fbf4d01d441195874e85788fd123\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9624.591225] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9624.626035] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9624.629668] agent flakelet[1801]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1
agent # [ 9624.630257] agent flakelet[1801]: app: up to date
agent # [ 9624.632714] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
relay # [ 9624.481360] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:02.443147Z","level":"INFO","message":"deploy finished","job":"56aec3139ffb9895cad78fd920f943b6","ok":false,"target":"flakelet_relay::relay::api"}
relay # [ 9624.536230] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:02.497987Z","level":"INFO","message":"deploy accepted","job":"40d0adcc8c1b1331239c07682fde03f6","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}
client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/app' 2>&1, in 0.68 seconds)
client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '*/app' 2>&1
relay # [ 9625.198483] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:03.160270Z","level":"INFO","message":"deploy finished","job":"40d0adcc8c1b1331239c07682fde03f6","ok":true,"target":"flakelet_relay::relay::api"}
relay # [ 9625.220546] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:03.182344Z","level":"INFO","message":"deploy accepted","job":"5eccb012153856d19cf624475c4572cb","caller":"admin@example.org","targets":"[\"agent/app\", \"ghost/app\"]","target":"flakelet_relay::relay::api"}
agent # [ 9625.197205] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.158940Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}
agent # [ 9625.220796] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.182602Z","level":"INFO","message":"start","id":"02052e4e1b1b0f4e26a4db34dfed3c54","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9625.221216] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.183036Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"02052e4e1b1b0f4e26a4db34dfed3c54\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9625.288480] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9625.320222] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9625.324233] agent flakelet[1850]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1
agent # [ 9625.324772] agent flakelet[1850]: app: up to date
agent # [ 9625.326614] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '*/app' 2>&1, in 0.69 seconds)
client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave '*/app' 2>&1
relay # [ 9625.887787] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:03.849565Z","level":"INFO","message":"deploy finished","job":"5eccb012153856d19cf624475c4572cb","ok":false,"target":"flakelet_relay::relay::api"}
relay # [ 9625.912366] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:03.874170Z","level":"INFO","message":"deploy accepted","job":"3f6c243b083622947815c325bc60a773","caller":"admin@example.org","targets":"[\"agent/app\", \"ghost/app\"]","target":"flakelet_relay::relay::api"}
agent # [ 9625.886552] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.848286Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}
agent # [ 9625.912639] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.874443Z","level":"INFO","message":"start","id":"13743ac4391a9051e03e0ccd5d136b0e","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9625.913058] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.874878Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"13743ac4391a9051e03e0ccd5d136b0e\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9625.982102] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9626.020101] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9626.024461] agent flakelet[1886]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1
agent # [ 9626.024976] agent flakelet[1886]: app: up to date
agent # [ 9626.026743] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave '*/app' 2>&1, in 0.70 seconds)
client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/nope' 2>&1
client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/nope' 2>&1, in 0.02 seconds)
(finished: subtest: host patterns expand on the relay, in 2.11 seconds)
subtest: detach returns once the job is accepted
agent: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
client: must succeed: timeout 20 push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --detach agent/app 2>&1
client: (finished: must succeed: timeout 20 push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --detach agent/app 2>&1, in 0.02 seconds)
agent: waiting for success: systemctl is-active flakelet-relay-job-app.service
relay # [ 9626.589313] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:04.551092Z","level":"INFO","message":"deploy finished","job":"3f6c243b083622947815c325bc60a773","ok":false,"target":"flakelet_relay::relay::api"}
relay # [ 9626.645164] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:04.606914Z","level":"INFO","message":"deploy accepted","job":"3015bdc86c3442a89c4b3e6a9bf00aaf","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}
agent # [ 9626.588056] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:04.549795Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}
agent # [ 9626.645394] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:04.607138Z","level":"INFO","message":"start","id":"55699d1ca2dee22c746350319ed619df","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9626.645868] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:04.607684Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"55699d1ca2dee22c746350319ed619df\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9626.711083] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9626.745400] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9626.750108] agent flakelet[1927]: app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2
agent # [ 9626.750781] agent flakelet[1927]: app: activating generation 4
agent # [ 9626.763261] agent systemd[1]: Stopping app v1...
agent # [ 9626.763630] agent systemd[1]: app.service: Deactivated successfully.
agent # [ 9626.763854] agent systemd[1]: Stopped app v1.
agent # [ 9626.778352] agent systemd[1]: Reload requested from client PID 1934 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9626.778408] agent systemd[1]: Reloading...
agent # [ 9627.110281] agent systemd[1]: Reloading finished in 331 ms.
agent # [ 9627.197122] agent systemd[1]: Reload requested from client PID 1967 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9627.197160] agent systemd[1]: Reloading...
agent # [ 9627.521768] agent systemd[1]: Reloading finished in 324 ms.
agent # [ 9627.599800] agent systemd[1]: Reload requested from client PID 1998 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9627.599834] agent systemd[1]: Reloading...
agent # [ 9627.925477] agent systemd[1]: Reloading finished in 325 ms.
agent: (finished: waiting for success: systemctl is-active flakelet-relay-job-app.service, in 1.35 seconds)
??? Warning (UserWarning): wait_until_fails(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.
    File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39
agent: waiting for failure: systemctl is-active flakelet-relay-job-app.service
??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.
    File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39
agent # [ 9627.997241] agent systemd[1]: Reload requested from client PID 2032 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9627.997275] agent systemd[1]: Reloading...
agent # [ 9628.322627] agent systemd[1]: Reloading finished in 325 ms.
agent # [ 9628.421963] agent systemd[1]: Starting app slow2...
agent: (finished: waiting for failure: systemctl is-active flakelet-relay-job-app.service, in 6.51 seconds)
(finished: subtest: detach returns once the job is accepted, in 7.89 seconds)
subtest: unknown host is final, missing agent is retried, agents listing
client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy nope/app 2>&1
client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy nope/app 2>&1, in 0.02 seconds)
agent: must succeed: systemctl stop flakelet-agent
agent: (finished: must succeed: systemctl stop flakelet-agent, in 0.06 seconds)
relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 0'
relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 0', in 0.02 seconds)
client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/other > /tmp/retry.out 2>&1 &
client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/other > /tmp/retry.out 2>&1 &, in 0.01 seconds)
client: waiting for success: grep -q 'agent_unavailable.*retrying' /tmp/retry.out
agent # [ 9634.471107] agent systemd[1]: Started app slow2.
agent # [ 9634.490816] agent flakelet[1927]: app: updated to generation 4
agent # [ 9634.493801] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
agent # [ 9634.533466] agent systemd[1]: Stopping flakelet-agent...
agent # [ 9634.568821] agent systemd[1]: flakelet-agent.service: Deactivated successfully.
agent # [ 9634.569125] agent systemd[1]: Stopped flakelet-agent.
agent # [ 9634.569429] agent systemd[1]: flakelet-agent.service: Consumed 2.437s CPU time over 27.620s wall clock time.
relay # [ 9634.534098] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:12.495829Z","level":"INFO","message":"agent disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","host":"agent","target":"flakelet_relay::relay::agent_conn"}
client: (finished: waiting for success: grep -q 'agent_unavailable.*retrying' /tmp/retry.out, in 1.02 seconds)
agent: must succeed: systemctl start flakelet-agent
agent: (finished: must succeed: systemctl start flakelet-agent, in 0.14 seconds)
??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.
    File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39
client: waiting for success: grep -q 'agent/other: unchanged' /tmp/retry.out
agent # [ 9635.642345] agent systemd[1]: Starting flakelet-agent...
agent # [ 9635.699430] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:13.661178Z","level":"INFO","message":"reattaching to running unit","flakelet":"app","ids":"[\"55699d1ca2dee22c746350319ed619df\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9635.771776] agent systemd[1]: Started flakelet-agent.
agent # [ 9635.781648] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:13.743460Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}
relay # [ 9635.781849] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:13.743562Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(4), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}
agent # [ 9636.266944] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:14.228681Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(4)","target":"flakelet_relay::agent::jobs"}
relay # [ 9637.632374] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:15.594115Z","level":"INFO","message":"deploy accepted","job":"a83facd19d92b3ea501c99e5f1689915","caller":"admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}
agent # [ 9637.632845] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:15.594587Z","level":"INFO","message":"start","id":"e0a706f85f3c8d2579755112c2fb6c5c","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9637.633710] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:15.595321Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"e0a706f85f3c8d2579755112c2fb6c5c\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9637.696228] agent systemd[1]: Starting flakelet update other (via relay)...
agent # [ 9637.731072] agent systemd[1]: Started flakelet update other (via relay).
agent # [ 9637.735211] agent flakelet[2582]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1
agent # [ 9637.735763] agent flakelet[2582]: other: up to date
agent # [ 9637.738762] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.
relay # [ 9638.307964] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:16.269672Z","level":"INFO","message":"deploy finished","job":"a83facd19d92b3ea501c99e5f1689915","ok":true,"target":"flakelet_relay::relay::api"}
agent # [ 9638.306352] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:16.268089Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}
client: (finished: waiting for success: grep -q 'agent/other: unchanged' /tmp/retry.out, in 3.04 seconds)
client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents
client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents, in 0.02 seconds)
client: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)
client: (finished: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.06 seconds)
client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODIxNiwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTE2LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJNTzJTS1BMQk03M0dVPT09IiwibmJmIjoxNzg4Njg3OTE2LCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.HbnlIqBBWUlA6fUBm1mSrd-bTOq7F4doBf-dqWC0PxGivtbyG1LnaxlEOU5O3mekAS7Fcml7XUcBdRRQx78UCQ' push --relay https://relay:7443 agents
client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODIxNiwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTE2LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJNTzJTS1BMQk03M0dVPT09IiwibmJmIjoxNzg4Njg3OTE2LCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.HbnlIqBBWUlA6fUBm1mSrd-bTOq7F4doBf-dqWC0PxGivtbyG1LnaxlEOU5O3mekAS7Fcml7XUcBdRRQx78UCQ' push --relay https://relay:7443 agents, in 0.02 seconds)
(finished: subtest: unknown host is final, missing agent is retried, agents listing, in 4.41 seconds)
subtest: out-of-band flakelet update reaches the relay without reconnect
agent: must succeed: flakelet status --json app | jq '.[0].generation'
agent: (finished: must succeed: flakelet status --json app | jq '.[0].generation', in 0.06 seconds)
agent: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true
agent: (finished: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true, in 0.01 seconds)
agent: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
agent: must succeed: flakelet update app
app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2
app: activating generation 5
agent # [ 9639.015646] agent systemd[1]: Stopping app slow2...
agent # [ 9639.016144] agent systemd[1]: app.service: Deactivated successfully.
agent # [ 9639.016416] agent systemd[1]: Stopped app slow2.
agent # [ 9639.034916] agent systemd[1]: Reload requested from client PID 2642 ('systemctl')...
agent # [ 9639.034985] agent systemd[1]: Reloading...
agent # [ 9639.372376] agent systemd[1]: Reloading finished in 336 ms.
agent # [ 9639.443193] agent systemd[1]: Reload requested from client PID 2671 ('systemctl')...
agent # [ 9639.443230] agent systemd[1]: Reloading...
agent # [ 9639.764668] agent systemd[1]: Reloading finished in 321 ms.
agent # [ 9639.854201] agent systemd[1]: Reload requested from client PID 2701 ('systemctl')...
agent # [ 9639.854246] agent systemd[1]: Reloading...
agent # [ 9640.179192] agent systemd[1]: Reloading finished in 324 ms.
agent # [ 9640.246894] agent systemd[1]: Reload requested from client PID 2732 ('systemctl')...
agent # [ 9640.246929] agent systemd[1]: Reloading...
agent: (finished: must succeed: flakelet update app, in 1.69 seconds)
client: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@5'
agent # [ 9640.569690] agent systemd[1]: Reloading finished in 322 ms.
agent # [ 9640.637272] agent systemd[1]: Starting app v2...
agent # [ 9640.671078] agent systemd[1]: Started app v2.
client: (finished: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@5', in 2.06 seconds)
agent: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true
agent: (finished: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true, in 0.01 seconds)
agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
agent: must succeed: flakelet update app
app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1
app: activating generation 6
agent # [ 9642.792287] agent systemd[1]: Stopping app v2...
agent # [ 9642.792754] agent systemd[1]: app.service: Deactivated successfully.
agent # [ 9642.793036] agent systemd[1]: Stopped app v2.
agent # [ 9642.810069] agent systemd[1]: Reload requested from client PID 2799 ('systemctl')...
agent # [ 9642.810137] agent systemd[1]: Reloading...
agent # [ 9643.150998] agent systemd[1]: Reloading finished in 340 ms.
agent # [ 9643.215659] agent systemd[1]: Reload requested from client PID 2828 ('systemctl')...
agent # [ 9643.215696] agent systemd[1]: Reloading...
agent # [ 9643.536843] agent systemd[1]: Reloading finished in 320 ms.
agent # [ 9643.599328] agent systemd[1]: Reload requested from client PID 2856 ('systemctl')...
agent # [ 9643.599365] agent systemd[1]: Reloading...
agent # [ 9643.923171] agent systemd[1]: Reloading finished in 323 ms.
agent # [ 9643.994471] agent systemd[1]: Reload requested from client PID 2888 ('systemctl')...
agent # [ 9643.994509] agent systemd[1]: Reloading...
agent: (finished: must succeed: flakelet update app, in 1.66 seconds)
client: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@6'
agent # [ 9644.317348] agent systemd[1]: Reloading finished in 322 ms.
agent # [ 9644.382318] agent systemd[1]: Starting app v1...
agent # [ 9644.413725] agent systemd[1]: Started app v1.
client: (finished: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@6', in 2.06 seconds)
(finished: subtest: out-of-band flakelet update reaches the relay without reconnect, in 7.58 seconds)
subtest: certificate not listed under agents cannot connect as agent
client: must succeed: curl -s -o /dev/null -w '%{http_code}' --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/cert.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/key.pem -H 'Upgrade: websocket' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' https://relay:7443/v1/agent
client: (finished: must succeed: curl -s -o /dev/null -w '%{http_code}' --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/cert.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/key.pem -H 'Upgrade: websocket' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' https://relay:7443/v1/agent, in 0.02 seconds)
(finished: subtest: certificate not listed under agents cannot connect as agent, in 0.02 seconds)
subtest: metrics
relay: must succeed: curl -sf http://127.0.0.1:7400/metrics
relay: (finished: must succeed: curl -sf http://127.0.0.1:7400/metrics, in 0.02 seconds)
(finished: subtest: metrics, in 0.02 seconds)
subtest: agent link drop mid-run replays missed lines once
agent: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
agent: must succeed: date +%s
agent: (finished: must succeed: date +%s, in 0.01 seconds)
client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/slow.out 2>&1 &
client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/slow.out 2>&1 &, in 0.01 seconds)
agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687924 | grep -q 'app-slow'
relay # [ 9646.509062] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:24.470808Z","level":"INFO","message":"agent rejected: no agents entry matches","principals":"[\"x509:dns:mallory\"]","target":"flakelet_relay::relay::agent_conn"}
relay # [ 9646.567733] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:24.529479Z","level":"INFO","message":"deploy accepted","job":"440f9582a38de6779d9609d608b774a0","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}
agent # [ 9646.568095] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:24.529758Z","level":"INFO","message":"start","id":"0df49d05b7e4e93b26d0ba61f2169a2a","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9646.568877] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:24.530441Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"0df49d05b7e4e93b26d0ba61f2169a2a\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9646.616797] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9646.646070] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9646.650196] agent flakelet[2962]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow
agent # [ 9646.650850] agent flakelet[2962]: app: activating generation 7
agent # [ 9646.658976] agent systemd[1]: Stopping app v1...
agent # [ 9646.659690] agent systemd[1]: app.service: Deactivated successfully.
agent # [ 9646.659818] agent systemd[1]: Stopped app v1.
agent # [ 9646.670123] agent systemd[1]: Reload requested from client PID 2969 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9646.670161] agent systemd[1]: Reloading...
agent # [ 9646.993737] agent systemd[1]: Reloading finished in 323 ms.
agent # [ 9647.058655] agent systemd[1]: Reload requested from client PID 2999 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9647.058692] agent systemd[1]: Reloading...
agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687924 | grep -q 'app-slow', in 1.02 seconds)
agent: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route add unreachable $a; done
agent: (finished: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route add unreachable $a; done, in 0.02 seconds)
relay: must succeed: for a in $(getent ahosts agent | cut -d' ' -f1 | sort -u); do ss -K dst "[$a]"; done
relay: (finished: must succeed: for a in $(getent ahosts agent | cut -d' ' -f1 | sort -u); do ss -K dst "[$a]"; done, in 0.06 seconds)
agent: must succeed: ss -K dport = 7443
agent # [ 9647.381714] agent systemd[1]: Reloading finished in 322 ms.
agent # [ 9647.445982] agent systemd[1]: Reload requested from client PID 3028 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9647.446027] agent systemd[1]: Reloading...
agent: (finished: must succeed: ss -K dport = 7443, in 0.03 seconds)
agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687924 | grep -q 'updated to generation'
relay # [ 9647.651228] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:25.612964Z","level":"INFO","message":"agent disconnected: IO error: Software caused connection abort (os error 103)","host":"agent","target":"flakelet_relay::relay::agent_conn"}
agent # [ 9647.684446] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:25.645988Z","level":"INFO","message":"disconnected: IO error: Software caused connection abort (os error 103)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}
agent # [ 9647.769252] agent systemd[1]: Reloading finished in 322 ms.
agent # [ 9647.852344] agent systemd[1]: Reload requested from client PID 3076 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9647.852380] agent systemd[1]: Reloading...
agent # [ 9648.175845] agent systemd[1]: Reloading finished in 323 ms.
agent # [ 9648.242016] agent systemd[1]: Starting app slow...
agent # [ 9648.687461] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:26.649213Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}
agent # [ 9650.689886] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:28.651638Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}
agent # [ 9654.309039] agent systemd[1]: Started app slow.
agent # [ 9654.328535] agent flakelet[2962]: app: updated to generation 7
agent # [ 9654.331209] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687924 | grep -q 'updated to generation', in 7.10 seconds)
agent: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route del unreachable $a; done
agent: (finished: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route del unreachable $a; done, in 0.02 seconds)
client: waiting for success: grep -q 'agent/app: updated' /tmp/slow.out
agent # [ 9654.692108] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:32.653640Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}
agent # [ 9655.462958] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:33.424697Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(7)","target":"flakelet_relay::agent::jobs"}
client: (finished: waiting for success: grep -q 'agent/app: updated' /tmp/slow.out, in 8.09 seconds)
client: must succeed: cat /tmp/slow.out
client: (finished: must succeed: cat /tmp/slow.out, in 0.01 seconds)
(finished: subtest: agent link drop mid-run replays missed lines once, in 16.38 seconds)
subtest: agent restart mid-run reattaches to the unit
agent: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/restart.out 2>&1 &
client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/restart.out 2>&1 &, in 0.01 seconds)
agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since -20s | grep -q 'app-slow2'
relay # [ 9662.696525] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:40.658261Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(7), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}
relay # [ 9662.697949] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:40.659770Z","level":"INFO","message":"deploy finished","job":"440f9582a38de6779d9609d608b774a0","ok":true,"target":"flakelet_relay::relay::api"}
relay # [ 9662.943118] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:40.904838Z","level":"INFO","message":"deploy accepted","job":"a70cd93e695c209c2c4be97c1a881505","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}
agent # [ 9662.696437] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:40.658181Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}
agent # [ 9662.697631] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:40.659450Z","level":"INFO","message":"start","id":"0df49d05b7e4e93b26d0ba61f2169a2a","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9662.943381] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:40.905165Z","level":"INFO","message":"start","id":"7de24a5e260e1768fc15f62fd80772f9","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9662.944372] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:40.906196Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"7de24a5e260e1768fc15f62fd80772f9\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9663.014426] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9663.045026] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9663.049392] agent flakelet[3295]: app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2
agent # [ 9663.050065] agent flakelet[3295]: app: activating generation 8
agent # [ 9663.062894] agent systemd[1]: Stopping app slow...
agent # [ 9663.063370] agent systemd[1]: app.service: Deactivated successfully.
agent # [ 9663.063636] agent systemd[1]: Stopped app slow.
agent # [ 9663.080462] agent systemd[1]: Reload requested from client PID 3302 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9663.080530] agent systemd[1]: Reloading...
agent # [ 9663.413196] agent systemd[1]: Reloading finished in 331 ms.
agent # [ 9663.484158] agent systemd[1]: Reload requested from client PID 3332 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9663.484193] agent systemd[1]: Reloading...
agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since -20s | grep -q 'app-slow2', in 1.02 seconds)
agent: must succeed: systemctl restart flakelet-agent
agent # [ 9663.808122] agent systemd[1]: Reloading finished in 323 ms.
agent # [ 9663.868393] agent systemd[1]: Reload requested from client PID 3363 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9663.868429] agent systemd[1]: Reloading...
agent # [ 9664.198583] agent systemd[1]: Reloading finished in 329 ms.
agent # [ 9664.269954] agent systemd[1]: Stopping flakelet-agent...
agent # [ 9664.273611] agent systemd[1]: Reload requested from client PID 3400 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9664.273648] agent systemd[1]: Reloading...
relay # [ 9664.270141] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:42.231810Z","level":"INFO","message":"agent disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","host":"agent","target":"flakelet_relay::relay::agent_conn"}
agent: (finished: must succeed: systemctl restart flakelet-agent, in 0.88 seconds)
agent: must succeed: test -n "$(ls /var/lib/flakelet-agent/jobs)"
agent: (finished: must succeed: test -n "$(ls /var/lib/flakelet-agent/jobs)", in 0.01 seconds)
client: waiting for success: grep -q 'agent/app: updated' /tmp/restart.out
agent # [ 9664.603790] agent systemd[1]: Reloading finished in 329 ms.
agent # [ 9664.660139] agent systemd[1]: flakelet-agent.service: Deactivated successfully.
agent # [ 9664.660381] agent systemd[1]: Stopped flakelet-agent.
agent # [ 9664.660397] agent systemd[1]: flakelet-agent.service: Consumed 1.936s CPU time over 29.017s wall clock time.
agent # [ 9664.678520] agent systemd[1]: Starting app slow2...
agent # [ 9664.679626] agent systemd[1]: Starting flakelet-agent...
agent # [ 9664.740160] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:42.701959Z","level":"INFO","message":"reattaching to running unit","flakelet":"app","ids":"[\"7de24a5e260e1768fc15f62fd80772f9\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9664.831208] agent systemd[1]: Started flakelet-agent.
agent # [ 9664.845126] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:42.806891Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}
agent # [ 9664.846434] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:42.808244Z","level":"INFO","message":"start","id":"7de24a5e260e1768fc15f62fd80772f9","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
relay # [ 9664.845244] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:42.806967Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(7), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}
agent # [ 9670.753096] agent systemd[1]: Started app slow2.
agent # [ 9670.774966] agent flakelet[3295]: app: updated to generation 8
agent # [ 9670.778144] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
relay # [ 9671.365701] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:49.327440Z","level":"INFO","message":"deploy finished","job":"a70cd93e695c209c2c4be97c1a881505","ok":true,"target":"flakelet_relay::relay::api"}
agent # [ 9671.364033] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:49.325777Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(8)","target":"flakelet_relay::agent::jobs"}
client: (finished: waiting for success: grep -q 'agent/app: updated' /tmp/restart.out, in 7.08 seconds)
client: must succeed: cat /tmp/restart.out
client: (finished: must succeed: cat /tmp/restart.out, in 0.01 seconds)
agent: must succeed: systemctl is-active app.service
agent: (finished: must succeed: systemctl is-active app.service, in 0.01 seconds)
(finished: subtest: agent restart mid-run reattaches to the unit, in 9.04 seconds)
subtest: relay restart mid-run, push resumes via /v1/jobs
agent: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json
agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)
agent: must succeed: date +%s
agent: (finished: must succeed: date +%s, in 0.01 seconds)
client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/resume.out 2>&1 &
client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/resume.out 2>&1 &, in 0.01 seconds)
agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687949 | grep -q 'app-slow'
agent # [ 9671.992389] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:49.954129Z","level":"INFO","message":"start","id":"f61a62e5ccbd9502ee565b40233040c0","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9671.992891] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:49.954711Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"f61a62e5ccbd9502ee565b40233040c0\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9672.064837] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9672.098010] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9672.102513] agent flakelet[3944]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow
agent # [ 9672.103162] agent flakelet[3944]: app: activating generation 9
agent # [ 9672.116706] agent systemd[1]: Stopping app slow2...
agent # [ 9672.118586] agent systemd[1]: app.service: Deactivated successfully.
agent # [ 9672.119024] agent systemd[1]: Stopped app slow2.
agent # [ 9672.134715] agent systemd[1]: Reload requested from client PID 3951 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9672.134780] agent systemd[1]: Reloading...
relay # [ 9671.992036] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:49.953814Z","level":"INFO","message":"deploy accepted","job":"98adb657cbe52468bad1fdcca0d30827","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}
agent # [ 9672.478099] agent systemd[1]: Reloading finished in 342 ms.
agent # [ 9672.555272] agent systemd[1]: Reload requested from client PID 3981 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9672.555310] agent systemd[1]: Reloading...
agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687949 | grep -q 'app-slow', in 1.03 seconds)
relay: must succeed: systemctl restart flakelet-relay
agent # [ 9672.881198] agent systemd[1]: Reloading finished in 325 ms.
agent # [ 9672.951193] agent systemd[1]: Reload requested from client PID 4012 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9672.951224] agent systemd[1]: Reloading...
agent # [ 9673.020628] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:50.982366Z","level":"INFO","message":"disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}
relay # [ 9673.019819] relay systemd[1]: Stopping flakelet-relay...
relay # [ 9673.075085] relay systemd[1]: flakelet-relay.service: Deactivated successfully.
relay # [ 9673.075551] relay systemd[1]: Stopped flakelet-relay.
relay # [ 9673.075596] relay systemd[1]: flakelet-relay.service: Consumed 2.504s CPU time over 1min 7.921s wall clock time.
relay # [ 9673.112746] relay systemd[1]: Starting flakelet-relay...
agent # [ 9673.283753] agent systemd[1]: Reloading finished in 332 ms.
agent # [ 9673.370761] agent systemd[1]: Reload requested from client PID 4048 ('systemctl') (unit flakelet-relay-job-app.service)...
agent # [ 9673.370796] agent systemd[1]: Reloading...
agent # [ 9673.699354] agent systemd[1]: Reloading finished in 328 ms.
agent # [ 9673.777499] agent systemd[1]: Starting app slow...
agent # [ 9674.023440] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:51.985195Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}
relay: (finished: must succeed: systemctl restart flakelet-relay, in 1.54 seconds)
client: waiting for success: grep -q '» ok' /tmp/resume.out
relay # [ 9674.546496] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:52.508248Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}
relay # [ 9674.547319] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:52.508671Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}
relay # [ 9674.547209] relay systemd[1]: Started flakelet-relay.
relay # [ 9676.038529] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:54.000243Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(8), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}
relay # [ 9676.040523] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:54.002332Z","level":"INFO","message":"job reattached","job":"98adb657cbe52468bad1fdcca0d30827","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}
agent # [ 9676.038221] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:53.999945Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}
agent # [ 9679.840017] agent systemd[1]: Started app slow.
agent # [ 9679.858819] agent flakelet[3944]: app: updated to generation 9
agent # [ 9679.861925] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
agent # [ 9680.979362] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:58.941069Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(9)","target":"flakelet_relay::agent::jobs"}
client: (finished: waiting for success: grep -q '» ok' /tmp/resume.out, in 7.09 seconds)
client: must succeed: cat /tmp/resume.out
client: (finished: must succeed: cat /tmp/resume.out, in 0.01 seconds)
(finished: subtest: relay restart mid-run, push resumes via /v1/jobs, in 9.70 seconds)
subtest: job history comes from the agents and survives a relay restart
client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id hist1 agent/other 2>&1
relay # [ 9681.666256] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:59.628018Z","level":"INFO","message":"deploy accepted","job":"188b06c4636b65849b3ba8daac22dc7c","caller":"admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}
agent # [ 9681.666684] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:59.628439Z","level":"INFO","message":"start","id":"5be20c693e5c0b60e3f85779794dfb86","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9681.667118] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:59.628944Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"5be20c693e5c0b60e3f85779794dfb86\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9681.731169] agent systemd[1]: Starting flakelet update other (via relay)...
agent # [ 9681.766045] agent systemd[1]: Started flakelet update other (via relay).
agent # [ 9681.771388] agent flakelet[4189]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1
agent # [ 9681.772236] agent flakelet[4189]: other: up to date
agent # [ 9681.772966] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.
client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id hist1 agent/other 2>&1, in 0.70 seconds)
relay: must succeed: systemctl restart flakelet-relay
relay # [ 9682.341554] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:46:00.303241Z","level":"INFO","message":"deploy finished","job":"188b06c4636b65849b3ba8daac22dc7c","ok":true,"target":"flakelet_relay::relay::api"}
relay # [ 9682.358613] relay systemd[1]: Stopping flakelet-relay...
relay # [ 9682.401155] relay systemd[1]: flakelet-relay.service: Deactivated successfully.
relay # [ 9682.401584] relay systemd[1]: Stopped flakelet-relay.
relay # [ 9682.401639] relay systemd[1]: flakelet-relay.service: Consumed 2.043s CPU time over 9.288s wall clock time.
relay # [ 9682.438561] relay systemd[1]: Starting flakelet-relay...
agent # [ 9682.339948] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:00.301679Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}
agent # [ 9682.358775] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:00.320581Z","level":"INFO","message":"disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}
agent # [ 9683.361035] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:01.322773Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}
relay: (finished: must succeed: systemctl restart flakelet-relay, in 1.49 seconds)
relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1'
relay # [ 9683.835070] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:01.796808Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}
relay # [ 9683.835885] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:01.797261Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}
relay # [ 9683.835668] relay systemd[1]: Started flakelet-relay.
relay # [ 9685.374625] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:03.336336Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(9), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}
agent # [ 9685.374456] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:03.336189Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}
relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1', in 2.06 seconds)
client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs
client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs, in 0.02 seconds)
client: must succeed: echo '{"groups": [], "repository": "example/app"}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)
client: (finished: must succeed: echo '{"groups": [], "repository": "example/app"}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.06 seconds)
client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODI2MywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTYzLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJYR0dGS0EyQU5WTk5BPT09IiwibmJmIjoxNzg4Njg3OTYzLCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.7p-lQPI3-f29LXUEnNzYGD8cTaFcIB_R5AQlq8oRsT-lm-Yy01z1jZpLaLD5rhAkwrt2I_R8tL6GHnUCQrdXDw' push --relay https://relay:7443 deploy --id hist2 agent/app
» job 6a239dbf87cfec796e49404c351d787f via relay 0.1.0 (agents: agent 0.1.0)
» wave 0
agent/app | Starting flakelet update app (via relay)...
agent/app | Started flakelet update app (via relay).
agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow
agent/app | app: up to date
agent # [ 9685.994588] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:03.956309Z","level":"INFO","message":"start","id":"dce6faec8455f85321e72ee634888126","flakelet":"app","rule":"ci","relay":"relay","caller_name":"Some(\"example/app\")","target":"flakelet_relay::agent::conn"}
agent # [ 9685.995209] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:03.957028Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"dce6faec8455f85321e72ee634888126\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9686.066532] agent systemd[1]: Starting flakelet update app (via relay)...
agent # [ 9686.097978] agent systemd[1]: Started flakelet update app (via relay).
agent # [ 9686.103766] agent flakelet[4253]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow
agent # [ 9686.104421] agent flakelet[4253]: app: up to date
agent # [ 9686.105604] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.
agent/app | flakelet-relay-job-app.service: Deactivated successfully.
relay # [ 9685.994039] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:03.955817Z","level":"INFO","message":"deploy accepted","job":"6a239dbf87cfec796e49404c351d787f","caller":"example/app","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}
» agent/app: unchanged (generation 9)
» ok
client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODI2MywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTYzLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJYR0dGS0EyQU5WTk5BPT09IiwibmJmIjoxNzg4Njg3OTYzLCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.7p-lQPI3-f29LXUEnNzYGD8cTaFcIB_R5AQlq8oRsT-lm-Yy01z1jZpLaLD5rhAkwrt2I_R8tL6GHnUCQrdXDw' push --relay https://relay:7443 deploy --id hist2 agent/app, in 0.70 seconds)
client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODI2MywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTYzLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJYR0dGS0EyQU5WTk5BPT09IiwibmJmIjoxNzg4Njg3OTYzLCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.7p-lQPI3-f29LXUEnNzYGD8cTaFcIB_R5AQlq8oRsT-lm-Yy01z1jZpLaLD5rhAkwrt2I_R8tL6GHnUCQrdXDw' push --relay https://relay:7443 jobs
client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODI2MywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTYzLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJYR0dGS0EyQU5WTk5BPT09IiwibmJmIjoxNzg4Njg3OTYzLCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.7p-lQPI3-f29LXUEnNzYGD8cTaFcIB_R5AQlq8oRsT-lm-Yy01z1jZpLaLD5rhAkwrt2I_R8tL6GHnUCQrdXDw' push --relay https://relay:7443 jobs, in 0.02 seconds)
(finished: subtest: job history comes from the agents and survives a relay restart, in 5.05 seconds)
subtest: dashboard login via authorization code flow
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code} %{redirect_url}' https://relay:7443/ui/ | grep -q '303 https://relay:7443/ui/login'
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code} %{redirect_url}' https://relay:7443/ui/ | grep -q '303 https://relay:7443/ui/login', in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -L https://relay:7443/ui/login
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -L https://relay:7443/ui/login, in 0.09 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs, in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -f https://relay:7443/v1/jobs | grep -q hist1
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -f https://relay:7443/v1/jobs | grep -q hist1, in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 10 https://relay:7443/ui/jobs/hist1/events || true
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 10 https://relay:7443/ui/jobs/hist1/events || true, in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -X POST 'https://relay:7443/ui/deploy?arg=other' | grep -q 403
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -X POST 'https://relay:7443/ui/deploy?arg=other' | grep -q 403, in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -D - -o /dev/null -H 'HX-Request: true' -X POST 'https://relay:7443/ui/deploy?arg=other'
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -D - -o /dev/null -H 'HX-Request: true' -X POST 'https://relay:7443/ui/deploy?arg=other', in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 30 https://relay:7443/ui/jobs/aa679b057ae16902f2e5b292dd90396d/events || true
relay # [ 9686.673607] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.635315Z","level":"INFO","message":"deploy finished","job":"6a239dbf87cfec796e49404c351d787f","ok":true,"target":"flakelet_relay::relay::api"}
relay # [ 9686.749987] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:46:04] "GET /authorize?response_type=code&client_id=dashboard&redirect_uri=https%3A%2F%2Frelay%3A7443%2Fui%2Fcallback&scope=openid%20profile%20email%20groups&state=VICnDwgXd_bb65-MlqOSv2B6kjB0J6Dzn3f9k9J2TAA&code_challenge=ux0X6GLHvycWdDfU70E0B4onkylU7pIQauSa2Q_5BWI&code_challenge_method=S256 HTTP/1.1" 302 -
relay # [ 9686.799906] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:46:04] "POST /token HTTP/1.1" 200 -
relay # [ 9686.800796] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.762608Z","level":"INFO","message":"login","name":"dev@example.org","principals":"[\"oidc:mock:someone\", \"oidc:mock:groups:wheel\"]","target":"flakelet_relay::relay::login"}
relay # [ 9686.860879] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.822684Z","level":"INFO","message":"job reattached","job":"188b06c4636b65849b3ba8daac22dc7c","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}
relay # [ 9686.899585] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.861403Z","level":"INFO","message":"deploy accepted","job":"189890aa3e83142428d6f07166d42001","caller":"dev@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}
relay # [ 9686.919501] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.881323Z","level":"INFO","message":"job reattached","job":"189890aa3e83142428d6f07166d42001","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}
agent # [ 9686.671750] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:04.633473Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(9)","target":"flakelet_relay::agent::jobs"}
agent # [ 9686.899806] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:04.861564Z","level":"INFO","message":"start","id":"a239886a520be8b32371f452d5c37574","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"dev@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9686.900089] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:04.861915Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"a239886a520be8b32371f452d5c37574\"]","target":"flakelet_relay::agent::jobs"}
agent # [ 9686.960961] agent systemd[1]: Starting flakelet update other (via relay)...
agent # [ 9686.994009] agent systemd[1]: Started flakelet update other (via relay).
agent # [ 9686.999542] agent flakelet[4303]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1
agent # [ 9687.000376] agent flakelet[4303]: other: up to date
agent # [ 9687.001739] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 30 https://relay:7443/ui/jobs/aa679b057ae16902f2e5b292dd90396d/events || true, in 1.68 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs/aa679b057ae16902f2e5b292dd90396d
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs/aa679b057ae16902f2e5b292dd90396d, in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -sf -o /dev/null https://relay:7443/ui/static/htmx.min.js
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -sf -o /dev/null https://relay:7443/ui/static/htmx.min.js, in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/?q=status%3Ahealthy+oth'
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/?q=status%3Ahealthy+oth', in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/flakelets/other
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/flakelets/other, in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/hosts?q=status%3Adisconnected'
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/hosts?q=status%3Adisconnected', in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -H 'HX-Request: true' -X POST 'https://relay:7443/ui/retry?arg=aa679b057ae16902f2e5b292dd90396d' | grep -q 400
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -H 'HX-Request: true' -X POST 'https://relay:7443/ui/retry?arg=aa679b057ae16902f2e5b292dd90396d' | grep -q 400, in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 15 'https://relay:7443/ui/events?path=jobs' > /tmp/ev.out 2>&1 &
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 15 'https://relay:7443/ui/events?path=jobs' > /tmp/ev.out 2>&1 &, in 0.01 seconds)
client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id live1 agent/other
» job 38796a7ba508eadfe82a1cd5def000bc via relay 0.1.0 (agents: agent 0.1.0)
» wave 0
agent/other | Starting flakelet update other (via relay)...
agent/other | Started flakelet update other (via relay).
agent/other | other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1
agent/other | other: up to date
agent # [ 9688.576693] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:06.538422Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}
agent # [ 9688.727758] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:06.689520Z","level":"INFO","message":"start","id":"5e317fc57863c2a7c03ebe1bdacd1356","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}
agent # [ 9688.728329] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:06.690148Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"5e317fc57863c2a7c03ebe1bdacd1356\"]","target":"flakelet_relay::agent::jobs"}
agent/otheragent # [ 9688.790759] agent systemd[1]: Starting flakelet update other (via relay)...
 | agent # [ 9688.819038] agent systemd[1]: Started flakelet update other (via relay).
flakelet-relay-job-other.service: Deactivated successfully.agent # [ 9688.824338] agent flakelet[4340]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1

agent # [ 9688.825199] agent flakelet[4340]: other: up to date
agent # [ 9688.826326] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.
relay # [ 9688.727322] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:06.689107Z","level":"INFO","message":"deploy accepted","job":"38796a7ba508eadfe82a1cd5def000bc","caller":"admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}
» agent/other: unchanged (generation 1)
» ok
client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id live1 agent/other, in 0.69 seconds)
client: waiting for success: grep -q live1 /tmp/ev.out
client: (finished: waiting for success: grep -q live1 /tmp/ev.out, in 0.01 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -X POST -o /dev/null https://relay:7443/ui/logout
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -X POST -o /dev/null https://relay:7443/ui/logout, in 0.02 seconds)
client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' https://relay:7443/ui/ | grep -q 303
client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' https://relay:7443/ui/ | grep -q 303, in 0.02 seconds)
(finished: subtest: dashboard login via authorization code flow, in 2.76 seconds)
(finished: run the VM test script, in 87.79 seconds)
test script finished in 87.82s
cleanup
kill NspawnMachine (pid 52)
kill NspawnMachine (pid 53)
kill NspawnMachine (pid 54)
Container agent terminated by signal KILL.
relay # [ 9689.399754] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:07.361541Z","level":"INFO","message":"deploy finished","job":"38796a7ba508eadfe82a1cd5def000bc","ok":true,"target":"flakelet_relay::relay::api"}
Container client terminated by signal KILL.
Container relay terminated by signal KILL.
(finished: cleanup, in 0.39 seconds)
additionally exposed symbols:
    agent, client, relay,
    vlan1,
    start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh
» job 3aa35c37e59eaf074982997d80ea382f via relay 0.1.0 (agents: agent 0.1.0)
» wave 0
agent/app | Starting flakelet update app (via relay)...
agent/app | Started flakelet update app (via relay).
agent/app | app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2
agent/app | app: activating generation 2
agent/app | app: updated to generation 2
agent/app | flakelet-relay-job-app.service: Deactivated successfully.
» agent/app: updated (generation 2)
» ok

» job 56aec3139ffb9895cad78fd920f943b6 via relay 0.1.0 (agents: agent 0.1.0)
» wave 0
agent/app | Starting flakelet update app (via relay)...
agent/app | Started flakelet update app (via relay).
agent/app | app: using prebuilt artifact /nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken
agent/app | app: activating generation 4
agent/app | app: deploy failed, rolled back: systemctl start app.service failed:
agent/app | failed units: app.service. See journalctl -u app.service
agent/app | flakelet-relay-job-app.service: Main process exited, code=exited, status=1/FAILURE
agent/app | flakelet-relay-job-app.service: Failed with result 'exit-code'.
» agent/app: rolled-back (generation 3)
agent/app │ Starting app v1...
agent/app │ Started app v1.
agent/app │ Stopping app v1...
agent/app │ app.service: Deactivated successfully.
agent/app │ Stopped app v1.
agent/app │ Starting app v2...
agent/app │ Started app v2.
agent/app │ Stopping app v2...
agent/app │ app.service: Deactivated successfully.
agent/app │ Stopped app v2.
agent/app │ Starting app v1...
agent/app │ Started app v1.
agent/app │ Stopping app v1...
agent/app │ app.service: Deactivated successfully.
agent/app │ Stopped app v1.
agent/app │ Starting app broken...
agent/app │ app.service: Unable to locate executable '/nonexistent': No such file or directory
agent/app │ app.service: Failed at step EXEC spawning /nonexistent: No such file or directory
agent/app │ app.service: Main process exited, code=exited, status=203/EXEC
agent/app │ app.service: Failed with result 'exit-code'.
agent/app │ Failed to start app broken.
agent/app │ Starting app v1...
agent/app │ Started app v1.
» skipped: agent/other
» failed

» job 440f9582a38de6779d9609d608b774a0 via relay 0.1.0 (agents: agent 0.1.0)
» wave 0
agent/app | Starting flakelet update app (via relay)...
agent/app | Started flakelet update app (via relay).
agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow
agent/app | app: activating generation 7
agent/app | app: updated to generation 7
agent/app | flakelet-relay-job-app.service: Deactivated successfully.
» agent/app: updated (generation 7)
» ok

» job a70cd93e695c209c2c4be97c1a881505 via relay 0.1.0 (agents: agent 0.1.0)
» wave 0
agent/app | Starting flakelet update app (via relay)...
agent/app | Started flakelet update app (via relay).
agent/app | app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2
agent/app | app: activating generation 8
agent/app | app: updated to generation 8
agent/app | flakelet-relay-job-app.service: Deactivated successfully.
» agent/app: updated (generation 8)
» ok

» job 98adb657cbe52468bad1fdcca0d30827 via relay 0.1.0 (agents: agent 0.1.0)
» wave 0
agent/app | Starting flakelet update app (via relay)...
agent/app | Started flakelet update app (via relay).
agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow
agent/app | app: activating generation 9
flakelet-push: stream: error reading a body from connection, resuming job 5c88c4be4e598f9a2cf7fd2570707b6f
flakelet-push: https://relay:7443: connect relay:7443: Connection refused (os error 111), retrying in 1s
flakelet-push: https://relay:7443: connect relay:7443: Connection refused (os error 111), retrying in 2s
» job 98adb657cbe52468bad1fdcca0d30827 via relay 0.1.0 (agents: agent 0.1.0)
» wave 0
agent/app | app: updated to generation 9
agent/app | flakelet-relay-job-app.service: Deactivated successfully.
» agent/app: updated (generation 9)
» ok

1788687959	hist1	admin@example.org	agent/other:unchanged
1788687949	5c88c4be4e598f9a2cf7fd2570707b6f	admin@example.org	agent/app:updated
1788687940	9ea811ff91edb1efcbed3d80c8cac0cd	admin@example.org	agent/app:updated
1788687924	fe8b9cf81c9212d5dcfc80af9ad59d33	admin@example.org	agent/app:updated
1788687915	ca6dbee687afe7ab341cc1e49c0db303	admin@example.org	agent/other:unchanged
1788687904	29d17930204ef426f4562129bc0deaf1	admin@example.org	agent/app:updated
1788687903	35e285f9c2176659c4e72565ef900704	admin@example.org	agent/app:unchanged
1788687903	f76e546af74a11a5a65c4949d33e9f0b	admin@example.org	agent/app:unchanged
1788687902	8c6fcee97235a03c2f58f74eb9a2043b	admin@example.org	agent/app:unchanged
1788687898	d86ec9ebef884696bfc46b98c20a284d	admin@example.org	agent/app:rolled-back
1788687897	c9461b2f44eb1a6985b90a3a2ea94dbe	dev@example.org	agent/other:unchanged
1788687896	03f353613bb9d215170e437bb7c8ffc8	dev@example.org	agent/other:unchanged
1788687894	7a11f9c99a78170bdd3222ceb9141d30	someone	agent/other:unchanged
1788687891	aeb452c85dd52cd446c4831375293ebf	repo:github:example/app:ref:refs/heads/main	agent/app:updated
1788687887	6ddb24702aeb90540341ab7554da8f06	admin@example.org	agent/app:updated