container-test-run-flakelet-relay
checks.x86_64-linux.nixos-test
· build #26
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Initializing machine ID from random generator.6Test will time out and terminate in 3600.0 seconds7run the VM test script8start all VMs9agent: systemd-nspawn running (pid 52)10relay: systemd-nspawn running (pid 54)11client: systemd-nspawn running (pid 53)12agent: Waiting for journal at /build/vm-state-agent/var/log/journal...13relay: Waiting for journal at /build/vm-state-relay/var/log/journal...14client: Waiting for journal at /build/vm-state-client/var/log/journal...15(finished: start all VMs, in 0.00 seconds)16relay: waiting for unit flakelet-relay.service17nixos-nspawn(relay): TAP vde-tap1 not found; container will be isolated from VDE18nixos-nspawn(relay): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.19nixos-nspawn(client): TAP vde-tap1 not found; container will be isolated from VDE20nixos-nspawn(client): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.21nixos-nspawn(agent): TAP vde-tap1 not found; container will be isolated from VDE22nixos-nspawn(agent): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.23Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.24Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.25░ Spawning container relay on /build/vm-state-relay.26Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.27░ Spawning container agent on /build/vm-state-agent.28░ Spawning container client on /build/vm-state-client.29relay # No journal files were found.30relay # No journal boot entry found for the specified boot (+0).31agent # No journal files were found.32agent # No journal boot entry found for the specified boot (+0).33client # No journal files were found.34client # No journal boot entry found for the specified boot (+0).35relay # [ 9602.610118] relay systemd-journald[57]: Journal started36relay # [ 9602.610158] relay systemd-journald[57]: Runtime Journal (/run/log/journal/248dcf733e5e4021bfbb9db19395b461) is 8M, max 4G, 3.9G free.37relay # [ 9602.621758] relay systemd[1]: Finished Apply Kernel Variables.38relay # [ 9602.633442] relay systemd[1]: Finished Create Static Device Nodes in /dev gracefully.39relay # [ 9602.647141] relay systemd[1]: Starting Flush Journal to Persistent Storage...40relay # [ 9602.647948] relay systemd[1]: Starting Create Static Device Nodes in /dev...41relay # [ 9602.687314] relay systemd-journald[57]: Time spent on flushing to /var/log/journal/248dcf733e5e4021bfbb9db19395b461 is 1.365ms for 6 entries.42agent # [ 9602.616034] agent systemd-journald[57]: Journal started43relay # [ 9602.687314] relay systemd-journald[57]: System Journal (/var/log/journal/248dcf733e5e4021bfbb9db19395b461) is 512B, max 4G, 3.9G free.44agent # [ 9602.616075] agent systemd-journald[57]: Runtime Journal (/run/log/journal/76045c452adf408887329d35b7f2c388) is 8M, max 4G, 3.9G free.45relay # [ 9602.695495] relay systemd[1]: Finished Create Static Device Nodes in /dev.46agent # [ 9602.626408] agent systemd[1]: Finished Apply Kernel Variables.47client # [ 9602.610676] client systemd-journald[57]: Journal started48agent # [ 9602.638157] agent systemd[1]: Finished Create Static Device Nodes in /dev gracefully.49client # [ 9602.610716] client systemd-journald[57]: Runtime Journal (/run/log/journal/6fda3615315145978b328a9b7132a79c) is 8M, max 4G, 3.9G free.50agent # [ 9602.681533] agent systemd[1]: Starting Flush Journal to Persistent Storage...51client # [ 9602.621023] client systemd[1]: Finished Apply Kernel Variables.52agent # [ 9602.682264] agent systemd[1]: Starting Create Static Device Nodes in /dev...53client # [ 9602.632673] client systemd[1]: Finished Create Static Device Nodes in /dev gracefully.54relay # [ 9602.696239] relay systemd[1]: Reached target Preparation for Local File Systems.55client # [ 9602.646406] client systemd[1]: Starting Flush Journal to Persistent Storage...56relay # [ 9602.696353] relay systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys57client # [ 9602.647113] client systemd[1]: Starting Create Static Device Nodes in /dev...58agent # [ 9602.690919] agent systemd-journald[57]: Time spent on flushing to /var/log/journal/76045c452adf408887329d35b7f2c388 is 2.576ms for 6 entries.59relay # [ 9602.710804] relay systemd[1]: Finished Flush Journal to Persistent Storage.60agent # [ 9602.690919] agent systemd-journald[57]: System Journal (/var/log/journal/76045c452adf408887329d35b7f2c388) is 512B, max 4G, 3.9G free.61client # [ 9602.686559] client systemd-journald[57]: Time spent on flushing to /var/log/journal/6fda3615315145978b328a9b7132a79c is 1.352ms for 6 entries.62relay # [ 9602.781445] relay systemd[1]: Finished Firewall.63client # [ 9602.686559] client systemd-journald[57]: System Journal (/var/log/journal/6fda3615315145978b328a9b7132a79c) is 512B, max 4G, 3.9G free.64relay # [ 9603.609591] relay systemd[1]: Mounting /run/wrappers...65agent # [ 9602.699822] agent systemd[1]: Finished Create Static Device Nodes in /dev.66client # [ 9602.695742] client systemd[1]: Finished Create Static Device Nodes in /dev.67agent # [ 9602.700505] agent systemd[1]: Reached target Preparation for Local File Systems.68client # [ 9602.696369] client systemd[1]: Reached target Preparation for Local File Systems.69relay # [ 9603.670252] relay systemd[1]: Mounted /run/wrappers.70client # [ 9602.696469] client systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys71agent # [ 9602.700602] agent systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys72relay # [ 9603.671301] relay systemd[1]: Reached target Local File Systems.73agent # [ 9602.717240] agent systemd[1]: Finished Flush Journal to Persistent Storage.74client # [ 9602.707708] client systemd[1]: Finished Flush Journal to Persistent Storage.75relay # [ 9603.672565] relay systemd[1]: Listening on Boot Loader Control Service Socket.76client # [ 9602.762678] client systemd[1]: Finished Firewall.77agent # [ 9602.779593] agent systemd[1]: Finished Firewall.78relay # [ 9603.674153] relay systemd[1]: Starting Create SUID/SGID Wrappers...79agent # [ 9603.638734] agent systemd[1]: Mounting /run/wrappers...80relay # [ 9603.674200] relay systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container81agent # [ 9603.669815] agent systemd[1]: Mounted /run/wrappers.82client # [ 9603.638746] client systemd[1]: Mounting /run/wrappers...83relay # [ 9603.675419] relay systemd[1]: Starting Save Transient machine-id to Disk...84client # [ 9603.670111] client systemd[1]: Mounted /run/wrappers.85agent # [ 9603.670856] agent systemd[1]: Reached target Local File Systems.86relay # [ 9603.676549] relay systemd[1]: Starting Create System Files and Directories...87agent # [ 9603.672150] agent systemd[1]: Listening on Boot Loader Control Service Socket.88client # [ 9603.671131] client systemd[1]: Reached target Local File Systems.89agent # [ 9603.673765] agent systemd[1]: Starting Create SUID/SGID Wrappers...90client # [ 9603.672384] client systemd[1]: Listening on Boot Loader Control Service Socket.91agent # [ 9603.673813] agent systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container92client # [ 9603.673660] client systemd[1]: Starting Create SUID/SGID Wrappers...93agent # [ 9603.674997] agent systemd[1]: Starting Save Transient machine-id to Disk...94client # [ 9603.673705] client systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container95agent # [ 9603.676423] agent systemd[1]: Starting Create System Files and Directories...96client # [ 9603.675232] client systemd[1]: Starting Save Transient machine-id to Disk...97client # [ 9603.676108] client systemd[1]: Starting Create System Files and Directories...98client # [ 9603.694199] client systemd-tmpfiles[161]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted99client # [ 9603.694651] client systemd-tmpfiles[161]: fchmod() of /var/log/journal failed: Operation not permitted100client # [ 9603.694986] client systemd-tmpfiles[161]: fchmod() of /var/log/journal/6fda3615315145978b328a9b7132a79c failed: Operation not permitted101client # [ 9603.695348] client systemd-tmpfiles[161]: fchmod() of /run/log/journal failed: Operation not permitted102relay # [ 9603.695481] relay systemd-tmpfiles[169]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted103relay # [ 9603.695901] relay systemd-tmpfiles[169]: fchmod() of /var/log/journal failed: Operation not permitted104agent # [ 9603.695681] agent systemd-tmpfiles[161]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted105relay # [ 9603.696231] relay systemd-tmpfiles[169]: fchmod() of /var/log/journal/248dcf733e5e4021bfbb9db19395b461 failed: Operation not permitted106agent # [ 9603.696105] agent systemd-tmpfiles[161]: fchmod() of /var/log/journal failed: Operation not permitted107agent # [ 9603.696427] agent systemd-tmpfiles[161]: fchmod() of /var/log/journal/76045c452adf408887329d35b7f2c388 failed: Operation not permitted108relay # [ 9603.696558] relay systemd-tmpfiles[169]: fchmod() of /run/log/journal failed: Operation not permitted109agent # [ 9603.696748] agent systemd-tmpfiles[161]: fchmod() of /run/log/journal failed: Operation not permitted110relay # [ 9603.697704] relay suid-sgid-wrappers-start[176]: chmod: changing permissions of '/run/wrappers/wrappers.Pr2sEjtRRY/chsh': Operation not permitted111relay # [ 9603.703128] relay systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE112relay # [ 9603.703168] relay systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.113relay # [ 9603.703228] relay systemd[1]: Failed to start Create SUID/SGID Wrappers.114relay # [ 9603.733739] relay systemd[1]: Finished Create System Files and Directories.115relay # [ 9603.761395] relay systemd[1]: etc-machine\x2did.mount: Deactivated successfully.116relay # [ 9603.774161] relay systemd[1]: Finished Save Transient machine-id to Disk.117relay # [ 9603.777209] relay systemd[1]: Starting Rebuild Journal Catalog...118relay # [ 9603.778032] relay systemd[1]: Starting Record System Boot/Shutdown in UTMP...119relay # [ 9603.818682] relay systemd[1]: Finished Record System Boot/Shutdown in UTMP.120relay # [ 9603.826188] relay systemd[1]: Finished Rebuild Journal Catalog.121relay # [ 9603.827248] relay systemd[1]: Starting Update is Completed...122relay # [ 9603.835825] relay systemd[1]: Finished Update is Completed.123relay # [ 9603.835902] relay systemd[1]: Reached target System Initialization.124relay # [ 9603.835962] relay systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container125relay # [ 9603.835981] relay systemd[1]: Started Daily Cleanup of Temporary Directories.126relay # [ 9603.835995] relay systemd[1]: Reached target Timer Units.127relay # [ 9603.836093] relay systemd[1]: Listening on D-Bus System Message Bus Socket.128relay # [ 9603.836270] relay systemd[1]: Listening on Nix Daemon Socket.129relay # [ 9603.836360] relay systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.130relay # [ 9603.836374] relay systemd[1]: Reached target Socket Units.131relay # [ 9603.836399] relay systemd[1]: Reached target Basic System.132relay # [ 9603.837388] relay systemd[1]: Started device-mock.service.133relay # [ 9603.838161] relay systemd[1]: Starting Re-link flakelet services at boot...134relay # [ 9603.838809] relay systemd[1]: Starting Import lastlog data into lastlog2 database...135relay # [ 9603.839587] relay systemd[1]: Starting Name Service Cache Daemon (nsncd)...136relay # [ 9603.840186] relay systemd[1]: Starting resolvconf update...137relay # [ 9603.849180] relay systemd[1]: Finished Re-link flakelet services at boot.138relay # [ 9603.850595] relay systemd[1]: Starting Reconcile flakelet services with the host configuration...139relay # [ 9603.856414] relay systemd[1]: Finished Import lastlog data into lastlog2 database.140agent # [ 9603.699231] agent suid-sgid-wrappers-start[168]: chmod: changing permissions of '/run/wrappers/wrappers.coaPTLOMMz/chsh': Operation not permitted141agent # [ 9603.701061] agent systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE142agent # [ 9603.701102] agent systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.143agent # [ 9603.701147] agent systemd[1]: Failed to start Create SUID/SGID Wrappers.144agent # [ 9603.733476] agent systemd[1]: Finished Create System Files and Directories.145agent # [ 9603.761836] agent systemd[1]: etc-machine\x2did.mount: Deactivated successfully.146agent # [ 9603.774383] agent systemd[1]: Finished Save Transient machine-id to Disk.147agent # [ 9603.777902] agent systemd[1]: Starting Rebuild Journal Catalog...148agent # [ 9603.778529] agent systemd[1]: Starting Record System Boot/Shutdown in UTMP...149agent # [ 9603.819061] agent systemd[1]: Finished Record System Boot/Shutdown in UTMP.150agent # [ 9603.826320] agent systemd[1]: Finished Rebuild Journal Catalog.151agent # [ 9603.827375] agent systemd[1]: Starting Update is Completed...152agent # [ 9603.837757] agent systemd[1]: Finished Update is Completed.153agent # [ 9603.837896] agent systemd[1]: Reached target System Initialization.154agent # [ 9603.837966] agent systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container155agent # [ 9603.837988] agent systemd[1]: Started Daily Cleanup of Temporary Directories.156agent # [ 9603.838010] agent systemd[1]: Reached target Timer Units.157agent # [ 9603.838103] agent systemd[1]: Listening on D-Bus System Message Bus Socket.158agent # [ 9603.838310] agent systemd[1]: Listening on Nix Daemon Socket.159agent # [ 9603.838413] agent systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.160agent # [ 9603.838426] agent systemd[1]: Reached target Socket Units.161agent # [ 9603.838454] agent systemd[1]: Reached target Basic System.162agent # [ 9603.839565] agent systemd[1]: Starting Re-link flakelet services at boot...163agent # [ 9603.840397] agent systemd[1]: Starting Import lastlog data into lastlog2 database...164agent # [ 9603.841140] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...165agent # [ 9603.841735] agent systemd[1]: Starting resolvconf update...166agent # [ 9603.852176] agent systemd[1]: Finished Re-link flakelet services at boot.167agent # [ 9603.853133] agent systemd[1]: Starting Reconcile flakelet services with the host configuration...168agent # [ 9603.858059] agent systemd[1]: Finished Import lastlog data into lastlog2 database.169agent # [ 9603.866420] agent systemd[1]: Finished Reconcile flakelet services with the host configuration.170client # [ 9603.695737] client suid-sgid-wrappers-start[168]: chmod: changing permissions of '/run/wrappers/wrappers.ZPV4x77r3M/chsh': Operation not permitted171client # [ 9603.700455] client systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE172client # [ 9603.700552] client systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.173client # [ 9603.700744] client systemd[1]: Failed to start Create SUID/SGID Wrappers.174client # [ 9603.733175] client systemd[1]: Finished Create System Files and Directories.175client # [ 9603.761410] client systemd[1]: etc-machine\x2did.mount: Deactivated successfully.176client # [ 9603.774150] client systemd[1]: Finished Save Transient machine-id to Disk.177client # [ 9603.776955] client systemd[1]: Starting Rebuild Journal Catalog...178client # [ 9603.777831] client systemd[1]: Starting Record System Boot/Shutdown in UTMP...179client # [ 9603.819023] client systemd[1]: Finished Record System Boot/Shutdown in UTMP.180client # [ 9603.825110] client systemd[1]: Finished Rebuild Journal Catalog.181client # [ 9603.826080] client systemd[1]: Starting Update is Completed...182client # [ 9603.834600] client systemd[1]: Finished Update is Completed.183client # [ 9603.834669] client systemd[1]: Reached target System Initialization.184client # [ 9603.834733] client systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container185client # [ 9603.834751] client systemd[1]: Started Daily Cleanup of Temporary Directories.186client # [ 9603.834764] client systemd[1]: Reached target Timer Units.187client # [ 9603.834853] client systemd[1]: Listening on D-Bus System Message Bus Socket.188client # [ 9603.835057] client systemd[1]: Listening on Nix Daemon Socket.189client # [ 9603.835146] client systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.190client # [ 9603.835157] client systemd[1]: Reached target Socket Units.191client # [ 9603.835183] client systemd[1]: Reached target Basic System.192client # [ 9603.836399] client systemd[1]: Starting Import lastlog data into lastlog2 database...193client # [ 9603.837198] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...194client # [ 9603.837970] client systemd[1]: Starting resolvconf update...195client # [ 9603.858944] client systemd[1]: Finished Import lastlog data into lastlog2 database.196relay # [ 9603.862834] relay systemd[1]: Finished Reconcile flakelet services with the host configuration.197relay # [ 9603.921846] relay systemd[1]: nscd.service: Deactivated successfully.198relay # [ 9603.921964] relay systemd[1]: Stopped Name Service Cache Daemon (nsncd).199relay # [ 9603.924357] relay systemd[1]: Starting Name Service Cache Daemon (nsncd)...200relay # [ 9603.959227] relay systemd[1]: Finished resolvconf update.201relay # [ 9603.959403] relay systemd[1]: Reached target Preparation for Network.202relay # [ 9603.960448] relay systemd[1]: Starting Address configuration of eth1...203relay # [ 9603.961373] relay systemd[1]: Starting Extra networking commands....204relay # [ 9603.979995] relay network-addresses-eth1-start[251]: adding address 192.168.1.3/24... done205relay # [ 9603.982660] relay network-addresses-eth1-start[251]: adding address 2001:db8:1::3/64... done206relay # [ 9603.986796] relay systemd[1]: Finished Address configuration of eth1.207relay # [ 9604.027856] relay systemd[1]: Finished Extra networking commands..208relay # [ 9604.027939] relay systemd[1]: Reached target Network.209relay # [ 9604.029306] relay systemd[1]: Starting Dnsmasq Daemon...210relay # [ 9604.030139] relay systemd[1]: Starting Nginx Web Server...211relay # [ 9604.030885] relay systemd[1]: Starting Update flakelet service flakelet-relay...212relay # [ 9604.079313] relay flakelet[311]: flakelet-relay: using prebuilt artifact /nix/store/j272yq4kj97x5q0nggal747653288wg4-flakelet-flakelet-relay213relay # [ 9604.079957] relay flakelet[311]: flakelet-relay: activating generation 1214relay # [ 9604.084159] relay systemd[1]: Reload requested from client PID 316 ('systemctl') (unit flakelet-flakelet-relay.service)...215relay # [ 9604.084205] relay systemd[1]: Reloading...216agent # [ 9603.918072] agent systemd[1]: nscd.service: Deactivated successfully.217agent # [ 9603.921628] agent systemd[1]: Stopped Name Service Cache Daemon (nsncd).218agent # [ 9603.924183] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...219agent # [ 9603.959220] agent systemd[1]: Finished resolvconf update.220agent # [ 9603.959470] agent systemd[1]: Reached target Preparation for Network.221agent # [ 9603.960517] agent systemd[1]: Starting Address configuration of eth1...222agent # [ 9603.961320] agent systemd[1]: Starting Extra networking commands....223agent # [ 9603.982950] agent network-addresses-eth1-start[242]: adding address 192.168.1.1/24... done224agent # [ 9603.986513] agent network-addresses-eth1-start[242]: adding address 2001:db8:1::1/64... done225agent # [ 9603.991163] agent systemd[1]: Finished Address configuration of eth1.226agent # [ 9604.071372] agent systemd[1]: Finished Extra networking commands..227agent # [ 9604.071633] agent systemd[1]: nscd.service: Deactivated successfully.228agent # [ 9604.071778] agent systemd[1]: Stopped Name Service Cache Daemon (nsncd).229agent # [ 9604.073975] agent systemd[1]: Reached target Network.230agent # [ 9604.075158] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...231agent # [ 9604.075967] agent systemd[1]: Starting Update flakelet service app...232agent # [ 9604.076917] agent systemd[1]: Starting Update flakelet service flakelet-agent...233agent # [ 9604.077742] agent systemd[1]: Starting Update flakelet service other...234agent # [ 9604.088011] agent flakelet[308]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1235agent # [ 9604.088315] agent flakelet[309]: flakelet-agent: using prebuilt artifact /nix/store/j44k16n69mi2k45ccgnkdvbqapv6qb5n-flakelet-flakelet-agent236agent # [ 9604.088620] agent flakelet[308]: app: activating generation 1237agent # [ 9604.088783] agent flakelet[309]: flakelet-agent: activating generation 1238agent # [ 9604.088806] agent flakelet[310]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1239agent # [ 9604.089319] agent flakelet[310]: other: activating generation 1240agent # [ 9604.093500] agent systemd[1]: Reload requested from client PID 312 ('systemctl') (unit flakelet-app.service)...241agent # [ 9604.093537] agent systemd[1]: Reloading...242client # [ 9603.930999] client systemd[1]: nscd.service: Deactivated successfully.243client # [ 9603.959218] client systemd[1]: Stopped Name Service Cache Daemon (nsncd).244client # [ 9603.960684] client systemd[1]: Finished resolvconf update.245client # [ 9603.961371] client systemd[1]: Reached target Preparation for Network.246client # [ 9603.962516] client systemd[1]: Starting Address configuration of eth1...247client # [ 9603.963264] client systemd[1]: Starting Extra networking commands....248client # [ 9603.964066] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...249client # [ 9603.983228] client network-addresses-eth1-start[236]: adding address 192.168.1.2/24... done250client # [ 9603.986035] client network-addresses-eth1-start[236]: adding address 2001:db8:1::2/64... done251client # [ 9603.989430] client systemd[1]: Finished Address configuration of eth1.252client # [ 9604.071362] client systemd[1]: Finished Extra networking commands..253client # [ 9604.071623] client systemd[1]: nscd.service: Deactivated successfully.254client # [ 9604.071754] client systemd[1]: Stopped Name Service Cache Daemon (nsncd).255client # [ 9604.073787] client systemd[1]: Reached target Network.256client # [ 9604.074912] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...257relay # [ 9604.414299] relay nsncd[246]: Sep 06 09:44:42.376 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"258relay # [ 9604.510585] relay systemd[1]: Reloading finished in 425 ms.259relay # [ 9604.635671] relay systemd[1]: Started Name Service Cache Daemon (nsncd).260relay # [ 9604.636133] relay systemd[1]: Reached target Host and Network Name Lookups.261relay # [ 9604.636219] relay systemd[1]: Reached target User and Group Name Lookups.262relay # [ 9604.637577] relay systemd[1]: Starting User Login Management...263relay # [ 9604.638410] relay systemd[1]: Starting Permit User Sessions...264relay # [ 9604.656364] relay systemd[1]: Reload requested from client PID 357 ('systemctl') (unit flakelet-flakelet-relay.service)...265relay # [ 9604.656428] relay systemd[1]: Reloading...266agent # [ 9604.418848] agent nsncd[307]: Sep 06 09:44:42.380 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"267agent # [ 9604.506421] agent systemd[1]: Reloading finished in 412 ms.268agent # [ 9604.638600] agent systemd[1]: Started Name Service Cache Daemon (nsncd).269agent # [ 9604.639166] agent systemd[1]: Reload requested from client PID 313 ('systemctl') (unit flakelet-flakelet-agent.service)...270agent # [ 9604.639225] agent systemd[1]: Reloading...271client # [ 9604.397042] client nsncd[303]: Sep 06 09:44:42.358 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"272client # [ 9604.397746] client systemd[1]: Started Name Service Cache Daemon (nsncd).273client # [ 9604.397836] client systemd[1]: Reached target Host and Network Name Lookups.274client # [ 9604.398231] client systemd[1]: Reached target User and Group Name Lookups.275client # [ 9604.408141] client systemd[1]: Starting User Login Management...276client # [ 9604.411657] client systemd[1]: Starting Permit User Sessions...277client # [ 9604.451043] client systemd[1]: Finished Permit User Sessions.278client # [ 9604.452721] client systemd[1]: Started Console Getty.279client # [ 9604.452764] client systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0280client # [ 9604.452784] client systemd[1]: Reached target Login Prompts.281relay # [ 9604.682213] relay dnsmasq-pre-start[366]: dnsmasq: syntax check OK.282agent # [ 9605.003751] agent systemd[1]: Reloading finished in 364 ms.283agent # [ 9605.134102] agent systemd[1]: Reload requested from client PID 314 ('systemctl') (unit flakelet-other.service)...284agent # [ 9605.134155] agent systemd[1]: Reloading...285relay # [ 9604.998725] relay systemd[1]: Reloading finished in 341 ms.286relay # [ 9605.131965] relay systemd[1]: Finished Permit User Sessions.287relay # [ 9605.136396] relay systemd[1]: Started Console Getty.288relay # [ 9605.136459] relay systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0289relay # [ 9605.136528] relay systemd[1]: Reached target Login Prompts.290relay # [ 9605.153269] relay systemd[1]: Starting flakelet-relay...291agent # [ 9605.488225] agent systemd[1]: Reloading finished in 353 ms.292agent # [ 9605.569219] agent systemd[1]: Reached target Host and Network Name Lookups.293agent # [ 9605.569356] agent systemd[1]: Reached target User and Group Name Lookups.294agent # [ 9605.570721] agent systemd[1]: Starting User Login Management...295agent # [ 9605.571816] agent systemd[1]: Starting Permit User Sessions...296agent # [ 9605.575832] agent systemd[1]: Reload requested from client PID 406 ('systemctl') (unit flakelet-app.service)...297agent # [ 9605.575862] agent systemd[1]: Reloading...298relay # [ 9605.591220] relay nginx-pre-start[402]: nginx: the configuration file /nix/store/01avjj26wcihp926xgasszcj5s0k7cbs-nginx.conf syntax is ok299relay # [ 9605.591964] relay nginx-pre-start[402]: nginx: configuration file /nix/store/01avjj26wcihp926xgasszcj5s0k7cbs-nginx.conf test is successful300relay # [ 9605.609707] relay systemd[1]: Started Nginx Web Server.301relay # [ 9605.808784] relay systemd[1]: Starting D-Bus System Message Bus...302client # [ 9605.744854] client systemd-logind[315]: New seat seat0.303client # [ 9605.745350] client systemd[1]: Started User Login Management.304client # [ 9605.747595] client systemd[1]: Starting D-Bus System Message Bus...305client # [ 9605.748889] client systemd[1]: Starting linger-users.service...306client # [ 9605.792153] client systemd[1]: linger-users.service: Deactivated successfully.307client # [ 9605.792507] client systemd[1]: Finished linger-users.service.308client # [ 9605.792844] client systemd[1]: Reached target Multi-User System.309agent # [ 9605.968678] agent systemd[1]: Reloading finished in 392 ms.310agent # [ 9606.043375] agent systemd[1]: Finished Permit User Sessions.311agent # [ 9606.049232] agent systemd[1]: Started Console Getty.312agent # [ 9606.049346] agent systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0313agent # [ 9606.049499] agent systemd[1]: Reached target Login Prompts.314agent # [ 9606.051850] agent systemd[1]: Reload requested from client PID 408 ('systemctl') (unit flakelet-flakelet-agent.service)...315agent # [ 9606.051881] agent systemd[1]: Reloading...316relay # [ 9606.063697] relay systemd-logind[354]: New seat seat0.317relay # [ 9606.064125] relay systemd[1]: Started User Login Management.318relay # [ 9606.066634] relay systemd[1]: Starting linger-users.service...319relay # [ 9606.118124] relay systemd[1]: linger-users.service: Deactivated successfully.320relay # [ 9606.118323] relay systemd[1]: Finished linger-users.service.321client # [ 9606.299528] client dbus-broker-launch[325]: Looking up NSS user entry for 'systemd-timesync'...322client # [ 9606.300887] client dbus-broker-launch[325]: NSS returned no entry for 'systemd-timesync'323client # [ 9606.300887] client dbus-broker-launch[325]: Invalid user-name in /nix/store/5cfppnr9iv53mjjvcafddv4813rd7x9c-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"324client # [ 9606.301809] client systemd[1]: Started D-Bus System Message Bus.325client # [ 9606.302355] client systemd[1]: Startup finished in 4.119s.326client # [ 9606.308524] client dbus-broker-launch[325]: Ready327relay # [ 9606.324608] relay dbus-broker-launch[408]: Looking up NSS user entry for 'systemd-timesync'...328relay # [ 9606.325816] relay dbus-broker-launch[408]: NSS returned no entry for 'systemd-timesync'329relay # [ 9606.325816] relay dbus-broker-launch[408]: Invalid user-name in /nix/store/27h4g2jdw18vrabfh6p1l0n8ss5dhbp5-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"330relay # [ 9606.326871] relay systemd[1]: Started D-Bus System Message Bus.331relay # [ 9606.332785] relay dbus-broker-launch[408]: Ready332relay # [ 9606.336320] relay systemd[1]: Started Dnsmasq Daemon.333relay # [ 9606.337903] relay dnsmasq[394]: started, version 2.93 cachesize 150334relay # [ 9606.337910] relay dnsmasq[394]: compile time options: IPv6 GNU-getopt DBus no-UBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset nftset auth DNSSEC loop-detect inotify dumpfile335relay # [ 9606.337916] relay dnsmasq[394]: DBus support enabled: connected to system bus336relay # [ 9606.337923] relay dnsmasq[394]: no upstream servers configured - please set them from DBus337relay # [ 9606.338012] relay dnsmasq[394]: read /etc/hosts - 9 names338agent # [ 9606.399636] agent systemd[1]: Reloading finished in 347 ms.339agent # [ 9606.485175] agent systemd[1]: Reload requested from client PID 444 ('systemctl') (unit flakelet-other.service)...340agent # [ 9606.485219] agent systemd[1]: Reloading...341agent # [ 9606.830030] agent systemd[1]: Reloading finished in 344 ms.342agent # [ 9606.915740] agent systemd[1]: Starting app v1...343agent # [ 9606.925467] agent systemd[1]: Starting other v1...344agent # [ 9606.948163] agent systemd[1]: Starting flakelet-agent...345agent # [ 9606.951460] agent systemd[1]: Started app v1.346agent # [ 9606.951673] agent systemd[1]: Started other v1.347agent # [ 9606.979964] agent flakelet[308]: app: updated to generation 1348agent # [ 9606.980550] agent systemd[1]: Finished Update flakelet service app.349agent # [ 9606.984056] agent flakelet[310]: other: updated to generation 1350agent # [ 9606.985840] agent systemd[1]: Finished Update flakelet service other.351agent # [ 9606.992233] agent systemd-logind[405]: New seat seat0.352agent # [ 9606.993154] agent systemd[1]: Starting D-Bus System Message Bus...353agent # [ 9606.993272] agent systemd[1]: Started User Login Management.354agent # [ 9606.995284] agent systemd[1]: Starting linger-users.service...355agent # [ 9607.041629] agent systemd[1]: Started flakelet-agent.356agent # [ 9607.042369] agent systemd[1]: linger-users.service: Deactivated successfully.357agent # [ 9607.042492] agent systemd[1]: Finished linger-users.service.358agent # [ 9607.045449] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:45.007143Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}359agent # [ 9607.058546] agent flakelet[309]: flakelet-agent: updated to generation 1360agent # [ 9607.058998] agent systemd[1]: Finished Update flakelet service flakelet-agent.361agent # [ 9607.059921] agent systemd[1]: Reached target flakelet managed services.362agent # [ 9607.059990] agent systemd[1]: Reached target Multi-User System.363relay # [ 9607.228364] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:45.190000Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}364relay # [ 9607.228879] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:45.190638Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}365relay # [ 9607.228982] relay systemd[1]: Started flakelet-relay.366relay # [ 9607.238917] relay systemd[1]: Starting health for flakelet-relay...367agent # [ 9607.463313] agent dbus-broker-launch[898]: Looking up NSS user entry for 'systemd-timesync'...368agent # [ 9607.464808] agent dbus-broker-launch[898]: NSS returned no entry for 'systemd-timesync'369agent # [ 9607.464808] agent dbus-broker-launch[898]: Invalid user-name in /nix/store/q3c2gxfm178wqgzv8m87na4aa72wxc04-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"370agent # [ 9607.465462] agent systemd[1]: Started D-Bus System Message Bus.371agent # [ 9607.465764] agent systemd[1]: Startup finished in 5.282s.372agent # [ 9607.477586] agent dbus-broker-launch[898]: Ready373relay: (finished: waiting for unit flakelet-relay.service, in 6.16 seconds)374relay: waiting for unit nginx.service375relay: (finished: waiting for unit nginx.service, in 0.01 seconds)376agent: waiting for unit flakelet.target377agent: (finished: waiting for unit flakelet.target, in 0.02 seconds)378agent: waiting for unit flakelet-agent.service379agent: (finished: waiting for unit flakelet-agent.service, in 0.02 seconds)380client: waiting for unit multi-user.target381client: (finished: waiting for unit multi-user.target, in 0.02 seconds)382relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1'383relay # [ 9607.506852] relay systemd[1]: flakelet-relay-health.service: Deactivated successfully.384relay # [ 9607.507146] relay systemd[1]: Finished health for flakelet-relay.385relay # [ 9607.530809] relay flakelet[311]: flakelet-relay: updated to generation 1386relay # [ 9607.532646] relay systemd[1]: Finished Update flakelet service flakelet-relay.387relay # [ 9607.532912] relay systemd[1]: Reached target flakelet managed services.388relay # [ 9607.532997] relay systemd[1]: Reached target Multi-User System.389relay # [ 9607.533120] relay systemd[1]: Startup finished in 5.353s.390relay # [ 9609.064591] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:47.026321Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(1), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}391agent # [ 9609.064140] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:47.025915Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}392relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1', in 2.05 seconds)393agent: must succeed: systemctl is-active app.service other.service394agent: (finished: must succeed: systemctl is-active app.service other.service, in 0.01 seconds)395subtest: deploy with client certificate, relay found via SRV396agent: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json397agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)398client: must succeed: push --relay-srv test --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app 2>&1399agent # [ 9610.009483] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:47.971202Z","level":"INFO","message":"start","id":"a545a9efc6d4f2130cea8d5a1427f0d9","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}400agent # [ 9610.010481] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:47.972297Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"a545a9efc6d4f2130cea8d5a1427f0d9\"]","target":"flakelet_relay::agent::jobs"}401agent # [ 9610.077740] agent systemd[1]: Starting flakelet update app (via relay)...402agent # [ 9610.117211] agent systemd[1]: Started flakelet update app (via relay).403agent # [ 9610.121513] agent flakelet[959]: app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2404agent # [ 9610.122207] agent flakelet[959]: app: activating generation 2405agent # [ 9610.134848] agent systemd[1]: Stopping app v1...406agent # [ 9610.136553] agent systemd[1]: app.service: Deactivated successfully.407agent # [ 9610.136883] agent systemd[1]: Stopped app v1.408agent # [ 9610.154574] agent systemd[1]: Reload requested from client PID 966 ('systemctl') (unit flakelet-relay-job-app.service)...409agent # [ 9610.154642] agent systemd[1]: Reloading...410relay # [ 9610.008511] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:47.970238Z","level":"INFO","message":"deploy accepted","job":"3aa35c37e59eaf074982997d80ea382f","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}411agent # [ 9610.495591] agent systemd[1]: Reloading finished in 340 ms.412agent # [ 9610.587252] agent systemd[1]: Reload requested from client PID 996 ('systemctl') (unit flakelet-relay-job-app.service)...413agent # [ 9610.587291] agent systemd[1]: Reloading...414agent # [ 9610.909313] agent systemd[1]: Reloading finished in 321 ms.415agent # [ 9610.992510] agent systemd[1]: Reload requested from client PID 1025 ('systemctl') (unit flakelet-relay-job-app.service)...416agent # [ 9610.992551] agent systemd[1]: Reloading...417agent # [ 9611.320156] agent systemd[1]: Reloading finished in 327 ms.418agent # [ 9611.414321] agent systemd[1]: Reload requested from client PID 1059 ('systemctl') (unit flakelet-relay-job-app.service)...419agent # [ 9611.414359] agent systemd[1]: Reloading...420agent # [ 9611.736285] agent systemd[1]: Reloading finished in 321 ms.421agent # [ 9611.806286] agent systemd[1]: Starting app v2...422agent # [ 9611.835070] agent systemd[1]: Started app v2.423agent # [ 9611.845482] agent flakelet[959]: app: updated to generation 2424agent # [ 9611.846458] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.425client: (finished: must succeed: push --relay-srv test --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app 2>&1, in 3.00 seconds)426agent: must succeed: systemctl show app.service -p Description | grep -q 'app v2'427agent: (finished: must succeed: systemctl show app.service -p Description | grep -q 'app v2', in 0.02 seconds)428agent: must succeed: journalctl -u flakelet-relay-job-app.service | grep -q 'generation 2'429agent: (finished: must succeed: journalctl -u flakelet-relay-job-app.service | grep -q 'generation 2', in 0.01 seconds)430(finished: subtest: deploy with client certificate, relay found via SRV, in 3.04 seconds)431subtest: deploy with OIDC token, sub principal432agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json433agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)434client: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)435client: (finished: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.05 seconds)436client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5MCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3ODkxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJBMzc1VkNBR0VVUEk2PT09IiwibmJmIjoxNzg4Njg3ODkxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.acsV-fWTCs2atdlnUplgoKj2zgGgpbzGmjEMx7QVbS038c-ChDSSg_q2kAr2-OUjsxn2hXuVsRn-CRI96qkmDQ' push --relay https://relay:7443 deploy agent/app 2>&1437agent # [ 9612.976475] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:50.938214Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(2)","target":"flakelet_relay::agent::jobs"}438agent # [ 9613.106090] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:51.067843Z","level":"INFO","message":"start","id":"55e95ed5d514288d49c36b7bd1becf61","flakelet":"app","rule":"ci","relay":"relay","caller_name":"Some(\"repo:github:example/app:ref:refs/heads/main\")","target":"flakelet_relay::agent::conn"}439agent # [ 9613.106376] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:51.068186Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"55e95ed5d514288d49c36b7bd1becf61\"]","target":"flakelet_relay::agent::jobs"}440agent # [ 9613.149642] agent systemd[1]: Starting flakelet update app (via relay)...441agent # [ 9613.185081] agent systemd[1]: Started flakelet update app (via relay).442agent # [ 9613.190260] agent flakelet[1160]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1443agent # [ 9613.190984] agent flakelet[1160]: app: activating generation 3444agent # [ 9613.202961] agent systemd[1]: Stopping app v2...445agent # [ 9613.204302] agent systemd[1]: app.service: Deactivated successfully.446agent # [ 9613.204522] agent systemd[1]: Stopped app v2.447agent # [ 9613.217802] agent systemd[1]: Reload requested from client PID 1167 ('systemctl') (unit flakelet-relay-job-app.service)...448agent # [ 9613.217860] agent systemd[1]: Reloading...449relay # [ 9612.978083] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:50.939756Z","level":"INFO","message":"deploy finished","job":"3aa35c37e59eaf074982997d80ea382f","ok":true,"target":"flakelet_relay::relay::api"}450relay # [ 9613.105746] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:51.067515Z","level":"INFO","message":"deploy accepted","job":"8680abbd92bba6531427a6116622e760","caller":"repo:github:example/app:ref:refs/heads/main","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}451agent # [ 9613.550665] agent systemd[1]: Reloading finished in 332 ms.452agent # [ 9613.633247] agent systemd[1]: Reload requested from client PID 1197 ('systemctl') (unit flakelet-relay-job-app.service)...453agent # [ 9613.633286] agent systemd[1]: Reloading...454agent # [ 9613.959766] agent systemd[1]: Reloading finished in 326 ms.455agent # [ 9614.052531] agent systemd[1]: Reload requested from client PID 1226 ('systemctl') (unit flakelet-relay-job-app.service)...456agent # [ 9614.052571] agent systemd[1]: Reloading...457agent # [ 9614.374874] agent systemd[1]: Reloading finished in 321 ms.458agent # [ 9614.442758] agent systemd[1]: Reload requested from client PID 1257 ('systemctl') (unit flakelet-relay-job-app.service)...459agent # [ 9614.442794] agent systemd[1]: Reloading...460agent # [ 9614.766894] agent systemd[1]: Reloading finished in 323 ms.461agent # [ 9614.863875] agent systemd[1]: Starting app v1...462agent # [ 9614.893024] agent systemd[1]: Started app v1.463agent # [ 9614.907825] agent flakelet[1160]: app: updated to generation 3464agent # [ 9614.909900] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.465client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5MCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3ODkxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJBMzc1VkNBR0VVUEk2PT09IiwibmJmIjoxNzg4Njg3ODkxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.acsV-fWTCs2atdlnUplgoKj2zgGgpbzGmjEMx7QVbS038c-ChDSSg_q2kAr2-OUjsxn2hXuVsRn-CRI96qkmDQ' push --relay https://relay:7443 deploy agent/app 2>&1, in 2.93 seconds)466(finished: subtest: deploy with OIDC token, sub principal, in 3.00 seconds)467subtest: policy denies uncovered target468client: must fail: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5MCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3ODkxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJBMzc1VkNBR0VVUEk2PT09IiwibmJmIjoxNzg4Njg3ODkxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.acsV-fWTCs2atdlnUplgoKj2zgGgpbzGmjEMx7QVbS038c-ChDSSg_q2kAr2-OUjsxn2hXuVsRn-CRI96qkmDQ' push --relay https://relay:7443 deploy agent/other 2>&1469client: (finished: must fail: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5MCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3ODkxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJBMzc1VkNBR0VVUEk2PT09IiwibmJmIjoxNzg4Njg3ODkxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.acsV-fWTCs2atdlnUplgoKj2zgGgpbzGmjEMx7QVbS038c-ChDSSg_q2kAr2-OUjsxn2hXuVsRn-CRI96qkmDQ' push --relay https://relay:7443 deploy agent/other 2>&1, in 0.02 seconds)470(finished: subtest: policy denies uncovered target, in 0.02 seconds)471subtest: claim principal from groups, unchanged counts as success472client: must succeed: echo '{"groups": ["wheel"]}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'someone' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)473client: (finished: must succeed: echo '{"groups": ["wheel"]}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'someone' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.05 seconds)474client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5NCwiZ3JvdXBzIjpbIndoZWVsIl0sImlhdCI6MTc4ODY4Nzg5NCwiaXNzIjoiaHR0cHM6Ly9yZWxheTo4NDQzIiwianRpIjoiWk1TVTZQMjJDRDVNTz09PSIsIm5iZiI6MTc4ODY4Nzg5NCwic3ViIjoic29tZW9uZSJ9.NdYpb78dD4bfhM4pgUNGi03z8pT9nGHtXny379QZyz_QTPBFZDzS0ekmZGStbbPKc52-ZtMFt3NgC64JICEtCA' push --relay https://relay:7443 deploy agent/other 2>&1475agent # [ 9616.003999] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:53.965744Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}476agent # [ 9616.102883] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:54.064676Z","level":"INFO","message":"start","id":"ae0de93b75e827ff3c5fcc67ba222394","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"someone\")","target":"flakelet_relay::agent::conn"}477agent # [ 9616.103455] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:54.065281Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"ae0de93b75e827ff3c5fcc67ba222394\"]","target":"flakelet_relay::agent::jobs"}478agent # [ 9616.172598] agent systemd[1]: Starting flakelet update other (via relay)...479agent # [ 9616.209026] agent systemd[1]: Started flakelet update other (via relay).480agent # [ 9616.213515] agent flakelet[1338]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1481agent # [ 9616.214083] agent flakelet[1338]: other: up to date482agent # [ 9616.216485] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.483relay # [ 9616.005594] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:53.967331Z","level":"INFO","message":"deploy finished","job":"8680abbd92bba6531427a6116622e760","ok":true,"target":"flakelet_relay::relay::api"}484relay # [ 9616.102615] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:54.064414Z","level":"INFO","message":"deploy accepted","job":"02c5ac753196a07ebb8cf94b7f603650","caller":"someone","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}485client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODE5NCwiZ3JvdXBzIjpbIndoZWVsIl0sImlhdCI6MTc4ODY4Nzg5NCwiaXNzIjoiaHR0cHM6Ly9yZWxheTo4NDQzIiwianRpIjoiWk1TVTZQMjJDRDVNTz09PSIsIm5iZiI6MTc4ODY4Nzg5NCwic3ViIjoic29tZW9uZSJ9.NdYpb78dD4bfhM4pgUNGi03z8pT9nGHtXny379QZyz_QTPBFZDzS0ekmZGStbbPKc52-ZtMFt3NgC64JICEtCA' push --relay https://relay:7443 deploy agent/other 2>&1, in 0.70 seconds)486(finished: subtest: claim principal from groups, unchanged counts as success, in 0.75 seconds)487subtest: push login via device flow, cached token used afterwards488client: must succeed: push login --issuer https://relay:8443 --client-id flakelet-relay 2>&1489relay # [ 9616.783592] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:54.745333Z","level":"INFO","message":"deploy finished","job":"02c5ac753196a07ebb8cf94b7f603650","ok":true,"target":"flakelet_relay::relay::api"}490relay # [ 9616.804817] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:44:54] "POST /device HTTP/1.1" 200 -491agent # [ 9616.782308] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:54.744063Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}492relay # [ 9617.810168] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:44:55] "POST /token HTTP/1.1" 400 -493client: (finished: must succeed: push login --issuer https://relay:8443 --client-id flakelet-relay 2>&1, in 2.08 seconds)494client: must succeed: test "$(stat -c %a /root/.local/state/flakelet-push/token.json)" = 600495client: (finished: must succeed: test "$(stat -c %a /root/.local/state/flakelet-push/token.json)" = 600, in 0.01 seconds)496client: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1497relay # [ 9618.860156] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:44:56] "POST /token HTTP/1.1" 200 -498relay # [ 9618.888991] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:56.850781Z","level":"INFO","message":"deploy accepted","job":"a72041d7fe18aade8f793775c6d74383","caller":"dev@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}499agent # [ 9618.889318] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:56.851096Z","level":"INFO","message":"start","id":"2df89cb357d1e6e88e4b1a680393be0c","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"dev@example.org\")","target":"flakelet_relay::agent::conn"}500agent # [ 9618.890118] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:56.851737Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"2df89cb357d1e6e88e4b1a680393be0c\"]","target":"flakelet_relay::agent::jobs"}501agent # [ 9618.953429] agent systemd[1]: Starting flakelet update other (via relay)...502agent # [ 9618.990231] agent systemd[1]: Started flakelet update other (via relay).503agent # [ 9618.993857] agent flakelet[1388]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1504agent # [ 9618.994400] agent flakelet[1388]: other: up to date505agent # [ 9618.997311] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.506client: (finished: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1, in 0.69 seconds)507client: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1508relay # [ 9619.564891] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:57.526629Z","level":"INFO","message":"deploy finished","job":"a72041d7fe18aade8f793775c6d74383","ok":true,"target":"flakelet_relay::relay::api"}509relay # [ 9619.584235] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:57.546039Z","level":"INFO","message":"deploy accepted","job":"dd20960cf993670ab011469990176db9","caller":"dev@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}510agent # [ 9619.563462] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:57.525203Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}511agent # [ 9619.584496] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:57.546304Z","level":"INFO","message":"start","id":"ff913cd8361cf0eb83cf85bd406f9c8f","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"dev@example.org\")","target":"flakelet_relay::agent::conn"}512agent # [ 9619.584871] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:57.546693Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"ff913cd8361cf0eb83cf85bd406f9c8f\"]","target":"flakelet_relay::agent::jobs"}513agent # [ 9619.648521] agent systemd[1]: Starting flakelet update other (via relay)...514agent # [ 9619.686129] agent systemd[1]: Started flakelet update other (via relay).515agent # [ 9619.690105] agent flakelet[1437]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1516agent # [ 9619.690659] agent flakelet[1437]: other: up to date517agent # [ 9619.691556] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.518client: (finished: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1, in 0.70 seconds)519(finished: subtest: push login via device flow, cached token used afterwards, in 3.48 seconds)520subtest: waves: failing first wave skips the second521agent: must succeed: jq '.services.app.prebuilt="/nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json522agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)523client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave agent/other 2>&1524agent # [ 9620.258461] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:58.220204Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}525agent # [ 9620.292319] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:58.254116Z","level":"INFO","message":"start","id":"d9b8e1683093e5b08b9ee45cc2a8c931","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}526agent # [ 9620.292716] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:44:58.254538Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"d9b8e1683093e5b08b9ee45cc2a8c931\"]","target":"flakelet_relay::agent::jobs"}527agent # [ 9620.349701] agent systemd[1]: Starting flakelet update app (via relay)...528agent # [ 9620.388120] agent systemd[1]: Started flakelet update app (via relay).529agent # [ 9620.393073] agent flakelet[1477]: app: using prebuilt artifact /nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken530agent # [ 9620.393740] agent flakelet[1477]: app: activating generation 4531agent # [ 9620.404788] agent systemd[1]: Stopping app v1...532agent # [ 9620.406140] agent systemd[1]: app.service: Deactivated successfully.533agent # [ 9620.406361] agent systemd[1]: Stopped app v1.534agent # [ 9620.420648] agent systemd[1]: Reload requested from client PID 1484 ('systemctl') (unit flakelet-relay-job-app.service)...535agent # [ 9620.420707] agent systemd[1]: Reloading...536relay # [ 9620.259864] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:58.221589Z","level":"INFO","message":"deploy finished","job":"dd20960cf993670ab011469990176db9","ok":true,"target":"flakelet_relay::relay::api"}537relay # [ 9620.292064] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:44:58.253872Z","level":"INFO","message":"deploy accepted","job":"56aec3139ffb9895cad78fd920f943b6","caller":"admin@example.org","targets":"[\"agent/app\", \"agent/other\"]","target":"flakelet_relay::relay::api"}538agent # [ 9620.752554] agent systemd[1]: Reloading finished in 331 ms.539agent # [ 9620.819850] agent systemd[1]: Reload requested from client PID 1514 ('systemctl') (unit flakelet-relay-job-app.service)...540agent # [ 9620.819886] agent systemd[1]: Reloading...541agent # [ 9621.147008] agent systemd[1]: Reloading finished in 326 ms.542agent # [ 9621.227745] agent systemd[1]: Reload requested from client PID 1545 ('systemctl') (unit flakelet-relay-job-app.service)...543agent # [ 9621.227782] agent systemd[1]: Reloading...544agent # [ 9621.552874] agent systemd[1]: Reloading finished in 324 ms.545agent # [ 9621.632640] agent systemd[1]: Reload requested from client PID 1579 ('systemctl') (unit flakelet-relay-job-app.service)...546agent # [ 9621.632676] agent systemd[1]: Reloading...547agent # [ 9621.958440] agent systemd[1]: Reloading finished in 325 ms.548agent # [ 9622.040561] agent systemd[1]: Starting app broken...549agent # [ 9622.077380] agent (nonexistent)[1611]: app.service: Unable to locate executable '/nonexistent': No such file or directory550agent # [ 9622.077405] agent (nonexistent)[1611]: app.service: Failed at step EXEC spawning /nonexistent: No such file or directory551agent # [ 9622.079402] agent systemd[1]: app.service: Main process exited, code=exited, status=203/EXEC552agent # [ 9622.079499] agent systemd[1]: app.service: Failed with result 'exit-code'.553agent # [ 9622.079671] agent systemd[1]: Failed to start app broken.554agent # [ 9622.108583] agent systemd[1]: Reload requested from client PID 1625 ('systemctl') (unit flakelet-relay-job-app.service)...555agent # [ 9622.108620] agent systemd[1]: Reloading...556agent # [ 9622.437760] agent systemd[1]: Reloading finished in 328 ms.557agent # [ 9622.501574] agent systemd[1]: Reload requested from client PID 1655 ('systemctl') (unit flakelet-relay-job-app.service)...558agent # [ 9622.501610] agent systemd[1]: Reloading...559agent # [ 9622.825726] agent systemd[1]: Reloading finished in 323 ms.560agent # [ 9622.909167] agent systemd[1]: Reload requested from client PID 1685 ('systemctl') (unit flakelet-relay-job-app.service)...561agent # [ 9622.909202] agent systemd[1]: Reloading...562agent # [ 9623.231471] agent systemd[1]: Reloading finished in 321 ms.563agent # [ 9623.320332] agent systemd[1]: Reload requested from client PID 1718 ('systemctl') (unit flakelet-relay-job-app.service)...564agent # [ 9623.320367] agent systemd[1]: Reloading...565agent # [ 9623.642862] agent systemd[1]: Reloading finished in 322 ms.566agent # [ 9623.709178] agent systemd[1]: Starting app v1...567agent # [ 9623.738040] agent systemd[1]: Started app v1.568agent # [ 9623.739267] agent flakelet[1477]: app: deploy failed, rolled back: systemctl start app.service failed:569agent # [ 9623.739267] agent flakelet[1477]: failed units: app.service. See journalctl -u app.service570agent # [ 9623.740852] agent systemd[1]: flakelet-relay-job-app.service: Main process exited, code=exited, status=1/FAILURE571agent # [ 9623.740946] agent systemd[1]: flakelet-relay-job-app.service: Failed with result 'exit-code'.572client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave agent/other 2>&1, in 4.21 seconds)573agent: must succeed: systemctl show app.service -p Description | grep -q 'app v1'574agent: (finished: must succeed: systemctl show app.service -p Description | grep -q 'app v1', in 0.02 seconds)575(finished: subtest: waves: failing first wave skips the second, in 4.24 seconds)576subtest: host patterns expand on the relay577agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json578agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)579client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/app' 2>&1580agent # [ 9624.480143] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:02.441884Z","level":"INFO","message":"update finished","flakelet":"app","status":"RolledBack","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}581agent # [ 9624.536527] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:02.498319Z","level":"INFO","message":"start","id":"c084fbf4d01d441195874e85788fd123","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}582agent # [ 9624.536974] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:02.498792Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"c084fbf4d01d441195874e85788fd123\"]","target":"flakelet_relay::agent::jobs"}583agent # [ 9624.591225] agent systemd[1]: Starting flakelet update app (via relay)...584agent # [ 9624.626035] agent systemd[1]: Started flakelet update app (via relay).585agent # [ 9624.629668] agent flakelet[1801]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1586agent # [ 9624.630257] agent flakelet[1801]: app: up to date587agent # [ 9624.632714] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.588relay # [ 9624.481360] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:02.443147Z","level":"INFO","message":"deploy finished","job":"56aec3139ffb9895cad78fd920f943b6","ok":false,"target":"flakelet_relay::relay::api"}589relay # [ 9624.536230] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:02.497987Z","level":"INFO","message":"deploy accepted","job":"40d0adcc8c1b1331239c07682fde03f6","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}590client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/app' 2>&1, in 0.68 seconds)591client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '*/app' 2>&1592relay # [ 9625.198483] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:03.160270Z","level":"INFO","message":"deploy finished","job":"40d0adcc8c1b1331239c07682fde03f6","ok":true,"target":"flakelet_relay::relay::api"}593relay # [ 9625.220546] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:03.182344Z","level":"INFO","message":"deploy accepted","job":"5eccb012153856d19cf624475c4572cb","caller":"admin@example.org","targets":"[\"agent/app\", \"ghost/app\"]","target":"flakelet_relay::relay::api"}594agent # [ 9625.197205] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.158940Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}595agent # [ 9625.220796] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.182602Z","level":"INFO","message":"start","id":"02052e4e1b1b0f4e26a4db34dfed3c54","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}596agent # [ 9625.221216] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.183036Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"02052e4e1b1b0f4e26a4db34dfed3c54\"]","target":"flakelet_relay::agent::jobs"}597agent # [ 9625.288480] agent systemd[1]: Starting flakelet update app (via relay)...598agent # [ 9625.320222] agent systemd[1]: Started flakelet update app (via relay).599agent # [ 9625.324233] agent flakelet[1850]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1600agent # [ 9625.324772] agent flakelet[1850]: app: up to date601agent # [ 9625.326614] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.602client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '*/app' 2>&1, in 0.69 seconds)603client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave '*/app' 2>&1604relay # [ 9625.887787] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:03.849565Z","level":"INFO","message":"deploy finished","job":"5eccb012153856d19cf624475c4572cb","ok":false,"target":"flakelet_relay::relay::api"}605relay # [ 9625.912366] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:03.874170Z","level":"INFO","message":"deploy accepted","job":"3f6c243b083622947815c325bc60a773","caller":"admin@example.org","targets":"[\"agent/app\", \"ghost/app\"]","target":"flakelet_relay::relay::api"}606agent # [ 9625.886552] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.848286Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}607agent # [ 9625.912639] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.874443Z","level":"INFO","message":"start","id":"13743ac4391a9051e03e0ccd5d136b0e","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}608agent # [ 9625.913058] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:03.874878Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"13743ac4391a9051e03e0ccd5d136b0e\"]","target":"flakelet_relay::agent::jobs"}609agent # [ 9625.982102] agent systemd[1]: Starting flakelet update app (via relay)...610agent # [ 9626.020101] agent systemd[1]: Started flakelet update app (via relay).611agent # [ 9626.024461] agent flakelet[1886]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1612agent # [ 9626.024976] agent flakelet[1886]: app: up to date613agent # [ 9626.026743] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.614client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave '*/app' 2>&1, in 0.70 seconds)615client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/nope' 2>&1616client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/nope' 2>&1, in 0.02 seconds)617(finished: subtest: host patterns expand on the relay, in 2.11 seconds)618subtest: detach returns once the job is accepted619agent: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json620agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)621client: must succeed: timeout 20 push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --detach agent/app 2>&1622client: (finished: must succeed: timeout 20 push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --detach agent/app 2>&1, in 0.02 seconds)623agent: waiting for success: systemctl is-active flakelet-relay-job-app.service624relay # [ 9626.589313] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:04.551092Z","level":"INFO","message":"deploy finished","job":"3f6c243b083622947815c325bc60a773","ok":false,"target":"flakelet_relay::relay::api"}625relay # [ 9626.645164] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:04.606914Z","level":"INFO","message":"deploy accepted","job":"3015bdc86c3442a89c4b3e6a9bf00aaf","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}626agent # [ 9626.588056] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:04.549795Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}627agent # [ 9626.645394] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:04.607138Z","level":"INFO","message":"start","id":"55699d1ca2dee22c746350319ed619df","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}628agent # [ 9626.645868] agent flakelet-agent[504]: {"timestamp":"2026-09-06T09:45:04.607684Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"55699d1ca2dee22c746350319ed619df\"]","target":"flakelet_relay::agent::jobs"}629agent # [ 9626.711083] agent systemd[1]: Starting flakelet update app (via relay)...630agent # [ 9626.745400] agent systemd[1]: Started flakelet update app (via relay).631agent # [ 9626.750108] agent flakelet[1927]: app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2632agent # [ 9626.750781] agent flakelet[1927]: app: activating generation 4633agent # [ 9626.763261] agent systemd[1]: Stopping app v1...634agent # [ 9626.763630] agent systemd[1]: app.service: Deactivated successfully.635agent # [ 9626.763854] agent systemd[1]: Stopped app v1.636agent # [ 9626.778352] agent systemd[1]: Reload requested from client PID 1934 ('systemctl') (unit flakelet-relay-job-app.service)...637agent # [ 9626.778408] agent systemd[1]: Reloading...638agent # [ 9627.110281] agent systemd[1]: Reloading finished in 331 ms.639agent # [ 9627.197122] agent systemd[1]: Reload requested from client PID 1967 ('systemctl') (unit flakelet-relay-job-app.service)...640agent # [ 9627.197160] agent systemd[1]: Reloading...641agent # [ 9627.521768] agent systemd[1]: Reloading finished in 324 ms.642agent # [ 9627.599800] agent systemd[1]: Reload requested from client PID 1998 ('systemctl') (unit flakelet-relay-job-app.service)...643agent # [ 9627.599834] agent systemd[1]: Reloading...644agent # [ 9627.925477] agent systemd[1]: Reloading finished in 325 ms.645agent: (finished: waiting for success: systemctl is-active flakelet-relay-job-app.service, in 1.35 seconds)646??? Warning (UserWarning): wait_until_fails(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.647 File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39648agent: waiting for failure: systemctl is-active flakelet-relay-job-app.service649??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.650 File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39651agent # [ 9627.997241] agent systemd[1]: Reload requested from client PID 2032 ('systemctl') (unit flakelet-relay-job-app.service)...652agent # [ 9627.997275] agent systemd[1]: Reloading...653agent # [ 9628.322627] agent systemd[1]: Reloading finished in 325 ms.654agent # [ 9628.421963] agent systemd[1]: Starting app slow2...655agent: (finished: waiting for failure: systemctl is-active flakelet-relay-job-app.service, in 6.51 seconds)656(finished: subtest: detach returns once the job is accepted, in 7.89 seconds)657subtest: unknown host is final, missing agent is retried, agents listing658client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy nope/app 2>&1659client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy nope/app 2>&1, in 0.02 seconds)660agent: must succeed: systemctl stop flakelet-agent661agent: (finished: must succeed: systemctl stop flakelet-agent, in 0.06 seconds)662relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 0'663relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 0', in 0.02 seconds)664client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/other > /tmp/retry.out 2>&1 &665client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/other > /tmp/retry.out 2>&1 &, in 0.01 seconds)666client: waiting for success: grep -q 'agent_unavailable.*retrying' /tmp/retry.out667agent # [ 9634.471107] agent systemd[1]: Started app slow2.668agent # [ 9634.490816] agent flakelet[1927]: app: updated to generation 4669agent # [ 9634.493801] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.670agent # [ 9634.533466] agent systemd[1]: Stopping flakelet-agent...671agent # [ 9634.568821] agent systemd[1]: flakelet-agent.service: Deactivated successfully.672agent # [ 9634.569125] agent systemd[1]: Stopped flakelet-agent.673agent # [ 9634.569429] agent systemd[1]: flakelet-agent.service: Consumed 2.437s CPU time over 27.620s wall clock time.674relay # [ 9634.534098] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:12.495829Z","level":"INFO","message":"agent disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","host":"agent","target":"flakelet_relay::relay::agent_conn"}675client: (finished: waiting for success: grep -q 'agent_unavailable.*retrying' /tmp/retry.out, in 1.02 seconds)676agent: must succeed: systemctl start flakelet-agent677agent: (finished: must succeed: systemctl start flakelet-agent, in 0.14 seconds)678??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.679 File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39680client: waiting for success: grep -q 'agent/other: unchanged' /tmp/retry.out681agent # [ 9635.642345] agent systemd[1]: Starting flakelet-agent...682agent # [ 9635.699430] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:13.661178Z","level":"INFO","message":"reattaching to running unit","flakelet":"app","ids":"[\"55699d1ca2dee22c746350319ed619df\"]","target":"flakelet_relay::agent::jobs"}683agent # [ 9635.771776] agent systemd[1]: Started flakelet-agent.684agent # [ 9635.781648] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:13.743460Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}685relay # [ 9635.781849] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:13.743562Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(4), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}686agent # [ 9636.266944] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:14.228681Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(4)","target":"flakelet_relay::agent::jobs"}687relay # [ 9637.632374] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:15.594115Z","level":"INFO","message":"deploy accepted","job":"a83facd19d92b3ea501c99e5f1689915","caller":"admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}688agent # [ 9637.632845] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:15.594587Z","level":"INFO","message":"start","id":"e0a706f85f3c8d2579755112c2fb6c5c","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}689agent # [ 9637.633710] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:15.595321Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"e0a706f85f3c8d2579755112c2fb6c5c\"]","target":"flakelet_relay::agent::jobs"}690agent # [ 9637.696228] agent systemd[1]: Starting flakelet update other (via relay)...691agent # [ 9637.731072] agent systemd[1]: Started flakelet update other (via relay).692agent # [ 9637.735211] agent flakelet[2582]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1693agent # [ 9637.735763] agent flakelet[2582]: other: up to date694agent # [ 9637.738762] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.695relay # [ 9638.307964] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:16.269672Z","level":"INFO","message":"deploy finished","job":"a83facd19d92b3ea501c99e5f1689915","ok":true,"target":"flakelet_relay::relay::api"}696agent # [ 9638.306352] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:16.268089Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}697client: (finished: waiting for success: grep -q 'agent/other: unchanged' /tmp/retry.out, in 3.04 seconds)698client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents699client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents, in 0.02 seconds)700client: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)701client: (finished: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.06 seconds)702client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODIxNiwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTE2LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJNTzJTS1BMQk03M0dVPT09IiwibmJmIjoxNzg4Njg3OTE2LCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.HbnlIqBBWUlA6fUBm1mSrd-bTOq7F4doBf-dqWC0PxGivtbyG1LnaxlEOU5O3mekAS7Fcml7XUcBdRRQx78UCQ' push --relay https://relay:7443 agents703client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODIxNiwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTE2LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJNTzJTS1BMQk03M0dVPT09IiwibmJmIjoxNzg4Njg3OTE2LCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.HbnlIqBBWUlA6fUBm1mSrd-bTOq7F4doBf-dqWC0PxGivtbyG1LnaxlEOU5O3mekAS7Fcml7XUcBdRRQx78UCQ' push --relay https://relay:7443 agents, in 0.02 seconds)704(finished: subtest: unknown host is final, missing agent is retried, agents listing, in 4.41 seconds)705subtest: out-of-band flakelet update reaches the relay without reconnect706agent: must succeed: flakelet status --json app | jq '.[0].generation'707agent: (finished: must succeed: flakelet status --json app | jq '.[0].generation', in 0.06 seconds)708agent: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true709agent: (finished: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true, in 0.01 seconds)710agent: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json711agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)712agent: must succeed: flakelet update app713app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2714app: activating generation 5715agent # [ 9639.015646] agent systemd[1]: Stopping app slow2...716agent # [ 9639.016144] agent systemd[1]: app.service: Deactivated successfully.717agent # [ 9639.016416] agent systemd[1]: Stopped app slow2.718agent # [ 9639.034916] agent systemd[1]: Reload requested from client PID 2642 ('systemctl')...719agent # [ 9639.034985] agent systemd[1]: Reloading...720agent # [ 9639.372376] agent systemd[1]: Reloading finished in 336 ms.721agent # [ 9639.443193] agent systemd[1]: Reload requested from client PID 2671 ('systemctl')...722agent # [ 9639.443230] agent systemd[1]: Reloading...723agent # [ 9639.764668] agent systemd[1]: Reloading finished in 321 ms.724agent # [ 9639.854201] agent systemd[1]: Reload requested from client PID 2701 ('systemctl')...725agent # [ 9639.854246] agent systemd[1]: Reloading...726agent # [ 9640.179192] agent systemd[1]: Reloading finished in 324 ms.727agent # [ 9640.246894] agent systemd[1]: Reload requested from client PID 2732 ('systemctl')...728agent # [ 9640.246929] agent systemd[1]: Reloading...729agent: (finished: must succeed: flakelet update app, in 1.69 seconds)730client: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@5'731agent # [ 9640.569690] agent systemd[1]: Reloading finished in 322 ms.732agent # [ 9640.637272] agent systemd[1]: Starting app v2...733agent # [ 9640.671078] agent systemd[1]: Started app v2.734client: (finished: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@5', in 2.06 seconds)735agent: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true736agent: (finished: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true, in 0.01 seconds)737agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json738agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)739agent: must succeed: flakelet update app740app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1741app: activating generation 6742agent # [ 9642.792287] agent systemd[1]: Stopping app v2...743agent # [ 9642.792754] agent systemd[1]: app.service: Deactivated successfully.744agent # [ 9642.793036] agent systemd[1]: Stopped app v2.745agent # [ 9642.810069] agent systemd[1]: Reload requested from client PID 2799 ('systemctl')...746agent # [ 9642.810137] agent systemd[1]: Reloading...747agent # [ 9643.150998] agent systemd[1]: Reloading finished in 340 ms.748agent # [ 9643.215659] agent systemd[1]: Reload requested from client PID 2828 ('systemctl')...749agent # [ 9643.215696] agent systemd[1]: Reloading...750agent # [ 9643.536843] agent systemd[1]: Reloading finished in 320 ms.751agent # [ 9643.599328] agent systemd[1]: Reload requested from client PID 2856 ('systemctl')...752agent # [ 9643.599365] agent systemd[1]: Reloading...753agent # [ 9643.923171] agent systemd[1]: Reloading finished in 323 ms.754agent # [ 9643.994471] agent systemd[1]: Reload requested from client PID 2888 ('systemctl')...755agent # [ 9643.994509] agent systemd[1]: Reloading...756agent: (finished: must succeed: flakelet update app, in 1.66 seconds)757client: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@6'758agent # [ 9644.317348] agent systemd[1]: Reloading finished in 322 ms.759agent # [ 9644.382318] agent systemd[1]: Starting app v1...760agent # [ 9644.413725] agent systemd[1]: Started app v1.761client: (finished: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@6', in 2.06 seconds)762(finished: subtest: out-of-band flakelet update reaches the relay without reconnect, in 7.58 seconds)763subtest: certificate not listed under agents cannot connect as agent764client: must succeed: curl -s -o /dev/null -w '%{http_code}' --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/cert.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/key.pem -H 'Upgrade: websocket' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' https://relay:7443/v1/agent765client: (finished: must succeed: curl -s -o /dev/null -w '%{http_code}' --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/cert.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/key.pem -H 'Upgrade: websocket' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' https://relay:7443/v1/agent, in 0.02 seconds)766(finished: subtest: certificate not listed under agents cannot connect as agent, in 0.02 seconds)767subtest: metrics768relay: must succeed: curl -sf http://127.0.0.1:7400/metrics769relay: (finished: must succeed: curl -sf http://127.0.0.1:7400/metrics, in 0.02 seconds)770(finished: subtest: metrics, in 0.02 seconds)771subtest: agent link drop mid-run replays missed lines once772agent: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json773agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)774agent: must succeed: date +%s775agent: (finished: must succeed: date +%s, in 0.01 seconds)776client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/slow.out 2>&1 &777client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/slow.out 2>&1 &, in 0.01 seconds)778agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687924 | grep -q 'app-slow'779relay # [ 9646.509062] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:24.470808Z","level":"INFO","message":"agent rejected: no agents entry matches","principals":"[\"x509:dns:mallory\"]","target":"flakelet_relay::relay::agent_conn"}780relay # [ 9646.567733] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:24.529479Z","level":"INFO","message":"deploy accepted","job":"440f9582a38de6779d9609d608b774a0","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}781agent # [ 9646.568095] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:24.529758Z","level":"INFO","message":"start","id":"0df49d05b7e4e93b26d0ba61f2169a2a","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}782agent # [ 9646.568877] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:24.530441Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"0df49d05b7e4e93b26d0ba61f2169a2a\"]","target":"flakelet_relay::agent::jobs"}783agent # [ 9646.616797] agent systemd[1]: Starting flakelet update app (via relay)...784agent # [ 9646.646070] agent systemd[1]: Started flakelet update app (via relay).785agent # [ 9646.650196] agent flakelet[2962]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow786agent # [ 9646.650850] agent flakelet[2962]: app: activating generation 7787agent # [ 9646.658976] agent systemd[1]: Stopping app v1...788agent # [ 9646.659690] agent systemd[1]: app.service: Deactivated successfully.789agent # [ 9646.659818] agent systemd[1]: Stopped app v1.790agent # [ 9646.670123] agent systemd[1]: Reload requested from client PID 2969 ('systemctl') (unit flakelet-relay-job-app.service)...791agent # [ 9646.670161] agent systemd[1]: Reloading...792agent # [ 9646.993737] agent systemd[1]: Reloading finished in 323 ms.793agent # [ 9647.058655] agent systemd[1]: Reload requested from client PID 2999 ('systemctl') (unit flakelet-relay-job-app.service)...794agent # [ 9647.058692] agent systemd[1]: Reloading...795agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687924 | grep -q 'app-slow', in 1.02 seconds)796agent: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route add unreachable $a; done797agent: (finished: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route add unreachable $a; done, in 0.02 seconds)798relay: must succeed: for a in $(getent ahosts agent | cut -d' ' -f1 | sort -u); do ss -K dst "[$a]"; done799relay: (finished: must succeed: for a in $(getent ahosts agent | cut -d' ' -f1 | sort -u); do ss -K dst "[$a]"; done, in 0.06 seconds)800agent: must succeed: ss -K dport = 7443801agent # [ 9647.381714] agent systemd[1]: Reloading finished in 322 ms.802agent # [ 9647.445982] agent systemd[1]: Reload requested from client PID 3028 ('systemctl') (unit flakelet-relay-job-app.service)...803agent # [ 9647.446027] agent systemd[1]: Reloading...804agent: (finished: must succeed: ss -K dport = 7443, in 0.03 seconds)805agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687924 | grep -q 'updated to generation'806relay # [ 9647.651228] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:25.612964Z","level":"INFO","message":"agent disconnected: IO error: Software caused connection abort (os error 103)","host":"agent","target":"flakelet_relay::relay::agent_conn"}807agent # [ 9647.684446] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:25.645988Z","level":"INFO","message":"disconnected: IO error: Software caused connection abort (os error 103)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}808agent # [ 9647.769252] agent systemd[1]: Reloading finished in 322 ms.809agent # [ 9647.852344] agent systemd[1]: Reload requested from client PID 3076 ('systemctl') (unit flakelet-relay-job-app.service)...810agent # [ 9647.852380] agent systemd[1]: Reloading...811agent # [ 9648.175845] agent systemd[1]: Reloading finished in 323 ms.812agent # [ 9648.242016] agent systemd[1]: Starting app slow...813agent # [ 9648.687461] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:26.649213Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}814agent # [ 9650.689886] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:28.651638Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}815agent # [ 9654.309039] agent systemd[1]: Started app slow.816agent # [ 9654.328535] agent flakelet[2962]: app: updated to generation 7817agent # [ 9654.331209] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.818agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687924 | grep -q 'updated to generation', in 7.10 seconds)819agent: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route del unreachable $a; done820agent: (finished: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route del unreachable $a; done, in 0.02 seconds)821client: waiting for success: grep -q 'agent/app: updated' /tmp/slow.out822agent # [ 9654.692108] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:32.653640Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}823agent # [ 9655.462958] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:33.424697Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(7)","target":"flakelet_relay::agent::jobs"}824client: (finished: waiting for success: grep -q 'agent/app: updated' /tmp/slow.out, in 8.09 seconds)825client: must succeed: cat /tmp/slow.out826client: (finished: must succeed: cat /tmp/slow.out, in 0.01 seconds)827(finished: subtest: agent link drop mid-run replays missed lines once, in 16.38 seconds)828subtest: agent restart mid-run reattaches to the unit829agent: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json830agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)831client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/restart.out 2>&1 &832client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/restart.out 2>&1 &, in 0.01 seconds)833agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since -20s | grep -q 'app-slow2'834relay # [ 9662.696525] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:40.658261Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(7), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}835relay # [ 9662.697949] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:40.659770Z","level":"INFO","message":"deploy finished","job":"440f9582a38de6779d9609d608b774a0","ok":true,"target":"flakelet_relay::relay::api"}836relay # [ 9662.943118] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:40.904838Z","level":"INFO","message":"deploy accepted","job":"a70cd93e695c209c2c4be97c1a881505","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}837agent # [ 9662.696437] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:40.658181Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}838agent # [ 9662.697631] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:40.659450Z","level":"INFO","message":"start","id":"0df49d05b7e4e93b26d0ba61f2169a2a","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}839agent # [ 9662.943381] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:40.905165Z","level":"INFO","message":"start","id":"7de24a5e260e1768fc15f62fd80772f9","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}840agent # [ 9662.944372] agent flakelet-agent[2133]: {"timestamp":"2026-09-06T09:45:40.906196Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"7de24a5e260e1768fc15f62fd80772f9\"]","target":"flakelet_relay::agent::jobs"}841agent # [ 9663.014426] agent systemd[1]: Starting flakelet update app (via relay)...842agent # [ 9663.045026] agent systemd[1]: Started flakelet update app (via relay).843agent # [ 9663.049392] agent flakelet[3295]: app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2844agent # [ 9663.050065] agent flakelet[3295]: app: activating generation 8845agent # [ 9663.062894] agent systemd[1]: Stopping app slow...846agent # [ 9663.063370] agent systemd[1]: app.service: Deactivated successfully.847agent # [ 9663.063636] agent systemd[1]: Stopped app slow.848agent # [ 9663.080462] agent systemd[1]: Reload requested from client PID 3302 ('systemctl') (unit flakelet-relay-job-app.service)...849agent # [ 9663.080530] agent systemd[1]: Reloading...850agent # [ 9663.413196] agent systemd[1]: Reloading finished in 331 ms.851agent # [ 9663.484158] agent systemd[1]: Reload requested from client PID 3332 ('systemctl') (unit flakelet-relay-job-app.service)...852agent # [ 9663.484193] agent systemd[1]: Reloading...853agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since -20s | grep -q 'app-slow2', in 1.02 seconds)854agent: must succeed: systemctl restart flakelet-agent855agent # [ 9663.808122] agent systemd[1]: Reloading finished in 323 ms.856agent # [ 9663.868393] agent systemd[1]: Reload requested from client PID 3363 ('systemctl') (unit flakelet-relay-job-app.service)...857agent # [ 9663.868429] agent systemd[1]: Reloading...858agent # [ 9664.198583] agent systemd[1]: Reloading finished in 329 ms.859agent # [ 9664.269954] agent systemd[1]: Stopping flakelet-agent...860agent # [ 9664.273611] agent systemd[1]: Reload requested from client PID 3400 ('systemctl') (unit flakelet-relay-job-app.service)...861agent # [ 9664.273648] agent systemd[1]: Reloading...862relay # [ 9664.270141] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:42.231810Z","level":"INFO","message":"agent disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","host":"agent","target":"flakelet_relay::relay::agent_conn"}863agent: (finished: must succeed: systemctl restart flakelet-agent, in 0.88 seconds)864agent: must succeed: test -n "$(ls /var/lib/flakelet-agent/jobs)"865agent: (finished: must succeed: test -n "$(ls /var/lib/flakelet-agent/jobs)", in 0.01 seconds)866client: waiting for success: grep -q 'agent/app: updated' /tmp/restart.out867agent # [ 9664.603790] agent systemd[1]: Reloading finished in 329 ms.868agent # [ 9664.660139] agent systemd[1]: flakelet-agent.service: Deactivated successfully.869agent # [ 9664.660381] agent systemd[1]: Stopped flakelet-agent.870agent # [ 9664.660397] agent systemd[1]: flakelet-agent.service: Consumed 1.936s CPU time over 29.017s wall clock time.871agent # [ 9664.678520] agent systemd[1]: Starting app slow2...872agent # [ 9664.679626] agent systemd[1]: Starting flakelet-agent...873agent # [ 9664.740160] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:42.701959Z","level":"INFO","message":"reattaching to running unit","flakelet":"app","ids":"[\"7de24a5e260e1768fc15f62fd80772f9\"]","target":"flakelet_relay::agent::jobs"}874agent # [ 9664.831208] agent systemd[1]: Started flakelet-agent.875agent # [ 9664.845126] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:42.806891Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}876agent # [ 9664.846434] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:42.808244Z","level":"INFO","message":"start","id":"7de24a5e260e1768fc15f62fd80772f9","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}877relay # [ 9664.845244] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:42.806967Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(7), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}878agent # [ 9670.753096] agent systemd[1]: Started app slow2.879agent # [ 9670.774966] agent flakelet[3295]: app: updated to generation 8880agent # [ 9670.778144] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.881relay # [ 9671.365701] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:49.327440Z","level":"INFO","message":"deploy finished","job":"a70cd93e695c209c2c4be97c1a881505","ok":true,"target":"flakelet_relay::relay::api"}882agent # [ 9671.364033] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:49.325777Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(8)","target":"flakelet_relay::agent::jobs"}883client: (finished: waiting for success: grep -q 'agent/app: updated' /tmp/restart.out, in 7.08 seconds)884client: must succeed: cat /tmp/restart.out885client: (finished: must succeed: cat /tmp/restart.out, in 0.01 seconds)886agent: must succeed: systemctl is-active app.service887agent: (finished: must succeed: systemctl is-active app.service, in 0.01 seconds)888(finished: subtest: agent restart mid-run reattaches to the unit, in 9.04 seconds)889subtest: relay restart mid-run, push resumes via /v1/jobs890agent: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json891agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)892agent: must succeed: date +%s893agent: (finished: must succeed: date +%s, in 0.01 seconds)894client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/resume.out 2>&1 &895client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/resume.out 2>&1 &, in 0.01 seconds)896agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687949 | grep -q 'app-slow'897agent # [ 9671.992389] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:49.954129Z","level":"INFO","message":"start","id":"f61a62e5ccbd9502ee565b40233040c0","flakelet":"app","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}898agent # [ 9671.992891] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:49.954711Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"f61a62e5ccbd9502ee565b40233040c0\"]","target":"flakelet_relay::agent::jobs"}899agent # [ 9672.064837] agent systemd[1]: Starting flakelet update app (via relay)...900agent # [ 9672.098010] agent systemd[1]: Started flakelet update app (via relay).901agent # [ 9672.102513] agent flakelet[3944]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow902agent # [ 9672.103162] agent flakelet[3944]: app: activating generation 9903agent # [ 9672.116706] agent systemd[1]: Stopping app slow2...904agent # [ 9672.118586] agent systemd[1]: app.service: Deactivated successfully.905agent # [ 9672.119024] agent systemd[1]: Stopped app slow2.906agent # [ 9672.134715] agent systemd[1]: Reload requested from client PID 3951 ('systemctl') (unit flakelet-relay-job-app.service)...907agent # [ 9672.134780] agent systemd[1]: Reloading...908relay # [ 9671.992036] relay flakelet-relay[400]: {"timestamp":"2026-09-06T09:45:49.953814Z","level":"INFO","message":"deploy accepted","job":"98adb657cbe52468bad1fdcca0d30827","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}909agent # [ 9672.478099] agent systemd[1]: Reloading finished in 342 ms.910agent # [ 9672.555272] agent systemd[1]: Reload requested from client PID 3981 ('systemctl') (unit flakelet-relay-job-app.service)...911agent # [ 9672.555310] agent systemd[1]: Reloading...912agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788687949 | grep -q 'app-slow', in 1.03 seconds)913relay: must succeed: systemctl restart flakelet-relay914agent # [ 9672.881198] agent systemd[1]: Reloading finished in 325 ms.915agent # [ 9672.951193] agent systemd[1]: Reload requested from client PID 4012 ('systemctl') (unit flakelet-relay-job-app.service)...916agent # [ 9672.951224] agent systemd[1]: Reloading...917agent # [ 9673.020628] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:50.982366Z","level":"INFO","message":"disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}918relay # [ 9673.019819] relay systemd[1]: Stopping flakelet-relay...919relay # [ 9673.075085] relay systemd[1]: flakelet-relay.service: Deactivated successfully.920relay # [ 9673.075551] relay systemd[1]: Stopped flakelet-relay.921relay # [ 9673.075596] relay systemd[1]: flakelet-relay.service: Consumed 2.504s CPU time over 1min 7.921s wall clock time.922relay # [ 9673.112746] relay systemd[1]: Starting flakelet-relay...923agent # [ 9673.283753] agent systemd[1]: Reloading finished in 332 ms.924agent # [ 9673.370761] agent systemd[1]: Reload requested from client PID 4048 ('systemctl') (unit flakelet-relay-job-app.service)...925agent # [ 9673.370796] agent systemd[1]: Reloading...926agent # [ 9673.699354] agent systemd[1]: Reloading finished in 328 ms.927agent # [ 9673.777499] agent systemd[1]: Starting app slow...928agent # [ 9674.023440] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:51.985195Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}929relay: (finished: must succeed: systemctl restart flakelet-relay, in 1.54 seconds)930client: waiting for success: grep -q '» ok' /tmp/resume.out931relay # [ 9674.546496] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:52.508248Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}932relay # [ 9674.547319] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:52.508671Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}933relay # [ 9674.547209] relay systemd[1]: Started flakelet-relay.934relay # [ 9676.038529] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:54.000243Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(8), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}935relay # [ 9676.040523] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:54.002332Z","level":"INFO","message":"job reattached","job":"98adb657cbe52468bad1fdcca0d30827","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}936agent # [ 9676.038221] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:53.999945Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}937agent # [ 9679.840017] agent systemd[1]: Started app slow.938agent # [ 9679.858819] agent flakelet[3944]: app: updated to generation 9939agent # [ 9679.861925] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.940agent # [ 9680.979362] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:58.941069Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(9)","target":"flakelet_relay::agent::jobs"}941client: (finished: waiting for success: grep -q '» ok' /tmp/resume.out, in 7.09 seconds)942client: must succeed: cat /tmp/resume.out943client: (finished: must succeed: cat /tmp/resume.out, in 0.01 seconds)944(finished: subtest: relay restart mid-run, push resumes via /v1/jobs, in 9.70 seconds)945subtest: job history comes from the agents and survives a relay restart946client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id hist1 agent/other 2>&1947relay # [ 9681.666256] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:45:59.628018Z","level":"INFO","message":"deploy accepted","job":"188b06c4636b65849b3ba8daac22dc7c","caller":"admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}948agent # [ 9681.666684] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:59.628439Z","level":"INFO","message":"start","id":"5be20c693e5c0b60e3f85779794dfb86","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}949agent # [ 9681.667118] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:45:59.628944Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"5be20c693e5c0b60e3f85779794dfb86\"]","target":"flakelet_relay::agent::jobs"}950agent # [ 9681.731169] agent systemd[1]: Starting flakelet update other (via relay)...951agent # [ 9681.766045] agent systemd[1]: Started flakelet update other (via relay).952agent # [ 9681.771388] agent flakelet[4189]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1953agent # [ 9681.772236] agent flakelet[4189]: other: up to date954agent # [ 9681.772966] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.955client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id hist1 agent/other 2>&1, in 0.70 seconds)956relay: must succeed: systemctl restart flakelet-relay957relay # [ 9682.341554] relay flakelet-relay[933]: {"timestamp":"2026-09-06T09:46:00.303241Z","level":"INFO","message":"deploy finished","job":"188b06c4636b65849b3ba8daac22dc7c","ok":true,"target":"flakelet_relay::relay::api"}958relay # [ 9682.358613] relay systemd[1]: Stopping flakelet-relay...959relay # [ 9682.401155] relay systemd[1]: flakelet-relay.service: Deactivated successfully.960relay # [ 9682.401584] relay systemd[1]: Stopped flakelet-relay.961relay # [ 9682.401639] relay systemd[1]: flakelet-relay.service: Consumed 2.043s CPU time over 9.288s wall clock time.962relay # [ 9682.438561] relay systemd[1]: Starting flakelet-relay...963agent # [ 9682.339948] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:00.301679Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}964agent # [ 9682.358775] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:00.320581Z","level":"INFO","message":"disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}965agent # [ 9683.361035] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:01.322773Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}966relay: (finished: must succeed: systemctl restart flakelet-relay, in 1.49 seconds)967relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1'968relay # [ 9683.835070] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:01.796808Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}969relay # [ 9683.835885] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:01.797261Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}970relay # [ 9683.835668] relay systemd[1]: Started flakelet-relay.971relay # [ 9685.374625] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:03.336336Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(9), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}972agent # [ 9685.374456] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:03.336189Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}973relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1', in 2.06 seconds)974client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs975client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs, in 0.02 seconds)976client: must succeed: echo '{"groups": [], "repository": "example/app"}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)977client: (finished: must succeed: echo '{"groups": [], "repository": "example/app"}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.06 seconds)978client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODI2MywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTYzLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJYR0dGS0EyQU5WTk5BPT09IiwibmJmIjoxNzg4Njg3OTYzLCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.7p-lQPI3-f29LXUEnNzYGD8cTaFcIB_R5AQlq8oRsT-lm-Yy01z1jZpLaLD5rhAkwrt2I_R8tL6GHnUCQrdXDw' push --relay https://relay:7443 deploy --id hist2 agent/app979» job 6a239dbf87cfec796e49404c351d787f via relay 0.1.0 (agents: agent 0.1.0)980» wave 0981agent/app | Starting flakelet update app (via relay)...982agent/app | Started flakelet update app (via relay).983agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow984agent/app | app: up to date985agent # [ 9685.994588] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:03.956309Z","level":"INFO","message":"start","id":"dce6faec8455f85321e72ee634888126","flakelet":"app","rule":"ci","relay":"relay","caller_name":"Some(\"example/app\")","target":"flakelet_relay::agent::conn"}986agent # [ 9685.995209] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:03.957028Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"dce6faec8455f85321e72ee634888126\"]","target":"flakelet_relay::agent::jobs"}987agent # [ 9686.066532] agent systemd[1]: Starting flakelet update app (via relay)...988agent # [ 9686.097978] agent systemd[1]: Started flakelet update app (via relay).989agent # [ 9686.103766] agent flakelet[4253]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow990agent # [ 9686.104421] agent flakelet[4253]: app: up to date991agent # [ 9686.105604] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.992agent/app | flakelet-relay-job-app.service: Deactivated successfully.993relay # [ 9685.994039] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:03.955817Z","level":"INFO","message":"deploy accepted","job":"6a239dbf87cfec796e49404c351d787f","caller":"example/app","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}994» agent/app: unchanged (generation 9)995» ok996client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODI2MywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTYzLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJYR0dGS0EyQU5WTk5BPT09IiwibmJmIjoxNzg4Njg3OTYzLCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.7p-lQPI3-f29LXUEnNzYGD8cTaFcIB_R5AQlq8oRsT-lm-Yy01z1jZpLaLD5rhAkwrt2I_R8tL6GHnUCQrdXDw' push --relay https://relay:7443 deploy --id hist2 agent/app, in 0.70 seconds)997client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODI2MywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTYzLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJYR0dGS0EyQU5WTk5BPT09IiwibmJmIjoxNzg4Njg3OTYzLCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.7p-lQPI3-f29LXUEnNzYGD8cTaFcIB_R5AQlq8oRsT-lm-Yy01z1jZpLaLD5rhAkwrt2I_R8tL6GHnUCQrdXDw' push --relay https://relay:7443 jobs998client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY4ODI2MywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njg3OTYzLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJYR0dGS0EyQU5WTk5BPT09IiwibmJmIjoxNzg4Njg3OTYzLCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.7p-lQPI3-f29LXUEnNzYGD8cTaFcIB_R5AQlq8oRsT-lm-Yy01z1jZpLaLD5rhAkwrt2I_R8tL6GHnUCQrdXDw' push --relay https://relay:7443 jobs, in 0.02 seconds)999(finished: subtest: job history comes from the agents and survives a relay restart, in 5.05 seconds)1000subtest: dashboard login via authorization code flow1001client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code} %{redirect_url}' https://relay:7443/ui/ | grep -q '303 https://relay:7443/ui/login'1002client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code} %{redirect_url}' https://relay:7443/ui/ | grep -q '303 https://relay:7443/ui/login', in 0.02 seconds)1003client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -L https://relay:7443/ui/login1004client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -L https://relay:7443/ui/login, in 0.09 seconds)1005client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs1006client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs, in 0.02 seconds)1007client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -f https://relay:7443/v1/jobs | grep -q hist11008client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -f https://relay:7443/v1/jobs | grep -q hist1, in 0.02 seconds)1009client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 10 https://relay:7443/ui/jobs/hist1/events || true1010client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 10 https://relay:7443/ui/jobs/hist1/events || true, in 0.02 seconds)1011client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -X POST 'https://relay:7443/ui/deploy?arg=other' | grep -q 4031012client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -X POST 'https://relay:7443/ui/deploy?arg=other' | grep -q 403, in 0.02 seconds)1013client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -D - -o /dev/null -H 'HX-Request: true' -X POST 'https://relay:7443/ui/deploy?arg=other'1014client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -D - -o /dev/null -H 'HX-Request: true' -X POST 'https://relay:7443/ui/deploy?arg=other', in 0.02 seconds)1015client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 30 https://relay:7443/ui/jobs/aa679b057ae16902f2e5b292dd90396d/events || true1016relay # [ 9686.673607] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.635315Z","level":"INFO","message":"deploy finished","job":"6a239dbf87cfec796e49404c351d787f","ok":true,"target":"flakelet_relay::relay::api"}1017relay # [ 9686.749987] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:46:04] "GET /authorize?response_type=code&client_id=dashboard&redirect_uri=https%3A%2F%2Frelay%3A7443%2Fui%2Fcallback&scope=openid%20profile%20email%20groups&state=VICnDwgXd_bb65-MlqOSv2B6kjB0J6Dzn3f9k9J2TAA&code_challenge=ux0X6GLHvycWdDfU70E0B4onkylU7pIQauSa2Q_5BWI&code_challenge_method=S256 HTTP/1.1" 302 -1018relay # [ 9686.799906] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [06/Sep/2026 09:46:04] "POST /token HTTP/1.1" 200 -1019relay # [ 9686.800796] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.762608Z","level":"INFO","message":"login","name":"dev@example.org","principals":"[\"oidc:mock:someone\", \"oidc:mock:groups:wheel\"]","target":"flakelet_relay::relay::login"}1020relay # [ 9686.860879] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.822684Z","level":"INFO","message":"job reattached","job":"188b06c4636b65849b3ba8daac22dc7c","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}1021relay # [ 9686.899585] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.861403Z","level":"INFO","message":"deploy accepted","job":"189890aa3e83142428d6f07166d42001","caller":"dev@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}1022relay # [ 9686.919501] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:04.881323Z","level":"INFO","message":"job reattached","job":"189890aa3e83142428d6f07166d42001","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}1023agent # [ 9686.671750] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:04.633473Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(9)","target":"flakelet_relay::agent::jobs"}1024agent # [ 9686.899806] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:04.861564Z","level":"INFO","message":"start","id":"a239886a520be8b32371f452d5c37574","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"dev@example.org\")","target":"flakelet_relay::agent::conn"}1025agent # [ 9686.900089] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:04.861915Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"a239886a520be8b32371f452d5c37574\"]","target":"flakelet_relay::agent::jobs"}1026agent # [ 9686.960961] agent systemd[1]: Starting flakelet update other (via relay)...1027agent # [ 9686.994009] agent systemd[1]: Started flakelet update other (via relay).1028agent # [ 9686.999542] agent flakelet[4303]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v11029agent # [ 9687.000376] agent flakelet[4303]: other: up to date1030agent # [ 9687.001739] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.1031client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 30 https://relay:7443/ui/jobs/aa679b057ae16902f2e5b292dd90396d/events || true, in 1.68 seconds)1032client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs/aa679b057ae16902f2e5b292dd90396d1033client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs/aa679b057ae16902f2e5b292dd90396d, in 0.02 seconds)1034client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -sf -o /dev/null https://relay:7443/ui/static/htmx.min.js1035client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -sf -o /dev/null https://relay:7443/ui/static/htmx.min.js, in 0.02 seconds)1036client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/?q=status%3Ahealthy+oth'1037client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/?q=status%3Ahealthy+oth', in 0.02 seconds)1038client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/flakelets/other1039client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/flakelets/other, in 0.02 seconds)1040client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/hosts?q=status%3Adisconnected'1041client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/hosts?q=status%3Adisconnected', in 0.02 seconds)1042client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -H 'HX-Request: true' -X POST 'https://relay:7443/ui/retry?arg=aa679b057ae16902f2e5b292dd90396d' | grep -q 4001043client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -H 'HX-Request: true' -X POST 'https://relay:7443/ui/retry?arg=aa679b057ae16902f2e5b292dd90396d' | grep -q 400, in 0.02 seconds)1044client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 15 'https://relay:7443/ui/events?path=jobs' > /tmp/ev.out 2>&1 &1045client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 15 'https://relay:7443/ui/events?path=jobs' > /tmp/ev.out 2>&1 &, in 0.01 seconds)1046client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id live1 agent/other1047» job 38796a7ba508eadfe82a1cd5def000bc via relay 0.1.0 (agents: agent 0.1.0)1048» wave 01049agent/other | Starting flakelet update other (via relay)...1050agent/other | Started flakelet update other (via relay).1051agent/other | other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v11052agent/other | other: up to date1053agent # [ 9688.576693] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:06.538422Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}1054agent # [ 9688.727758] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:06.689520Z","level":"INFO","message":"start","id":"5e317fc57863c2a7c03ebe1bdacd1356","flakelet":"other","rule":"admins","relay":"relay","caller_name":"Some(\"admin@example.org\")","target":"flakelet_relay::agent::conn"}1055agent # [ 9688.728329] agent flakelet-agent[3432]: {"timestamp":"2026-09-06T09:46:06.690148Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"5e317fc57863c2a7c03ebe1bdacd1356\"]","target":"flakelet_relay::agent::jobs"}1056agent/otheragent # [ 9688.790759] agent systemd[1]: Starting flakelet update other (via relay)...1057 | agent # [ 9688.819038] agent systemd[1]: Started flakelet update other (via relay).1058flakelet-relay-job-other.service: Deactivated successfully.agent # [ 9688.824338] agent flakelet[4340]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v110591060agent # [ 9688.825199] agent flakelet[4340]: other: up to date1061agent # [ 9688.826326] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.1062relay # [ 9688.727322] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:06.689107Z","level":"INFO","message":"deploy accepted","job":"38796a7ba508eadfe82a1cd5def000bc","caller":"admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}1063» agent/other: unchanged (generation 1)1064» ok1065client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id live1 agent/other, in 0.69 seconds)1066client: waiting for success: grep -q live1 /tmp/ev.out1067client: (finished: waiting for success: grep -q live1 /tmp/ev.out, in 0.01 seconds)1068client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -X POST -o /dev/null https://relay:7443/ui/logout1069client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -X POST -o /dev/null https://relay:7443/ui/logout, in 0.02 seconds)1070client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' https://relay:7443/ui/ | grep -q 3031071client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' https://relay:7443/ui/ | grep -q 303, in 0.02 seconds)1072(finished: subtest: dashboard login via authorization code flow, in 2.76 seconds)1073(finished: run the VM test script, in 87.79 seconds)1074test script finished in 87.82s1075cleanup1076kill NspawnMachine (pid 52)1077kill NspawnMachine (pid 53)1078kill NspawnMachine (pid 54)1079Container agent terminated by signal KILL.1080relay # [ 9689.399754] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T09:46:07.361541Z","level":"INFO","message":"deploy finished","job":"38796a7ba508eadfe82a1cd5def000bc","ok":true,"target":"flakelet_relay::relay::api"}1081Container client terminated by signal KILL.1082Container relay terminated by signal KILL.1083(finished: cleanup, in 0.39 seconds)1084additionally exposed symbols:1085 agent, client, relay,1086 vlan1,1087 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh1088» job 3aa35c37e59eaf074982997d80ea382f via relay 0.1.0 (agents: agent 0.1.0)1089» wave 01090agent/app | Starting flakelet update app (via relay)...1091agent/app | Started flakelet update app (via relay).1092agent/app | app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v21093agent/app | app: activating generation 21094agent/app | app: updated to generation 21095agent/app | flakelet-relay-job-app.service: Deactivated successfully.1096» agent/app: updated (generation 2)1097» ok10981099» job 56aec3139ffb9895cad78fd920f943b6 via relay 0.1.0 (agents: agent 0.1.0)1100» wave 01101agent/app | Starting flakelet update app (via relay)...1102agent/app | Started flakelet update app (via relay).1103agent/app | app: using prebuilt artifact /nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken1104agent/app | app: activating generation 41105agent/app | app: deploy failed, rolled back: systemctl start app.service failed:1106agent/app | failed units: app.service. See journalctl -u app.service1107agent/app | flakelet-relay-job-app.service: Main process exited, code=exited, status=1/FAILURE1108agent/app | flakelet-relay-job-app.service: Failed with result 'exit-code'.1109» agent/app: rolled-back (generation 3)1110agent/app │ Starting app v1...1111agent/app │ Started app v1.1112agent/app │ Stopping app v1...1113agent/app │ app.service: Deactivated successfully.1114agent/app │ Stopped app v1.1115agent/app │ Starting app v2...1116agent/app │ Started app v2.1117agent/app │ Stopping app v2...1118agent/app │ app.service: Deactivated successfully.1119agent/app │ Stopped app v2.1120agent/app │ Starting app v1...1121agent/app │ Started app v1.1122agent/app │ Stopping app v1...1123agent/app │ app.service: Deactivated successfully.1124agent/app │ Stopped app v1.1125agent/app │ Starting app broken...1126agent/app │ app.service: Unable to locate executable '/nonexistent': No such file or directory1127agent/app │ app.service: Failed at step EXEC spawning /nonexistent: No such file or directory1128agent/app │ app.service: Main process exited, code=exited, status=203/EXEC1129agent/app │ app.service: Failed with result 'exit-code'.1130agent/app │ Failed to start app broken.1131agent/app │ Starting app v1...1132agent/app │ Started app v1.1133» skipped: agent/other1134» failed11351136» job 440f9582a38de6779d9609d608b774a0 via relay 0.1.0 (agents: agent 0.1.0)1137» wave 01138agent/app | Starting flakelet update app (via relay)...1139agent/app | Started flakelet update app (via relay).1140agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow1141agent/app | app: activating generation 71142agent/app | app: updated to generation 71143agent/app | flakelet-relay-job-app.service: Deactivated successfully.1144» agent/app: updated (generation 7)1145» ok11461147» job a70cd93e695c209c2c4be97c1a881505 via relay 0.1.0 (agents: agent 0.1.0)1148» wave 01149agent/app | Starting flakelet update app (via relay)...1150agent/app | Started flakelet update app (via relay).1151agent/app | app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow21152agent/app | app: activating generation 81153agent/app | app: updated to generation 81154agent/app | flakelet-relay-job-app.service: Deactivated successfully.1155» agent/app: updated (generation 8)1156» ok11571158» job 98adb657cbe52468bad1fdcca0d30827 via relay 0.1.0 (agents: agent 0.1.0)1159» wave 01160agent/app | Starting flakelet update app (via relay)...1161agent/app | Started flakelet update app (via relay).1162agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow1163agent/app | app: activating generation 91164flakelet-push: stream: error reading a body from connection, resuming job 5c88c4be4e598f9a2cf7fd2570707b6f1165flakelet-push: https://relay:7443: connect relay:7443: Connection refused (os error 111), retrying in 1s1166flakelet-push: https://relay:7443: connect relay:7443: Connection refused (os error 111), retrying in 2s1167» job 98adb657cbe52468bad1fdcca0d30827 via relay 0.1.0 (agents: agent 0.1.0)1168» wave 01169agent/app | app: updated to generation 91170agent/app | flakelet-relay-job-app.service: Deactivated successfully.1171» agent/app: updated (generation 9)1172» ok117311741788687959 hist1 admin@example.org agent/other:unchanged11751788687949 5c88c4be4e598f9a2cf7fd2570707b6f admin@example.org agent/app:updated11761788687940 9ea811ff91edb1efcbed3d80c8cac0cd admin@example.org agent/app:updated11771788687924 fe8b9cf81c9212d5dcfc80af9ad59d33 admin@example.org agent/app:updated11781788687915 ca6dbee687afe7ab341cc1e49c0db303 admin@example.org agent/other:unchanged11791788687904 29d17930204ef426f4562129bc0deaf1 admin@example.org agent/app:updated11801788687903 35e285f9c2176659c4e72565ef900704 admin@example.org agent/app:unchanged11811788687903 f76e546af74a11a5a65c4949d33e9f0b admin@example.org agent/app:unchanged11821788687902 8c6fcee97235a03c2f58f74eb9a2043b admin@example.org agent/app:unchanged11831788687898 d86ec9ebef884696bfc46b98c20a284d admin@example.org agent/app:rolled-back11841788687897 c9461b2f44eb1a6985b90a3a2ea94dbe dev@example.org agent/other:unchanged11851788687896 03f353613bb9d215170e437bb7c8ffc8 dev@example.org agent/other:unchanged11861788687894 7a11f9c99a78170bdd3222ceb9141d30 someone agent/other:unchanged11871788687891 aeb452c85dd52cd446c4831375293ebf repo:github:example/app:ref:refs/heads/main agent/app:updated11881788687887 6ddb24702aeb90540341ab7554da8f06 admin@example.org agent/app:updated1189