container-test-run-flakelet-relay
checks.x86_64-linux.nixos-test
· build #11
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Initializing machine ID from random generator.6Test will time out and terminate in 3600.0 seconds7run the VM test script8start all VMs9agent: systemd-nspawn running (pid 52)10client: systemd-nspawn running (pid 53)11relay: systemd-nspawn running (pid 54)12agent: Waiting for journal at /build/vm-state-agent/var/log/journal...13client: Waiting for journal at /build/vm-state-client/var/log/journal...14relay: Waiting for journal at /build/vm-state-relay/var/log/journal...15(finished: start all VMs, in 0.00 seconds)16relay: waiting for unit flakelet-relay.service17nixos-nspawn(relay): TAP vde-tap1 not found; container will be isolated from VDE18nixos-nspawn(relay): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.19nixos-nspawn(client): TAP vde-tap1 not found; container will be isolated from VDE20nixos-nspawn(client): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.21nixos-nspawn(agent): TAP vde-tap1 not found; container will be isolated from VDE22nixos-nspawn(agent): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.23Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.24░ Spawning container relay on /build/vm-state-relay.25Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.26Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.27░ Spawning container client on /build/vm-state-client.28░ Spawning container agent on /build/vm-state-agent.29agent # No journal files were found.30agent # No journal boot entry found for the specified boot (+0).31relay # No journal files were found.32client # No journal files were found.33relay # No journal boot entry found for the specified boot (+0).34client # No journal boot entry found for the specified boot (+0).35agent # [84890.930571] agent systemd-journald[57]: Journal started36agent # [84890.930615] agent systemd-journald[57]: Runtime Journal (/run/log/journal/da2ad364eaca41d083583b4df0c739d0) is 8M, max 4G, 3.9G free.37agent # [84890.943147] agent systemd[1]: Finished Apply Kernel Variables.38agent # [84890.955776] agent systemd[1]: Finished Create Static Device Nodes in /dev gracefully.39agent # [84890.971440] agent systemd[1]: Starting Flush Journal to Persistent Storage...40agent # [84890.972023] agent systemd[1]: Starting Create Static Device Nodes in /dev...41agent # [84891.008892] agent systemd-journald[57]: Time spent on flushing to /var/log/journal/da2ad364eaca41d083583b4df0c739d0 is 1.841ms for 6 entries.42agent # [84891.008892] agent systemd-journald[57]: System Journal (/var/log/journal/da2ad364eaca41d083583b4df0c739d0) is 512B, max 4G, 3.9G free.43agent # [84891.018894] agent systemd[1]: Finished Create Static Device Nodes in /dev.44agent # [84891.019760] agent systemd[1]: Reached target Preparation for Local File Systems.45agent # [84891.019870] agent systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys46agent # [84891.031492] agent systemd[1]: Finished Flush Journal to Persistent Storage.47agent # [84891.093873] agent systemd[1]: Finished Firewall.48client # [84890.930865] client systemd-journald[57]: Journal started49relay # [84890.931745] relay systemd-journald[57]: Journal started50client # [84890.930906] client systemd-journald[57]: Runtime Journal (/run/log/journal/09dbdfcda81c48e694ba1667ad174263) is 8M, max 4G, 3.9G free.51relay # [84890.931789] relay systemd-journald[57]: Runtime Journal (/run/log/journal/23e76c64862744d29b57d3fa42ba33dc) is 8M, max 4G, 3.9G free.52client # [84890.942809] client systemd[1]: Finished Apply Kernel Variables.53relay # [84890.942760] relay systemd[1]: Finished Apply Kernel Variables.54client # [84890.955518] client systemd[1]: Finished Create Static Device Nodes in /dev gracefully.55relay # [84890.955477] relay systemd[1]: Finished Create Static Device Nodes in /dev gracefully.56client # [84890.970322] client systemd[1]: Starting Flush Journal to Persistent Storage...57relay # [84890.970419] relay systemd[1]: Starting Flush Journal to Persistent Storage...58client # [84890.971227] client systemd[1]: Starting Create Static Device Nodes in /dev...59relay # [84890.971274] relay systemd[1]: Starting Create Static Device Nodes in /dev...60client # [84891.009110] client systemd-journald[57]: Time spent on flushing to /var/log/journal/09dbdfcda81c48e694ba1667ad174263 is 2.986ms for 6 entries.61relay # [84891.008807] relay systemd-journald[57]: Time spent on flushing to /var/log/journal/23e76c64862744d29b57d3fa42ba33dc is 2.651ms for 6 entries.62client # [84891.009110] client systemd-journald[57]: System Journal (/var/log/journal/09dbdfcda81c48e694ba1667ad174263) is 512B, max 4G, 3.9G free.63relay # [84891.008807] relay systemd-journald[57]: System Journal (/var/log/journal/23e76c64862744d29b57d3fa42ba33dc) is 512B, max 4G, 3.9G free.64client # [84891.018119] client systemd[1]: Finished Create Static Device Nodes in /dev.65relay # [84891.018246] relay systemd[1]: Finished Create Static Device Nodes in /dev.66client # [84891.018926] client systemd[1]: Reached target Preparation for Local File Systems.67relay # [84891.019118] relay systemd[1]: Reached target Preparation for Local File Systems.68client # [84891.019043] client systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys69relay # [84891.019234] relay systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys70client # [84891.036317] client systemd[1]: Finished Flush Journal to Persistent Storage.71relay # [84891.038355] relay systemd[1]: Finished Flush Journal to Persistent Storage.72client # [84891.098525] client systemd[1]: Finished Firewall.73relay # [84891.117134] relay systemd[1]: Finished Firewall.74relay # [84891.930761] relay systemd[1]: Mounting /run/wrappers...75agent # [84891.960787] agent systemd[1]: Mounting /run/wrappers...76relay # [84891.993348] relay systemd[1]: Mounted /run/wrappers.77agent # [84891.999777] agent systemd[1]: Mounted /run/wrappers.78relay # [84891.994524] relay systemd[1]: Reached target Local File Systems.79agent # [84892.001024] agent systemd[1]: Reached target Local File Systems.80relay # [84891.995846] relay systemd[1]: Listening on Boot Loader Control Service Socket.81agent # [84892.002432] agent systemd[1]: Listening on Boot Loader Control Service Socket.82relay # [84891.997305] relay systemd[1]: Starting Create SUID/SGID Wrappers...83agent # [84892.003651] agent systemd[1]: Starting Create SUID/SGID Wrappers...84relay # [84891.997356] relay systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container85agent # [84892.003704] agent systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container86relay # [84891.998656] relay systemd[1]: Starting Save Transient machine-id to Disk...87agent # [84892.004896] agent systemd[1]: Starting Save Transient machine-id to Disk...88relay # [84891.999759] relay systemd[1]: Starting Create System Files and Directories...89agent # [84892.006150] agent systemd[1]: Starting Create System Files and Directories...90relay # [84892.023367] relay suid-sgid-wrappers-start[176]: chmod: changing permissions of '/run/wrappers/wrappers.Xm0BKQ9bun/chsh': Operation not permitted91client # [84891.960804] client systemd[1]: Mounting /run/wrappers...92relay # [84892.026741] relay systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE93agent # [84892.028626] agent systemd-tmpfiles[161]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted94relay # [84892.027017] relay systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.95agent # [84892.029080] agent systemd-tmpfiles[161]: fchmod() of /var/log/journal failed: Operation not permitted96relay # [84892.027272] relay systemd[1]: Failed to start Create SUID/SGID Wrappers.97agent # [84892.029427] agent systemd-tmpfiles[161]: fchmod() of /var/log/journal/da2ad364eaca41d083583b4df0c739d0 failed: Operation not permitted98relay # [84892.028549] relay systemd-tmpfiles[169]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted99agent # [84892.029795] agent systemd-tmpfiles[161]: fchmod() of /run/log/journal failed: Operation not permitted100client # [84892.000912] client systemd[1]: Mounted /run/wrappers.101relay # [84892.029046] relay systemd-tmpfiles[169]: fchmod() of /var/log/journal failed: Operation not permitted102client # [84892.002132] client systemd[1]: Reached target Local File Systems.103relay # [84892.029381] relay systemd-tmpfiles[169]: fchmod() of /var/log/journal/23e76c64862744d29b57d3fa42ba33dc failed: Operation not permitted104client # [84892.003547] client systemd[1]: Listening on Boot Loader Control Service Socket.105relay # [84892.029758] relay systemd-tmpfiles[169]: fchmod() of /run/log/journal failed: Operation not permitted106client # [84892.004938] client systemd[1]: Starting Create SUID/SGID Wrappers...107client # [84892.004985] client systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container108client # [84892.006075] client systemd[1]: Starting Save Transient machine-id to Disk...109client # [84892.007052] client systemd[1]: Starting Create System Files and Directories...110client # [84892.029730] client systemd-tmpfiles[161]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted111client # [84892.030195] client systemd-tmpfiles[161]: fchmod() of /var/log/journal failed: Operation not permitted112client # [84892.030541] client systemd-tmpfiles[161]: fchmod() of /var/log/journal/09dbdfcda81c48e694ba1667ad174263 failed: Operation not permitted113client # [84892.030900] client systemd-tmpfiles[161]: fchmod() of /run/log/journal failed: Operation not permitted114relay # [84892.031669] relay systemd[1]: Finished Create System Files and Directories.115relay # [84892.045412] relay systemd[1]: Starting Rebuild Journal Catalog...116relay # [84892.046130] relay systemd[1]: Starting Record System Boot/Shutdown in UTMP...117relay # [84892.058055] relay systemd[1]: Finished Record System Boot/Shutdown in UTMP.118relay # [84892.068171] relay systemd[1]: Finished Rebuild Journal Catalog.119relay # [84892.069396] relay systemd[1]: Starting Update is Completed...120relay # [84892.078562] relay systemd[1]: Finished Update is Completed.121relay # [84892.078673] relay systemd[1]: Reached target System Initialization.122relay # [84892.078774] relay systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container123relay # [84892.078806] relay systemd[1]: Started Daily Cleanup of Temporary Directories.124relay # [84892.078826] relay systemd[1]: Reached target Timer Units.125relay # [84892.078957] relay systemd[1]: Listening on D-Bus System Message Bus Socket.126relay # [84892.079171] relay systemd[1]: Listening on Nix Daemon Socket.127relay # [84892.079279] relay systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.128relay # [84892.079294] relay systemd[1]: Reached target Socket Units.129relay # [84892.079327] relay systemd[1]: Reached target Basic System.130relay # [84892.080625] relay systemd[1]: Started device-mock.service.131relay # [84892.081350] relay systemd[1]: Starting Re-link flakelet services at boot...132relay # [84892.082085] relay systemd[1]: Starting Import lastlog data into lastlog2 database...133relay # [84892.083118] relay systemd[1]: Starting Name Service Cache Daemon (nsncd)...134relay # [84892.083923] relay systemd[1]: Starting resolvconf update...135relay # [84892.092646] relay systemd[1]: Finished Re-link flakelet services at boot.136relay # [84892.094320] relay systemd[1]: Starting Reconcile flakelet services with the host configuration...137relay # [84892.101205] relay systemd[1]: Finished Import lastlog data into lastlog2 database.138relay # [84892.106734] relay systemd[1]: Finished Reconcile flakelet services with the host configuration.139relay # [84892.160251] relay systemd[1]: Finished resolvconf update.140relay # [84892.161023] relay systemd[1]: Reached target Preparation for Network.141relay # [84892.162152] relay systemd[1]: Starting Address configuration of eth1...142relay # [84892.162985] relay systemd[1]: Starting Extra networking commands....143client # [84892.032475] client suid-sgid-wrappers-start[168]: chmod: changing permissions of '/run/wrappers/wrappers.qJXk1qpsET/chsh': Operation not permitted144client # [84892.038641] client systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE145client # [84892.038684] client systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.146client # [84892.038735] client systemd[1]: Failed to start Create SUID/SGID Wrappers.147client # [84892.039190] client systemd[1]: Finished Create System Files and Directories.148client # [84892.053772] client systemd[1]: Starting Rebuild Journal Catalog...149client # [84892.054372] client systemd[1]: Starting Record System Boot/Shutdown in UTMP...150client # [84892.066585] client systemd[1]: Finished Record System Boot/Shutdown in UTMP.151client # [84892.075076] client systemd[1]: Finished Rebuild Journal Catalog.152client # [84892.075992] client systemd[1]: Starting Update is Completed...153client # [84892.087079] client systemd[1]: Finished Update is Completed.154client # [84892.087745] client systemd[1]: Reached target System Initialization.155client # [84892.087918] client systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container156client # [84892.087987] client systemd[1]: Started Daily Cleanup of Temporary Directories.157client # [84892.088041] client systemd[1]: Reached target Timer Units.158client # [84892.088246] client systemd[1]: Listening on D-Bus System Message Bus Socket.159client # [84892.088514] client systemd[1]: Listening on Nix Daemon Socket.160client # [84892.088720] client systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.161client # [84892.088749] client systemd[1]: Reached target Socket Units.162client # [84892.088805] client systemd[1]: Reached target Basic System.163client # [84892.090451] client systemd[1]: Starting Import lastlog data into lastlog2 database...164client # [84892.091750] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...165client # [84892.092693] client systemd[1]: Starting resolvconf update...166client # [84892.101422] client systemd[1]: lastlog2-import.service: Failed to spawn executor: No such file or directory167client # [84892.101492] client systemd[1]: lastlog2-import.service: Failed to spawn 'start-post' task: No such file or directory168client # [84892.101563] client systemd[1]: lastlog2-import.service: Failed with result 'resources'.169client # [84892.101649] client systemd[1]: Failed to start Import lastlog data into lastlog2 database.170client # [84892.174509] client systemd[1]: Finished resolvconf update.171client # [84892.175393] client systemd[1]: Reached target Preparation for Network.172client # [84892.176499] client systemd[1]: Starting Address configuration of eth1...173client # [84892.177211] client systemd[1]: Starting Extra networking commands....174client # [84892.201441] client network-addresses-eth1-start[230]: adding address 192.168.1.2/24... done175client # [84892.205825] client network-addresses-eth1-start[230]: adding address 2001:db8:1::2/64... done176agent # [84892.035528] agent suid-sgid-wrappers-start[168]: chmod: changing permissions of '/run/wrappers/wrappers.yRkVWS4xRL/chsh': Operation not permitted177agent # [84892.036532] agent systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE178agent # [84892.036649] agent systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.179agent # [84892.036873] agent systemd[1]: Failed to start Create SUID/SGID Wrappers.180agent # [84892.037405] agent systemd[1]: Finished Create System Files and Directories.181agent # [84892.051920] agent systemd[1]: Starting Rebuild Journal Catalog...182agent # [84892.052538] agent systemd[1]: Starting Record System Boot/Shutdown in UTMP...183agent # [84892.064247] agent systemd[1]: Finished Record System Boot/Shutdown in UTMP.184agent # [84892.074639] agent systemd[1]: Finished Rebuild Journal Catalog.185agent # [84892.075718] agent systemd[1]: Starting Update is Completed...186agent # [84892.085214] agent systemd[1]: Finished Update is Completed.187agent # [84892.085978] agent systemd[1]: Reached target System Initialization.188agent # [84892.086103] agent systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container189agent # [84892.086141] agent systemd[1]: Started Daily Cleanup of Temporary Directories.190agent # [84892.086161] agent systemd[1]: Reached target Timer Units.191agent # [84892.086303] agent systemd[1]: Listening on D-Bus System Message Bus Socket.192agent # [84892.086540] agent systemd[1]: Listening on Nix Daemon Socket.193agent # [84892.086679] agent systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.194agent # [84892.086703] agent systemd[1]: Reached target Socket Units.195agent # [84892.086733] agent systemd[1]: Reached target Basic System.196agent # [84892.088071] agent systemd[1]: Starting Re-link flakelet services at boot...197agent # [84892.088791] agent systemd[1]: Starting Import lastlog data into lastlog2 database...198agent # [84892.089552] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...199agent # [84892.090152] agent systemd[1]: Starting resolvconf update...200agent # [84892.100767] agent systemd[1]: Finished Re-link flakelet services at boot.201agent # [84892.101921] agent systemd[1]: Starting Reconcile flakelet services with the host configuration...202agent # [84892.102211] agent systemd[1]: lastlog2-import.service: Failed to spawn executor: No such file or directory203agent # [84892.102231] agent systemd[1]: lastlog2-import.service: Failed to spawn 'start-post' task: No such file or directory204agent # [84892.102265] agent systemd[1]: lastlog2-import.service: Failed with result 'resources'.205agent # [84892.102309] agent systemd[1]: Failed to start Import lastlog data into lastlog2 database.206agent # [84892.112678] agent systemd[1]: Finished Reconcile flakelet services with the host configuration.207agent # [84892.167614] agent systemd[1]: Finished resolvconf update.208agent # [84892.168386] agent systemd[1]: Reached target Preparation for Network.209agent # [84892.169422] agent systemd[1]: Starting Address configuration of eth1...210agent # [84892.170113] agent systemd[1]: Starting Extra networking commands....211agent # [84892.188216] agent network-addresses-eth1-start[234]: adding address 192.168.1.1/24... done212agent # [84892.194467] agent network-addresses-eth1-start[234]: adding address 2001:db8:1::1/64... done213agent # [84892.200894] agent systemd[1]: Finished Address configuration of eth1.214agent # [84892.225773] agent systemd[1]: nscd.service: Deactivated successfully.215agent # [84892.226071] agent systemd[1]: Stopped Name Service Cache Daemon (nsncd).216agent # [84892.231635] agent systemd[1]: nscd.service: Failed to spawn executor: No such file or directory217agent # [84892.231675] agent systemd[1]: nscd.service: Failed to spawn 'start' task: No such file or directory218agent # [84892.231715] agent systemd[1]: nscd.service: Failed with result 'resources'.219agent # [84892.231756] agent systemd[1]: Failed to start Name Service Cache Daemon (nsncd).220agent # [84892.231769] agent systemd[1]: Dependency failed for Host and Network Name Lookups.221agent # [84892.231779] agent systemd[1]: nss-lookup.target: Job nss-lookup.target/start failed with result 'dependency'.222agent # [84892.231787] agent systemd[1]: Dependency failed for User and Group Name Lookups.223agent # [84892.231795] agent systemd[1]: nss-user-lookup.target: Job nss-user-lookup.target/start failed with result 'dependency'.224agent # [84892.233565] agent systemd[1]: nscd.service: Scheduled restart job immediately on client request, restart counter is at 1.225agent # [84892.234680] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...226agent # [84892.284076] agent systemd[1]: etc-machine\x2did.mount: Deactivated successfully.227agent # [84892.285059] agent systemd[1]: Finished Save Transient machine-id to Disk.228agent # [84892.323240] agent systemd[1]: nscd.service: Deactivated successfully.229agent # [84892.323381] agent systemd[1]: Stopped Name Service Cache Daemon (nsncd).230agent # [84892.325080] agent systemd[1]: Finished Extra networking commands..231agent # [84892.325985] agent systemd[1]: Reached target Network.232agent # [84892.327193] agent systemd[1]: Starting flakelet-agent...233agent # [84892.327986] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...234agent # [84892.328761] agent systemd[1]: Starting Update flakelet service app...235agent # [84892.329785] agent systemd[1]: Starting Update flakelet service other...236agent # [84892.341276] agent flakelet[309]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1237agent # [84892.342129] agent flakelet[309]: app: activating generation 1238agent # [84892.342901] agent flakelet[310]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1239agent # [84892.343696] agent flakelet[310]: other: activating generation 1240agent # [84892.350607] agent systemd[1]: Reload requested from client PID 313 ('systemctl') (unit flakelet-app.service)...241agent # [84892.350694] agent systemd[1]: Reloading...242agent # [84892.401926] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:13.531082Z","level":"WARN","message":"SRV _flakelet-relay._tcp.test: lookup failed","target":"flakelet_agent"}243client # [84892.211627] client systemd[1]: Finished Address configuration of eth1.244client # [84892.231248] client systemd[1]: nscd.service: Deactivated successfully.245client # [84892.231518] client systemd[1]: Stopped Name Service Cache Daemon (nsncd).246client # [84892.283109] client systemd[1]: etc-machine\x2did.mount: Deactivated successfully.247client # [84892.283911] client systemd[1]: Finished Save Transient machine-id to Disk.248client # [84892.288244] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...249client # [84892.298483] client systemd[1]: nscd.service: Deactivated successfully.250client # [84892.323218] client systemd[1]: Stopped Name Service Cache Daemon (nsncd).251client # [84892.325027] client systemd[1]: Finished Extra networking commands..252client # [84892.327113] client systemd[1]: Reached target Network.253client # [84892.328075] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...254relay # [84892.184552] relay network-addresses-eth1-start[245]: adding address 192.168.1.3/24... done255relay # [84892.189330] relay network-addresses-eth1-start[245]: adding address 2001:db8:1::3/64... done256relay # [84892.198248] relay systemd[1]: Finished Address configuration of eth1.257relay # [84892.280413] relay systemd[1]: etc-machine\x2did.mount: Deactivated successfully.258relay # [84892.281413] relay systemd[1]: Finished Save Transient machine-id to Disk.259relay # [84892.282074] relay systemd[1]: nscd.service: Deactivated successfully.260relay # [84892.282191] relay systemd[1]: Stopped Name Service Cache Daemon (nsncd).261relay # [84892.283919] relay systemd[1]: Finished Extra networking commands..262relay # [84892.285951] relay systemd[1]: Reached target Network.263relay # [84892.287366] relay systemd[1]: Starting Dnsmasq Daemon...264relay # [84892.288855] relay systemd[1]: Starting Nginx Web Server...265relay # [84892.323581] relay systemd[1]: Starting Name Service Cache Daemon (nsncd)...266relay # [84892.324713] relay systemd[1]: Starting Update flakelet service flakelet-relay...267relay # [84892.337771] relay flakelet[312]: flakelet-relay: using prebuilt artifact /nix/store/7bw7cva8az30xy0f6nl7gpd2zd83nz7q-flakelet-flakelet-relay268relay # [84892.338481] relay flakelet[312]: flakelet-relay: activating generation 1269relay # [84892.344049] relay systemd[1]: Reload requested from client PID 316 ('systemctl') (unit flakelet-flakelet-relay.service)...270relay # [84892.344131] relay systemd[1]: Reloading...271agent # [84892.670597] agent nsncd[308]: Sep 06 06:14:13.799 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"272agent # [84892.797271] agent systemd[1]: Reloading finished in 445 ms.273agent # [84892.938984] agent systemd[1]: Started flakelet-agent.274relay # [84892.692886] relay nsncd[309]: Sep 06 06:14:13.822 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"275relay # [84892.793585] relay systemd[1]: Reloading finished in 448 ms.276relay # [84892.931488] relay systemd[1]: Started Name Service Cache Daemon (nsncd).277relay # [84892.935060] relay systemd[1]: Reached target Host and Network Name Lookups.278relay # [84892.935186] relay systemd[1]: Reached target User and Group Name Lookups.279relay # [84892.937572] relay systemd[1]: Starting User Login Management...280relay # [84892.939359] relay systemd[1]: Starting Permit User Sessions...281relay # [84892.954624] relay systemd[1]: Reload requested from client PID 357 ('systemctl') (unit flakelet-flakelet-relay.service)...282client # [84892.687215] client nsncd[299]: Sep 06 06:14:13.816 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"283client # [84892.687242] client systemd[1]: Started Name Service Cache Daemon (nsncd).284client # [84892.687305] client systemd[1]: Reached target Host and Network Name Lookups.285client # [84892.687356] client systemd[1]: Reached target User and Group Name Lookups.286client # [84892.689289] client systemd[1]: Starting User Login Management...287client # [84892.690447] client systemd[1]: Starting Permit User Sessions...288client # [84892.729411] client systemd[1]: Finished Permit User Sessions.289client # [84892.732720] client systemd[1]: Started Console Getty.290client # [84892.732782] client systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0291client # [84892.732815] client systemd[1]: Reached target Login Prompts.292agent # [84892.939121] agent systemd[1]: Started Name Service Cache Daemon (nsncd).293agent # [84892.940183] agent systemd[1]: Reload requested from client PID 314 ('systemctl') (unit flakelet-other.service)...294agent # [84892.940236] agent systemd[1]: Reloading...295relay # [84892.954677] relay systemd[1]: Reloading...296relay # [84892.956359] relay dnsmasq-pre-start[360]: dnsmasq: syntax check OK.297relay # [84893.337634] relay systemd[1]: Reloading finished in 382 ms.298relay # [84893.466770] relay systemd[1]: Finished Permit User Sessions.299relay # [84893.469690] relay systemd[1]: Started Console Getty.300relay # [84893.469760] relay systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0301relay # [84893.469852] relay systemd[1]: Reached target Login Prompts.302relay # [84893.499088] relay systemd[1]: Starting flakelet-relay...303agent # [84893.337887] agent systemd[1]: Reloading finished in 397 ms.304agent # [84893.466152] agent systemd[1]: Reached target Host and Network Name Lookups.305agent # [84893.466291] agent systemd[1]: Reached target User and Group Name Lookups.306agent # [84893.467765] agent systemd[1]: Starting User Login Management...307agent # [84893.468767] agent systemd[1]: Starting Permit User Sessions...308agent # [84893.479458] agent systemd[1]: Reload requested from client PID 765 ('systemctl') (unit flakelet-app.service)...309agent # [84893.479496] agent systemd[1]: Reloading...310agent # [84893.884314] agent systemd[1]: Reloading finished in 404 ms.311agent # [84893.950928] agent systemd[1]: Finished Permit User Sessions.312agent # [84893.958426] agent systemd[1]: Started Console Getty.313agent # [84893.958494] agent systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0314agent # [84893.958565] agent systemd[1]: Reached target Login Prompts.315agent # [84893.960871] agent systemd[1]: Reload requested from client PID 767 ('systemctl') (unit flakelet-other.service)...316agent # [84893.960905] agent systemd[1]: Reloading...317relay # [84893.950169] relay nginx-pre-start[402]: nginx: the configuration file /nix/store/01avjj26wcihp926xgasszcj5s0k7cbs-nginx.conf syntax is ok318relay # [84893.950861] relay nginx-pre-start[402]: nginx: configuration file /nix/store/01avjj26wcihp926xgasszcj5s0k7cbs-nginx.conf test is successful319relay # [84893.966872] relay systemd[1]: Started Nginx Web Server.320relay # [84894.136763] relay systemd[1]: Starting D-Bus System Message Bus...321client # [84894.189886] client systemd-logind[311]: New seat seat0.322client # [84894.190209] client systemd[1]: Started User Login Management.323client # [84894.192391] client systemd[1]: Starting D-Bus System Message Bus...324client # [84894.193997] client systemd[1]: Starting linger-users.service...325client # [84894.235468] client systemd[1]: linger-users.service: Deactivated successfully.326client # [84894.235714] client systemd[1]: Finished linger-users.service.327client # [84894.235919] client systemd[1]: Reached target Multi-User System.328agent # [84894.334973] agent systemd[1]: Reloading finished in 373 ms.329agent # [84894.431397] agent systemd[1]: Starting app v1...330agent # [84894.432479] agent systemd[1]: Starting other v1...331agent # [84894.441064] agent systemd[1]: Started app v1.332agent # [84894.444390] agent systemd[1]: Started other v1.333agent # [84894.465381] agent flakelet[309]: app: updated to generation 1334agent # [84894.468583] agent systemd[1]: Finished Update flakelet service app.335agent # [84894.471480] agent flakelet[310]: other: updated to generation 1336agent # [84894.473460] agent systemd[1]: Finished Update flakelet service other.337agent # [84894.473603] agent systemd[1]: Reached target flakelet managed services.338relay # [84894.428816] relay systemd-logind[354]: New seat seat0.339relay # [84894.429094] relay systemd[1]: Started User Login Management.340relay # [84894.431217] relay systemd[1]: Starting linger-users.service...341relay # [84894.447868] relay systemd[1]: linger-users.service: Deactivated successfully.342relay # [84894.448372] relay systemd[1]: Finished linger-users.service.343relay # [84894.663277] relay dbus-broker-launch[408]: Looking up NSS user entry for 'systemd-timesync'...344relay # [84894.664424] relay dbus-broker-launch[408]: NSS returned no entry for 'systemd-timesync'345relay # [84894.664424] relay dbus-broker-launch[408]: Invalid user-name in /nix/store/27h4g2jdw18vrabfh6p1l0n8ss5dhbp5-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"346relay # [84894.665356] relay systemd[1]: Started D-Bus System Message Bus.347relay # [84894.672152] relay dbus-broker-launch[408]: Ready348relay # [84894.676211] relay systemd[1]: Started Dnsmasq Daemon.349client # [84894.694911] client dbus-broker-launch[321]: Looking up NSS user entry for 'systemd-timesync'...350client # [84894.696542] client dbus-broker-launch[321]: NSS returned no entry for 'systemd-timesync'351client # [84894.696542] client dbus-broker-launch[321]: Invalid user-name in /nix/store/v60nbi08l4cvn6sw37db35riam3hjjkp-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"352client # [84894.697561] client systemd[1]: Started D-Bus System Message Bus.353client # [84894.698156] client systemd[1]: Startup finished in 4.228s.354client # [84894.703931] client dbus-broker-launch[321]: Ready355relay # [84894.683875] relay dnsmasq[394]: started, version 2.93 cachesize 150356relay # [84894.683886] relay dnsmasq[394]: compile time options: IPv6 GNU-getopt DBus no-UBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset nftset auth DNSSEC loop-detect inotify dumpfile357relay # [84894.683892] relay dnsmasq[394]: DBus support enabled: connected to system bus358relay # [84894.683902] relay dnsmasq[394]: no upstream servers configured - please set them from DBus359relay # [84894.684008] relay dnsmasq[394]: read /etc/hosts - 9 names360agent # [84894.955341] agent systemd-logind[763]: New seat seat0.361agent # [84894.956796] agent systemd[1]: Starting D-Bus System Message Bus...362agent # [84894.956854] agent systemd[1]: Started User Login Management.363agent # [84894.957645] agent systemd[1]: Starting linger-users.service...364agent # [84894.995734] agent systemd[1]: linger-users.service: Deactivated successfully.365agent # [84894.996183] agent systemd[1]: Finished linger-users.service.366agent # [84894.997457] agent systemd[1]: Reached target Multi-User System.367relay # [84895.441560] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:16.570565Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}368relay # [84895.442425] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:16.571215Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}369relay # [84895.441982] relay systemd[1]: Started flakelet-relay.370relay # [84895.454975] relay systemd[1]: Starting health for flakelet-relay...371relay # [84895.690358] relay systemd[1]: flakelet-relay-health.service: Deactivated successfully.372relay # [84895.690850] relay systemd[1]: Finished health for flakelet-relay.373agent # [84895.394496] agent dbus-broker-launch[839]: Looking up NSS user entry for 'systemd-timesync'...374agent # [84895.396209] agent dbus-broker-launch[839]: NSS returned no entry for 'systemd-timesync'375agent # [84895.396209] agent dbus-broker-launch[839]: Invalid user-name in /nix/store/q3c2gxfm178wqgzv8m87na4aa72wxc04-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"376agent # [84895.397247] agent systemd[1]: Started D-Bus System Message Bus.377agent # [84895.397860] agent systemd[1]: Startup finished in 4.920s.378agent # [84895.403638] agent dbus-broker-launch[839]: Ready379relay: (finished: waiting for unit flakelet-relay.service, in 6.16 seconds)380relay: waiting for unit nginx.service381relay: (finished: waiting for unit nginx.service, in 0.02 seconds)382agent: waiting for unit flakelet.target383agent: (finished: waiting for unit flakelet.target, in 0.01 seconds)384agent: waiting for unit flakelet-agent.service385agent: (finished: waiting for unit flakelet-agent.service, in 0.02 seconds)386client: waiting for unit multi-user.target387relay # [84895.739040] relay flakelet[312]: flakelet-relay: updated to generation 1388relay # [84895.739999] relay systemd[1]: Finished Update flakelet service flakelet-relay.389relay # [84895.741865] relay systemd[1]: Reached target flakelet managed services.390relay # [84895.742215] relay systemd[1]: Reached target Multi-User System.391relay # [84895.742506] relay systemd[1]: Startup finished in 5.264s.392client: (finished: waiting for unit multi-user.target, in 0.02 seconds)393relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1'394agent # [84897.510779] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:18.640010Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}395relay # [84897.511095] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:18.640286Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(1), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}396relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1', in 2.05 seconds)397agent: must succeed: systemctl is-active app.service other.service398agent: (finished: must succeed: systemctl is-active app.service other.service, in 0.01 seconds)399subtest: deploy with client certificate, relay found via SRV400agent: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json401agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)402client: must succeed: push --relay-srv test --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app 2>&1403agent # [84898.252132] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:19.381338Z","level":"INFO","message":"start","id":"c71387b48c60d0082f847f3bb1320c43","flakelet":"app","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}404agent # [84898.252967] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:19.382209Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"c71387b48c60d0082f847f3bb1320c43\"]","target":"flakelet_relay::agent::jobs"}405agent # [84898.315877] agent systemd[1]: Starting flakelet update app (via relay)...406agent # [84898.355731] agent systemd[1]: Started flakelet update app (via relay).407agent # [84898.360347] agent flakelet[868]: app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2408agent # [84898.361162] agent flakelet[868]: app: activating generation 2409agent # [84898.375984] agent systemd[1]: Stopping app v1...410agent # [84898.377632] agent systemd[1]: app.service: Deactivated successfully.411agent # [84898.378052] agent systemd[1]: Stopped app v1.412agent # [84898.397804] agent systemd[1]: Reload requested from client PID 875 ('systemctl') (unit flakelet-relay-job-app.service)...413agent # [84898.397896] agent systemd[1]: Reloading...414relay # [84898.251430] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:19.380666Z","level":"INFO","message":"deploy accepted","job":"64f7f2b288e4b08effff548ddbe8c0a6","caller":"x509:email:admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}415agent # [84898.785466] agent systemd[1]: Reloading finished in 386 ms.416agent # [84898.868930] agent systemd[1]: Reload requested from client PID 905 ('systemctl') (unit flakelet-relay-job-app.service)...417agent # [84898.868980] agent systemd[1]: Reloading...418agent # [84899.215075] agent systemd[1]: Reloading finished in 345 ms.419agent # [84899.301481] agent systemd[1]: Reload requested from client PID 934 ('systemctl') (unit flakelet-relay-job-app.service)...420agent # [84899.301526] agent systemd[1]: Reloading...421agent # [84899.646747] agent systemd[1]: Reloading finished in 344 ms.422agent # [84899.736479] agent systemd[1]: Reload requested from client PID 965 ('systemctl') (unit flakelet-relay-job-app.service)...423agent # [84899.736529] agent systemd[1]: Reloading...424agent # [84900.078697] agent systemd[1]: Reloading finished in 341 ms.425agent # [84900.185949] agent systemd[1]: Starting app v2...426agent # [84900.224163] agent systemd[1]: Started app v2.427agent # [84900.244334] agent flakelet[868]: app: updated to generation 2428agent # [84900.247512] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.429client: (finished: must succeed: push --relay-srv test --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app 2>&1, in 3.07 seconds)430agent: must succeed: systemctl show app.service -p Description | grep -q 'app v2'431agent: (finished: must succeed: systemctl show app.service -p Description | grep -q 'app v2', in 0.02 seconds)432agent: must succeed: journalctl -u flakelet-relay-job-app.service | grep -q 'generation 2'433agent: (finished: must succeed: journalctl -u flakelet-relay-job-app.service | grep -q 'generation 2', in 0.01 seconds)434(finished: subtest: deploy with client certificate, relay found via SRV, in 3.11 seconds)435subtest: deploy with OIDC token, sub principal436agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json437agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)438client: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)439client: (finished: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.05 seconds)440client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTU2MiwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njc1MjYyLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiI3UE5DMzI1TVFGTUVDPT09IiwibmJmIjoxNzg4Njc1MjYyLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.pzsRuQ342V_9xYJozPAtqQQCCfZ-GZKDoC1XzqSA5I0XJIY8XpLYTkVe7lAkqk2igIjR0e8FF7MgCbXNhkeVDA' push --relay https://relay:7443 deploy agent/app 2>&1441agent # [84901.293286] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:22.422520Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(2)","target":"flakelet_relay::agent::jobs"}442agent # [84901.419748] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:22.548949Z","level":"INFO","message":"start","id":"0764d2594c59cf0d5e788838926e09e6","flakelet":"app","rule":"ci","relay":"relay","caller":"Some(\"oidc:mock:repo:github:example/app:ref:refs/heads/main\")","target":"flakelet_relay::agent::conn"}443agent # [84901.420402] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:22.549641Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"0764d2594c59cf0d5e788838926e09e6\"]","target":"flakelet_relay::agent::jobs"}444agent # [84901.479689] agent systemd[1]: Starting flakelet update app (via relay)...445agent # [84901.512782] agent systemd[1]: Started flakelet update app (via relay).446agent # [84901.516755] agent flakelet[1023]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1447agent # [84901.517336] agent flakelet[1023]: app: activating generation 3448agent # [84901.530235] agent systemd[1]: Stopping app v2...449agent # [84901.531465] agent systemd[1]: app.service: Deactivated successfully.450agent # [84901.531827] agent systemd[1]: Stopped app v2.451agent # [84901.546237] agent systemd[1]: Reload requested from client PID 1030 ('systemctl') (unit flakelet-relay-job-app.service)...452agent # [84901.546288] agent systemd[1]: Reloading...453relay # [84901.294242] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:22.423512Z","level":"INFO","message":"deploy finished","job":"64f7f2b288e4b08effff548ddbe8c0a6","ok":true,"target":"flakelet_relay::relay::api"}454relay # [84901.419244] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:22.548460Z","level":"INFO","message":"deploy accepted","job":"6476ff633eea33410ea9031111ce8cd8","caller":"oidc:mock:repo:github:example/app:ref:refs/heads/main","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}455agent # [84901.889805] agent systemd[1]: Reloading finished in 343 ms.456agent # [84902.000447] agent systemd[1]: Reload requested from client PID 1060 ('systemctl') (unit flakelet-relay-job-app.service)...457agent # [84902.000540] agent systemd[1]: Reloading...458agent # [84902.483661] agent systemd[1]: Reloading finished in 482 ms.459agent # [84902.566414] agent systemd[1]: Reload requested from client PID 1090 ('systemctl') (unit flakelet-relay-job-app.service)...460agent # [84902.566465] agent systemd[1]: Reloading...461agent # [84902.936521] agent systemd[1]: Reloading finished in 369 ms.462agent # [84903.028172] agent systemd[1]: Reload requested from client PID 1120 ('systemctl') (unit flakelet-relay-job-app.service)...463agent # [84903.028263] agent systemd[1]: Reloading...464agent # [84903.395189] agent systemd[1]: Reloading finished in 366 ms.465agent # [84903.488691] agent systemd[1]: Starting app v1...466agent # [84903.531229] agent systemd[1]: Started app v1.467agent # [84903.555061] agent flakelet[1023]: app: updated to generation 3468agent # [84903.558274] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.469client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTU2MiwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njc1MjYyLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiI3UE5DMzI1TVFGTUVDPT09IiwibmJmIjoxNzg4Njc1MjYyLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.pzsRuQ342V_9xYJozPAtqQQCCfZ-GZKDoC1XzqSA5I0XJIY8XpLYTkVe7lAkqk2igIjR0e8FF7MgCbXNhkeVDA' push --relay https://relay:7443 deploy agent/app 2>&1, in 2.74 seconds)470(finished: subtest: deploy with OIDC token, sub principal, in 2.81 seconds)471subtest: policy denies uncovered target472client: must fail: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTU2MiwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njc1MjYyLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiI3UE5DMzI1TVFGTUVDPT09IiwibmJmIjoxNzg4Njc1MjYyLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.pzsRuQ342V_9xYJozPAtqQQCCfZ-GZKDoC1XzqSA5I0XJIY8XpLYTkVe7lAkqk2igIjR0e8FF7MgCbXNhkeVDA' push --relay https://relay:7443 deploy agent/other 2>&1473client: (finished: must fail: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTU2MiwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njc1MjYyLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiI3UE5DMzI1TVFGTUVDPT09IiwibmJmIjoxNzg4Njc1MjYyLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.pzsRuQ342V_9xYJozPAtqQQCCfZ-GZKDoC1XzqSA5I0XJIY8XpLYTkVe7lAkqk2igIjR0e8FF7MgCbXNhkeVDA' push --relay https://relay:7443 deploy agent/other 2>&1, in 0.02 seconds)474(finished: subtest: policy denies uncovered target, in 0.02 seconds)475subtest: claim principal from groups, unchanged counts as success476client: must succeed: echo '{"groups": ["wheel"]}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'someone' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)477client: (finished: must succeed: echo '{"groups": ["wheel"]}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'someone' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.06 seconds)478client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTU2NSwiZ3JvdXBzIjpbIndoZWVsIl0sImlhdCI6MTc4ODY3NTI2NSwiaXNzIjoiaHR0cHM6Ly9yZWxheTo4NDQzIiwianRpIjoiNVFXTlRXRURKRVRPWT09PSIsIm5iZiI6MTc4ODY3NTI2NSwic3ViIjoic29tZW9uZSJ9.-7ZuQGs607b52Yb_DtIHF-aQyI_WlNSJjcJULe50GYHgxqeuv3rTALO_JplK3DRSQB_SftME4lpDomR8a5dsBQ' push --relay https://relay:7443 deploy agent/other 2>&1479relay # [84904.128302] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:25.257557Z","level":"INFO","message":"deploy finished","job":"6476ff633eea33410ea9031111ce8cd8","ok":true,"target":"flakelet_relay::relay::api"}480relay # [84904.232357] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:25.361522Z","level":"INFO","message":"deploy accepted","job":"1e4440b23112bc61e3383192bc14d361","caller":"oidc:mock:groups:wheel\noidc:mock:someone","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}481agent # [84904.127154] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:25.256395Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}482agent # [84904.232668] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:25.361907Z","level":"INFO","message":"start","id":"76bc1d9561d9628ca52dba041a75c71c","flakelet":"other","rule":"admins","relay":"relay","caller":"Some(\"oidc:mock:groups:wheel\\noidc:mock:someone\")","target":"flakelet_relay::agent::conn"}483agent # [84904.232962] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:25.362263Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"76bc1d9561d9628ca52dba041a75c71c\"]","target":"flakelet_relay::agent::jobs"}484agent # [84904.286950] agent systemd[1]: Starting flakelet update other (via relay)...485agent # [84904.317745] agent systemd[1]: Started flakelet update other (via relay).486agent # [84904.321910] agent flakelet[1169]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1487agent # [84904.322442] agent flakelet[1169]: other: up to date488agent # [84904.325579] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.489client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTU2NSwiZ3JvdXBzIjpbIndoZWVsIl0sImlhdCI6MTc4ODY3NTI2NSwiaXNzIjoiaHR0cHM6Ly9yZWxheTo4NDQzIiwianRpIjoiNVFXTlRXRURKRVRPWT09PSIsIm5iZiI6MTc4ODY3NTI2NSwic3ViIjoic29tZW9uZSJ9.-7ZuQGs607b52Yb_DtIHF-aQyI_WlNSJjcJULe50GYHgxqeuv3rTALO_JplK3DRSQB_SftME4lpDomR8a5dsBQ' push --relay https://relay:7443 deploy agent/other 2>&1, in 0.67 seconds)490(finished: subtest: claim principal from groups, unchanged counts as success, in 0.73 seconds)491subtest: push login via device flow, cached token used afterwards492client: must succeed: push login --issuer https://relay:8443 --client-id flakelet-relay 2>&1493agent # [84904.883708] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:26.012929Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}494relay # [84904.884526] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:26.013741Z","level":"INFO","message":"deploy finished","job":"1e4440b23112bc61e3383192bc14d361","ok":true,"target":"flakelet_relay::relay::api"}495relay # [84904.906729] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[185]: 127.0.0.1 - - [06/Sep/2026 06:14:26] "POST /device HTTP/1.1" 200 -496relay # [84905.911763] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[185]: 127.0.0.1 - - [06/Sep/2026 06:14:27] "POST /token HTTP/1.1" 400 -497client: (finished: must succeed: push login --issuer https://relay:8443 --client-id flakelet-relay 2>&1, in 2.07 seconds)498client: must succeed: test "$(stat -c %a /root/.local/state/flakelet-push/token.json)" = 600499client: (finished: must succeed: test "$(stat -c %a /root/.local/state/flakelet-push/token.json)" = 600, in 0.01 seconds)500client: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1501agent # [84906.989522] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:28.118760Z","level":"INFO","message":"start","id":"991113e51b43c22235c1464a87111fde","flakelet":"other","rule":"admins","relay":"relay","caller":"Some(\"oidc:mock:groups:wheel\\noidc:mock:someone\")","target":"flakelet_relay::agent::conn"}502agent # [84906.990418] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:28.119366Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"991113e51b43c22235c1464a87111fde\"]","target":"flakelet_relay::agent::jobs"}503agent # [84907.047406] agent systemd[1]: Starting flakelet update other (via relay)...504agent # [84907.085894] agent systemd[1]: Started flakelet update other (via relay).505agent # [84907.089888] agent flakelet[1187]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1506agent # [84907.090618] agent flakelet[1187]: other: up to date507agent # [84907.093655] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.508relay # [84906.959257] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[185]: 127.0.0.1 - - [06/Sep/2026 06:14:28] "POST /token HTTP/1.1" 200 -509relay # [84906.989159] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:28.118385Z","level":"INFO","message":"deploy accepted","job":"9e1d862b4d6bdc0e323cf88d6efb31c4","caller":"oidc:mock:groups:wheel\noidc:mock:someone","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}510client: (finished: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1, in 0.68 seconds)511client: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1512agent # [84907.652096] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:28.781282Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}513agent # [84907.672059] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:28.801293Z","level":"INFO","message":"start","id":"da7d292a0aee6dd59d22d12c6a230e07","flakelet":"other","rule":"admins","relay":"relay","caller":"Some(\"oidc:mock:groups:wheel\\noidc:mock:someone\")","target":"flakelet_relay::agent::conn"}514agent # [84907.672573] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:28.801861Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"da7d292a0aee6dd59d22d12c6a230e07\"]","target":"flakelet_relay::agent::jobs"}515agent # [84907.737808] agent systemd[1]: Starting flakelet update other (via relay)...516agent # [84907.769874] agent systemd[1]: Started flakelet update other (via relay).517agent # [84907.773939] agent flakelet[1205]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1518agent # [84907.774676] agent flakelet[1205]: other: up to date519agent # [84907.777595] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.520relay # [84907.652864] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:28.782126Z","level":"INFO","message":"deploy finished","job":"9e1d862b4d6bdc0e323cf88d6efb31c4","ok":true,"target":"flakelet_relay::relay::api"}521relay # [84907.671681] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:28.800898Z","level":"INFO","message":"deploy accepted","job":"e85d2118b961ea5b94230608a0b50531","caller":"oidc:mock:groups:wheel\noidc:mock:someone","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}522client: (finished: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1, in 0.68 seconds)523(finished: subtest: push login via device flow, cached token used afterwards, in 3.45 seconds)524subtest: waves: failing first wave skips the second525agent: must succeed: jq '.services.app.prebuilt="/nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json526agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)527client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave agent/other 2>&1528relay # [84908.333350] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:29.462576Z","level":"INFO","message":"deploy finished","job":"e85d2118b961ea5b94230608a0b50531","ok":true,"target":"flakelet_relay::relay::api"}529relay # [84908.368872] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:29.498090Z","level":"INFO","message":"deploy accepted","job":"d08097230363f39d28594bf67127750f","caller":"x509:email:admin@example.org","targets":"[\"agent/app\", \"agent/other\"]","target":"flakelet_relay::relay::api"}530agent # [84908.332497] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:29.461709Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}531agent # [84908.369276] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:29.498487Z","level":"INFO","message":"start","id":"34761b44e9b831e525d4ebf0f71ea282","flakelet":"app","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}532agent # [84908.369726] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:29.498993Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"34761b44e9b831e525d4ebf0f71ea282\"]","target":"flakelet_relay::agent::jobs"}533agent # [84908.423926] agent systemd[1]: Starting flakelet update app (via relay)...534agent # [84908.463134] agent systemd[1]: Started flakelet update app (via relay).535agent # [84908.468210] agent flakelet[1226]: app: using prebuilt artifact /nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken536agent # [84908.469115] agent flakelet[1226]: app: activating generation 4537agent # [84908.483503] agent systemd[1]: Stopping app v1...538agent # [84908.485346] agent systemd[1]: app.service: Deactivated successfully.539agent # [84908.485769] agent systemd[1]: Stopped app v1.540agent # [84908.505375] agent systemd[1]: Reload requested from client PID 1233 ('systemctl') (unit flakelet-relay-job-app.service)...541agent # [84908.505464] agent systemd[1]: Reloading...542agent # [84908.864914] agent systemd[1]: Reloading finished in 358 ms.543agent # [84908.955085] agent systemd[1]: Reload requested from client PID 1263 ('systemctl') (unit flakelet-relay-job-app.service)...544agent # [84908.955144] agent systemd[1]: Reloading...545agent # [84909.297071] agent systemd[1]: Reloading finished in 341 ms.546agent # [84909.398277] agent systemd[1]: Reload requested from client PID 1292 ('systemctl') (unit flakelet-relay-job-app.service)...547agent # [84909.398323] agent systemd[1]: Reloading...548agent # [84909.740451] agent systemd[1]: Reloading finished in 341 ms.549agent # [84909.833805] agent systemd[1]: Reload requested from client PID 1323 ('systemctl') (unit flakelet-relay-job-app.service)...550agent # [84909.833856] agent systemd[1]: Reloading...551agent # [84910.179547] agent systemd[1]: Reloading finished in 345 ms.552agent # [84910.262215] agent systemd[1]: Starting app broken...553agent # [84910.301479] agent (nonexistent)[1353]: app.service: Unable to locate executable '/nonexistent': No such file or directory554agent # [84910.301504] agent (nonexistent)[1353]: app.service: Failed at step EXEC spawning /nonexistent: No such file or directory555agent # [84910.302321] agent systemd[1]: app.service: Main process exited, code=exited, status=203/EXEC556agent # [84910.302561] agent systemd[1]: app.service: Failed with result 'exit-code'.557agent # [84910.303027] agent systemd[1]: Failed to start app broken.558agent # [84910.364469] agent systemd[1]: Reload requested from client PID 1360 ('systemctl') (unit flakelet-relay-job-app.service)...559agent # [84910.364566] agent systemd[1]: Reloading...560agent # [84910.713512] agent systemd[1]: Reloading finished in 348 ms.561agent # [84910.798946] agent systemd[1]: Reload requested from client PID 1390 ('systemctl') (unit flakelet-relay-job-app.service)...562agent # [84910.799019] agent systemd[1]: Reloading...563agent # [84911.142118] agent systemd[1]: Reloading finished in 342 ms.564agent # [84911.230076] agent systemd[1]: Reload requested from client PID 1420 ('systemctl') (unit flakelet-relay-job-app.service)...565agent # [84911.230117] agent systemd[1]: Reloading...566agent # [84911.574801] agent systemd[1]: Reloading finished in 344 ms.567agent # [84911.661198] agent systemd[1]: Reload requested from client PID 1450 ('systemctl') (unit flakelet-relay-job-app.service)...568agent # [84911.661255] agent systemd[1]: Reloading...569agent # [84912.008418] agent systemd[1]: Reloading finished in 346 ms.570agent # [84912.110117] agent systemd[1]: Starting app v1...571agent # [84912.144425] agent systemd[1]: Started app v1.572agent # [84912.145554] agent flakelet[1226]: app: deploy failed, rolled back: systemctl start app.service failed:573agent # [84912.145554] agent flakelet[1226]: failed units: app.service. See journalctl -u app.service574agent # [84912.148155] agent systemd[1]: flakelet-relay-job-app.service: Main process exited, code=exited, status=1/FAILURE575agent # [84912.148392] agent systemd[1]: flakelet-relay-job-app.service: Failed with result 'exit-code'.576client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave agent/other 2>&1, in 4.46 seconds)577agent: must succeed: systemctl show app.service -p Description | grep -q 'app v1'578agent: (finished: must succeed: systemctl show app.service -p Description | grep -q 'app v1', in 0.02 seconds)579(finished: subtest: waves: failing first wave skips the second, in 4.49 seconds)580subtest: detach returns once the job is accepted581agent: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json582agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)583client: must succeed: timeout 20 push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --detach agent/app 2>&1584client: (finished: must succeed: timeout 20 push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --detach agent/app 2>&1, in 0.02 seconds)585agent: waiting for success: systemctl is-active flakelet-relay-job-app.service586agent # [84912.804508] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:33.933724Z","level":"INFO","message":"update finished","flakelet":"app","status":"RolledBack","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}587agent # [84912.858136] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:33.987374Z","level":"INFO","message":"start","id":"400a14c4928937bd6c6a5d490babd64e","flakelet":"app","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}588agent # [84912.858411] agent flakelet-agent[307]: {"timestamp":"2026-09-06T06:14:33.987712Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"400a14c4928937bd6c6a5d490babd64e\"]","target":"flakelet_relay::agent::jobs"}589agent # [84912.913297] agent systemd[1]: Starting flakelet update app (via relay)...590agent # [84912.946272] agent systemd[1]: Started flakelet update app (via relay).591agent # [84912.950886] agent flakelet[1505]: app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2592agent # [84912.951624] agent flakelet[1505]: app: activating generation 4593agent # [84912.971176] agent systemd[1]: Stopping app v1...594agent # [84912.971649] agent systemd[1]: app.service: Deactivated successfully.595agent # [84912.972033] agent systemd[1]: Stopped app v1.596agent # [84912.992375] agent systemd[1]: Reload requested from client PID 1512 ('systemctl') (unit flakelet-relay-job-app.service)...597agent # [84912.992466] agent systemd[1]: Reloading...598relay # [84912.805272] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:33.934504Z","level":"INFO","message":"deploy finished","job":"d08097230363f39d28594bf67127750f","ok":false,"target":"flakelet_relay::relay::api"}599relay # [84912.857770] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:33.987015Z","level":"INFO","message":"deploy accepted","job":"af8f1867d888b4135eca3529e668a551","caller":"x509:email:admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}600agent # [84913.340635] agent systemd[1]: Reloading finished in 347 ms.601agent # [84913.431562] agent systemd[1]: Reload requested from client PID 1542 ('systemctl') (unit flakelet-relay-job-app.service)...602agent # [84913.431652] agent systemd[1]: Reloading...603agent # [84913.777595] agent systemd[1]: Reloading finished in 345 ms.604agent # [84913.869057] agent systemd[1]: Reload requested from client PID 1571 ('systemctl') (unit flakelet-relay-job-app.service)...605agent # [84913.869106] agent systemd[1]: Reloading...606agent: (finished: waiting for success: systemctl is-active flakelet-relay-job-app.service, in 1.43 seconds)607??? Warning (UserWarning): wait_until_fails(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.608 File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39609agent: waiting for failure: systemctl is-active flakelet-relay-job-app.service610??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.611 File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39612agent # [84914.209982] agent systemd[1]: Reloading finished in 340 ms.613agent # [84914.298213] agent systemd[1]: Reload requested from client PID 1604 ('systemctl') (unit flakelet-relay-job-app.service)...614agent # [84914.298266] agent systemd[1]: Reloading...615agent # [84914.639982] agent systemd[1]: Reloading finished in 341 ms.616agent # [84914.722789] agent systemd[1]: Starting app slow2...617agent # [84920.790571] agent systemd[1]: Started app slow2.618agent # [84920.813816] agent flakelet[1505]: app: updated to generation 4619agent # [84920.817092] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.620agent: (finished: waiting for failure: systemctl is-active flakelet-relay-job-app.service, in 7.51 seconds)621(finished: subtest: detach returns once the job is accepted, in 8.98 seconds)622subtest: unknown host is final, missing agent is retried, agents listing623client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy nope/app 2>&1624client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy nope/app 2>&1, in 0.02 seconds)625agent: must succeed: systemctl stop flakelet-agent626agent: (finished: must succeed: systemctl stop flakelet-agent, in 0.04 seconds)627relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 0'628relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 0', in 0.02 seconds)629client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/other > /tmp/retry.out 2>&1 &630client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/other > /tmp/retry.out 2>&1 &, in 0.01 seconds)631client: waiting for success: grep -q 'agent_unavailable.*retrying' /tmp/retry.out632relay # [84921.833460] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:42.962664Z","level":"INFO","message":"agent disconnected: websocket closed","host":"agent","target":"flakelet_relay::relay::agent_conn"}633agent # [84921.833316] agent systemd[1]: Stopping flakelet-agent...634agent # [84921.857649] agent systemd[1]: flakelet-agent.service: Deactivated successfully.635agent # [84921.858365] agent systemd[1]: Stopped flakelet-agent.636agent # [84921.858835] agent systemd[1]: flakelet-agent.service: Consumed 1.048s CPU time over 29.530s wall clock time.637client: (finished: waiting for success: grep -q 'agent_unavailable.*retrying' /tmp/retry.out, in 1.02 seconds)638agent: must succeed: systemctl start flakelet-agent639agent: (finished: must succeed: systemctl start flakelet-agent, in 0.07 seconds)640??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.641 File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39642client: waiting for success: grep -q 'agent/other: unchanged' /tmp/retry.out643agent # [84922.917491] agent systemd[1]: Starting flakelet-agent...644agent # [84922.973809] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:44.103092Z","level":"INFO","message":"reattaching to running unit","flakelet":"app","ids":"[\"400a14c4928937bd6c6a5d490babd64e\"]","target":"flakelet_relay::agent::jobs"}645agent # [84922.973875] agent systemd[1]: Started flakelet-agent.646agent # [84923.054913] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:44.184159Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}647relay # [84923.054920] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:44.184171Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(4), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}648agent # [84923.529302] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:44.658521Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(4)","target":"flakelet_relay::agent::jobs"}649agent # [84924.909800] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:46.039007Z","level":"INFO","message":"start","id":"ce1f0daf2126c10edb69e12920652642","flakelet":"other","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}650agent # [84924.910844] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:46.039747Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"ce1f0daf2126c10edb69e12920652642\"]","target":"flakelet_relay::agent::jobs"}651agent # [84924.960123] agent systemd[1]: Starting flakelet update other (via relay)...652agent # [84924.988224] agent systemd[1]: Started flakelet update other (via relay).653agent # [84924.992824] agent flakelet[2071]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1654agent # [84924.993589] agent flakelet[2071]: other: up to date655agent # [84924.995890] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.656relay # [84924.909314] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:46.038571Z","level":"INFO","message":"deploy accepted","job":"8da82e42f16b0787ca00efd205363268","caller":"x509:email:admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}657agent # [84925.541030] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:46.670245Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}658relay # [84925.541572] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:46.670833Z","level":"INFO","message":"deploy finished","job":"8da82e42f16b0787ca00efd205363268","ok":true,"target":"flakelet_relay::relay::api"}659client: (finished: waiting for success: grep -q 'agent/other: unchanged' /tmp/retry.out, in 3.04 seconds)660client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents661client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents, in 0.02 seconds)662client: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)663client: (finished: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.05 seconds)664client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTU4NywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njc1Mjg3LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiI3RUZCMk02U1lLRVhLPT09IiwibmJmIjoxNzg4Njc1Mjg3LCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.wsgI63JxG-gIc8X6drKo_FtIZ-KwbEsizY98qWMy4oOLpnFoeex_d9WS5hdtEbpfVzxkB7Nc7rMpan-o6ijSBQ' push --relay https://relay:7443 agents665client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTU4NywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njc1Mjg3LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiI3RUZCMk02U1lLRVhLPT09IiwibmJmIjoxNzg4Njc1Mjg3LCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.wsgI63JxG-gIc8X6drKo_FtIZ-KwbEsizY98qWMy4oOLpnFoeex_d9WS5hdtEbpfVzxkB7Nc7rMpan-o6ijSBQ' push --relay https://relay:7443 agents, in 0.02 seconds)666(finished: subtest: unknown host is final, missing agent is retried, agents listing, in 4.30 seconds)667subtest: certificate not listed under agents cannot connect as agent668client: must succeed: curl -s -o /dev/null -w '%{http_code}' --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/cert.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/key.pem -H 'Upgrade: websocket' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' https://relay:7443/v1/agent669client: (finished: must succeed: curl -s -o /dev/null -w '%{http_code}' --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/cert.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/key.pem -H 'Upgrade: websocket' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' https://relay:7443/v1/agent, in 0.02 seconds)670(finished: subtest: certificate not listed under agents cannot connect as agent, in 0.02 seconds)671subtest: metrics672relay: must succeed: curl -sf http://127.0.0.1:7400/metrics673relay: (finished: must succeed: curl -sf http://127.0.0.1:7400/metrics, in 0.02 seconds)674(finished: subtest: metrics, in 0.02 seconds)675subtest: agent link drop mid-run replays missed lines once676agent: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json677agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)678agent: must succeed: date +%s679agent: (finished: must succeed: date +%s, in 0.01 seconds)680client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/slow.out 2>&1 &681client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/slow.out 2>&1 &, in 0.01 seconds)682agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788675287 | grep -q 'app-slow'683relay # [84926.116795] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:47.246042Z","level":"INFO","message":"agent rejected: no agents entry matches","principals":"[\"x509:dns:mallory\"]","target":"flakelet_relay::relay::agent_conn"}684relay # [84926.174711] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:47.303971Z","level":"INFO","message":"deploy accepted","job":"132127c9efc7e2f693d64052e27ca6b3","caller":"x509:email:admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}685agent # [84926.175178] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:47.304419Z","level":"INFO","message":"start","id":"b8ce431af3a942fe458f59c0d7b3800b","flakelet":"app","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}686agent # [84926.176057] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:47.305156Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"b8ce431af3a942fe458f59c0d7b3800b\"]","target":"flakelet_relay::agent::jobs"}687agent # [84926.224100] agent systemd[1]: Starting flakelet update app (via relay)...688agent # [84926.259892] agent systemd[1]: Started flakelet update app (via relay).689agent # [84926.264201] agent flakelet[2096]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow690agent # [84926.264833] agent flakelet[2096]: app: activating generation 5691agent # [84926.278324] agent systemd[1]: Stopping app slow2...692agent # [84926.279222] agent systemd[1]: app.service: Deactivated successfully.693agent # [84926.279436] agent systemd[1]: Stopped app slow2.694agent # [84926.291408] agent systemd[1]: Reload requested from client PID 2103 ('systemctl') (unit flakelet-relay-job-app.service)...695agent # [84926.291451] agent systemd[1]: Reloading...696agent # [84926.638929] agent systemd[1]: Reloading finished in 347 ms.697agent # [84926.733682] agent systemd[1]: Reload requested from client PID 2133 ('systemctl') (unit flakelet-relay-job-app.service)...698agent # [84926.733728] agent systemd[1]: Reloading...699agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788675287 | grep -q 'app-slow', in 1.02 seconds)700agent: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route add unreachable $a; done701agent: (finished: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route add unreachable $a; done, in 0.02 seconds)702relay: must succeed: for a in $(getent ahosts agent | cut -d' ' -f1 | sort -u); do ss -K dst "[$a]"; done703relay: (finished: must succeed: for a in $(getent ahosts agent | cut -d' ' -f1 | sort -u); do ss -K dst "[$a]"; done, in 0.06 seconds)704agent: must succeed: ss -K dport = 7443705agent: (finished: must succeed: ss -K dport = 7443, in 0.03 seconds)706agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788675287 | grep -q 'updated to generation'707agent # [84927.078135] agent systemd[1]: Reloading finished in 343 ms.708agent # [84927.154298] agent systemd[1]: Reload requested from client PID 2162 ('systemctl') (unit flakelet-relay-job-app.service)...709agent # [84927.154346] agent systemd[1]: Reloading...710agent # [84927.291210] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:48.420375Z","level":"INFO","message":"disconnected: Software caused connection abort (os error 103)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}711relay # [84927.256402] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:48.385609Z","level":"INFO","message":"agent disconnected: Software caused connection abort (os error 103)","host":"agent","target":"flakelet_relay::relay::agent_conn"}712agent # [84927.494537] agent systemd[1]: Reloading finished in 339 ms.713agent # [84927.575363] agent systemd[1]: Reload requested from client PID 2207 ('systemctl') (unit flakelet-relay-job-app.service)...714agent # [84927.575402] agent systemd[1]: Reloading...715agent # [84927.914536] agent systemd[1]: Reloading finished in 338 ms.716agent # [84927.988105] agent systemd[1]: Starting app slow...717agent # [84928.379328] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:49.508547Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}718agent # [84930.463627] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:51.592861Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}719agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788675287 | grep -q 'updated to generation', in 7.10 seconds)720agent: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route del unreachable $a; done721agent: (finished: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route del unreachable $a; done, in 0.02 seconds)722client: waiting for success: grep -q 'agent/app: updated' /tmp/slow.out723agent # [84934.061479] agent systemd[1]: Started app slow.724agent # [84934.080176] agent flakelet[2096]: app: updated to generation 5725agent # [84934.080790] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.726agent # [84934.543991] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:55.673071Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}727agent # [84934.545311] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:55.674598Z","level":"INFO","message":"start","id":"b8ce431af3a942fe458f59c0d7b3800b","flakelet":"app","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}728relay # [84934.543918] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:55.673133Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(5), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}729client: (finished: waiting for success: grep -q 'agent/app: updated' /tmp/slow.out, in 1.02 seconds)730client: must succeed: cat /tmp/slow.out731client: (finished: must succeed: cat /tmp/slow.out, in 0.01 seconds)732(finished: subtest: agent link drop mid-run replays missed lines once, in 9.30 seconds)733subtest: agent restart mid-run reattaches to the unit734agent: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json735agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)736client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/restart.out 2>&1 &737client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/restart.out 2>&1 &, in 0.01 seconds)738agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since -20s | grep -q 'app-slow2'739relay # [84935.163428] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:56.292678Z","level":"INFO","message":"deploy finished","job":"132127c9efc7e2f693d64052e27ca6b3","ok":true,"target":"flakelet_relay::relay::api"}740relay # [84935.465359] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:56.594516Z","level":"INFO","message":"deploy accepted","job":"53596fefcdd8b6fa05d0f13f8544d3cd","caller":"x509:email:admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}741agent # [84935.162791] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:56.292030Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(5)","target":"flakelet_relay::agent::jobs"}742agent # [84935.465758] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:56.594884Z","level":"INFO","message":"start","id":"e64e06db46a14587f0092f1df49c8540","flakelet":"app","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}743agent # [84935.465980] agent flakelet-agent[1658]: {"timestamp":"2026-09-06T06:14:56.595271Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"e64e06db46a14587f0092f1df49c8540\"]","target":"flakelet_relay::agent::jobs"}744agent # [84935.518695] agent systemd[1]: Starting flakelet update app (via relay)...745agent # [84935.548734] agent systemd[1]: Started flakelet update app (via relay).746agent # [84935.553095] agent flakelet[2328]: app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2747agent # [84935.553748] agent flakelet[2328]: app: activating generation 6748agent # [84935.565886] agent systemd[1]: Stopping app slow...749agent # [84935.566788] agent systemd[1]: app.service: Deactivated successfully.750agent # [84935.566995] agent systemd[1]: Stopped app slow.751agent # [84935.582995] agent systemd[1]: Reload requested from client PID 2335 ('systemctl') (unit flakelet-relay-job-app.service)...752agent # [84935.583046] agent systemd[1]: Reloading...753agent # [84935.930575] agent systemd[1]: Reloading finished in 347 ms.754agent # [84936.014627] agent systemd[1]: Reload requested from client PID 2365 ('systemctl') (unit flakelet-relay-job-app.service)...755agent # [84936.014671] agent systemd[1]: Reloading...756agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since -20s | grep -q 'app-slow2', in 1.02 seconds)757agent: must succeed: systemctl restart flakelet-agent758agent # [84936.350791] agent systemd[1]: Reloading finished in 335 ms.759agent # [84936.435515] agent systemd[1]: Reload requested from client PID 2394 ('systemctl') (unit flakelet-relay-job-app.service)...760agent # [84936.435562] agent systemd[1]: Reloading...761agent # [84936.809436] agent systemd[1]: Reloading finished in 373 ms.762agent # [84936.895163] agent systemd[1]: Stopping flakelet-agent...763agent # [84936.898252] agent systemd[1]: Reload requested from client PID 2429 ('systemctl') (unit flakelet-relay-job-app.service)...764agent # [84936.898291] agent systemd[1]: Reloading...765agent # [84937.266395] agent systemd[1]: Reloading finished in 367 ms.766relay # [84936.895393] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:58.024578Z","level":"INFO","message":"agent disconnected: websocket closed","host":"agent","target":"flakelet_relay::relay::agent_conn"}767agent: (finished: must succeed: systemctl restart flakelet-agent, in 0.94 seconds)768agent: must succeed: test -n "$(ls /var/lib/flakelet-agent/jobs)"769agent: (finished: must succeed: test -n "$(ls /var/lib/flakelet-agent/jobs)", in 0.01 seconds)770client: waiting for success: grep -q 'agent/app: updated' /tmp/restart.out771agent # [84937.350585] agent systemd[1]: flakelet-agent.service: Deactivated successfully.772agent # [84937.351020] agent systemd[1]: Stopped flakelet-agent.773agent # [84937.356729] agent systemd[1]: Starting app slow2...774agent # [84937.357574] agent systemd[1]: Starting flakelet-agent...775agent # [84937.417369] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:14:58.546589Z","level":"INFO","message":"reattaching to running unit","flakelet":"app","ids":"[\"e64e06db46a14587f0092f1df49c8540\"]","target":"flakelet_relay::agent::jobs"}776agent # [84937.417488] agent systemd[1]: Started flakelet-agent.777agent # [84937.481421] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:14:58.610671Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}778agent # [84937.482539] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:14:58.611832Z","level":"INFO","message":"start","id":"e64e06db46a14587f0092f1df49c8540","flakelet":"app","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}779relay # [84937.481465] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:14:58.610707Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(5), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}780agent # [84943.428892] agent systemd[1]: Started app slow2.781agent # [84943.449456] agent flakelet[2328]: app: updated to generation 6782agent # [84943.452980] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.783agent # [84944.007507] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:05.136763Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(6)","target":"flakelet_relay::agent::jobs"}784relay # [84944.007980] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:15:05.137251Z","level":"INFO","message":"deploy finished","job":"53596fefcdd8b6fa05d0f13f8544d3cd","ok":true,"target":"flakelet_relay::relay::api"}785client: (finished: waiting for success: grep -q 'agent/app: updated' /tmp/restart.out, in 7.07 seconds)786client: must succeed: cat /tmp/restart.out787client: (finished: must succeed: cat /tmp/restart.out, in 0.01 seconds)788agent: must succeed: systemctl is-active app.service789agent: (finished: must succeed: systemctl is-active app.service, in 0.01 seconds)790(finished: subtest: agent restart mid-run reattaches to the unit, in 9.08 seconds)791subtest: relay restart mid-run, push resumes via /v1/jobs792agent: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json793agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)794agent: must succeed: date +%s795agent: (finished: must succeed: date +%s, in 0.01 seconds)796client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/resume.out 2>&1 &797client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/resume.out 2>&1 &, in 0.01 seconds)798agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788675305 | grep -q 'app-slow'799agent # [84944.554402] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:05.683600Z","level":"INFO","message":"start","id":"02852836acd1733833227e5f8ef706e7","flakelet":"app","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}800agent # [84944.554925] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:05.684211Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"02852836acd1733833227e5f8ef706e7\"]","target":"flakelet_relay::agent::jobs"}801agent # [84944.594812] agent systemd[1]: Starting flakelet update app (via relay)...802agent # [84944.632284] agent systemd[1]: Started flakelet update app (via relay).803agent # [84944.637035] agent flakelet[2895]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow804agent # [84944.637797] agent flakelet[2895]: app: activating generation 7805agent # [84944.648907] agent systemd[1]: Stopping app slow2...806agent # [84944.649820] agent systemd[1]: app.service: Deactivated successfully.807agent # [84944.649997] agent systemd[1]: Stopped app slow2.808agent # [84944.663859] agent systemd[1]: Reload requested from client PID 2902 ('systemctl') (unit flakelet-relay-job-app.service)...809agent # [84944.663942] agent systemd[1]: Reloading...810relay # [84944.553936] relay flakelet-relay[400]: {"timestamp":"2026-09-06T06:15:05.683181Z","level":"INFO","message":"deploy accepted","job":"0f0aee819ff790b1bb0db833bd1e306c","caller":"x509:email:admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}811agent # [84945.020725] agent systemd[1]: Reloading finished in 355 ms.812agent # [84945.109626] agent systemd[1]: Reload requested from client PID 2932 ('systemctl') (unit flakelet-relay-job-app.service)...813agent # [84945.109673] agent systemd[1]: Reloading...814agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1788675305 | grep -q 'app-slow', in 1.02 seconds)815relay: must succeed: systemctl restart flakelet-relay816agent # [84945.468979] agent systemd[1]: Reloading finished in 358 ms.817agent # [84945.552344] agent systemd[1]: Reload requested from client PID 2961 ('systemctl') (unit flakelet-relay-job-app.service)...818agent # [84945.552395] agent systemd[1]: Reloading...819agent # [84945.576990] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:06.706233Z","level":"INFO","message":"disconnected: websocket closed","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}820relay # [84945.576400] relay systemd[1]: Stopping flakelet-relay...821relay # [84945.624666] relay systemd[1]: flakelet-relay.service: Deactivated successfully.822relay # [84945.632631] relay systemd[1]: Stopped flakelet-relay.823relay # [84945.632655] relay systemd[1]: flakelet-relay.service: Consumed 2.200s CPU time over 52.125s wall clock time.824relay # [84945.657449] relay systemd[1]: Starting flakelet-relay...825agent # [84945.954430] agent systemd[1]: Reloading finished in 401 ms.826agent # [84946.052259] agent systemd[1]: Reload requested from client PID 2995 ('systemctl') (unit flakelet-relay-job-app.service)...827agent # [84946.052300] agent systemd[1]: Reloading...828agent # [84946.562669] agent systemd[1]: Reloading finished in 509 ms.829agent # [84946.654158] agent systemd[1]: Starting app slow...830agent # [84946.700550] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:07.829790Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}831relay: (finished: must succeed: systemctl restart flakelet-relay, in 2.72 seconds)832client: waiting for success: grep -q '» ok' /tmp/resume.out833relay # [84948.280183] relay flakelet-relay[933]: {"timestamp":"2026-09-06T06:15:09.409436Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}834relay # [84948.280681] relay flakelet-relay[933]: {"timestamp":"2026-09-06T06:15:09.409892Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}835relay # [84948.280729] relay systemd[1]: Started flakelet-relay.836agent # [84948.757020] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:09.886250Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}837relay # [84948.757148] relay flakelet-relay[933]: {"timestamp":"2026-09-06T06:15:09.886298Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(6), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}838relay # [84952.590758] relay flakelet-relay[933]: {"timestamp":"2026-09-06T06:15:13.719994Z","level":"INFO","message":"job reattached","job":"0f0aee819ff790b1bb0db833bd1e306c","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}839agent # [84952.675408] agent systemd[1]: Started app slow.840agent # [84952.694168] agent flakelet[2895]: app: updated to generation 7841agent # [84952.697615] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.842agent # [84953.634674] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:14.763839Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(7)","target":"flakelet_relay::agent::jobs"}843client: (finished: waiting for success: grep -q '» ok' /tmp/resume.out, in 6.07 seconds)844client: must succeed: cat /tmp/resume.out845client: (finished: must succeed: cat /tmp/resume.out, in 0.01 seconds)846(finished: subtest: relay restart mid-run, push resumes via /v1/jobs, in 9.84 seconds)847subtest: job history comes from the agents and survives a relay restart848client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id hist1 agent/other 2>&1849agent # [84954.381988] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:15.511211Z","level":"INFO","message":"start","id":"5be20c693e5c0b60e3f85779794dfb86","flakelet":"other","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}850agent # [84954.382463] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:15.511750Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"5be20c693e5c0b60e3f85779794dfb86\"]","target":"flakelet_relay::agent::jobs"}851agent # [84954.430875] agent systemd[1]: Starting flakelet update other (via relay)...852agent # [84954.458722] agent systemd[1]: Started flakelet update other (via relay).853agent # [84954.463255] agent flakelet[3072]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1854agent # [84954.463982] agent flakelet[3072]: other: up to date855agent # [84954.466754] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.856relay # [84954.381552] relay flakelet-relay[933]: {"timestamp":"2026-09-06T06:15:15.510799Z","level":"INFO","message":"deploy accepted","job":"188b06c4636b65849b3ba8daac22dc7c","caller":"x509:email:admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}857client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id hist1 agent/other 2>&1, in 0.66 seconds)858relay: must succeed: systemctl restart flakelet-relay859agent # [84955.020911] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:16.150127Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}860agent # [84955.037104] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:16.166383Z","level":"INFO","message":"disconnected: websocket closed","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}861relay # [84955.021673] relay flakelet-relay[933]: {"timestamp":"2026-09-06T06:15:16.150901Z","level":"INFO","message":"deploy finished","job":"188b06c4636b65849b3ba8daac22dc7c","ok":true,"target":"flakelet_relay::relay::api"}862relay # [84955.036904] relay systemd[1]: Stopping flakelet-relay...863relay # [84955.069719] relay systemd[1]: flakelet-relay.service: Deactivated successfully.864relay # [84955.070109] relay systemd[1]: Stopped flakelet-relay.865relay # [84955.070132] relay systemd[1]: flakelet-relay.service: Consumed 2.164s CPU time over 9.412s wall clock time.866relay # [84955.135721] relay systemd[1]: Starting flakelet-relay...867agent # [84956.094191] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:17.223442Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}868relay: (finished: must succeed: systemctl restart flakelet-relay, in 2.94 seconds)869relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1'870relay # [84957.958651] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T06:15:19.087881Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}871relay # [84957.959459] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T06:15:19.088435Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}872relay # [84957.959184] relay systemd[1]: Started flakelet-relay.873relay # [84958.180064] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T06:15:19.309279Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(7), revision: Some(\"prebuilt:app\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}874agent # [84958.179953] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:19.309205Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}875relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1', in 1.03 seconds)876client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs877client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs, in 0.02 seconds)878client: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)879client: (finished: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.06 seconds)880client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTYyMCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njc1MzIwLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJZUldCQk5OR0NTVjNJPT09IiwibmJmIjoxNzg4Njc1MzIwLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.1WPwCx0pkkQRh-N9upJtcUvispfWc8Knqg1ngc43zzK4wekzXhzXWu_J9Klq-9Qv_K_7axmJlgjjESYnxhq_BA' push --relay https://relay:7443 jobs881client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4ODY3NTYyMCwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg4Njc1MzIwLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJZUldCQk5OR0NTVjNJPT09IiwibmJmIjoxNzg4Njc1MzIwLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.1WPwCx0pkkQRh-N9upJtcUvispfWc8Knqg1ngc43zzK4wekzXhzXWu_J9Klq-9Qv_K_7axmJlgjjESYnxhq_BA' push --relay https://relay:7443 jobs, in 0.03 seconds)882(finished: subtest: job history comes from the agents and survives a relay restart, in 4.74 seconds)883subtest: dashboard login via authorization code flow884client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code} %{redirect_url}' https://relay:7443/ui/ | grep -q '303 https://relay:7443/ui/login'885client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code} %{redirect_url}' https://relay:7443/ui/ | grep -q '303 https://relay:7443/ui/login', in 0.02 seconds)886client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -L https://relay:7443/ui/login887client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -L https://relay:7443/ui/login, in 0.09 seconds)888client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs889client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs, in 0.02 seconds)890client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -f https://relay:7443/v1/jobs | grep -q hist1891client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -f https://relay:7443/v1/jobs | grep -q hist1, in 0.02 seconds)892client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 10 https://relay:7443/ui/jobs/hist1/events || true893client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 10 https://relay:7443/ui/jobs/hist1/events || true, in 0.02 seconds)894client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -X POST 'https://relay:7443/ui/deploy?arg=other' | grep -q 403895client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -X POST 'https://relay:7443/ui/deploy?arg=other' | grep -q 403, in 0.02 seconds)896client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -D - -o /dev/null -H 'HX-Request: true' -X POST 'https://relay:7443/ui/deploy?arg=other'897client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -D - -o /dev/null -H 'HX-Request: true' -X POST 'https://relay:7443/ui/deploy?arg=other', in 0.02 seconds)898client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 30 https://relay:7443/ui/jobs/9498e306df5bbcb8c0b0c9da9cc7f56d/events || true899relay # [84959.155416] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[185]: 127.0.0.1 - - [06/Sep/2026 06:15:20] "GET /authorize?response_type=code&client_id=dashboard&redirect_uri=https%3A%2F%2Frelay%3A7443%2Fui%2Fcallback&scope=openid%20profile%20email%20groups&state=DrdzKNR9cunN46VJv6vtIZ_whQRBG8q9wLVQl-IsQqQ&code_challenge=JdaLK4xsp8jrTYE9u868GQaTQrhlYcWomlNTMoOOBTg&code_challenge_method=S256 HTTP/1.1" 302 -900relay # [84959.200069] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[185]: 127.0.0.1 - - [06/Sep/2026 06:15:20] "POST /token HTTP/1.1" 200 -901relay # [84959.201059] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T06:15:20.330330Z","level":"INFO","message":"login","name":"someone","principals":"[\"oidc:mock:someone\", \"oidc:mock:groups:wheel\"]","target":"flakelet_relay::relay::login"}902relay # [84959.267834] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T06:15:20.397102Z","level":"INFO","message":"job reattached","job":"188b06c4636b65849b3ba8daac22dc7c","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}903relay # [84959.306027] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T06:15:20.435275Z","level":"INFO","message":"deploy accepted","job":"d8d57b005fe61f82a76fbedd599cd92a","caller":"oidc:mock:someone\noidc:mock:groups:wheel","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}904relay # [84959.324092] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T06:15:20.453361Z","level":"INFO","message":"job reattached","job":"d8d57b005fe61f82a76fbedd599cd92a","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}905agent # [84959.306224] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:20.435494Z","level":"INFO","message":"start","id":"b95f14405d6a0e605f52bc8ffa3ccb1e","flakelet":"other","rule":"admins","relay":"relay","caller":"Some(\"oidc:mock:someone\\noidc:mock:groups:wheel\")","target":"flakelet_relay::agent::conn"}906agent # [84959.306739] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:20.435782Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"b95f14405d6a0e605f52bc8ffa3ccb1e\"]","target":"flakelet_relay::agent::jobs"}907agent # [84959.355564] agent systemd[1]: Starting flakelet update other (via relay)...908agent # [84959.363679] agent systemd[1]: Started flakelet update other (via relay).909agent # [84959.368249] agent flakelet[3110]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1910agent # [84959.368734] agent flakelet[3110]: other: up to date911agent # [84959.369328] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.912client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 30 https://relay:7443/ui/jobs/9498e306df5bbcb8c0b0c9da9cc7f56d/events || true, in 0.60 seconds)913client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs/9498e306df5bbcb8c0b0c9da9cc7f56d914client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs/9498e306df5bbcb8c0b0c9da9cc7f56d, in 0.02 seconds)915client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -sf -o /dev/null https://relay:7443/ui/static/htmx.min.js916client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -sf -o /dev/null https://relay:7443/ui/static/htmx.min.js, in 0.02 seconds)917client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/?q=status%3Ahealthy+oth'918client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/?q=status%3Ahealthy+oth', in 0.02 seconds)919client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/flakelets/other920client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/flakelets/other, in 0.02 seconds)921client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/hosts?q=status%3Adisconnected'922client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/hosts?q=status%3Adisconnected', in 0.02 seconds)923client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -H 'HX-Request: true' -X POST 'https://relay:7443/ui/retry?arg=9498e306df5bbcb8c0b0c9da9cc7f56d' | grep -q 400924client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -H 'HX-Request: true' -X POST 'https://relay:7443/ui/retry?arg=9498e306df5bbcb8c0b0c9da9cc7f56d' | grep -q 400, in 0.03 seconds)925client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 15 'https://relay:7443/ui/events?path=jobs' > /tmp/ev.out 2>&1 &926client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 15 'https://relay:7443/ui/events?path=jobs' > /tmp/ev.out 2>&1 &, in 0.01 seconds)927client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id live1 agent/other928» job 38796a7ba508eadfe82a1cd5def000bc via relay 0.1.0 (agents: agent 0.1.0)929» wave 0930agent/other | Starting flakelet update other (via relay)...931agent/other | Started flakelet update other (via relay).932agent/other | other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1933agent/other | other: up to date934agent # [84959.909385] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:21.038638Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}935agent # [84960.055384] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:21.184653Z","level":"INFO","message":"start","id":"5e317fc57863c2a7c03ebe1bdacd1356","flakelet":"other","rule":"admins","relay":"relay","caller":"Some(\"x509:email:admin@example.org\")","target":"flakelet_relay::agent::conn"}936agent # [84960.055657] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:21.184951Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"5e317fc57863c2a7c03ebe1bdacd1356\"]","target":"flakelet_relay::agent::jobs"}937agent # [84960.109752] agent systemd[1]: Starting flakelet update other (via relay)...938agent # [84960.119863] agent systemd[1]: Started flakelet update other (via relay).939agent # [84960.123936] agent flakelet[3128]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1940agent # [84960.124421] agent flakelet[3128]: other: up to date941agent # [84960.125895] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.942agent/other | flakelet-relay-job-other.service: Deactivated successfully.943relay # [84960.055138] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T06:15:21.184405Z","level":"INFO","message":"deploy accepted","job":"38796a7ba508eadfe82a1cd5def000bc","caller":"x509:email:admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}944» agent/other: unchanged (generation 1)945» ok946client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id live1 agent/other, in 0.64 seconds)947client: waiting for success: grep -q live1 /tmp/ev.out948client: (finished: waiting for success: grep -q live1 /tmp/ev.out, in 0.01 seconds)949client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -X POST -o /dev/null https://relay:7443/ui/logout950client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -X POST -o /dev/null https://relay:7443/ui/logout, in 0.02 seconds)951client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' https://relay:7443/ui/ | grep -q 303952client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' https://relay:7443/ui/ | grep -q 303, in 0.02 seconds)953(finished: subtest: dashboard login via authorization code flow, in 1.64 seconds)954(finished: run the VM test script, in 70.83 seconds)955agent # [84960.674665] agent flakelet-agent[2461]: {"timestamp":"2026-09-06T06:15:21.803911Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}956relay # [84960.675362] relay flakelet-relay[1323]: {"timestamp":"2026-09-06T06:15:21.804608Z","level":"INFO","message":"deploy finished","job":"38796a7ba508eadfe82a1cd5def000bc","ok":true,"target":"flakelet_relay::relay::api"}957test script finished in 71.17s958cleanup959kill NspawnMachine (pid 52)960kill NspawnMachine (pid 53)961Container agent terminated by signal KILL.962kill NspawnMachine (pid 54)963Container client terminated by signal KILL.964Container relay terminated by signal KILL.965(finished: cleanup, in 0.39 seconds)966additionally exposed symbols:967 agent, client, relay,968 vlan1,969 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh970» job 64f7f2b288e4b08effff548ddbe8c0a6 via relay 0.1.0 (agents: agent 0.1.0)971» wave 0972agent/app | Starting flakelet update app (via relay)...973agent/app | Started flakelet update app (via relay).974agent/app | app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2975agent/app | app: activating generation 2976agent/app | app: updated to generation 2977agent/app | flakelet-relay-job-app.service: Deactivated successfully.978» agent/app: updated (generation 2)979» ok980981» job d08097230363f39d28594bf67127750f via relay 0.1.0 (agents: agent 0.1.0)982» wave 0983agent/app | Starting flakelet update app (via relay)...984agent/app | Started flakelet update app (via relay).985agent/app | app: using prebuilt artifact /nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken986agent/app | app: activating generation 4987agent/app | app: deploy failed, rolled back: systemctl start app.service failed:988agent/app | failed units: app.service. See journalctl -u app.service989agent/app | flakelet-relay-job-app.service: Main process exited, code=exited, status=1/FAILURE990agent/app | flakelet-relay-job-app.service: Failed with result 'exit-code'.991» agent/app: rolled-back (generation 3)992agent/app │ Starting app v1...993agent/app │ Started app v1.994agent/app │ Stopping app v1...995agent/app │ app.service: Deactivated successfully.996agent/app │ Stopped app v1.997agent/app │ Starting app v2...998agent/app │ Started app v2.999agent/app │ Stopping app v2...1000agent/app │ app.service: Deactivated successfully.1001agent/app │ Stopped app v2.1002agent/app │ Starting app v1...1003agent/app │ Started app v1.1004agent/app │ Stopping app v1...1005agent/app │ app.service: Deactivated successfully.1006agent/app │ Stopped app v1.1007agent/app │ Starting app broken...1008agent/app │ app.service: Unable to locate executable '/nonexistent': No such file or directory1009agent/app │ app.service: Failed at step EXEC spawning /nonexistent: No such file or directory1010agent/app │ app.service: Main process exited, code=exited, status=203/EXEC1011agent/app │ app.service: Failed with result 'exit-code'.1012agent/app │ Failed to start app broken.1013agent/app │ Starting app v1...1014agent/app │ Started app v1.1015» skipped: agent/other1016» failed10171018» job 132127c9efc7e2f693d64052e27ca6b3 via relay 0.1.0 (agents: agent 0.1.0)1019» wave 01020agent/app | Starting flakelet update app (via relay)...1021agent/app | Started flakelet update app (via relay).1022agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow1023agent/app | app: activating generation 51024agent/app | app: updated to generation 51025agent/app | flakelet-relay-job-app.service: Deactivated successfully.1026» agent/app: updated (generation 5)1027» ok10281029» job 53596fefcdd8b6fa05d0f13f8544d3cd via relay 0.1.0 (agents: agent 0.1.0)1030» wave 01031agent/app | Starting flakelet update app (via relay)...1032agent/app | Started flakelet update app (via relay).1033agent/app | app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow21034agent/app | app: activating generation 61035agent/app | app: updated to generation 61036agent/app | flakelet-relay-job-app.service: Deactivated successfully.1037» agent/app: updated (generation 6)1038» ok10391040» job 0f0aee819ff790b1bb0db833bd1e306c via relay 0.1.0 (agents: agent 0.1.0)1041» wave 01042agent/app | Starting flakelet update app (via relay)...1043agent/app | Started flakelet update app (via relay).1044agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow1045agent/app | app: activating generation 71046flakelet-push: stream: error reading a body from connection, resuming job 7a143c97e450cf93712ea7389cbc4e9c1047flakelet-push: https://relay:7443: connect relay:7443: Connection refused (os error 111), retrying in 1s1048flakelet-push: https://relay:7443: connect relay:7443: Connection refused (os error 111), retrying in 2s1049flakelet-push: https://relay:7443: 404 Not Found: unknown_job: no connected agent knows this job, retrying in 4s1050» job 0f0aee819ff790b1bb0db833bd1e306c via relay 0.1.0 (agents: agent 0.1.0)1051» wave 01052agent/app | app: updated to generation 71053agent/app | flakelet-relay-job-app.service: Deactivated successfully.1054» agent/app: updated (generation 7)1055» ok105610571788675315 hist1 x509:email:admin@example.org agent/other:unchanged10581788675305 7a143c97e450cf93712ea7389cbc4e9c x509:email:admin@example.org agent/app:updated10591788675296 8cbce986e3f8a2eb98fc7555de1c7c67 x509:email:admin@example.org agent/app:updated10601788675287 aba313c3cc6a087b0c28bf793c16c354 x509:email:admin@example.org agent/app:updated10611788675286 5d94a1288b42acf74ee69475ef79e1cc x509:email:admin@example.org agent/other:unchanged10621788675273 d6af219a091a2ab4efb6fcddc299dec2 x509:email:admin@example.org agent/app:updated10631788675269 37c9644033c94bb6baf06912bf7a6207 x509:email:admin@example.org agent/app:rolled-back10641788675268 dda6ee7fce8a4a36b1cad364d3aeedbf oidc:mock:groups:wheel agent/other:unchanged10651788675268 f7e24fe8e29116f45effd5fadc310385 oidc:mock:groups:wheel agent/other:unchanged10661788675265 6c0a1cffc045c8e438f3c9a5b726c0c2 oidc:mock:groups:wheel agent/other:unchanged10671788675262 766cf7c5daa2a57fcceb46ef2124c398 oidc:mock:repo:github:example/app:ref:refs/heads/main agent/app:updated10681788675259 616c8f64918a382e4e2356eb161d7880 x509:email:admin@example.org agent/app:updated1069