container-test-run-flakelet-relay
checks.x86_64-linux.nixos-test
· build #38
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.01 seconds)5Initializing machine ID from random generator.6Test will time out and terminate in 3600.0 seconds7run the VM test script8start all VMs9client: systemd-nspawn running (pid 54)10relay: systemd-nspawn running (pid 53)11agent: systemd-nspawn running (pid 52)12client: Waiting for journal at /build/vm-state-client/var/log/journal...13relay: Waiting for journal at /build/vm-state-relay/var/log/journal...14agent: Waiting for journal at /build/vm-state-agent/var/log/journal...15(finished: start all VMs, in 0.01 seconds)16relay: waiting for unit flakelet-relay.service17nixos-nspawn(relay): TAP vde-tap1 not found; container will be isolated from VDE18nixos-nspawn(relay): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.19nixos-nspawn(client): TAP vde-tap1 not found; container will be isolated from VDE20nixos-nspawn(client): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.21nixos-nspawn(agent): TAP vde-tap1 not found; container will be isolated from VDE22nixos-nspawn(agent): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.23Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.24Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.25Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.26░ Spawning container agent on /build/vm-state-agent.27░ Spawning container relay on /build/vm-state-relay.28░ Spawning container client on /build/vm-state-client.29relay # No journal files were found.30relay # No journal boot entry found for the specified boot (+0).31client # No journal files were found.32client # No journal boot entry found for the specified boot (+0).33agent # No journal files were found.34agent # No journal boot entry found for the specified boot (+0).35relay # [ 1389.371134] relay systemd-journald[57]: Journal started36relay # [ 1389.371179] relay systemd-journald[57]: Runtime Journal (/run/log/journal/7a241eec43194eb9ad4c15024e98398d) is 8M, max 4G, 3.9G free.37client # [ 1389.370797] client systemd-journald[57]: Journal started38relay # [ 1389.379779] relay systemd[1]: Finished Apply Kernel Variables.39agent # [ 1389.373385] agent systemd-journald[57]: Journal started40relay # [ 1389.391956] relay systemd[1]: Finished Create Static Device Nodes in /dev gracefully.41agent # [ 1389.373430] agent systemd-journald[57]: Runtime Journal (/run/log/journal/d35762c222d248b0b1a4e5cdd59bce09) is 8M, max 4G, 3.9G free.42client # [ 1389.370886] client systemd-journald[57]: Runtime Journal (/run/log/journal/bdfbe268e326457bbc5316b557416a30) is 8M, max 4G, 3.9G free.43agent # [ 1389.383672] agent systemd[1]: Finished Apply Kernel Variables.44client # [ 1389.379737] client systemd[1]: Finished Apply Kernel Variables.45agent # [ 1389.395398] agent systemd[1]: Finished Create Static Device Nodes in /dev gracefully.46client # [ 1389.391688] client systemd[1]: Finished Create Static Device Nodes in /dev gracefully.47agent # [ 1389.429426] agent systemd[1]: Starting Flush Journal to Persistent Storage...48client # [ 1389.405878] client systemd[1]: Starting Flush Journal to Persistent Storage...49agent # [ 1389.430335] agent systemd[1]: Starting Create Static Device Nodes in /dev...50relay # [ 1389.406405] relay systemd[1]: Starting Flush Journal to Persistent Storage...51agent # [ 1389.439232] agent systemd-journald[57]: Time spent on flushing to /var/log/journal/d35762c222d248b0b1a4e5cdd59bce09 is 2.807ms for 6 entries.52relay # [ 1389.406906] relay systemd[1]: Starting Create Static Device Nodes in /dev...53agent # [ 1389.439232] agent systemd-journald[57]: System Journal (/var/log/journal/d35762c222d248b0b1a4e5cdd59bce09) is 512B, max 4G, 3.9G free.54client # [ 1389.406593] client systemd[1]: Starting Create Static Device Nodes in /dev...55agent # [ 1389.450604] agent systemd[1]: Finished Create Static Device Nodes in /dev.56relay # [ 1389.436541] relay systemd-journald[57]: Time spent on flushing to /var/log/journal/7a241eec43194eb9ad4c15024e98398d is 1.953ms for 6 entries.57client # [ 1389.436180] client systemd-journald[57]: Time spent on flushing to /var/log/journal/bdfbe268e326457bbc5316b557416a30 is 3.019ms for 6 entries.58client # [ 1389.436180] client systemd-journald[57]: System Journal (/var/log/journal/bdfbe268e326457bbc5316b557416a30) is 512B, max 4G, 3.9G free.59relay # [ 1389.436541] relay systemd-journald[57]: System Journal (/var/log/journal/7a241eec43194eb9ad4c15024e98398d) is 512B, max 4G, 3.9G free.60agent # [ 1389.451434] agent systemd[1]: Reached target Preparation for Local File Systems.61relay # [ 1389.445307] relay systemd[1]: Finished Create Static Device Nodes in /dev.62client # [ 1389.445246] client systemd[1]: Finished Create Static Device Nodes in /dev.63relay # [ 1389.446045] relay systemd[1]: Reached target Preparation for Local File Systems.64client # [ 1389.445988] client systemd[1]: Reached target Preparation for Local File Systems.65relay # [ 1389.446170] relay systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys66client # [ 1389.446150] client systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys67relay # [ 1389.459850] relay systemd[1]: Finished Flush Journal to Persistent Storage.68client # [ 1389.464801] client systemd[1]: Finished Flush Journal to Persistent Storage.69relay # [ 1389.588810] relay systemd[1]: Finished Firewall.70client # [ 1389.573955] client systemd[1]: Finished Firewall.71agent # [ 1389.451557] agent systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys72agent # [ 1389.469157] agent systemd[1]: Finished Flush Journal to Persistent Storage.73agent # [ 1389.586828] agent systemd[1]: Finished Firewall.74relay # [ 1390.365835] relay systemd[1]: Mounting /run/wrappers...75relay # [ 1390.424127] relay systemd[1]: Mounted /run/wrappers.76relay # [ 1390.425299] relay systemd[1]: Reached target Local File Systems.77relay # [ 1390.426635] relay systemd[1]: Listening on Boot Loader Control Service Socket.78relay # [ 1390.428244] relay systemd[1]: Starting Create SUID/SGID Wrappers...79relay # [ 1390.428294] relay systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container80relay # [ 1390.429691] relay systemd[1]: Starting Save Transient machine-id to Disk...81relay # [ 1390.430955] relay systemd[1]: Starting Create System Files and Directories...82relay # [ 1390.451493] relay systemd-tmpfiles[169]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted83relay # [ 1390.451977] relay systemd-tmpfiles[169]: fchmod() of /var/log/journal failed: Operation not permitted84relay # [ 1390.452321] relay systemd-tmpfiles[169]: fchmod() of /var/log/journal/7a241eec43194eb9ad4c15024e98398d failed: Operation not permitted85relay # [ 1390.452718] relay systemd-tmpfiles[169]: fchmod() of /run/log/journal failed: Operation not permitted86client # [ 1390.388819] client systemd[1]: Mounting /run/wrappers...87client # [ 1390.426462] client systemd[1]: Mounted /run/wrappers.88client # [ 1390.427584] client systemd[1]: Reached target Local File Systems.89client # [ 1390.428842] client systemd[1]: Listening on Boot Loader Control Service Socket.90client # [ 1390.430221] client systemd[1]: Starting Create SUID/SGID Wrappers...91client # [ 1390.430265] client systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container92client # [ 1390.432964] client systemd[1]: Starting Save Transient machine-id to Disk...93client # [ 1390.433959] client systemd[1]: Starting Create System Files and Directories...94client # [ 1390.453591] client systemd-tmpfiles[161]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted95client # [ 1390.454021] client systemd-tmpfiles[161]: fchmod() of /var/log/journal failed: Operation not permitted96client # [ 1390.454364] client systemd-tmpfiles[161]: fchmod() of /var/log/journal/bdfbe268e326457bbc5316b557416a30 failed: Operation not permitted97client # [ 1390.454717] client systemd-tmpfiles[161]: fchmod() of /run/log/journal failed: Operation not permitted98agent # [ 1390.388829] agent systemd[1]: Mounting /run/wrappers...99agent # [ 1390.427565] agent systemd[1]: Mounted /run/wrappers.100agent # [ 1390.428784] agent systemd[1]: Reached target Local File Systems.101agent # [ 1390.430205] agent systemd[1]: Listening on Boot Loader Control Service Socket.102agent # [ 1390.433050] agent systemd[1]: Starting Create SUID/SGID Wrappers...103agent # [ 1390.433103] agent systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container104agent # [ 1390.434378] agent systemd[1]: Starting Save Transient machine-id to Disk...105agent # [ 1390.435462] agent systemd[1]: Starting Create System Files and Directories...106agent # [ 1390.454641] agent systemd-tmpfiles[161]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted107agent # [ 1390.455084] agent systemd-tmpfiles[161]: fchmod() of /var/log/journal failed: Operation not permitted108agent # [ 1390.455451] agent systemd-tmpfiles[161]: fchmod() of /var/log/journal/d35762c222d248b0b1a4e5cdd59bce09 failed: Operation not permitted109agent # [ 1390.455821] agent systemd-tmpfiles[161]: fchmod() of /run/log/journal failed: Operation not permitted110relay # [ 1390.453283] relay suid-sgid-wrappers-start[176]: chmod: changing permissions of '/run/wrappers/wrappers.6zbHVG0lMx/chsh': Operation not permitted111relay # [ 1390.459608] relay systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE112relay # [ 1390.459722] relay systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.113relay # [ 1390.460044] relay systemd[1]: Failed to start Create SUID/SGID Wrappers.114relay # [ 1390.460513] relay systemd[1]: Finished Create System Files and Directories.115relay # [ 1390.479555] relay systemd[1]: Starting Rebuild Journal Catalog...116relay # [ 1390.480276] relay systemd[1]: Starting Record System Boot/Shutdown in UTMP...117relay # [ 1390.507561] relay systemd[1]: Finished Record System Boot/Shutdown in UTMP.118relay # [ 1390.521608] relay systemd[1]: Finished Rebuild Journal Catalog.119relay # [ 1390.523392] relay systemd[1]: Starting Update is Completed...120relay # [ 1390.536171] relay systemd[1]: Finished Update is Completed.121relay # [ 1390.558977] relay systemd[1]: etc-machine\x2did.mount: Deactivated successfully.122relay # [ 1390.559572] relay systemd[1]: Finished Save Transient machine-id to Disk.123relay # [ 1390.560306] relay systemd[1]: Reached target System Initialization.124relay # [ 1390.560397] relay systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container125relay # [ 1390.560470] relay systemd[1]: Started Daily Cleanup of Temporary Directories.126relay # [ 1390.560491] relay systemd[1]: Reached target Timer Units.127relay # [ 1390.560610] relay systemd[1]: Listening on D-Bus System Message Bus Socket.128relay # [ 1390.560801] relay systemd[1]: Listening on Nix Daemon Socket.129relay # [ 1390.560901] relay systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.130relay # [ 1390.560917] relay systemd[1]: Reached target Socket Units.131relay # [ 1390.560948] relay systemd[1]: Reached target Basic System.132relay # [ 1390.562600] relay systemd[1]: Started device-mock.service.133relay # [ 1390.563525] relay systemd[1]: Starting Re-link flakelet services at boot...134relay # [ 1390.564368] relay systemd[1]: Starting Import lastlog data into lastlog2 database...135relay # [ 1390.565474] relay systemd[1]: Starting Name Service Cache Daemon (nsncd)...136relay # [ 1390.566262] relay systemd[1]: Starting resolvconf update...137relay # [ 1390.580419] relay systemd[1]: Finished Re-link flakelet services at boot.138relay # [ 1390.581677] relay systemd[1]: Starting Reconcile flakelet services with the host configuration...139relay # [ 1390.585716] relay systemd[1]: Finished Import lastlog data into lastlog2 database.140relay # [ 1390.593456] relay systemd[1]: Finished Reconcile flakelet services with the host configuration.141client # [ 1390.456807] client suid-sgid-wrappers-start[168]: chmod: changing permissions of '/run/wrappers/wrappers.nluH3A4leo/chsh': Operation not permitted142client # [ 1390.468916] client systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE143client # [ 1390.468963] client systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.144client # [ 1390.469036] client systemd[1]: Failed to start Create SUID/SGID Wrappers.145agent # [ 1390.456878] agent suid-sgid-wrappers-start[168]: chmod: changing permissions of '/run/wrappers/wrappers.P3wsBigCW6/chsh': Operation not permitted146client # [ 1390.469724] client systemd[1]: Finished Create System Files and Directories.147agent # [ 1390.464859] agent systemd[1]: suid-sgid-wrappers.service: Main process exited, code=exited, status=1/FAILURE148client # [ 1390.535076] client systemd[1]: Starting Rebuild Journal Catalog...149agent # [ 1390.464909] agent systemd[1]: suid-sgid-wrappers.service: Failed with result 'exit-code'.150client # [ 1390.535709] client systemd[1]: Starting Record System Boot/Shutdown in UTMP...151agent # [ 1390.464969] agent systemd[1]: Failed to start Create SUID/SGID Wrappers.152client # [ 1390.548890] client systemd[1]: etc-machine\x2did.mount: Deactivated successfully.153agent # [ 1390.465547] agent systemd[1]: Finished Create System Files and Directories.154client # [ 1390.549586] client systemd[1]: Finished Save Transient machine-id to Disk.155agent # [ 1390.531756] agent systemd[1]: Starting Rebuild Journal Catalog...156client # [ 1390.554604] client systemd[1]: Finished Record System Boot/Shutdown in UTMP.157agent # [ 1390.532629] agent systemd[1]: Starting Record System Boot/Shutdown in UTMP...158client # [ 1390.555689] client systemd[1]: Finished Rebuild Journal Catalog.159agent # [ 1390.547481] agent systemd[1]: etc-machine\x2did.mount: Deactivated successfully.160client # [ 1390.556481] client systemd[1]: Starting Update is Completed...161agent # [ 1390.548502] agent systemd[1]: Finished Save Transient machine-id to Disk.162client # [ 1390.564859] client systemd[1]: Finished Update is Completed.163agent # [ 1390.551390] agent systemd[1]: Finished Record System Boot/Shutdown in UTMP.164client # [ 1390.565551] client systemd[1]: Reached target System Initialization.165agent # [ 1390.551573] agent systemd[1]: Finished Rebuild Journal Catalog.166client # [ 1390.565670] client systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container167agent # [ 1390.554014] agent systemd[1]: Starting Update is Completed...168client # [ 1390.565716] client systemd[1]: Started Daily Cleanup of Temporary Directories.169agent # [ 1390.563066] agent systemd[1]: Finished Update is Completed.170client # [ 1390.565738] client systemd[1]: Reached target Timer Units.171agent # [ 1390.563849] agent systemd[1]: Reached target System Initialization.172client # [ 1390.565889] client systemd[1]: Listening on D-Bus System Message Bus Socket.173agent # [ 1390.563970] agent systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container174client # [ 1390.566065] client systemd[1]: Listening on Nix Daemon Socket.175agent # [ 1390.564030] agent systemd[1]: Started Daily Cleanup of Temporary Directories.176client # [ 1390.566182] client systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.177agent # [ 1390.564049] agent systemd[1]: Reached target Timer Units.178client # [ 1390.566198] client systemd[1]: Reached target Socket Units.179agent # [ 1390.564178] agent systemd[1]: Listening on D-Bus System Message Bus Socket.180client # [ 1390.566232] client systemd[1]: Reached target Basic System.181agent # [ 1390.564377] agent systemd[1]: Listening on Nix Daemon Socket.182agent # [ 1390.564493] agent systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.183client # [ 1390.567399] client systemd[1]: Starting Import lastlog data into lastlog2 database...184agent # [ 1390.564507] agent systemd[1]: Reached target Socket Units.185client # [ 1390.568507] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...186agent # [ 1390.564536] agent systemd[1]: Reached target Basic System.187client # [ 1390.569097] client systemd[1]: Starting resolvconf update...188agent # [ 1390.565951] agent systemd[1]: Starting Re-link flakelet services at boot...189client # [ 1390.588753] client systemd[1]: Finished Import lastlog data into lastlog2 database.190agent # [ 1390.566590] agent systemd[1]: Starting Import lastlog data into lastlog2 database...191agent # [ 1390.567322] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...192agent # [ 1390.568187] agent systemd[1]: Starting resolvconf update...193agent # [ 1390.580417] agent systemd[1]: Finished Re-link flakelet services at boot.194agent # [ 1390.581613] agent systemd[1]: Starting Reconcile flakelet services with the host configuration...195agent # [ 1390.586524] agent systemd[1]: Finished Import lastlog data into lastlog2 database.196agent # [ 1390.592395] agent systemd[1]: Finished Reconcile flakelet services with the host configuration.197agent # [ 1390.652281] agent systemd[1]: Finished resolvconf update.198agent # [ 1390.652433] agent systemd[1]: Reached target Preparation for Network.199agent # [ 1390.654392] agent systemd[1]: Starting Address configuration of eth1...200agent # [ 1390.655892] agent systemd[1]: Starting Extra networking commands....201agent # [ 1390.678322] agent systemd[1]: nscd.service: Deactivated successfully.202agent # [ 1390.678417] agent systemd[1]: Stopped Name Service Cache Daemon (nsncd).203agent # [ 1390.682280] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...204agent # [ 1390.694704] agent network-addresses-eth1-start[237]: adding address 192.168.1.1/24... done205agent # [ 1390.697996] agent network-addresses-eth1-start[237]: adding address 2001:db8:1::1/64... done206agent # [ 1390.703713] agent systemd[1]: Finished Address configuration of eth1.207agent # [ 1390.748775] agent systemd[1]: Finished Extra networking commands..208agent # [ 1390.749495] agent systemd[1]: Reached target Network.209agent # [ 1390.750843] agent systemd[1]: Starting Update flakelet service app...210agent # [ 1390.751797] agent systemd[1]: Starting Update flakelet service flakelet-agent...211agent # [ 1390.752712] agent systemd[1]: Starting Update flakelet service other...212agent # [ 1390.763552] agent flakelet[304]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1213agent # [ 1390.763989] agent flakelet[305]: flakelet-agent: using prebuilt artifact /nix/store/m3qlnnpailq8ia2iyfmnfxd81q1l3xqy-flakelet-flakelet-agent214agent # [ 1390.764686] agent flakelet[304]: app: activating generation 1215agent # [ 1390.764734] agent flakelet[305]: flakelet-agent: activating generation 1216agent # [ 1390.765796] agent flakelet[306]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1217agent # [ 1390.766632] agent flakelet[306]: other: activating generation 1218agent # [ 1390.775483] agent systemd[1]: Reload requested from client PID 307 ('systemctl') (unit flakelet-app.service)...219agent # [ 1390.775571] agent systemd[1]: Reloading...220client # [ 1390.651485] client systemd[1]: Finished resolvconf update.221client # [ 1390.652220] client systemd[1]: Reached target Preparation for Network.222client # [ 1390.653343] client systemd[1]: Starting Address configuration of eth1...223client # [ 1390.654527] client systemd[1]: Starting Extra networking commands....224client # [ 1390.663931] client systemd[1]: nscd.service: Deactivated successfully.225client # [ 1390.678285] client systemd[1]: Stopped Name Service Cache Daemon (nsncd).226client # [ 1390.681281] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...227client # [ 1390.695699] client network-addresses-eth1-start[233]: adding address 192.168.1.2/24... done228client # [ 1390.698767] client network-addresses-eth1-start[233]: adding address 2001:db8:1::2/64... done229client # [ 1390.703791] client systemd[1]: Finished Address configuration of eth1.230client # [ 1390.748788] client systemd[1]: Finished Extra networking commands..231client # [ 1390.748951] client systemd[1]: Reached target Network.232client # [ 1390.778605] client systemd[1]: nscd.service: Deactivated successfully.233client # [ 1390.778799] client systemd[1]: Stopped Name Service Cache Daemon (nsncd).234client # [ 1390.781522] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...235relay # [ 1390.651929] relay systemd[1]: Finished resolvconf update.236relay # [ 1390.652049] relay systemd[1]: Reached target Preparation for Network.237relay # [ 1390.653607] relay systemd[1]: Starting Address configuration of eth1...238relay # [ 1390.654611] relay systemd[1]: Starting Extra networking commands....239relay # [ 1390.678278] relay systemd[1]: nscd.service: Deactivated successfully.240relay # [ 1390.678334] relay systemd[1]: Stopped Name Service Cache Daemon (nsncd).241relay # [ 1390.681280] relay systemd[1]: Starting Name Service Cache Daemon (nsncd)...242relay # [ 1390.696184] relay network-addresses-eth1-start[246]: adding address 192.168.1.3/24... done243relay # [ 1390.699306] relay network-addresses-eth1-start[246]: adding address 2001:db8:1::3/64... done244relay # [ 1390.704506] relay systemd[1]: Finished Address configuration of eth1.245relay # [ 1390.745381] relay systemd[1]: Finished Extra networking commands..246relay # [ 1390.745677] relay systemd[1]: Reached target Network.247relay # [ 1390.747921] relay systemd[1]: Starting Dnsmasq Daemon...248relay # [ 1390.749698] relay systemd[1]: Starting Nginx Web Server...249relay # [ 1390.751117] relay systemd[1]: Starting Update flakelet service flakelet-relay...250relay # [ 1390.764993] relay flakelet[311]: flakelet-relay: using prebuilt artifact /nix/store/y9h8ggd4mavd2sybyw1jzksdqy7z09bf-flakelet-flakelet-relay251relay # [ 1390.766126] relay flakelet[311]: flakelet-relay: activating generation 1252relay # [ 1390.770556] relay systemd[1]: Reload requested from client PID 316 ('systemctl') (unit flakelet-flakelet-relay.service)...253relay # [ 1390.770619] relay systemd[1]: Reloading...254relay # [ 1391.091489] relay nsncd[251]: Sep 10 11:57:52.586 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"255relay # [ 1391.201831] relay systemd[1]: Reloading finished in 430 ms.256relay # [ 1391.298212] relay systemd[1]: Started Name Service Cache Daemon (nsncd).257relay # [ 1391.298752] relay systemd[1]: Reached target Host and Network Name Lookups.258relay # [ 1391.298844] relay systemd[1]: Reached target User and Group Name Lookups.259relay # [ 1391.300309] relay systemd[1]: Starting User Login Management...260relay # [ 1391.301129] relay systemd[1]: Starting Permit User Sessions...261relay # [ 1391.319796] relay systemd[1]: Reload requested from client PID 357 ('systemctl') (unit flakelet-flakelet-relay.service)...262relay # [ 1391.319882] relay systemd[1]: Reloading...263client # [ 1391.086665] client nsncd[303]: Sep 10 11:57:52.581 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"264client # [ 1391.086883] client systemd[1]: Started Name Service Cache Daemon (nsncd).265client # [ 1391.086989] client systemd[1]: Reached target Host and Network Name Lookups.266client # [ 1391.087108] client systemd[1]: Reached target User and Group Name Lookups.267client # [ 1391.089597] client systemd[1]: Starting User Login Management...268client # [ 1391.091027] client systemd[1]: Starting Permit User Sessions...269client # [ 1391.102342] client systemd[1]: Finished Permit User Sessions.270client # [ 1391.105370] client systemd[1]: Started Console Getty.271client # [ 1391.105428] client systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0272client # [ 1391.105460] client systemd[1]: Reached target Login Prompts.273agent # [ 1391.189450] agent systemd[1]: Reloading finished in 412 ms.274agent # [ 1391.301184] agent systemd[1]: nscd.service: Deactivated successfully.275agent # [ 1391.301339] agent systemd[1]: Stopped Name Service Cache Daemon (nsncd).276agent # [ 1391.304183] agent systemd[1]: Reload requested from client PID 308 ('systemctl') (unit flakelet-flakelet-agent.service)...277agent # [ 1391.304226] agent systemd[1]: Reloading...278relay # [ 1391.488030] relay dnsmasq-pre-start[393]: dnsmasq: syntax check OK.279relay # [ 1391.699792] relay systemd[1]: Reloading finished in 379 ms.280relay # [ 1391.765197] relay systemd[1]: Finished Permit User Sessions.281relay # [ 1391.768234] relay systemd[1]: Started Console Getty.282relay # [ 1391.768310] relay systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0283relay # [ 1391.768403] relay systemd[1]: Reached target Login Prompts.284relay # [ 1391.789712] relay systemd[1]: Starting flakelet-relay...285agent # [ 1391.686947] agent systemd[1]: Reloading finished in 382 ms.286agent # [ 1391.767183] agent systemd[1]: Reload requested from client PID 309 ('systemctl') (unit flakelet-other.service)...287agent # [ 1391.767237] agent systemd[1]: Reloading...288agent # [ 1392.132327] agent systemd[1]: Reloading finished in 364 ms.289agent # [ 1392.191599] agent systemd[1]: Starting Name Service Cache Daemon (nsncd)...290agent # [ 1392.198179] agent systemd[1]: Reload requested from client PID 393 ('systemctl') (unit flakelet-app.service)...291agent # [ 1392.198225] agent systemd[1]: Reloading...292relay # [ 1392.401861] relay systemd[1]: Starting D-Bus System Message Bus...293relay # [ 1392.487095] relay nginx-pre-start[403]: nginx: the configuration file /nix/store/01avjj26wcihp926xgasszcj5s0k7cbs-nginx.conf syntax is ok294relay # [ 1392.487774] relay nginx-pre-start[403]: nginx: configuration file /nix/store/01avjj26wcihp926xgasszcj5s0k7cbs-nginx.conf test is successful295relay # [ 1392.501929] relay systemd[1]: Started Nginx Web Server.296agent # [ 1392.479909] agent nsncd[392]: Sep 10 11:57:53.974 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"297agent # [ 1392.578674] agent systemd[1]: Reloading finished in 380 ms.298agent # [ 1392.644367] agent systemd[1]: Started Name Service Cache Daemon (nsncd).299agent # [ 1392.646984] agent systemd[1]: Reached target Host and Network Name Lookups.300agent # [ 1392.647156] agent systemd[1]: Reached target User and Group Name Lookups.301agent # [ 1392.648675] agent systemd[1]: Starting User Login Management...302agent # [ 1392.649865] agent systemd[1]: Starting Permit User Sessions...303agent # [ 1392.651466] agent systemd[1]: Reload requested from client PID 394 ('systemctl') (unit flakelet-flakelet-agent.service)...304agent # [ 1392.651494] agent systemd[1]: Reloading...305client # [ 1392.581677] client systemd-logind[315]: New seat seat0.306client # [ 1392.583494] client systemd[1]: Starting D-Bus System Message Bus...307client # [ 1392.583616] client systemd[1]: Started User Login Management.308client # [ 1392.585079] client systemd[1]: Starting linger-users.service...309client # [ 1392.614409] client systemd[1]: linger-users.service: Deactivated successfully.310client # [ 1392.614819] client systemd[1]: Finished linger-users.service.311client # [ 1392.616176] client systemd[1]: Reached target Multi-User System.312relay # [ 1392.901987] relay systemd-logind[354]: New seat seat0.313relay # [ 1392.902351] relay systemd[1]: Started User Login Management.314relay # [ 1392.905053] relay systemd[1]: Starting linger-users.service...315relay # [ 1392.918314] relay systemd[1]: linger-users.service: Deactivated successfully.316relay # [ 1392.918710] relay systemd[1]: Finished linger-users.service.317relay # [ 1392.997147] relay dbus-broker-launch[402]: Looking up NSS user entry for 'systemd-timesync'...318relay # [ 1392.998419] relay dbus-broker-launch[402]: NSS returned no entry for 'systemd-timesync'319relay # [ 1392.998419] relay dbus-broker-launch[402]: Invalid user-name in /nix/store/jc4z11b21x1n6xiiply0x9mfc84afs4d-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"320relay # [ 1393.002234] relay systemd[1]: Started D-Bus System Message Bus.321relay # [ 1393.011180] relay dbus-broker-launch[402]: Ready322relay # [ 1393.015326] relay systemd[1]: Started Dnsmasq Daemon.323relay # [ 1393.017817] relay dnsmasq[394]: started, version 2.93 cachesize 150324relay # [ 1393.017841] relay dnsmasq[394]: compile time options: IPv6 GNU-getopt DBus no-UBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset nftset auth DNSSEC loop-detect inotify dumpfile325relay # [ 1393.017854] relay dnsmasq[394]: DBus support enabled: connected to system bus326relay # [ 1393.017886] relay dnsmasq[394]: no upstream servers configured - please set them from DBus327relay # [ 1393.018251] relay dnsmasq[394]: read /etc/hosts - 9 names328agent # [ 1393.012244] agent systemd[1]: Reloading finished in 360 ms.329agent # [ 1393.073809] agent systemd[1]: Finished Permit User Sessions.330agent # [ 1393.074506] agent systemd[1]: Reload requested from client PID 395 ('systemctl') (unit flakelet-other.service)...331agent # [ 1393.074554] agent systemd[1]: Reloading...332client # [ 1393.095448] client dbus-broker-launch[325]: Looking up NSS user entry for 'systemd-timesync'...333client # [ 1393.096778] client dbus-broker-launch[325]: NSS returned no entry for 'systemd-timesync'334client # [ 1393.096778] client dbus-broker-launch[325]: Invalid user-name in /nix/store/692g0slrqzpwy8pwhsv3byjhwldybkla-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"335client # [ 1393.097766] client systemd[1]: Started D-Bus System Message Bus.336client # [ 1393.098326] client systemd[1]: Startup finished in 4.181s.337client # [ 1393.104756] client dbus-broker-launch[325]: Ready338agent # [ 1393.426048] agent systemd[1]: Reloading finished in 351 ms.339agent # [ 1393.501591] agent systemd[1]: Started Console Getty.340agent # [ 1393.501722] agent systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0341agent # [ 1393.501886] agent systemd[1]: Reached target Login Prompts.342agent # [ 1393.505877] agent systemd[1]: Starting app v1...343agent # [ 1393.506849] agent systemd[1]: Starting flakelet-agent...344agent # [ 1393.510570] agent systemd[1]: Starting other v1...345agent # [ 1393.542079] agent systemd[1]: Started app v1.346agent # [ 1393.542296] agent systemd[1]: Started other v1.347agent # [ 1393.547337] agent flakelet[304]: app: updated to generation 1348agent # [ 1393.548037] agent systemd[1]: Finished Update flakelet service app.349agent # [ 1393.550304] agent flakelet[306]: other: updated to generation 1350agent # [ 1393.551961] agent systemd[1]: Finished Update flakelet service other.351agent # [ 1393.574441] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:57:55.069118Z","level":"INFO","message":"local change","flakelet":"app","generation":1,"by":"host activation","target":"flakelet_relay::agent::jobs"}352agent # [ 1393.575107] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:57:55.069413Z","level":"INFO","message":"local change","flakelet":"other","generation":1,"by":"host activation","target":"flakelet_relay::agent::jobs"}353agent # [ 1393.575079] agent systemd[1]: Started flakelet-agent.354agent # [ 1393.581896] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:57:55.076629Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}355agent # [ 1393.582876] agent flakelet[305]: flakelet-agent: updated to generation 1356agent # [ 1393.585008] agent systemd[1]: Finished Update flakelet service flakelet-agent.357agent # [ 1393.586191] agent systemd[1]: Reached target flakelet managed services.358agent # [ 1393.596407] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:57:55.091152Z","level":"INFO","message":"local change","flakelet":"flakelet-agent","generation":1,"by":"host activation","target":"flakelet_relay::agent::jobs"}359relay # [ 1393.845151] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:57:55.339553Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}360relay # [ 1393.846087] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:57:55.340098Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}361relay # [ 1393.845586] relay systemd[1]: Started flakelet-relay.362relay # [ 1393.857305] relay systemd[1]: Starting health for flakelet-relay...363agent # [ 1393.933597] agent systemd-logind[433]: New seat seat0.364agent # [ 1393.933835] agent systemd[1]: Started User Login Management.365agent # [ 1393.935335] agent systemd[1]: Starting D-Bus System Message Bus...366agent # [ 1393.936298] agent systemd[1]: Starting linger-users.service...367agent # [ 1393.962185] agent systemd[1]: linger-users.service: Deactivated successfully.368agent # [ 1393.962602] agent systemd[1]: Finished linger-users.service.369agent # [ 1393.964038] agent systemd[1]: Reached target Multi-User System.370relay: (finished: waiting for unit flakelet-relay.service, in 6.17 seconds)371relay: waiting for unit nginx.service372relay: (finished: waiting for unit nginx.service, in 0.02 seconds)373agent: waiting for unit flakelet.target374agent: (finished: waiting for unit flakelet.target, in 0.01 seconds)375agent: waiting for unit flakelet-agent.service376agent: (finished: waiting for unit flakelet-agent.service, in 0.02 seconds)377client: waiting for unit multi-user.target378client: (finished: waiting for unit multi-user.target, in 0.02 seconds)379relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1'380relay # [ 1394.151339] relay systemd[1]: flakelet-relay-health.service: Deactivated successfully.381relay # [ 1394.154326] relay systemd[1]: Finished health for flakelet-relay.382relay # [ 1394.163365] relay flakelet[311]: flakelet-relay: updated to generation 1383relay # [ 1394.167269] relay systemd[1]: Finished Update flakelet service flakelet-relay.384relay # [ 1394.167891] relay systemd[1]: Reached target flakelet managed services.385relay # [ 1394.168084] relay systemd[1]: Reached target Multi-User System.386relay # [ 1394.168359] relay systemd[1]: Startup finished in 5.251s.387agent # [ 1394.344019] agent dbus-broker-launch[902]: Looking up NSS user entry for 'systemd-timesync'...388agent # [ 1394.345562] agent dbus-broker-launch[902]: NSS returned no entry for 'systemd-timesync'389agent # [ 1394.345562] agent dbus-broker-launch[902]: Invalid user-name in /nix/store/9jm4z4kmif1yf3786ghlinqlwra7q4q4-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"390agent # [ 1394.346811] agent systemd[1]: Started D-Bus System Message Bus.391agent # [ 1394.347446] agent systemd[1]: Startup finished in 5.430s.392agent # [ 1394.352445] agent dbus-broker-launch[902]: Ready393relay # [ 1395.602444] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:57:57.097151Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(1), revision: Some(\"prebuilt:app\") }, Named { name: \"flakelet-agent\", generation: Some(1), revision: Some(\"prebuilt:flakelet-agent\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}394agent # [ 1395.602113] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:57:57.096698Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}395relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1', in 2.05 seconds)396agent: must succeed: systemctl is-active app.service other.service397agent: (finished: must succeed: systemctl is-active app.service other.service, in 0.01 seconds)398subtest: deploy with client certificate, relay found via SRV399agent: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json400agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.02 seconds)401client: must succeed: push --relay-srv test --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app 2>&1402relay # [ 1396.676220] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:57:58.170911Z","level":"INFO","message":"deploy accepted","job":"4dee1fef3671457119b29328889c5ff9","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}403agent # [ 1396.677137] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:57:58.171737Z","level":"INFO","message":"start","id":"094fc03cd0ef4611ee8599b75ddf1072","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}404agent # [ 1396.678130] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:57:58.172662Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"094fc03cd0ef4611ee8599b75ddf1072\"]","target":"flakelet_relay::agent::jobs"}405agent # [ 1396.720768] agent systemd[1]: Starting flakelet update app (via relay)...406agent # [ 1396.745387] agent systemd[1]: Started flakelet update app (via relay).407agent # [ 1396.750635] agent flakelet[924]: app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2408agent # [ 1396.751386] agent flakelet[924]: app: activating generation 2409agent # [ 1396.770184] agent systemd[1]: Stopping app v1...410agent # [ 1396.770710] agent systemd[1]: app.service: Deactivated successfully.411agent # [ 1396.771073] agent systemd[1]: Stopped app v1.412agent # [ 1396.790800] agent systemd[1]: Reload requested from client PID 931 ('systemctl') (unit flakelet-relay-job-app.service)...413agent # [ 1396.790887] agent systemd[1]: Reloading...414agent # [ 1397.149314] agent systemd[1]: Reloading finished in 357 ms.415agent # [ 1397.238352] agent systemd[1]: Reload requested from client PID 961 ('systemctl') (unit flakelet-relay-job-app.service)...416agent # [ 1397.238402] agent systemd[1]: Reloading...417agent # [ 1397.588895] agent systemd[1]: Reloading finished in 350 ms.418agent # [ 1397.670603] agent systemd[1]: Reload requested from client PID 992 ('systemctl') (unit flakelet-relay-job-app.service)...419agent # [ 1397.670655] agent systemd[1]: Reloading...420agent # [ 1398.021115] agent systemd[1]: Reloading finished in 349 ms.421agent # [ 1398.089279] agent systemd[1]: Reload requested from client PID 1025 ('systemctl') (unit flakelet-relay-job-app.service)...422agent # [ 1398.089333] agent systemd[1]: Reloading...423agent # [ 1398.432957] agent systemd[1]: Reloading finished in 343 ms.424agent # [ 1398.485585] agent systemd[1]: Starting app v2...425agent # [ 1398.504298] agent systemd[1]: Started app v2.426agent # [ 1398.511018] agent flakelet[924]: app: updated to generation 2427agent # [ 1398.513657] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.428client: (finished: must succeed: push --relay-srv test --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app 2>&1, in 2.97 seconds)429agent: must succeed: systemctl show app.service -p Description | grep -q 'app v2'430agent: (finished: must succeed: systemctl show app.service -p Description | grep -q 'app v2', in 0.02 seconds)431agent: must succeed: journalctl -u flakelet-relay-job-app.service | grep -q 'generation 2'432agent: (finished: must succeed: journalctl -u flakelet-relay-job-app.service | grep -q 'generation 2', in 0.01 seconds)433agent: must succeed: flakelet status --json app | jq -c '.[0].changed.by'434agent: (finished: must succeed: flakelet status --json app | jq -c '.[0].changed.by', in 0.03 seconds)435(finished: subtest: deploy with client certificate, relay found via SRV, in 3.06 seconds)436subtest: deploy with OIDC token, sub principal437agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json438agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)439client: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)440client: (finished: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.14 seconds)441client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTc4MSwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNDgxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJXTkhMRDZXRk9ERDdLPT09IiwibmJmIjoxNzg5MDQxNDgxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.VtehAOQR44OQsOZtzDrjBvWHyWr7MfDH6mQB_5S9HhFl6cF6YfCKOKLfL2i9yibwNIlGZYAo-23538_tPF-RDA' push --relay https://relay:7443 deploy agent/app 2>&1442relay # [ 1399.621772] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:01.116429Z","level":"INFO","message":"deploy finished","job":"4dee1fef3671457119b29328889c5ff9","ok":true,"target":"flakelet_relay::relay::api"}443agent # [ 1399.620193] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:01.114782Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(2)","target":"flakelet_relay::agent::jobs"}444agent # [ 1399.882329] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:01.376910Z","level":"INFO","message":"start","id":"467be3c3c9b72efa4fea1778a3d13f7b","flakelet":"app","rule":"ci","relay":"relay","caller_name":"\"repo:github:example/app:ref:refs/heads/main\"","target":"flakelet_relay::agent::conn"}445agent # [ 1399.882742] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:01.377460Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"467be3c3c9b72efa4fea1778a3d13f7b\"]","target":"flakelet_relay::agent::jobs"}446agent # [ 1399.921275] agent systemd[1]: Starting flakelet update app (via relay)...447agent # [ 1399.949278] agent systemd[1]: Started flakelet update app (via relay).448agent # [ 1399.954447] agent flakelet[1094]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1449agent # [ 1399.955194] agent flakelet[1094]: app: activating generation 3450agent # [ 1399.970052] agent systemd[1]: Stopping app v2...451agent # [ 1399.970578] agent systemd[1]: app.service: Deactivated successfully.452agent # [ 1399.970912] agent systemd[1]: Stopped app v2.453agent # [ 1399.991201] agent systemd[1]: Reload requested from client PID 1101 ('systemctl') (unit flakelet-relay-job-app.service)...454agent # [ 1399.991283] agent systemd[1]: Reloading...455relay # [ 1399.881775] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:01.376515Z","level":"INFO","message":"deploy accepted","job":"aff01a078adf4aaca6a92a3203d034df","caller":"repo:github:example/app:ref:refs/heads/main","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}456agent # [ 1400.347248] agent systemd[1]: Reloading finished in 355 ms.457agent # [ 1400.421722] agent systemd[1]: Reload requested from client PID 1131 ('systemctl') (unit flakelet-relay-job-app.service)...458agent # [ 1400.421773] agent systemd[1]: Reloading...459agent # [ 1400.784363] agent systemd[1]: Reloading finished in 362 ms.460agent # [ 1400.843806] agent systemd[1]: Reload requested from client PID 1160 ('systemctl') (unit flakelet-relay-job-app.service)...461agent # [ 1400.843891] agent systemd[1]: Reloading...462agent # [ 1401.196947] agent systemd[1]: Reloading finished in 352 ms.463agent # [ 1401.242325] agent systemd[1]: Reload requested from client PID 1191 ('systemctl') (unit flakelet-relay-job-app.service)...464agent # [ 1401.242373] agent systemd[1]: Reloading...465agent # [ 1401.591245] agent systemd[1]: Reloading finished in 348 ms.466agent # [ 1401.655061] agent systemd[1]: Starting app v1...467agent # [ 1401.672395] agent systemd[1]: Started app v1.468agent # [ 1401.678379] agent flakelet[1094]: app: updated to generation 3469agent # [ 1401.681144] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.470client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTc4MSwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNDgxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJXTkhMRDZXRk9ERDdLPT09IiwibmJmIjoxNzg5MDQxNDgxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.VtehAOQR44OQsOZtzDrjBvWHyWr7MfDH6mQB_5S9HhFl6cF6YfCKOKLfL2i9yibwNIlGZYAo-23538_tPF-RDA' push --relay https://relay:7443 deploy agent/app 2>&1, in 2.94 seconds)471(finished: subtest: deploy with OIDC token, sub principal, in 3.09 seconds)472subtest: policy denies uncovered target473client: must fail: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTc4MSwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNDgxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJXTkhMRDZXRk9ERDdLPT09IiwibmJmIjoxNzg5MDQxNDgxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.VtehAOQR44OQsOZtzDrjBvWHyWr7MfDH6mQB_5S9HhFl6cF6YfCKOKLfL2i9yibwNIlGZYAo-23538_tPF-RDA' push --relay https://relay:7443 deploy agent/other 2>&1474client: (finished: must fail: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTc4MSwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNDgxLCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJXTkhMRDZXRk9ERDdLPT09IiwibmJmIjoxNzg5MDQxNDgxLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.VtehAOQR44OQsOZtzDrjBvWHyWr7MfDH6mQB_5S9HhFl6cF6YfCKOKLfL2i9yibwNIlGZYAo-23538_tPF-RDA' push --relay https://relay:7443 deploy agent/other 2>&1, in 0.02 seconds)475(finished: subtest: policy denies uncovered target, in 0.02 seconds)476subtest: claim principal from groups, unchanged counts as success477client: must succeed: echo '{"groups": ["wheel"]}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'someone' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)478client: (finished: must succeed: echo '{"groups": ["wheel"]}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'someone' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.05 seconds)479client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTc4NCwiZ3JvdXBzIjpbIndoZWVsIl0sImlhdCI6MTc4OTA0MTQ4NCwiaXNzIjoiaHR0cHM6Ly9yZWxheTo4NDQzIiwianRpIjoiNk1UN0VaNjNFRjdVND09PSIsIm5iZiI6MTc4OTA0MTQ4NCwic3ViIjoic29tZW9uZSJ9.lHoYeHEm9bYJsNFUP4zgg93HYftWfLkAxA5VRmSRAtYrOwnQQkiMmfeZigS89y7nMyG2rT1gBPMJJMFoEyxRDg' push --relay https://relay:7443 deploy agent/other 2>&1480relay # [ 1402.781326] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:04.276020Z","level":"INFO","message":"deploy finished","job":"aff01a078adf4aaca6a92a3203d034df","ok":true,"target":"flakelet_relay::relay::api"}481relay # [ 1402.883560] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:04.378249Z","level":"INFO","message":"deploy accepted","job":"64ce5a9610b5f3fb978d0b6e6326babd","caller":"someone","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}482agent # [ 1402.779562] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:04.274243Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}483agent # [ 1402.884028] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:04.378698Z","level":"INFO","message":"start","id":"381ce55ae5036272a704ee692f016fe8","flakelet":"other","rule":"admins","relay":"relay","caller_name":"\"someone\"","target":"flakelet_relay::agent::conn"}484agent # [ 1402.884486] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:04.379173Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"381ce55ae5036272a704ee692f016fe8\"]","target":"flakelet_relay::agent::jobs"}485agent # [ 1402.927078] agent systemd[1]: Starting flakelet update other (via relay)...486agent # [ 1402.954636] agent systemd[1]: Started flakelet update other (via relay).487agent # [ 1402.959434] agent flakelet[1245]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1488agent # [ 1402.959896] agent flakelet[1245]: other: up to date489agent # [ 1402.962078] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.490client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTc4NCwiZ3JvdXBzIjpbIndoZWVsIl0sImlhdCI6MTc4OTA0MTQ4NCwiaXNzIjoiaHR0cHM6Ly9yZWxheTo4NDQzIiwianRpIjoiNk1UN0VaNjNFRjdVND09PSIsIm5iZiI6MTc4OTA0MTQ4NCwic3ViIjoic29tZW9uZSJ9.lHoYeHEm9bYJsNFUP4zgg93HYftWfLkAxA5VRmSRAtYrOwnQQkiMmfeZigS89y7nMyG2rT1gBPMJJMFoEyxRDg' push --relay https://relay:7443 deploy agent/other 2>&1, in 0.64 seconds)491(finished: subtest: claim principal from groups, unchanged counts as success, in 0.70 seconds)492subtest: push login via device flow, cached token used afterwards493client: must succeed: push login --issuer https://relay:8443 --client-id flakelet-relay 2>&1494relay # [ 1403.503537] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:04.998232Z","level":"INFO","message":"deploy finished","job":"64ce5a9610b5f3fb978d0b6e6326babd","ok":true,"target":"flakelet_relay::relay::api"}495relay # [ 1403.525506] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [10/Sep/2026 11:58:05] "POST /device HTTP/1.1" 200 -496agent # [ 1403.501807] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:04.996488Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}497relay # [ 1404.530993] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [10/Sep/2026 11:58:06] "POST /token HTTP/1.1" 400 -498client: (finished: must succeed: push login --issuer https://relay:8443 --client-id flakelet-relay 2>&1, in 2.08 seconds)499client: must succeed: test "$(stat -c %a /root/.local/state/flakelet-push/token.json)" = 600500client: (finished: must succeed: test "$(stat -c %a /root/.local/state/flakelet-push/token.json)" = 600, in 0.01 seconds)501client: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1502relay # [ 1405.579744] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [10/Sep/2026 11:58:07] "POST /token HTTP/1.1" 200 -503relay # [ 1405.614189] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:07.108913Z","level":"INFO","message":"deploy accepted","job":"482de79a49c78282b3bc7f2c99866a06","caller":"dev@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}504agent # [ 1405.614733] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:07.109412Z","level":"INFO","message":"start","id":"0b46ba0b5a77d4f06e2eaedf9a980e7f","flakelet":"other","rule":"admins","relay":"relay","caller_name":"\"dev@example.org\"","target":"flakelet_relay::agent::conn"}505agent # [ 1405.615867] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:07.110126Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"0b46ba0b5a77d4f06e2eaedf9a980e7f\"]","target":"flakelet_relay::agent::jobs"}506agent # [ 1405.676803] agent systemd[1]: Starting flakelet update other (via relay)...507agent # [ 1405.684647] agent systemd[1]: Started flakelet update other (via relay).508agent # [ 1405.692337] agent flakelet[1273]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1509agent # [ 1405.693629] agent flakelet[1273]: other: up to date510agent # [ 1405.696939] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.511client: (finished: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1, in 1.65 seconds)512client: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1513relay # [ 1407.242642] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:08.737369Z","level":"INFO","message":"deploy finished","job":"482de79a49c78282b3bc7f2c99866a06","ok":true,"target":"flakelet_relay::relay::api"}514relay # [ 1407.263504] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:08.758202Z","level":"INFO","message":"deploy accepted","job":"57723744fe04bb9406a0b98570a7eebc","caller":"dev@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}515agent # [ 1407.241146] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:08.735761Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}516agent # [ 1407.264052] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:08.758654Z","level":"INFO","message":"start","id":"28d7e80a2767a41436c18db0c1d9cab8","flakelet":"other","rule":"admins","relay":"relay","caller_name":"\"dev@example.org\"","target":"flakelet_relay::agent::conn"}517agent # [ 1407.264695] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:08.759412Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"28d7e80a2767a41436c18db0c1d9cab8\"]","target":"flakelet_relay::agent::jobs"}518agent # [ 1407.302359] agent systemd[1]: Starting flakelet update other (via relay)...519agent # [ 1407.323928] agent systemd[1]: Started flakelet update other (via relay).520agent # [ 1407.328678] agent flakelet[1294]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1521agent # [ 1407.329388] agent flakelet[1294]: other: up to date522agent # [ 1407.331536] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.523client: (finished: must succeed: push --relay https://relay:7443 deploy agent/other 2>&1, in 0.63 seconds)524(finished: subtest: push login via device flow, cached token used afterwards, in 4.37 seconds)525subtest: waves: failing first wave skips the second526agent: must succeed: jq '.services.app.prebuilt="/nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json527agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)528client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave agent/other 2>&1529relay # [ 1407.874960] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:09.369607Z","level":"INFO","message":"deploy finished","job":"57723744fe04bb9406a0b98570a7eebc","ok":true,"target":"flakelet_relay::relay::api"}530relay # [ 1407.913165] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:09.407892Z","level":"INFO","message":"deploy accepted","job":"8788c66594d8e3430f58bb2d4619446b","caller":"admin@example.org","targets":"[\"agent/app\", \"agent/other\"]","target":"flakelet_relay::relay::api"}531agent # [ 1407.873498] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:09.368131Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}532agent # [ 1407.913706] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:09.408380Z","level":"INFO","message":"start","id":"999563ee8f4c5f25d0b595db2ef0c4c8","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}533agent # [ 1407.914343] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:09.409100Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"999563ee8f4c5f25d0b595db2ef0c4c8\"]","target":"flakelet_relay::agent::jobs"}534agent # [ 1407.956239] agent systemd[1]: Starting flakelet update app (via relay)...535agent # [ 1407.965075] agent systemd[1]: Started flakelet update app (via relay).536agent # [ 1407.970518] agent flakelet[1321]: app: using prebuilt artifact /nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken537agent # [ 1407.971250] agent flakelet[1321]: app: activating generation 4538agent # [ 1407.991659] agent systemd[1]: Stopping app v1...539agent # [ 1407.992208] agent systemd[1]: app.service: Deactivated successfully.540agent # [ 1407.992549] agent systemd[1]: Stopped app v1.541agent # [ 1408.008970] agent systemd[1]: Reload requested from client PID 1328 ('systemctl') (unit flakelet-relay-job-app.service)...542agent # [ 1408.009083] agent systemd[1]: Reloading...543agent # [ 1408.380219] agent systemd[1]: Reloading finished in 370 ms.544agent # [ 1408.462126] agent systemd[1]: Reload requested from client PID 1358 ('systemctl') (unit flakelet-relay-job-app.service)...545agent # [ 1408.462223] agent systemd[1]: Reloading...546agent # [ 1408.822418] agent systemd[1]: Reloading finished in 359 ms.547agent # [ 1408.898643] agent systemd[1]: Reload requested from client PID 1387 ('systemctl') (unit flakelet-relay-job-app.service)...548agent # [ 1408.898687] agent systemd[1]: Reloading...549agent # [ 1409.256341] agent systemd[1]: Reloading finished in 357 ms.550agent # [ 1409.342707] agent systemd[1]: Reload requested from client PID 1418 ('systemctl') (unit flakelet-relay-job-app.service)...551agent # [ 1409.342754] agent systemd[1]: Reloading...552agent # [ 1409.711821] agent systemd[1]: Reloading finished in 368 ms.553agent # [ 1409.790217] agent systemd[1]: Starting app broken...554agent # [ 1409.796914] agent (nonexistent)[1452]: app.service: Unable to locate executable '/nonexistent': No such file or directory555agent # [ 1409.796938] agent (nonexistent)[1452]: app.service: Failed at step EXEC spawning /nonexistent: No such file or directory556agent # [ 1409.799216] agent systemd[1]: app.service: Main process exited, code=exited, status=203/EXEC557agent # [ 1409.800089] agent systemd[1]: app.service: Failed with result 'exit-code'.558agent # [ 1409.800299] agent systemd[1]: Failed to start app broken.559agent # [ 1409.826921] agent systemd[1]: Reload requested from client PID 1458 ('systemctl') (unit flakelet-relay-job-app.service)...560agent # [ 1409.826963] agent systemd[1]: Reloading...561agent # [ 1410.184840] agent systemd[1]: Reloading finished in 357 ms.562agent # [ 1410.273143] agent systemd[1]: Reload requested from client PID 1488 ('systemctl') (unit flakelet-relay-job-app.service)...563agent # [ 1410.273186] agent systemd[1]: Reloading...564agent # [ 1410.625839] agent systemd[1]: Reloading finished in 352 ms.565agent # [ 1410.712480] agent systemd[1]: Reload requested from client PID 1518 ('systemctl') (unit flakelet-relay-job-app.service)...566agent # [ 1410.712524] agent systemd[1]: Reloading...567agent # [ 1411.072487] agent systemd[1]: Reloading finished in 359 ms.568agent # [ 1411.140188] agent systemd[1]: Reload requested from client PID 1548 ('systemctl') (unit flakelet-relay-job-app.service)...569agent # [ 1411.140227] agent systemd[1]: Reloading...570agent # [ 1411.498260] agent systemd[1]: Reloading finished in 357 ms.571agent # [ 1411.583073] agent systemd[1]: Starting app v1...572agent # [ 1411.612634] agent systemd[1]: Started app v1.573agent # [ 1411.613930] agent flakelet[1321]: app: deploy failed, rolled back: systemctl start app.service failed:574agent # [ 1411.613930] agent flakelet[1321]: failed units: app.service. See journalctl -u app.service575agent # [ 1411.616712] agent systemd[1]: flakelet-relay-job-app.service: Main process exited, code=exited, status=1/FAILURE576agent # [ 1411.616930] agent systemd[1]: flakelet-relay-job-app.service: Failed with result 'exit-code'.577client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave agent/other 2>&1, in 4.37 seconds)578agent: must succeed: systemctl show app.service -p Description | grep -q 'app v1'579agent: (finished: must succeed: systemctl show app.service -p Description | grep -q 'app v1', in 0.02 seconds)580(finished: subtest: waves: failing first wave skips the second, in 4.41 seconds)581subtest: host patterns expand on the relay582agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json583agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)584client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/app' 2>&1585relay # [ 1412.258494] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:13.753197Z","level":"INFO","message":"deploy finished","job":"8788c66594d8e3430f58bb2d4619446b","ok":false,"target":"flakelet_relay::relay::api"}586relay # [ 1412.317811] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:13.812488Z","level":"INFO","message":"deploy accepted","job":"18bf8af9a3c30d1b21340110a7df6683","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}587agent # [ 1412.256936] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:13.751617Z","level":"INFO","message":"update finished","flakelet":"app","status":"RolledBack","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}588agent # [ 1412.318062] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:13.812720Z","level":"INFO","message":"start","id":"9db2c624451a617a4f5ed5e1d8273e51","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}589agent # [ 1412.318441] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:13.813207Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"9db2c624451a617a4f5ed5e1d8273e51\"]","target":"flakelet_relay::agent::jobs"}590agent # [ 1412.359939] agent systemd[1]: Starting flakelet update app (via relay)...591agent # [ 1412.389809] agent systemd[1]: Started flakelet update app (via relay).592agent # [ 1412.395180] agent flakelet[1608]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1593agent # [ 1412.395748] agent flakelet[1608]: app: up to date594agent # [ 1412.396929] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.595client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/app' 2>&1, in 0.65 seconds)596client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '*/app' 2>&1597relay # [ 1412.945045] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:14.439708Z","level":"INFO","message":"deploy finished","job":"18bf8af9a3c30d1b21340110a7df6683","ok":true,"target":"flakelet_relay::relay::api"}598agent # [ 1412.943255] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:14.437913Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}599relay # [ 1412.969269] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:14.464019Z","level":"INFO","message":"deploy accepted","job":"ccc347d1e54a47dd14c08d8ca6974b01","caller":"admin@example.org","targets":"[\"agent/app\", \"ghost/app\"]","target":"flakelet_relay::relay::api"}600agent # [ 1412.969751] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:14.464380Z","level":"INFO","message":"start","id":"7d8d5ad796adc0ca8f9f1798b707bb51","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}601agent # [ 1412.970362] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:14.465081Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"7d8d5ad796adc0ca8f9f1798b707bb51\"]","target":"flakelet_relay::agent::jobs"}602agent # [ 1413.011503] agent systemd[1]: Starting flakelet update app (via relay)...603agent # [ 1413.020093] agent systemd[1]: Started flakelet update app (via relay).604agent # [ 1413.024857] agent flakelet[1628]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1605agent # [ 1413.025567] agent flakelet[1628]: app: up to date606agent # [ 1413.028629] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.607client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '*/app' 2>&1, in 0.63 seconds)608client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave '*/app' 2>&1609relay # [ 1413.571655] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:15.066353Z","level":"INFO","message":"deploy finished","job":"ccc347d1e54a47dd14c08d8ca6974b01","ok":false,"target":"flakelet_relay::relay::api"}610relay # [ 1413.593705] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:15.088353Z","level":"INFO","message":"deploy accepted","job":"493bfdf1231834babfee085e9e84d195","caller":"admin@example.org","targets":"[\"agent/app\", \"ghost/app\"]","target":"flakelet_relay::relay::api"}611agent # [ 1413.569840] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:15.064475Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}612agent # [ 1413.594381] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:15.088963Z","level":"INFO","message":"start","id":"aa42f49ed86c71b2664c300b7b903e19","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}613agent # [ 1413.594873] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:15.089579Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"aa42f49ed86c71b2664c300b7b903e19\"]","target":"flakelet_relay::agent::jobs"}614agent # [ 1413.640509] agent systemd[1]: Starting flakelet update app (via relay)...615agent # [ 1413.649130] agent systemd[1]: Started flakelet update app (via relay).616agent # [ 1413.654118] agent flakelet[1652]: app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1617agent # [ 1413.654687] agent flakelet[1652]: app: up to date618agent # [ 1413.657386] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.619client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app --wave '*/app' 2>&1, in 0.63 seconds)620client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/nope' 2>&1621client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy '@all/nope' 2>&1, in 0.02 seconds)622(finished: subtest: host patterns expand on the relay, in 1.94 seconds)623subtest: detach returns once the job is accepted624agent: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json625agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)626client: must succeed: timeout 20 push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --detach agent/app 2>&1627client: (finished: must succeed: timeout 20 push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --detach agent/app 2>&1, in 0.02 seconds)628agent: waiting for success: systemctl is-active flakelet-relay-job-app.service629relay # [ 1414.204153] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:15.698776Z","level":"INFO","message":"deploy finished","job":"493bfdf1231834babfee085e9e84d195","ok":false,"target":"flakelet_relay::relay::api"}630relay # [ 1414.259673] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:15.754395Z","level":"INFO","message":"deploy accepted","job":"a25cd5b83857025398020e4de6df0589","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}631agent # [ 1414.202397] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:15.697090Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(3)","target":"flakelet_relay::agent::jobs"}632agent # [ 1414.260407] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:15.755004Z","level":"INFO","message":"start","id":"c3fefc1b8cab185b67de820dfd88f70f","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}633agent # [ 1414.261068] agent flakelet-agent[501]: {"timestamp":"2026-09-10T11:58:15.755728Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"c3fefc1b8cab185b67de820dfd88f70f\"]","target":"flakelet_relay::agent::jobs"}634agent # [ 1414.296813] agent systemd[1]: Starting flakelet update app (via relay)...635agent # [ 1414.305720] agent systemd[1]: Started flakelet update app (via relay).636agent # [ 1414.310247] agent flakelet[1676]: app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2637agent # [ 1414.310947] agent flakelet[1676]: app: activating generation 4638agent # [ 1414.325900] agent systemd[1]: Stopping app v1...639agent # [ 1414.326453] agent systemd[1]: app.service: Deactivated successfully.640agent # [ 1414.326810] agent systemd[1]: Stopped app v1.641agent # [ 1414.348852] agent systemd[1]: Reload requested from client PID 1683 ('systemctl') (unit flakelet-relay-job-app.service)...642agent # [ 1414.348937] agent systemd[1]: Reloading...643agent # [ 1414.710209] agent systemd[1]: Reloading finished in 360 ms.644agent # [ 1414.785841] agent systemd[1]: Reload requested from client PID 1713 ('systemctl') (unit flakelet-relay-job-app.service)...645agent # [ 1414.785893] agent systemd[1]: Reloading...646agent # [ 1415.152453] agent systemd[1]: Reloading finished in 366 ms.647agent # [ 1415.223825] agent systemd[1]: Reload requested from client PID 1742 ('systemctl') (unit flakelet-relay-job-app.service)...648agent # [ 1415.223877] agent systemd[1]: Reloading...649agent: (finished: waiting for success: systemctl is-active flakelet-relay-job-app.service, in 1.41 seconds)650??? Warning (UserWarning): wait_until_fails(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.651 File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39652agent: waiting for failure: systemctl is-active flakelet-relay-job-app.service653??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.654 File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39655agent # [ 1415.585940] agent systemd[1]: Reloading finished in 361 ms.656agent # [ 1415.676634] agent systemd[1]: Reload requested from client PID 1777 ('systemctl') (unit flakelet-relay-job-app.service)...657agent # [ 1415.676680] agent systemd[1]: Reloading...658agent # [ 1416.034375] agent systemd[1]: Reloading finished in 357 ms.659agent # [ 1416.112939] agent systemd[1]: Starting app slow2...660agent: (finished: waiting for failure: systemctl is-active flakelet-relay-job-app.service, in 6.52 seconds)661(finished: subtest: detach returns once the job is accepted, in 7.96 seconds)662subtest: unknown host is final, missing agent is retried, agents listing663client: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy nope/app 2>&1664client: (finished: must fail: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy nope/app 2>&1, in 0.02 seconds)665agent: must succeed: systemctl stop flakelet-agent666agent: (finished: must succeed: systemctl stop flakelet-agent, in 0.07 seconds)667relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 0'668relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 0', in 0.02 seconds)669client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/other > /tmp/retry.out 2>&1 &670client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/other > /tmp/retry.out 2>&1 &, in 0.01 seconds)671client: waiting for success: grep -q 'agent_unavailable.*retrying' /tmp/retry.out672agent # [ 1422.162703] agent systemd[1]: Started app slow2.673agent # [ 1422.170111] agent flakelet[1676]: app: updated to generation 4674agent # [ 1422.172678] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.675agent # [ 1422.223843] agent systemd[1]: Stopping flakelet-agent...676agent # [ 1422.248716] agent systemd[1]: flakelet-agent.service: Deactivated successfully.677agent # [ 1422.249325] agent systemd[1]: Stopped flakelet-agent.678agent # [ 1422.249826] agent systemd[1]: flakelet-agent.service: Consumed 1.601s CPU time over 28.741s wall clock time.679relay # [ 1422.223916] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:23.718648Z","level":"INFO","message":"agent disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","host":"agent","target":"flakelet_relay::relay::agent_conn"}680client: (finished: waiting for success: grep -q 'agent_unavailable.*retrying' /tmp/retry.out, in 1.02 seconds)681agent: must succeed: systemctl start flakelet-agent682agent: (finished: must succeed: systemctl start flakelet-agent, in 0.09 seconds)683??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.684 File "/nix/store/fv4yr9c0f9kyrfc00100i4rlshiiidbd-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39685client: waiting for success: grep -q 'agent/other: unchanged' /tmp/retry.out686agent # [ 1423.340765] agent systemd[1]: Starting flakelet-agent...687agent # [ 1423.394631] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:24.889318Z","level":"INFO","message":"reattaching to running unit","flakelet":"app","ids":"[\"c3fefc1b8cab185b67de820dfd88f70f\"]","target":"flakelet_relay::agent::jobs"}688agent # [ 1423.417795] agent systemd[1]: Started flakelet-agent.689agent # [ 1423.427873] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:24.922611Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}690relay # [ 1423.428074] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:24.922748Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(4), revision: Some(\"prebuilt:app\") }, Named { name: \"flakelet-agent\", generation: Some(1), revision: Some(\"prebuilt:flakelet-agent\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}691agent # [ 1423.940185] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:25.434784Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(4)","target":"flakelet_relay::agent::jobs"}692relay # [ 1425.327422] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:26.822155Z","level":"INFO","message":"deploy accepted","job":"79a56225857a3307d333780a9fd58b56","caller":"admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}693agent # [ 1425.327823] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:26.822539Z","level":"INFO","message":"start","id":"cde89db0fa5a525c89e207771d282c66","flakelet":"other","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}694agent # [ 1425.328375] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:26.823141Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"cde89db0fa5a525c89e207771d282c66\"]","target":"flakelet_relay::agent::jobs"}695agent # [ 1425.373160] agent systemd[1]: Starting flakelet update other (via relay)...696agent # [ 1425.393081] agent systemd[1]: Started flakelet update other (via relay).697agent # [ 1425.399417] agent flakelet[2254]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1698agent # [ 1425.400166] agent flakelet[2254]: other: up to date699agent # [ 1425.401045] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.700relay # [ 1425.942985] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:27.437716Z","level":"INFO","message":"deploy finished","job":"79a56225857a3307d333780a9fd58b56","ok":true,"target":"flakelet_relay::relay::api"}701agent # [ 1425.942053] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:27.436726Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}702client: (finished: waiting for success: grep -q 'agent/other: unchanged' /tmp/retry.out, in 3.04 seconds)703client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents704client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents, in 0.02 seconds)705client: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)706client: (finished: must succeed: echo '{"groups": []}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.05 seconds)707client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTgwNywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNTA4LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJNQkU3QUhSSFpENlFJPT09IiwibmJmIjoxNzg5MDQxNTA4LCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.fp7xxbVct-O6h5fy7DQdpTsfkBl3jaz_6cZ1FkXsfq_P8peymd7gGtQLZgRIQw8a5lRpP4j1MM82myW5oa15Bg' push --relay https://relay:7443 agents708client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTgwNywiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNTA4LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJNQkU3QUhSSFpENlFJPT09IiwibmJmIjoxNzg5MDQxNTA4LCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.fp7xxbVct-O6h5fy7DQdpTsfkBl3jaz_6cZ1FkXsfq_P8peymd7gGtQLZgRIQw8a5lRpP4j1MM82myW5oa15Bg' push --relay https://relay:7443 agents, in 0.02 seconds)709(finished: subtest: unknown host is final, missing agent is retried, agents listing, in 4.36 seconds)710subtest: out-of-band flakelet update reaches the relay without reconnect711agent: must succeed: flakelet status --json app | jq '.[0].generation'712agent: (finished: must succeed: flakelet status --json app | jq '.[0].generation', in 0.03 seconds)713agent: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true714agent: (finished: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true, in 0.01 seconds)715agent: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json716agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)717agent: must succeed: flakelet update app718app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v2719app: activating generation 5720agent # [ 1426.648897] agent systemd[1]: Stopping app slow2...721agent # [ 1426.649393] agent systemd[1]: app.service: Deactivated successfully.722agent # [ 1426.649739] agent systemd[1]: Stopped app slow2.723agent # [ 1426.670968] agent systemd[1]: Reload requested from client PID 2288 ('systemctl')...724agent # [ 1426.671069] agent systemd[1]: Reloading...725agent # [ 1427.034165] agent systemd[1]: Reloading finished in 362 ms.726agent # [ 1427.082199] agent systemd[1]: Reload requested from client PID 2317 ('systemctl')...727agent # [ 1427.082234] agent systemd[1]: Reloading...728agent # [ 1427.425896] agent systemd[1]: Reloading finished in 343 ms.729agent # [ 1427.482972] agent systemd[1]: Reload requested from client PID 2348 ('systemctl')...730agent # [ 1427.483025] agent systemd[1]: Reloading...731agent # [ 1427.829405] agent systemd[1]: Reloading finished in 345 ms.732agent # [ 1427.876428] agent systemd[1]: Reload requested from client PID 2378 ('systemctl')...733agent # [ 1427.876465] agent systemd[1]: Reloading...734agent: (finished: must succeed: flakelet update app, in 1.71 seconds)735client: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@5'736agent # [ 1428.215237] agent systemd[1]: Reloading finished in 338 ms.737agent # [ 1428.286226] agent systemd[1]: Starting app v2...738agent # [ 1428.321129] agent systemd[1]: Started app v2.739agent # [ 1429.449048] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:30.943746Z","level":"INFO","message":"local change","flakelet":"app","generation":5,"by":"root (shell)","target":"flakelet_relay::agent::jobs"}740client: (finished: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@5', in 2.07 seconds)741agent: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true742agent: (finished: must succeed: journalctl -u flakelet-agent -o cat | grep -c '"connected"' || true, in 0.01 seconds)743client: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs | grep local-app-5-744client: (finished: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs | grep local-app-5-, in 0.02 seconds)745client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs | grep local-app-746client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs | grep local-app-, in 0.02 seconds)747agent: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json748agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)749agent: must succeed: flakelet update app750app: using prebuilt artifact /nix/store/hysydfr96ds875vr0jgw5rjxi4p39h2j-flakelet-app-v1751app: activating generation 6752agent # [ 1430.490315] agent systemd[1]: Stopping app v2...753agent # [ 1430.492292] agent systemd[1]: app.service: Deactivated successfully.754agent # [ 1430.492717] agent systemd[1]: Stopped app v2.755agent # [ 1430.509699] agent systemd[1]: Reload requested from client PID 2424 ('systemctl')...756agent # [ 1430.509776] agent systemd[1]: Reloading...757agent # [ 1430.869669] agent systemd[1]: Reloading finished in 359 ms.758agent # [ 1430.916783] agent systemd[1]: Reload requested from client PID 2453 ('systemctl')...759agent # [ 1430.916825] agent systemd[1]: Reloading...760agent # [ 1431.262669] agent systemd[1]: Reloading finished in 345 ms.761agent # [ 1431.336989] agent systemd[1]: Reload requested from client PID 2481 ('systemctl')...762agent # [ 1431.337043] agent systemd[1]: Reloading...763agent # [ 1431.686315] agent systemd[1]: Reloading finished in 348 ms.764agent # [ 1431.768877] agent systemd[1]: Reload requested from client PID 2513 ('systemctl')...765agent # [ 1431.768921] agent systemd[1]: Reloading...766agent # [ 1432.118701] agent systemd[1]: Reloading finished in 349 ms.767agent: (finished: must succeed: flakelet update app, in 1.77 seconds)768client: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@6'769agent # [ 1432.196638] agent systemd[1]: Starting app v1...770agent # [ 1432.220722] agent systemd[1]: Started app v1.771agent # [ 1433.447037] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:34.941712Z","level":"INFO","message":"local change","flakelet":"app","generation":6,"by":"root (shell)","target":"flakelet_relay::agent::jobs"}772client: (finished: waiting for success: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem agents | grep -q 'app@6', in 2.07 seconds)773(finished: subtest: out-of-band flakelet update reaches the relay without reconnect, in 7.75 seconds)774subtest: certificate not listed under agents cannot connect as agent775client: must succeed: curl -s -o /dev/null -w '%{http_code}' --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/cert.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/key.pem -H 'Upgrade: websocket' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' https://relay:7443/v1/agent776client: (finished: must succeed: curl -s -o /dev/null -w '%{http_code}' --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/cert.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/mallory/key.pem -H 'Upgrade: websocket' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' https://relay:7443/v1/agent, in 0.03 seconds)777(finished: subtest: certificate not listed under agents cannot connect as agent, in 0.03 seconds)778subtest: metrics779relay: must succeed: curl -sf http://127.0.0.1:7400/metrics780relay: (finished: must succeed: curl -sf http://127.0.0.1:7400/metrics, in 0.02 seconds)781(finished: subtest: metrics, in 0.02 seconds)782subtest: agent link drop mid-run replays missed lines once783agent: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json784agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.01 seconds)785agent: must succeed: date +%s786agent: (finished: must succeed: date +%s, in 0.01 seconds)787client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/slow.out 2>&1 &788client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/slow.out 2>&1 &, in 0.01 seconds)789agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1789041515 | grep -q 'app-slow'790relay # [ 1434.325156] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:35.819838Z","level":"INFO","message":"agent rejected: no agents entry matches","principals":"[\"x509:dns:mallory\"]","target":"flakelet_relay::relay::agent_conn"}791relay # [ 1434.381558] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:35.876258Z","level":"INFO","message":"deploy accepted","job":"d4b764c17aa68a4783eadb7160113c67","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}792agent # [ 1434.381866] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:35.876565Z","level":"INFO","message":"start","id":"ab9a6084853663f8866ecf51aa8c19a9","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}793agent # [ 1434.382469] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:35.877234Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"ab9a6084853663f8866ecf51aa8c19a9\"]","target":"flakelet_relay::agent::jobs"}794agent # [ 1434.427359] agent systemd[1]: Starting flakelet update app (via relay)...795agent # [ 1434.445807] agent systemd[1]: Started flakelet update app (via relay).796agent # [ 1434.452149] agent flakelet[2563]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow797agent # [ 1434.452850] agent flakelet[2563]: app: activating generation 7798agent # [ 1434.471612] agent systemd[1]: Stopping app v1...799agent # [ 1434.472458] agent systemd[1]: app.service: Deactivated successfully.800agent # [ 1434.472630] agent systemd[1]: Stopped app v1.801agent # [ 1434.492891] agent systemd[1]: Reload requested from client PID 2570 ('systemctl') (unit flakelet-relay-job-app.service)...802agent # [ 1434.492979] agent systemd[1]: Reloading...803agent # [ 1434.849889] agent systemd[1]: Reloading finished in 356 ms.804agent # [ 1434.901355] agent systemd[1]: Reload requested from client PID 2600 ('systemctl') (unit flakelet-relay-job-app.service)...805agent # [ 1434.901413] agent systemd[1]: Reloading...806agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1789041515 | grep -q 'app-slow', in 1.03 seconds)807agent: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route add unreachable $a; done808agent: (finished: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route add unreachable $a; done, in 0.02 seconds)809relay: must succeed: for a in $(getent ahosts agent | cut -d' ' -f1 | sort -u); do ss -K dst "[$a]"; done810relay: (finished: must succeed: for a in $(getent ahosts agent | cut -d' ' -f1 | sort -u); do ss -K dst "[$a]"; done, in 0.09 seconds)811agent: must succeed: ss -K dport = 7443812agent # [ 1435.250941] agent systemd[1]: Reloading finished in 349 ms.813agent # [ 1435.290038] agent systemd[1]: Reload requested from client PID 2629 ('systemctl') (unit flakelet-relay-job-app.service)...814agent # [ 1435.290124] agent systemd[1]: Reloading...815agent: (finished: must succeed: ss -K dport = 7443, in 0.03 seconds)816agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1789041515 | grep -q 'updated to generation'817relay # [ 1435.509008] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:37.003730Z","level":"INFO","message":"agent disconnected: IO error: Software caused connection abort (os error 103)","host":"agent","target":"flakelet_relay::relay::agent_conn"}818agent # [ 1435.542526] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:37.037110Z","level":"INFO","message":"disconnected: IO error: Software caused connection abort (os error 103)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}819agent # [ 1435.639386] agent systemd[1]: Reloading finished in 348 ms.820agent # [ 1435.693529] agent systemd[1]: Reload requested from client PID 2677 ('systemctl') (unit flakelet-relay-job-app.service)...821agent # [ 1435.693575] agent systemd[1]: Reloading...822agent # [ 1436.040659] agent systemd[1]: Reloading finished in 346 ms.823agent # [ 1436.085098] agent systemd[1]: Starting app slow...824agent # [ 1436.546351] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:38.041023Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}825agent # [ 1438.549663] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:40.044350Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}826agent # [ 1442.125681] agent systemd[1]: Started app slow.827agent # [ 1442.134171] agent flakelet[2563]: app: updated to generation 7828agent # [ 1442.137791] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.829agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1789041515 | grep -q 'updated to generation', in 7.11 seconds)830agent: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route del unreachable $a; done831agent: (finished: must succeed: for a in $(getent ahosts relay | cut -d' ' -f1 | sort -u); do ip route del unreachable $a; done, in 0.02 seconds)832client: waiting for success: grep -q 'agent/app: updated' /tmp/slow.out833agent # [ 1442.552749] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:44.047365Z","level":"WARN","message":"connect failed: connect relay:7443: No route to host (os error 113)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}834agent # [ 1443.274276] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:44.768979Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(7)","target":"flakelet_relay::agent::jobs"}835client: (finished: waiting for success: grep -q 'agent/app: updated' /tmp/slow.out, in 8.09 seconds)836client: must succeed: cat /tmp/slow.out837client: (finished: must succeed: cat /tmp/slow.out, in 0.01 seconds)838(finished: subtest: agent link drop mid-run replays missed lines once, in 16.43 seconds)839subtest: agent restart mid-run reattaches to the unit840agent: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json841agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.02 seconds)842client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/restart.out 2>&1 &843client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/restart.out 2>&1 &, in 0.01 seconds)844agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since -20s | grep -q 'app-slow2'845relay # [ 1450.558225] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:52.052939Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(7), revision: Some(\"prebuilt:app\") }, Named { name: \"flakelet-agent\", generation: Some(1), revision: Some(\"prebuilt:flakelet-agent\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}846agent # [ 1450.558158] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:52.052844Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}847agent # [ 1450.559167] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:52.053137Z","level":"INFO","message":"start","id":"ab9a6084853663f8866ecf51aa8c19a9","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}848relay # [ 1450.559174] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:52.053683Z","level":"INFO","message":"deploy finished","job":"d4b764c17aa68a4783eadb7160113c67","ok":true,"target":"flakelet_relay::relay::api"}849relay # [ 1450.810305] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:52.304772Z","level":"INFO","message":"deploy accepted","job":"3b5ec6d8de6456198515baa836550434","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}850agent # [ 1450.810517] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:52.305166Z","level":"INFO","message":"start","id":"e42c314886aa80b6eea666cc5fbf5e8e","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}851agent # [ 1450.810910] agent flakelet-agent[1841]: {"timestamp":"2026-09-10T11:58:52.305677Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"e42c314886aa80b6eea666cc5fbf5e8e\"]","target":"flakelet_relay::agent::jobs"}852agent # [ 1450.857344] agent systemd[1]: Starting flakelet update app (via relay)...853agent # [ 1450.877215] agent systemd[1]: Started flakelet update app (via relay).854agent # [ 1450.883392] agent flakelet[2799]: app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow2855agent # [ 1450.884142] agent flakelet[2799]: app: activating generation 8856agent # [ 1450.904494] agent systemd[1]: Stopping app slow...857agent # [ 1450.906447] agent systemd[1]: app.service: Deactivated successfully.858agent # [ 1450.906882] agent systemd[1]: Stopped app slow.859agent # [ 1450.926488] agent systemd[1]: Reload requested from client PID 2806 ('systemctl') (unit flakelet-relay-job-app.service)...860agent # [ 1450.926577] agent systemd[1]: Reloading...861agent # [ 1451.284740] agent systemd[1]: Reloading finished in 357 ms.862agent # [ 1451.328377] agent systemd[1]: Reload requested from client PID 2836 ('systemctl') (unit flakelet-relay-job-app.service)...863agent # [ 1451.328415] agent systemd[1]: Reloading...864agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since -20s | grep -q 'app-slow2', in 1.03 seconds)865agent: must succeed: systemctl restart flakelet-agent866agent # [ 1451.669337] agent systemd[1]: Reloading finished in 340 ms.867agent # [ 1451.729654] agent systemd[1]: Reload requested from client PID 2868 ('systemctl') (unit flakelet-relay-job-app.service)...868agent # [ 1451.729694] agent systemd[1]: Reloading...869agent # [ 1452.071879] agent systemd[1]: Reloading finished in 341 ms.870agent # [ 1452.140641] agent systemd[1]: Stopping flakelet-agent...871agent # [ 1452.150257] agent systemd[1]: Reload requested from client PID 2904 ('systemctl') (unit flakelet-relay-job-app.service)...872agent # [ 1452.150347] agent systemd[1]: Reloading...873agent: (finished: must succeed: systemctl restart flakelet-agent, in 0.80 seconds)874agent: must succeed: test -n "$(ls /var/lib/flakelet-agent/jobs)"875relay # [ 1452.141269] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:53.635942Z","level":"INFO","message":"agent disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","host":"agent","target":"flakelet_relay::relay::agent_conn"}876agent: (finished: must succeed: test -n "$(ls /var/lib/flakelet-agent/jobs)", in 0.01 seconds)877client: waiting for success: grep -q 'agent/app: updated' /tmp/restart.out878agent # [ 1452.504776] agent systemd[1]: Reloading finished in 353 ms.879agent # [ 1452.538843] agent systemd[1]: flakelet-agent.service: Deactivated successfully.880agent # [ 1452.539106] agent systemd[1]: Stopped flakelet-agent.881agent # [ 1452.539122] agent systemd[1]: flakelet-agent.service: Consumed 1.078s CPU time over 29.198s wall clock time.882agent # [ 1452.557274] agent systemd[1]: Starting app slow2...883agent # [ 1452.558328] agent systemd[1]: Starting flakelet-agent...884agent # [ 1452.603022] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:58:54.097728Z","level":"INFO","message":"reattaching to running unit","flakelet":"app","ids":"[\"e42c314886aa80b6eea666cc5fbf5e8e\"]","target":"flakelet_relay::agent::jobs"}885agent # [ 1452.624193] agent systemd[1]: Started flakelet-agent.886agent # [ 1452.634777] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:58:54.129483Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}887agent # [ 1452.636070] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:58:54.130836Z","level":"INFO","message":"start","id":"e42c314886aa80b6eea666cc5fbf5e8e","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}888relay # [ 1452.634870] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:58:54.129595Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(7), revision: Some(\"prebuilt:app\") }, Named { name: \"flakelet-agent\", generation: Some(1), revision: Some(\"prebuilt:flakelet-agent\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}889agent # [ 1458.616776] agent systemd[1]: Started app slow2.890agent # [ 1458.625819] agent flakelet[2799]: app: updated to generation 8891agent # [ 1458.629732] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.892relay # [ 1459.196794] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:59:00.691439Z","level":"INFO","message":"deploy finished","job":"3b5ec6d8de6456198515baa836550434","ok":true,"target":"flakelet_relay::relay::api"}893agent # [ 1459.195387] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:00.690078Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(8)","target":"flakelet_relay::agent::jobs"}894client: (finished: waiting for success: grep -q 'agent/app: updated' /tmp/restart.out, in 7.09 seconds)895client: must succeed: cat /tmp/restart.out896client: (finished: must succeed: cat /tmp/restart.out, in 0.01 seconds)897agent: must succeed: systemctl is-active app.service898agent: (finished: must succeed: systemctl is-active app.service, in 0.01 seconds)899(finished: subtest: agent restart mid-run reattaches to the unit, in 8.98 seconds)900subtest: relay restart mid-run, push resumes via /v1/jobs901agent: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json902agent: (finished: must succeed: jq '.services.app.prebuilt="/nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow"' /etc/flakelet/config.json > /tmp/config.json && cp --remove-destination /tmp/config.json /etc/flakelet/config.json, in 0.02 seconds)903agent: must succeed: date +%s904agent: (finished: must succeed: date +%s, in 0.01 seconds)905client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/resume.out 2>&1 &906client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy agent/app > /tmp/resume.out 2>&1 &, in 0.01 seconds)907agent: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1789041541 | grep -q 'app-slow'908relay # [ 1459.796759] relay flakelet-relay[400]: {"timestamp":"2026-09-10T11:59:01.291426Z","level":"INFO","message":"deploy accepted","job":"2457763d27299411effb90a481d8634e","caller":"admin@example.org","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}909agent # [ 1459.797190] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:01.291877Z","level":"INFO","message":"start","id":"08f1102b4fe1c2e0e953c6ce4c312dd1","flakelet":"app","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}910agent # [ 1459.797835] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:01.292553Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"08f1102b4fe1c2e0e953c6ce4c312dd1\"]","target":"flakelet_relay::agent::jobs"}911agent # [ 1459.844991] agent systemd[1]: Starting flakelet update app (via relay)...912agent # [ 1459.862917] agent systemd[1]: Started flakelet update app (via relay).913agent # [ 1459.868852] agent flakelet[3381]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow914agent # [ 1459.869981] agent flakelet[3381]: app: activating generation 9915agent # [ 1459.890981] agent systemd[1]: Stopping app slow2...916agent # [ 1459.891515] agent systemd[1]: app.service: Deactivated successfully.917agent # [ 1459.891856] agent systemd[1]: Stopped app slow2.918agent # [ 1459.912427] agent systemd[1]: Reload requested from client PID 3388 ('systemctl') (unit flakelet-relay-job-app.service)...919agent # [ 1459.912515] agent systemd[1]: Reloading...920agent # [ 1460.268922] agent systemd[1]: Reloading finished in 355 ms.921agent # [ 1460.348045] agent systemd[1]: Reload requested from client PID 3418 ('systemctl') (unit flakelet-relay-job-app.service)...922agent # [ 1460.348087] agent systemd[1]: Reloading...923agent: (finished: waiting for success: journalctl -u flakelet-relay-job-app.service --since=@1789041541 | grep -q 'app-slow', in 1.02 seconds)924relay: must succeed: systemctl restart flakelet-relay925relay # [ 1460.821894] relay systemd[1]: Stopping flakelet-relay...926relay # [ 1460.861269] relay systemd[1]: flakelet-relay.service: Deactivated successfully.927relay # [ 1460.861691] relay systemd[1]: Stopped flakelet-relay.928relay # [ 1460.861718] relay systemd[1]: flakelet-relay.service: Consumed 2.241s CPU time over 1min 9.071s wall clock time.929relay # [ 1460.879903] relay systemd[1]: Starting flakelet-relay...930agent # [ 1460.718494] agent systemd[1]: Reloading finished in 369 ms.931agent # [ 1460.768619] agent systemd[1]: Reload requested from client PID 3450 ('systemctl') (unit flakelet-relay-job-app.service)...932agent # [ 1460.768662] agent systemd[1]: Reloading...933agent # [ 1460.822519] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:02.317233Z","level":"INFO","message":"disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}934agent # [ 1461.172984] agent systemd[1]: Reloading finished in 403 ms.935agent # [ 1461.238599] agent systemd[1]: Reload requested from client PID 3485 ('systemctl') (unit flakelet-relay-job-app.service)...936agent # [ 1461.238637] agent systemd[1]: Reloading...937agent # [ 1461.601582] agent systemd[1]: Reloading finished in 362 ms.938agent # [ 1461.679102] agent systemd[1]: Starting app slow...939agent # [ 1461.824976] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:03.319700Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}940relay: (finished: must succeed: systemctl restart flakelet-relay, in 1.53 seconds)941client: waiting for success: grep -q '» ok' /tmp/resume.out942relay # [ 1462.338710] relay flakelet-relay[933]: {"timestamp":"2026-09-10T11:59:03.833397Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}943relay # [ 1462.339640] relay flakelet-relay[933]: {"timestamp":"2026-09-10T11:59:03.833881Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}944relay # [ 1462.339162] relay systemd[1]: Started flakelet-relay.945relay # [ 1463.840185] relay flakelet-relay[933]: {"timestamp":"2026-09-10T11:59:05.334851Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(8), revision: Some(\"prebuilt:app\") }, Named { name: \"flakelet-agent\", generation: Some(1), revision: Some(\"prebuilt:flakelet-agent\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}946relay # [ 1463.841484] relay flakelet-relay[933]: {"timestamp":"2026-09-10T11:59:05.336161Z","level":"INFO","message":"job reattached","job":"2457763d27299411effb90a481d8634e","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}947agent # [ 1463.839816] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:05.334481Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}948agent # [ 1467.728213] agent systemd[1]: Started app slow.949agent # [ 1467.738199] agent flakelet[3381]: app: updated to generation 9950agent # [ 1467.741814] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.951agent # [ 1468.821975] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:10.316670Z","level":"INFO","message":"update finished","flakelet":"app","status":"Updated","generation":"Some(9)","target":"flakelet_relay::agent::jobs"}952client: (finished: waiting for success: grep -q '» ok' /tmp/resume.out, in 7.08 seconds)953client: must succeed: cat /tmp/resume.out954client: (finished: must succeed: cat /tmp/resume.out, in 0.01 seconds)955(finished: subtest: relay restart mid-run, push resumes via /v1/jobs, in 9.68 seconds)956subtest: job history comes from the agents and survives a relay restart957client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id hist1 agent/other 2>&1958relay # [ 1469.453367] relay flakelet-relay[933]: {"timestamp":"2026-09-10T11:59:10.948058Z","level":"INFO","message":"deploy accepted","job":"188b06c4636b65849b3ba8daac22dc7c","caller":"admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}959agent # [ 1469.453671] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:10.948366Z","level":"INFO","message":"start","id":"5be20c693e5c0b60e3f85779794dfb86","flakelet":"other","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}960agent # [ 1469.454291] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:10.948970Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"5be20c693e5c0b60e3f85779794dfb86\"]","target":"flakelet_relay::agent::jobs"}961agent # [ 1469.498963] agent systemd[1]: Starting flakelet update other (via relay)...962agent # [ 1469.516818] agent systemd[1]: Started flakelet update other (via relay).963agent # [ 1469.521971] agent flakelet[3559]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1964agent # [ 1469.522729] agent flakelet[3559]: other: up to date965agent # [ 1469.525016] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.966client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id hist1 agent/other 2>&1, in 0.64 seconds)967relay: must succeed: systemctl restart flakelet-relay968relay # [ 1470.068254] relay flakelet-relay[933]: {"timestamp":"2026-09-10T11:59:11.562950Z","level":"INFO","message":"deploy finished","job":"188b06c4636b65849b3ba8daac22dc7c","ok":true,"target":"flakelet_relay::relay::api"}969relay # [ 1470.085291] relay systemd[1]: Stopping flakelet-relay...970relay # [ 1470.135095] relay systemd[1]: flakelet-relay.service: Deactivated successfully.971relay # [ 1470.135640] relay systemd[1]: Stopped flakelet-relay.972relay # [ 1470.135682] relay systemd[1]: flakelet-relay.service: Consumed 1.721s CPU time over 9.255s wall clock time.973relay # [ 1470.172335] relay systemd[1]: Starting flakelet-relay...974agent # [ 1470.066531] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:11.561226Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}975agent # [ 1470.085653] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:11.580401Z","level":"INFO","message":"disconnected: IO error: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}976agent # [ 1471.088523] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:12.583202Z","level":"WARN","message":"connect failed: connect relay:7443: Connection refused (os error 111)","relay":"https://relay:7443","target":"flakelet_relay::agent::conn"}977relay: (finished: must succeed: systemctl restart flakelet-relay, in 1.50 seconds)978relay: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1'979relay # [ 1471.572392] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:13.067079Z","level":"INFO","message":"listening on 127.0.0.1:7400","target":"flakelet_relay::relay::server"}980relay # [ 1471.573250] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:13.067502Z","level":"INFO","message":"listening on [::]:7443","target":"flakelet_relay::relay::server"}981relay # [ 1471.572835] relay systemd[1]: Started flakelet-relay.982relay # [ 1473.103654] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:14.598291Z","level":"INFO","message":"agent connected","host":"agent","version":"0.1.0","flakelets":"[Named { name: \"app\", generation: Some(9), revision: Some(\"prebuilt:app\") }, Named { name: \"flakelet-agent\", generation: Some(1), revision: Some(\"prebuilt:flakelet-agent\") }, Named { name: \"other\", generation: Some(1), revision: Some(\"prebuilt:other\") }]","target":"flakelet_relay::relay::agent_conn"}983agent # [ 1473.103451] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:14.598144Z","level":"INFO","message":"connected","relay":"https://relay:7443","relay_name":"relay","relay_version":"0.1.0","host":"agent","target":"flakelet_relay::agent::conn"}984relay: (finished: waiting for success: curl -sf http://127.0.0.1:7400/metrics | grep -q 'agent_up{host="agent"} 1', in 2.06 seconds)985client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs986client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem jobs, in 0.02 seconds)987client: must succeed: echo '{"groups": [], "repository": "example/app"}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32)988client: (finished: must succeed: echo '{"groups": [], "repository": "example/app"}' | step crypto jwt sign --key /nix/store/r7sclamg09v434w8zjlfnrcrn1w79wh6-flakelet-relay-test-issuer/jwk.json --iss https://relay:8443 --aud flakelet-relay --sub 'repo:github:example/app:ref:refs/heads/main' --exp $(( $(date +%s) + 300 )) --jti $(head -c8 /dev/urandom | base32), in 0.06 seconds)989client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTg1NSwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNTU1LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJXWVpXTkJNQzdYUzJRPT09IiwibmJmIjoxNzg5MDQxNTU1LCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.EGfXvXkQztBk8ELcHzpQhPBgJErbZ_dgCZNTLkEqWz35rahS4CBnyy7dZFmr1o4Z0WUnuqieap7HedYwb8ISAw' push --relay https://relay:7443 deploy --id hist2 agent/app990» job 6a239dbf87cfec796e49404c351d787f via relay 0.1.0 (agents: agent 0.1.0)991» wave 0992agent/app | Starting flakelet update app (via relay)...993agent/app | Started flakelet update app (via relay).994relay # [ 1473.731546] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:15.226240Z","level":"INFO","message":"deploy accepted","job":"6a239dbf87cfec796e49404c351d787f","caller":"example/app","targets":"[\"agent/app\"]","target":"flakelet_relay::relay::api"}995agent # [ 1473.732036] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:15.226693Z","level":"INFO","message":"start","id":"dce6faec8455f85321e72ee634888126","flakelet":"app","rule":"ci","relay":"relay","caller_name":"\"example/app\"","target":"flakelet_relay::agent::conn"}996agent # [ 1473.732783] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:15.227460Z","level":"INFO","message":"update starting","flakelet":"app","ids":"[\"dce6faec8455f85321e72ee634888126\"]","target":"flakelet_relay::agent::jobs"}997agent # [ 1473.779986] agent systemd[1]: Starting flakelet update app (via relay)...998agent # [ 1473.801704] agent systemd[1]: Started flakelet update app (via relay).999agent # [ 1473.807721] agent flakelet[3587]: app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow1000agent # [ 1473.808449] agent flakelet[3587]: app: up to date1001agent # [ 1473.810743] agent systemd[1]: flakelet-relay-job-app.service: Deactivated successfully.1002agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow1003agent/app | app: up to date1004agent/app | flakelet-relay-job-app.service: Deactivated successfully.1005» agent/app: unchanged (generation 9)1006» ok1007client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTg1NSwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNTU1LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJXWVpXTkJNQzdYUzJRPT09IiwibmJmIjoxNzg5MDQxNTU1LCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.EGfXvXkQztBk8ELcHzpQhPBgJErbZ_dgCZNTLkEqWz35rahS4CBnyy7dZFmr1o4Z0WUnuqieap7HedYwb8ISAw' push --relay https://relay:7443 deploy --id hist2 agent/app, in 0.64 seconds)1008client: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTg1NSwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNTU1LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJXWVpXTkJNQzdYUzJRPT09IiwibmJmIjoxNzg5MDQxNTU1LCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.EGfXvXkQztBk8ELcHzpQhPBgJErbZ_dgCZNTLkEqWz35rahS4CBnyy7dZFmr1o4Z0WUnuqieap7HedYwb8ISAw' push --relay https://relay:7443 jobs1009client: (finished: must succeed: FLAKELET_RELAY_TOKEN_COMMAND='echo eyJhbGciOiJFZERTQSIsImtpZCI6ImsxIiwidHlwIjoiSldUIn0.eyJhdWQiOiJmbGFrZWxldC1yZWxheSIsImV4cCI6MTc4OTA0MTg1NSwiZ3JvdXBzIjpbXSwiaWF0IjoxNzg5MDQxNTU1LCJpc3MiOiJodHRwczovL3JlbGF5Ojg0NDMiLCJqdGkiOiJXWVpXTkJNQzdYUzJRPT09IiwibmJmIjoxNzg5MDQxNTU1LCJyZXBvc2l0b3J5IjoiZXhhbXBsZS9hcHAiLCJzdWIiOiJyZXBvOmdpdGh1YjpleGFtcGxlL2FwcDpyZWY6cmVmcy9oZWFkcy9tYWluIn0.EGfXvXkQztBk8ELcHzpQhPBgJErbZ_dgCZNTLkEqWz35rahS4CBnyy7dZFmr1o4Z0WUnuqieap7HedYwb8ISAw' push --relay https://relay:7443 jobs, in 0.02 seconds)1010(finished: subtest: job history comes from the agents and survives a relay restart, in 4.94 seconds)1011subtest: dashboard login via authorization code flow1012client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code} %{redirect_url}' https://relay:7443/ui/ | grep -q '303 https://relay:7443/ui/login'1013client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code} %{redirect_url}' https://relay:7443/ui/ | grep -q '303 https://relay:7443/ui/login', in 0.02 seconds)1014client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -L https://relay:7443/ui/login1015client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -L https://relay:7443/ui/login, in 0.08 seconds)1016client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs1017client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs, in 0.02 seconds)1018client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -f https://relay:7443/v1/jobs | grep -q hist11019client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -f https://relay:7443/v1/jobs | grep -q hist1, in 0.02 seconds)1020client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 10 https://relay:7443/ui/jobs/hist1/events || true1021client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 10 https://relay:7443/ui/jobs/hist1/events || true, in 0.02 seconds)1022client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -X POST 'https://relay:7443/ui/deploy?arg=other' | grep -q 4031023client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -X POST 'https://relay:7443/ui/deploy?arg=other' | grep -q 403, in 0.02 seconds)1024client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -D - -o /dev/null -H 'HX-Request: true' -X POST 'https://relay:7443/ui/deploy?arg=other'1025client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -D - -o /dev/null -H 'HX-Request: true' -X POST 'https://relay:7443/ui/deploy?arg=other', in 0.02 seconds)1026client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 30 https://relay:7443/ui/jobs/b53d4fcba08cbe9fbd49535e39e6cf45/events || true1027relay # [ 1474.353047] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:15.847742Z","level":"INFO","message":"deploy finished","job":"6a239dbf87cfec796e49404c351d787f","ok":true,"target":"flakelet_relay::relay::api"}1028relay # [ 1474.432266] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [10/Sep/2026 11:59:15] "GET /authorize?response_type=code&client_id=dashboard&redirect_uri=https%3A%2F%2Frelay%3A7443%2Fui%2Fcallback&scope=openid%20profile%20email%20groups&state=dJ0YOaTs22d_PjiWReQns1NRtFW_YIEKSCYDvDIYrFM&code_challenge=Mwgqipfh6ASuMxDxhGGnLjrX5pMUC9pP3dQmF1uCrrE&code_challenge_method=S256 HTTP/1.1" 302 -1029relay # [ 1474.480026] relay 93sn3sv24fj26ksccpppzwg6kbf10s4w-device-mock[186]: 127.0.0.1 - - [10/Sep/2026 11:59:15] "POST /token HTTP/1.1" 200 -1030relay # [ 1474.481191] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:15.975933Z","level":"INFO","message":"login","name":"dev@example.org","principals":"[\"oidc:mock:someone\", \"oidc:mock:groups:wheel\"]","target":"flakelet_relay::relay::login"}1031relay # [ 1474.546493] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:16.041188Z","level":"INFO","message":"job reattached","job":"188b06c4636b65849b3ba8daac22dc7c","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}1032relay # [ 1474.591720] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:16.086409Z","level":"INFO","message":"deploy accepted","job":"45e800226ec9983360aca748123679fa","caller":"dev@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}1033relay # [ 1474.614508] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:16.109102Z","level":"INFO","message":"job reattached","job":"45e800226ec9983360aca748123679fa","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}1034agent # [ 1474.351513] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:15.846206Z","level":"INFO","message":"update finished","flakelet":"app","status":"Unchanged","generation":"Some(9)","target":"flakelet_relay::agent::jobs"}1035agent # [ 1474.592368] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:16.087032Z","level":"INFO","message":"start","id":"6c5521945a6a3649fa84c579b0e19fab","flakelet":"other","rule":"admins","relay":"relay","caller_name":"\"dev@example.org\"","target":"flakelet_relay::agent::conn"}1036agent # [ 1474.592959] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:16.087719Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"6c5521945a6a3649fa84c579b0e19fab\"]","target":"flakelet_relay::agent::jobs"}1037agent # [ 1474.637883] agent systemd[1]: Starting flakelet update other (via relay)...1038agent # [ 1474.658579] agent systemd[1]: Started flakelet update other (via relay).1039agent # [ 1474.662785] agent flakelet[3609]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v11040agent # [ 1474.663504] agent flakelet[3609]: other: up to date1041agent # [ 1474.666633] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.1042client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 30 https://relay:7443/ui/jobs/b53d4fcba08cbe9fbd49535e39e6cf45/events || true, in 0.62 seconds)1043client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs/b53d4fcba08cbe9fbd49535e39e6cf451044client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/jobs/b53d4fcba08cbe9fbd49535e39e6cf45, in 0.02 seconds)1045client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -sf -o /dev/null https://relay:7443/ui/static/htmx.min.js1046client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -sf -o /dev/null https://relay:7443/ui/static/htmx.min.js, in 0.02 seconds)1047client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/?q=status%3Ahealthy+oth'1048client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/?q=status%3Ahealthy+oth', in 0.02 seconds)1049client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/flakelets/other1050client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar https://relay:7443/ui/flakelets/other, in 0.02 seconds)1051client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/hosts?q=status%3Adisconnected'1052client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar 'https://relay:7443/ui/hosts?q=status%3Adisconnected', in 0.02 seconds)1053client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -H 'HX-Request: true' -X POST 'https://relay:7443/ui/retry?arg=b53d4fcba08cbe9fbd49535e39e6cf45' | grep -q 4001054client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' -H 'HX-Request: true' -X POST 'https://relay:7443/ui/retry?arg=b53d4fcba08cbe9fbd49535e39e6cf45' | grep -q 400, in 0.02 seconds)1055client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 15 'https://relay:7443/ui/events?path=jobs' > /tmp/ev.out 2>&1 &1056client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -N --max-time 15 'https://relay:7443/ui/events?path=jobs' > /tmp/ev.out 2>&1 &, in 0.01 seconds)1057client: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id live1 agent/other1058» job 38796a7ba508eadfe82a1cd5def000bc via relay 0.1.0 (agents: agent 0.1.0)1059» wave 01060agent/other | Starting flakelet update other (via relay)...1061agent/other | Started flakelet update other (via relay).1062relay # [ 1475.354836] relay flakelet-relay[1323]: {"timestamp":"2026-09-10T11:59:16.849529Z","level":"INFO","message":"deploy accepted","job":"38796a7ba508eadfe82a1cd5def000bc","caller":"admin@example.org","targets":"[\"agent/other\"]","target":"flakelet_relay::relay::api"}1063agent # [ 1475.206668] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:16.701362Z","level":"INFO","message":"update finished","flakelet":"other","status":"Unchanged","generation":"Some(1)","target":"flakelet_relay::agent::jobs"}1064agent # [ 1475.355205] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:16.849893Z","level":"INFO","message":"start","id":"5e317fc57863c2a7c03ebe1bdacd1356","flakelet":"other","rule":"admins","relay":"relay","caller_name":"\"admin@example.org\"","target":"flakelet_relay::agent::conn"}1065agent # [ 1475.356257] agent flakelet-agent[2936]: {"timestamp":"2026-09-10T11:59:16.850244Z","level":"INFO","message":"update starting","flakelet":"other","ids":"[\"5e317fc57863c2a7c03ebe1bdacd1356\"]","target":"flakelet_relay::agent::jobs"}1066agent # [ 1475.400133] agent systemd[1]: Starting flakelet update other (via relay)...1067agent/otheragent # [ 1475.416596] agent systemd[1]: Started flakelet update other (via relay).1068 | other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v1agent # [ 1475.421709] agent flakelet[3631]: other: using prebuilt artifact /nix/store/hmr990n0b4v6rpzrcpyyw738r9xi7z95-flakelet-other-v110691070agent # [ 1475.422409] agent flakelet[3631]: other: up to date1071agent/other | agent # [ 1475.424704] agent systemd[1]: flakelet-relay-job-other.service: Deactivated successfully.1072other: up to date1073agent/other | flakelet-relay-job-other.service: Deactivated successfully.1074» agent/other: unchanged (generation 1)1075» ok1076client: (finished: must succeed: push --relay https://relay:7443 --cert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin.pem --key /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/admin-key.pem deploy --id live1 agent/other, in 0.63 seconds)1077client: waiting for success: grep -q live1 /tmp/ev.out1078client: (finished: waiting for success: grep -q live1 /tmp/ev.out, in 0.01 seconds)1079client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -X POST -o /dev/null https://relay:7443/ui/logout1080client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -X POST -o /dev/null https://relay:7443/ui/logout, in 0.02 seconds)1081client: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' https://relay:7443/ui/ | grep -q 3031082client: (finished: must succeed: curl -sS --cacert /nix/store/ydql3jgw8j7khsad67k605wlnywsd36n-flakelet-relay-test-certs/ca.pem -b /tmp/jar -c /tmp/jar -o /dev/null -w '%{http_code}' https://relay:7443/ui/ | grep -q 303, in 0.02 seconds)1083(finished: subtest: dashboard login via authorization code flow, in 1.65 seconds)1084(finished: run the VM test script, in 87.70 seconds)1085test script finished in 87.74s1086cleanup1087kill NspawnMachine (pid 52)1088kill NspawnMachine (pid 54)1089Container agent terminated by signal KILL.1090kill NspawnMachine (pid 53)1091Container client terminated by signal KILL.1092(finished: cleanup, in 0.29 seconds)1093additionally exposed symbols:1094 agent, client, relay,1095 vlan1,1096 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh1097» job 4dee1fef3671457119b29328889c5ff9 via relay 0.1.0 (agents: agent 0.1.0)1098» wave 01099agent/app | Starting flakelet update app (via relay)...1100agent/app | Started flakelet update app (via relay).1101agent/app | app: using prebuilt artifact /nix/store/6ili1vxjlag9d7dy0bxrbxk6bpi5qh9p-flakelet-app-v21102agent/app | app: activating generation 21103agent/app | app: updated to generation 21104agent/app | flakelet-relay-job-app.service: Deactivated successfully.1105» agent/app: updated (generation 2)1106» ok11071108» job 8788c66594d8e3430f58bb2d4619446b via relay 0.1.0 (agents: agent 0.1.0)1109» wave 01110agent/app | Starting flakelet update app (via relay)...1111agent/app | Started flakelet update app (via relay).1112agent/app | app: using prebuilt artifact /nix/store/8282rnbvyk847q4l4zby3zwcq3jpzrvw-flakelet-app-broken1113agent/app | app: activating generation 41114agent/app | app: deploy failed, rolled back: systemctl start app.service failed:1115agent/app | failed units: app.service. See journalctl -u app.service1116agent/app | flakelet-relay-job-app.service: Main process exited, code=exited, status=1/FAILURE1117agent/app | flakelet-relay-job-app.service: Failed with result 'exit-code'.1118» agent/app: rolled-back (generation 3)1119agent/app │ Starting app v1...1120agent/app │ Started app v1.1121agent/app │ Stopping app v1...1122agent/app │ app.service: Deactivated successfully.1123agent/app │ Stopped app v1.1124agent/app │ Starting app v2...1125agent/app │ Started app v2.1126agent/app │ Stopping app v2...1127agent/app │ app.service: Deactivated successfully.1128agent/app │ Stopped app v2.1129agent/app │ Starting app v1...1130agent/app │ Started app v1.1131agent/app │ Stopping app v1...1132agent/app │ app.service: Deactivated successfully.1133agent/app │ Stopped app v1.1134agent/app │ Starting app broken...1135agent/app │ app.service: Unable to locate executable '/nonexistent': No such file or directory1136agent/app │ app.service: Failed at step EXEC spawning /nonexistent: No such file or directory1137agent/app │ app.service: Main process exited, code=exited, status=203/EXEC1138agent/app │ app.service: Failed with result 'exit-code'.1139agent/app │ Failed to start app broken.1140agent/app │ Starting app v1...1141agent/app │ Started app v1.1142» skipped: agent/other1143» failed11441145» job d4b764c17aa68a4783eadb7160113c67 via relay 0.1.0 (agents: agent 0.1.0)1146» wave 01147agent/app | Starting flakelet update app (via relay)...1148agent/app | Started flakelet update app (via relay).1149agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow1150agent/app | app: activating generation 71151agent/app | app: updated to generation 71152agent/app | flakelet-relay-job-app.service: Deactivated successfully.1153» agent/app: updated (generation 7)1154» ok11551156» job 3b5ec6d8de6456198515baa836550434 via relay 0.1.0 (agents: agent 0.1.0)1157» wave 01158agent/app | Starting flakelet update app (via relay)...1159agent/app | Started flakelet update app (via relay).1160agent/app | app: using prebuilt artifact /nix/store/ln47rsd5km8a04hd5zlkz4q1v352zqh3-flakelet-app-slow21161agent/app | app: activating generation 81162agent/app | app: updated to generation 81163agent/app | flakelet-relay-job-app.service: Deactivated successfully.1164» agent/app: updated (generation 8)1165» ok11661167» job 2457763d27299411effb90a481d8634e via relay 0.1.0 (agents: agent 0.1.0)1168» wave 01169agent/app | Starting flakelet update app (via relay)...1170agent/app | Started flakelet update app (via relay).1171agent/app | app: using prebuilt artifact /nix/store/c67jj8kx3m50nyj874aqmdqf21w5iznd-flakelet-app-slow1172agent/app | app: activating generation 91173flakelet-push: stream: error reading a body from connection, resuming job c095c3c4a5d6ef24a325f9a8276721b01174flakelet-push: https://relay:7443: connect relay:7443: Connection refused (os error 111), retrying in 1s1175flakelet-push: https://relay:7443: connect relay:7443: Connection refused (os error 111), retrying in 2s1176» job 2457763d27299411effb90a481d8634e via relay 0.1.0 (agents: agent 0.1.0)1177» wave 01178agent/app | app: updated to generation 91179agent/app | flakelet-relay-job-app.service: Deactivated successfully.1180» agent/app: updated (generation 9)1181» ok118211831789041550 hist1 admin@example.org agent/other:unchanged11841789041541 c095c3c4a5d6ef24a325f9a8276721b0 admin@example.org agent/app:updated11851789041532 cb79325055f86f578ad9b8b18e5dbabf admin@example.org agent/app:updated11861789041515 6f269babd0aead46e35a8a54144d743c admin@example.org agent/app:updated11871789041513 local-app-6-1789041513 root (shell) agent/app:updated11881789041509 local-app-5-1789041509 root (shell) agent/app:updated11891789041506 ee6d03be48f5ecb53da41abc7ae80ae9 admin@example.org agent/other:unchanged11901789041495 2771421bb7d1dbd496ebeedfa1c14037 admin@example.org agent/app:updated11911789041495 95eee09327942bc6b464ee41dda01e3a admin@example.org agent/app:unchanged11921789041494 0ecb982896a198beb747ca6b081b9a42 admin@example.org agent/app:unchanged11931789041493 2ccaf530a175a0796e0188abaa406eff admin@example.org agent/app:unchanged11941789041489 9f21d698dd2aa16e77c0b8294f164846 admin@example.org agent/app:rolled-back11951789041488 955f2eb43188d9f24d0aeadb440d49a5 dev@example.org agent/other:unchanged11961789041487 c69338f02482cf208ff5e15b7b346b22 dev@example.org agent/other:unchanged11971789041484 399192e2a688385ad197dcda43ebbf1c someone agent/other:unchanged11981789041481 0b090f0debec0d8e08b4c6a24cfaf136 repo:github:example/app:ref:refs/heads/main agent/app:updated11991789041478 8953dcf5a5d08f9c6bf7ff31027dd8ee admin@example.org agent/app:updated12001789041475 local-app-1-1789041475 host activation agent/app:updated12011789041475 local-flakelet-agent-1-1789041475 host activation agent/flakelet-agent:updated12021789041475 local-other-1-1789041475 host activation agent/other:updated12031204Container relay terminated by signal KILL.