vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #553
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1: QEMU running (pid 50)13worker1 # Disk image does not exist, creating the virtualisation disk image...14client # Disk image does not exist, creating the virtualisation disk image...15lb # Disk image does not exist, creating the virtualisation disk image...16worker2: QEMU running (pid 52)17worker1 # Formatting '/build/vm-state-worker1/tmp.XwgTyYEsFS', fmt=raw size=107374182418client # Formatting '/build/vm-state-client/tmp.AVUj4YRNCL', fmt=raw size=107374182419lb # Formatting '/build/vm-state-lb/tmp.aJHrpj7Z0U', fmt=raw size=107374182420worker2 # Disk image does not exist, creating the virtualisation disk image...21lb: QEMU running (pid 51)22client: QEMU running (pid 49)23worker1 # mke2fs 1.47.4 (6-Mar-2025)24client # mke2fs 1.47.4 (6-Mar-2025)25worker2 # Formatting '/build/vm-state-worker2/tmp.1rPIU1kSIt', fmt=raw size=107374182426lb # mke2fs 1.47.4 (6-Mar-2025)27worker1 # Discarding device blocks: 0/262144 done28(finished: start all VMs, in 0.49 seconds)29lb # Discarding device blocks: 0/262144 done30worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes31client: waiting for unit network-addresses-eth1.service32lb # Creating filesystem with 262144 4k blocks and 65536 inodes33worker1 # Filesystem UUID: 3a4ec32c-988f-408a-b03b-2d3a9a3ac7cf34client: waiting for the VM to finish booting35lb # Filesystem UUID: 9b06c72b-8ef5-4a99-9d75-485fbf91b70336worker1 # Superblock backups stored on blocks:37worker2 # mke2fs 1.47.4 (6-Mar-2025)38client # Discarding device blocks: 0/262144 done39lb # Superblock backups stored on blocks:40worker1 # 32768, 98304, 163840, 22937641worker2 # Discarding device blocks: 0/262144 done42client # Creating filesystem with 262144 4k blocks and 65536 inodes43lb # 32768, 98304, 163840, 22937644worker1 # 45worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes46client # Filesystem UUID: 035d415d-5322-4590-8d64-eefee6bc2e1947lb # 48worker1 # Allocating group tables: 0/8 done49worker2 # Filesystem UUID: 800684ce-d338-428b-a2a7-3282b37aab5e50client # Superblock backups stored on blocks:51lb # Allocating group tables: 0/8 done52worker1 # Writing inode tables: 0/8 done53worker2 # Superblock backups stored on blocks:54client # 32768, 98304, 163840, 22937655lb # Writing inode tables: 0/8 done56worker1 # Creating journal (8192 blocks): done57worker2 # 32768, 98304, 163840, 22937658client # 59lb # Creating journal (8192 blocks): done60worker1 # Writing superblocks and filesystem accounting information: 0/8 done61worker2 # 62client # Allocating group tables: 0/8 done63lb # Writing superblocks and filesystem accounting information: 0/8 done64worker1 # 65worker2 # Allocating group tables: 0/8 done66client # Writing inode tables: 0/8 done67lb # 68worker1 # Virtualisation disk image created.69worker2 # Writing inode tables: 0/8 done70client # Creating journal (8192 blocks): done71lb # Virtualisation disk image created.72worker1 # Starting virtiofs daemons...73worker2 # Creating journal (8192 blocks): done74client # Writing superblocks and filesystem accounting information: 0/8 done75lb # Starting virtiofs daemons...76worker1 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)77worker2 # Writing superblocks and filesystem accounting information: 0/8 done78client # 79lb # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80worker1 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether81worker2 # 82client # Virtualisation disk image created.83lb # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84worker1 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker2 # Virtualisation disk image created.86client # Starting virtiofs daemons...87lb # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...88worker1 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # Starting virtiofs daemons...90client # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91lb # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)92worker1 # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker2 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94client # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95lb # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether96worker1 # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...97worker2 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether98client # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...99lb # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...100worker1 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)101worker2 # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...102client # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)103lb # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)104worker1 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether105worker2 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)106worker2 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether107lb # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether108client # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether109worker2 # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...110lb # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...111worker1 # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...112lb # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests113worker1 # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests114worker2 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)115client # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...116lb # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests117worker1 # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests118worker2 # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether119client # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests121worker1 # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests122worker2 # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...123client # [2026-09-24T12:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker2 # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests125client # [2026-09-24T12:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...126worker2 # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests127client # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests128worker2 # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-24T12:25:33Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133client # [ 0.000000] KASLR enabled134client # [ 0.000000] random: crng init done135client # [ 0.000000] Machine model: linux,dummy-virt136client # [ 0.000000] efi: UEFI not found.137client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140client # [ 0.000000] Zone ranges:141client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] DMA32 empty143client # [ 0.000000] Normal empty144client # [ 0.000000] Device empty145client # [ 0.000000] Movable zone start for each node146client # [ 0.000000] Early memory node ranges147client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]151client # [ 0.000000] psci: probing for conduit method from DT.152client # [ 0.000000] psci: PSCIv1.3 detected in firmware.153client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs154worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026155lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]156worker1 # [ 0.000000] KASLR enabled157client # [ 0.000000] psci: Trusted OS migration not required158worker1 # [ 0.000000] random: crng init done159client # [ 0.000000] psci: SMC Calling Convention v1.1160worker1 # [ 0.000000] Machine model: linux,dummy-virt161worker1 # [ 0.000000] efi: UEFI not found.162lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026163lb # [ 0.000000] KASLR enabled164client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)165lb # [ 0.000000] random: crng init done166worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT167client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296168lb # [ 0.000000] Machine model: linux,dummy-virt169lb # [ 0.000000] efi: UEFI not found.170client # [ 0.000000] Detected PIPT I-cache on CPU0171worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]172worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]173lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT174client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)175worker1 # [ 0.000000] Zone ranges:176client # [ 0.000000] CPU features: detected: GICv3 CPU interface177lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]178worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]179client # [ 0.000000] CPU features: detected: Spectre-v4180worker1 # [ 0.000000] DMA32 empty181lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]182worker1 # [ 0.000000] Normal empty183lb # [ 0.000000] Zone ranges:184client # [ 0.000000] CPU features: detected: Spectre-BHB185worker1 # [ 0.000000] Device empty186worker1 # [ 0.000000] Movable zone start for each node187client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38188lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]189worker1 # [ 0.000000] Early memory node ranges190lb # [ 0.000000] DMA32 empty191lb # [ 0.000000] Normal empty192client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23193lb # [ 0.000000] Device empty194worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]195client # [ 0.000000] alternatives: applying boot alternatives196lb # [ 0.000000] Movable zone start for each node197worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]198lb # [ 0.000000] Early memory node ranges199worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000200lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]201worker1 # [ 0.000000] psci: probing for conduit method from DT.202lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]203worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.204lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000205worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs206lb # [ 0.000000] psci: probing for conduit method from DT.207worker1 # [ 0.000000] psci: Trusted OS migration not required208client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/p452xpx22hq7f138nw4ylrz3gvzaggdz-nixos-system-client-test/init regInfo=/nix/store/h7i8sijidwyfk2ni1m8lf2c50jgxf2kd-closure-info/registration console=ttyAMA0,115200n8 console=tty0209lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.210worker1 # [ 0.000000] psci: SMC Calling Convention v1.1211lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs212lb # [ 0.000000] psci: Trusted OS migration not required213worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)214client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/h7i8sijidwyfk2ni1m8lf2c50jgxf2kd-closure-info/registration", will be passed to user space.215lb # [ 0.000000] psci: SMC Calling Convention v1.1216worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296217client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes218worker1 # [ 0.000000] Detected PIPT I-cache on CPU0219lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)220client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)221worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)222lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296223client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)224lb # [ 0.000000] Detected PIPT I-cache on CPU0225worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface226worker1 # [ 0.000000] CPU features: detected: Spectre-v4227client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB228lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)229worker1 # [ 0.000000] CPU features: detected: Spectre-BHB230client # [ 0.000000] software IO TLB: area num 1.231lb # [ 0.000000] CPU features: detected: GICv3 CPU interface232worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38233client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB234lb # [ 0.000000] CPU features: detected: Spectre-v4235lb # [ 0.000000] CPU features: detected: Spectre-BHB236worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23237client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)238worker1 # [ 0.000000] alternatives: applying boot alternatives239lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38240client # [ 0.000000] Fallback order for Node 0: 0241lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23242client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216243client # [ 0.000000] Policy zone: DMA244lb # [ 0.000000] alternatives: applying boot alternatives245client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off246client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1247client # [ 0.000000] allocated 3145728 bytes of page_ext248worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/489h0ln9x3if8fnpxg61abrp8jrcvvvn-nixos-system-worker1-test/init regInfo=/nix/store/sng1nm0awbs6sx972rwrh6bkhzwg3i4g-closure-info/registration console=ttyAMA0,115200n8 console=tty0249client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages250client # [ 0.000000] ftrace: allocated 294 pages with 4 groups251lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hmvppg1fs8xkiccvl0m50n4dxjz92fr-nixos-system-lb-test/init regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration console=ttyAMA0,115200n8 console=tty0252client # [ 0.000000] rcu: Hierarchical RCU implementation.253worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]254client # [ 0.000000] rcu: RCU event tracing is enabled.255worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/sng1nm0awbs6sx972rwrh6bkhzwg3i4g-closure-info/registration", will be passed to user space.256client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.257worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes258worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026259client # [ 0.000000] Trampoline variant of Tasks RCU enabled.260worker2 # [ 0.000000] KASLR enabled261lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration", will be passed to user space.262client # [ 0.000000] Rude variant of Tasks RCU enabled.263worker2 # [ 0.000000] random: crng init done264worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)265client # [ 0.000000] Tracing variant of Tasks RCU enabled.266lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes267worker2 # [ 0.000000] Machine model: linux,dummy-virt268worker2 # [ 0.000000] efi: UEFI not found.269worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)270client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.271lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)272worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT273worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB274client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1275worker1 # [ 0.000000] software IO TLB: area num 1.276lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)277worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]278worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB279lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB280client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.281worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]282worker2 # [ 0.000000] Zone ranges:283lb # [ 0.000000] software IO TLB: area num 1.284worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)285client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.286worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]287lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB288worker1 # [ 0.000000] Fallback order for Node 0: 0289worker2 # [ 0.000000] DMA32 empty290worker2 # [ 0.000000] Normal empty291client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.292worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216293lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)294worker2 # [ 0.000000] Device empty295worker1 # [ 0.000000] Policy zone: DMA296lb # [ 0.000000] Fallback order for Node 0: 0297client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0298worker2 # [ 0.000000] Movable zone start for each node299worker2 # [ 0.000000] Early memory node ranges300client # [ 0.000000] GICv3: 256 SPIs implemented301worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off302lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216303lb # [ 0.000000] Policy zone: DMA304client # [ 0.000000] GICv3: 0 Extended SPIs implemented305worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]306worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1307client # [ 0.000000] Root IRQ handler: gic_handle_irq308lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off309worker1 # [ 0.000000] allocated 3145728 bytes of page_ext310worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]311client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI312worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages313worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000314lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1315client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0316worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups317lb # [ 0.000000] allocated 3145728 bytes of page_ext318worker2 # [ 0.000000] psci: probing for conduit method from DT.319client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000320worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.321worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.322lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages323client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]324worker1 # [ 0.000000] rcu: RCU event tracing is enabled.325lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups326worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs327lb # [ 0.000000] rcu: Hierarchical RCU implementation.328worker2 # [ 0.000000] psci: Trusted OS migration not required329worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.330client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)331lb # [ 0.000000] rcu: RCU event tracing is enabled.332worker2 # [ 0.000000] psci: SMC Calling Convention v1.1333worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.334worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.335lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.336client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)337worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)338worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.339lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.340client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000341worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296342lb # [ 0.000000] Rude variant of Tasks RCU enabled.343worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.344worker2 # [ 0.000000] Detected PIPT I-cache on CPU0345client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000346lb # [ 0.000000] Tracing variant of Tasks RCU enabled.347worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1348client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.349lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.350worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)351worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.352worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface353lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1354client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns355worker2 # [ 0.000000] CPU features: detected: Spectre-v4356client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).357worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.358worker2 # [ 0.000000] CPU features: detected: Spectre-BHB359lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.360worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38361worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.362client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns363lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.364worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23365worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0366worker2 # [ 0.000000] alternatives: applying boot alternatives367client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns368worker1 # [ 0.000000] GICv3: 256 SPIs implemented369lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.370client # [ 0.000036] arm-pv: using stolen time PV371worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented372lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0373worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq374lb # [ 0.000000] GICv3: 256 SPIs implemented375client # [ 0.000668] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)376worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI377lb # [ 0.000000] GICv3: 0 Extended SPIs implemented378client # [ 0.000828] Console: colour dummy device 80x25379lb # [ 0.000000] Root IRQ handler: gic_handle_irq380worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0381client # [ 0.000837] printk: legacy console [tty0] enabled382lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI383worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000384lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0385worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]386worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ws7q0h3ljasrk1kpv4c15197d8wayb1b-nixos-system-worker2-test/init regInfo=/nix/store/kl46v6ljm9401myhdlw65m9qa6w0zvih-closure-info/registration console=ttyAMA0,115200n8 console=tty0387client # [ 0.001040] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)388lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000389client # [ 0.001047] pid_max: default: 32768 minimum: 301390worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)391lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]392client # [ 0.001122] LSM: initializing lsm=capability,landlock,yama,bpf,ima393worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kl46v6ljm9401myhdlw65m9qa6w0zvih-closure-info/registration", will be passed to user space.394client # [ 0.001298] landlock: Up and running.395lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)396worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)397client # [ 0.001301] Yama: becoming mindful.398worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes399client # [ 0.001969] LSM support for eBPF active400worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000401lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)402worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)403client # [ 0.002158] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)404worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000405lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000406worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.407worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)408client # [ 0.002191] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)409lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000410client # [ 0.004147] rcu: Hierarchical SRCU implementation.411lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.412worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns413client # [ 0.004153] rcu: Max phase no-delay instances is 1000.414worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB415client # [ 0.005412] fsl-mc MSI: its@8080000 domain created416worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).417worker2 # [ 0.000000] software IO TLB: area num 1.418lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns419client # [ 0.005501] EFI services will not be available.420worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB421client # [ 0.005590] smp: Bringing up secondary CPUs ...422lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).423worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns424client # [ 0.005598] smp: Brought up 1 node, 1 CPU425worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)426client # [ 0.005601] SMP: Total of 1 processors activated.427worker2 # [ 0.000000] Fallback order for Node 0: 0428worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns429lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns430client # [ 0.005608] CPU: All CPU(s) started at EL1431worker1 # [ 0.000032] arm-pv: using stolen time PV432worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216433lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns434client # [ 0.005618] CPU features: detected: Branch Target Identification435worker2 # [ 0.000000] Policy zone: DMA436lb # [ 0.000031] arm-pv: using stolen time PV437worker1 # [ 0.000457] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)438client # [ 0.005623] CPU features: detected: ARMv8.4 Translation Table Level439worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off440worker1 # [ 0.000633] Console: colour dummy device 80x25441lb # [ 0.000431] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)442worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1443worker1 # [ 0.000640] printk: legacy console [tty0] enabled444client # [ 0.005626] CPU features: detected: Instruction cache invalidation not required for I/D coherence445lb # [ 0.000598] Console: colour dummy device 80x25446worker2 # [ 0.000000] allocated 3145728 bytes of page_ext447lb # [ 0.000605] printk: legacy console [tty0] enabled448worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages449client # [ 0.005630] CPU features: detected: Data cache clean to the PoU not required for I/D coherence450worker1 # [ 0.000830] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)451worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups452worker1 # [ 0.000837] pid_max: default: 32768 minimum: 301453client # [ 0.005634] CPU features: detected: Common not Private translations454lb # [ 0.000793] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)455worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.456client # [ 0.005637] CPU features: detected: CRC32 instructions457worker1 # [ 0.000908] LSM: initializing lsm=capability,landlock,yama,bpf,ima458worker2 # [ 0.000000] rcu: RCU event tracing is enabled.459lb # [ 0.000800] pid_max: default: 32768 minimum: 301460worker1 # [ 0.001047] landlock: Up and running.461client # [ 0.005640] CPU features: detected: Data cache clean to Point of Deep Persistence462worker1 # [ 0.001049] Yama: becoming mindful.463lb # [ 0.000888] LSM: initializing lsm=capability,landlock,yama,bpf,ima464worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.465worker1 # [ 0.001527] LSM support for eBPF active466lb # [ 0.001023] landlock: Up and running.467client # [ 0.005643] CPU features: detected: Data cache clean to Point of Persistence468worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.469lb # [ 0.001026] Yama: becoming mindful.470worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.471worker1 # [ 0.001649] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)472lb # [ 0.001598] LSM support for eBPF active473client # [ 0.005646] CPU features: detected: Data independent timing control (DIT)474worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.475worker1 # [ 0.001686] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)476lb # [ 0.001735] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)477client # [ 0.005649] CPU features: detected: E0PD478worker1 # [ 0.003542] rcu: Hierarchical SRCU implementation.479worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.480lb # [ 0.001783] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)481worker1 # [ 0.003546] rcu: Max phase no-delay instances is 1000.482client # [ 0.005652] CPU features: detected: Enhanced Counter Virtualization483worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1484lb # [ 0.003625] rcu: Hierarchical SRCU implementation.485worker1 # [ 0.004828] fsl-mc MSI: its@8080000 domain created486lb # [ 0.003631] rcu: Max phase no-delay instances is 1000.487client # [ 0.005655] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)488worker1 # [ 0.004919] EFI services will not be available.489worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.490lb # [ 0.004917] fsl-mc MSI: its@8080000 domain created491worker1 # [ 0.004986] smp: Bringing up secondary CPUs ...492client # [ 0.005658] CPU features: detected: Enhanced Virtualization Traps493lb # [ 0.005016] EFI services will not be available.494worker1 # [ 0.004995] smp: Brought up 1 node, 1 CPU495worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.496client # [ 0.005661] CPU features: detected: Fine Grained Traps497lb # [ 0.005095] smp: Bringing up secondary CPUs ...498worker1 # [ 0.004998] SMP: Total of 1 processors activated.499lb # [ 0.005103] smp: Brought up 1 node, 1 CPU500worker1 # [ 0.005003] CPU: All CPU(s) started at EL1501client # [ 0.005665] CPU features: detected: Generic authentication (architected QARMA5 algorithm)502worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.503lb # [ 0.005105] SMP: Total of 1 processors activated.504worker1 # [ 0.005014] CPU features: detected: Branch Target Identification505worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0506client # [ 0.005669] CPU features: detected: RCpc load-acquire (LDAPR)507lb # [ 0.005111] CPU: All CPU(s) started at EL1508worker2 # [ 0.000000] GICv3: 256 SPIs implemented509client # [ 0.005672] CPU features: detected: LSE atomic instructions510worker1 # [ 0.005018] CPU features: detected: ARMv8.4 Translation Table Level511lb # [ 0.005120] CPU features: detected: Branch Target Identification512worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented513client # [ 0.005675] CPU features: detected: Privileged Access Never514worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq515lb # [ 0.005125] CPU features: detected: ARMv8.4 Translation Table Level516client # [ 0.005678] CPU features: detected: PMUv3517worker1 # [ 0.005021] CPU features: detected: Instruction cache invalidation not required for I/D coherence518worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI519client # [ 0.005681] CPU features: detected: RAS Extension Support520worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0521lb # [ 0.005128] CPU features: detected: Instruction cache invalidation not required for I/D coherence522worker1 # [ 0.005025] CPU features: detected: Data cache clean to the PoU not required for I/D coherence523client # [ 0.005684] CPU features: detected: RASv1p1 Extension Support524worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000525worker1 # [ 0.005029] CPU features: detected: Common not Private translations526lb # [ 0.005132] CPU features: detected: Data cache clean to the PoU not required for I/D coherence527client # [ 0.005686] CPU features: detected: Random Number Generator528worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]529worker1 # [ 0.005032] CPU features: detected: CRC32 instructions530lb # [ 0.005136] CPU features: detected: Common not Private translations531client # [ 0.005689] CPU features: detected: Speculation barrier (SB)532worker1 # [ 0.005035] CPU features: detected: Data cache clean to Point of Deep Persistence533lb # [ 0.005139] CPU features: detected: CRC32 instructions534client # [ 0.005692] CPU features: detected: Stage-2 Force Write-Back535worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)536worker1 # [ 0.005038] CPU features: detected: Data cache clean to Point of Persistence537client # [ 0.005695] CPU features: detected: TLB range maintenance instructions538lb # [ 0.005142] CPU features: detected: Data cache clean to Point of Deep Persistence539worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)540client # [ 0.005700] CPU features: detected: Speculative Store Bypassing Safe (SSBS)541worker1 # [ 0.005041] CPU features: detected: Data independent timing control (DIT)542lb # [ 0.005145] CPU features: detected: Data cache clean to Point of Persistence543worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000544worker1 # [ 0.005045] CPU features: detected: E0PD545client # [ 0.005738] alternatives: applying system-wide alternatives546lb # [ 0.005148] CPU features: detected: Data independent timing control (DIT)547worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000548worker1 # [ 0.005047] CPU features: detected: Enhanced Counter Virtualization549client # [ 0.008750] CPU features: detected: BBM Level 2 without TLB conflict abort550lb # [ 0.005151] CPU features: detected: E0PD551worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.552worker1 # [ 0.005050] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)553lb # [ 0.005153] CPU features: detected: Enhanced Counter Virtualization554worker1 # [ 0.005054] CPU features: detected: Enhanced Virtualization Traps555client # [ 0.008949] Memory: 1405668K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)556lb # [ 0.005156] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)557worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns558worker1 # [ 0.005057] CPU features: detected: Fine Grained Traps559client # [ 0.009359] devtmpfs: initialized560lb # [ 0.005160] CPU features: detected: Enhanced Virtualization Traps561worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).562worker1 # [ 0.005061] CPU features: detected: Generic authentication (architected QARMA5 algorithm)563client # [ 0.011200] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)564lb # [ 0.005163] CPU features: detected: Fine Grained Traps565worker1 # [ 0.005067] CPU features: detected: RCpc load-acquire (LDAPR)566worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns567lb # [ 0.005166] CPU features: detected: Generic authentication (architected QARMA5 algorithm)568client # [ 0.011221] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).569worker1 # [ 0.005070] CPU features: detected: LSE atomic instructions570lb # [ 0.005170] CPU features: detected: RCpc load-acquire (LDAPR)571worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns572client # [ 0.011413] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL573worker1 # [ 0.005073] CPU features: detected: Privileged Access Never574worker2 # [ 0.000029] arm-pv: using stolen time PV575client # [ 0.011417] 0 pages in range for non-PLT usage576lb # [ 0.005173] CPU features: detected: LSE atomic instructions577worker1 # [ 0.005075] CPU features: detected: PMUv3578client # [ 0.011418] 508272 pages in range for PLT usage579lb # [ 0.005176] CPU features: detected: Privileged Access Never580worker1 # [ 0.005078] CPU features: detected: RAS Extension Support581client # [ 0.011558] pinctrl core: initialized pinctrl subsystem582worker2 # [ 0.000389] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)583lb # [ 0.005178] CPU features: detected: PMUv3584worker1 # [ 0.005081] CPU features: detected: RASv1p1 Extension Support585client # [ 0.012325] DMI not present or invalid.586worker2 # [ 0.000547] Console: colour dummy device 80x25587lb # [ 0.005181] CPU features: detected: RAS Extension Support588worker1 # [ 0.005084] CPU features: detected: Random Number Generator589worker2 # [ 0.000555] printk: legacy console [tty0] enabled590client # [ 0.015508] NET: Registered PF_NETLINK/PF_ROUTE protocol family591lb # [ 0.005184] CPU features: detected: RASv1p1 Extension Support592worker1 # [ 0.005086] CPU features: detected: Speculation barrier (SB)593client # [ 0.017901] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations594lb # [ 0.005186] CPU features: detected: Random Number Generator595worker2 # [ 0.000767] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)596worker1 # [ 0.005089] CPU features: detected: Stage-2 Force Write-Back597lb # [ 0.005189] CPU features: detected: Speculation barrier (SB)598client # [ 0.018095] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations599worker2 # [ 0.000774] pid_max: default: 32768 minimum: 301600worker1 # [ 0.005092] CPU features: detected: TLB range maintenance instructions601lb # [ 0.005191] CPU features: detected: Stage-2 Force Write-Back602worker2 # [ 0.000854] LSM: initializing lsm=capability,landlock,yama,bpf,ima603client # [ 0.018296] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations604worker2 # [ 0.000984] landlock: Up and running.605worker1 # [ 0.005097] CPU features: detected: Speculative Store Bypassing Safe (SSBS)606lb # [ 0.005194] CPU features: detected: TLB range maintenance instructions607client # [ 0.018319] audit: initializing netlink subsys (disabled)608worker2 # [ 0.000987] Yama: becoming mindful.609worker1 # [ 0.005135] alternatives: applying system-wide alternatives610worker2 # [ 0.001433] LSM support for eBPF active611lb # [ 0.005198] CPU features: detected: Speculative Store Bypassing Safe (SSBS)612client # [ 0.018906] thermal_sys: Registered thermal governor 'fair_share'613worker1 # [ 0.008175] CPU features: detected: BBM Level 2 without TLB conflict abort614lb # [ 0.005235] alternatives: applying system-wide alternatives615worker2 # [ 0.001572] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)616client # [ 0.018908] thermal_sys: Registered thermal governor 'bang_bang'617client # [ 0.018912] thermal_sys: Registered thermal governor 'step_wise'618lb # [ 0.008264] CPU features: detected: BBM Level 2 without TLB conflict abort619worker2 # [ 0.001611] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)620worker2 # [ 0.003438] rcu: Hierarchical SRCU implementation.621worker1 # [ 0.008437] Memory: 1405632K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)622client # [ 0.018915] thermal_sys: Registered thermal governor 'user_space'623worker1 # [ 0.008864] devtmpfs: initialized624worker2 # [ 0.003443] rcu: Max phase no-delay instances is 1000.625client # [ 0.018920] thermal_sys: Registered thermal governor 'power_allocator'626lb # [ 0.008511] Memory: 1405692K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)627worker2 # [ 0.004716] fsl-mc MSI: its@8080000 domain created628worker1 # [ 0.010726] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)629lb # [ 0.008884] devtmpfs: initialized630client # [ 0.018945] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1631worker2 # [ 0.004801] EFI services will not be available.632client # [ 0.018954] cpuidle: using governor ladder633lb # [ 0.010770] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)634worker1 # [ 0.010749] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).635worker2 # [ 0.004905] smp: Bringing up secondary CPUs ...636client # [ 0.018960] cpuidle: using governor menu637worker2 # [ 0.004914] smp: Brought up 1 node, 1 CPU638worker1 # [ 0.010926] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL639lb # [ 0.010793] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).640worker2 # [ 0.004917] SMP: Total of 1 processors activated.641client # [ 0.019147] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.642worker2 # [ 0.004922] CPU: All CPU(s) started at EL1643lb # [ 0.010992] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL644client # [ 0.019163] ASID allocator initialised with 65536 entries645worker2 # [ 0.004932] CPU features: detected: Branch Target Identification646client # [ 0.020430] Serial: AMBA PL011 UART driver647worker2 # [ 0.004937] CPU features: detected: ARMv8.4 Translation Table Level648client # [ 0.025828] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1649client # [ 0.025958] printk: console [ttyAMA0] enabled650worker2 # [ 0.004940] CPU features: detected: Instruction cache invalidation not required for I/D coherence651worker2 # [ 0.004944] CPU features: detected: Data cache clean to the PoU not required for I/D coherence652worker2 # [ 0.004948] CPU features: detected: Common not Private translations653worker2 # [ 0.004951] CPU features: detected: CRC32 instructions654worker2 # [ 0.004954] CPU features: detected: Data cache clean to Point of Deep Persistence655worker2 # [ 0.004957] CPU features: detected: Data cache clean to Point of Persistence656worker2 # [ 0.004960] CPU features: detected: Data independent timing control (DIT)657worker1 # [ 0.010930] 0 pages in range for non-PLT usage658worker2 # [ 0.004963] CPU features: detected: E0PD659worker1 # [ 0.010932] 508272 pages in range for PLT usage660worker2 # [ 0.004966] CPU features: detected: Enhanced Counter Virtualization661lb # [ 0.010997] 0 pages in range for non-PLT usage662worker1 # [ 0.011049] pinctrl core: initialized pinctrl subsystem663lb # [ 0.010998] 508272 pages in range for PLT usage664worker1 # [ 0.011913] DMI not present or invalid.665worker2 # [ 0.004969] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)666client # [ 0.143174] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages667lb # [ 0.011107] pinctrl core: initialized pinctrl subsystem668worker1 # [ 0.015179] NET: Registered PF_NETLINK/PF_ROUTE protocol family669worker2 # [ 0.004972] CPU features: detected: Enhanced Virtualization Traps670lb # [ 0.011813] DMI not present or invalid.671client # [ 0.143193] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page672worker2 # [ 0.004975] CPU features: detected: Fine Grained Traps673worker1 # [ 0.017381] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations674lb # [ 0.015057] NET: Registered PF_NETLINK/PF_ROUTE protocol family675client # [ 0.143199] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages676lb # [ 0.017270] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations677worker1 # [ 0.017557] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations678worker2 # [ 0.004978] CPU features: detected: Generic authentication (architected QARMA5 algorithm)679client # [ 0.143203] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page680worker2 # [ 0.004982] CPU features: detected: RCpc load-acquire (LDAPR)681lb # [ 0.017446] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations682worker1 # [ 0.017764] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations683client # [ 0.143208] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages684worker2 # [ 0.004985] CPU features: detected: LSE atomic instructions685worker1 # [ 0.017785] audit: initializing netlink subsys (disabled)686client # [ 0.143212] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page687lb # [ 0.017649] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations688worker2 # [ 0.004988] CPU features: detected: Privileged Access Never689worker1 # [ 0.018324] thermal_sys: Registered thermal governor 'fair_share'690client # [ 0.143216] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages691lb # [ 0.017669] audit: initializing netlink subsys (disabled)692worker2 # [ 0.004991] CPU features: detected: PMUv3693worker1 # [ 0.018326] thermal_sys: Registered thermal governor 'bang_bang'694lb # [ 0.018291] thermal_sys: Registered thermal governor 'fair_share'695worker2 # [ 0.004993] CPU features: detected: RAS Extension Support696client # [ 0.143221] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page697worker1 # [ 0.018330] thermal_sys: Registered thermal governor 'step_wise'698worker2 # [ 0.004996] CPU features: detected: RASv1p1 Extension Support699lb # [ 0.018293] thermal_sys: Registered thermal governor 'bang_bang'700worker1 # [ 0.018333] thermal_sys: Registered thermal governor 'user_space'701lb # [ 0.018297] thermal_sys: Registered thermal governor 'step_wise'702worker2 # [ 0.004999] CPU features: detected: Random Number Generator703worker2 # [ 0.005001] CPU features: detected: Speculation barrier (SB)704lb # [ 0.018300] thermal_sys: Registered thermal governor 'user_space'705worker1 # [ 0.018338] thermal_sys: Registered thermal governor 'power_allocator'706client # [ 0.150441] fbcon: Taking over console707worker2 # [ 0.005004] CPU features: detected: Stage-2 Force Write-Back708client # [ 0.150458] ACPI: Interpreter disabled.709lb # [ 0.018305] thermal_sys: Registered thermal governor 'power_allocator'710worker1 # [ 0.018375] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1711worker2 # [ 0.005006] CPU features: detected: TLB range maintenance instructions712worker1 # [ 0.018384] cpuidle: using governor ladder713lb # [ 0.018329] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1714worker1 # [ 0.018389] cpuidle: using governor menu715lb # [ 0.018338] cpuidle: using governor ladder716worker2 # [ 0.005010] CPU features: detected: Speculative Store Bypassing Safe (SSBS)717lb # [ 0.018344] cpuidle: using governor menu718worker1 # [ 0.018584] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.719worker2 # [ 0.005047] alternatives: applying system-wide alternatives720worker1 # [ 0.018601] ASID allocator initialised with 65536 entries721lb # [ 0.018524] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.722worker1 # [ 0.019776] Serial: AMBA PL011 UART driver723worker2 # [ 0.007975] CPU features: detected: BBM Level 2 without TLB conflict abort724lb # [ 0.018539] ASID allocator initialised with 65536 entries725lb # [ 0.019737] Serial: AMBA PL011 UART driver726worker1 # [ 0.024988] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1727worker1 # [ 0.025110] printk: console [ttyAMA0] enabled728worker2 # [ 0.008193] Memory: 1405684K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)729lb # [ 0.025052] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1730worker2 # [ 0.008595] devtmpfs: initialized731lb # [ 0.025211] printk: console [ttyAMA0] enabled732worker1 # [ 0.145041] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages733worker2 # [ 0.010416] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)734lb # [ 0.144143] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages735worker1 # [ 0.145062] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page736worker2 # [ 0.010438] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).737lb # [ 0.144163] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page738worker1 # [ 0.145068] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages739worker2 # [ 0.010633] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL740lb # [ 0.144168] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages741worker1 # [ 0.145072] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page742client # [ 0.159492] iommu: Default domain type: Translated743worker2 # [ 0.010637] 0 pages in range for non-PLT usage744worker2 # [ 0.010639] 508272 pages in range for PLT usage745client # [ 0.159503] iommu: DMA domain TLB invalidation policy: strict mode746lb # [ 0.144173] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page747worker1 # [ 0.145076] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages748client # [ 0.159842] SCSI subsystem initialized749worker2 # [ 0.010744] pinctrl core: initialized pinctrl subsystem750lb # [ 0.144177] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages751worker1 # [ 0.145081] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page752worker2 # [ 0.011468] DMI not present or invalid.753lb # [ 0.144181] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page754worker1 # [ 0.145085] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages755worker2 # [ 0.014587] NET: Registered PF_NETLINK/PF_ROUTE protocol family756client # [ 0.161770] usbcore: registered new interface driver usbfs757lb # [ 0.144186] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages758worker1 # [ 0.145089] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page759client # [ 0.161810] usbcore: registered new interface driver hub760lb # [ 0.144190] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page761client # [ 0.161826] usbcore: registered new device driver usb762client # [ 0.162096] pps_core: LinuxPPS API ver. 1 registered763client # [ 0.162103] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>764worker1 # [ 0.152480] fbcon: Taking over console765client # [ 0.162112] PTP clock support registered766worker1 # [ 0.152495] ACPI: Interpreter disabled.767lb # [ 0.151543] fbcon: Taking over console768client # [ 0.162182] EDAC MC: Ver: 3.0.0769lb # [ 0.151558] ACPI: Interpreter disabled.770client # [ 0.166759] scmi_core: SCMI protocol bus registered771lb # [ 0.153414] iommu: Default domain type: Translated772lb # [ 0.153424] iommu: DMA domain TLB invalidation policy: strict mode773client # [ 0.167783] FPGA manager framework774client # [ 0.168742] vgaarb: loaded775lb # [ 0.155131] SCSI subsystem initialized776client # [ 0.169387] clocksource: Switched to clocksource arch_sys_counter777worker2 # [ 0.016768] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations778worker1 # [ 0.159475] iommu: Default domain type: Translated779worker2 # [ 0.016942] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations780worker1 # [ 0.159483] iommu: DMA domain TLB invalidation policy: strict mode781worker2 # [ 0.017138] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations782client # [ 0.173028] VFS: Disk quotas dquot_6.6.0783worker1 # [ 0.161191] SCSI subsystem initialized784worker2 # [ 0.017161] audit: initializing netlink subsys (disabled)785client # [ 0.173064] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)786worker2 # [ 0.017710] thermal_sys: Registered thermal governor 'fair_share'787worker1 # [ 0.161892] usbcore: registered new interface driver usbfs788worker2 # [ 0.017712] thermal_sys: Registered thermal governor 'bang_bang'789worker1 # [ 0.161923] usbcore: registered new interface driver hub790worker2 # [ 0.017716] thermal_sys: Registered thermal governor 'step_wise'791worker1 # [ 0.161938] usbcore: registered new device driver usb792worker1 # [ 0.162208] pps_core: LinuxPPS API ver. 1 registered793worker2 # [ 0.017718] thermal_sys: Registered thermal governor 'user_space'794client # [ 0.176815] netfs: FS-Cache loaded795lb # [ 0.162796] usbcore: registered new interface driver usbfs796worker2 # [ 0.017724] thermal_sys: Registered thermal governor 'power_allocator'797client # [ 0.176958] pnp: PnP ACPI: disabled798worker1 # [ 0.162214] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>799lb # [ 0.162834] usbcore: registered new interface driver hub800worker1 # [ 0.162224] PTP clock support registered801worker2 # [ 0.017748] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1802lb # [ 0.162860] usbcore: registered new device driver usb803worker1 # [ 0.162278] EDAC MC: Ver: 3.0.0804worker2 # [ 0.017756] cpuidle: using governor ladder805lb # [ 0.163163] pps_core: LinuxPPS API ver. 1 registered806worker2 # [ 0.017761] cpuidle: using governor menu807worker1 # [ 0.166903] scmi_core: SCMI protocol bus registered808worker2 # [ 0.017957] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.809lb # [ 0.163170] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>810worker1 # [ 0.167885] FPGA manager framework811lb # [ 0.163180] PTP clock support registered812worker2 # [ 0.017972] ASID allocator initialised with 65536 entries813worker1 # [ 0.168846] vgaarb: loaded814lb # [ 0.163220] EDAC MC: Ver: 3.0.0815worker2 # [ 0.019089] Serial: AMBA PL011 UART driver816client # [ 0.180846] NET: Registered PF_INET protocol family817lb # [ 0.167804] scmi_core: SCMI protocol bus registered818worker1 # [ 0.169486] clocksource: Switched to clocksource arch_sys_counter819worker2 # [ 0.024265] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1820client # [ 0.181115] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)821worker2 # [ 0.024418] printk: console [ttyAMA0] enabled822lb # [ 0.168745] FPGA manager framework823lb # [ 0.169735] vgaarb: loaded824worker2 # [ 0.144610] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages825worker2 # [ 0.144630] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page826lb # [ 0.170379] clocksource: Switched to clocksource arch_sys_counter827lb # [ 0.170981] VFS: Disk quotas dquot_6.6.0828worker2 # [ 0.144636] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages829worker2 # [ 0.144640] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page830lb # [ 0.171008] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)831worker2 # [ 0.144645] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages832lb # [ 0.173395] netfs: FS-Cache loaded833worker1 # [ 0.173127] VFS: Disk quotas dquot_6.6.0834lb # [ 0.173514] pnp: PnP ACPI: disabled835worker2 # [ 0.144649] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page836worker1 # [ 0.173163] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)837worker2 # [ 0.144654] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages838worker1 # [ 0.176773] netfs: FS-Cache loaded839worker1 # [ 0.176876] pnp: PnP ACPI: disabled840worker2 # [ 0.144658] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page841worker2 # [ 0.152104] fbcon: Taking over console842worker2 # [ 0.152118] ACPI: Interpreter disabled.843worker2 # [ 0.153997] iommu: Default domain type: Translated844worker1 # [ 0.180707] NET: Registered PF_INET protocol family845worker2 # [ 0.154006] iommu: DMA domain TLB invalidation policy: strict mode846worker1 # [ 0.180964] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)847worker2 # [ 0.155678] SCSI subsystem initialized848lb # [ 0.179602] NET: Registered PF_INET protocol family849lb # [ 0.179862] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)850worker2 # [ 0.161032] usbcore: registered new interface driver usbfs851worker2 # [ 0.161063] usbcore: registered new interface driver hub852worker2 # [ 0.161078] usbcore: registered new device driver usb853worker2 # [ 0.161342] pps_core: LinuxPPS API ver. 1 registered854worker2 # [ 0.161348] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>855worker2 # [ 0.161357] PTP clock support registered856worker2 # [ 0.161405] EDAC MC: Ver: 3.0.0857worker2 # [ 0.166011] scmi_core: SCMI protocol bus registered858worker2 # [ 0.166941] FPGA manager framework859worker2 # [ 0.167907] vgaarb: loaded860worker2 # [ 0.168546] clocksource: Switched to clocksource arch_sys_counter861worker2 # [ 0.172352] VFS: Disk quotas dquot_6.6.0862worker2 # [ 0.172382] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)863worker2 # [ 0.176082] netfs: FS-Cache loaded864worker2 # [ 0.176204] pnp: PnP ACPI: disabled865client # [ 0.210363] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)866client # [ 0.210414] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)867worker2 # [ 0.179989] NET: Registered PF_INET protocol family868client # [ 0.210443] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)869worker2 # [ 0.180247] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)870client # [ 0.210513] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)871client # [ 0.210657] TCP: Hash tables configured (established 16384 bind 16384)872client # [ 0.210744] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)873client # [ 0.210782] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)874client # [ 0.210818] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)875client # [ 0.210912] NET: Registered PF_UNIX/PF_LOCAL protocol family876client # [ 0.210932] NET: Registered PF_XDP protocol family877client # [ 0.210950] PCI: CLS 0 bytes, default 64878client # [ 0.211210] Trying to unpack rootfs image as initramfs...879worker1 # [ 0.210595] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)880worker1 # [ 0.210644] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)881worker1 # [ 0.210671] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)882lb # [ 0.209261] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)883worker1 # [ 0.210752] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)884client # [ 0.227081] kvm [1]: HYP mode not available885lb # [ 0.209306] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)886worker1 # [ 0.210895] TCP: Hash tables configured (established 16384 bind 16384)887lb # [ 0.209333] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)888worker1 # [ 0.210988] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)889worker1 # [ 0.211025] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)890lb # [ 0.209410] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)891lb # [ 0.209554] TCP: Hash tables configured (established 16384 bind 16384)892worker1 # [ 0.211061] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)893worker1 # [ 0.211178] NET: Registered PF_UNIX/PF_LOCAL protocol family894lb # [ 0.209643] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)895worker1 # [ 0.211198] NET: Registered PF_XDP protocol family896lb # [ 0.209679] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)897worker1 # [ 0.211216] PCI: CLS 0 bytes, default 64898worker1 # [ 0.211481] Trying to unpack rootfs image as initramfs...899lb # [ 0.209714] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)900lb # [ 0.209820] NET: Registered PF_UNIX/PF_LOCAL protocol family901lb # [ 0.209846] NET: Registered PF_XDP protocol family902lb # [ 0.209864] PCI: CLS 0 bytes, default 64903lb # [ 0.210102] Trying to unpack rootfs image as initramfs...904worker1 # [ 0.227338] kvm [1]: HYP mode not available905lb # [ 0.228249] kvm [1]: HYP mode not available906worker2 # [ 0.210769] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)907worker2 # [ 0.210826] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)908worker2 # [ 0.210857] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909worker2 # [ 0.210929] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910worker2 # [ 0.211072] TCP: Hash tables configured (established 16384 bind 16384)911worker2 # [ 0.211160] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912worker2 # [ 0.211209] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913worker2 # [ 0.211246] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914worker2 # [ 0.211338] NET: Registered PF_UNIX/PF_LOCAL protocol family915worker2 # [ 0.211358] NET: Registered PF_XDP protocol family916worker2 # [ 0.211376] PCI: CLS 0 bytes, default 64917worker2 # [ 0.211632] Trying to unpack rootfs image as initramfs...918worker2 # [ 0.226365] kvm [1]: HYP mode not available919client # [ 0.375049] Initialise system trusted keyrings920lb # [ 0.362356] Initialise system trusted keyrings921lb # [ 0.364877] workingset: timestamp_bits=42 max_order=19 bucket_order=0922client # [ 0.385454] workingset: timestamp_bits=42 max_order=19 bucket_order=0923client # [ 0.386886] squashfs: version 4.0 (2009/01/31) Phillip Lougher924lb # [ 0.374920] squashfs: version 4.0 (2009/01/31) Phillip Lougher925client # [ 0.387676] 9p: Installing v9fs 9p2000 file system support926worker1 # [ 0.378051] Initialise system trusted keyrings927worker1 # [ 0.378821] workingset: timestamp_bits=42 max_order=19 bucket_order=0928worker1 # [ 0.380110] squashfs: version 4.0 (2009/01/31) Phillip Lougher929worker1 # [ 0.380900] 9p: Installing v9fs 9p2000 file system support930lb # [ 0.375755] 9p: Installing v9fs 9p2000 file system support931client # [ 0.408341] Key type asymmetric registered932client # [ 0.408367] Asymmetric key parser 'x509' registered933lb # [ 0.395726] Key type asymmetric registered934client # [ 0.408440] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)935lb # [ 0.395754] Asymmetric key parser 'x509' registered936lb # [ 0.395832] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)937lb # [ 0.397965] io scheduler mq-deadline registered938lb # [ 0.397977] io scheduler kyber registered939client # [ 0.417468] io scheduler mq-deadline registered940client # [ 0.417492] io scheduler kyber registered941worker2 # [ 0.381149] Initialise system trusted keyrings942worker2 # [ 0.381915] workingset: timestamp_bits=42 max_order=19 bucket_order=0943worker1 # [ 0.409583] Key type asymmetric registered944worker2 # [ 0.383172] squashfs: version 4.0 (2009/01/31) Phillip Lougher945worker1 # [ 0.409614] Asymmetric key parser 'x509' registered946client # [ 0.422573] pl061_gpio 9030000.pl061: PL061 GPIO chip registered947worker1 # [ 0.409685] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)948worker2 # [ 0.383955] 9p: Installing v9fs 9p2000 file system support949worker1 # [ 0.411854] io scheduler mq-deadline registered950worker1 # [ 0.411866] io scheduler kyber registered951client # [ 0.423868] ledtrig-cpu: registered to indicate activity on CPUs952lb # [ 0.411305] pl061_gpio 9030000.pl061: PL061 GPIO chip registered953client # [ 0.424249] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:954client # [ 0.424267] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000955lb # [ 0.412625] ledtrig-cpu: registered to indicate activity on CPUs956client # [ 0.424288] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000957lb # [ 0.413040] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:958client # [ 0.424297] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000959lb # [ 0.413057] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000960client # [ 0.424318] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits961lb # [ 0.413069] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000962client # [ 0.424341] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]963lb # [ 0.413077] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000964client # [ 0.424414] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00965lb # [ 0.413097] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits966client # [ 0.424424] pci_bus 0000:00: root bus resource [bus 00-ff]967client # [ 0.424430] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]968lb # [ 0.413118] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]969worker1 # [ 0.421635] pl061_gpio 9030000.pl061: PL061 GPIO chip registered970client # [ 0.424435] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]971lb # [ 0.413188] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00972client # [ 0.424440] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]973lb # [ 0.413197] pci_bus 0000:00: root bus resource [bus 00-ff]974lb # [ 0.413203] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]975client # [ 0.424496] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint976lb # [ 0.413208] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]977worker1 # [ 0.423049] ledtrig-cpu: registered to indicate activity on CPUs978client # [ 0.424935] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint979lb # [ 0.413213] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]980client # [ 0.425120] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]981worker1 # [ 0.423470] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:982client # [ 0.425137] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]983lb # [ 0.413270] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint984worker1 # [ 0.423487] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000985client # [ 0.425167] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]986lb # [ 0.413704] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint987client # [ 0.425183] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]988worker1 # [ 0.423500] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000989lb # [ 0.413908] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]990lb # [ 0.413924] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]991worker1 # [ 0.423508] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000992lb # [ 0.413954] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]993worker1 # [ 0.423531] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits994lb # [ 0.413971] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]995worker1 # [ 0.423558] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]996lb # [ 0.430193] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint997worker1 # [ 0.423639] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00998worker1 # [ 0.423649] pci_bus 0000:00: root bus resource [bus 00-ff]999worker1 # [ 0.423655] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1000worker1 # [ 0.423660] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1001worker1 # [ 0.423665] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1002lb # [ 0.431369] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1003lb # [ 0.431386] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1004worker1 # [ 0.423723] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1005lb # [ 0.431416] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006client # [ 0.445737] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1007worker1 # [ 0.424177] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1008client # [ 0.445950] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1009lb # [ 0.431876] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1010worker1 # [ 0.424366] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1011client # [ 0.445968] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1012lb # [ 0.432063] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1013worker1 # [ 0.424383] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1014worker2 # [ 0.412666] Key type asymmetric registered1015lb # [ 0.432078] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1016client # [ 0.445998] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1017worker1 # [ 0.424412] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1018worker2 # [ 0.412694] Asymmetric key parser 'x509' registered1019lb # [ 0.432108] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1020worker1 # [ 0.424428] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1021client # [ 0.446471] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1022worker2 # [ 0.412771] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1023client # [ 0.446654] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1024lb # [ 0.432575] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1025worker2 # [ 0.414829] io scheduler mq-deadline registered1026worker1 # [ 0.424880] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1027client # [ 0.446671] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1028lb # [ 0.432758] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1029worker2 # [ 0.414840] io scheduler kyber registered1030worker1 # [ 0.425065] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1031lb # [ 0.432774] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1032client # [ 0.446701] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033worker1 # [ 0.425080] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1034lb # [ 0.432803] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035worker1 # [ 0.425109] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1036client # [ 0.447149] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1037lb # [ 0.432821] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1038client # [ 0.447334] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1039client # [ 0.447351] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1040lb # [ 0.433275] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1041client # [ 0.447381] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042lb # [ 0.433461] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1043client # [ 0.447400] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1044lb # [ 0.433491] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1045client # [ 0.447870] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1046lb # [ 0.433946] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1047client # [ 0.448056] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1048lb # [ 0.434134] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1049client # [ 0.448087] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1050lb # [ 0.434163] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1051worker1 # [ 0.444479] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1052client # [ 0.448537] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1053worker1 # [ 0.444670] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1054client # [ 0.448724] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1055worker1 # [ 0.444685] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1056client # [ 0.448755] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057worker1 # [ 0.444715] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1058worker2 # [ 0.424752] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1059client # [ 0.449155] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1060worker1 # [ 0.445174] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1061client # [ 0.449337] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1062worker1 # [ 0.445362] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1063worker1 # [ 0.445378] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1064worker1 # [ 0.445407] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065worker2 # [ 0.426153] ledtrig-cpu: registered to indicate activity on CPUs1066worker1 # [ 0.445426] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1067worker2 # [ 0.426541] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1068worker1 # [ 0.445892] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1069worker2 # [ 0.426558] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001070worker1 # [ 0.446080] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1071worker1 # [ 0.446110] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1072worker2 # [ 0.426571] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001073worker1 # [ 0.446567] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1074worker2 # [ 0.426579] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001075worker1 # [ 0.446755] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1076worker2 # [ 0.426603] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1077worker1 # [ 0.446784] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078client # [ 0.469729] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1079worker2 # [ 0.426634] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1080worker1 # [ 0.447172] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1081client # [ 0.469935] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1082worker2 # [ 0.426715] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001083worker1 # [ 0.447355] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1084client # [ 0.469966] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085worker2 # [ 0.426725] pci_bus 0000:00: root bus resource [bus 00-ff]1086worker1 # [ 0.447669] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1087worker2 # [ 0.426731] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1088client # [ 0.470432] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1089lb # [ 0.454718] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1090worker1 # [ 0.447860] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1091worker2 # [ 0.426736] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1092client # [ 0.470619] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1093lb # [ 0.454923] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1094worker1 # [ 0.447890] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1095client # [ 0.470650] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1096worker2 # [ 0.426742] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1097lb # [ 0.455178] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1098worker1 # [ 0.448342] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1099worker2 # [ 0.426837] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1100client # [ 0.471109] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1101lb # [ 0.455367] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1102worker1 # [ 0.448530] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1103client # [ 0.471296] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1104lb # [ 0.455401] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1105worker2 # [ 0.427290] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1106worker1 # [ 0.448559] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1107client # [ 0.471326] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108worker2 # [ 0.427478] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1109lb # [ 0.455860] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1110worker1 # [ 0.449007] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1111worker2 # [ 0.427495] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1112client # [ 0.471814] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1113lb # [ 0.456048] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1114worker1 # [ 0.449197] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1115client # [ 0.472118] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1116worker2 # [ 0.427527] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1117lb # [ 0.456077] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118worker1 # [ 0.449227] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1119client # [ 0.472136] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1120worker2 # [ 0.427544] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1121lb # [ 0.456532] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1122client # [ 0.472166] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123worker2 # [ 0.428009] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1124lb # [ 0.456720] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1125worker2 # [ 0.428196] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1126client # [ 0.472625] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1127lb # [ 0.456752] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1128client # [ 0.472807] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1129worker2 # [ 0.428212] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1130lb # [ 0.457228] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1131client # [ 0.472824] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1132worker2 # [ 0.428241] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133lb # [ 0.457525] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1134client # [ 0.472854] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135lb # [ 0.457543] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1136lb # [ 0.457572] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137worker2 # [ 0.447551] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1138worker2 # [ 0.447738] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1139lb # [ 0.458045] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1140worker2 # [ 0.447754] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1141lb # [ 0.458231] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1142lb # [ 0.458247] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1143worker2 # [ 0.447784] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1144lb # [ 0.458277] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1145worker2 # [ 0.448235] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1146lb # [ 0.458876] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1147worker2 # [ 0.448417] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1148worker2 # [ 0.448433] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1149lb # [ 0.458887] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1150worker2 # [ 0.448462] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1151lb # [ 0.458892] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1152worker2 # [ 0.448480] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1153lb # [ 0.458939] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1154lb # [ 0.458986] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1155lb # [ 0.459033] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1156lb # [ 0.459081] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1157client # [ 0.493561] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1158lb # [ 0.459129] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1159client # [ 0.493586] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1160lb # [ 0.459176] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1161client # [ 0.493592] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1162worker1 # [ 0.481844] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1163worker1 # [ 0.482183] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1164lb # [ 0.459223] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1165client # [ 0.493645] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1166worker1 # [ 0.482202] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1167lb # [ 0.459269] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1168client # [ 0.493695] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1169worker1 # [ 0.482232] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1170lb # [ 0.459315] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1171client # [ 0.493744] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1172worker1 # [ 0.482703] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1173lb # [ 0.459386] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1174client # [ 0.493793] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1175worker1 # [ 0.482889] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1176lb # [ 0.459440] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1177worker1 # [ 0.482905] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1178client # [ 0.493842] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1179lb # [ 0.459461] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1180worker1 # [ 0.482935] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1181client # [ 0.493891] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1182lb # [ 0.459482] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1183worker1 # [ 0.483527] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1184client # [ 0.493939] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1185lb # [ 0.459503] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1186worker1 # [ 0.483538] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1187client # [ 0.493987] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1188lb # [ 0.459525] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1189worker1 # [ 0.483543] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1190worker2 # [ 0.461054] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1191client # [ 0.494035] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1192lb # [ 0.459546] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1193worker1 # [ 0.483589] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1194worker2 # [ 0.461265] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1195lb # [ 0.459568] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1196client # [ 0.494166] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1197worker2 # [ 0.461296] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1198worker1 # [ 0.483637] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1199lb # [ 0.459593] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1200client # [ 0.494216] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1201worker2 # [ 0.461754] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1202worker1 # [ 0.483685] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1203lb # [ 0.459615] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1204client # [ 0.494239] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1205worker2 # [ 0.461943] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1206lb # [ 0.459636] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1207client # [ 0.494261] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1208worker1 # [ 0.483733] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1209worker2 # [ 0.461973] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1210lb # [ 0.459659] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1211client # [ 0.494284] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1212worker1 # [ 0.483781] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1213client # [ 0.494306] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1214worker2 # [ 0.462362] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1215lb # [ 0.459681] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1216worker1 # [ 0.483829] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1217lb # [ 0.459702] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1218client # [ 0.494328] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1219worker2 # [ 0.462544] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1220lb # [ 0.459723] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1221worker1 # [ 0.483877] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1222worker2 # [ 0.462832] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1223client # [ 0.494351] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1224lb # [ 0.459744] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1225worker2 # [ 0.463021] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1226client # [ 0.494381] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1227worker1 # [ 0.483924] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1228lb # [ 0.459765] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1229client # [ 0.494405] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1230worker2 # [ 0.463051] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1231worker1 # [ 0.483972] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1232lb # [ 0.459786] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1233client # [ 0.494428] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1234worker2 # [ 0.463503] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1235worker1 # [ 0.484054] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1236lb # [ 0.459807] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1237client # [ 0.494451] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1238worker2 # [ 0.463689] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1239lb # [ 0.459833] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1240worker1 # [ 0.484108] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1241client # [ 0.494474] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1242lb # [ 0.459842] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1243worker2 # [ 0.463719] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1244worker1 # [ 0.484130] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1245client # [ 0.494496] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1246lb # [ 0.459847] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1247worker2 # [ 0.464168] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1248worker1 # [ 0.484151] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1249client # [ 0.494518] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1250lb # [ 0.460661] pci 0000:00:07.0: enabling device (0000 -> 0002)1251worker2 # [ 0.464353] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1252worker1 # [ 0.484172] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1253client # [ 0.494539] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1254worker2 # [ 0.464383] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1255worker1 # [ 0.484194] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1256client # [ 0.494561] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1257client # [ 0.494583] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1258worker1 # [ 0.484215] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1259worker2 # [ 0.464863] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1260client # [ 0.494605] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1261worker2 # [ 0.465155] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1262worker1 # [ 0.484236] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1263client # [ 0.494632] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1264worker2 # [ 0.465173] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1265worker1 # [ 0.484263] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1266client # [ 0.494642] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1267worker2 # [ 0.465203] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1268worker1 # [ 0.484284] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1269client # [ 0.494646] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1270worker1 # [ 0.484307] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1271worker2 # [ 0.465675] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1272client # [ 0.495479] pci 0000:00:07.0: enabling device (0000 -> 0002)1273worker2 # [ 0.465860] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1274worker1 # [ 0.484329] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1275worker2 # [ 0.465876] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1276worker1 # [ 0.484351] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1277worker2 # [ 0.465905] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1278worker1 # [ 0.484372] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1279worker2 # [ 0.466475] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1280worker1 # [ 0.484393] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1281worker1 # [ 0.484414] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1282worker2 # [ 0.466487] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1283worker1 # [ 0.484435] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1284worker2 # [ 0.466492] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1285worker1 # [ 0.484456] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1286worker2 # [ 0.466538] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1287worker1 # [ 0.484477] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1288worker1 # [ 0.484502] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1289worker2 # [ 0.466586] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1290worker1 # [ 0.484512] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1291worker2 # [ 0.466634] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1292worker1 # [ 0.484516] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1293worker2 # [ 0.466682] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1294worker1 # [ 0.485326] pci 0000:00:07.0: enabling device (0000 -> 0002)1295worker2 # [ 0.466728] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1296worker2 # [ 0.466774] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1297worker2 # [ 0.466820] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1298worker2 # [ 0.466867] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1299worker2 # [ 0.466913] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1300worker2 # [ 0.466979] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1301worker2 # [ 0.467025] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1302worker2 # [ 0.467048] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1303worker2 # [ 0.467070] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1304worker2 # [ 0.467091] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1305worker2 # [ 0.467112] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1306worker2 # [ 0.467138] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1307client # [ 0.544351] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1308worker2 # [ 0.467164] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1309worker2 # [ 0.467192] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1310worker2 # [ 0.467216] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1311worker2 # [ 0.467242] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1312worker2 # [ 0.467264] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1313worker2 # [ 0.467287] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1314worker2 # [ 0.467308] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1315worker2 # [ 0.467329] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1316worker2 # [ 0.467350] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1317lb # [ 0.537267] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1318worker2 # [ 0.467371] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1319worker2 # [ 0.467392] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1320client # [ 0.550683] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1321worker2 # [ 0.467414] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1322worker1 # [ 0.541215] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1323worker2 # [ 0.467440] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1324client # [ 0.552929] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1325worker2 # [ 0.467449] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1326lb # [ 0.540351] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1327worker2 # [ 0.467454] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1328worker2 # [ 0.468270] pci 0000:00:07.0: enabling device (0000 -> 0002)1329worker2 # [ 0.524727] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55137 usecs1330client # [ 0.562839] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1331client # [ 0.565026] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1332worker1 # [ 0.550931] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1333worker1 # [ 0.553424] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1334lb # [ 0.550484] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1335lb # [ 0.552578] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1336lb # [ 0.554659] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1337lb # [ 0.556905] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1338client # [ 0.576727] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1339worker1 # [ 0.562809] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1340client # [ 0.579471] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1341lb # [ 0.566612] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1342lb # [ 0.568635] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1343worker1 # [ 0.570089] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1344worker1 # [ 0.572195] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1345worker2 # [ 0.545359] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1346worker2 # [ 0.547865] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1347client # [ 0.589431] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1348client # [ 0.591356] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1349client # [ 0.593232] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1350lb # [ 0.578603] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1351lb # [ 0.580931] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1352worker2 # [ 0.558639] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1353worker1 # [ 0.582077] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1354worker1 # [ 0.584023] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1355worker1 # [ 0.585949] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1356worker1 # [ 0.587857] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1357client # [ 0.603854] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1358lb # [ 0.592645] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1359client # [ 0.614627] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1360client # [ 0.617353] msm_serial: driver initialized1361lb # [ 0.603729] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1362worker2 # [ 0.572657] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1363worker2 # [ 0.574663] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1364worker2 # [ 0.576792] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1365worker1 # [ 0.606162] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1366worker2 # [ 0.578887] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1367lb # [ 0.606674] msm_serial: driver initialized1368lb # [ 0.606828] SuperH (H)SCI(F) driver initialized1369worker1 # [ 0.611997] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1370lb # [ 0.606884] STM32 USART driver initialized1371client # [ 0.621654] SuperH (H)SCI(F) driver initialized1372client # [ 0.621719] STM32 USART driver initialized1373worker1 # [ 0.614520] msm_serial: driver initialized1374worker1 # [ 0.614663] SuperH (H)SCI(F) driver initialized1375worker1 # [ 0.614715] STM32 USART driver initialized1376worker2 # [ 0.588736] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1377worker2 # [ 0.590649] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1378worker2 # [ 0.600639] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1379worker2 # [ 0.603563] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1380worker2 # [ 0.613761] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1381worker2 # [ 0.616448] msm_serial: driver initialized1382client # [ 0.654032] loop: module loaded1383lb # [ 0.641618] loop: module loaded1384client # [ 0.654231] virtio_blk virtio2: 1/0/0 default/read/poll queues1385lb # [ 0.641873] virtio_blk virtio2: 1/0/0 default/read/poll queues1386client # [ 0.655032] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1387lb # [ 0.644024] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1388worker2 # [ 0.617182] SuperH (H)SCI(F) driver initialized1389worker2 # [ 0.617240] STM32 USART driver initialized1390worker1 # [ 0.651060] loop: module loaded1391worker1 # [ 0.651263] virtio_blk virtio2: 1/0/0 default/read/poll queues1392worker1 # [ 0.652081] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1393client # [ 0.662056] megasas: 07.734.00.00-rc11394client # [ 0.662729] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1395client # [ 0.665213] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001396client # [ 0.665242] Intel/Sharp Extended Query Table at 0x00311397client # [ 0.666953] Using buffer write method1398client # [ 0.667028] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1399client # [ 0.668756] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001400worker1 # [ 0.658128] megasas: 07.734.00.00-rc11401lb # [ 0.655218] megasas: 07.734.00.00-rc11402client # [ 0.668780] Intel/Sharp Extended Query Table at 0x00311403worker1 # [ 0.658853] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1404lb # [ 0.656022] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1405lb # [ 0.658280] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406lb # [ 0.658324] Intel/Sharp Extended Query Table at 0x00311407lb # [ 0.659916] Using buffer write method1408lb # [ 0.660026] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1409lb # [ 0.661689] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001410lb # [ 0.661774] Intel/Sharp Extended Query Table at 0x00311411worker1 # [ 0.670539] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001412client # [ 0.687510] Using buffer write method1413worker1 # [ 0.670606] Intel/Sharp Extended Query Table at 0x00311414client # [ 0.687551] Concatenating MTD devices:1415worker1 # [ 0.672205] Using buffer write method1416client # [ 0.687556] (0): "0.flash"1417client # [ 0.687560] (1): "0.flash"1418client # [ 0.687571] into device "0.flash"1419worker1 # [ 0.672276] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1420worker2 # [ 0.651462] loop: module loaded1421worker2 # [ 0.651640] virtio_blk virtio2: 1/0/0 default/read/poll queues1422worker2 # [ 0.652387] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1423lb # [ 0.680446] Using buffer write method1424lb # [ 0.680506] Concatenating MTD devices:1425lb # [ 0.680510] (0): "0.flash"1426lb # [ 0.680514] (1): "0.flash"1427lb # [ 0.680523] into device "0.flash"1428worker2 # [ 0.657176] megasas: 07.734.00.00-rc11429worker2 # [ 0.657825] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1430worker1 # [ 0.686347] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001431worker1 # [ 0.686378] Intel/Sharp Extended Query Table at 0x00311432worker1 # [ 0.698051] Using buffer write method1433worker1 # [ 0.698086] Concatenating MTD devices:1434worker1 # [ 0.698090] (0): "0.flash"1435worker1 # [ 0.698094] (1): "0.flash"1436worker1 # [ 0.698103] into device "0.flash"1437worker2 # [ 0.669624] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001438worker2 # [ 0.669658] Intel/Sharp Extended Query Table at 0x00311439worker2 # [ 0.671210] Using buffer write method1440worker2 # [ 0.671280] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1441worker2 # [ 0.686543] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001442worker2 # [ 0.686579] Intel/Sharp Extended Query Table at 0x00311443worker2 # [ 0.688197] Using buffer write method1444worker2 # [ 0.688225] Concatenating MTD devices:1445worker2 # [ 0.688229] (0): "0.flash"1446worker2 # [ 0.688240] (1): "0.flash"1447worker2 # [ 0.688244] into device "0.flash"1448client # [ 0.941000] Freeing initrd memory: 26972K1449client # [ 0.947005] tun: Universal TUN/TAP device driver, 1.61450lb # [ 0.938312] Freeing initrd memory: 26948K1451client # [ 0.950805] thunder_xcv, ver 1.01452client # [ 0.950850] thunder_bgx, ver 1.01453client # [ 0.950878] nicpf, ver 1.01454client # [ 0.951429] e1000: Intel(R) PRO/1000 Network Driver1455client # [ 0.951436] e1000: Copyright (c) 1999-2006 Intel Corporation.1456client # [ 0.951475] e1000e: Intel(R) PRO/1000 Network Driver1457client # [ 0.951483] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1458lb # [ 0.944431] tun: Universal TUN/TAP device driver, 1.61459client # [ 0.951514] igb: Intel(R) Gigabit Ethernet Network Driver1460client # [ 0.951520] igb: Copyright (c) 2007-2014 Intel Corporation.1461client # [ 0.951560] igbvf: Intel(R) Gigabit Virtual Function Network Driver1462client # [ 0.951566] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1463client # [ 0.951698] sky2: driver version 1.301464client # [ 0.953229] usbcore: registered new interface driver usb-storage1465client # [ 0.953313] usbcore: registered new interface driver usbserial_generic1466client # [ 0.953327] usbserial: USB Serial support registered for generic1467client # [ 0.954164] ehci-pci 0000:00:07.0: EHCI Host Controller1468lb # [ 0.948442] thunder_xcv, ver 1.01469lb # [ 0.948487] thunder_bgx, ver 1.01470lb # [ 0.948514] nicpf, ver 1.01471client # [ 0.954194] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11472lb # [ 0.949085] e1000: Intel(R) PRO/1000 Network Driver1473worker1 # [ 0.953179] Freeing initrd memory: 26976K1474client # [ 0.954447] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001475lb # [ 0.949093] e1000: Copyright (c) 1999-2006 Intel Corporation.1476lb # [ 0.949119] e1000e: Intel(R) PRO/1000 Network Driver1477client # [ 0.966020] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001478lb # [ 0.949127] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1479client # [ 0.966324] hub 1-0:1.0: USB hub found1480client # [ 0.966339] hub 1-0:1.0: 6 ports detected1481lb # [ 0.949160] igb: Intel(R) Gigabit Ethernet Network Driver1482lb # [ 0.949166] igb: Copyright (c) 2007-2014 Intel Corporation.1483lb # [ 0.949193] igbvf: Intel(R) Gigabit Virtual Function Network Driver1484client # [ 0.968913] hv_vmbus: registering driver hyperv_keyboard1485lb # [ 0.949199] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1486lb # [ 0.949333] sky2: driver version 1.301487worker1 # [ 0.959143] tun: Universal TUN/TAP device driver, 1.61488client # [ 0.970469] rtc-pl031 9010000.pl031: registered as rtc01489lb # [ 0.958024] usbcore: registered new interface driver usb-storage1490client # [ 0.970501] rtc-pl031 9010000.pl031: setting system clock to 2026-09-24T12:25:35 UTC (1790252735)1491lb # [ 0.958085] usbcore: registered new interface driver usbserial_generic1492client # [ 0.970804] i2c_dev: i2c /dev entries driver1493lb # [ 0.958100] usbserial: USB Serial support registered for generic1494lb # [ 0.961446] ehci-pci 0000:00:07.0: EHCI Host Controller1495lb # [ 0.961479] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11496client # [ 0.976169] sdhci: Secure Digital Host Controller Interface driver1497lb # [ 0.961840] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001498client # [ 0.976179] sdhci: Copyright(c) Pierre Ossman1499worker1 # [ 0.962930] thunder_xcv, ver 1.01500client # [ 0.976439] Synopsys Designware Multimedia Card Interface Driver1501worker1 # [ 0.962974] thunder_bgx, ver 1.01502worker1 # [ 0.962995] nicpf, ver 1.01503lb # [ 0.965005] hv_vmbus: registering driver hyperv_keyboard1504client # [ 0.976807] sdhci-pltfm: SDHCI platform and OF driver helper1505worker1 # [ 0.963573] e1000: Intel(R) PRO/1000 Network Driver1506worker1 # [ 0.963581] e1000: Copyright (c) 1999-2006 Intel Corporation.1507worker1 # [ 0.963604] e1000e: Intel(R) PRO/1000 Network Driver1508client # [ 0.981023] hid: raw HID events driver (C) Jiri Kosina1509worker2 # [ 0.940948] Freeing initrd memory: 26944K1510client # [ 0.981273] usbcore: registered new interface driver usbhid1511worker1 # [ 0.963613] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1512lb # [ 0.966718] rtc-pl031 9010000.pl031: registered as rtc01513worker1 # [ 0.963642] igb: Intel(R) Gigabit Ethernet Network Driver1514client # [ 0.981280] usbhid: USB HID core driver1515lb # [ 0.966748] rtc-pl031 9010000.pl031: setting system clock to 2026-09-24T12:25:35 UTC (1790252735)1516lb # [ 0.967074] i2c_dev: i2c /dev entries driver1517worker1 # [ 0.963647] igb: Copyright (c) 2007-2014 Intel Corporation.1518worker1 # [ 0.963670] igbvf: Intel(R) Gigabit Virtual Function Network Driver1519worker1 # [ 0.963675] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1520worker1 # [ 0.963807] sky2: driver version 1.301521worker2 # [ 0.946812] tun: Universal TUN/TAP device driver, 1.61522client # [ 0.984117] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1523worker1 # [ 0.965365] usbcore: registered new interface driver usb-storage1524worker1 # [ 0.965411] usbcore: registered new interface driver usbserial_generic1525client # [ 0.986734] drop_monitor: Initializing network drop monitor service1526worker1 # [ 0.965425] usbserial: USB Serial support registered for generic1527client # [ 0.986928] NET: Registered PF_INET6 protocol family1528worker1 # [ 0.966252] ehci-pci 0000:00:07.0: EHCI Host Controller1529lb # [ 0.972539] sdhci: Secure Digital Host Controller Interface driver1530lb # [ 0.972551] sdhci: Copyright(c) Pierre Ossman1531client # [ 0.988955] Segment Routing with IPv61532worker1 # [ 0.966285] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11533client # [ 0.988974] In-situ OAM (IOAM) with IPv61534lb # [ 0.972821] Synopsys Designware Multimedia Card Interface Driver1535worker1 # [ 0.966555] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001536client # [ 0.989003] NET: Registered PF_PACKET protocol family1537lb # [ 0.973195] sdhci-pltfm: SDHCI platform and OF driver helper1538worker1 # [ 0.978217] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001539worker2 # [ 0.950552] thunder_xcv, ver 1.01540worker1 # [ 0.978524] hub 1-0:1.0: USB hub found1541worker2 # [ 0.950595] thunder_bgx, ver 1.01542lb # [ 0.977218] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001543worker1 # [ 0.978543] hub 1-0:1.0: 6 ports detected1544worker2 # [ 0.950616] nicpf, ver 1.01545lb # [ 0.977544] hub 1-0:1.0: USB hub found1546worker2 # [ 0.951140] e1000: Intel(R) PRO/1000 Network Driver1547lb # [ 0.977563] hub 1-0:1.0: 6 ports detected1548worker2 # [ 0.951147] e1000: Copyright (c) 1999-2006 Intel Corporation.1549client # [ 0.990816] 9pnet: Installing 9P2000 support1550worker1 # [ 0.981139] hv_vmbus: registering driver hyperv_keyboard1551worker2 # [ 0.951171] e1000e: Intel(R) PRO/1000 Network Driver1552client # [ 0.990865] Key type dns_resolver registered1553worker2 # [ 0.951178] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1554lb # [ 0.980063] hid: raw HID events driver (C) Jiri Kosina1555worker2 # [ 0.951206] igb: Intel(R) Gigabit Ethernet Network Driver1556lb # [ 0.980306] usbcore: registered new interface driver usbhid1557worker1 # [ 0.982635] rtc-pl031 9010000.pl031: registered as rtc01558worker2 # [ 0.951211] igb: Copyright (c) 2007-2014 Intel Corporation.1559lb # [ 0.980312] usbhid: USB HID core driver1560worker2 # [ 0.951232] igbvf: Intel(R) Gigabit Virtual Function Network Driver1561worker1 # [ 0.982662] rtc-pl031 9010000.pl031: setting system clock to 2026-09-24T12:25:35 UTC (1790252735)1562worker2 # [ 0.951238] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1563worker1 # [ 0.982960] i2c_dev: i2c /dev entries driver1564worker2 # [ 0.951367] sky2: driver version 1.301565lb # [ 0.983307] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1566worker2 # [ 0.960275] ehci-pci 0000:00:07.0: EHCI Host Controller1567lb # [ 0.984876] drop_monitor: Initializing network drop monitor service1568worker2 # [ 0.960307] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11569lb # [ 0.985013] NET: Registered PF_INET6 protocol family1570worker2 # [ 0.960637] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001571client # [ 0.997580] registered taskstats version 11572worker1 # [ 0.988322] sdhci: Secure Digital Host Controller Interface driver1573lb # [ 0.988195] Segment Routing with IPv61574client # [ 0.997722] Loading compiled-in X.509 certificates1575worker1 # [ 0.988334] sdhci: Copyright(c) Pierre Ossman1576lb # [ 0.988216] In-situ OAM (IOAM) with IPv61577worker1 # [ 0.988598] Synopsys Designware Multimedia Card Interface Driver1578lb # [ 0.988261] NET: Registered PF_PACKET protocol family1579worker2 # [ 0.963355] usbcore: registered new interface driver usb-storage1580worker1 # [ 0.988968] sdhci-pltfm: SDHCI platform and OF driver helper1581lb # [ 0.989984] 9pnet: Installing 9P2000 support1582worker2 # [ 0.963476] usbcore: registered new interface driver usbserial_generic1583lb # [ 0.990037] Key type dns_resolver registered1584worker2 # [ 0.963490] usbserial: USB Serial support registered for generic1585worker1 # [ 0.993155] hid: raw HID events driver (C) Jiri Kosina1586worker2 # [ 0.964062] hv_vmbus: registering driver hyperv_keyboard1587worker1 # [ 0.993394] usbcore: registered new interface driver usbhid1588worker1 # [ 0.993400] usbhid: USB HID core driver1589client # [ 1.006444] Demotion targets for Node 0: null1590client # [ 1.006547] Key type .fscrypt registered1591worker2 # [ 0.967875] rtc-pl031 9010000.pl031: registered as rtc01592client # [ 1.006553] Key type fscrypt-provisioning registered1593client # [ 1.006646] ima: No TPM chip found, activating TPM-bypass!1594worker2 # [ 0.967904] rtc-pl031 9010000.pl031: setting system clock to 2026-09-24T12:25:35 UTC (1790252735)1595worker2 # [ 0.968198] i2c_dev: i2c /dev entries driver1596client # [ 1.006665] ima: Allocated hash algorithm: sha11597client # [ 1.006688] ima: No architecture policies found1598worker1 # [ 0.996345] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1599client # [ 1.010721] input: gpio-keys as /devices/platform/gpio-keys/input/input01600worker1 # [ 0.997856] drop_monitor: Initializing network drop monitor service1601lb # [ 0.996983] registered taskstats version 11602worker1 # [ 0.997988] NET: Registered PF_INET6 protocol family1603lb # [ 0.997141] Loading compiled-in X.509 certificates1604worker1 # [ 1.001046] Segment Routing with IPv61605worker1 # [ 1.001081] In-situ OAM (IOAM) with IPv61606worker1 # [ 1.001109] NET: Registered PF_PACKET protocol family1607worker2 # [ 0.973438] sdhci: Secure Digital Host Controller Interface driver1608worker2 # [ 0.973448] sdhci: Copyright(c) Pierre Ossman1609worker2 # [ 0.973701] Synopsys Designware Multimedia Card Interface Driver1610worker1 # [ 1.002902] 9pnet: Installing 9P2000 support1611worker2 # [ 0.974054] sdhci-pltfm: SDHCI platform and OF driver helper1612worker1 # [ 1.002948] Key type dns_resolver registered1613worker2 # [ 0.975508] hid: raw HID events driver (C) Jiri Kosina1614worker2 # [ 0.975734] usbcore: registered new interface driver usbhid1615worker2 # [ 0.975740] usbhid: USB HID core driver1616lb # [ 1.006043] Demotion targets for Node 0: null1617worker2 # [ 0.976587] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001618lb # [ 1.006168] Key type .fscrypt registered1619worker2 # [ 0.982104] hub 1-0:1.0: USB hub found1620lb # [ 1.006175] Key type fscrypt-provisioning registered1621worker1 # [ 1.009293] registered taskstats version 11622worker2 # [ 0.982125] hub 1-0:1.0: 6 ports detected1623lb # [ 1.006280] ima: No TPM chip found, activating TPM-bypass!1624worker1 # [ 1.009433] Loading compiled-in X.509 certificates1625lb # [ 1.006300] ima: Allocated hash algorithm: sha11626lb # [ 1.006322] ima: No architecture policies found1627worker2 # [ 0.983656] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1628lb # [ 1.010661] input: gpio-keys as /devices/platform/gpio-keys/input/input01629worker2 # [ 0.986245] drop_monitor: Initializing network drop monitor service1630worker2 # [ 0.986430] NET: Registered PF_INET6 protocol family1631worker2 # [ 0.988304] Segment Routing with IPv61632worker2 # [ 0.988323] In-situ OAM (IOAM) with IPv61633worker2 # [ 0.988349] NET: Registered PF_PACKET protocol family1634client # [ 1.029082] clk: Disabling unused clocks1635client # [ 1.029105] PM: genpd: Disabling unused power domains1636worker1 # [ 1.017979] Demotion targets for Node 0: null1637worker1 # [ 1.018080] Key type .fscrypt registered1638worker2 # [ 0.990017] 9pnet: Installing 9P2000 support1639worker1 # [ 1.018088] Key type fscrypt-provisioning registered1640worker2 # [ 0.990067] Key type dns_resolver registered1641worker1 # [ 1.018179] ima: No TPM chip found, activating TPM-bypass!1642worker1 # [ 1.018198] ima: Allocated hash algorithm: sha11643worker1 # [ 1.018221] ima: No architecture policies found1644client # [ 1.033225] Freeing unused kernel memory: 4736K1645client # [ 1.034017] Run /init as init process1646worker1 # [ 1.022440] input: gpio-keys as /devices/platform/gpio-keys/input/input01647worker2 # [ 0.996650] registered taskstats version 11648worker2 # [ 0.996788] Loading compiled-in X.509 certificates1649lb # [ 1.028539] clk: Disabling unused clocks1650lb # [ 1.028571] PM: genpd: Disabling unused power domains1651worker2 # [ 1.005480] Demotion targets for Node 0: null1652worker2 # [ 1.005585] Key type .fscrypt registered1653worker2 # [ 1.005591] Key type fscrypt-provisioning registered1654worker2 # [ 1.005691] ima: No TPM chip found, activating TPM-bypass!1655lb # [ 1.032758] Freeing unused kernel memory: 4736K1656worker2 # [ 1.005710] ima: Allocated hash algorithm: sha11657lb # [ 1.032963] Run /init as init process1658worker2 # [ 1.005731] ima: No architecture policies found1659client # [ 1.047651] systemd[1]: Successfully made /usr/ read-only.1660worker2 # [ 1.009965] input: gpio-keys as /devices/platform/gpio-keys/input/input01661worker1 # [ 1.041095] clk: Disabling unused clocks1662worker1 # [ 1.041127] PM: genpd: Disabling unused power domains1663worker1 # [ 1.045337] Freeing unused kernel memory: 4736K1664worker1 # [ 1.046133] Run /init as init process1665lb # [ 1.048746] systemd[1]: Successfully made /usr/ read-only.1666worker2 # [ 1.027924] clk: Disabling unused clocks1667worker2 # [ 1.027950] PM: genpd: Disabling unused power domains1668worker2 # [ 1.032075] Freeing unused kernel memory: 4736K1669worker2 # [ 1.032295] Run /init as init process1670worker1 # [ 1.059735] systemd[1]: Successfully made /usr/ read-only.1671worker2 # [ 1.046724] systemd[1]: Successfully made /usr/ read-only.1672client # [ 1.213497] usb 1-1: new high-speed USB device number 2 using ehci-pci1673lb # [ 1.222452] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker1 # [ 1.225560] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker2 # [ 1.228648] usb 1-1: new high-speed USB device number 2 using ehci-pci1676client # [ 1.366172] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677client # [ 1.382379] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678client # [ 1.388602] systemd[1]: Detected virtualization qemu.1679client # [ 1.389686] systemd[1]: Detected architecture arm64.1680worker1 # [ 1.375600] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11681client # [ 1.390683] systemd[1]: Running in initrd.1682client # [ 1.392081] systemd[1]: Initializing machine ID from random generator.1683lb # [ 1.375116] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684client # [ 1.393611] systemd[1]: Hostname set to <client>.1685lb # [ 1.383719] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1686lb # [ 1.396105] systemd[1]: Detected virtualization qemu.1687lb # [ 1.398195] systemd[1]: Detected architecture arm64.1688lb # [ 1.400188] systemd[1]: Running in initrd.1689worker1 # [ 1.394585] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1690lb # [ 1.402871] systemd[1]: Initializing machine ID from random generator.1691worker2 # [ 1.378685] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11692worker1 # [ 1.406865] systemd[1]: Detected virtualization qemu.1693lb # [ 1.405699] systemd[1]: Hostname set to <lb>.1694worker1 # [ 1.408855] systemd[1]: Detected architecture arm64.1695worker1 # [ 1.410829] systemd[1]: Running in initrd.1696worker2 # [ 1.381938] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1697worker1 # [ 1.413388] systemd[1]: Initializing machine ID from random generator.1698worker2 # [ 1.381968] systemd[1]: Detected virtualization qemu.1699worker2 # [ 1.382014] systemd[1]: Detected architecture arm64.1700worker1 # [ 1.416418] systemd[1]: Hostname set to <worker1>.1701worker2 # [ 1.382026] systemd[1]: Running in initrd.1702worker2 # [ 1.382547] systemd[1]: Initializing machine ID from random generator.1703worker2 # [ 1.382693] systemd[1]: Hostname set to <worker2>.1704client # [ 1.445645] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705lb # [ 1.454666] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker1 # [ 1.461794] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker2 # [ 1.456762] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708client # [ 1.565473] usb 1-2: new high-speed USB device number 3 using ehci-pci1709lb # [ 1.574498] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker1 # [ 1.589592] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker2 # [ 1.576615] usb 1-2: new high-speed USB device number 3 using ehci-pci1712worker2 # [ 1.666372] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713client # [ 1.706115] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714lb # [ 1.712977] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715client # [ 1.726179] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21716client # [ 1.733778] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01717worker1 # [ 1.731227] systemd[1]: bpf-restrict-fs: LSM BPF program attached1718lb # [ 1.736443] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719lb # [ 1.738484] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01720worker1 # [ 1.748084] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721worker1 # [ 1.754062] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722worker2 # [ 1.740047] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21723worker2 # [ 1.743569] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01724lb # [ 1.778971] systemd[1]: Queued start job for default target Initrd Default Target.1725lb # [ 1.788396] systemd[1]: Created slice Slice /system/modprobe.1726lb # [ 1.789445] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1727lb # [ 1.790568] systemd[1]: Expecting device /dev/disk/by-label/nixos...1728lb # [ 1.791407] systemd[1]: Reached target Path Units.1729lb # [ 1.792065] systemd[1]: Reached target Slice Units.1730lb # [ 1.792720] systemd[1]: Reached target Swaps.1731lb # [ 1.793310] systemd[1]: Reached target Timer Units.1732lb # [ 1.794155] systemd[1]: Listening on D-Bus System Message Bus Socket.1733lb # [ 1.795187] systemd[1]: Listening on Journal Socket (/dev/log).1734lb # [ 1.796054] systemd[1]: Listening on Journal Sockets.1735lb # [ 1.796808] systemd[1]: Listening on udev Control Socket.1736lb # [ 1.797648] systemd[1]: Listening on udev Kernel Socket.1737lb # [ 1.798437] systemd[1]: Reached target Socket Units.1738worker2 # [ 1.773555] systemd[1]: Queued start job for default target Initrd Default Target.1739lb # [ 1.800470] systemd[1]: Starting Create List of Static Device Nodes...1740lb # [ 1.801425] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1741client # [ 1.820542] systemd[1]: Queued start job for default target Initrd Default Target.1742worker2 # [ 1.783092] systemd[1]: Created slice Slice /system/modprobe.1743worker2 # [ 1.784317] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1744lb # [ 1.810240] systemd[1]: Mounting Kernel Configuration File System...1745worker2 # [ 1.785806] systemd[1]: Expecting device /dev/disk/by-label/nixos...1746client # [ 1.827575] systemd[1]: Created slice Slice /system/modprobe.1747worker2 # [ 1.785845] systemd[1]: Reached target Path Units.1748worker2 # [ 1.785867] systemd[1]: Reached target Slice Units.1749client # [ 1.828660] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1750worker2 # [ 1.785887] systemd[1]: Reached target Swaps.1751client # [ 1.829878] systemd[1]: Expecting device /dev/disk/by-label/nixos...1752worker2 # [ 1.785905] systemd[1]: Reached target Timer Units.1753client # [ 1.830816] systemd[1]: Reached target Path Units.1754worker2 # [ 1.786117] systemd[1]: Listening on D-Bus System Message Bus Socket.1755client # [ 1.831548] systemd[1]: Reached target Slice Units.1756client # [ 1.832283] systemd[1]: Reached target Swaps.1757worker2 # [ 1.786281] systemd[1]: Listening on Journal Socket (/dev/log).1758lb # [ 1.818794] systemd[1]: Starting Journal Service...1759client # [ 1.833037] systemd[1]: Reached target Timer Units.1760worker2 # [ 1.786433] systemd[1]: Listening on Journal Sockets.1761worker2 # [ 1.786569] systemd[1]: Listening on udev Control Socket.1762client # [ 1.834016] systemd[1]: Listening on D-Bus System Message Bus Socket.1763worker2 # [ 1.786697] systemd[1]: Listening on udev Kernel Socket.1764client # [ 1.835100] systemd[1]: Listening on Journal Socket (/dev/log).1765worker2 # [ 1.786720] systemd[1]: Reached target Socket Units.1766client # [ 1.836106] systemd[1]: Listening on Journal Sockets.1767client # [ 1.836977] systemd[1]: Listening on udev Control Socket.1768worker2 # [ 1.798131] systemd[1]: Starting Create List of Static Device Nodes...1769client # [ 1.837948] systemd[1]: Listening on udev Kernel Socket.1770worker2 # [ 1.799316] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1771client # [ 1.837976] systemd[1]: Reached target Socket Units.1772client # [ 1.840942] systemd[1]: Starting Create List of Static Device Nodes...1773client # [ 1.841994] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1774worker2 # [ 1.808850] systemd[1]: Mounting Kernel Configuration File System...1775client # [ 1.852390] systemd[1]: Mounting Kernel Configuration File System...1776worker2 # [ 1.816886] systemd[1]: Starting Journal Service...1777worker1 # [ 1.845782] systemd[1]: Queued start job for default target Initrd Default Target.1778lb # [ 1.842158] systemd[1]: Starting Load Kernel Modules...1779lb # [ 1.842284] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1780lb # [ 1.848744] systemd[1]: Starting Coldplug All udev Devices...1781worker1 # [ 1.855892] systemd[1]: Created slice Slice /system/modprobe.1782worker1 # [ 1.857060] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1783worker1 # [ 1.858452] systemd[1]: Expecting device /dev/disk/by-label/nixos...1784worker1 # [ 1.858490] systemd[1]: Reached target Path Units.1785worker1 # [ 1.858509] systemd[1]: Reached target Slice Units.1786worker1 # [ 1.858529] systemd[1]: Reached target Swaps.1787worker1 # [ 1.858547] systemd[1]: Reached target Timer Units.1788worker1 # [ 1.858752] systemd[1]: Listening on D-Bus System Message Bus Socket.1789worker1 # [ 1.858909] systemd[1]: Listening on Journal Socket (/dev/log).1790worker1 # [ 1.859054] systemd[1]: Listening on Journal Sockets.1791worker1 # [ 1.859192] systemd[1]: Listening on udev Control Socket.1792worker2 # [ 1.841043] systemd[1]: Starting Load Kernel Modules...1793worker1 # [ 1.859320] systemd[1]: Listening on udev Kernel Socket.1794worker1 # [ 1.859342] systemd[1]: Reached target Socket Units.1795worker2 # [ 1.843149] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1796lb # [ 1.866572] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1797client # [ 1.883339] systemd[1]: Starting Journal Service...1798worker1 # [ 1.872003] systemd[1]: Starting Create List of Static Device Nodes...1799worker1 # [ 1.873594] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1800lb # [ 1.870515] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1801lb # [ 1.876886] systemd[1]: Finished Create List of Static Device Nodes.1802lb # [ 1.880814] systemd[1]: Mounted Kernel Configuration File System.1803worker1 # [ 1.884268] systemd[1]: Mounting Kernel Configuration File System...1804client # [ 1.905584] systemd[1]: Starting Load Kernel Modules...1805worker2 # [ 1.867145] systemd[1]: Starting Coldplug All udev Devices...1806client # [ 1.905718] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1807client # [ 1.909083] systemd[1]: Starting Coldplug All udev Devices...1808lb # [ 1.896844] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01809lb # [ 1.897122] [drm] features: -virgl +edid -resource_blob -host_visible1810worker2 # [ 1.878387] systemd-journald[72]: Collecting audit messages is disabled.1811lb # [ 1.897132] [drm] features: -context_init1812lb # [ 1.898013] [drm] number of scanouts: 11813lb # [ 1.898034] [drm] number of cap sets: 01814lb # [ 1.900519] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1815worker1 # [ 1.906163] systemd[1]: Starting Journal Service...1816client # [ 1.921825] systemd[1]: Finished Create List of Static Device Nodes.1817worker2 # [ 1.884720] systemd[1]: Finished Create List of Static Device Nodes.1818worker2 # [ 1.885558] systemd[1]: Mounted Kernel Configuration File System.1819worker1 # [ 1.925792] systemd[1]: Starting Load Kernel Modules...1820client # [ 1.937937] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1821worker1 # [ 1.926763] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1822client # [ 1.939605] systemd[1]: Mounted Kernel Configuration File System.1823client # [ 1.942808] systemd-journald[72]: Collecting audit messages is disabled.1824lb # [ 1.922911] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1825lb # [ 1.922938] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01826worker1 # [ 1.933668] systemd[1]: Starting Coldplug All udev Devices...1827worker2 # [ 1.905040] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1828worker1 # [ 1.936776] systemd[1]: Finished Create List of Static Device Nodes.1829worker1 # [ 1.938494] systemd[1]: Mounted Kernel Configuration File System.1830worker1 # [ 1.951377] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1831lb # [ 1.951246] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1832lb # [ 1.956445] systemd-journald[72]: Collecting audit messages is disabled.1833lb # [ 1.958876] systemd[1]: Starting Create Static Device Nodes in /dev...1834lb # [ 1.962984] Console: switching to colour frame buffer device 160x501835worker2 # [ 1.960038] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1836client # [ 2.002081] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1837client # [ 2.004625] systemd[1]: Starting Create Static Device Nodes in /dev...1838lb # [ 1.990995] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1839worker2 # [ 1.968607] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1840worker2 # [ 1.973293] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1841client # [ 2.016549] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1842worker1 # [ 2.001971] systemd-journald[72]: Collecting audit messages is disabled.1843worker1 # [ 2.010431] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1844worker1 # [ 2.013687] systemd[1]: Starting Create Static Device Nodes in /dev...1845lb # [ 2.011176] systemd[1]: Finished Load Kernel Modules.1846worker2 # [ 1.985000] systemd[1]: Starting Create Static Device Nodes in /dev...1847lb # [ 2.014528] systemd[1]: Starting Apply Kernel Variables...1848client # [ 2.029519] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1849worker1 # [ 2.020301] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1850worker2 # [ 2.000848] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01851client # [ 2.039730] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01852worker2 # [ 2.001087] [drm] features: -virgl +edid -resource_blob -host_visible1853client # [ 2.039974] [drm] features: -virgl +edid -resource_blob -host_visible1854worker2 # [ 2.001097] [drm] features: -context_init1855client # [ 2.039984] [drm] features: -context_init1856worker2 # [ 2.001857] [drm] number of scanouts: 11857client # [ 2.040699] [drm] number of scanouts: 11858worker2 # [ 2.001875] [drm] number of cap sets: 01859client # [ 2.040717] [drm] number of cap sets: 01860worker2 # [ 2.005636] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1861worker2 # [ 2.005651] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01862worker1 # [ 2.031667] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1863lb # [ 2.042748] systemd[1]: Finished Create Static Device Nodes in /dev.1864client # [ 2.058146] systemd[1]: Finished Create Static Device Nodes in /dev.1865lb # [ 2.042999] systemd[1]: Reached target Preparation for Local File Systems.1866client # [ 2.058350] systemd[1]: Reached target Preparation for Local File Systems.1867lb # [ 2.043033] systemd[1]: Reached target Local File Systems.1868client # [ 2.058378] systemd[1]: Reached target Local File Systems.1869client # [ 2.062484] systemd[1]: Starting Rule-based Manager for Device Events and Files...1870lb # [ 2.050872] systemd[1]: Starting Rule-based Manager for Device Events and Files...1871worker1 # [ 2.054111] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01872worker1 # [ 2.054379] [drm] features: -virgl +edid -resource_blob -host_visible1873worker1 # [ 2.054392] [drm] features: -context_init1874worker1 # [ 2.055167] [drm] number of scanouts: 11875worker1 # [ 2.055189] [drm] number of cap sets: 01876client # [ 2.073735] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1877worker1 # [ 2.064038] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1878client # [ 2.073755] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01879worker1 # [ 2.064068] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01880worker2 # [ 2.036778] Console: switching to colour frame buffer device 160x501881worker2 # [ 2.043460] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1882worker2 # [ 2.052802] systemd[1]: Finished Create Static Device Nodes in /dev.1883worker2 # [ 2.053212] systemd[1]: Reached target Preparation for Local File Systems.1884worker2 # [ 2.053249] systemd[1]: Reached target Local File Systems.1885lb # [ 2.082746] systemd[1]: Finished Apply Kernel Variables.1886worker2 # [ 2.060962] systemd[1]: Starting Rule-based Manager for Device Events and Files...1887client # [ 2.097707] Console: switching to colour frame buffer device 160x501888worker1 # [ 2.093655] systemd[1]: Finished Create Static Device Nodes in /dev.1889worker2 # [ 2.052664] systemd-modules-load[74]: Inserted module 'dm_mod'1890worker1 # [ 2.093916] systemd[1]: Reached target Preparation for Local File Systems.1891worker1 # [ 2.093947] systemd[1]: Reached target Local File Systems.1892worker2 # [ 2.054561] systemd-modules-load[74]: Module 'virtio_balloon' is built in1893worker2 # [ 2.072808] systemd[1]: Started Journal Service.1894worker2 # [ 2.060475] systemd-modules-load[74]: Module 'virtio_console' is built in1895worker1 # [ 2.098075] Console: switching to colour frame buffer device 160x501896worker1 # [ 2.104968] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1897worker2 # [ 2.064341] systemd-modules-load[74]: Inserted module 'virtio_gpu'1898worker2 # [ 2.066063] systemd-modules-load[74]: Module 'virtio_rng' is built in1899worker1 # [ 2.108389] systemd[1]: Starting Rule-based Manager for Device Events and Files...1900worker2 # [ 2.073376] systemd[1]: Finished Load Kernel Modules.1901client # [ 2.130095] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1902client # [ 2.133760] systemd[1]: Started Journal Service.1903lb # [ 2.120222] systemd[1]: Started Journal Service.1904client # [ 2.123733] systemd-modules-load[73]: Inserted module 'dm_mod'1905lb # [ 2.109237] systemd-modules-load[73]: Inserted module 'dm_mod'1906lb # [ 2.110331] systemd-modules-load[73]: Module 'virtio_balloon' is built in1907client # [ 2.130354] systemd-modules-load[73]: Module 'virtio_balloon' is built in1908client # [ 2.131510] systemd-modules-load[73]: Module 'virtio_console' is built in1909lb # [ 2.120343] systemd-modules-load[73]: Module 'virtio_console' is built in1910lb # [ 2.121523] systemd-modules-load[73]: Inserted module 'virtio_gpu'1911lb # [ 2.124284] systemd-modules-load[73]: Module 'virtio_rng' is built in1912worker2 # [ 2.100483] systemd[1]: Starting Apply Kernel Variables...1913client # [ 2.144434] systemd-modules-load[73]: Inserted module 'virtio_gpu'1914client # [ 2.145449] systemd-modules-load[73]: Module 'virtio_rng' is built in1915client # [ 2.146496] systemd[1]: Starting Create System Files and Directories...1916worker1 # [ 2.145790] systemd[1]: Finished Load Kernel Modules.1917lb # [ 2.136316] systemd[1]: Starting Create System Files and Directories...1918worker2 # [ 2.110031] systemd[1]: Starting Create System Files and Directories...1919worker1 # [ 2.149408] systemd[1]: Starting Apply Kernel Variables...1920lb # [ 2.139629] systemd-udevd[80]: Using default interface naming scheme 'v261'.1921client # [ 2.156192] systemd[1]: Finished Load Kernel Modules.1922client # [ 2.159361] systemd-udevd[78]: Using default interface naming scheme 'v261'.1923lb # [ 2.160813] systemd[1]: Finished Create System Files and Directories.1924worker2 # [ 2.134357] systemd[1]: Finished Apply Kernel Variables.1925client # [ 2.178142] systemd[1]: Starting Apply Kernel Variables...1926worker2 # [ 2.142587] systemd-udevd[78]: Using default interface naming scheme 'v261'.1927worker1 # [ 2.189606] systemd[1]: Started Journal Service.1928lb # [ 2.178034] systemd[1]: Started Rule-based Manager for Device Events and Files.1929worker2 # [ 2.156099] systemd[1]: Finished Create System Files and Directories.1930client # [ 2.206406] systemd[1]: Finished Create System Files and Directories.1931worker1 # [ 2.196475] systemd-modules-load[73]: Inserted module 'dm_mod'1932worker1 # [ 2.197809] systemd-modules-load[73]: Module 'virtio_balloon' is built in1933worker1 # [ 2.198987] systemd-modules-load[73]: Module 'virtio_console' is built in1934client # [ 2.212575] systemd[1]: Finished Apply Kernel Variables.1935worker2 # [ 2.178441] systemd[1]: Started Rule-based Manager for Device Events and Files.1936worker1 # [ 2.210076] systemd-modules-load[73]: Inserted module 'virtio_gpu'1937worker1 # [ 2.211997] systemd-modules-load[73]: Module 'virtio_rng' is built in1938client # [ 2.227089] systemd[1]: Started Rule-based Manager for Device Events and Files.1939worker1 # [ 2.216304] systemd[1]: Starting Create System Files and Directories...1940worker1 # [ 2.224593] systemd[1]: Finished Apply Kernel Variables.1941worker1 # [ 2.232632] systemd-udevd[78]: Using default interface naming scheme 'v261'.1942lb # [ 2.241282] systemd[1]: Starting Virtual Console Setup...1943worker1 # [ 2.252108] systemd[1]: Finished Create System Files and Directories.1944worker1 # [ 2.266383] systemd[1]: Started Rule-based Manager for Device Events and Files.1945worker2 # [ 2.240482] systemd[1]: Starting Virtual Console Setup...1946client # [ 2.288267] systemd[1]: Starting Virtual Console Setup...1947lb # [ 2.288516] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1948lb # [ 2.291951] systemd[1]: Finished Virtual Console Setup.1949worker2 # [ 2.288479] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1950worker2 # [ 2.291883] systemd[1]: Finished Virtual Console Setup.1951client # [ 2.338174] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1952client # [ 2.340078] systemd[1]: Finished Virtual Console Setup.1953worker1 # [ 2.328409] systemd[1]: Starting Virtual Console Setup...1954worker1 # [ 2.388487] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1955worker1 # [ 2.391902] systemd[1]: Finished Virtual Console Setup.1956lb # [ 2.899332] systemd[1]: Finished Coldplug All udev Devices.1957lb # [ 2.900399] systemd[1]: Reached target System Initialization.1958lb # [ 2.901202] systemd[1]: Reached target Basic System.1959client # [ 2.927261] systemd[1]: Finished Coldplug All udev Devices.1960client # [ 2.932176] systemd[1]: Reached target System Initialization.1961worker2 # [ 2.890352] systemd[1]: Finished Coldplug All udev Devices.1962client # [ 2.933002] systemd[1]: Reached target Basic System.1963worker2 # [ 2.891321] systemd[1]: Reached target System Initialization.1964worker2 # [ 2.896131] systemd[1]: Reached target Basic System.1965worker1 # [ 2.976198] systemd[1]: Finished Coldplug All udev Devices.1966worker1 # [ 2.977979] systemd[1]: Reached target System Initialization.1967worker1 # [ 2.979076] systemd[1]: Reached target Basic System.1968lb # [ 3.040618] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1969worker2 # [ 3.044352] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1970lb # [ 3.074032] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1971lb # [ 3.080483] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1972client # [ 3.100628] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973client # [ 3.105434] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1974client # [ 3.124631] (udev-worker)[105]: Network interface NamePolicy= disabled on kernel command line.1975worker2 # [ 3.085192] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1976worker2 # [ 3.094749] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1977worker1 # [ 3.124372] (udev-worker)[104]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1978worker1 # [ 3.161629] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line.1979lb # [ 3.166022] systemd[1]: Found device /dev/disk/by-label/nixos.1980worker1 # [ 3.170474] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1981lb # [ 3.169896] systemd[1]: Reached target Initrd Root Device.1982lb # [ 3.172712] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1983worker2 # [ 3.163326] systemd[1]: Found device /dev/disk/by-label/nixos.1984client # [ 3.207385] systemd[1]: Found device /dev/disk/by-label/nixos.1985client # [ 3.208971] systemd[1]: Reached target Initrd Root Device.1986worker2 # [ 3.168172] systemd[1]: Reached target Initrd Root Device.1987client # [ 3.210871] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1988worker2 # [ 3.172122] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989lb # [ 3.217124] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1990worker1 # [ 3.225948] systemd[1]: Found device /dev/disk/by-label/nixos.1991lb # [ 3.225114] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1992lb # [ 3.228163] systemd[1]: Mounting /sysroot...1993worker1 # [ 3.231321] systemd[1]: Reached target Initrd Root Device.1994worker1 # [ 3.235966] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1995client # [ 3.255453] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1996worker2 # [ 3.221172] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1997client # [ 3.263156] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1998client # [ 3.264997] systemd[1]: Mounting /sysroot...1999worker2 # [ 3.229535] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.2000worker2 # [ 3.231565] systemd[1]: Mounting /sysroot...
loading 2,046 hidden lines…
4047worker1 # deleting specified paths...4048worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4049worker1 # deleting unused links...4050worker1 # note: hard linking is currently saving 0.0 KiB4051worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds)4052worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4053worker2 # finding garbage collector roots...4054worker2 # deleting specified paths...4055worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4056worker2 # deleting unused links...4057worker2 # note: hard linking is currently saving 0.0 KiB4058worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)4059client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24060client # don't know how to build these paths:4061client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4062worker1 # [ 74.749644] nix-grpc-daemon[486]: ts=2026-09-24T12:26:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588364063worker1 # [ 74.757535] nix-grpc-daemon[486]: ts=2026-09-24T12:26:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364064worker1 # [ 74.781648] nix-grpc-daemon[486]: ts=2026-09-24T12:26:49Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14065client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.29 seconds)4066client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4067client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.14 seconds)4068lb: must succeed: curl -sf localhost:9901/clusters4069lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4070lb: must succeed: curl -sf localhost:9901/clusters4071lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4072worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4073worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4074(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.14 seconds)4075subtest: inputs already in the cache are not sent to the scheduler4076lb: must succeed: curl -sf localhost:9901/clusters4077lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4078lb: must succeed: curl -sf localhost:9901/clusters4079lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4080worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4081worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4082client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24083worker2 # [ 75.572731] nix-grpc-daemon[786]: ts=2026-09-24T12:26:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:431324084client # don't know how to build these paths:4085client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4086worker1 # [ 75.613866] nix-grpc-daemon[486]: ts=2026-09-24T12:26:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364087worker1 # [ 75.630774] nix-grpc-daemon[486]: ts=2026-09-24T12:26:50Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04088worker2 # [ 75.604228] nix-grpc-daemon[786]: ts=2026-09-24T12:26:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324089worker1 # [ 75.637100] nix-grpc-daemon[486]: ts=2026-09-24T12:26:50Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14090worker2 # [ 75.620261] nix-grpc-daemon[786]: ts=2026-09-24T12:26:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324091client # copying 1 paths to 'lb:50051'...4092worker2 # [ 75.635988] nix-grpc-daemon[856]: time=2026-09-24T12:26:50.183Z level=INFO msg="All 2 paths already cached"4093client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4094worker2 # [ 75.643431] nix-grpc-daemon[786]: ts=2026-09-24T12:26:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431324095worker2 # [ 75.658771] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4096worker2 # [ 75.821670] nix-daemon[844]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4097worker2 # [ 75.844306] nix-daemon[844]: reaped child process 984, status = succeeded4098lb # [ 75.870808] niks3-server[1025]: 2026/09/24 12:26:50 INFO Received push request method=POST path=/api/pushes4099worker2 # [ 75.855221] nix-grpc-daemon[856]: time=2026-09-24T12:26:50.403Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4100worker2 # [ 75.857059] nix-grpc-daemon[856]: time=2026-09-24T12:26:50.403Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4101lb # [ 75.906649] niks3-server[1025]: 2026/09/24 12:26:50 INFO Received sign narinfos request method=POST path=/api/pushes/7/sign4102worker2 # [ 75.886547] nix-grpc-daemon[856]: time=2026-09-24T12:26:50.434Z level=INFO msg="Uploading 1 narinfos"4103lb # [ 75.918541] niks3-server[1025]: 2026/09/24 12:26:50 INFO Signed narinfos id=7 count=14104lb # [ 75.927734] niks3-server[1025]: 2026/09/24 12:26:50 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4105lb # [ 75.937766] niks3-server[1025]: 2026/09/24 12:26:50 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4106lb # [ 75.954919] niks3-server[1025]: 2026/09/24 12:26:50 INFO Received complete push request method=POST path=/api/pushes/7/complete4107worker2 # [ 75.932378] nix-grpc-daemon[856]: time=2026-09-24T12:26:50.480Z level=INFO msg="Upload complete. (256ms)"4108lb # [ 75.961413] niks3-server[1025]: 2026/09/24 12:26:50 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4109worker2 # [ 75.943194] nix-grpc-daemon[786]: ts=2026-09-24T12:26:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=0 paths=1 nar_bytes_in=7524110worker2 # [ 75.960911] nix-grpc-daemon[786]: ts=2026-09-24T12:26:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324111worker2 # [ 75.968571] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4112worker2 # [ 75.979490] nix-daemon[844]: reaped child process 979, status = succeeded4113client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4114worker2 # [ 75.987260] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4115client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4116client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4117client # farm-top> LOG-farm-top-t1b4118worker2 # [ 77.275828] nix-daemon[844]: reaped child process 988, status = succeeded4119worker2 # [ 77.342265] nix-daemon[844]: accepted connection from pid 1004, user nix-grpc-daemon (trusted)4120worker2 # [ 77.362940] nix-daemon[844]: reaped child process 1007, status = succeeded4121lb # [ 77.398184] niks3-server[1025]: 2026/09/24 12:26:51 INFO Received push request method=POST path=/api/pushes4122worker2 # [ 77.378545] nix-grpc-daemon[856]: time=2026-09-24T12:26:51.926Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4123worker2 # [ 77.383085] nix-grpc-daemon[856]: time=2026-09-24T12:26:51.928Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4124lb # [ 77.440260] niks3-server[1025]: 2026/09/24 12:26:51 INFO Received sign narinfos request method=POST path=/api/pushes/8/sign4125worker2 # [ 77.422424] nix-grpc-daemon[856]: time=2026-09-24T12:26:51.970Z level=INFO msg="Uploading 1 narinfos"4126lb # [ 77.455625] niks3-server[1025]: 2026/09/24 12:26:51 INFO Signed narinfos id=8 count=14127lb # [ 77.462458] niks3-server[1025]: 2026/09/24 12:26:52 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4128lb # [ 77.475620] niks3-server[1025]: 2026/09/24 12:26:52 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4129lb # [ 77.483379] niks3-server[1025]: 2026/09/24 12:26:52 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4130worker2 # [ 77.456302] nix-grpc-daemon[856]: time=2026-09-24T12:26:52.004Z level=INFO msg="Upload complete. (259ms)"4131lb # [ 77.488117] niks3-server[1025]: 2026/09/24 12:26:52 INFO Received complete push request method=POST path=/api/pushes/8/complete4132lb # [ 77.493527] niks3-server[1025]: 2026/09/24 12:26:52 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4133worker2 # [ 77.466834] nix-grpc-daemon[786]: ts=2026-09-24T12:26:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04134worker2 # [ 77.487199] nix-daemon[844]: reaped child process 986, status = succeeded4135client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.17 seconds)4136client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4137client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4138lb: must succeed: curl -sf localhost:9901/clusters4139lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4140lb: must succeed: curl -sf localhost:9901/clusters4141lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4142worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4143worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4144(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.68 seconds)4145subtest: outputs read back through any worker carry the cache signature4146client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24147worker2 # [ 78.077050] nix-grpc-daemon[786]: ts=2026-09-24T12:26:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:431324148client # don't know how to build these paths:4149client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4150worker1 # [ 78.117978] nix-grpc-daemon[486]: ts=2026-09-24T12:26:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364151worker1 # [ 78.141506] nix-grpc-daemon[486]: ts=2026-09-24T12:26:52Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04152worker1 # [ 78.148814] nix-grpc-daemon[486]: ts=2026-09-24T12:26:52Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=14153worker1 # [ 78.156447] nix-grpc-daemon[486]: ts=2026-09-24T12:26:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364154worker1 # [ 78.171361] nix-grpc-daemon[486]: ts=2026-09-24T12:26:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364155client # copying 1 paths to 'lb:50051'...4156client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4157worker1 # [ 78.187638] nix-grpc-daemon[486]: ts=2026-09-24T12:26:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588364158worker1 # [ 78.203389] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4159worker1 # [ 78.231416] nix-grpc-daemon[486]: ts=2026-09-24T12:26:52Z level=info event=niks3_push_started pid=9144160worker1 # [ 78.408438] nix-grpc-daemon[914]: time=2026-09-24T12:26:52.938Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4161worker1 # [ 78.517941] nix-daemon[855]: accepted connection from pid 927, user nix-grpc-daemon (trusted)4162worker1 # [ 78.540982] nix-daemon[855]: reaped child process 930, status = succeeded4163lb # [ 78.541868] niks3-server[1025]: 2026/09/24 12:26:53 INFO Received push request method=POST path=/api/pushes4164worker1 # [ 78.554953] nix-grpc-daemon[914]: time=2026-09-24T12:26:53.084Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4165worker1 # [ 78.558732] nix-grpc-daemon[914]: time=2026-09-24T12:26:53.084Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4166lb # [ 78.580359] niks3-server[1025]: 2026/09/24 12:26:53 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4167worker1 # [ 78.594553] nix-grpc-daemon[914]: time=2026-09-24T12:26:53.124Z level=INFO msg="Uploading 1 narinfos"4168lb # [ 78.591740] niks3-server[1025]: 2026/09/24 12:26:53 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4169lb # [ 78.604276] niks3-server[1025]: 2026/09/24 12:26:53 INFO Signed narinfos id=9 count=14170lb # [ 78.607219] niks3-server[1025]: 2026/09/24 12:26:53 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4171lb # [ 78.624980] niks3-server[1025]: 2026/09/24 12:26:53 INFO Received complete push request method=POST path=/api/pushes/9/complete4172worker1 # [ 78.633531] nix-grpc-daemon[914]: time=2026-09-24T12:26:53.164Z level=INFO msg="Upload complete. (220ms)"4173lb # [ 78.632263] niks3-server[1025]: 2026/09/24 12:26:53 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4174worker1 # [ 78.642821] nix-grpc-daemon[486]: ts=2026-09-24T12:26:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_in=5124175worker1 # [ 78.652472] nix-grpc-daemon[486]: ts=2026-09-24T12:26:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364176worker1 # [ 78.664454] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4177worker1 # [ 78.669417] nix-daemon[855]: reaped child process 912, status = succeeded4178client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4179worker1 # [ 78.679268] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4180client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4181lb # [ 79.245030] envoy[906]: conn peer=[2001:db8:1::1]:59890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9476 tx=4382 ms=49874182lb # [ 79.250204] envoy[906]: conn peer=[2001:db8:1::1]:33432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=554183lb # [ 79.255008] envoy[906]: conn peer=[2001:db8:1::1]:33444 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1919 ms=19134184client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4185client # farm-a> LOG-farm-a-sig4186worker1 # [ 80.360315] nix-daemon[855]: reaped child process 934, status = succeeded4187worker1 # [ 80.475309] nix-daemon[855]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4188worker1 # [ 80.491930] nix-daemon[855]: reaped child process 953, status = succeeded4189lb # [ 80.497381] niks3-server[1025]: 2026/09/24 12:26:55 INFO Received push request method=POST path=/api/pushes4190worker1 # [ 80.512463] nix-grpc-daemon[914]: time=2026-09-24T12:26:55.040Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4191worker1 # [ 80.514321] nix-grpc-daemon[914]: time=2026-09-24T12:26:55.040Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4192lb # [ 80.529590] niks3-server[1025]: 2026/09/24 12:26:55 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4193worker1 # [ 80.553069] nix-grpc-daemon[914]: time=2026-09-24T12:26:55.083Z level=INFO msg="Uploading 1 narinfos"4194lb # [ 80.549980] niks3-server[1025]: 2026/09/24 12:26:55 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4195lb # [ 80.560203] niks3-server[1025]: 2026/09/24 12:26:55 INFO Signed narinfos id=10 count=14196lb # [ 80.566943] niks3-server[1025]: 2026/09/24 12:26:55 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4197lb # [ 80.578373] niks3-server[1025]: 2026/09/24 12:26:55 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4198lb # [ 80.590653] niks3-server[1025]: 2026/09/24 12:26:55 INFO Received complete push request method=POST path=/api/pushes/10/complete4199worker1 # [ 80.599496] nix-grpc-daemon[914]: time=2026-09-24T12:26:55.130Z level=INFO msg="Upload complete. (308ms)"4200lb # [ 80.597861] niks3-server[1025]: 2026/09/24 12:26:55 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4201worker1 # [ 80.620666] nix-grpc-daemon[486]: ts=2026-09-24T12:26:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04202worker1 # [ 80.640333] nix-grpc-daemon[486]: ts=2026-09-24T12:26:55Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04203worker1 # [ 80.650938] nix-grpc-daemon[486]: ts=2026-09-24T12:26:55Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14204worker1 # [ 80.656812] nix-grpc-daemon[486]: ts=2026-09-24T12:26:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364205worker1 # [ 80.662575] nix-grpc-daemon[486]: ts=2026-09-24T12:26:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364206worker1 # [ 80.668323] nix-daemon[855]: reaped child process 932, status = succeeded4207client # copying 1 paths to 'lb:50051'...4208worker1 # [ 80.670641] nix-grpc-daemon[914]: time=2026-09-24T12:26:55.197Z level=INFO msg="All 1 paths already cached"4209client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4210worker1 # [ 80.674848] nix-grpc-daemon[486]: ts=2026-09-24T12:26:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588364211worker1 # [ 80.690351] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4212worker1 # [ 80.821909] nix-daemon[855]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4213worker1 # [ 80.840978] nix-daemon[855]: reaped child process 962, status = succeeded4214lb # [ 80.843844] niks3-server[1025]: 2026/09/24 12:26:55 INFO Received push request method=POST path=/api/pushes4215worker1 # [ 80.855170] nix-grpc-daemon[914]: time=2026-09-24T12:26:55.385Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4216worker1 # [ 80.856764] nix-grpc-daemon[914]: time=2026-09-24T12:26:55.385Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4217lb # [ 80.872989] niks3-server[1025]: 2026/09/24 12:26:55 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4218worker1 # [ 80.885112] nix-grpc-daemon[914]: time=2026-09-24T12:26:55.415Z level=INFO msg="Uploading 1 narinfos"4219lb # [ 80.887871] niks3-server[1025]: 2026/09/24 12:26:55 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4220lb # [ 80.897834] niks3-server[1025]: 2026/09/24 12:26:55 INFO Signed narinfos id=11 count=14221lb # [ 80.909375] niks3-server[1025]: 2026/09/24 12:26:55 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4222worker1 # [ 80.918760] nix-grpc-daemon[914]: time=2026-09-24T12:26:55.449Z level=INFO msg="Upload complete. (220ms)"4223lb # [ 80.917073] niks3-server[1025]: 2026/09/24 12:26:55 INFO Received complete push request method=POST path=/api/pushes/11/complete4224worker1 # [ 80.926838] nix-grpc-daemon[486]: ts=2026-09-24T12:26:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_in=6244225lb # [ 80.926049] niks3-server[1025]: 2026/09/24 12:26:55 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4226worker1 # [ 80.942482] nix-grpc-daemon[486]: ts=2026-09-24T12:26:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364227worker1 # [ 80.948933] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4228worker1 # [ 80.954039] nix-daemon[855]: reaped child process 957, status = succeeded4229client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4230client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4231worker1 # [ 80.966740] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4232client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4233client # farm-b> LOG-farm-b-sig4234worker1 # [ 82.240865] nix-daemon[855]: reaped child process 966, status = succeeded4235worker1 # [ 82.325871] nix-daemon[855]: accepted connection from pid 982, user nix-grpc-daemon (trusted)4236worker1 # [ 82.348193] nix-daemon[855]: reaped child process 985, status = succeeded4237lb # [ 82.351274] niks3-server[1025]: 2026/09/24 12:26:56 INFO Received push request method=POST path=/api/pushes4238worker1 # [ 82.365030] nix-grpc-daemon[914]: time=2026-09-24T12:26:56.894Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4239worker1 # [ 82.366511] nix-grpc-daemon[914]: time=2026-09-24T12:26:56.894Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4240lb # [ 82.393410] niks3-server[1025]: 2026/09/24 12:26:56 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4241worker1 # [ 82.405729] nix-grpc-daemon[914]: time=2026-09-24T12:26:56.936Z level=INFO msg="Uploading 1 narinfos"4242lb # [ 82.405610] niks3-server[1025]: 2026/09/24 12:26:56 INFO Signed narinfos id=12 count=14243lb # [ 82.416103] niks3-server[1025]: 2026/09/24 12:26:56 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4244lb # [ 82.428262] niks3-server[1025]: 2026/09/24 12:26:56 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4245worker1 # [ 82.445586] nix-grpc-daemon[914]: time=2026-09-24T12:26:56.976Z level=INFO msg="Upload complete. (268ms)"4246lb # [ 82.443505] niks3-server[1025]: 2026/09/24 12:26:56 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4247lb # [ 82.451616] niks3-server[1025]: 2026/09/24 12:26:56 INFO Received complete push request method=POST path=/api/pushes/12/complete4248lb # [ 82.455627] niks3-server[1025]: 2026/09/24 12:26:56 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4249worker1 # [ 82.460191] nix-grpc-daemon[486]: ts=2026-09-24T12:26:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04250worker1 # [ 82.481918] nix-grpc-daemon[486]: ts=2026-09-24T12:26:57Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04251worker1 # [ 82.488990] nix-grpc-daemon[486]: ts=2026-09-24T12:26:57Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14252worker1 # [ 82.494196] nix-grpc-daemon[486]: ts=2026-09-24T12:26:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364253worker1 # [ 82.499134] nix-daemon[855]: reaped child process 964, status = succeeded4254worker1 # [ 82.502779] nix-grpc-daemon[486]: ts=2026-09-24T12:26:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364255worker1 # [ 82.514224] nix-grpc-daemon[914]: time=2026-09-24T12:26:57.044Z level=INFO msg="All 2 paths already cached"4256client # copying 1 paths to 'lb:50051'...4257client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4258worker1 # [ 82.524278] nix-grpc-daemon[486]: ts=2026-09-24T12:26:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588364259worker1 # [ 82.541858] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4260worker1 # [ 82.680779] nix-daemon[855]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4261worker1 # [ 82.700402] nix-daemon[855]: reaped child process 994, status = succeeded4262lb # [ 82.700783] niks3-server[1025]: 2026/09/24 12:26:57 INFO Received push request method=POST path=/api/pushes4263worker1 # [ 82.712750] nix-grpc-daemon[914]: time=2026-09-24T12:26:57.243Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4264worker1 # [ 82.714348] nix-grpc-daemon[914]: time=2026-09-24T12:26:57.243Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4265lb # [ 82.731487] niks3-server[1025]: 2026/09/24 12:26:57 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4266worker1 # [ 82.740457] nix-grpc-daemon[914]: time=2026-09-24T12:26:57.270Z level=INFO msg="Uploading 1 narinfos"4267lb # [ 82.746172] niks3-server[1025]: 2026/09/24 12:26:57 INFO Signed narinfos id=13 count=14268lb # [ 82.755148] niks3-server[1025]: 2026/09/24 12:26:57 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4269lb # [ 82.764742] niks3-server[1025]: 2026/09/24 12:26:57 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4270lb # [ 82.777689] niks3-server[1025]: 2026/09/24 12:26:57 INFO Received complete push request method=POST path=/api/pushes/13/complete4271lb # [ 82.783289] niks3-server[1025]: 2026/09/24 12:26:57 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4272worker1 # [ 82.790532] nix-grpc-daemon[914]: time=2026-09-24T12:26:57.321Z level=INFO msg="Upload complete. (234ms)"4273worker1 # [ 82.800760] nix-grpc-daemon[486]: ts=2026-09-24T12:26:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_in=7524274worker1 # [ 82.817491] nix-grpc-daemon[486]: ts=2026-09-24T12:26:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364275worker1 # [ 82.828379] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4276worker1 # [ 82.832913] nix-daemon[855]: reaped child process 989, status = succeeded4277client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4278worker1 # [ 82.842530] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4279client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4280client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4281client # farm-top> LOG-farm-top-sig4282worker1 # [ 84.089822] nix-daemon[855]: reaped child process 998, status = succeeded4283worker1 # [ 84.199434] nix-daemon[855]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4284worker1 # [ 84.215187] nix-daemon[855]: reaped child process 1017, status = succeeded4285lb # [ 84.221418] niks3-server[1025]: 2026/09/24 12:26:58 INFO Received push request method=POST path=/api/pushes4286worker1 # [ 84.234877] nix-grpc-daemon[914]: time=2026-09-24T12:26:58.764Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4287worker1 # [ 84.237009] nix-grpc-daemon[914]: time=2026-09-24T12:26:58.764Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4288lb # [ 84.259699] niks3-server[1025]: 2026/09/24 12:26:58 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4289worker1 # [ 84.270248] nix-grpc-daemon[914]: time=2026-09-24T12:26:58.800Z level=INFO msg="Uploading 1 narinfos"4290lb # [ 84.271754] niks3-server[1025]: 2026/09/24 12:26:58 INFO Signed narinfos id=14 count=14291lb # [ 84.281453] niks3-server[1025]: 2026/09/24 12:26:58 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4292lb # [ 84.290240] niks3-server[1025]: 2026/09/24 12:26:58 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4293lb # [ 84.301438] niks3-server[1025]: 2026/09/24 12:26:58 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4294worker1 # [ 84.309821] nix-grpc-daemon[914]: time=2026-09-24T12:26:58.840Z level=INFO msg="Upload complete. (255ms)"4295lb # [ 84.309090] niks3-server[1025]: 2026/09/24 12:26:58 INFO Received complete push request method=POST path=/api/pushes/14/complete4296lb # [ 84.313346] niks3-server[1025]: 2026/09/24 12:26:58 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4297worker1 # [ 84.325054] nix-grpc-daemon[486]: ts=2026-09-24T12:26:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04298worker1 # [ 84.344546] nix-daemon[855]: reaped child process 996, status = succeeded4299client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.48 seconds)4300client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4301client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.18 seconds)4302client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4303worker1 # [ 84.676573] nix-grpc-daemon[486]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:607984304worker1 # [ 84.685021] nix-grpc-daemon[486]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:607984305client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.15 seconds)4306client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4307worker2 # [ 84.801864] nix-grpc-daemon[786]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:524424308client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4309client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4310client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4311client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4312worker2 # [ 84.825630] nix-grpc-daemon[786]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:524424313worker2 # [ 84.834762] nix-grpc-daemon[786]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4314worker2 # [ 84.850920] nix-grpc-daemon[786]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4315worker2 # [ 84.867719] nix-grpc-daemon[786]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4316client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.24 seconds)4317client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4318client # finding garbage collector roots...4319client # removing stale temporary roots file "/nix/var/nix/temproots/855"4320client # removing stale temporary roots file "/nix/var/nix/temproots/931"4321client # removing stale temporary roots file "/nix/var/nix/temproots/891"4322client # removing stale temporary roots file "/nix/var/nix/temproots/870"4323client # removing stale temporary roots file "/nix/var/nix/temproots/923"4324client # removing stale temporary roots file "/nix/var/nix/temproots/946"4325client # removing stale temporary roots file "/nix/var/nix/temproots/910"4326client # removing stale temporary roots file "/nix/var/nix/temproots/902"4327client # deleting specified paths...4328client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4329client # deleting unused links...4330client # note: hard linking is currently saving 0.0 KiB4331worker2 # [ 85.239770] nix-grpc-daemon[786]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:431324332worker1 # [ 85.274046] nix-grpc-daemon[486]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364333worker1 # [ 85.285797] nix-grpc-daemon[914]: time=2026-09-24T12:26:59.815Z level=INFO msg="All 1 paths already cached"4334worker1 # [ 85.294814] nix-grpc-daemon[486]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588364335worker2 # [ 85.274312] nix-grpc-daemon[786]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:431324336client # copying 3 paths...4337worker1 # [ 85.318998] nix-grpc-daemon[486]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588364338client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4339worker2 # [ 85.315926] nix-grpc-daemon[786]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:431324340worker2 # [ 85.323223] nix-grpc-daemon[786]: ts=2026-09-24T12:26:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=0 paths=1 nar_bytes_out=2324341worker1 # [ 85.364974] nix-grpc-daemon[486]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588364342worker1 # [ 85.372987] nix-grpc-daemon[486]: ts=2026-09-24T12:26:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_out=1764343worker1 # [ 85.382634] nix-grpc-daemon[486]: ts=2026-09-24T12:26:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588364344client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4345worker1 # [ 85.387706] nix-grpc-daemon[486]: ts=2026-09-24T12:26:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_out=1284346client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4347client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.48 seconds)4348(finished: subtest: outputs read back through any worker carry the cache signature, in 7.53 seconds)4349subtest: two clients wanting the same drv build it once4350client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4351client # [ 85.478846] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4352client # Running as unit: dedup1.service; invocation ID: b6629b0d2d3a46d6b759513c2ce76fc14353client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.06 seconds)4354client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4355worker2 # [ 85.614951] nix-grpc-daemon[786]: ts=2026-09-24T12:27:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:431324356client # [ 85.664898] nix[999]: don't know how to build these paths:4357client # [ 85.665877] nix[999]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4358worker1 # [ 85.661448] nix-grpc-daemon[486]: ts=2026-09-24T12:27:00Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04359worker1 # [ 85.667757] nix-grpc-daemon[486]: ts=2026-09-24T12:27:00Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14360worker2 # [ 85.640950] nix-grpc-daemon[786]: ts=2026-09-24T12:27:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324361worker2 # [ 85.655691] nix-grpc-daemon[786]: ts=2026-09-24T12:27:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324362client # [ 85.711456] nix[999]: copying 1 paths to 'lb:50051'...4363client # [ 85.713086] nix[999]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4364worker2 # [ 85.670016] nix-grpc-daemon[786]: ts=2026-09-24T12:27:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431324365worker2 # [ 85.685113] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4366worker2 # [ 85.848811] nix-daemon[844]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4367worker2 # [ 85.867532] nix-daemon[844]: reaped child process 1017, status = succeeded4368lb # [ 85.902277] niks3-server[1025]: 2026/09/24 12:27:00 INFO Received push request method=POST path=/api/pushes4369worker2 # [ 85.885245] nix-grpc-daemon[856]: time=2026-09-24T12:27:00.433Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4370worker2 # [ 85.886901] nix-grpc-daemon[856]: time=2026-09-24T12:27:00.433Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4371lb # [ 85.931195] niks3-server[1025]: 2026/09/24 12:27:00 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4372worker2 # [ 85.914071] nix-grpc-daemon[856]: time=2026-09-24T12:27:00.462Z level=INFO msg="Uploading 1 narinfos"4373lb # [ 85.946534] niks3-server[1025]: 2026/09/24 12:27:00 INFO Signed narinfos id=15 count=14374lb # [ 85.953442] niks3-server[1025]: 2026/09/24 12:27:00 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4375lb # [ 85.965025] niks3-server[1025]: 2026/09/24 12:27:00 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4376lb # [ 85.978285] niks3-server[1025]: 2026/09/24 12:27:00 INFO Received complete push request method=POST path=/api/pushes/15/complete4377worker2 # [ 85.957440] nix-grpc-daemon[856]: time=2026-09-24T12:27:00.505Z level=INFO msg="Upload complete. (255ms)"4378lb # [ 85.986443] niks3-server[1025]: 2026/09/24 12:27:00 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4379worker2 # [ 85.969071] nix-grpc-daemon[786]: ts=2026-09-24T12:27:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=0 paths=1 nar_bytes_in=4324380worker2 # [ 85.985667] nix-grpc-daemon[786]: ts=2026-09-24T12:27:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324381worker2 # [ 85.992392] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4382worker2 # [ 85.997593] nix-daemon[844]: reaped child process 1012, status = succeeded4383client # [ 86.049594] nix[999]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4384worker2 # [ 86.010619] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4385client # [ 86.563024] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4386client # Running as unit: dedup2.service; invocation ID: 54c9d436fb674c1aa2b71c3896b1d8ce4387client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.10 seconds)4388worker1 # [ 86.788870] nix-grpc-daemon[486]: ts=2026-09-24T12:27:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588364389client # [ 86.810956] nix[1015]: don't know how to build these paths:4390client # [ 86.812093] nix[1015]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4391worker1 # [ 86.806575] nix-grpc-daemon[486]: ts=2026-09-24T12:27:01Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04392worker2 # [ 86.780346] nix-grpc-daemon[786]: ts=2026-09-24T12:27:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324393client # [ 86.829475] nix[1015]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4394client: must succeed: sleep 24395lb # [ 89.244677] envoy[906]: conn peer=[2001:db8:1::1]:33450 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9541 tx=4372 ms=62304396lb # [ 89.250110] envoy[906]: conn peer=[2001:db8:1::1]:43260 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1444397lb # [ 89.255008] envoy[906]: conn peer=[2001:db8:1::1]:43270 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=684398lb # [ 89.259805] envoy[906]: conn peer=[2001:db8:1::1]:43288 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=374399lb # [ 89.264561] envoy[906]: conn peer=[2001:db8:1::1]:43274 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=514400client: (finished: must succeed: sleep 2, in 2.05 seconds)4401client: waiting for success: ! systemctl is-active dedup1 dedup24402worker2 # [ 90.272312] nix-daemon[844]: reaped child process 1021, status = succeeded4403worker2 # [ 90.408642] nix-daemon[844]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)4404worker2 # [ 90.429154] nix-daemon[844]: reaped child process 1054, status = succeeded4405lb # [ 90.462276] niks3-server[1025]: 2026/09/24 12:27:05 INFO Received push request method=POST path=/api/pushes4406worker2 # [ 90.447667] nix-grpc-daemon[856]: time=2026-09-24T12:27:04.993Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4407worker2 # [ 90.451838] nix-grpc-daemon[856]: time=2026-09-24T12:27:04.993Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4408lb # [ 90.508796] niks3-server[1025]: 2026/09/24 12:27:05 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4409lb # [ 90.521423] niks3-server[1025]: 2026/09/24 12:27:05 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4410worker2 # [ 90.504439] nix-grpc-daemon[856]: time=2026-09-24T12:27:05.052Z level=INFO msg="Uploading 1 narinfos"4411lb # [ 90.532864] niks3-server[1025]: 2026/09/24 12:27:05 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4412lb # [ 90.540904] niks3-server[1025]: 2026/09/24 12:27:05 INFO Signed narinfos id=16 count=14413lb # [ 90.545898] niks3-server[1025]: 2026/09/24 12:27:05 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4414lb # [ 90.555823] niks3-server[1025]: 2026/09/24 12:27:05 INFO Received complete push request method=POST path=/api/pushes/16/complete4415lb # [ 90.565539] niks3-server[1025]: 2026/09/24 12:27:05 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4416worker2 # [ 90.537361] nix-grpc-daemon[856]: time=2026-09-24T12:27:05.085Z level=INFO msg="Upload complete. (311ms)"4417worker2 # [ 90.549374] nix-grpc-daemon[786]: ts=2026-09-24T12:27:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04418worker2 # [ 90.571873] nix-grpc-daemon[786]: ts=2026-09-24T12:27:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04419worker2 # [ 90.582124] nix-daemon[844]: reaped child process 1019, status = succeeded4420client # [ 90.650230] systemd[1]: dedup2.service: Deactivated successfully.4421client # [ 90.654129] systemd[1]: dedup2.service: Consumed 212ms CPU time over 4.090s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4422client # [ 90.663281] systemd[1]: dedup1.service: Deactivated successfully.4423client # [ 90.666914] systemd[1]: dedup1.service: Consumed 163ms CPU time over 5.183s wall clock time, 43.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4424client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4425client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4426client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4427worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4428worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4429worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4430worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4431worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4432worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4433worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4434worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4435(finished: subtest: two clients wanting the same drv build it once, in 6.09 seconds)4436subtest: --timeout and --max-silent-time reach the builder4437worker1 # [ 91.739956] nix-grpc-daemon[486]: ts=2026-09-24T12:27:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588364438worker1 # [ 91.753781] nix-grpc-daemon[486]: ts=2026-09-24T12:27:06Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04439worker1 # [ 91.761010] nix-grpc-daemon[486]: ts=2026-09-24T12:27:06Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14440worker1 # [ 91.769717] nix-grpc-daemon[486]: ts=2026-09-24T12:27:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364441worker1 # [ 91.780564] nix-grpc-daemon[486]: ts=2026-09-24T12:27:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364442worker1 # [ 91.794595] nix-grpc-daemon[486]: ts=2026-09-24T12:27:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588364443worker1 # [ 91.810087] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4444worker1 # [ 91.968114] nix-daemon[855]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)4445worker1 # [ 91.986435] nix-daemon[855]: reaped child process 1051, status = succeeded4446lb # [ 91.989538] niks3-server[1025]: 2026/09/24 12:27:06 INFO Received push request method=POST path=/api/pushes4447worker1 # [ 92.002091] nix-grpc-daemon[914]: time=2026-09-24T12:27:06.532Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4448worker1 # [ 92.003550] nix-grpc-daemon[914]: time=2026-09-24T12:27:06.532Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4449lb # [ 92.018413] niks3-server[1025]: 2026/09/24 12:27:06 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4450worker1 # [ 92.030052] nix-grpc-daemon[914]: time=2026-09-24T12:27:06.560Z level=INFO msg="Uploading 1 narinfos"4451lb # [ 92.032877] niks3-server[1025]: 2026/09/24 12:27:06 INFO Signed narinfos id=17 count=14452lb # [ 92.041787] niks3-server[1025]: 2026/09/24 12:27:06 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4453lb # [ 92.057974] niks3-server[1025]: 2026/09/24 12:27:06 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4454lb # [ 92.067285] niks3-server[1025]: 2026/09/24 12:27:06 INFO Received complete push request method=POST path=/api/pushes/17/complete4455worker1 # [ 92.068648] nix-grpc-daemon[914]: time=2026-09-24T12:27:06.599Z level=INFO msg="Upload complete. (241ms)"4456lb # [ 92.073895] niks3-server[1025]: 2026/09/24 12:27:06 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4457worker1 # [ 92.080471] nix-grpc-daemon[486]: ts=2026-09-24T12:27:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_in=4244458worker1 # [ 92.096644] nix-grpc-daemon[486]: ts=2026-09-24T12:27:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364459worker1 # [ 92.105061] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4460worker1 # [ 92.110721] nix-daemon[855]: reaped child process 1046, status = succeeded4461worker1 # [ 92.122177] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4462worker1 # [ 98.143585] nix-grpc-daemon[486]: ts=2026-09-24T12:27:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04463worker1 # [ 98.187950] nix-daemon[855]: reaped child process 1053, status = succeeded4464worker1 # [ 98.191927] nix-daemon[855]: reaped child process 1055, status = succeeded4465worker2 # [ 98.373657] nix-grpc-daemon[786]: ts=2026-09-24T12:27:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:431324466worker1 # [ 98.415041] nix-grpc-daemon[486]: ts=2026-09-24T12:27:12Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04467worker2 # [ 98.389602] nix-grpc-daemon[786]: ts=2026-09-24T12:27:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324468worker1 # [ 98.421025] nix-grpc-daemon[486]: ts=2026-09-24T12:27:12Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14469worker2 # [ 98.406235] nix-grpc-daemon[786]: ts=2026-09-24T12:27:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324470worker2 # [ 98.420896] nix-grpc-daemon[786]: ts=2026-09-24T12:27:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431324471worker2 # [ 98.438012] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4472worker2 # [ 98.600282] nix-daemon[844]: accepted connection from pid 1067, user nix-grpc-daemon (trusted)4473worker2 # [ 98.623877] nix-daemon[844]: reaped child process 1070, status = succeeded4474lb # [ 98.649097] niks3-server[1025]: 2026/09/24 12:27:13 INFO Received push request method=POST path=/api/pushes4475worker2 # [ 98.634225] nix-grpc-daemon[856]: time=2026-09-24T12:27:13.181Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4476worker2 # [ 98.635937] nix-grpc-daemon[856]: time=2026-09-24T12:27:13.181Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4477lb # [ 98.687840] niks3-server[1025]: 2026/09/24 12:27:13 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4478worker2 # [ 98.670633] nix-grpc-daemon[856]: time=2026-09-24T12:27:13.218Z level=INFO msg="Uploading 1 narinfos"4479lb # [ 98.704771] niks3-server[1025]: 2026/09/24 12:27:13 INFO Signed narinfos id=18 count=14480lb # [ 98.714786] niks3-server[1025]: 2026/09/24 12:27:13 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4481lb # [ 98.725576] niks3-server[1025]: 2026/09/24 12:27:13 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4482lb # [ 98.739025] niks3-server[1025]: 2026/09/24 12:27:13 INFO Received complete push request method=POST path=/api/pushes/18/complete4483lb # [ 98.746888] niks3-server[1025]: 2026/09/24 12:27:13 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4484worker2 # [ 98.717096] nix-grpc-daemon[856]: time=2026-09-24T12:27:13.265Z level=INFO msg="Upload complete. (258ms)"4485worker2 # [ 98.727958] nix-grpc-daemon[786]: ts=2026-09-24T12:27:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=0 paths=1 nar_bytes_in=4244486worker2 # [ 98.744468] nix-grpc-daemon[786]: ts=2026-09-24T12:27:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324487worker2 # [ 98.756219] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4488worker2 # [ 98.762137] nix-daemon[844]: reaped child process 1065, status = succeeded4489worker2 # [ 98.773866] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4490lb # [ 99.246295] envoy[906]: conn peer=[2001:db8:1::1]:43302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1194 ms=38074491lb # [ 99.251300] envoy[906]: conn peer=[2001:db8:1::1]:43294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3031 tx=1565 ms=49504492lb # [ 99.256200] envoy[906]: conn peer=[2001:db8:1::1]:51820 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2958 tx=958 ms=64344493worker2 # [ 105.192805] nix-grpc-daemon[786]: ts=2026-09-24T12:27:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04494worker2 # [ 105.236428] nix-daemon[844]: reaped child process 1074, status = succeeded4495worker2 # [ 105.239040] nix-daemon[844]: reaped child process 1072, status = succeeded4496(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.90 seconds)4497subtest: the first client leaving does not fail the second4498client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4499client # [ 105.493386] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4500client # Running as unit: share1.service; invocation ID: a4b735851dcc4dca872948cca9cb6fe24501client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4502client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4503client # [ 105.713592] nix[1088]: don't know how to build these paths:4504client # [ 105.714571] nix[1088]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4505worker1 # [ 105.701148] nix-grpc-daemon[486]: ts=2026-09-24T12:27:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588364506worker1 # [ 105.715536] nix-grpc-daemon[486]: ts=2026-09-24T12:27:20Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04507worker1 # [ 105.725229] nix-grpc-daemon[486]: ts=2026-09-24T12:27:20Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14508worker1 # [ 105.734318] nix-grpc-daemon[486]: ts=2026-09-24T12:27:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364509worker1 # [ 105.742252] nix-grpc-daemon[486]: ts=2026-09-24T12:27:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364510client # [ 105.762534] nix[1088]: copying 1 paths to 'lb:50051'...4511client # [ 105.765268] nix[1088]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4512worker1 # [ 105.754712] nix-grpc-daemon[486]: ts=2026-09-24T12:27:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588364513worker1 # [ 105.770603] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4514worker1 # [ 105.924293] nix-daemon[855]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)4515worker1 # [ 105.950071] nix-daemon[855]: reaped child process 1085, status = succeeded4516lb # [ 105.946643] niks3-server[1025]: 2026/09/24 12:27:20 INFO Received push request method=POST path=/api/pushes4517worker1 # [ 105.962123] nix-grpc-daemon[914]: time=2026-09-24T12:27:20.491Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4518worker1 # [ 105.963604] nix-grpc-daemon[914]: time=2026-09-24T12:27:20.491Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4519lb # [ 105.981126] niks3-server[1025]: 2026/09/24 12:27:20 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4520worker1 # [ 105.992366] nix-grpc-daemon[914]: time=2026-09-24T12:27:20.522Z level=INFO msg="Uploading 1 narinfos"4521lb # [ 105.995033] niks3-server[1025]: 2026/09/24 12:27:20 INFO Signed narinfos id=19 count=14522lb # [ 106.004651] niks3-server[1025]: 2026/09/24 12:27:20 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4523lb # [ 106.018671] niks3-server[1025]: 2026/09/24 12:27:20 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4524lb # [ 106.033146] niks3-server[1025]: 2026/09/24 12:27:20 INFO Received complete push request method=POST path=/api/pushes/19/complete4525worker1 # [ 106.039900] nix-grpc-daemon[914]: time=2026-09-24T12:27:20.570Z level=INFO msg="Upload complete. (253ms)"4526lb # [ 106.038883] niks3-server[1025]: 2026/09/24 12:27:20 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4527worker1 # [ 106.051948] nix-grpc-daemon[486]: ts=2026-09-24T12:27:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_in=4324528worker1 # [ 106.068303] nix-grpc-daemon[486]: ts=2026-09-24T12:27:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364529worker1 # [ 106.076351] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4530worker1 # [ 106.082360] nix-daemon[855]: reaped child process 1080, status = succeeded4531worker1 # [ 106.095117] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4532client # [ 106.105988] nix[1088]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4533client # [ 106.599803] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4534client # Running as unit: share2.service; invocation ID: ac67a3815c7e493ca6351c6595dbc0df4535client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4536client # [ 106.833874] nix[1104]: don't know how to build these paths:4537client # [ 106.835608] nix[1104]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4538worker2 # [ 106.791470] nix-grpc-daemon[786]: ts=2026-09-24T12:27:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:431324539worker1 # [ 106.834299] nix-grpc-daemon[486]: ts=2026-09-24T12:27:21Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04540client # [ 106.854926] nix[1104]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4541worker1 # [ 106.842298] nix-grpc-daemon[486]: ts=2026-09-24T12:27:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364542client: must succeed: sleep 2; systemctl kill -s INT share14543lb # [ 109.252748] envoy[906]: conn peer=[2001:db8:1::1]:51828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3002 tx=953 ms=68414544client # [ 109.983078] systemd[1]: share1.service: Sent signal SIGINT to main process 1088 (nix) on client request.4545client # [ 109.994488] nix[1088]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4546client # [ 109.995578] nix[1088]: error: interrupted by the user4547client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.11 seconds)4548client: waiting for success: ! systemctl is-active share14549client # [ 110.051571] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4550client # [ 110.055338] systemd[1]: share1.service: Failed with result 'exit-code'.4551client # [ 110.059310] systemd[1]: share1.service: Consumed 184ms CPU time over 4.561s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic.4552client: (finished: waiting for success: ! systemctl is-active share1, in 0.05 seconds)4553client: waiting for success: ! systemctl is-active share24554worker1 # [ 110.328512] nix-daemon[855]: reaped child process 1089, status = succeeded4555worker1 # [ 110.432493] nix-daemon[855]: accepted connection from pid 1125, user nix-grpc-daemon (trusted)4556worker1 # [ 110.455525] nix-daemon[855]: reaped child process 1128, status = succeeded4557lb # [ 110.452580] niks3-server[1025]: 2026/09/24 12:27:24 INFO Received push request method=POST path=/api/pushes4558worker1 # [ 110.468382] nix-grpc-daemon[914]: time=2026-09-24T12:27:24.997Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4559worker1 # [ 110.469843] nix-grpc-daemon[914]: time=2026-09-24T12:27:24.997Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4560lb # [ 110.506748] niks3-server[1025]: 2026/09/24 12:27:25 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign4561worker1 # [ 110.515573] nix-grpc-daemon[914]: time=2026-09-24T12:27:25.045Z level=INFO msg="Uploading 1 narinfos"4562lb # [ 110.518243] niks3-server[1025]: 2026/09/24 12:27:25 INFO Signed narinfos id=20 count=14563lb # [ 110.525951] niks3-server[1025]: 2026/09/24 12:27:25 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4564lb # [ 110.535330] niks3-server[1025]: 2026/09/24 12:27:25 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4565lb # [ 110.548233] niks3-server[1025]: 2026/09/24 12:27:25 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4566lb # [ 110.551635] niks3-server[1025]: 2026/09/24 12:27:25 INFO Received complete push request method=POST path=/api/pushes/20/complete4567lb # [ 110.557251] niks3-server[1025]: 2026/09/24 12:27:25 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4568worker1 # [ 110.563472] nix-grpc-daemon[914]: time=2026-09-24T12:27:25.094Z level=INFO msg="Upload complete. (247ms)"4569worker1 # [ 110.577340] nix-grpc-daemon[486]: ts=2026-09-24T12:27:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04570worker1 # [ 110.597402] nix-grpc-daemon[486]: ts=2026-09-24T12:27:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=3 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04571worker1 # [ 110.604521] nix-daemon[855]: reaped child process 1087, status = succeeded4572client # [ 110.623168] systemd[1]: share2.service: Deactivated successfully.4573client # [ 110.628456] systemd[1]: share2.service: Consumed 206ms CPU time over 4.023s wall clock time, 33.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4574client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4575client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4576client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4577(finished: subtest: the first client leaving does not fail the second, in 6.08 seconds)4578subtest: build hook: nix-daemon with builders = grpc://lb4579client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4580client # [ 111.636911] systemd[1]: Started Nix Daemon.4581client # [ 111.762173] nix-daemon[1151]: accepted connection from pid 1148, user root (trusted)4582worker1 # [ 111.910651] nix-grpc-daemon[486]: ts=2026-09-24T12:27:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:588364583worker1 # [ 111.918049] nix-grpc-daemon[486]: ts=2026-09-24T12:27:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364584worker1 # [ 111.932172] nix-grpc-daemon[486]: ts=2026-09-24T12:27:26Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04585worker1 # [ 111.938421] nix-grpc-daemon[486]: ts=2026-09-24T12:27:26Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14586worker2 # [ 111.913769] nix-grpc-daemon[786]: ts=2026-09-24T12:27:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324587worker2 # [ 111.930163] nix-grpc-daemon[786]: ts=2026-09-24T12:27:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324588worker2 # [ 111.945793] nix-grpc-daemon[786]: ts=2026-09-24T12:27:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431324589worker2 # [ 111.961019] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4590worker2 # [ 112.126068] nix-daemon[844]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4591worker2 # [ 112.147349] nix-daemon[844]: reaped child process 1116, status = succeeded4592lb # [ 112.182205] niks3-server[1025]: 2026/09/24 12:27:26 INFO Received push request method=POST path=/api/pushes4593worker2 # [ 112.163549] nix-grpc-daemon[856]: time=2026-09-24T12:27:26.711Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4594worker2 # [ 112.165446] nix-grpc-daemon[856]: time=2026-09-24T12:27:26.711Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4595worker2 # [ 112.207065] nix-grpc-daemon[856]: time=2026-09-24T12:27:26.755Z level=INFO msg="Uploading 1 narinfos"4596lb # [ 112.227077] niks3-server[1025]: 2026/09/24 12:27:26 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4597lb # [ 112.240072] niks3-server[1025]: 2026/09/24 12:27:26 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign4598lb # [ 112.248777] niks3-server[1025]: 2026/09/24 12:27:26 INFO Signed narinfos id=21 count=14599lb # [ 112.251685] niks3-server[1025]: 2026/09/24 12:27:26 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4600lb # [ 112.270459] niks3-server[1025]: 2026/09/24 12:27:26 INFO Received complete push request method=POST path=/api/pushes/21/complete4601lb # [ 112.277226] niks3-server[1025]: 2026/09/24 12:27:26 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4602worker2 # [ 112.256541] nix-grpc-daemon[856]: time=2026-09-24T12:27:26.805Z level=INFO msg="Upload complete. (278ms)"4603worker2 # [ 112.267409] nix-grpc-daemon[786]: ts=2026-09-24T12:27:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=0 paths=1 nar_bytes_in=5124604worker2 # [ 112.280662] nix-grpc-daemon[786]: ts=2026-09-24T12:27:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324605worker2 # [ 112.295139] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4606worker2 # [ 112.302254] nix-daemon[844]: reaped child process 1111, status = succeeded4607worker2 # [ 112.316074] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4608worker2 # [ 113.188298] nix-daemon[844]: reaped child process 1120, status = succeeded4609worker2 # [ 113.291372] nix-daemon[844]: accepted connection from pid 1136, user nix-grpc-daemon (trusted)4610worker2 # [ 113.309539] nix-daemon[844]: reaped child process 1139, status = succeeded4611lb # [ 113.344247] niks3-server[1025]: 2026/09/24 12:27:27 INFO Received push request method=POST path=/api/pushes4612worker2 # [ 113.328464] nix-grpc-daemon[856]: time=2026-09-24T12:27:27.875Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4613worker2 # [ 113.330177] nix-grpc-daemon[856]: time=2026-09-24T12:27:27.875Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4614lb # [ 113.392219] niks3-server[1025]: 2026/09/24 12:27:27 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4615worker2 # [ 113.375920] nix-grpc-daemon[856]: time=2026-09-24T12:27:27.924Z level=INFO msg="Uploading 1 narinfos"4616lb # [ 113.406761] niks3-server[1025]: 2026/09/24 12:27:27 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4617lb # [ 113.416872] niks3-server[1025]: 2026/09/24 12:27:27 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign4618lb # [ 113.419813] niks3-server[1025]: 2026/09/24 12:27:27 INFO Signed narinfos id=22 count=14619lb # [ 113.429730] niks3-server[1025]: 2026/09/24 12:27:27 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4620lb # [ 113.432865] niks3-server[1025]: 2026/09/24 12:27:27 INFO Received complete push request method=POST path=/api/pushes/22/complete4621lb # [ 113.437213] niks3-server[1025]: 2026/09/24 12:27:27 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4622worker2 # [ 113.408751] nix-grpc-daemon[856]: time=2026-09-24T12:27:27.957Z level=INFO msg="Upload complete. (283ms)"4623worker2 # [ 113.421246] nix-grpc-daemon[786]: ts=2026-09-24T12:27:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04624worker2 # [ 113.444567] nix-grpc-daemon[786]: ts=2026-09-24T12:27:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:431324625worker2 # [ 113.450340] nix-daemon[844]: reaped child process 1118, status = succeeded4626worker1 # [ 113.504998] nix-grpc-daemon[486]: ts=2026-09-24T12:27:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588364627worker1 # [ 113.519608] nix-grpc-daemon[486]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4628worker1 # [ 113.538599] nix-grpc-daemon[486]: ts=2026-09-24T12:27:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_out=1284629worker1 # [ 113.774933] nix-grpc-daemon[486]: ts=2026-09-24T12:27:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:588364630worker1 # [ 113.782839] nix-grpc-daemon[486]: ts=2026-09-24T12:27:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364631worker1 # [ 113.790429] nix-grpc-daemon[914]: time=2026-09-24T12:27:28.319Z level=INFO msg="All 1 paths already cached"4632worker1 # [ 113.802749] nix-grpc-daemon[486]: ts=2026-09-24T12:27:28Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04633worker2 # [ 113.776519] nix-grpc-daemon[786]: ts=2026-09-24T12:27:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324634worker1 # [ 113.808833] nix-grpc-daemon[486]: ts=2026-09-24T12:27:28Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14635worker2 # [ 113.792147] nix-grpc-daemon[786]: ts=2026-09-24T12:27:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324636worker2 # [ 113.806581] nix-grpc-daemon[856]: time=2026-09-24T12:27:28.355Z level=INFO msg="All 1 paths already cached"4637worker2 # [ 113.816858] nix-grpc-daemon[786]: ts=2026-09-24T12:27:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431324638worker2 # [ 113.831278] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4639worker2 # [ 113.989576] nix-daemon[844]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4640worker2 # [ 114.008565] nix-daemon[844]: reaped child process 1146, status = succeeded4641lb # [ 114.042974] niks3-server[1025]: 2026/09/24 12:27:28 INFO Received push request method=POST path=/api/pushes4642worker2 # [ 114.025263] nix-grpc-daemon[856]: time=2026-09-24T12:27:28.573Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4643worker2 # [ 114.026987] nix-grpc-daemon[856]: time=2026-09-24T12:27:28.573Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4644lb # [ 114.073085] niks3-server[1025]: 2026/09/24 12:27:28 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign4645worker2 # [ 114.052501] nix-grpc-daemon[856]: time=2026-09-24T12:27:28.600Z level=INFO msg="Uploading 1 narinfos"4646lb # [ 114.085997] niks3-server[1025]: 2026/09/24 12:27:28 INFO Signed narinfos id=23 count=14647lb # [ 114.095544] niks3-server[1025]: 2026/09/24 12:27:28 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4648lb # [ 114.109234] niks3-server[1025]: 2026/09/24 12:27:28 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4649lb # [ 114.121960] niks3-server[1025]: 2026/09/24 12:27:28 INFO Received complete push request method=POST path=/api/pushes/23/complete4650worker2 # [ 114.099267] nix-grpc-daemon[856]: time=2026-09-24T12:27:28.647Z level=INFO msg="Upload complete. (250ms)"4651lb # [ 114.127445] niks3-server[1025]: 2026/09/24 12:27:28 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4652worker2 # [ 114.109248] nix-grpc-daemon[786]: ts=2026-09-24T12:27:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=0 paths=1 nar_bytes_in=6244653worker2 # [ 114.124464] nix-grpc-daemon[786]: ts=2026-09-24T12:27:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324654worker2 # [ 114.131595] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4655worker2 # [ 114.140356] nix-daemon[844]: reaped child process 1141, status = succeeded4656worker2 # [ 114.148440] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4657worker2 # [ 115.056746] nix-daemon[844]: reaped child process 1150, status = succeeded4658worker2 # [ 115.130910] nix-daemon[844]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)4659worker2 # [ 115.148499] nix-daemon[844]: reaped child process 1169, status = succeeded4660lb # [ 115.182866] niks3-server[1025]: 2026/09/24 12:27:29 INFO Received push request method=POST path=/api/pushes4661worker2 # [ 115.168439] nix-grpc-daemon[856]: time=2026-09-24T12:27:29.715Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4662worker2 # [ 115.170164] nix-grpc-daemon[856]: time=2026-09-24T12:27:29.715Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4663lb # [ 115.260408] niks3-server[1025]: 2026/09/24 12:27:29 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4664worker2 # [ 115.243383] nix-grpc-daemon[856]: time=2026-09-24T12:27:29.791Z level=INFO msg="Uploading 1 narinfos"4665lb # [ 115.270996] niks3-server[1025]: 2026/09/24 12:27:29 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign4666lb # [ 115.282383] niks3-server[1025]: 2026/09/24 12:27:29 INFO Signed narinfos id=24 count=14667lb # [ 115.286998] niks3-server[1025]: 2026/09/24 12:27:29 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4668lb # [ 115.299257] niks3-server[1025]: 2026/09/24 12:27:29 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4669worker2 # [ 115.271247] nix-grpc-daemon[856]: time=2026-09-24T12:27:29.819Z level=INFO msg="Upload complete. (311ms)"4670lb # [ 115.302292] niks3-server[1025]: 2026/09/24 12:27:29 INFO Received complete push request method=POST path=/api/pushes/24/complete4671lb # [ 115.307035] niks3-server[1025]: 2026/09/24 12:27:29 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4672worker2 # [ 115.285282] nix-grpc-daemon[786]: ts=2026-09-24T12:27:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04673worker1 # [ 115.319657] nix-grpc-daemon[486]: ts=2026-09-24T12:27:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588364674worker1 # [ 115.331131] nix-grpc-daemon[486]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4675worker2 # [ 115.304882] nix-daemon[844]: reaped child process 1148, status = succeeded4676worker1 # [ 115.367583] nix-grpc-daemon[486]: ts=2026-09-24T12:27:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588364677worker1 # [ 115.374792] nix-grpc-daemon[486]: ts=2026-09-24T12:27:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_out=1764678worker1 # [ 115.642112] nix-grpc-daemon[486]: ts=2026-09-24T12:27:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:588364679worker2 # [ 115.618451] nix-grpc-daemon[786]: ts=2026-09-24T12:27:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324680worker2 # [ 115.629903] nix-grpc-daemon[856]: time=2026-09-24T12:27:30.178Z level=INFO msg="All 2 paths already cached"4681worker1 # [ 115.687875] nix-grpc-daemon[486]: ts=2026-09-24T12:27:30Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04682worker1 # [ 115.697291] nix-grpc-daemon[486]: ts=2026-09-24T12:27:30Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14683worker2 # [ 115.672719] nix-grpc-daemon[786]: ts=2026-09-24T12:27:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324684worker2 # [ 115.693833] nix-grpc-daemon[786]: ts=2026-09-24T12:27:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324685worker2 # [ 115.709187] nix-grpc-daemon[856]: time=2026-09-24T12:27:30.257Z level=INFO msg="All 2 paths already cached"4686worker2 # [ 115.718037] nix-grpc-daemon[786]: ts=2026-09-24T12:27:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431324687worker2 # [ 115.726410] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4688worker2 # [ 115.881851] nix-daemon[844]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)4689worker2 # [ 115.906430] nix-daemon[844]: reaped child process 1176, status = succeeded4690lb # [ 115.935512] niks3-server[1025]: 2026/09/24 12:27:30 INFO Received push request method=POST path=/api/pushes4691worker2 # [ 115.918717] nix-grpc-daemon[856]: time=2026-09-24T12:27:30.466Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4692worker2 # [ 115.920523] nix-grpc-daemon[856]: time=2026-09-24T12:27:30.466Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4693lb # [ 115.971300] niks3-server[1025]: 2026/09/24 12:27:30 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4694worker2 # [ 115.960647] nix-grpc-daemon[856]: time=2026-09-24T12:27:30.508Z level=INFO msg="Uploading 1 narinfos"4695lb # [ 115.981530] niks3-server[1025]: 2026/09/24 12:27:30 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign4696lb # [ 115.995245] niks3-server[1025]: 2026/09/24 12:27:30 INFO Signed narinfos id=25 count=14697lb # [ 116.001748] niks3-server[1025]: 2026/09/24 12:27:30 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4698lb # [ 116.015407] niks3-server[1025]: 2026/09/24 12:27:30 INFO Received complete push request method=POST path=/api/pushes/25/complete4699worker2 # [ 115.996064] nix-grpc-daemon[856]: time=2026-09-24T12:27:30.544Z level=INFO msg="Upload complete. (260ms)"4700lb # [ 116.025222] niks3-server[1025]: 2026/09/24 12:27:30 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4701worker2 # [ 116.008109] nix-grpc-daemon[786]: ts=2026-09-24T12:27:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=0 paths=1 nar_bytes_in=7524702worker2 # [ 116.024329] nix-grpc-daemon[786]: ts=2026-09-24T12:27:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324703worker2 # [ 116.033015] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4704worker2 # [ 116.035927] nix-daemon[844]: reaped child process 1171, status = succeeded4705worker2 # [ 116.047343] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4706worker2 # [ 117.563570] nix-daemon[844]: reaped child process 1180, status = succeeded4707worker2 # [ 117.635237] nix-daemon[844]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)4708worker2 # [ 117.655366] nix-daemon[844]: reaped child process 1199, status = succeeded4709lb # [ 117.689647] niks3-server[1025]: 2026/09/24 12:27:32 INFO Received push request method=POST path=/api/pushes4710worker2 # [ 117.672236] nix-grpc-daemon[856]: time=2026-09-24T12:27:32.219Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4711worker2 # [ 117.673988] nix-grpc-daemon[856]: time=2026-09-24T12:27:32.219Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4712lb # [ 117.727381] niks3-server[1025]: 2026/09/24 12:27:32 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign4713worker2 # [ 117.707328] nix-grpc-daemon[856]: time=2026-09-24T12:27:32.255Z level=INFO msg="Uploading 1 narinfos"4714lb # [ 117.741432] niks3-server[1025]: 2026/09/24 12:27:32 INFO Signed narinfos id=26 count=14715lb # [ 117.748169] niks3-server[1025]: 2026/09/24 12:27:32 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4716lb # [ 117.759301] niks3-server[1025]: 2026/09/24 12:27:32 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4717lb # [ 117.772194] niks3-server[1025]: 2026/09/24 12:27:32 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4718lb # [ 117.775788] niks3-server[1025]: 2026/09/24 12:27:32 INFO Received complete push request method=POST path=/api/pushes/26/complete4719lb # [ 117.784506] niks3-server[1025]: 2026/09/24 12:27:32 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4720worker2 # [ 117.754536] nix-grpc-daemon[856]: time=2026-09-24T12:27:32.302Z level=INFO msg="Upload complete. (292ms)"4721worker1 # [ 117.801829] nix-grpc-daemon[486]: ts=2026-09-24T12:27:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588364722worker2 # [ 117.769628] nix-grpc-daemon[786]: ts=2026-09-24T12:27:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04723worker1 # [ 117.814374] nix-grpc-daemon[486]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4724worker2 # [ 117.789411] nix-daemon[844]: reaped child process 1178, status = succeeded4725worker1 # [ 117.853691] nix-grpc-daemon[486]: ts=2026-09-24T12:27:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588364726worker1 # [ 117.860939] nix-grpc-daemon[486]: ts=2026-09-24T12:27:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_out=2324727client # [ 117.958057] nix-daemon[1151]: reaped child process 1153, status = succeeded4728client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.48 seconds)4729client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4730client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4731client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4732client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4733client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4734client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4735(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.59 seconds)4736subtest: an input only one worker has still reaches the builder4737client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4738client # this derivation will be built:4739client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4740client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4741client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)4742client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4743client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4744worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4745worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4746worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4747worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4748client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24749client # [ 118.918098] nix-daemon[1151]: accepted connection from pid 1234, user root (trusted)4750client # this derivation will be built:4751client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4752client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4753worker1 # [ 119.067489] nix-grpc-daemon[486]: ts=2026-09-24T12:27:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:588364754worker2 # [ 119.042169] nix-grpc-daemon[786]: ts=2026-09-24T12:27:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324755client # copying 1 paths...4756client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4757worker2 # [ 119.058614] nix-grpc-daemon[786]: ts=2026-09-24T12:27:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431324758worker2 # [ 119.074221] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4759lb # [ 119.247924] envoy[906]: conn peer=[2001:db8:1::1]:36040 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2990 tx=788 ms=43054760lb # [ 119.253011] envoy[906]: conn peer=[2001:db8:1::1]:36056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=37674761lb # [ 119.258128] envoy[906]: conn peer=[2001:db8:1::1]:45466 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=334 ms=564762lb # [ 119.263625] envoy[906]: conn peer=[2001:db8:1::1]:45464 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1944 ms=16514763lb # [ 119.268772] envoy[906]: conn peer=[2001:db8:1::1]:45498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=284764worker2 # [ 119.246232] nix-daemon[844]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)4765lb # [ 119.274579] envoy[906]: conn peer=[2001:db8:1::1]:45482 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3926 tx=2158 ms=16194766lb # [ 119.279444] envoy[906]: conn peer=[2001:db8:1::1]:45512 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4326 tx=2388 ms=22374767lb # [ 119.284384] envoy[906]: conn peer=[2001:db8:1::1]:45520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=294768worker2 # [ 119.270332] nix-daemon[844]: reaped child process 1209, status = succeeded4769lb # [ 119.296833] niks3-server[1025]: 2026/09/24 12:27:33 INFO Received push request method=POST path=/api/pushes4770worker2 # [ 119.278506] nix-grpc-daemon[856]: time=2026-09-24T12:27:33.826Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4771worker2 # [ 119.280136] nix-grpc-daemon[856]: time=2026-09-24T12:27:33.826Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4772lb # [ 119.330796] niks3-server[1025]: 2026/09/24 12:27:33 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign4773worker2 # [ 119.309764] nix-grpc-daemon[856]: time=2026-09-24T12:27:33.857Z level=INFO msg="Uploading 1 narinfos"4774lb # [ 119.340280] niks3-server[1025]: 2026/09/24 12:27:33 INFO Signed narinfos id=27 count=14775lb # [ 119.348365] niks3-server[1025]: 2026/09/24 12:27:33 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4776lb # [ 119.355273] niks3-server[1025]: 2026/09/24 12:27:33 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4777lb # [ 119.366425] niks3-server[1025]: 2026/09/24 12:27:33 INFO Received complete push request method=POST path=/api/pushes/27/complete4778worker2 # [ 119.345929] nix-grpc-daemon[856]: time=2026-09-24T12:27:33.894Z level=INFO msg="Upload complete. (254ms)"4779lb # [ 119.375110] niks3-server[1025]: 2026/09/24 12:27:33 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4780worker2 # [ 119.356312] nix-grpc-daemon[786]: ts=2026-09-24T12:27:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=0 paths=1 nar_bytes_in=1204781worker1 # [ 119.398838] nix-grpc-daemon[486]: ts=2026-09-24T12:27:33Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04782worker2 # [ 119.372586] nix-daemon[844]: reaped child process 1204, status = succeeded4783worker1 # [ 119.406343] nix-grpc-daemon[486]: ts=2026-09-24T12:27:33Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14784worker1 # [ 119.416082] nix-grpc-daemon[486]: ts=2026-09-24T12:27:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364785worker1 # [ 119.427543] nix-grpc-daemon[486]: ts=2026-09-24T12:27:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364786client # copying 2 paths to 'lb:50051'...4787worker1 # [ 119.445746] nix-grpc-daemon[486]: ts=2026-09-24T12:27:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588364788client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4789client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4790worker1 # [ 119.461123] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4791worker1 # [ 119.622829] nix-daemon[855]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)4792worker1 # [ 119.642799] nix-daemon[855]: reaped child process 1151, status = succeeded4793lb # [ 119.643851] niks3-server[1025]: 2026/09/24 12:27:34 INFO Received push request method=POST path=/api/pushes4794worker1 # [ 119.658850] nix-grpc-daemon[914]: time=2026-09-24T12:27:34.187Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4795worker1 # [ 119.661514] nix-grpc-daemon[914]: time=2026-09-24T12:27:34.187Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4796worker1 # [ 119.663620] nix-grpc-daemon[914]: time=2026-09-24T12:27:34.187Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4797lb # [ 119.692373] niks3-server[1025]: 2026/09/24 12:27:34 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4798worker1 # [ 119.707251] nix-grpc-daemon[914]: time=2026-09-24T12:27:34.237Z level=INFO msg="Uploading 2 narinfos"4799lb # [ 119.704558] niks3-server[1025]: 2026/09/24 12:27:34 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign4800lb # [ 119.707876] niks3-server[1025]: 2026/09/24 12:27:34 INFO Signed narinfos id=28 count=24801lb # [ 119.718815] niks3-server[1025]: 2026/09/24 12:27:34 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4802lb # [ 119.726276] niks3-server[1025]: 2026/09/24 12:27:34 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4803lb # [ 119.735245] niks3-server[1025]: 2026/09/24 12:27:34 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4804lb # [ 119.744081] niks3-server[1025]: 2026/09/24 12:27:34 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4805lb # [ 119.752363] niks3-server[1025]: 2026/09/24 12:27:34 INFO Received complete push request method=POST path=/api/pushes/28/complete4806lb # [ 119.762962] niks3-server[1025]: 2026/09/24 12:27:34 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4807worker1 # [ 119.764511] nix-grpc-daemon[914]: time=2026-09-24T12:27:34.295Z level=INFO msg="Upload complete. (268ms)"4808worker1 # [ 119.776485] nix-grpc-daemon[486]: ts=2026-09-24T12:27:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=2 nar_bytes_in=9844809worker1 # [ 119.793522] nix-grpc-daemon[486]: ts=2026-09-24T12:27:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364810worker1 # [ 119.800820] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4811worker1 # [ 119.812673] nix-daemon[855]: reaped child process 1146, status = succeeded4812client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4813worker1 # [ 119.821703] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4814client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4815client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4816worker1 # [ 120.685470] nix-daemon[855]: reaped child process 1155, status = succeeded4817worker1 # [ 120.800567] nix-daemon[855]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)4818worker1 # [ 120.821461] nix-daemon[855]: reaped child process 1174, status = succeeded4819lb # [ 120.826123] niks3-server[1025]: 2026/09/24 12:27:35 INFO Received push request method=POST path=/api/pushes4820worker1 # [ 120.838497] nix-grpc-daemon[914]: time=2026-09-24T12:27:35.367Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4821lb # [ 120.854866] niks3-server[1025]: 2026/09/24 12:27:35 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4822worker1 # [ 120.871155] nix-grpc-daemon[914]: time=2026-09-24T12:27:35.401Z level=INFO msg="Uploading 1 narinfos"4823lb # [ 120.868373] niks3-server[1025]: 2026/09/24 12:27:35 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign4824lb # [ 120.876149] niks3-server[1025]: 2026/09/24 12:27:35 INFO Signed narinfos id=29 count=14825lb # [ 120.879386] niks3-server[1025]: 2026/09/24 12:27:35 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4826lb # [ 120.899169] niks3-server[1025]: 2026/09/24 12:27:35 INFO Received complete push request method=POST path=/api/pushes/29/complete4827lb # [ 120.906279] niks3-server[1025]: 2026/09/24 12:27:35 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4828worker1 # [ 120.914629] nix-grpc-daemon[914]: time=2026-09-24T12:27:35.445Z level=INFO msg="Upload complete. (278ms)"4829client # copying 1 paths...4830worker1 # [ 120.929220] nix-grpc-daemon[486]: ts=2026-09-24T12:27:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04831worker2 # [ 120.903635] nix-grpc-daemon[786]: ts=2026-09-24T12:27:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:431324832worker2 # [ 120.916994] nix-grpc-daemon[786]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4833worker1 # [ 120.950081] nix-daemon[855]: reaped child process 1153, status = succeeded4834client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4835worker1 # [ 120.991362] nix-grpc-daemon[486]: ts=2026-09-24T12:27:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588364836worker1 # [ 120.998394] nix-grpc-daemon[486]: ts=2026-09-24T12:27:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_out=1204837client # [ 121.096227] nix-daemon[1151]: reaped child process 1237, status = succeeded4838client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.37 seconds)4839client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24840client # [ 121.268738] nix-daemon[1151]: accepted connection from pid 1256, user root (trusted)4841client # this derivation will be built:4842client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4843worker2 # [ 121.399670] nix-grpc-daemon[786]: ts=2026-09-24T12:27:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:431324844client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4845worker2 # [ 121.408710] nix-grpc-daemon[786]: ts=2026-09-24T12:27:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324846worker2 # [ 121.418028] nix-grpc-daemon[856]: time=2026-09-24T12:27:35.963Z level=INFO msg="All 1 paths already cached"4847worker1 # [ 121.462787] nix-grpc-daemon[486]: ts=2026-09-24T12:27:35Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04848worker1 # [ 121.470014] nix-grpc-daemon[486]: ts=2026-09-24T12:27:35Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14849worker2 # [ 121.444345] nix-grpc-daemon[786]: ts=2026-09-24T12:27:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324850worker2 # [ 121.462260] nix-grpc-daemon[786]: ts=2026-09-24T12:27:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324851client # copying 1 paths to 'lb:50051'...4852client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4853worker2 # [ 121.485954] nix-grpc-daemon[786]: ts=2026-09-24T12:27:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431324854worker2 # [ 121.501839] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4855worker2 # [ 121.524926] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 1/5)4856worker2 # [ 121.588860] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 113 ms (attempt 2/5)4857worker2 # [ 121.711236] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 119 ms (attempt 3/5)4858worker2 # [ 121.832101] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 4/5)4859worker2 # [ 121.927909] nix-grpc-daemon[786]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4860worker2 # [ 121.936428] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4861worker2 # [ 121.943828] nix-grpc-daemon[786]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4862worker2 # [ 122.112133] nix-daemon[844]: accepted connection from pid 1219, user nix-grpc-daemon (trusted)4863worker2 # [ 122.134398] nix-daemon[844]: reaped child process 1222, status = succeeded4864lb # [ 122.167362] niks3-server[1025]: 2026/09/24 12:27:36 INFO Received push request method=POST path=/api/pushes4865worker2 # [ 122.149529] nix-grpc-daemon[856]: time=2026-09-24T12:27:36.697Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4866worker2 # [ 122.151255] nix-grpc-daemon[856]: time=2026-09-24T12:27:36.697Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4867lb # [ 122.199183] niks3-server[1025]: 2026/09/24 12:27:36 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4868worker2 # [ 122.182354] nix-grpc-daemon[856]: time=2026-09-24T12:27:36.730Z level=INFO msg="Uploading 1 narinfos"4869lb # [ 122.211235] niks3-server[1025]: 2026/09/24 12:27:36 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign4870lb # [ 122.223221] niks3-server[1025]: 2026/09/24 12:27:36 INFO Signed narinfos id=30 count=14871lb # [ 122.231217] niks3-server[1025]: 2026/09/24 12:27:36 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4872lb # [ 122.248303] niks3-server[1025]: 2026/09/24 12:27:36 INFO Received complete push request method=POST path=/api/pushes/30/complete4873worker2 # [ 122.226487] nix-grpc-daemon[856]: time=2026-09-24T12:27:36.774Z level=INFO msg="Upload complete. (268ms)"4874lb # [ 122.256684] niks3-server[1025]: 2026/09/24 12:27:36 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4875worker2 # [ 122.236341] nix-grpc-daemon[786]: ts=2026-09-24T12:27:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=0 paths=1 nar_bytes_in=5684876worker2 # [ 122.256121] nix-grpc-daemon[786]: ts=2026-09-24T12:27:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431324877worker2 # [ 122.266212] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4878worker2 # [ 122.278876] nix-daemon[844]: reaped child process 1212, status = succeeded4879client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4880worker2 # [ 122.294556] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4881client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4882client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4883worker2 # [ 123.565786] nix-daemon[844]: reaped child process 1226, status = succeeded4884worker2 # [ 123.681347] nix-daemon[844]: accepted connection from pid 1242, user nix-grpc-daemon (trusted)4885worker2 # [ 123.703665] nix-daemon[844]: reaped child process 1245, status = succeeded4886lb # [ 123.735930] niks3-server[1025]: 2026/09/24 12:27:38 INFO Received push request method=POST path=/api/pushes4887worker2 # [ 123.720458] nix-grpc-daemon[856]: time=2026-09-24T12:27:38.266Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4888lb # [ 123.768686] niks3-server[1025]: 2026/09/24 12:27:38 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign4889worker2 # [ 123.750274] nix-grpc-daemon[856]: time=2026-09-24T12:27:38.298Z level=INFO msg="Uploading 1 narinfos"4890lb # [ 123.782100] niks3-server[1025]: 2026/09/24 12:27:38 INFO Signed narinfos id=31 count=14891lb # [ 123.791268] niks3-server[1025]: 2026/09/24 12:27:38 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4892lb # [ 123.803648] niks3-server[1025]: 2026/09/24 12:27:38 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4893lb # [ 123.815165] niks3-server[1025]: 2026/09/24 12:27:38 INFO Received complete push request method=POST path=/api/pushes/31/complete4894worker2 # [ 123.786597] nix-grpc-daemon[856]: time=2026-09-24T12:27:38.335Z level=INFO msg="Upload complete. (277ms)"4895lb # [ 123.819478] niks3-server[1025]: 2026/09/24 12:27:38 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4896client # copying 1 paths...4897worker2 # [ 123.797391] nix-grpc-daemon[786]: ts=2026-09-24T12:27:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43132 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04898worker2 # [ 123.819497] nix-grpc-daemon[786]: ts=2026-09-24T12:27:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:431324899worker2 # [ 123.825378] nix-daemon[844]: reaped child process 1224, status = succeeded4900client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4901worker1 # [ 123.882326] nix-grpc-daemon[486]: ts=2026-09-24T12:27:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588364902worker1 # [ 123.896318] nix-grpc-daemon[486]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4903worker1 # [ 123.913785] nix-grpc-daemon[486]: ts=2026-09-24T12:27:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_out=1204904client # [ 124.021359] nix-daemon[1151]: reaped child process 1259, status = succeeded4905client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.92 seconds)4906client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24907client # [ 124.194388] nix-daemon[1151]: accepted connection from pid 1278, user root (trusted)4908client # this derivation will be built:4909client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4910worker1 # [ 124.341576] nix-grpc-daemon[486]: ts=2026-09-24T12:27:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:588364911client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4912worker2 # [ 124.315520] nix-grpc-daemon[786]: ts=2026-09-24T12:27:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431324913worker2 # [ 124.323609] nix-grpc-daemon[856]: time=2026-09-24T12:27:38.872Z level=INFO msg="All 1 paths already cached"4914worker1 # [ 124.370870] nix-grpc-daemon[486]: ts=2026-09-24T12:27:38Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04915worker1 # [ 124.378584] nix-grpc-daemon[486]: ts=2026-09-24T12:27:38Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14916worker1 # [ 124.387796] nix-grpc-daemon[486]: ts=2026-09-24T12:27:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364917worker1 # [ 124.399368] nix-grpc-daemon[486]: ts=2026-09-24T12:27:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588364918worker1 # [ 124.415341] nix-grpc-daemon[914]: time=2026-09-24T12:27:38.944Z level=INFO msg="All 1 paths already cached"4919client # copying 1 paths to 'lb:50051'...4920client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4921worker1 # [ 124.425491] nix-grpc-daemon[486]: ts=2026-09-24T12:27:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588364922worker1 # [ 124.440519] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4923worker1 # [ 124.617377] nix-daemon[855]: accepted connection from pid 1183, user nix-grpc-daemon (trusted)4924worker1 # [ 124.641222] nix-daemon[855]: reaped child process 1186, status = succeeded4925lb # [ 124.643298] niks3-server[1025]: 2026/09/24 12:27:39 INFO Received push request method=POST path=/api/pushes4926worker1 # [ 124.652979] nix-grpc-daemon[914]: time=2026-09-24T12:27:39.183Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4927worker1 # [ 124.654473] nix-grpc-daemon[914]: time=2026-09-24T12:27:39.183Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4928lb # [ 124.668630] niks3-server[1025]: 2026/09/24 12:27:39 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign4929worker1 # [ 124.674665] nix-grpc-daemon[914]: time=2026-09-24T12:27:39.205Z level=INFO msg="Uploading 1 narinfos"4930lb # [ 124.676197] niks3-server[1025]: 2026/09/24 12:27:39 INFO Signed narinfos id=32 count=14931lb # [ 124.678058] niks3-server[1025]: 2026/09/24 12:27:39 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4932lb # [ 124.691110] niks3-server[1025]: 2026/09/24 12:27:39 INFO Received complete push request method=POST path=/api/pushes/32/complete4933lb # [ 124.693794] niks3-server[1025]: 2026/09/24 12:27:39 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4934lb # [ 124.698376] niks3-server[1025]: 2026/09/24 12:27:39 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4935worker1 # [ 124.704557] nix-grpc-daemon[914]: time=2026-09-24T12:27:39.235Z level=INFO msg="Upload complete. (247ms)"4936worker1 # [ 124.712474] nix-grpc-daemon[486]: ts=2026-09-24T12:27:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_in=5684937worker1 # [ 124.724144] nix-grpc-daemon[486]: ts=2026-09-24T12:27:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588364938worker1 # [ 124.730119] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4939worker1 # [ 124.734926] nix-daemon[855]: reaped child process 1181, status = succeeded4940worker1 # [ 124.744714] nix-daemon[855]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4941client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4942client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4943client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4944worker1 # [ 126.200291] nix-daemon[855]: reaped child process 1190, status = succeeded4945worker1 # [ 126.327344] nix-daemon[855]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)4946worker1 # [ 126.351575] nix-daemon[855]: reaped child process 1209, status = succeeded4947lb # [ 126.347675] niks3-server[1025]: 2026/09/24 12:27:40 INFO Received push request method=POST path=/api/pushes4948worker1 # [ 126.360506] nix-grpc-daemon[914]: time=2026-09-24T12:27:40.890Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4949lb # [ 126.388594] niks3-server[1025]: 2026/09/24 12:27:40 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4950worker1 # [ 126.403369] nix-grpc-daemon[914]: time=2026-09-24T12:27:40.933Z level=INFO msg="Uploading 1 narinfos"4951lb # [ 126.400990] niks3-server[1025]: 2026/09/24 12:27:40 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign4952lb # [ 126.411796] niks3-server[1025]: 2026/09/24 12:27:40 INFO Signed narinfos id=33 count=14953lb # [ 126.418207] niks3-server[1025]: 2026/09/24 12:27:40 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4954lb # [ 126.432195] niks3-server[1025]: 2026/09/24 12:27:40 INFO Received complete push request method=POST path=/api/pushes/33/complete4955worker1 # [ 126.441596] nix-grpc-daemon[914]: time=2026-09-24T12:27:40.972Z level=INFO msg="Upload complete. (284ms)"4956lb # [ 126.440757] niks3-server[1025]: 2026/09/24 12:27:40 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4957client # copying 1 paths...4958worker2 # [ 126.430874] nix-grpc-daemon[786]: ts=2026-09-24T12:27:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:431324959worker1 # [ 126.457238] nix-grpc-daemon[486]: ts=2026-09-24T12:27:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04960worker2 # [ 126.444454] nix-grpc-daemon[786]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4961worker1 # [ 126.477282] nix-daemon[855]: reaped child process 1188, status = succeeded4962client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4963worker1 # [ 126.518124] nix-grpc-daemon[486]: ts=2026-09-24T12:27:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588364964worker1 # [ 126.525414] nix-grpc-daemon[486]: ts=2026-09-24T12:27:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58836 duration_s=0 paths=1 nar_bytes_out=1204965client # [ 126.621477] nix-daemon[1151]: reaped child process 1281, status = succeeded4966client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.60 seconds)4967(finished: subtest: an input only one worker has still reaches the builder, in 8.54 seconds)4968subtest: upload whose reference the worker lacks is completed from the cache4969client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4970client # this derivation will be built:4971client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4972client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4973client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.34 seconds)4974client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4975client # this derivation will be built:4976client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4977client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4978client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.32 seconds)4979client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4980client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.15 seconds)4981worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4982worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.18 seconds)4983worker2 # [ 127.616476] systemd[1]: Stopping Nix worker-protocol over gRPC...4984worker2 # [ 128.552151] nix-grpc-daemon[786]: ts=2026-09-24T12:27:43Z level=info event=signal_exit4985lb # [ 129.251114] envoy[906]: conn peer=[2001:db8:1::1]:34800 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=294986lb # [ 129.256289] envoy[906]: conn peer=[2001:db8:1::1]:45532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4237 tx=2054 ms=19494987lb # [ 129.256601] envoy[906]: conn peer=[2001:db8:1::1]:34808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3739 tx=2127 ms=25034988lb # [ 129.256757] envoy[906]: conn peer=[2001:db8:1::1]:34820 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=329 ms=534989lb # [ 129.256856] envoy[906]: conn peer=[2001:db8:1::1]:34836 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3779 tx=2124 ms=22014990lb # [ 129.256950] envoy[906]: conn peer=[2001:db8:1::1]:34846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=294991worker2 # [ 130.526581] nix-grpc-daemon[786]: ts=2026-09-24T12:27:45Z level=info event=scheduler_restarting4992worker1 # [ 130.555859] nix-grpc-daemon[486]: ts=2026-09-24T12:27:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:517684993worker1 # [ 130.564244] nix-grpc-daemon[486]: ts=2026-09-24T12:27:45Z level=info event=worker_gone addr=192.168.1.4:500514994worker2 # [ 130.555258] nix-grpc-daemon[786]: ts=2026-09-24T12:27:45Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4995worker2 # [ 130.588232] nix-daemon[844]: reaped child process 846, status = succeeded4996worker2 # [ 131.553928] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4997worker2 # [ 131.558548] systemd[1]: Stopped Nix worker-protocol over gRPC.4998worker2 # [ 131.564712] systemd[1]: nix-grpc-daemon.service: Consumed 3.755s CPU time over 1min 36.236s wall clock time, 65.6M memory peak, 195.9K incoming IP traffic, 312.2K outgoing IP traffic.4999lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux5000lb: must succeed: curl -sf localhost:9901/clusters5001lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5002lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.08 seconds)5003client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25004worker1 # [ 131.915178] nix-grpc-daemon[486]: ts=2026-09-24T12:27:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588365005worker1 # [ 131.924785] nix-grpc-daemon[486]: ts=2026-09-24T12:27:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588365006client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5007client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)5008worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5009worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5010worker2 # [ 132.015552] systemd[1]: Starting scheduler-token.service...5011worker2 # [ 132.084097] systemd[1]: scheduler-token.service: Deactivated successfully.5012worker2 # [ 132.091613] systemd[1]: Finished scheduler-token.service.5013worker2 # [ 132.093478] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 69ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5014worker2 # [ 132.098807] systemd[1]: Starting Nix worker-protocol over gRPC...5015worker2 # [ 132.220564] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5016worker2 # [ 132.222542] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5017worker2 # [ 132.226421] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15018worker2 # [ 132.232565] nix-grpc-daemon[1270]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5019worker2 # [ 132.234072] nix-grpc-daemon[1270]: E0000 00:00:1790252866.779585 1270 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5020worker2 # [ 132.238623] nix-grpc-daemon[1270]: E0000 00:00:1790252866.786968 1270 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5021worker2 # [ 132.283130] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:46Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5022worker2 # [ 132.286082] nix-grpc-daemon[1270]: E0000 00:00:1790252866.795876 1313 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5023worker2 # [ 132.289094] nix-grpc-daemon[1270]: E0000 00:00:1790252866.814672 1313 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5024worker2 # [ 132.291550] nix-grpc-daemon[1270]: E0000 00:00:1790252866.814880 1313 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5025worker2 # [ 132.297258] nix-grpc-daemon[1270]: E0000 00:00:1790252866.831518 1313 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5026worker2 # [ 132.302088] systemd[1]: Started Nix worker-protocol over gRPC.5027worker2 # [ 132.303114] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:46Z level=info event=scheduler_connected addr=lb:500515028worker1 # [ 132.335206] nix-grpc-daemon[486]: ts=2026-09-24T12:27:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:517685029worker1 # [ 132.351763] nix-grpc-daemon[486]: ts=2026-09-24T12:27:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05030worker1 # [ 132.389820] systemd[1]: Stopping Nix worker-protocol over gRPC...5031worker1 # [ 132.728906] nix-grpc-daemon[486]: ts=2026-09-24T12:27:47Z level=info event=signal_exit5032lb # [ 132.914689] niks3-server[1025]: 2026/09/24 12:27:47 INFO lead: released remote=192.168.1.3:452725033worker1 # [ 133.912343] nix-grpc-daemon[486]: ts=2026-09-24T12:27:48Z level=info event=scheduler_restarting5034lb # [ 133.909169] envoy[906]: [2026-09-24 12:27:48.455][906][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5035worker1 # [ 133.923577] nix-grpc-daemon[486]: ts=2026-09-24T12:27:48Z level=info event=scheduler_restarting addr=lb:500515036worker2 # [ 133.894689] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:48Z level=info event=scheduler_restarting addr=lb:500515037worker1 # [ 133.934849] nix-grpc-daemon[486]: ts=2026-09-24T12:27:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:517685038worker1 # [ 133.940510] nix-grpc-daemon[486]: ts=2026-09-24T12:27:48Z level=info event=worker_gone addr=192.168.1.4:500515039worker1 # [ 133.944720] nix-grpc-daemon[486]: ts=2026-09-24T12:27:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:517685040worker1 # [ 133.949852] nix-grpc-daemon[486]: ts=2026-09-24T12:27:48Z level=info event=worker_gone addr=192.168.1.3:500515041worker1 # [ 133.977827] nix-daemon[855]: reaped child process 857, status = succeeded5042worker2 # [ 134.094498] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:48Z level=info event=scheduler_connected addr=lb:500515043worker2 # [ 134.100276] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5044worker1 # [ 134.157311] nix-grpc-daemon[486]: ts=2026-09-24T12:27:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5045worker2 # [ 134.301719] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:48Z level=info event=scheduler_connected addr=lb:500515046worker2 # [ 134.307664] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5047worker2 # [ 134.509061] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_connected addr=lb:500515048worker2 # [ 134.514750] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5049worker2 # [ 134.716074] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_connected addr=lb:500515050worker2 # [ 134.721663] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5051worker2 # [ 134.924844] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_connected addr=lb:500515052worker2 # [ 134.929215] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5053worker2 # [ 135.125964] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_connected addr=lb:500515054worker2 # [ 135.131602] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5055worker2 # [ 135.332855] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_connected addr=lb:500515056worker2 # [ 135.338600] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5057worker1 # [ 135.522746] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5058worker1 # [ 135.527399] systemd[1]: Stopped Nix worker-protocol over gRPC.5059worker1 # [ 135.535903] systemd[1]: nix-grpc-daemon.service: Consumed 2.936s CPU time over 2min 5.191s wall clock time, 67.2M memory peak, 429.7K incoming IP traffic, 289.1K outgoing IP traffic.5060lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5061lb: must succeed: curl -sf localhost:9901/clusters5062worker2 # [ 135.544101] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_connected addr=lb:500515063worker2 # [ 135.549950] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5064lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5065lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.09 seconds)5066client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25067worker2 # [ 135.751518] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_connected addr=lb:500515068worker2 # [ 135.757317] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5069worker2 # [ 135.863368] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525070worker2 # [ 135.871891] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5071worker2 # [ 135.887080] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5)5072worker2 # [ 135.935959] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 2/5)5073worker2 # [ 135.958189] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_connected addr=lb:500515074worker2 # [ 135.963465] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5075worker2 # [ 135.971900] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 315 ms (attempt 3/5)5076worker2 # [ 136.164870] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_connected addr=lb:500515077worker2 # [ 136.170503] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5078worker2 # [ 136.297049] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 4/5)5079worker2 # [ 136.322344] nix-grpc-daemon[1270]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5080worker2 # [ 136.331778] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5081client # copying 2 paths...5082client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5083worker2 # [ 136.359260] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:576525084client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5085worker2 # [ 136.372444] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_connected addr=lb:500515086worker2 # [ 136.390439] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5087worker2 # [ 136.408188] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5088worker2 # [ 136.433118] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:50Z level=info event=niks3_push_started pid=13305089worker2 # [ 136.572372] nix-grpc-daemon[1330]: time=2026-09-24T12:27:51.120Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5090worker2 # [ 136.598460] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_connected addr=lb:500515091worker2 # [ 136.600896] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5092worker2 # [ 136.693267] nix-daemon[844]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5093worker2 # [ 136.715952] nix-daemon[844]: reaped child process 1346, status = succeeded5094lb # [ 136.745285] niks3-server[1025]: 2026/09/24 12:27:51 INFO Received push request method=POST path=/api/pushes5095worker2 # [ 136.731498] nix-grpc-daemon[1330]: time=2026-09-24T12:27:51.278Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5096worker2 # [ 136.735519] nix-grpc-daemon[1330]: time=2026-09-24T12:27:51.278Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5097worker2 # [ 136.738085] nix-grpc-daemon[1330]: time=2026-09-24T12:27:51.278Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5098lb # [ 136.814805] niks3-server[1025]: 2026/09/24 12:27:51 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5099worker2 # [ 136.800097] nix-grpc-daemon[1330]: time=2026-09-24T12:27:51.347Z level=INFO msg="Uploading 2 narinfos"5100worker2 # [ 136.805337] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_connected addr=lb:500515101lb # [ 136.834985] niks3-server[1025]: 2026/09/24 12:27:51 INFO Signed narinfos id=34 count=25102worker2 # [ 136.808846] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5103lb # [ 136.844315] niks3-server[1025]: 2026/09/24 12:27:51 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5104lb # [ 136.847049] niks3-server[1025]: 2026/09/24 12:27:51 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5105lb # [ 136.861899] niks3-server[1025]: 2026/09/24 12:27:51 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5106lb # [ 136.869363] niks3-server[1025]: 2026/09/24 12:27:51 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5107lb # [ 136.880569] niks3-server[1025]: 2026/09/24 12:27:51 INFO Received complete push request method=POST path=/api/pushes/34/complete5108lb # [ 136.888906] niks3-server[1025]: 2026/09/24 12:27:51 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5109worker2 # [ 136.860719] nix-grpc-daemon[1330]: time=2026-09-24T12:27:51.409Z level=INFO msg="Upload complete. (284ms)"5110lb # [ 136.892153] niks3-server[1025]: 2026/09/24 12:27:51 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5111worker2 # [ 136.872363] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 paths=2 nar_bytes_in=3125112worker2 # [ 136.891696] nix-daemon[844]: reaped child process 1328, status = succeeded5113client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.28 seconds)5114worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5115worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5116worker2 # [ 137.012169] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_connected addr=lb:500515117worker1 # [ 137.046109] systemd[1]: Starting Nix worker-protocol over gRPC...5118worker2 # [ 137.016315] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5119worker1 # [ 137.186980] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5120worker1 # [ 137.189454] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5121worker1 # [ 137.193433] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15122worker1 # [ 137.200072] nix-grpc-daemon[1232]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5123worker1 # [ 137.201603] nix-grpc-daemon[1232]: E0000 00:00:1790252871.728723 1232 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5124worker1 # [ 137.204584] nix-grpc-daemon[1232]: E0000 00:00:1790252871.735195 1232 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5125worker1 # [ 137.208800] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:51Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5126worker1 # [ 137.210558] systemd[1]: Started Nix worker-protocol over gRPC.5127worker1 # [ 137.217973] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:51Z level=info event=scheduler_take_over5128lb # [ 137.214252] niks3-server[1025]: 2026/09/24 12:27:51 INFO lead: acquired remote=192.168.1.3:579005129lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5130lb: must succeed: curl -sf localhost:9901/clusters5131worker1 # [ 137.240135] nix-grpc-daemon[1232]: E0000 00:00:1790252871.750957 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5132worker1 # [ 137.242505] nix-grpc-daemon[1232]: E0000 00:00:1790252871.767652 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5133worker2 # [ 137.213345] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_connected addr=lb:500515134worker2 # [ 137.218567] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5135worker1 # [ 137.257591] nix-grpc-daemon[1232]: E0000 00:00:1790252871.767820 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5136worker1 # [ 137.260511] nix-grpc-daemon[1232]: E0000 00:00:1790252871.787321 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5137worker1 # [ 137.275637] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:51Z level=info event=scheduler_connected addr=lb:500515138worker1 # [ 137.277582] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5139lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5140worker2 # [ 137.419628] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_connected addr=lb:500515141worker2 # [ 137.425122] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5142worker1 # [ 137.478629] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:52Z level=info event=scheduler_connected addr=lb:500515143worker1 # [ 137.483850] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5144worker2 # [ 137.626122] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:52Z level=info event=scheduler_connected addr=lb:500515145worker1 # [ 137.671574] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:52Z level=info event=oidc_jwks provider=mock keys=15146worker1 # [ 137.676238] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:519465147worker1 # [ 137.681460] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05148worker1 # [ 137.687876] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:52Z level=info event=scheduler_connected addr=lb:500515149worker1 # [ 137.693197] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:519465150worker1 # [ 137.698408] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05151lb: must succeed: curl -sf localhost:9901/clusters5152lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5153lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.15 seconds)5154(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.75 seconds)5155subtest: a niks3 restart does not cost the leader its role5156lb: must succeed: curl -sf localhost:9901/clusters5157lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5158lb: must succeed: curl -sf localhost:9901/clusters5159lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5160lb # [ 138.502554] niks3-server[1025]: 2026/09/24 12:27:53 INFO Shutdown signal received, draining in-flight requests timeout=10s5161worker2 # [ 138.477850] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:53Z level=info event=scheduler_lock_lost status=2005162lb # [ 138.509453] systemd[1]: Stopping niks3 server...5163worker1 # [ 138.513248] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:53Z level=info event=scheduler_lock_lost status=2005164lb # [ 138.525176] niks3-server[1025]: 2026/09/24 12:27:53 INFO lead: released remote=192.168.1.3:579005165lb # [ 138.536161] systemd[1]: niks3.service: Deactivated successfully.5166lb # [ 138.537823] systemd[1]: Stopped niks3 server.5167lb # [ 138.538824] systemd[1]: niks3.service: Consumed 641ms CPU time over 1min 48.910s wall clock time, 27.7M memory peak, 96.3K incoming IP traffic, 139.5K outgoing IP traffic.5168lb # [ 138.549192] systemd[1]: Starting niks3 server...5169lb # [ 138.610637] niks3-server[1231]: 2026/09/24 12:27:53 goose: no migrations to run. current version: 202609231200005170lb # [ 138.617461] niks3-server[1231]: 2026/09/24 12:27:53 OK 1_commit_pending_closure.sql (4.94ms)5171lb # [ 138.621093] niks3-server[1231]: 2026/09/24 12:27:53 OK 2_object_stats_trigger.sql (2.22ms)5172lb # [ 138.624278] niks3-server[1231]: 2026/09/24 12:27:53 OK 3_commit_push.sql (2.8ms)5173lb # [ 138.626251] niks3-server[1231]: 2026/09/24 12:27:53 goose: up to current file version: 35174lb # [ 138.627793] niks3-server[1231]: 2026/09/24 12:27:53 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5175lb # [ 138.638798] niks3-server[1231]: 2026/09/24 12:27:53 INFO Read proxy enabled — serving cache objects from S35176lb # [ 138.643417] niks3-server[1231]: 2026/09/24 12:27:53 INFO Using socket-activated listener address=0.0.0.0:57515177lb # [ 138.647222] niks3-server[1231]: 2026/09/24 12:27:53 INFO systemd watchdog enabled interval=15s5178lb # [ 138.649689] niks3-server[1231]: 2026/09/24 12:27:53 INFO Starting HTTP server address=0.0.0.0:57515179lb # [ 138.650897] systemd[1]: Started niks3 server.5180lb: waiting for unit niks3.service5181lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5182lb: must succeed: curl -sf localhost:9901/clusters5183lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5184worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5185worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5186lb: must succeed: curl -sf localhost:9901/clusters5187lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5188lb: must succeed: curl -sf localhost:9901/clusters5189lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5190worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5191worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5192client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25193worker1 # [ 139.131109] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:519705194worker1 # [ 139.147545] nix-daemon[855]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5195client # don't know how to build these paths:5196client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5197worker2 # [ 139.142908] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525198worker1 # [ 139.206956] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:53Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05199worker1 # [ 139.213906] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:53Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15200worker2 # [ 139.187887] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525201worker2 # [ 139.204432] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525202client # copying 1 paths to 'lb:50051'...5203client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5204worker2 # [ 139.221123] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:576525205lb # [ 139.253592] envoy[906]: conn peer=[2001:db8:1::4]:45272 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=20637 tx=1942 ms=949645206lb # [ 139.259283] envoy[906]: conn peer=[2001:db8:1::1]:33072 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=325207lb # [ 139.264275] envoy[906]: conn peer=192.168.1.3:34714 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1994 tx=920 ms=969625208worker2 # [ 139.236847] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5209lb # [ 139.269688] envoy[906]: conn peer=[2001:db8:1::1]:33074 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=305 ms=10245210worker2 # [ 139.398985] nix-daemon[844]: accepted connection from pid 1353, user nix-grpc-daemon (trusted)5211worker2 # [ 139.419308] nix-daemon[844]: reaped child process 1356, status = succeeded5212lb # [ 139.452995] niks3-server[1231]: 2026/09/24 12:27:54 INFO Received push request method=POST path=/api/pushes5213worker2 # [ 139.439152] nix-grpc-daemon[1330]: time=2026-09-24T12:27:53.987Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5214worker2 # [ 139.441794] nix-grpc-daemon[1330]: time=2026-09-24T12:27:53.989Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5215lb # [ 139.497709] niks3-server[1231]: 2026/09/24 12:27:54 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5216lb # [ 139.530747] niks3-server[1231]: 2026/09/24 12:27:54 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5217lb # [ 139.553707] niks3-server[1231]: 2026/09/24 12:27:54 INFO Signed narinfos id=35 count=15218lb # [ 139.560397] niks3-server[1231]: 2026/09/24 12:27:54 INFO lead: acquired remote=192.168.1.3:579185219worker2 # [ 139.531579] nix-grpc-daemon[1330]: time=2026-09-24T12:27:54.078Z level=INFO msg="Uploading 1 narinfos"5220lb # [ 139.564842] niks3-server[1231]: 2026/09/24 12:27:54 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5221lb # [ 139.572725] niks3-server[1231]: 2026/09/24 12:27:54 INFO Received complete push request method=POST path=/api/pushes/35/complete5222worker2 # [ 139.556776] nix-grpc-daemon[1330]: time=2026-09-24T12:27:54.104Z level=INFO msg="Upload complete. (300ms)"5223lb # [ 139.586903] niks3-server[1231]: 2026/09/24 12:27:54 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5224worker2 # [ 139.569243] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 paths=1 nar_bytes_in=5445225worker2 # [ 139.588795] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525226worker2 # [ 139.596762] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5227worker2 # [ 139.602711] nix-daemon[844]: reaped child process 1351, status = succeeded5228client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5229client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5230worker2 # [ 139.618697] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5231client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5232client # farm-a> LOG-farm-a-after-niks3-restart5233worker2 # [ 140.664980] nix-daemon[844]: reaped child process 1362, status = succeeded5234worker2 # [ 140.765826] nix-daemon[844]: accepted connection from pid 1378, user nix-grpc-daemon (trusted)5235worker2 # [ 140.785669] nix-daemon[844]: reaped child process 1381, status = succeeded5236lb # [ 140.822767] niks3-server[1231]: 2026/09/24 12:27:55 INFO Received push request method=POST path=/api/pushes5237worker2 # [ 140.812448] nix-grpc-daemon[1330]: time=2026-09-24T12:27:55.359Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5238worker2 # [ 140.814136] nix-grpc-daemon[1330]: time=2026-09-24T12:27:55.359Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5239worker2 # [ 140.865209] nix-grpc-daemon[1330]: time=2026-09-24T12:27:55.413Z level=INFO msg="Uploading 1 narinfos"5240lb # [ 140.880716] niks3-server[1231]: 2026/09/24 12:27:55 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5241lb # [ 140.897748] niks3-server[1231]: 2026/09/24 12:27:55 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5242lb # [ 140.909297] niks3-server[1231]: 2026/09/24 12:27:55 INFO Signed narinfos id=36 count=15243worker1 # [ 140.914107] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:55Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:568605244lb # [ 140.911973] niks3-server[1231]: 2026/09/24 12:27:55 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5245lb # [ 140.930937] niks3-server[1231]: 2026/09/24 12:27:55 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5246lb # [ 140.936332] niks3-server[1231]: 2026/09/24 12:27:55 INFO Received complete push request method=POST path=/api/pushes/36/complete5247worker2 # [ 140.913196] nix-grpc-daemon[1330]: time=2026-09-24T12:27:55.461Z level=INFO msg="Upload complete. (281ms)"5248lb # [ 140.942116] niks3-server[1231]: 2026/09/24 12:27:55 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5249worker2 # [ 140.925357] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05250worker1 # [ 140.962381] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:55Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05251worker1 # [ 140.968911] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:55Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15252worker2 # [ 140.949831] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525253worker2 # [ 140.958840] nix-daemon[844]: reaped child process 1360, status = succeeded5254worker2 # [ 140.967440] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525255worker2 # [ 140.981544] nix-grpc-daemon[1330]: time=2026-09-24T12:27:55.530Z level=INFO msg="All 1 paths already cached"5256client # copying 1 paths to 'lb:50051'...5257client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5258worker2 # [ 140.992973] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:576525259worker2 # [ 141.009447] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5260worker2 # [ 141.158360] nix-daemon[844]: accepted connection from pid 1385, user nix-grpc-daemon (trusted)5261worker2 # [ 141.179123] nix-daemon[844]: reaped child process 1388, status = succeeded5262lb # [ 141.214268] niks3-server[1231]: 2026/09/24 12:27:55 INFO Received push request method=POST path=/api/pushes5263worker2 # [ 141.199455] nix-grpc-daemon[1330]: time=2026-09-24T12:27:55.747Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5264worker2 # [ 141.201140] nix-grpc-daemon[1330]: time=2026-09-24T12:27:55.747Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5265worker2 # [ 141.234169] nix-grpc-daemon[1330]: time=2026-09-24T12:27:55.782Z level=INFO msg="Uploading 1 narinfos"5266lb # [ 141.250509] niks3-server[1231]: 2026/09/24 12:27:55 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5267lb # [ 141.267008] niks3-server[1231]: 2026/09/24 12:27:55 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5268lb # [ 141.275901] niks3-server[1231]: 2026/09/24 12:27:55 INFO Signed narinfos id=37 count=15269lb # [ 141.285992] niks3-server[1231]: 2026/09/24 12:27:55 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5270lb # [ 141.292403] niks3-server[1231]: 2026/09/24 12:27:55 INFO Received complete push request method=POST path=/api/pushes/37/complete5271worker2 # [ 141.264849] nix-grpc-daemon[1330]: time=2026-09-24T12:27:55.813Z level=INFO msg="Upload complete. (241ms)"5272lb # [ 141.295660] niks3-server[1231]: 2026/09/24 12:27:55 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5273worker2 # [ 141.275212] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 paths=1 nar_bytes_in=6565274worker2 # [ 141.284872] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525275worker2 # [ 141.298001] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5276worker2 # [ 141.303267] nix-daemon[844]: reaped child process 1383, status = succeeded5277client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5278worker2 # [ 141.314916] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5279client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5280client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5281client # farm-b> LOG-farm-b-after-niks3-restart5282worker2 # [ 142.610496] nix-daemon[844]: reaped child process 1392, status = succeeded5283worker2 # [ 142.696888] nix-daemon[844]: accepted connection from pid 1408, user nix-grpc-daemon (trusted)5284worker2 # [ 142.720812] nix-daemon[844]: reaped child process 1411, status = succeeded5285lb # [ 142.746373] niks3-server[1231]: 2026/09/24 12:27:57 INFO Received push request method=POST path=/api/pushes5286worker2 # [ 142.731619] nix-grpc-daemon[1330]: time=2026-09-24T12:27:57.279Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5287worker2 # [ 142.733318] nix-grpc-daemon[1330]: time=2026-09-24T12:27:57.279Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5288lb # [ 142.783383] niks3-server[1231]: 2026/09/24 12:27:57 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5289worker2 # [ 142.772805] nix-grpc-daemon[1330]: time=2026-09-24T12:27:57.321Z level=INFO msg="Uploading 1 narinfos"5290lb # [ 142.799875] niks3-server[1231]: 2026/09/24 12:27:57 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5291lb # [ 142.809858] niks3-server[1231]: 2026/09/24 12:27:57 INFO Signed narinfos id=38 count=15292lb # [ 142.816543] niks3-server[1231]: 2026/09/24 12:27:57 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5293lb # [ 142.825691] niks3-server[1231]: 2026/09/24 12:27:57 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5294lb # [ 142.836206] niks3-server[1231]: 2026/09/24 12:27:57 INFO Received complete push request method=POST path=/api/pushes/38/complete5295worker2 # [ 142.816613] nix-grpc-daemon[1330]: time=2026-09-24T12:27:57.365Z level=INFO msg="Upload complete. (288ms)"5296lb # [ 142.845859] niks3-server[1231]: 2026/09/24 12:27:57 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5297worker1 # [ 142.863218] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:57Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05298worker2 # [ 142.829344] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05299worker1 # [ 142.869619] nix-grpc-daemon[1232]: ts=2026-09-24T12:27:57Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15300worker2 # [ 142.852411] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525301worker2 # [ 142.862478] nix-daemon[844]: reaped child process 1390, status = succeeded5302worker2 # [ 142.866088] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525303worker2 # [ 142.879092] nix-grpc-daemon[1330]: time=2026-09-24T12:27:57.427Z level=INFO msg="All 2 paths already cached"5304client # copying 1 paths to 'lb:50051'...5305client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5306worker2 # [ 142.889324] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:576525307worker2 # [ 142.903505] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5308worker2 # [ 143.051001] nix-daemon[844]: accepted connection from pid 1415, user nix-grpc-daemon (trusted)5309worker2 # [ 143.070476] nix-daemon[844]: reaped child process 1418, status = succeeded5310lb # [ 143.106441] niks3-server[1231]: 2026/09/24 12:27:57 INFO Received push request method=POST path=/api/pushes5311worker2 # [ 143.089794] nix-grpc-daemon[1330]: time=2026-09-24T12:27:57.638Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5312worker2 # [ 143.091982] nix-grpc-daemon[1330]: time=2026-09-24T12:27:57.640Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5313lb # [ 143.145252] niks3-server[1231]: 2026/09/24 12:27:57 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5314lb # [ 143.159166] niks3-server[1231]: 2026/09/24 12:27:57 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5315worker2 # [ 143.136761] nix-grpc-daemon[1330]: time=2026-09-24T12:27:57.684Z level=INFO msg="Uploading 1 narinfos"5316lb # [ 143.169253] niks3-server[1231]: 2026/09/24 12:27:57 INFO Signed narinfos id=39 count=15317lb # [ 143.171422] niks3-server[1231]: 2026/09/24 12:27:57 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5318lb # [ 143.180977] niks3-server[1231]: 2026/09/24 12:27:57 INFO Received complete push request method=POST path=/api/pushes/39/complete5319lb # [ 143.188453] niks3-server[1231]: 2026/09/24 12:27:57 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5320worker2 # [ 143.167571] nix-grpc-daemon[1330]: time=2026-09-24T12:27:57.716Z level=INFO msg="Upload complete. (251ms)"5321worker2 # [ 143.180318] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 paths=1 nar_bytes_in=7845322worker2 # [ 143.192891] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525323worker2 # [ 143.204820] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5324worker2 # [ 143.211495] nix-daemon[844]: reaped child process 1413, status = succeeded5325worker2 # [ 143.221957] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5326client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5327client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5328client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5329client # farm-top> LOG-farm-top-after-niks3-restart5330worker2 # [ 144.696805] nix-daemon[844]: reaped child process 1422, status = succeeded5331worker2 # [ 144.812699] nix-daemon[844]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)5332worker2 # [ 144.830865] nix-daemon[844]: reaped child process 1441, status = succeeded5333lb # [ 144.867714] niks3-server[1231]: 2026/09/24 12:27:59 INFO Received push request method=POST path=/api/pushes5334worker2 # [ 144.853442] nix-grpc-daemon[1330]: time=2026-09-24T12:27:59.400Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5335worker2 # [ 144.855457] nix-grpc-daemon[1330]: time=2026-09-24T12:27:59.400Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5336lb # [ 144.914374] niks3-server[1231]: 2026/09/24 12:27:59 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5337worker2 # [ 144.896585] nix-grpc-daemon[1330]: time=2026-09-24T12:27:59.444Z level=INFO msg="Uploading 1 narinfos"5338lb # [ 144.928568] niks3-server[1231]: 2026/09/24 12:27:59 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5339lb # [ 144.938497] niks3-server[1231]: 2026/09/24 12:27:59 INFO Signed narinfos id=40 count=15340lb # [ 144.942879] niks3-server[1231]: 2026/09/24 12:27:59 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5341lb # [ 144.948890] niks3-server[1231]: 2026/09/24 12:27:59 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5342lb # [ 144.956388] niks3-server[1231]: 2026/09/24 12:27:59 INFO Received complete push request method=POST path=/api/pushes/40/complete5343worker2 # [ 144.934055] nix-grpc-daemon[1330]: time=2026-09-24T12:27:59.482Z level=INFO msg="Upload complete. (297ms)"5344lb # [ 144.962999] niks3-server[1231]: 2026/09/24 12:27:59 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5345worker2 # [ 144.949309] nix-grpc-daemon[1270]: ts=2026-09-24T12:27:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05346worker2 # [ 144.967318] nix-daemon[844]: reaped child process 1420, status = succeeded5347client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.11 seconds)5348client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5349client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds)5350(finished: subtest: a niks3 restart does not cost the leader its role, in 6.83 seconds)5351subtest: scheduler down: the other node takes the lock and keeps it5352lb: must succeed: curl -sf localhost:9901/clusters5353lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5354lb: must succeed: curl -sf localhost:9901/clusters5355lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5356lb: must succeed: curl -sf localhost:9901/clusters5357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5358lb: must succeed: curl -sf localhost:9901/clusters5359lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5360worker1 # [ 145.465720] systemd[1]: Stopping Nix worker-protocol over gRPC...5361worker1 # [ 146.217408] nix-grpc-daemon[1232]: ts=2026-09-24T12:28:00Z level=info event=signal_exit5362lb # [ 146.521310] niks3-server[1231]: 2026/09/24 12:28:01 INFO lead: released remote=192.168.1.3:579185363worker2 # [ 147.499104] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:02Z level=info event=scheduler_restarting addr=lb:500515364worker1 # [ 147.529203] nix-grpc-daemon[1232]: ts=2026-09-24T12:28:02Z level=info event=scheduler_restarting5365lb # [ 147.522470] envoy[906]: [2026-09-24 12:28:02.068][906][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5366worker1 # [ 147.529701] nix-grpc-daemon[1232]: ts=2026-09-24T12:28:02Z level=info event=scheduler_restarting addr=lb:500515367worker1 # [ 147.539678] nix-grpc-daemon[1232]: ts=2026-09-24T12:28:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:519465368worker1 # [ 147.552536] nix-grpc-daemon[1232]: ts=2026-09-24T12:28:02Z level=info event=worker_gone addr=192.168.1.4:500515369worker1 # [ 147.561660] nix-grpc-daemon[1232]: ts=2026-09-24T12:28:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:519465370worker1 # [ 147.574445] nix-grpc-daemon[1232]: ts=2026-09-24T12:28:02Z level=info event=worker_gone addr=192.168.1.3:500515371worker1 # [ 147.585870] nix-daemon[855]: reaped child process 1292, status = succeeded5372worker2 # [ 147.699466] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:02Z level=info event=scheduler_connected addr=lb:500515373worker2 # [ 147.701985] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5374worker2 # [ 147.903228] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:02Z level=info event=scheduler_connected addr=lb:500515375worker2 # [ 147.908763] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker2 # [ 148.110160] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:02Z level=info event=scheduler_connected addr=lb:500515377worker2 # [ 148.115608] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5378worker2 # [ 148.316854] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:02Z level=info event=scheduler_connected addr=lb:500515379worker2 # [ 148.322376] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5380worker2 # [ 148.524869] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_connected addr=lb:500515381worker2 # [ 148.529728] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5382worker2 # [ 148.731062] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_connected addr=lb:500515383worker2 # [ 148.736648] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker1 # [ 148.816223] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5385worker1 # [ 148.821632] systemd[1]: Stopped Nix worker-protocol over gRPC.5386worker1 # [ 148.829321] systemd[1]: nix-grpc-daemon.service: Consumed 308ms CPU time over 11.774s wall clock time, 15.4M memory peak, 148.9K incoming IP traffic, 33.4K outgoing IP traffic.5387worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5388worker2 # [ 148.937821] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_connected addr=lb:500515389worker2 # [ 148.943227] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5390worker2 # [ 149.144312] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_connected addr=lb:500515391worker2 # [ 149.149535] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5392lb # [ 149.251394] envoy[906]: conn peer=[2001:db8:1::1]:33082 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9779 tx=4444 ms=58615393lb # [ 149.256586] envoy[906]: conn peer=[2001:db8:1::3]:50982 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=758 tx=293 ms=104705394worker2 # [ 149.350656] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_connected addr=lb:500515395worker2 # [ 149.356104] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5396worker2 # [ 149.486343] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=info event=scheduler_take_over5397lb # [ 149.514989] niks3-server[1231]: 2026/09/24 12:28:04 INFO lead: acquired remote=192.168.1.4:340745398worker2 # [ 149.559023] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=info event=scheduler_connected addr=lb:500515399worker2 # [ 149.563329] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5400worker2 # [ 149.760395] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=info event=scheduler_connected addr=lb:500515401worker2 # [ 149.787431] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=info event=oidc_jwks provider=mock keys=15402worker2 # [ 149.793645] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:417325403worker2 # [ 149.799469] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05404worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.15 seconds)5405worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5406worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5407client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25408worker2 # [ 150.243884] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:576525409client # don't know how to build these paths:5410client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5411worker2 # [ 150.253995] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525412worker2 # [ 150.284303] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05413worker2 # [ 150.291743] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15414worker2 # [ 150.301261] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525415worker2 # [ 150.313557] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525416client # copying 1 paths to 'lb:50051'...5417client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5418worker2 # [ 150.330952] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:576525419worker2 # [ 150.346876] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5420worker2 # [ 150.497440] nix-daemon[844]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5421worker2 # [ 150.518181] nix-daemon[844]: reaped child process 1465, status = succeeded5422lb # [ 150.553408] niks3-server[1231]: 2026/09/24 12:28:05 INFO Received push request method=POST path=/api/pushes5423worker2 # [ 150.540499] nix-grpc-daemon[1330]: time=2026-09-24T12:28:05.088Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5424worker2 # [ 150.542796] nix-grpc-daemon[1330]: time=2026-09-24T12:28:05.088Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5425lb # [ 150.589797] niks3-server[1231]: 2026/09/24 12:28:05 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5426worker2 # [ 150.571011] nix-grpc-daemon[1330]: time=2026-09-24T12:28:05.119Z level=INFO msg="Uploading 1 narinfos"5427lb # [ 150.606412] niks3-server[1231]: 2026/09/24 12:28:05 INFO Signed narinfos id=41 count=15428lb # [ 150.613558] niks3-server[1231]: 2026/09/24 12:28:05 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5429lb # [ 150.626018] niks3-server[1231]: 2026/09/24 12:28:05 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5430worker2 # [ 150.606042] nix-grpc-daemon[1330]: time=2026-09-24T12:28:05.154Z level=INFO msg="Upload complete. (243ms)"5431lb # [ 150.635363] niks3-server[1231]: 2026/09/24 12:28:05 INFO Received complete push request method=POST path=/api/pushes/41/complete5432lb # [ 150.638795] niks3-server[1231]: 2026/09/24 12:28:05 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5433worker2 # [ 150.617174] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 paths=1 nar_bytes_in=5205434worker2 # [ 150.634592] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525435worker2 # [ 150.646021] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5436worker2 # [ 150.650211] nix-daemon[844]: reaped child process 1460, status = succeeded5437client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5438worker2 # [ 150.664565] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5439client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5440worker2 # [ 151.444888] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:05Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:605045441client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5442client # farm-a> LOG-farm-a-on-standby5443worker2 # [ 152.348306] nix-daemon[844]: reaped child process 1469, status = succeeded5444worker2 # [ 152.445422] nix-daemon[844]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)5445worker2 # [ 152.467066] nix-daemon[844]: reaped child process 1488, status = succeeded5446lb # [ 152.502154] niks3-server[1231]: 2026/09/24 12:28:07 INFO Received push request method=POST path=/api/pushes5447worker2 # [ 152.484447] nix-grpc-daemon[1330]: time=2026-09-24T12:28:07.031Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5448worker2 # [ 152.486157] nix-grpc-daemon[1330]: time=2026-09-24T12:28:07.032Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5449lb # [ 152.545035] niks3-server[1231]: 2026/09/24 12:28:07 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5450worker2 # [ 152.525461] nix-grpc-daemon[1330]: time=2026-09-24T12:28:07.073Z level=INFO msg="Uploading 1 narinfos"5451lb # [ 152.565234] niks3-server[1231]: 2026/09/24 12:28:07 INFO Signed narinfos id=42 count=15452lb # [ 152.580353] niks3-server[1231]: 2026/09/24 12:28:07 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5453lb # [ 152.596562] niks3-server[1231]: 2026/09/24 12:28:07 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5454lb # [ 152.615720] niks3-server[1231]: 2026/09/24 12:28:07 INFO Received complete push request method=POST path=/api/pushes/42/complete5455lb # [ 152.618982] niks3-server[1231]: 2026/09/24 12:28:07 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5456lb # [ 152.624664] niks3-server[1231]: 2026/09/24 12:28:07 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5457worker2 # [ 152.597670] nix-grpc-daemon[1330]: time=2026-09-24T12:28:07.146Z level=INFO msg="Upload complete. (317ms)"5458worker2 # [ 152.621488] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05459worker2 # [ 152.640450] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:07Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05460worker2 # [ 152.651473] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:07Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15461worker2 # [ 152.657818] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525462worker2 # [ 152.664225] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525463worker2 # [ 152.670262] nix-daemon[844]: reaped child process 1467, status = succeeded5464worker2 # [ 152.674001] nix-grpc-daemon[1330]: time=2026-09-24T12:28:07.222Z level=INFO msg="All 1 paths already cached"5465client # copying 1 paths to 'lb:50051'...5466client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5467worker2 # [ 152.683847] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:576525468worker2 # [ 152.699650] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5469worker2 # [ 152.843169] nix-daemon[844]: accepted connection from pid 1494, user nix-grpc-daemon (trusted)5470worker2 # [ 152.863795] nix-daemon[844]: reaped child process 1497, status = succeeded5471lb # [ 152.895742] niks3-server[1231]: 2026/09/24 12:28:07 INFO Received push request method=POST path=/api/pushes5472worker2 # [ 152.877757] nix-grpc-daemon[1330]: time=2026-09-24T12:28:07.425Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5473worker2 # [ 152.879471] nix-grpc-daemon[1330]: time=2026-09-24T12:28:07.425Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5474lb # [ 152.926589] niks3-server[1231]: 2026/09/24 12:28:07 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5475worker2 # [ 152.912158] nix-grpc-daemon[1330]: time=2026-09-24T12:28:07.460Z level=INFO msg="Uploading 1 narinfos"5476lb # [ 152.941746] niks3-server[1231]: 2026/09/24 12:28:07 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5477lb # [ 152.951272] niks3-server[1231]: 2026/09/24 12:28:07 INFO Signed narinfos id=43 count=15478lb # [ 152.960708] niks3-server[1231]: 2026/09/24 12:28:07 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5479lb # [ 152.977222] niks3-server[1231]: 2026/09/24 12:28:07 INFO Received complete push request method=POST path=/api/pushes/43/complete5480lb # [ 152.984329] niks3-server[1231]: 2026/09/24 12:28:07 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5481worker2 # [ 152.964136] nix-grpc-daemon[1330]: time=2026-09-24T12:28:07.512Z level=INFO msg="Upload complete. (253ms)"5482worker2 # [ 152.975095] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 paths=1 nar_bytes_in=6405483worker2 # [ 152.997434] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525484worker2 # [ 153.001927] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5485worker2 # [ 153.008479] nix-daemon[844]: reaped child process 1492, status = succeeded5486client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5487worker2 # [ 153.020958] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5488client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5489client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5490client # farm-b> LOG-farm-b-on-standby5491worker2 # [ 153.650984] nix-daemon[844]: reaped child process 1501, status = succeeded5492worker2 # [ 153.758874] nix-daemon[844]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5493worker2 # [ 153.779588] nix-daemon[844]: reaped child process 1520, status = succeeded5494lb # [ 153.814241] niks3-server[1231]: 2026/09/24 12:28:08 INFO Received push request method=POST path=/api/pushes5495worker2 # [ 153.800431] nix-grpc-daemon[1330]: time=2026-09-24T12:28:08.346Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5496worker2 # [ 153.801996] nix-grpc-daemon[1330]: time=2026-09-24T12:28:08.346Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5497lb # [ 153.856767] niks3-server[1231]: 2026/09/24 12:28:08 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5498worker2 # [ 153.843391] nix-grpc-daemon[1330]: time=2026-09-24T12:28:08.391Z level=INFO msg="Uploading 1 narinfos"5499lb # [ 153.870690] niks3-server[1231]: 2026/09/24 12:28:08 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5500lb # [ 153.881543] niks3-server[1231]: 2026/09/24 12:28:08 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5501lb # [ 153.888775] niks3-server[1231]: 2026/09/24 12:28:08 INFO Signed narinfos id=44 count=15502lb # [ 153.891525] niks3-server[1231]: 2026/09/24 12:28:08 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5503lb # [ 153.903572] niks3-server[1231]: 2026/09/24 12:28:08 INFO Received complete push request method=POST path=/api/pushes/44/complete5504worker2 # [ 153.882316] nix-grpc-daemon[1330]: time=2026-09-24T12:28:08.430Z level=INFO msg="Upload complete. (252ms)"5505lb # [ 153.911253] niks3-server[1231]: 2026/09/24 12:28:08 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5506worker2 # [ 153.898625] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05507worker2 # [ 153.920972] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:08Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05508worker2 # [ 153.928405] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:08Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15509worker2 # [ 153.935134] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525510worker2 # [ 153.941693] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525511worker2 # [ 153.947606] nix-daemon[844]: reaped child process 1499, status = succeeded5512worker2 # [ 153.952420] nix-grpc-daemon[1330]: time=2026-09-24T12:28:08.500Z level=INFO msg="All 2 paths already cached"5513client # copying 1 paths to 'lb:50051'...5514client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5515worker2 # [ 153.962749] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:576525516worker2 # [ 153.975240] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5517worker2 # [ 154.110886] nix-daemon[844]: accepted connection from pid 1526, user nix-grpc-daemon (trusted)5518worker2 # [ 154.134706] nix-daemon[844]: reaped child process 1529, status = succeeded5519lb # [ 154.161681] niks3-server[1231]: 2026/09/24 12:28:08 INFO Received push request method=POST path=/api/pushes5520worker2 # [ 154.144281] nix-grpc-daemon[1330]: time=2026-09-24T12:28:08.692Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5521worker2 # [ 154.146373] nix-grpc-daemon[1330]: time=2026-09-24T12:28:08.694Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5522lb # [ 154.198240] niks3-server[1231]: 2026/09/24 12:28:08 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5523worker2 # [ 154.180970] nix-grpc-daemon[1330]: time=2026-09-24T12:28:08.729Z level=INFO msg="Uploading 1 narinfos"5524lb # [ 154.212848] niks3-server[1231]: 2026/09/24 12:28:08 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign5525lb # [ 154.220604] niks3-server[1231]: 2026/09/24 12:28:08 INFO Signed narinfos id=45 count=15526lb # [ 154.226410] niks3-server[1231]: 2026/09/24 12:28:08 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5527lb # [ 154.234762] niks3-server[1231]: 2026/09/24 12:28:08 INFO Received complete push request method=POST path=/api/pushes/45/complete5528lb # [ 154.242977] niks3-server[1231]: 2026/09/24 12:28:08 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5529worker2 # [ 154.214862] nix-grpc-daemon[1330]: time=2026-09-24T12:28:08.763Z level=INFO msg="Upload complete. (227ms)"5530worker2 # [ 154.224325] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 paths=1 nar_bytes_in=7685531worker2 # [ 154.244734] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525532worker2 # [ 154.248559] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5533worker2 # [ 154.259183] nix-daemon[844]: reaped child process 1524, status = succeeded5534client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5535client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5536worker2 # [ 154.266919] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5537client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5538client # farm-top> LOG-farm-top-on-standby5539worker2 # [ 154.898144] nix-daemon[844]: reaped child process 1533, status = succeeded5540worker2 # [ 154.991683] nix-daemon[844]: accepted connection from pid 1549, user nix-grpc-daemon (trusted)5541worker2 # [ 155.009309] nix-daemon[844]: reaped child process 1552, status = succeeded5542lb # [ 155.045860] niks3-server[1231]: 2026/09/24 12:28:09 INFO Received push request method=POST path=/api/pushes5543worker2 # [ 155.036453] nix-grpc-daemon[1330]: time=2026-09-24T12:28:09.583Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5544worker2 # [ 155.038010] nix-grpc-daemon[1330]: time=2026-09-24T12:28:09.583Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5545worker2 # [ 155.082937] nix-grpc-daemon[1330]: time=2026-09-24T12:28:09.631Z level=INFO msg="Uploading 1 narinfos"5546lb # [ 155.101654] niks3-server[1231]: 2026/09/24 12:28:09 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign5547lb # [ 155.118705] niks3-server[1231]: 2026/09/24 12:28:09 INFO Signed narinfos id=46 count=15548lb # [ 155.126534] niks3-server[1231]: 2026/09/24 12:28:09 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5549lb # [ 155.134278] niks3-server[1231]: 2026/09/24 12:28:09 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5550lb # [ 155.144595] niks3-server[1231]: 2026/09/24 12:28:09 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5551lb # [ 155.147501] niks3-server[1231]: 2026/09/24 12:28:09 INFO Received complete push request method=POST path=/api/pushes/46/complete5552worker2 # [ 155.126103] nix-grpc-daemon[1330]: time=2026-09-24T12:28:09.674Z level=INFO msg="Upload complete. (250ms)"5553lb # [ 155.155091] niks3-server[1231]: 2026/09/24 12:28:09 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5554worker2 # [ 155.141258] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05555worker2 # [ 155.160537] nix-daemon[844]: reaped child process 1531, status = succeeded5556client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.17 seconds)5557client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5558client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds)5559worker1 # [ 155.471265] systemd[1]: Starting Nix worker-protocol over gRPC...5560worker1 # [ 155.610717] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5561worker1 # [ 155.613157] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5562worker1 # [ 155.617139] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15563worker1 # [ 155.623336] nix-grpc-daemon[1318]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5564worker1 # [ 155.626674] nix-grpc-daemon[1318]: E0000 00:00:1790252890.153229 1318 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5565worker1 # [ 155.648191] nix-grpc-daemon[1318]: E0000 00:00:1790252890.157141 1318 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5566worker1 # [ 155.650599] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:10Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5567lb: must succeed: curl -sf localhost:9901/clusters5568worker1 # [ 155.676672] nix-grpc-daemon[1318]: E0000 00:00:1790252890.167821 1358 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5569worker1 # [ 155.679230] systemd[1]: Started Nix worker-protocol over gRPC.5570worker1 # [ 155.680082] nix-grpc-daemon[1318]: E0000 00:00:1790252890.189215 1358 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5571worker1 # [ 155.682299] nix-grpc-daemon[1318]: E0000 00:00:1790252890.189400 1358 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5572worker1 # [ 155.684724] nix-grpc-daemon[1318]: E0000 00:00:1790252890.205875 1358 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5573worker1 # [ 155.693867] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:10Z level=info event=scheduler_connected addr=lb:500515574worker2 # [ 155.665725] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:417325575worker2 # [ 155.674371] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05576lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5577worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5578worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5579lb: must succeed: curl -sf localhost:9901/clusters5580lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5581lb: must succeed: curl -sf localhost:9901/clusters5582lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5583worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5584worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5585(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.74 seconds)5586subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5587client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5588client # [ 156.023617] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5589client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5590client # Running as unit: rs.service; invocation ID: 9474b1f3c0674535b4823f162455f8325591client # [ 156.233312] nix[1405]: don't know how to build these paths:5592client # [ 156.235304] nix[1405]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5593worker2 # [ 156.190133] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:576525594worker2 # [ 156.205247] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:10Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05595worker2 # [ 156.214748] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:10Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15596worker2 # [ 156.223501] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525597worker2 # [ 156.231337] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576525598client # [ 156.283295] nix[1405]: copying 1 paths to 'lb:50051'...5599client # [ 156.285468] nix[1405]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5600worker2 # [ 156.242441] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:576525601worker2 # [ 156.255964] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5602worker2 # [ 156.407149] nix-daemon[844]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5603worker2 # [ 156.426996] nix-daemon[844]: reaped child process 1568, status = succeeded5604lb # [ 156.461660] niks3-server[1231]: 2026/09/24 12:28:11 INFO Received push request method=POST path=/api/pushes5605worker2 # [ 156.445635] nix-grpc-daemon[1330]: time=2026-09-24T12:28:10.993Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5606worker2 # [ 156.447195] nix-grpc-daemon[1330]: time=2026-09-24T12:28:10.993Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5607lb # [ 156.495498] niks3-server[1231]: 2026/09/24 12:28:11 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign5608worker2 # [ 156.479950] nix-grpc-daemon[1330]: time=2026-09-24T12:28:11.028Z level=INFO msg="Uploading 1 narinfos"5609lb # [ 156.512565] niks3-server[1231]: 2026/09/24 12:28:11 INFO Signed narinfos id=47 count=15610lb # [ 156.515685] niks3-server[1231]: 2026/09/24 12:28:11 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5611lb # [ 156.528051] niks3-server[1231]: 2026/09/24 12:28:11 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5612lb # [ 156.544248] niks3-server[1231]: 2026/09/24 12:28:11 INFO Received complete push request method=POST path=/api/pushes/47/complete5613lb # [ 156.548138] niks3-server[1231]: 2026/09/24 12:28:11 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5614worker2 # [ 156.525494] nix-grpc-daemon[1330]: time=2026-09-24T12:28:11.073Z level=INFO msg="Upload complete. (253ms)"5615worker2 # [ 156.535653] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=0 paths=1 nar_bytes_in=4165616worker2 # [ 156.552554] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576525617worker2 # [ 156.562827] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5618worker2 # [ 156.569184] nix-daemon[844]: reaped child process 1563, status = succeeded5619client # [ 156.623201] nix[1405]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5620worker2 # [ 156.582714] nix-daemon[844]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5621lb: must succeed: curl -sf localhost:9901/clusters5622lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5623lb: must succeed: curl -sf localhost:9901/clusters5624lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5625lb: must succeed: curl -sf localhost:9901/clusters5626lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5627lb: must succeed: curl -sf localhost:9901/clusters5628lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5629worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5630worker1 # [ 158.860550] systemd-journald[270]: Received client request to rotate journal, rotating.5631worker1 # [ 158.882712] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e253ff6eeaba407ea73cee4d67ae01c8.5632worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5633worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5634worker2 # [ 158.907499] systemd[1]: Stopping Nix worker-protocol over gRPC...5635worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5636client: must succeed: sleep 5.05637lb # [ 159.251932] envoy[906]: conn peer=[2001:db8:1::1]:42650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9576 tx=4374 ms=49085638worker2 # [ 159.262176] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:13Z level=info event=draining5639client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5640worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5641worker2 # [ 164.501322] nix-daemon[844]: reaped child process 1572, status = succeeded5642worker2 # [ 164.572590] nix-daemon[844]: accepted connection from pid 1613, user nix-grpc-daemon (trusted)5643worker2 # [ 164.594246] nix-daemon[844]: reaped child process 1616, status = succeeded5644lb # [ 164.626306] niks3-server[1231]: 2026/09/24 12:28:19 INFO Received push request method=POST path=/api/pushes5645worker2 # [ 164.611764] nix-grpc-daemon[1330]: time=2026-09-24T12:28:19.158Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5646worker2 # [ 164.613675] nix-grpc-daemon[1330]: time=2026-09-24T12:28:19.158Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5647lb # [ 164.668491] niks3-server[1231]: 2026/09/24 12:28:19 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign5648worker2 # [ 164.649836] nix-grpc-daemon[1330]: time=2026-09-24T12:28:19.198Z level=INFO msg="Uploading 1 narinfos"5649lb # [ 164.683732] niks3-server[1231]: 2026/09/24 12:28:19 INFO Signed narinfos id=48 count=15650lb # [ 164.691030] niks3-server[1231]: 2026/09/24 12:28:19 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5651lb # [ 164.708956] niks3-server[1231]: 2026/09/24 12:28:19 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5652lb # [ 164.714045] niks3-server[1231]: 2026/09/24 12:28:19 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5653lb # [ 164.717568] niks3-server[1231]: 2026/09/24 12:28:19 INFO Received complete push request method=POST path=/api/pushes/48/complete5654worker2 # [ 164.693807] nix-grpc-daemon[1330]: time=2026-09-24T12:28:19.242Z level=INFO msg="Upload complete. (290ms)"5655lb # [ 164.725311] niks3-server[1231]: 2026/09/24 12:28:19 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5656worker2 # [ 164.709179] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57652 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05657worker2 # [ 164.729574] nix-daemon[844]: reaped child process 1570, status = succeeded5658client # [ 164.784595] systemd[1]: rs.service: Deactivated successfully.5659client # [ 164.788916] systemd[1]: rs.service: Consumed 202ms CPU time over 8.764s wall clock time, 41.4M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic.5660worker2 # [ 165.264199] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:19Z level=info event=signal_exit5661lb # [ 165.524775] niks3-server[1231]: 2026/09/24 12:28:20 INFO lead: released remote=192.168.1.4:340745662lb # [ 165.634865] niks3-server[1231]: 2026/09/24 12:28:20 INFO lead: acquired remote=192.168.1.3:382725663worker1 # [ 165.642925] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:20Z level=info event=scheduler_take_over5664worker1 # [ 166.524712] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:21Z level=info event=scheduler_restarting addr=lb:500515665worker2 # [ 166.495652] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:21Z level=info event=scheduler_restarting5666lb # [ 166.521177] envoy[906]: [2026-09-24 12:28:21.065][906][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5667worker2 # [ 166.499474] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:21Z level=info event=scheduler_restarting addr=lb:500515668worker2 # [ 166.506349] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:21Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:417325669worker2 # [ 166.514288] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:21Z level=info event=worker_gone addr=192.168.1.4:500515670worker2 # [ 166.518833] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:21Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:417325671worker2 # [ 166.524151] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:21Z level=info event=worker_gone addr=192.168.1.3:500515672worker2 # [ 166.553271] nix-daemon[844]: reaped child process 1322, status = succeeded5673worker1 # [ 166.727252] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:21Z level=info event=scheduler_connected addr=lb:500515674worker1 # [ 166.737270] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:372625675worker1 # [ 166.743201] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05676worker2 # [ 166.720455] nix-grpc-daemon[1270]: ts=2026-09-24T12:28:21Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5677worker1 # [ 166.905871] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:21Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:372785678worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.26 seconds)5679client: waiting for success: ! systemctl is-active rs5680client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5681client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5682client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5683client: must fail: journalctl -u rs | grep -q 'reconnecting'5684client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5685lb: must succeed: curl -sf localhost:9901/clusters5686lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5687worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5688worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5689worker2 # [ 167.819542] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5690worker2 # [ 167.826183] systemd[1]: Stopped Nix worker-protocol over gRPC.5691worker2 # [ 167.831830] systemd[1]: nix-grpc-daemon.service: Consumed 2.591s CPU time over 35.722s wall clock time, 47.1M memory peak, 380.7K incoming IP traffic, 238.8K outgoing IP traffic.5692worker2 # [ 167.850360] systemd[1]: Starting scheduler-token.service...5693worker2 # [ 167.927489] systemd[1]: scheduler-token.service: Deactivated successfully.5694worker2 # [ 167.932469] systemd[1]: Finished scheduler-token.service.5695worker2 # [ 167.933991] systemd[1]: scheduler-token.service: Consumed 47ms CPU time over 79ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5696worker2 # [ 167.942454] systemd[1]: Starting Nix worker-protocol over gRPC...5697worker2 # [ 168.065113] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5698worker2 # [ 168.067447] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5699worker2 # [ 168.071634] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15700worker2 # [ 168.077015] nix-grpc-daemon[1627]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5701worker2 # [ 168.078528] nix-grpc-daemon[1627]: E0000 00:00:1790252902.624799 1627 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5702worker2 # [ 168.082578] nix-grpc-daemon[1627]: E0000 00:00:1790252902.630907 1627 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5703worker2 # [ 168.121540] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:22Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5704worker2 # [ 168.124403] nix-grpc-daemon[1627]: E0000 00:00:1790252902.639682 1667 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5705worker2 # [ 168.127307] nix-grpc-daemon[1627]: E0000 00:00:1790252902.655483 1667 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5706worker2 # [ 168.129995] nix-grpc-daemon[1627]: E0000 00:00:1790252902.655635 1667 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5707worker2 # [ 168.135427] nix-grpc-daemon[1627]: E0000 00:00:1790252902.670009 1667 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5708worker2 # [ 168.140101] systemd[1]: Started Nix worker-protocol over gRPC.5709worker2 # [ 168.141073] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:22Z level=info event=scheduler_connected addr=lb:500515710worker1 # [ 168.181307] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:22Z level=info event=oidc_jwks provider=mock keys=15711worker1 # [ 168.181678] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:372625712worker1 # [ 168.181868] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05713lb: must succeed: curl -sf localhost:9901/clusters5714lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5715worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5716worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5717(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.98 seconds)5718subtest: scheduler restart mid-build: build finishes, no double build5719client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5720client # [ 169.006428] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5721client # Running as unit: mid.service; invocation ID: 29d7364aa66a436aa2027bc80fdf05db5722client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5723worker1 # [ 169.215122] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372805724worker1 # [ 169.229376] nix-daemon[855]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5725client # [ 169.259745] nix[1441]: don't know how to build these paths:5726client # [ 169.261639] nix[1441]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5727lb # [ 169.257758] envoy[906]: conn peer=[2001:db8:1::1]:52516 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2983 tx=1557 ms=85315728lb # [ 169.264063] envoy[906]: conn peer=[2001:db8:1::4]:54628 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=24699 tx=1859 ms=344085729worker1 # [ 169.273326] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:23Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05730worker1 # [ 169.280282] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:23Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15731worker1 # [ 169.298341] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372805732worker1 # [ 169.303682] nix-grpc-daemon[1318]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)5733worker1 # [ 169.374146] nix-grpc-daemon[1318]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 2/5)5734worker1 # [ 169.385014] nix-grpc-daemon[1318]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 333 ms (attempt 3/5)5735worker1 # [ 169.727490] nix-grpc-daemon[1318]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 161 ms (attempt 4/5)5736worker1 # [ 169.899173] nix-grpc-daemon[1318]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5737worker1 # [ 169.904978] nix-grpc-daemon[1318]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5738worker1 # [ 169.923422] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372805739client # [ 169.951738] nix[1441]: copying 1 paths to 'lb:50051'...5740worker1 # [ 169.939147] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372805741client # [ 169.952253] nix[1441]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5742worker1 # [ 169.953792] nix-daemon[855]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5743worker1 # [ 169.984341] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:24Z level=info event=niks3_push_started pid=14465744worker1 # [ 170.142456] nix-grpc-daemon[1446]: time=2026-09-24T12:28:24.672Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5745worker1 # [ 170.279401] nix-daemon[855]: accepted connection from pid 1459, user nix-grpc-daemon (trusted)5746worker1 # [ 170.307915] nix-daemon[855]: reaped child process 1466, status = succeeded5747lb # [ 170.306684] niks3-server[1231]: 2026/09/24 12:28:24 INFO Received push request method=POST path=/api/pushes5748worker1 # [ 170.317531] nix-grpc-daemon[1446]: time=2026-09-24T12:28:24.848Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5749worker1 # [ 170.322203] nix-grpc-daemon[1446]: time=2026-09-24T12:28:24.849Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5750lb # [ 170.345113] niks3-server[1231]: 2026/09/24 12:28:24 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign5751worker1 # [ 170.357363] nix-grpc-daemon[1446]: time=2026-09-24T12:28:24.887Z level=INFO msg="Uploading 1 narinfos"5752lb # [ 170.360544] niks3-server[1231]: 2026/09/24 12:28:24 INFO Signed narinfos id=49 count=15753lb # [ 170.368312] niks3-server[1231]: 2026/09/24 12:28:24 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5754lb # [ 170.371892] niks3-server[1231]: 2026/09/24 12:28:24 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5755lb # [ 170.386889] niks3-server[1231]: 2026/09/24 12:28:24 INFO Received complete push request method=POST path=/api/pushes/49/complete5756lb # [ 170.391194] niks3-server[1231]: 2026/09/24 12:28:24 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5757worker1 # [ 170.398898] nix-grpc-daemon[1446]: time=2026-09-24T12:28:24.929Z level=INFO msg="Upload complete. (252ms)"5758worker1 # [ 170.408862] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37280 duration_s=0 paths=1 nar_bytes_in=4245759worker1 # [ 170.422609] nix-grpc-daemon[1318]: ts=2026-09-24T12:28:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372805760worker1 # [ 170.428507] nix-daemon[855]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5761worker1 # [ 170.433031] nix-daemon[855]: reaped child process 1444, status = succeeded5762worker1 # [ 170.444904] nix-daemon[855]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5763client # [ 170.455480] nix[1441]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5764lb: must succeed: curl -sf localhost:9901/clusters5765lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5766lb: must succeed: curl -sf localhost:9901/clusters5767lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5768worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5769worker1 # [ 172.928786] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1318 (nix-grpc-daemon) on client request.5770worker1 # [ 172.930386] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5771lb # [ 172.949696] niks3-server[1231]: 2026/09/24 12:28:27 INFO lead: released remote=192.168.1.3:382725772client # [ 172.972732] nix[1441]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5773worker2 # [ 172.930155] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:27Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5774lb # [ 172.957533] envoy[906]: [2026-09-24 12:28:27.503][906][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5775client # [ 172.973118] nix[1441]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5776worker1 # [ 172.971360] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5777worker1 # [ 172.978979] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5778worker1 # [ 172.985343] systemd[1]: nix-grpc-daemon.service: Consumed 554ms CPU time over 17.484s wall clock time, 35M memory peak, 216.9K incoming IP traffic, 53.2K outgoing IP traffic.5779worker1 # [ 172.987351] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5780worker1 # [ 172.992146] systemd[1]: Starting Nix worker-protocol over gRPC...5781worker1 # [ 173.035581] nix-daemon[855]: reaped child process 1468, status = succeeded5782worker1 # [ 173.039847] nix-daemon[855]: reaped child process 1436, status = succeeded5783worker1 # [ 173.048844] nix-daemon[1470]: unexpected Nix daemon error: error: interrupted by the user5784worker1 # [ 173.104316] nix-daemon[855]: reaped child process 1470, status = failed with exit code 15785worker1 # [ 173.109123] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5786worker1 # [ 173.111293] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5787worker1 # [ 173.115132] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15788worker1 # [ 173.120079] nix-grpc-daemon[1506]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5789worker1 # [ 173.123416] nix-grpc-daemon[1506]: E0000 00:00:1790252907.650396 1506 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5790lb # [ 173.120297] niks3-server[1231]: 2026/09/24 12:28:27 INFO lead: acquired remote=192.168.1.4:398345791worker2 # [ 173.093523] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:27Z level=info event=scheduler_take_over5792worker1 # [ 173.132833] nix-grpc-daemon[1506]: E0000 00:00:1790252907.653984 1506 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5793worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.24 seconds)5794client: must succeed: sleep 5.05795worker1 # [ 173.137308] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:27Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5796worker1 # [ 173.162011] systemd[1]: Started Nix worker-protocol over gRPC.5797worker2 # [ 173.130517] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:27Z level=info event=scheduler_connected addr=lb:500515798worker1 # [ 173.163061] nix-grpc-daemon[1506]: E0000 00:00:1790252907.670569 1547 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5799worker1 # [ 173.178035] nix-grpc-daemon[1506]: E0000 00:00:1790252907.685591 1547 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5800worker1 # [ 173.180938] nix-grpc-daemon[1506]: E0000 00:00:1790252907.685733 1547 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5801worker2 # [ 173.148717] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5802worker1 # [ 173.183503] nix-grpc-daemon[1506]: E0000 00:00:1790252907.708009 1547 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5803worker1 # [ 173.191043] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:27Z level=info event=scheduler_connected addr=lb:500515804worker1 # [ 173.193850] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5805worker2 # [ 173.350138] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:27Z level=info event=scheduler_connected addr=lb:500515806worker2 # [ 173.355572] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5807worker1 # [ 173.394850] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:27Z level=info event=scheduler_connected addr=lb:500515808worker1 # [ 173.401297] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5809client # [ 173.476611] nix[1441]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting5810worker2 # [ 173.556586] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515811worker2 # [ 173.561909] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5812worker1 # [ 173.602358] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515813worker1 # [ 173.608577] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5814worker2 # [ 173.762946] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515815worker2 # [ 173.768321] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5816worker1 # [ 173.809444] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515817worker1 # [ 173.814629] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5818worker2 # [ 173.971101] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515819worker2 # [ 173.975356] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5820worker1 # [ 174.015411] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515821worker1 # [ 174.020697] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5822worker2 # [ 174.172863] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515823worker2 # [ 174.177886] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5824worker1 # [ 174.223315] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515825worker1 # [ 174.227542] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5826worker2 # [ 174.379062] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515827worker2 # [ 174.384448] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5828worker1 # [ 174.423646] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_connected addr=lb:500515829worker1 # [ 174.429059] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5830client # [ 174.480337] nix[1441]: scheduler at lb:50051: no healthy upstream, reconnecting5831worker2 # [ 174.585736] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:29Z level=info event=scheduler_connected addr=lb:500515832worker2 # [ 174.590956] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5833worker1 # [ 174.629789] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:29Z level=info event=scheduler_connected addr=lb:500515834worker1 # [ 174.634934] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5835worker2 # [ 174.792115] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:29Z level=info event=scheduler_connected addr=lb:500515836worker1 # [ 174.835717] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:29Z level=info event=scheduler_connected addr=lb:500515837worker2 # [ 174.813382] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:471765838worker2 # [ 174.819617] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05839worker2 # [ 174.832128] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:29Z level=info event=oidc_jwks provider=mock keys=15840worker2 # [ 174.838391] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:471765841worker2 # [ 174.843923] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05842worker2 # [ 175.292599] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:29Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:471845843worker2 # [ 176.450223] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:30Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05844worker2 # [ 176.456780] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:30Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15845worker1 # [ 176.497346] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:465825846worker1 # [ 176.512681] nix-daemon[855]: accepted connection from pid 1506, user nix-grpc-daemon (trusted)5847worker1 # [ 176.532728] nix-daemon[855]: accepted connection from pid 1506, user nix-grpc-daemon (trusted)5848client # [ 176.567204] nix[1441]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5849worker1 # [ 176.556394] nix-daemon[855]: accepted connection from pid 1506, user nix-grpc-daemon (trusted)5850client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5851client: waiting for success: ! systemctl is-active mid5852worker1 # [ 178.581798] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:33Z level=info event=niks3_push_started pid=15885853worker1 # [ 178.668729] nix-daemon[855]: reaped child process 1563, status = succeeded5854worker1 # [ 178.773299] nix-grpc-daemon[1588]: time=2026-09-24T12:28:33.303Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5855worker1 # [ 178.881333] nix-daemon[855]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)5856worker1 # [ 178.898155] nix-daemon[855]: reaped child process 1604, status = succeeded5857lb # [ 178.904593] niks3-server[1231]: 2026/09/24 12:28:33 INFO Received push request method=POST path=/api/pushes5858worker1 # [ 178.918161] nix-grpc-daemon[1588]: time=2026-09-24T12:28:33.448Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5859worker1 # [ 178.919683] nix-grpc-daemon[1588]: time=2026-09-24T12:28:33.448Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5860lb # [ 178.970728] niks3-server[1231]: 2026/09/24 12:28:33 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign5861lb # [ 178.982111] niks3-server[1231]: 2026/09/24 12:28:33 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5862lb # [ 179.000641] niks3-server[1231]: 2026/09/24 12:28:33 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5863lb # [ 179.016742] niks3-server[1231]: 2026/09/24 12:28:33 INFO Signed narinfos id=50 count=15864worker1 # [ 179.019564] nix-grpc-daemon[1588]: time=2026-09-24T12:28:33.549Z level=INFO msg="Uploading 1 narinfos"5865lb # [ 179.028186] niks3-server[1231]: 2026/09/24 12:28:33 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5866lb # [ 179.040669] niks3-server[1231]: 2026/09/24 12:28:33 INFO Received complete push request method=POST path=/api/pushes/50/complete5867lb # [ 179.052201] niks3-server[1231]: 2026/09/24 12:28:33 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5868worker1 # [ 179.059031] nix-grpc-daemon[1588]: time=2026-09-24T12:28:33.589Z level=INFO msg="Upload complete. (282ms)"5869worker1 # [ 179.072340] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46582 duration_s=2 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05870worker1 # [ 179.092493] nix-daemon[855]: reaped child process 1561, status = succeeded5871client # [ 179.119462] systemd[1]: mid.service: Deactivated successfully.5872client # [ 179.123659] systemd[1]: mid.service: Consumed 213ms CPU time over 10.114s wall clock time, 43.5M memory peak, 9.2K incoming IP traffic, 11.3K outgoing IP traffic.5873lb # [ 179.257786] envoy[906]: conn peer=[2001:db8:1::3]:43832 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=696 tx=328 ms=172805874lb # [ 179.263231] envoy[906]: conn peer=[2001:db8:1::1]:60822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4301 tx=2090 ms=98855875client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)5876client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5877client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5878(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.69 seconds)5879subtest: interrupting the client stops the build on the worker5880client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5881client # [ 179.668363] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5882client # Running as unit: intr.service; invocation ID: 86ec19ca036449db82241a0cdc5893be5883client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)5884worker2 # [ 179.818129] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471965885worker2 # [ 179.832518] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)5886client # [ 179.890828] nix[1477]: don't know how to build these paths:5887client # [ 179.893097] nix[1477]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5888worker2 # [ 179.858542] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:34Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05889worker2 # [ 179.871042] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:34Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15890worker2 # [ 179.878211] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471965891worker2 # [ 179.884202] nix-grpc-daemon[1627]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)5892worker2 # [ 179.947208] nix-grpc-daemon[1627]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5)5893worker2 # [ 179.956705] nix-grpc-daemon[1627]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 227 ms (attempt 3/5)5894worker2 # [ 180.185439] nix-grpc-daemon[1627]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 206 ms (attempt 4/5)5895worker2 # [ 180.394092] nix-grpc-daemon[1627]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5896worker2 # [ 180.401658] nix-grpc-daemon[1627]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5897worker2 # [ 180.419093] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471965898client # [ 180.474293] nix[1477]: copying 1 paths to 'lb:50051'...5899client # [ 180.479962] nix[1477]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5900worker2 # [ 180.437745] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471965901worker2 # [ 180.453095] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)5902worker2 # [ 180.480548] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:35Z level=info event=niks3_push_started pid=17235903worker2 # [ 180.629694] nix-grpc-daemon[1723]: time=2026-09-24T12:28:35.177Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5904worker2 # [ 180.736929] nix-daemon[844]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)5905worker2 # [ 180.758210] nix-daemon[844]: reaped child process 1739, status = succeeded5906lb # [ 180.785666] niks3-server[1231]: 2026/09/24 12:28:35 INFO Received push request method=POST path=/api/pushes5907worker2 # [ 180.769722] nix-grpc-daemon[1723]: time=2026-09-24T12:28:35.317Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5908worker2 # [ 180.771258] nix-grpc-daemon[1723]: time=2026-09-24T12:28:35.317Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5909lb # [ 180.830697] niks3-server[1231]: 2026/09/24 12:28:35 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign5910worker2 # [ 180.815685] nix-grpc-daemon[1723]: time=2026-09-24T12:28:35.363Z level=INFO msg="Uploading 1 narinfos"5911lb # [ 180.843858] niks3-server[1231]: 2026/09/24 12:28:35 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5912lb # [ 180.854275] niks3-server[1231]: 2026/09/24 12:28:35 INFO Signed narinfos id=51 count=15913lb # [ 180.865130] niks3-server[1231]: 2026/09/24 12:28:35 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5914lb # [ 180.875719] niks3-server[1231]: 2026/09/24 12:28:35 INFO Received complete push request method=POST path=/api/pushes/51/complete5915worker2 # [ 180.855734] nix-grpc-daemon[1723]: time=2026-09-24T12:28:35.404Z level=INFO msg="Upload complete. (223ms)"5916lb # [ 180.882334] niks3-server[1231]: 2026/09/24 12:28:35 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5917worker2 # [ 180.865241] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=0 paths=1 nar_bytes_in=4245918worker2 # [ 180.881083] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471965919worker2 # [ 180.893422] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)5920worker2 # [ 180.903206] nix-daemon[844]: reaped child process 1721, status = succeeded5921client # [ 180.963358] nix[1477]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5922worker2 # [ 180.926079] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)5923client: must succeed: systemctl kill -s INT intr5924client # [ 183.308812] systemd[1]: intr.service: Sent signal SIGINT to main process 1477 (nix) on client request.5925worker2 # [ 183.271015] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:37Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5926client # [ 183.317975] nix[1477]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5927client # [ 183.331589] nix[1477]: error: interrupted by the user5928client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5929client # [ 183.360813] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5930client # [ 183.366468] systemd[1]: intr.service: Failed with result 'exit-code'.5931client # [ 183.369240] systemd[1]: intr.service: Consumed 172ms CPU time over 3.696s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.5932worker2 # [ 183.362610] nix-daemon[1747]: unexpected Nix daemon error: error: interrupted by the user5933worker2 # [ 183.375991] nix-daemon[844]: reaped child process 1744, status = succeeded5934worker2 # [ 183.379080] nix-daemon[844]: reaped child process 1747, status = failed with exit code 15935(finished: subtest: interrupting the client stops the build on the worker, in 3.82 seconds)5936subtest: a deploy mid-build drains: the build finishes, the new generation takes over5937client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5938client # [ 183.516650] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5939client # Running as unit: sw.service; invocation ID: 086ee4bc300a4b5e944afdd3a48d846e5940client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)5941client # [ 183.732665] nix[1498]: don't know how to build these paths:5942client # [ 183.733680] nix[1498]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5943worker2 # [ 183.688138] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471965944worker2 # [ 183.706685] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:38Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05945worker2 # [ 183.714224] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:38Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15946worker1 # [ 183.751004] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:465825947worker1 # [ 183.772300] nix-grpc-daemon[1506]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)5948worker1 # [ 183.803524] nix-grpc-daemon[1506]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)5949worker1 # [ 183.962110] nix-grpc-daemon[1506]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 3/5)5950worker1 # [ 184.155118] nix-grpc-daemon[1506]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 433 ms (attempt 4/5)5951worker1 # [ 184.591583] nix-grpc-daemon[1506]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5952worker1 # [ 184.598415] nix-grpc-daemon[1506]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5953worker1 # [ 184.623749] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:465825954client # [ 184.652503] nix[1498]: copying 1 paths to 'lb:50051'...5955client # [ 184.655704] nix[1498]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5956worker1 # [ 184.645336] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:465825957worker1 # [ 184.664800] nix-daemon[855]: accepted connection from pid 1506, user nix-grpc-daemon (trusted)5958worker1 # [ 184.863589] nix-daemon[855]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)5959worker1 # [ 184.895522] nix-daemon[855]: reaped child process 1643, status = succeeded5960lb # [ 184.891368] niks3-server[1231]: 2026/09/24 12:28:39 INFO Received push request method=POST path=/api/pushes5961worker1 # [ 184.909258] nix-grpc-daemon[1588]: time=2026-09-24T12:28:39.439Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5962worker1 # [ 184.912355] nix-grpc-daemon[1588]: time=2026-09-24T12:28:39.441Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5963lb # [ 184.935992] niks3-server[1231]: 2026/09/24 12:28:39 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5964lb # [ 184.950181] niks3-server[1231]: 2026/09/24 12:28:39 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign5965worker1 # [ 184.962173] nix-grpc-daemon[1588]: time=2026-09-24T12:28:39.491Z level=INFO msg="Uploading 1 narinfos"5966lb # [ 184.962506] niks3-server[1231]: 2026/09/24 12:28:39 INFO Signed narinfos id=52 count=15967lb # [ 184.968518] niks3-server[1231]: 2026/09/24 12:28:39 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5968lb # [ 184.984258] niks3-server[1231]: 2026/09/24 12:28:39 INFO Received complete push request method=POST path=/api/pushes/52/complete5969lb # [ 184.995796] niks3-server[1231]: 2026/09/24 12:28:39 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5970worker1 # [ 184.997258] nix-grpc-daemon[1588]: time=2026-09-24T12:28:39.527Z level=INFO msg="Upload complete. (311ms)"5971worker1 # [ 185.012570] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46582 duration_s=0 paths=1 nar_bytes_in=4165972worker1 # [ 185.025395] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:465825973worker1 # [ 185.038814] nix-daemon[855]: accepted connection from pid 1506, user nix-grpc-daemon (trusted)5974worker1 # [ 185.051682] nix-daemon[855]: reaped child process 1634, status = succeeded5975client # [ 185.070606] nix[1498]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5976worker1 # [ 185.061068] nix-daemon[855]: accepted connection from pid 1506, user nix-grpc-daemon (trusted)5977worker1: must succeed: systemctl reload nix-grpc-daemon.service5978worker1 # [ 186.245403] systemd[1]: Reloading Nix worker-protocol over gRPC...5979worker1 # [ 186.280267] systemd[1]: Reloaded Nix worker-protocol over gRPC.5980worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)5981worker1: must succeed: pgrep -f 'read -t [9]0 x'5982worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5983client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&25984worker1 # [ 186.566611] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:465825985client # don't know how to build these paths:5986client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5987worker1 # [ 186.572321] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:465825988worker2 # [ 186.572286] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:41Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05989worker2 # [ 186.572655] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:41Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15990worker2 # [ 186.585547] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471965991worker2 # [ 186.596915] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471965992client # copying 1 paths to 'lb:50051'...5993worker2 # [ 186.618882] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471965994client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5995worker2 # [ 186.643322] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)5996worker2 # [ 186.805275] nix-daemon[844]: accepted connection from pid 1798, user nix-grpc-daemon (trusted)5997worker2 # [ 186.829263] nix-daemon[844]: reaped child process 1801, status = succeeded5998lb # [ 186.859589] niks3-server[1231]: 2026/09/24 12:28:41 INFO Received push request method=POST path=/api/pushes5999worker2 # [ 186.843172] nix-grpc-daemon[1723]: time=2026-09-24T12:28:41.390Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6000worker2 # [ 186.844853] nix-grpc-daemon[1723]: time=2026-09-24T12:28:41.390Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6001lb # [ 186.896195] niks3-server[1231]: 2026/09/24 12:28:41 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6002worker2 # [ 186.875860] nix-grpc-daemon[1723]: time=2026-09-24T12:28:41.423Z level=INFO msg="Uploading 1 narinfos"6003lb # [ 186.908505] niks3-server[1231]: 2026/09/24 12:28:41 INFO Signed narinfos id=53 count=16004lb # [ 186.911663] niks3-server[1231]: 2026/09/24 12:28:41 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6005lb # [ 186.922900] niks3-server[1231]: 2026/09/24 12:28:41 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6006lb # [ 186.935231] niks3-server[1231]: 2026/09/24 12:28:41 INFO Received complete push request method=POST path=/api/pushes/53/complete6007worker2 # [ 186.912353] nix-grpc-daemon[1723]: time=2026-09-24T12:28:41.460Z level=INFO msg="Upload complete. (249ms)"6008lb # [ 186.941068] niks3-server[1231]: 2026/09/24 12:28:41 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6009worker2 # [ 186.920303] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=0 paths=1 nar_bytes_in=5286010worker2 # [ 186.936442] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966011worker2 # [ 186.945435] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6012worker2 # [ 186.950844] nix-daemon[844]: reaped child process 1796, status = succeeded6013client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6014client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6015worker2 # [ 186.966407] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6016worker1 # [ 187.136166] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:41Z level=info event=draining6017client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6018client # farm-a> LOG-farm-a-during-drain6019worker2 # [ 188.635933] nix-daemon[844]: reaped child process 1805, status = succeeded6020worker2 # [ 188.757164] nix-daemon[844]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6021worker2 # [ 188.780423] nix-daemon[844]: reaped child process 1824, status = succeeded6022lb # [ 188.814849] niks3-server[1231]: 2026/09/24 12:28:43 INFO Received push request method=POST path=/api/pushes6023worker2 # [ 188.799710] nix-grpc-daemon[1723]: time=2026-09-24T12:28:43.346Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6024worker2 # [ 188.802835] nix-grpc-daemon[1723]: time=2026-09-24T12:28:43.346Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6025lb # [ 188.862727] niks3-server[1231]: 2026/09/24 12:28:43 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6026lb # [ 188.873870] niks3-server[1231]: 2026/09/24 12:28:43 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6027worker2 # [ 188.857540] nix-grpc-daemon[1723]: time=2026-09-24T12:28:43.405Z level=INFO msg="Uploading 1 narinfos"6028lb # [ 188.887318] niks3-server[1231]: 2026/09/24 12:28:43 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6029lb # [ 188.894322] niks3-server[1231]: 2026/09/24 12:28:43 INFO Signed narinfos id=54 count=16030lb # [ 188.899381] niks3-server[1231]: 2026/09/24 12:28:43 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6031lb # [ 188.906414] niks3-server[1231]: 2026/09/24 12:28:43 INFO Received complete push request method=POST path=/api/pushes/54/complete6032lb # [ 188.918686] niks3-server[1231]: 2026/09/24 12:28:43 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6033worker2 # [ 188.896863] nix-grpc-daemon[1723]: time=2026-09-24T12:28:43.445Z level=INFO msg="Upload complete. (318ms)"6034worker2 # [ 188.915348] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06035worker2 # [ 188.939441] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:43Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06036worker2 # [ 188.946157] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:43Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16037worker2 # [ 188.953539] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966038worker2 # [ 188.959775] nix-daemon[844]: reaped child process 1803, status = succeeded6039worker2 # [ 188.962808] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966040worker2 # [ 188.972683] nix-grpc-daemon[1723]: time=2026-09-24T12:28:43.521Z level=INFO msg="All 1 paths already cached"6041client # copying 1 paths to 'lb:50051'...6042worker2 # [ 188.983006] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471966043client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6044worker2 # [ 189.005221] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6045worker2 # [ 189.158430] nix-daemon[844]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6046worker2 # [ 189.178716] nix-daemon[844]: reaped child process 1833, status = succeeded6047lb # [ 189.208955] niks3-server[1231]: 2026/09/24 12:28:43 INFO Received push request method=POST path=/api/pushes6048worker2 # [ 189.190383] nix-grpc-daemon[1723]: time=2026-09-24T12:28:43.738Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6049worker2 # [ 189.191966] nix-grpc-daemon[1723]: time=2026-09-24T12:28:43.738Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6050worker2 # [ 189.225605] nix-grpc-daemon[1723]: time=2026-09-24T12:28:43.773Z level=INFO msg="Uploading 1 narinfos"6051lb # [ 189.241232] niks3-server[1231]: 2026/09/24 12:28:43 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6052lb # [ 189.257496] niks3-server[1231]: 2026/09/24 12:28:43 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6053lb # [ 189.267239] niks3-server[1231]: 2026/09/24 12:28:43 INFO Signed narinfos id=55 count=16054lb # [ 189.276854] niks3-server[1231]: 2026/09/24 12:28:43 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6055lb # [ 189.279906] niks3-server[1231]: 2026/09/24 12:28:43 INFO Received complete push request method=POST path=/api/pushes/55/complete6056lb # [ 189.289259] envoy[906]: conn peer=[2001:db8:1::1]:57568 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=828 ms=34956057lb # [ 189.293200] niks3-server[1231]: 2026/09/24 12:28:43 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6058worker2 # [ 189.265753] nix-grpc-daemon[1723]: time=2026-09-24T12:28:43.814Z level=INFO msg="Upload complete. (244ms)"6059worker2 # [ 189.276566] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=0 paths=1 nar_bytes_in=6406060worker2 # [ 189.288901] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966061worker2 # [ 189.301275] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6062worker2 # [ 189.306930] nix-daemon[844]: reaped child process 1828, status = succeeded6063client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6064client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6065worker2 # [ 189.321966] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6066client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6067client # farm-b> LOG-farm-b-during-drain6068worker2 # [ 190.224293] nix-daemon[844]: reaped child process 1837, status = succeeded6069worker2 # [ 190.297848] nix-daemon[844]: accepted connection from pid 1853, user nix-grpc-daemon (trusted)6070worker2 # [ 190.317742] nix-daemon[844]: reaped child process 1856, status = succeeded6071lb # [ 190.351198] niks3-server[1231]: 2026/09/24 12:28:44 INFO Received push request method=POST path=/api/pushes6072worker2 # [ 190.334713] nix-grpc-daemon[1723]: time=2026-09-24T12:28:44.882Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6073worker2 # [ 190.336499] nix-grpc-daemon[1723]: time=2026-09-24T12:28:44.882Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6074lb # [ 190.389938] niks3-server[1231]: 2026/09/24 12:28:44 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6075worker2 # [ 190.374195] nix-grpc-daemon[1723]: time=2026-09-24T12:28:44.922Z level=INFO msg="Uploading 1 narinfos"6076lb # [ 190.402889] niks3-server[1231]: 2026/09/24 12:28:44 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6077lb # [ 190.414760] niks3-server[1231]: 2026/09/24 12:28:44 INFO Signed narinfos id=56 count=16078lb # [ 190.420374] niks3-server[1231]: 2026/09/24 12:28:44 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6079lb # [ 190.423955] niks3-server[1231]: 2026/09/24 12:28:44 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6080lb # [ 190.437818] niks3-server[1231]: 2026/09/24 12:28:44 INFO Received complete push request method=POST path=/api/pushes/56/complete6081lb # [ 190.444380] niks3-server[1231]: 2026/09/24 12:28:44 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6082worker2 # [ 190.421807] nix-grpc-daemon[1723]: time=2026-09-24T12:28:44.970Z level=INFO msg="Upload complete. (289ms)"6083worker2 # [ 190.439959] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06084worker2 # [ 190.460419] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:44Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06085worker2 # [ 190.470647] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:44Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16086worker2 # [ 190.476991] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966087client # copying 1 paths to 'lb:50051'...6088worker2 # [ 190.482713] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966089worker2 # [ 190.486697] nix-daemon[844]: reaped child process 1835, status = succeeded6090worker2 # [ 190.488774] nix-grpc-daemon[1723]: time=2026-09-24T12:28:45.029Z level=INFO msg="All 2 paths already cached"6091worker2 # [ 190.494496] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471966092client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6093worker2 # [ 190.516603] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6094worker2 # [ 190.657410] nix-daemon[844]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6095worker2 # [ 190.677225] nix-daemon[844]: reaped child process 1865, status = succeeded6096lb # [ 190.711462] niks3-server[1231]: 2026/09/24 12:28:45 INFO Received push request method=POST path=/api/pushes6097worker2 # [ 190.693991] nix-grpc-daemon[1723]: time=2026-09-24T12:28:45.242Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6098worker2 # [ 190.695528] nix-grpc-daemon[1723]: time=2026-09-24T12:28:45.242Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6099lb # [ 190.741925] niks3-server[1231]: 2026/09/24 12:28:45 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6100worker2 # [ 190.721282] nix-grpc-daemon[1723]: time=2026-09-24T12:28:45.269Z level=INFO msg="Uploading 1 narinfos"6101lb # [ 190.756434] niks3-server[1231]: 2026/09/24 12:28:45 INFO Signed narinfos id=57 count=16102lb # [ 190.759764] niks3-server[1231]: 2026/09/24 12:28:45 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6103lb # [ 190.780329] niks3-server[1231]: 2026/09/24 12:28:45 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6104lb # [ 190.787852] niks3-server[1231]: 2026/09/24 12:28:45 INFO Received complete push request method=POST path=/api/pushes/57/complete6105worker2 # [ 190.760729] nix-grpc-daemon[1723]: time=2026-09-24T12:28:45.309Z level=INFO msg="Upload complete. (229ms)"6106lb # [ 190.792720] niks3-server[1231]: 2026/09/24 12:28:45 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6107worker2 # [ 190.773280] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=0 paths=1 nar_bytes_in=7686108worker2 # [ 190.786656] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966109worker2 # [ 190.792991] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6110worker2 # [ 190.798141] nix-daemon[844]: reaped child process 1860, status = succeeded6111client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6112worker2 # [ 190.810533] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6113client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6114client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6115client # farm-top> LOG-farm-top-during-drain6116worker2 # [ 192.258583] nix-daemon[844]: reaped child process 1869, status = succeeded6117worker2 # [ 192.378949] nix-daemon[844]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6118worker2 # [ 192.397473] nix-daemon[844]: reaped child process 1888, status = succeeded6119lb # [ 192.435323] niks3-server[1231]: 2026/09/24 12:28:46 INFO Received push request method=POST path=/api/pushes6120worker2 # [ 192.420315] nix-grpc-daemon[1723]: time=2026-09-24T12:28:46.967Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6121worker2 # [ 192.421855] nix-grpc-daemon[1723]: time=2026-09-24T12:28:46.967Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6122worker2 # [ 192.461744] nix-grpc-daemon[1723]: time=2026-09-24T12:28:47.010Z level=INFO msg="Uploading 1 narinfos"6123lb # [ 192.479366] niks3-server[1231]: 2026/09/24 12:28:47 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6124lb # [ 192.494524] niks3-server[1231]: 2026/09/24 12:28:47 INFO Signed narinfos id=58 count=16125lb # [ 192.505369] niks3-server[1231]: 2026/09/24 12:28:47 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6126lb # [ 192.514667] niks3-server[1231]: 2026/09/24 12:28:47 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6127lb # [ 192.521253] niks3-server[1231]: 2026/09/24 12:28:47 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6128lb # [ 192.530989] niks3-server[1231]: 2026/09/24 12:28:47 INFO Received complete push request method=POST path=/api/pushes/58/complete6129worker2 # [ 192.508509] nix-grpc-daemon[1723]: time=2026-09-24T12:28:47.056Z level=INFO msg="Upload complete. (287ms)"6130lb # [ 192.536842] niks3-server[1231]: 2026/09/24 12:28:47 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6131worker2 # [ 192.525156] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06132worker2 # [ 192.542133] nix-daemon[844]: reaped child process 1867, status = succeeded6133client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.25 seconds)6134client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6135client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6136worker1: must succeed: pkill -f 'read -t [9]0 x'6137worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.11 seconds)6138client: waiting for success: ! systemctl is-active sw6139worker1 # [ 193.181510] nix-daemon[855]: reaped child process 1647, status = succeeded6140worker1 # [ 193.249169] nix-daemon[855]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6141worker1 # [ 193.274090] nix-daemon[855]: reaped child process 1688, status = succeeded6142lb # [ 193.270137] niks3-server[1231]: 2026/09/24 12:28:47 INFO Received push request method=POST path=/api/pushes6143worker1 # [ 193.283447] nix-grpc-daemon[1588]: time=2026-09-24T12:28:47.813Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6144worker1 # [ 193.285152] nix-grpc-daemon[1588]: time=2026-09-24T12:28:47.813Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6145lb # [ 193.314667] niks3-server[1231]: 2026/09/24 12:28:47 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6146worker1 # [ 193.329472] nix-grpc-daemon[1588]: time=2026-09-24T12:28:47.859Z level=INFO msg="Uploading 1 narinfos"6147lb # [ 193.327902] niks3-server[1231]: 2026/09/24 12:28:47 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6148lb # [ 193.339335] niks3-server[1231]: 2026/09/24 12:28:47 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6149lb # [ 193.347018] niks3-server[1231]: 2026/09/24 12:28:47 INFO Signed narinfos id=59 count=16150worker1 # [ 193.361204] nix-grpc-daemon[1588]: time=2026-09-24T12:28:47.891Z level=INFO msg="Upload complete. (280ms)"6151lb # [ 193.358986] niks3-server[1231]: 2026/09/24 12:28:47 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6152lb # [ 193.363068] niks3-server[1231]: 2026/09/24 12:28:47 INFO Received complete push request method=POST path=/api/pushes/59/complete6153lb # [ 193.371117] niks3-server[1231]: 2026/09/24 12:28:47 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6154worker1 # [ 193.377235] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46582 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06155worker1 # [ 193.396405] nix-daemon[855]: reaped child process 1645, status = succeeded6156client # [ 193.429382] systemd[1]: sw.service: Deactivated successfully.6157client # [ 193.434775] systemd[1]: sw.service: Consumed 208ms CPU time over 9.916s wall clock time, 39.2M memory peak, 7.6K incoming IP traffic, 8.5K outgoing IP traffic.6158client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6159client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6160client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6161worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6162worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.06 seconds)6163lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6164lb: must succeed: curl -sf localhost:9901/clusters6165worker1 # [ 194.139772] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:48Z level=info event=signal_exit6166lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6167worker1 # [ 195.149334] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:49Z level=info event=scheduler_restarting6168worker2 # [ 195.123533] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:471766169worker2 # [ 195.128539] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:49Z level=info event=worker_gone addr=192.168.1.3:500516170lb: must succeed: curl -sf localhost:9901/clusters6171worker1 # [ 195.181965] nix-grpc-daemon[1506]: ts=2026-09-24T12:28:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6172worker1 # [ 195.205724] nix-daemon[855]: reaped child process 1559, status = succeeded6173lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6174lb: must succeed: curl -sf localhost:9901/clusters6175lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6176worker1 # [ 197.166702] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6177worker1 # [ 197.173320] systemd[1]: nix-grpc-daemon.service: Consumed 875ms CPU time over 24.214s wall clock time, 36M memory peak, 115.5K incoming IP traffic, 60.7K outgoing IP traffic.6178lb: must succeed: curl -sf localhost:9901/clusters6179worker1 # [ 197.363671] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 2.6180worker1 # [ 197.380501] systemd[1]: Starting Nix worker-protocol over gRPC...6181lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6182worker1 # [ 197.546737] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6183worker1 # [ 197.551605] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6184worker1 # [ 197.553902] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16185worker1 # [ 197.574163] nix-grpc-daemon[1701]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6186worker1 # [ 197.575632] nix-grpc-daemon[1701]: E0000 00:00:1790252932.088356 1701 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6187worker1 # [ 197.601251] nix-grpc-daemon[1701]: E0000 00:00:1790252932.090496 1701 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6188worker1 # [ 197.604055] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:52Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6189worker1 # [ 197.605901] nix-grpc-daemon[1701]: E0000 00:00:1790252932.096973 1743 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6190worker1 # [ 197.608266] systemd[1]: Started Nix worker-protocol over gRPC.6191worker1 # [ 197.612002] nix-grpc-daemon[1701]: E0000 00:00:1790252932.116358 1743 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6192worker1 # [ 197.616322] nix-grpc-daemon[1701]: E0000 00:00:1790252932.116511 1743 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6193worker1 # [ 197.618834] nix-grpc-daemon[1701]: E0000 00:00:1790252932.131067 1743 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6194worker1 # [ 197.621748] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:52Z level=info event=scheduler_connected addr=lb:500516195worker2 # [ 197.589714] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:471766196worker2 # [ 197.590100] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06197lb: must succeed: curl -sf localhost:9901/clusters6198lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6199lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 4.39 seconds)6200lb: must succeed: curl -sf localhost:9901/clusters6201lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6202worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6203worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6204(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.17 seconds)6205subtest: queue gauges show a burst while every build blocks6206lb: must succeed: curl -sf localhost:9901/clusters6207lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6208lb: must succeed: curl -sf localhost:9901/clusters6209lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6210worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6211worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6212client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6213client # [ 198.866006] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6214client # Running as unit: burst.service; invocation ID: 8a04256aaee74fc091699f46afb27c5d6215client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.07 seconds)6216lb: must succeed: curl -sf localhost:9901/clusters6217lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6218lb: must succeed: curl -sf localhost:9901/clusters6219lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6220worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6221client # [ 199.056498] nix[1554]: don't know how to build these paths:6222client # [ 199.057431] nix[1554]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6223client # [ 199.058458] nix[1554]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6224worker2 # [ 199.013512] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471966225client # [ 199.059421] nix[1554]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6226client # [ 199.061311] nix[1554]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6227worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6228worker2 # [ 199.027142] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06229worker2 # [ 199.033090] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.4:50051 clients=16230worker2 # [ 199.041719] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06231worker1 # [ 199.073433] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086232worker2 # [ 199.048160] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.3:50051 clients=16233worker2 # [ 199.054792] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06234worker1 # [ 199.089485] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6235worker2 # [ 199.059737] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06236client # [ 199.108134] nix[1554]: copying 1 paths to 'lb:50051'...6237worker2 # [ 199.064384] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966238client # [ 199.111701] nix[1554]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6239worker2 # [ 199.069066] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966240worker2 # [ 199.074739] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471966241worker2 # [ 199.088856] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6242worker1 # [ 199.114569] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 1/5)6243worker1 # [ 199.173700] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 2/5)6244worker2 # [ 199.210723] nix-daemon[844]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)6245worker2 # [ 199.232748] nix-daemon[844]: reaped child process 1906, status = succeeded6246lb # [ 199.258523] niks3-server[1231]: 2026/09/24 12:28:53 INFO Received push request method=POST path=/api/pushes6247worker1 # [ 199.259633] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 109 ms (attempt 3/5)6248lb # [ 199.266569] envoy[906]: conn peer=[2001:db8:1::1]:35438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9582 tx=4434 ms=59986249lb # [ 199.272789] envoy[906]: conn peer=[2001:db8:1::1]:35430 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=1559 ms=96766250worker2 # [ 199.252663] nix-grpc-daemon[1723]: time=2026-09-24T12:28:53.800Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6251worker2 # [ 199.254193] nix-grpc-daemon[1723]: time=2026-09-24T12:28:53.800Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6252lb # [ 199.279425] envoy[906]: conn peer=[2001:db8:1::3]:55752 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1965 tx=679 ms=219786253lb # [ 199.308502] niks3-server[1231]: 2026/09/24 12:28:53 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6254worker2 # [ 199.286190] nix-grpc-daemon[1723]: time=2026-09-24T12:28:53.834Z level=INFO msg="Uploading 1 narinfos"6255lb # [ 199.317173] niks3-server[1231]: 2026/09/24 12:28:53 INFO Signed narinfos id=60 count=16256lb # [ 199.326504] niks3-server[1231]: 2026/09/24 12:28:53 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6257lb # [ 199.336362] niks3-server[1231]: 2026/09/24 12:28:53 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6258lb # [ 199.352653] niks3-server[1231]: 2026/09/24 12:28:53 INFO Received complete push request method=POST path=/api/pushes/60/complete6259worker2 # [ 199.331145] nix-grpc-daemon[1723]: time=2026-09-24T12:28:53.879Z level=INFO msg="Upload complete. (236ms)"6260lb # [ 199.360975] niks3-server[1231]: 2026/09/24 12:28:53 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6261worker1 # [ 199.370338] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 792 ms (attempt 4/5)6262worker2 # [ 199.344376] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=0 paths=1 nar_bytes_in=4166263worker2 # [ 199.360733] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966264worker2 # [ 199.367817] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6265worker2 # [ 199.373979] nix-daemon[844]: reaped child process 1901, status = succeeded6266worker2 # [ 199.386928] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6267client # [ 199.427596] nix[1554]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6268lb: must succeed: curl -sf localhost:9901/clusters6269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6270lb: must succeed: curl -sf localhost:9901/clusters6271worker1 # [ 200.167422] nix-grpc-daemon[1701]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6272lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6273worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6274worker1 # [ 200.167772] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6275worker1 # [ 200.189410] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086276client # [ 200.217907] nix[1554]: copying 1 paths to 'lb:50051'...6277worker1 # [ 200.204597] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086278client # [ 200.223005] nix[1554]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6279worker1 # [ 200.227026] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6280worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6281worker1 # [ 200.261334] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:54Z level=info event=niks3_push_started pid=17646282worker1 # [ 200.431719] nix-grpc-daemon[1764]: time=2026-09-24T12:28:54.961Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6283worker1 # [ 200.545130] nix-daemon[855]: accepted connection from pid 1777, user nix-grpc-daemon (trusted)6284worker1 # [ 200.564604] nix-daemon[855]: reaped child process 1780, status = succeeded6285lb # [ 200.570559] niks3-server[1231]: 2026/09/24 12:28:55 INFO Received push request method=POST path=/api/pushes6286worker1 # [ 200.584077] nix-grpc-daemon[1764]: time=2026-09-24T12:28:55.114Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6287worker1 # [ 200.585589] nix-grpc-daemon[1764]: time=2026-09-24T12:28:55.114Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6288lb # [ 200.612601] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6289worker1 # [ 200.626043] nix-grpc-daemon[1764]: time=2026-09-24T12:28:55.155Z level=INFO msg="Uploading 1 narinfos"6290lb # [ 200.627611] niks3-server[1231]: 2026/09/24 12:28:55 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6291lb # [ 200.639483] niks3-server[1231]: 2026/09/24 12:28:55 INFO Signed narinfos id=61 count=16292lb # [ 200.649710] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6293worker1 # [ 200.660716] nix-grpc-daemon[1764]: time=2026-09-24T12:28:55.191Z level=INFO msg="Upload complete. (225ms)"6294lb # [ 200.658594] niks3-server[1231]: 2026/09/24 12:28:55 INFO Received complete push request method=POST path=/api/pushes/61/complete6295lb # [ 200.663842] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6296worker2 # [ 200.638875] nix-daemon[844]: reaped child process 1910, status = succeeded6297worker1 # [ 200.672307] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=4166298worker1 # [ 200.683358] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086299worker1 # [ 200.690167] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6300worker1 # [ 200.694762] nix-daemon[855]: reaped child process 1759, status = succeeded6301worker1 # [ 200.705285] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6302client # [ 200.716566] nix[1554]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6303worker2 # [ 200.781272] nix-daemon[844]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)6304worker2 # [ 200.802169] nix-daemon[844]: reaped child process 1937, status = succeeded6305lb # [ 200.834245] niks3-server[1231]: 2026/09/24 12:28:55 INFO Received push request method=POST path=/api/pushes6306worker2 # [ 200.819080] nix-grpc-daemon[1723]: time=2026-09-24T12:28:55.364Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6307worker2 # [ 200.820890] nix-grpc-daemon[1723]: time=2026-09-24T12:28:55.364Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6308worker2 # [ 200.847085] nix-grpc-daemon[1723]: time=2026-09-24T12:28:55.395Z level=INFO msg="Uploading 1 narinfos"6309lb # [ 200.873947] niks3-server[1231]: 2026/09/24 12:28:55 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6310lb # [ 200.887460] niks3-server[1231]: 2026/09/24 12:28:55 INFO Signed narinfos id=62 count=16311lb # [ 200.895763] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6312lb # [ 200.902770] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6313lb # [ 200.911003] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6314lb # [ 200.918663] niks3-server[1231]: 2026/09/24 12:28:55 INFO Received complete push request method=POST path=/api/pushes/62/complete6315worker2 # [ 200.891317] nix-grpc-daemon[1723]: time=2026-09-24T12:28:55.439Z level=INFO msg="Upload complete. (289ms)"6316lb # [ 200.922849] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6317worker2 # [ 200.906276] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06318worker2 # [ 200.928252] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:55Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=16319worker2 # [ 200.934851] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966320client # [ 200.982212] nix[1554]: copying 1 paths to 'lb:50051'...6321worker2 # [ 200.939312] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966322client # [ 200.984661] nix[1554]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6323worker2 # [ 200.942790] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471966324worker2 # [ 200.947401] nix-daemon[844]: reaped child process 1908, status = succeeded6325worker2 # [ 200.960135] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6326worker2 # [ 201.087949] nix-daemon[844]: accepted connection from pid 1941, user nix-grpc-daemon (trusted)6327worker2 # [ 201.109778] nix-daemon[844]: reaped child process 1944, status = succeeded6328lb # [ 201.135345] niks3-server[1231]: 2026/09/24 12:28:55 INFO Received push request method=POST path=/api/pushes6329worker2 # [ 201.116344] nix-grpc-daemon[1723]: time=2026-09-24T12:28:55.664Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6330worker2 # [ 201.118332] nix-grpc-daemon[1723]: time=2026-09-24T12:28:55.666Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6331lb # [ 201.169777] niks3-server[1231]: 2026/09/24 12:28:55 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6332worker2 # [ 201.148592] nix-grpc-daemon[1723]: time=2026-09-24T12:28:55.696Z level=INFO msg="Uploading 1 narinfos"6333lb # [ 201.181780] niks3-server[1231]: 2026/09/24 12:28:55 INFO Signed narinfos id=63 count=16334lb # [ 201.190728] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6335lb # [ 201.208449] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6336worker2 # [ 201.190346] nix-grpc-daemon[1723]: time=2026-09-24T12:28:55.738Z level=INFO msg="Upload complete. (224ms)"6337lb # [ 201.218186] niks3-server[1231]: 2026/09/24 12:28:55 INFO Received complete push request method=POST path=/api/pushes/63/complete6338lb # [ 201.223850] niks3-server[1231]: 2026/09/24 12:28:55 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6339worker2 # [ 201.200285] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=0 paths=1 nar_bytes_in=4166340worker2 # [ 201.216409] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966341worker2 # [ 201.223177] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6342worker2 # [ 201.229488] nix-daemon[844]: reaped child process 1939, status = succeeded6343client # [ 201.282672] nix[1554]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6344worker2 # [ 201.243101] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6345worker1 # [ 201.792149] nix-daemon[855]: reaped child process 1784, status = succeeded6346worker1 # [ 201.881329] nix-daemon[855]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)6347worker1 # [ 201.901859] nix-daemon[855]: reaped child process 1808, status = succeeded6348lb # [ 201.905440] niks3-server[1231]: 2026/09/24 12:28:56 INFO Received push request method=POST path=/api/pushes6349worker1 # [ 201.919245] nix-grpc-daemon[1764]: time=2026-09-24T12:28:56.447Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6350worker1 # [ 201.921464] nix-grpc-daemon[1764]: time=2026-09-24T12:28:56.447Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6351lb # [ 201.938901] niks3-server[1231]: 2026/09/24 12:28:56 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6352lb # [ 201.957710] niks3-server[1231]: 2026/09/24 12:28:56 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6353worker1 # [ 201.970120] nix-grpc-daemon[1764]: time=2026-09-24T12:28:56.500Z level=INFO msg="Uploading 1 narinfos"6354lb # [ 201.968675] niks3-server[1231]: 2026/09/24 12:28:56 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6355lb # [ 201.976746] niks3-server[1231]: 2026/09/24 12:28:56 INFO Signed narinfos id=64 count=16356lb # [ 201.985215] niks3-server[1231]: 2026/09/24 12:28:56 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6357lb # [ 201.998024] niks3-server[1231]: 2026/09/24 12:28:56 INFO Received complete push request method=POST path=/api/pushes/64/complete6358lb # [ 202.006271] niks3-server[1231]: 2026/09/24 12:28:56 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6359worker1 # [ 202.008555] nix-grpc-daemon[1764]: time=2026-09-24T12:28:56.539Z level=INFO msg="Upload complete. (291ms)"6360worker2 # [ 201.989565] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:56Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=16361worker1 # [ 202.022624] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06362worker1 # [ 202.047418] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086363worker1 # [ 202.051725] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086364worker1 # [ 202.056411] nix-daemon[855]: reaped child process 1782, status = succeeded6365client # [ 202.072752] nix[1554]: copying 1 paths to 'lb:50051'...6366worker1 # [ 202.065657] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086367client # [ 202.080116] nix[1554]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6368worker1 # [ 202.081288] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6369worker1 # [ 202.230563] nix-daemon[855]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6370worker1 # [ 202.251951] nix-daemon[855]: reaped child process 1815, status = succeeded6371lb # [ 202.246756] niks3-server[1231]: 2026/09/24 12:28:56 INFO Received push request method=POST path=/api/pushes6372worker1 # [ 202.258141] nix-grpc-daemon[1764]: time=2026-09-24T12:28:56.788Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6373worker1 # [ 202.259693] nix-grpc-daemon[1764]: time=2026-09-24T12:28:56.788Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6374lb # [ 202.279627] niks3-server[1231]: 2026/09/24 12:28:56 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6375worker1 # [ 202.291760] nix-grpc-daemon[1764]: time=2026-09-24T12:28:56.822Z level=INFO msg="Uploading 1 narinfos"6376lb # [ 202.294838] niks3-server[1231]: 2026/09/24 12:28:56 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6377lb # [ 202.304530] niks3-server[1231]: 2026/09/24 12:28:56 INFO Signed narinfos id=65 count=16378lb # [ 202.314718] niks3-server[1231]: 2026/09/24 12:28:56 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6379worker1 # [ 202.328948] nix-grpc-daemon[1764]: time=2026-09-24T12:28:56.859Z level=INFO msg="Upload complete. (233ms)"6380lb # [ 202.326088] niks3-server[1231]: 2026/09/24 12:28:56 INFO Received complete push request method=POST path=/api/pushes/65/complete6381lb # [ 202.330242] niks3-server[1231]: 2026/09/24 12:28:56 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6382worker1 # [ 202.337825] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=4166383worker1 # [ 202.356334] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086384worker1 # [ 202.362697] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6385worker1 # [ 202.368300] nix-daemon[855]: reaped child process 1810, status = succeeded6386worker1 # [ 202.379430] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6387client # [ 202.391772] nix[1554]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6388worker2 # [ 202.865267] nix-daemon[844]: reaped child process 1948, status = succeeded6389worker2 # [ 202.964366] nix-daemon[844]: accepted connection from pid 1973, user nix-grpc-daemon (trusted)6390worker2 # [ 202.986180] nix-daemon[844]: reaped child process 1976, status = succeeded6391lb # [ 203.012899] niks3-server[1231]: 2026/09/24 12:28:57 INFO Received push request method=POST path=/api/pushes6392worker2 # [ 202.996406] nix-grpc-daemon[1723]: time=2026-09-24T12:28:57.542Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6393worker2 # [ 202.997928] nix-grpc-daemon[1723]: time=2026-09-24T12:28:57.542Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6394lb # [ 203.057374] niks3-server[1231]: 2026/09/24 12:28:57 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6395worker2 # [ 203.040155] nix-grpc-daemon[1723]: time=2026-09-24T12:28:57.588Z level=INFO msg="Uploading 1 narinfos"6396lb # [ 203.072712] niks3-server[1231]: 2026/09/24 12:28:57 INFO Signed narinfos id=66 count=16397lb # [ 203.075327] niks3-server[1231]: 2026/09/24 12:28:57 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6398lb # [ 203.086091] niks3-server[1231]: 2026/09/24 12:28:57 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6399worker2 # [ 203.072362] nix-grpc-daemon[1723]: time=2026-09-24T12:28:57.620Z level=INFO msg="Upload complete. (219ms)"6400lb # [ 203.099608] niks3-server[1231]: 2026/09/24 12:28:57 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6401lb # [ 203.103515] niks3-server[1231]: 2026/09/24 12:28:57 INFO Received complete push request method=POST path=/api/pushes/66/complete6402lb # [ 203.108580] niks3-server[1231]: 2026/09/24 12:28:57 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6403worker2 # [ 203.080792] nix-grpc-daemon[1627]: ts=2026-09-24T12:28:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06404worker2 # [ 203.096339] nix-daemon[844]: reaped child process 1946, status = succeeded6405worker1 # [ 204.006265] nix-daemon[855]: reaped child process 1819, status = succeeded6406worker1 # [ 204.104214] nix-daemon[855]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6407worker1 # [ 204.119886] nix-daemon[855]: reaped child process 1847, status = succeeded6408lb # [ 204.125804] niks3-server[1231]: 2026/09/24 12:28:58 INFO Received push request method=POST path=/api/pushes6409worker1 # [ 204.137920] nix-grpc-daemon[1764]: time=2026-09-24T12:28:58.667Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6410worker1 # [ 204.139356] nix-grpc-daemon[1764]: time=2026-09-24T12:28:58.667Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6411worker1 # [ 204.175632] nix-grpc-daemon[1764]: time=2026-09-24T12:28:58.706Z level=INFO msg="Uploading 1 narinfos"6412lb # [ 204.166603] niks3-server[1231]: 2026/09/24 12:28:58 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6413lb # [ 204.179480] niks3-server[1231]: 2026/09/24 12:28:58 INFO Signed narinfos id=67 count=16414lb # [ 204.187710] niks3-server[1231]: 2026/09/24 12:28:58 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6415lb # [ 204.202790] niks3-server[1231]: 2026/09/24 12:28:58 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6416worker1 # [ 204.212406] nix-grpc-daemon[1764]: time=2026-09-24T12:28:58.743Z level=INFO msg="Upload complete. (221ms)"6417lb # [ 204.211782] niks3-server[1231]: 2026/09/24 12:28:58 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6418lb # [ 204.217254] niks3-server[1231]: 2026/09/24 12:28:58 INFO Received complete push request method=POST path=/api/pushes/67/complete6419lb # [ 204.222893] niks3-server[1231]: 2026/09/24 12:28:58 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6420worker1 # [ 204.224947] nix-grpc-daemon[1701]: ts=2026-09-24T12:28:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06421worker1 # [ 204.238717] nix-daemon[855]: reaped child process 1817, status = succeeded6422client # [ 204.272792] systemd[1]: burst.service: Deactivated successfully.6423client # [ 204.278088] systemd[1]: burst.service: Consumed 221ms CPU time over 5.408s wall clock time, 53.6M memory peak, 13.8K incoming IP traffic, 18.8K outgoing IP traffic.6424client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6425client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6426lb: must succeed: curl -sf localhost:9901/clusters6427lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6428lb: must succeed: curl -sf localhost:9901/clusters6429lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6430worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6431worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6432lb: must succeed: curl -sf localhost:9901/clusters6433lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6434lb: must succeed: curl -sf localhost:9901/clusters6435lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6436worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6437worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6438(finished: subtest: queue gauges show a burst while every build blocks, in 7.14 seconds)6439subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6440client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16441worker1 # [ 205.967744] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:387086442worker2 # [ 205.942713] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966443worker2 # [ 205.985948] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:00Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06444worker2 # [ 205.992323] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:00Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16445worker1 # [ 206.027654] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086446worker2 # [ 205.998809] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:00Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06447worker2 # [ 206.004273] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:00Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06448worker1 # [ 206.044169] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086449worker1 # [ 206.058455] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086450worker1 # [ 206.073848] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6451worker1 # [ 206.243937] nix-daemon[855]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)6452worker1 # [ 206.268720] nix-daemon[855]: reaped child process 1858, status = succeeded6453lb # [ 206.267882] niks3-server[1231]: 2026/09/24 12:29:00 INFO Received push request method=POST path=/api/pushes6454worker1 # [ 206.280506] nix-grpc-daemon[1764]: time=2026-09-24T12:29:00.810Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6455worker1 # [ 206.282014] nix-grpc-daemon[1764]: time=2026-09-24T12:29:00.810Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6456lb # [ 206.303397] niks3-server[1231]: 2026/09/24 12:29:00 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6457worker1 # [ 206.319079] nix-grpc-daemon[1764]: time=2026-09-24T12:29:00.849Z level=INFO msg="Uploading 1 narinfos"6458lb # [ 206.319722] niks3-server[1231]: 2026/09/24 12:29:00 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6459lb # [ 206.330877] niks3-server[1231]: 2026/09/24 12:29:00 INFO Signed narinfos id=68 count=16460lb # [ 206.338951] niks3-server[1231]: 2026/09/24 12:29:00 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6461lb # [ 206.352811] niks3-server[1231]: 2026/09/24 12:29:00 INFO Received complete push request method=POST path=/api/pushes/68/complete6462lb # [ 206.363665] niks3-server[1231]: 2026/09/24 12:29:00 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6463worker1 # [ 206.365241] nix-grpc-daemon[1764]: time=2026-09-24T12:29:00.895Z level=INFO msg="Upload complete. (274ms)"6464worker1 # [ 206.376678] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=5126465worker1 # [ 206.388338] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086466worker1 # [ 206.401678] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6467worker1 # [ 206.407672] nix-daemon[855]: reaped child process 1853, status = succeeded6468worker1 # [ 206.426459] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6469worker1 # [ 207.662270] nix-daemon[855]: reaped child process 1862, status = succeeded6470worker1 # [ 207.762574] nix-daemon[855]: accepted connection from pid 1878, user nix-grpc-daemon (trusted)6471worker1 # [ 207.781381] nix-daemon[855]: reaped child process 1881, status = succeeded6472lb # [ 207.790611] niks3-server[1231]: 2026/09/24 12:29:02 INFO Received push request method=POST path=/api/pushes6473worker1 # [ 207.800541] nix-grpc-daemon[1764]: time=2026-09-24T12:29:02.330Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6474worker1 # [ 207.803787] nix-grpc-daemon[1764]: time=2026-09-24T12:29:02.332Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6475lb # [ 207.836167] niks3-server[1231]: 2026/09/24 12:29:02 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6476lb # [ 207.849708] niks3-server[1231]: 2026/09/24 12:29:02 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6477worker1 # [ 207.857093] nix-grpc-daemon[1764]: time=2026-09-24T12:29:02.387Z level=INFO msg="Uploading 1 narinfos"6478lb # [ 207.859844] niks3-server[1231]: 2026/09/24 12:29:02 INFO Signed narinfos id=69 count=16479lb # [ 207.864498] niks3-server[1231]: 2026/09/24 12:29:02 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6480lb # [ 207.867438] niks3-server[1231]: 2026/09/24 12:29:02 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6481lb # [ 207.881071] niks3-server[1231]: 2026/09/24 12:29:02 INFO Received complete push request method=POST path=/api/pushes/69/complete6482lb # [ 207.883698] niks3-server[1231]: 2026/09/24 12:29:02 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6483worker1 # [ 207.889631] nix-grpc-daemon[1764]: time=2026-09-24T12:29:02.420Z level=INFO msg="Upload complete. (251ms)"6484worker2 # [ 207.868810] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:02Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16485worker1 # [ 207.905191] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06486worker2 # [ 207.882430] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:02Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06487worker1 # [ 207.928872] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086488worker1 # [ 207.933278] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086489worker1 # [ 207.939958] nix-daemon[855]: reaped child process 1860, status = succeeded6490worker1 # [ 207.947967] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086491worker1 # [ 207.959825] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6492worker1 # [ 208.097117] nix-daemon[855]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6493worker1 # [ 208.119773] nix-daemon[855]: reaped child process 1888, status = succeeded6494lb # [ 208.116191] niks3-server[1231]: 2026/09/24 12:29:02 INFO Received push request method=POST path=/api/pushes6495worker1 # [ 208.128784] nix-grpc-daemon[1764]: time=2026-09-24T12:29:02.659Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6496worker1 # [ 208.130246] nix-grpc-daemon[1764]: time=2026-09-24T12:29:02.659Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6497lb # [ 208.152768] niks3-server[1231]: 2026/09/24 12:29:02 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6498worker1 # [ 208.166222] nix-grpc-daemon[1764]: time=2026-09-24T12:29:02.696Z level=INFO msg="Uploading 1 narinfos"6499lb # [ 208.168417] niks3-server[1231]: 2026/09/24 12:29:02 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6500lb # [ 208.176389] niks3-server[1231]: 2026/09/24 12:29:02 INFO Signed narinfos id=70 count=16501worker1 # [ 208.189305] nix-grpc-daemon[1764]: time=2026-09-24T12:29:02.719Z level=INFO msg="Upload complete. (216ms)"6502lb # [ 208.186077] niks3-server[1231]: 2026/09/24 12:29:02 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6503worker1 # [ 208.195938] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=5126504lb # [ 208.195039] niks3-server[1231]: 2026/09/24 12:29:02 INFO Received complete push request method=POST path=/api/pushes/70/complete6505lb # [ 208.198122] niks3-server[1231]: 2026/09/24 12:29:02 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6506worker1 # [ 208.206334] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086507worker1 # [ 208.210836] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6508worker1 # [ 208.215627] nix-daemon[855]: reaped child process 1883, status = succeeded6509worker1 # [ 208.226538] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6510lb # [ 209.263701] envoy[906]: conn peer=[2001:db8:1::1]:45772 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9693 tx=5178 ms=51996511worker1 # [ 209.661443] nix-daemon[855]: reaped child process 1892, status = succeeded6512worker1 # [ 209.782673] nix-daemon[855]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6513worker1 # [ 209.803234] nix-daemon[855]: reaped child process 1911, status = succeeded6514lb # [ 209.799323] niks3-server[1231]: 2026/09/24 12:29:04 INFO Received push request method=POST path=/api/pushes6515worker1 # [ 209.812480] nix-grpc-daemon[1764]: time=2026-09-24T12:29:04.342Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6516worker1 # [ 209.813915] nix-grpc-daemon[1764]: time=2026-09-24T12:29:04.342Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6517lb # [ 209.840165] niks3-server[1231]: 2026/09/24 12:29:04 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6518lb # [ 209.854053] niks3-server[1231]: 2026/09/24 12:29:04 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6519worker1 # [ 209.861910] nix-grpc-daemon[1764]: time=2026-09-24T12:29:04.392Z level=INFO msg="Uploading 1 narinfos"6520lb # [ 209.864367] niks3-server[1231]: 2026/09/24 12:29:04 INFO Signed narinfos id=71 count=16521lb # [ 209.870086] niks3-server[1231]: 2026/09/24 12:29:04 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6522lb # [ 209.882948] niks3-server[1231]: 2026/09/24 12:29:04 INFO Received complete push request method=POST path=/api/pushes/71/complete6523worker1 # [ 209.890738] nix-grpc-daemon[1764]: time=2026-09-24T12:29:04.421Z level=INFO msg="Upload complete. (255ms)"6524lb # [ 209.889968] niks3-server[1231]: 2026/09/24 12:29:04 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6525lb # [ 209.892909] niks3-server[1231]: 2026/09/24 12:29:04 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6526worker2 # [ 209.872857] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:04Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16527worker1 # [ 209.901042] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06528worker2 # [ 209.885789] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:04Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06529worker1 # [ 209.920413] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086530worker1 # [ 209.925856] nix-daemon[855]: reaped child process 1890, status = succeeded6531worker1 # [ 209.932808] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086532worker1 # [ 209.944673] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086533worker1 # [ 209.953833] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6534worker1 # [ 210.100490] nix-daemon[855]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6535worker1 # [ 210.122993] nix-daemon[855]: reaped child process 1918, status = succeeded6536lb # [ 210.126406] niks3-server[1231]: 2026/09/24 12:29:04 INFO Received push request method=POST path=/api/pushes6537worker1 # [ 210.139249] nix-grpc-daemon[1764]: time=2026-09-24T12:29:04.669Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6538worker1 # [ 210.141030] nix-grpc-daemon[1764]: time=2026-09-24T12:29:04.669Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6539lb # [ 210.161928] niks3-server[1231]: 2026/09/24 12:29:04 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6540worker1 # [ 210.172361] nix-grpc-daemon[1764]: time=2026-09-24T12:29:04.702Z level=INFO msg="Uploading 1 narinfos"6541lb # [ 210.176246] niks3-server[1231]: 2026/09/24 12:29:04 INFO Signed narinfos id=72 count=16542lb # [ 210.186664] niks3-server[1231]: 2026/09/24 12:29:04 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6543lb # [ 210.195260] niks3-server[1231]: 2026/09/24 12:29:04 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6544lb # [ 210.206280] niks3-server[1231]: 2026/09/24 12:29:04 INFO Received complete push request method=POST path=/api/pushes/72/complete6545worker1 # [ 210.216658] nix-grpc-daemon[1764]: time=2026-09-24T12:29:04.747Z level=INFO msg="Upload complete. (251ms)"6546lb # [ 210.213808] niks3-server[1231]: 2026/09/24 12:29:04 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6547worker1 # [ 210.224259] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=5126548worker1 # [ 210.232519] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086549worker1 # [ 210.243806] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6550worker1 # [ 210.246229] nix-daemon[855]: reaped child process 1913, status = succeeded6551worker1 # [ 210.256896] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6552worker1 # [ 211.720914] nix-daemon[855]: reaped child process 1922, status = succeeded6553worker1 # [ 211.824136] nix-daemon[855]: accepted connection from pid 1939, user nix-grpc-daemon (trusted)6554worker1 # [ 211.844893] nix-daemon[855]: reaped child process 1942, status = succeeded6555lb # [ 211.840855] niks3-server[1231]: 2026/09/24 12:29:06 INFO Received push request method=POST path=/api/pushes6556worker1 # [ 211.855745] nix-grpc-daemon[1764]: time=2026-09-24T12:29:06.383Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6557worker1 # [ 211.857659] nix-grpc-daemon[1764]: time=2026-09-24T12:29:06.383Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6558lb # [ 211.884599] niks3-server[1231]: 2026/09/24 12:29:06 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6559worker1 # [ 211.897951] nix-grpc-daemon[1764]: time=2026-09-24T12:29:06.428Z level=INFO msg="Uploading 1 narinfos"6560lb # [ 211.898633] niks3-server[1231]: 2026/09/24 12:29:06 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6561lb # [ 211.907515] niks3-server[1231]: 2026/09/24 12:29:06 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign6562lb # [ 211.917175] niks3-server[1231]: 2026/09/24 12:29:06 INFO Signed narinfos id=73 count=16563worker1 # [ 211.919793] nix-grpc-daemon[1764]: time=2026-09-24T12:29:06.450Z level=INFO msg="Upload complete. (253ms)"6564lb # [ 211.919106] niks3-server[1231]: 2026/09/24 12:29:06 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6565lb # [ 211.925138] niks3-server[1231]: 2026/09/24 12:29:06 INFO Received complete push request method=POST path=/api/pushes/73/complete6566lb # [ 211.929667] niks3-server[1231]: 2026/09/24 12:29:06 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6567worker1 # [ 211.928888] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06568worker2 # [ 211.899749] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:06Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16569worker2 # [ 211.913775] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:06Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06570worker1 # [ 211.946744] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086571worker1 # [ 211.951741] nix-daemon[855]: reaped child process 1920, status = succeeded6572worker1 # [ 211.958618] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086573worker1 # [ 211.968914] nix-grpc-daemon[1764]: time=2026-09-24T12:29:06.499Z level=INFO msg="All 1 paths already cached"6574worker1 # [ 211.975954] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086575worker1 # [ 211.985237] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6576worker1 # [ 212.119975] nix-daemon[855]: accepted connection from pid 1946, user nix-grpc-daemon (trusted)6577worker1 # [ 212.140970] nix-daemon[855]: reaped child process 1949, status = succeeded6578lb # [ 212.140887] niks3-server[1231]: 2026/09/24 12:29:06 INFO Received push request method=POST path=/api/pushes6579worker1 # [ 212.152928] nix-grpc-daemon[1764]: time=2026-09-24T12:29:06.683Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6580worker1 # [ 212.154419] nix-grpc-daemon[1764]: time=2026-09-24T12:29:06.683Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6581lb # [ 212.172905] niks3-server[1231]: 2026/09/24 12:29:06 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6582worker1 # [ 212.185243] nix-grpc-daemon[1764]: time=2026-09-24T12:29:06.715Z level=INFO msg="Uploading 1 narinfos"6583lb # [ 212.188213] niks3-server[1231]: 2026/09/24 12:29:06 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign6584lb # [ 212.198450] niks3-server[1231]: 2026/09/24 12:29:06 INFO Signed narinfos id=74 count=16585lb # [ 212.204150] niks3-server[1231]: 2026/09/24 12:29:06 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6586lb # [ 212.211868] niks3-server[1231]: 2026/09/24 12:29:06 INFO Received complete push request method=POST path=/api/pushes/74/complete6587lb # [ 212.219206] niks3-server[1231]: 2026/09/24 12:29:06 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6588worker1 # [ 212.221241] nix-grpc-daemon[1764]: time=2026-09-24T12:29:06.751Z level=INFO msg="Upload complete. (225ms)"6589worker1 # [ 212.230858] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=6246590worker1 # [ 212.247433] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086591worker1 # [ 212.253229] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6592worker1 # [ 212.258205] nix-daemon[855]: reaped child process 1944, status = succeeded6593worker1 # [ 212.270682] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6594worker1 # [ 213.923672] nix-daemon[855]: reaped child process 1953, status = succeeded6595worker1 # [ 214.058709] nix-daemon[855]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)6596worker1 # [ 214.080941] nix-daemon[855]: reaped child process 1972, status = succeeded6597lb # [ 214.082883] niks3-server[1231]: 2026/09/24 12:29:08 INFO Received push request method=POST path=/api/pushes6598worker1 # [ 214.096432] nix-grpc-daemon[1764]: time=2026-09-24T12:29:08.625Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6599worker1 # [ 214.097957] nix-grpc-daemon[1764]: time=2026-09-24T12:29:08.625Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6600lb # [ 214.133339] niks3-server[1231]: 2026/09/24 12:29:08 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6601worker1 # [ 214.146535] nix-grpc-daemon[1764]: time=2026-09-24T12:29:08.676Z level=INFO msg="Uploading 1 narinfos"6602lb # [ 214.143634] niks3-server[1231]: 2026/09/24 12:29:08 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign6603lb # [ 214.152913] niks3-server[1231]: 2026/09/24 12:29:08 INFO Signed narinfos id=75 count=16604lb # [ 214.155370] niks3-server[1231]: 2026/09/24 12:29:08 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6605lb # [ 214.167211] niks3-server[1231]: 2026/09/24 12:29:08 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6606worker1 # [ 214.181769] nix-grpc-daemon[1764]: time=2026-09-24T12:29:08.712Z level=INFO msg="Upload complete. (299ms)"6607lb # [ 214.179040] niks3-server[1231]: 2026/09/24 12:29:08 INFO Received complete push request method=POST path=/api/pushes/75/complete6608lb # [ 214.184624] niks3-server[1231]: 2026/09/24 12:29:08 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6609worker1 # [ 214.193205] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06610worker2 # [ 214.167966] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:08Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16611worker2 # [ 214.177043] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:08Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06612worker1 # [ 214.217055] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086613worker1 # [ 214.227221] nix-daemon[855]: reaped child process 1951, status = succeeded6614worker1 # [ 214.230830] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086615worker1 # [ 214.242696] nix-grpc-daemon[1764]: time=2026-09-24T12:29:08.773Z level=INFO msg="All 1 paths already cached"6616worker1 # [ 214.252183] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086617worker1 # [ 214.268221] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6618worker1 # [ 214.423024] nix-daemon[855]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6619worker1 # [ 214.442282] nix-daemon[855]: reaped child process 1979, status = succeeded6620lb # [ 214.445155] niks3-server[1231]: 2026/09/24 12:29:08 INFO Received push request method=POST path=/api/pushes6621worker1 # [ 214.456935] nix-grpc-daemon[1764]: time=2026-09-24T12:29:08.987Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6622worker1 # [ 214.458549] nix-grpc-daemon[1764]: time=2026-09-24T12:29:08.987Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6623lb # [ 214.482080] niks3-server[1231]: 2026/09/24 12:29:09 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6624worker1 # [ 214.493198] nix-grpc-daemon[1764]: time=2026-09-24T12:29:09.023Z level=INFO msg="Uploading 1 narinfos"6625lb # [ 214.494534] niks3-server[1231]: 2026/09/24 12:29:09 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign6626lb # [ 214.502159] niks3-server[1231]: 2026/09/24 12:29:09 INFO Signed narinfos id=76 count=16627lb # [ 214.511328] niks3-server[1231]: 2026/09/24 12:29:09 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6628worker1 # [ 214.519711] nix-grpc-daemon[1764]: time=2026-09-24T12:29:09.050Z level=INFO msg="Upload complete. (237ms)"6629lb # [ 214.518145] niks3-server[1231]: 2026/09/24 12:29:09 INFO Received complete push request method=POST path=/api/pushes/76/complete6630lb # [ 214.521382] niks3-server[1231]: 2026/09/24 12:29:09 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6631worker1 # [ 214.528339] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=6246632worker1 # [ 214.540504] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086633worker1 # [ 214.551388] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6634worker1 # [ 214.560400] nix-daemon[855]: reaped child process 1974, status = succeeded6635worker1 # [ 214.570133] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6636worker1 # [ 216.004289] nix-daemon[855]: reaped child process 1983, status = succeeded6637worker1 # [ 216.116924] nix-daemon[855]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)6638worker1 # [ 216.136079] nix-daemon[855]: reaped child process 2002, status = succeeded6639lb # [ 216.137422] niks3-server[1231]: 2026/09/24 12:29:10 INFO Received push request method=POST path=/api/pushes6640worker1 # [ 216.152393] nix-grpc-daemon[1764]: time=2026-09-24T12:29:10.681Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6641worker1 # [ 216.153834] nix-grpc-daemon[1764]: time=2026-09-24T12:29:10.681Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6642lb # [ 216.190839] niks3-server[1231]: 2026/09/24 12:29:10 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign6643worker1 # [ 216.202075] nix-grpc-daemon[1764]: time=2026-09-24T12:29:10.732Z level=INFO msg="Uploading 1 narinfos"6644lb # [ 216.204856] niks3-server[1231]: 2026/09/24 12:29:10 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6645lb # [ 216.214258] niks3-server[1231]: 2026/09/24 12:29:10 INFO Signed narinfos id=77 count=16646lb # [ 216.224390] niks3-server[1231]: 2026/09/24 12:29:10 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6647worker1 # [ 216.234469] nix-grpc-daemon[1764]: time=2026-09-24T12:29:10.765Z level=INFO msg="Upload complete. (253ms)"6648lb # [ 216.232402] niks3-server[1231]: 2026/09/24 12:29:10 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6649lb # [ 216.237502] niks3-server[1231]: 2026/09/24 12:29:10 INFO Received complete push request method=POST path=/api/pushes/77/complete6650lb # [ 216.242749] niks3-server[1231]: 2026/09/24 12:29:10 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6651worker2 # [ 216.217690] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:10Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16652worker1 # [ 216.249340] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06653worker2 # [ 216.230533] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:10Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06654worker1 # [ 216.268401] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086655worker1 # [ 216.275693] nix-daemon[855]: reaped child process 1981, status = succeeded6656worker1 # [ 216.283437] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086657worker1 # [ 216.296475] nix-grpc-daemon[1764]: time=2026-09-24T12:29:10.827Z level=INFO msg="All 1 paths already cached"6658worker1 # [ 216.305180] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086659worker1 # [ 216.319669] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6660worker1 # [ 216.459229] nix-daemon[855]: accepted connection from pid 2006, user nix-grpc-daemon (trusted)6661worker1 # [ 216.478860] nix-daemon[855]: reaped child process 2009, status = succeeded6662lb # [ 216.483870] niks3-server[1231]: 2026/09/24 12:29:11 INFO Received push request method=POST path=/api/pushes6663worker1 # [ 216.494784] nix-grpc-daemon[1764]: time=2026-09-24T12:29:11.025Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6664worker1 # [ 216.496820] nix-grpc-daemon[1764]: time=2026-09-24T12:29:11.027Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6665lb # [ 216.516417] niks3-server[1231]: 2026/09/24 12:29:11 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6666worker1 # [ 216.528681] nix-grpc-daemon[1764]: time=2026-09-24T12:29:11.059Z level=INFO msg="Uploading 1 narinfos"6667lb # [ 216.531297] niks3-server[1231]: 2026/09/24 12:29:11 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign6668lb # [ 216.540963] niks3-server[1231]: 2026/09/24 12:29:11 INFO Signed narinfos id=78 count=16669lb # [ 216.549966] niks3-server[1231]: 2026/09/24 12:29:11 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6670lb # [ 216.556497] niks3-server[1231]: 2026/09/24 12:29:11 INFO Received complete push request method=POST path=/api/pushes/78/complete6671lb # [ 216.560059] niks3-server[1231]: 2026/09/24 12:29:11 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6672worker1 # [ 216.558485] nix-grpc-daemon[1764]: time=2026-09-24T12:29:11.089Z level=INFO msg="Upload complete. (226ms)"6673worker1 # [ 216.568430] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=6246674worker1 # [ 216.576950] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086675worker1 # [ 216.589112] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6676worker1 # [ 216.594889] nix-daemon[855]: reaped child process 2004, status = succeeded6677worker1 # [ 216.605235] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6678worker1 # [ 218.292318] nix-daemon[855]: reaped child process 2013, status = succeeded6679worker1 # [ 218.397846] nix-daemon[855]: accepted connection from pid 2029, user nix-grpc-daemon (trusted)6680worker1 # [ 218.416113] nix-daemon[855]: reaped child process 2032, status = succeeded6681lb # [ 218.419366] niks3-server[1231]: 2026/09/24 12:29:12 INFO Received push request method=POST path=/api/pushes6682worker1 # [ 218.433228] nix-grpc-daemon[1764]: time=2026-09-24T12:29:12.962Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6683worker1 # [ 218.434808] nix-grpc-daemon[1764]: time=2026-09-24T12:29:12.962Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6684lb # [ 218.460364] niks3-server[1231]: 2026/09/24 12:29:13 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign6685worker1 # [ 218.469973] nix-grpc-daemon[1764]: time=2026-09-24T12:29:13.000Z level=INFO msg="Uploading 1 narinfos"6686lb # [ 218.472979] niks3-server[1231]: 2026/09/24 12:29:13 INFO Signed narinfos id=79 count=16687lb # [ 218.479877] niks3-server[1231]: 2026/09/24 12:29:13 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6688lb # [ 218.488620] niks3-server[1231]: 2026/09/24 12:29:13 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6689lb # [ 218.500566] niks3-server[1231]: 2026/09/24 12:29:13 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6690lb # [ 218.503820] niks3-server[1231]: 2026/09/24 12:29:13 INFO Received complete push request method=POST path=/api/pushes/79/complete6691worker1 # [ 218.511146] nix-grpc-daemon[1764]: time=2026-09-24T12:29:13.041Z level=INFO msg="Upload complete. (290ms)"6692lb # [ 218.512962] niks3-server[1231]: 2026/09/24 12:29:13 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6693worker2 # [ 218.492776] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:13Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16694worker1 # [ 218.525448] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06695worker2 # [ 218.507008] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:13Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06696worker1 # [ 218.546514] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086697worker1 # [ 218.555011] nix-daemon[855]: reaped child process 2011, status = succeeded6698worker1 # [ 218.562783] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086699worker1 # [ 218.576312] nix-grpc-daemon[1764]: time=2026-09-24T12:29:13.107Z level=INFO msg="All 2 paths already cached"6700worker1 # [ 218.585885] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086701worker1 # [ 218.599264] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6702worker1 # [ 218.742718] nix-daemon[855]: accepted connection from pid 2036, user nix-grpc-daemon (trusted)6703worker1 # [ 218.764272] nix-daemon[855]: reaped child process 2039, status = succeeded6704lb # [ 218.767956] niks3-server[1231]: 2026/09/24 12:29:13 INFO Received push request method=POST path=/api/pushes6705worker1 # [ 218.780123] nix-grpc-daemon[1764]: time=2026-09-24T12:29:13.310Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6706worker1 # [ 218.781586] nix-grpc-daemon[1764]: time=2026-09-24T12:29:13.310Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6707worker1 # [ 218.815511] nix-grpc-daemon[1764]: time=2026-09-24T12:29:13.346Z level=INFO msg="Uploading 1 narinfos"6708lb # [ 218.804181] niks3-server[1231]: 2026/09/24 12:29:13 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6709lb # [ 218.818412] niks3-server[1231]: 2026/09/24 12:29:13 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign6710lb # [ 218.828300] niks3-server[1231]: 2026/09/24 12:29:13 INFO Signed narinfos id=80 count=16711lb # [ 218.831396] niks3-server[1231]: 2026/09/24 12:29:13 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6712worker1 # [ 218.845348] nix-grpc-daemon[1764]: time=2026-09-24T12:29:13.376Z level=INFO msg="Upload complete. (233ms)"6713lb # [ 218.839066] niks3-server[1231]: 2026/09/24 12:29:13 INFO Received complete push request method=POST path=/api/pushes/80/complete6714lb # [ 218.847472] niks3-server[1231]: 2026/09/24 12:29:13 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6715worker1 # [ 218.856270] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=7526716worker1 # [ 218.871470] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086717worker1 # [ 218.878282] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6718worker1 # [ 218.883514] nix-daemon[855]: reaped child process 2034, status = succeeded6719worker1 # [ 218.894755] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6720worker1 # [ 219.731645] nix-daemon[855]: reaped child process 2043, status = succeeded6721worker1 # [ 219.845104] nix-daemon[855]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)6722worker1 # [ 219.864464] nix-daemon[855]: reaped child process 2062, status = succeeded6723lb # [ 219.867520] niks3-server[1231]: 2026/09/24 12:29:14 INFO Received push request method=POST path=/api/pushes6724worker1 # [ 219.882908] nix-grpc-daemon[1764]: time=2026-09-24T12:29:14.411Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6725worker1 # [ 219.884539] nix-grpc-daemon[1764]: time=2026-09-24T12:29:14.411Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6726lb # [ 219.905901] niks3-server[1231]: 2026/09/24 12:29:14 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign6727worker1 # [ 219.918006] nix-grpc-daemon[1764]: time=2026-09-24T12:29:14.448Z level=INFO msg="Uploading 1 narinfos"6728lb # [ 219.917534] niks3-server[1231]: 2026/09/24 12:29:14 INFO Signed narinfos id=81 count=16729lb # [ 219.927999] niks3-server[1231]: 2026/09/24 12:29:14 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6730lb # [ 219.931516] niks3-server[1231]: 2026/09/24 12:29:14 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6731lb # [ 219.946754] niks3-server[1231]: 2026/09/24 12:29:14 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6732worker1 # [ 219.953774] nix-grpc-daemon[1764]: time=2026-09-24T12:29:14.484Z level=INFO msg="Upload complete. (246ms)"6733lb # [ 219.951644] niks3-server[1231]: 2026/09/24 12:29:14 INFO Received complete push request method=POST path=/api/pushes/81/complete6734lb # [ 219.955762] niks3-server[1231]: 2026/09/24 12:29:14 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6735worker2 # [ 219.935428] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:14Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16736worker1 # [ 219.966253] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06737worker1 # [ 219.984600] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086738worker1 # [ 219.989969] nix-daemon[855]: reaped child process 2041, status = succeeded6739worker1 # [ 219.997446] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086740worker1 # [ 220.009094] nix-grpc-daemon[1764]: time=2026-09-24T12:29:14.539Z level=INFO msg="All 2 paths already cached"6741worker1 # [ 220.017564] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086742worker1 # [ 220.028994] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6743worker1 # [ 220.170901] nix-daemon[855]: accepted connection from pid 2066, user nix-grpc-daemon (trusted)6744worker1 # [ 220.191462] nix-daemon[855]: reaped child process 2069, status = succeeded6745lb # [ 220.187777] niks3-server[1231]: 2026/09/24 12:29:14 INFO Received push request method=POST path=/api/pushes6746worker1 # [ 220.200421] nix-grpc-daemon[1764]: time=2026-09-24T12:29:14.730Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6747worker1 # [ 220.201890] nix-grpc-daemon[1764]: time=2026-09-24T12:29:14.730Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6748lb # [ 220.220715] niks3-server[1231]: 2026/09/24 12:29:14 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign6749worker1 # [ 220.229345] nix-grpc-daemon[1764]: time=2026-09-24T12:29:14.759Z level=INFO msg="Uploading 1 narinfos"6750lb # [ 220.233200] niks3-server[1231]: 2026/09/24 12:29:14 INFO Signed narinfos id=82 count=16751lb # [ 220.235942] niks3-server[1231]: 2026/09/24 12:29:14 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6752lb # [ 220.248828] niks3-server[1231]: 2026/09/24 12:29:14 INFO Received complete push request method=POST path=/api/pushes/82/complete6753lb # [ 220.256991] niks3-server[1231]: 2026/09/24 12:29:14 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6754worker1 # [ 220.265993] nix-grpc-daemon[1764]: time=2026-09-24T12:29:14.796Z level=INFO msg="Upload complete. (224ms)"6755lb # [ 220.263968] niks3-server[1231]: 2026/09/24 12:29:14 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6756worker1 # [ 220.274615] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=7526757worker1 # [ 220.284653] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086758worker1 # [ 220.294462] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6759worker1 # [ 220.301178] nix-daemon[855]: reaped child process 2064, status = succeeded6760worker1 # [ 220.312207] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6761worker1 # [ 221.164615] nix-daemon[855]: reaped child process 2073, status = succeeded6762worker1 # [ 221.276701] nix-daemon[855]: accepted connection from pid 2089, user nix-grpc-daemon (trusted)6763worker1 # [ 221.297509] nix-daemon[855]: reaped child process 2092, status = succeeded6764lb # [ 221.300767] niks3-server[1231]: 2026/09/24 12:29:15 INFO Received push request method=POST path=/api/pushes6765worker1 # [ 221.315201] nix-grpc-daemon[1764]: time=2026-09-24T12:29:15.844Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6766worker1 # [ 221.317068] nix-grpc-daemon[1764]: time=2026-09-24T12:29:15.844Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6767worker1 # [ 221.356174] nix-grpc-daemon[1764]: time=2026-09-24T12:29:15.886Z level=INFO msg="Uploading 1 narinfos"6768lb # [ 221.344566] niks3-server[1231]: 2026/09/24 12:29:15 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign6769lb # [ 221.358944] niks3-server[1231]: 2026/09/24 12:29:15 INFO Signed narinfos id=83 count=16770lb # [ 221.368353] niks3-server[1231]: 2026/09/24 12:29:15 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6771lb # [ 221.382745] niks3-server[1231]: 2026/09/24 12:29:15 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6772worker1 # [ 221.391572] nix-grpc-daemon[1764]: time=2026-09-24T12:29:15.922Z level=INFO msg="Upload complete. (268ms)"6773lb # [ 221.391821] niks3-server[1231]: 2026/09/24 12:29:15 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6774lb # [ 221.396960] niks3-server[1231]: 2026/09/24 12:29:15 INFO Received complete push request method=POST path=/api/pushes/83/complete6775lb # [ 221.401625] niks3-server[1231]: 2026/09/24 12:29:15 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6776worker2 # [ 221.371958] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:15Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16777worker1 # [ 221.405788] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06778worker1 # [ 221.428155] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086779worker1 # [ 221.434262] nix-daemon[855]: reaped child process 2071, status = succeeded6780worker1 # [ 221.442427] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387086781worker1 # [ 221.456811] nix-grpc-daemon[1764]: time=2026-09-24T12:29:15.987Z level=INFO msg="All 2 paths already cached"6782worker1 # [ 221.467286] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387086783worker1 # [ 221.481920] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6784worker1 # [ 221.629400] nix-daemon[855]: accepted connection from pid 2096, user nix-grpc-daemon (trusted)6785worker1 # [ 221.648938] nix-daemon[855]: reaped child process 2099, status = succeeded6786lb # [ 221.650639] niks3-server[1231]: 2026/09/24 12:29:16 INFO Received push request method=POST path=/api/pushes6787worker1 # [ 221.662968] nix-grpc-daemon[1764]: time=2026-09-24T12:29:16.193Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6788worker1 # [ 221.664518] nix-grpc-daemon[1764]: time=2026-09-24T12:29:16.193Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6789lb # [ 221.685763] niks3-server[1231]: 2026/09/24 12:29:16 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign6790worker1 # [ 221.697485] nix-grpc-daemon[1764]: time=2026-09-24T12:29:16.227Z level=INFO msg="Uploading 1 narinfos"6791lb # [ 221.698102] niks3-server[1231]: 2026/09/24 12:29:16 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6792lb # [ 221.706967] niks3-server[1231]: 2026/09/24 12:29:16 INFO Signed narinfos id=84 count=16793lb # [ 221.716398] niks3-server[1231]: 2026/09/24 12:29:16 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6794worker1 # [ 221.724477] nix-grpc-daemon[1764]: time=2026-09-24T12:29:16.255Z level=INFO msg="Upload complete. (228ms)"6795lb # [ 221.722944] niks3-server[1231]: 2026/09/24 12:29:16 INFO Received complete push request method=POST path=/api/pushes/84/complete6796lb # [ 221.726411] niks3-server[1231]: 2026/09/24 12:29:16 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6797worker1 # [ 221.736711] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=0 paths=1 nar_bytes_in=7526798worker1 # [ 221.744789] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387086799worker1 # [ 221.758647] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6800worker1 # [ 221.760964] nix-daemon[855]: reaped child process 2094, status = succeeded6801worker1 # [ 221.774629] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6802worker1 # [ 222.633445] nix-daemon[855]: reaped child process 2103, status = succeeded6803worker1 # [ 222.759072] nix-daemon[855]: accepted connection from pid 2119, user nix-grpc-daemon (trusted)6804worker1 # [ 222.777113] nix-daemon[855]: reaped child process 2122, status = succeeded6805lb # [ 222.779598] niks3-server[1231]: 2026/09/24 12:29:17 INFO Received push request method=POST path=/api/pushes6806worker1 # [ 222.793168] nix-grpc-daemon[1764]: time=2026-09-24T12:29:17.323Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6807worker1 # [ 222.796780] nix-grpc-daemon[1764]: time=2026-09-24T12:29:17.325Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6808lb # [ 222.824081] niks3-server[1231]: 2026/09/24 12:29:17 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign6809worker1 # [ 222.836001] nix-grpc-daemon[1764]: time=2026-09-24T12:29:17.366Z level=INFO msg="Uploading 1 narinfos"6810lb # [ 222.837960] niks3-server[1231]: 2026/09/24 12:29:17 INFO Signed narinfos id=85 count=16811lb # [ 222.845253] niks3-server[1231]: 2026/09/24 12:29:17 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6812lb # [ 222.857661] niks3-server[1231]: 2026/09/24 12:29:17 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6813lb # [ 222.864591] niks3-server[1231]: 2026/09/24 12:29:17 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6814worker1 # [ 222.865990] nix-grpc-daemon[1764]: time=2026-09-24T12:29:17.396Z level=INFO msg="Upload complete. (281ms)"6815lb # [ 222.867052] niks3-server[1231]: 2026/09/24 12:29:17 INFO Received complete push request method=POST path=/api/pushes/85/complete6816lb # [ 222.870753] niks3-server[1231]: 2026/09/24 12:29:17 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6817worker1 # [ 222.877196] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38708 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06818worker1 # [ 222.898489] nix-daemon[855]: reaped child process 2101, status = succeeded6819client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.20 seconds)6820worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6821worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6822client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6823client # [ 223.085292] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6824client # Running as unit: gpu.service; invocation ID: 3a1fb1abf0424b2195de8b785d59d7a56825client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6826lb: must succeed: curl -sf localhost:9901/clusters6827lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6828lb: must succeed: curl -sf localhost:9901/clusters6829lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6830worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6831worker2 # [ 223.238836] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471966832client # [ 223.288662] nix[1623]: don't know how to build these paths:6833client # [ 223.289625] nix[1623]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6834worker2 # [ 223.252240] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966835worker2 # [ 223.270755] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:17Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06836client # [ 223.313649] nix[1623]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6837worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)6838client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6839lb # [ 229.262617] envoy[906]: conn peer=[2001:db8:1::1]:33660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27114 tx=12322 ms=169286840client # [ 253.321384] nix[1623]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6841client # [ 253.336127] nix[1623]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6842client # [ 253.430231] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6843client # [ 253.434880] systemd[1]: gpu.service: Failed with result 'exit-code'.6844client # [ 253.438230] systemd[1]: gpu.service: Consumed 204ms CPU time over 30.348s wall clock time, 37.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.6845client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 31.18 seconds)6846client: must succeed: journalctl -u gpu -o cat6847client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6848lb: must succeed: curl -sf localhost:9901/clusters6849lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6850lb: must succeed: curl -sf localhost:9901/clusters6851lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6852worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6853worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6854(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.96 seconds)6855subtest: low disk drains a worker and builds go to the other6856worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6857worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.20 seconds)6858worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6859worker1 # [ 256.594319] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:51Z level=info event=unhealthy reason=min_free6860worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6861client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26862client # don't know how to build these paths:6863client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6864worker2 # [ 257.196177] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471966865worker2 # [ 257.203259] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966866worker2 # [ 257.229093] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:51Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06867worker2 # [ 257.235997] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:51Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16868worker2 # [ 257.245399] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966869worker2 # [ 257.257364] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966870client # copying 1 paths to 'lb:50051'...6871client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6872worker2 # [ 257.272654] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471966873worker2 # [ 257.287460] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6874worker2 # [ 257.438426] nix-daemon[844]: accepted connection from pid 2018, user nix-grpc-daemon (trusted)6875worker2 # [ 257.457092] nix-daemon[844]: reaped child process 2021, status = succeeded6876lb # [ 257.491574] niks3-server[1231]: 2026/09/24 12:29:52 INFO Received push request method=POST path=/api/pushes6877worker2 # [ 257.474055] nix-grpc-daemon[1723]: time=2026-09-24T12:29:52.021Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6878worker2 # [ 257.475764] nix-grpc-daemon[1723]: time=2026-09-24T12:29:52.021Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6879lb # [ 257.523663] niks3-server[1231]: 2026/09/24 12:29:52 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign6880worker2 # [ 257.505964] nix-grpc-daemon[1723]: time=2026-09-24T12:29:52.054Z level=INFO msg="Uploading 1 narinfos"6881lb # [ 257.537790] niks3-server[1231]: 2026/09/24 12:29:52 INFO Signed narinfos id=86 count=16882lb # [ 257.544245] niks3-server[1231]: 2026/09/24 12:29:52 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6883lb # [ 257.555087] niks3-server[1231]: 2026/09/24 12:29:52 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6884lb # [ 257.584189] niks3-server[1231]: 2026/09/24 12:29:52 INFO Received complete push request method=POST path=/api/pushes/86/complete6885lb # [ 257.590878] niks3-server[1231]: 2026/09/24 12:29:52 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6886worker2 # [ 257.562671] nix-grpc-daemon[1723]: time=2026-09-24T12:29:52.111Z level=INFO msg="Upload complete. (260ms)"6887worker2 # [ 257.573817] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=0 paths=1 nar_bytes_in=5126888worker2 # [ 257.588633] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966889worker2 # [ 257.600318] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6890worker2 # [ 257.606468] nix-daemon[844]: reaped child process 2016, status = succeeded6891client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6892worker2 # [ 257.617783] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6893client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6894client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6895client # farm-a> LOG-farm-a-drain6896worker2 # [ 258.851627] nix-daemon[844]: reaped child process 2025, status = succeeded6897worker2 # [ 258.956980] nix-daemon[844]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6898worker2 # [ 258.978731] nix-daemon[844]: reaped child process 2044, status = succeeded6899lb # [ 259.006896] niks3-server[1231]: 2026/09/24 12:29:53 INFO Received push request method=POST path=/api/pushes6900worker2 # [ 258.996436] nix-grpc-daemon[1723]: time=2026-09-24T12:29:53.543Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6901worker2 # [ 258.997956] nix-grpc-daemon[1723]: time=2026-09-24T12:29:53.543Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6902lb # [ 259.054527] niks3-server[1231]: 2026/09/24 12:29:53 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign6903worker2 # [ 259.032956] nix-grpc-daemon[1723]: time=2026-09-24T12:29:53.581Z level=INFO msg="Uploading 1 narinfos"6904lb # [ 259.064001] niks3-server[1231]: 2026/09/24 12:29:53 INFO Signed narinfos id=87 count=16905lb # [ 259.071899] niks3-server[1231]: 2026/09/24 12:29:53 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6906lb # [ 259.084802] niks3-server[1231]: 2026/09/24 12:29:53 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6907worker2 # [ 259.062977] nix-grpc-daemon[1723]: time=2026-09-24T12:29:53.611Z level=INFO msg="Upload complete. (230ms)"6908lb # [ 259.092344] niks3-server[1231]: 2026/09/24 12:29:53 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6909lb # [ 259.097471] niks3-server[1231]: 2026/09/24 12:29:53 INFO Received complete push request method=POST path=/api/pushes/87/complete6910lb # [ 259.102214] niks3-server[1231]: 2026/09/24 12:29:53 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6911worker2 # [ 259.073041] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06912worker2 # [ 259.096312] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:53Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06913worker2 # [ 259.099731] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:53Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16914worker2 # [ 259.105145] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966915worker2 # [ 259.109984] nix-daemon[844]: reaped child process 2023, status = succeeded6916worker2 # [ 259.113236] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966917worker2 # [ 259.123621] nix-grpc-daemon[1723]: time=2026-09-24T12:29:53.672Z level=INFO msg="All 1 paths already cached"6918client # copying 1 paths to 'lb:50051'...6919client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6920worker2 # [ 259.132741] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471966921worker2 # [ 259.142934] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6922lb # [ 259.269546] envoy[906]: conn peer=[2001:db8:1::1]:57576 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=610 ms=300676923worker2 # [ 259.277637] nix-daemon[844]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)6924worker2 # [ 259.295220] nix-daemon[844]: reaped child process 2053, status = succeeded6925lb # [ 259.329983] niks3-server[1231]: 2026/09/24 12:29:53 INFO Received push request method=POST path=/api/pushes6926worker2 # [ 259.312681] nix-grpc-daemon[1723]: time=2026-09-24T12:29:53.860Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6927worker2 # [ 259.314206] nix-grpc-daemon[1723]: time=2026-09-24T12:29:53.860Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6928lb # [ 259.364749] niks3-server[1231]: 2026/09/24 12:29:53 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6929worker2 # [ 259.351011] nix-grpc-daemon[1723]: time=2026-09-24T12:29:53.899Z level=INFO msg="Uploading 1 narinfos"6930lb # [ 259.373384] niks3-server[1231]: 2026/09/24 12:29:53 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign6931lb # [ 259.383940] niks3-server[1231]: 2026/09/24 12:29:53 INFO Signed narinfos id=88 count=16932lb # [ 259.390628] niks3-server[1231]: 2026/09/24 12:29:53 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6933lb # [ 259.404302] niks3-server[1231]: 2026/09/24 12:29:53 INFO Received complete push request method=POST path=/api/pushes/88/complete6934worker2 # [ 259.382920] nix-grpc-daemon[1723]: time=2026-09-24T12:29:53.931Z level=INFO msg="Upload complete. (227ms)"6935lb # [ 259.411729] niks3-server[1231]: 2026/09/24 12:29:53 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6936worker2 # [ 259.392863] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=0 paths=1 nar_bytes_in=6326937worker2 # [ 259.409288] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966938worker2 # [ 259.415321] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6939worker2 # [ 259.423915] nix-daemon[844]: reaped child process 2048, status = succeeded6940client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6941worker2 # [ 259.433001] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6942client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6943client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6944client # farm-b> LOG-farm-b-drain6945worker2 # [ 260.503904] nix-daemon[844]: reaped child process 2057, status = succeeded6946worker2 # [ 260.614470] nix-daemon[844]: accepted connection from pid 2073, user nix-grpc-daemon (trusted)6947worker2 # [ 260.630295] nix-daemon[844]: reaped child process 2076, status = succeeded6948lb # [ 260.665930] niks3-server[1231]: 2026/09/24 12:29:55 INFO Received push request method=POST path=/api/pushes6949worker2 # [ 260.651195] nix-grpc-daemon[1723]: time=2026-09-24T12:29:55.196Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6950worker2 # [ 260.653061] nix-grpc-daemon[1723]: time=2026-09-24T12:29:55.196Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6951lb # [ 260.708537] niks3-server[1231]: 2026/09/24 12:29:55 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6952worker2 # [ 260.693459] nix-grpc-daemon[1723]: time=2026-09-24T12:29:55.241Z level=INFO msg="Uploading 1 narinfos"6953lb # [ 260.721756] niks3-server[1231]: 2026/09/24 12:29:55 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign6954lb # [ 260.732971] niks3-server[1231]: 2026/09/24 12:29:55 INFO Signed narinfos id=89 count=16955lb # [ 260.738910] niks3-server[1231]: 2026/09/24 12:29:55 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6956lb # [ 260.753442] niks3-server[1231]: 2026/09/24 12:29:55 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6957lb # [ 260.759563] niks3-server[1231]: 2026/09/24 12:29:55 INFO Received complete push request method=POST path=/api/pushes/89/complete6958worker2 # [ 260.732247] nix-grpc-daemon[1723]: time=2026-09-24T12:29:55.280Z level=INFO msg="Upload complete. (289ms)"6959lb # [ 260.763858] niks3-server[1231]: 2026/09/24 12:29:55 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6960worker2 # [ 260.753111] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06961worker2 # [ 260.769765] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:55Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06962worker2 # [ 260.780877] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:55Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16963worker2 # [ 260.787779] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966964client # copying 1 paths to 'lb:50051'...6965worker2 # [ 260.794708] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471966966client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6967worker2 # [ 260.800493] nix-daemon[844]: reaped child process 2055, status = succeeded6968worker2 # [ 260.802914] nix-grpc-daemon[1723]: time=2026-09-24T12:29:55.341Z level=INFO msg="All 2 paths already cached"6969worker2 # [ 260.818890] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471966970worker2 # [ 260.825195] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6971worker2 # [ 260.944932] nix-daemon[844]: accepted connection from pid 2082, user nix-grpc-daemon (trusted)6972worker2 # [ 260.963050] nix-daemon[844]: reaped child process 2085, status = succeeded6973lb # [ 260.994396] niks3-server[1231]: 2026/09/24 12:29:55 INFO Received push request method=POST path=/api/pushes6974worker2 # [ 260.976109] nix-grpc-daemon[1723]: time=2026-09-24T12:29:55.524Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6975worker2 # [ 260.977759] nix-grpc-daemon[1723]: time=2026-09-24T12:29:55.524Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6976lb # [ 261.026851] niks3-server[1231]: 2026/09/24 12:29:55 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6977worker2 # [ 261.010772] nix-grpc-daemon[1723]: time=2026-09-24T12:29:55.559Z level=INFO msg="Uploading 1 narinfos"6978lb # [ 261.042495] niks3-server[1231]: 2026/09/24 12:29:55 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign6979lb # [ 261.052349] niks3-server[1231]: 2026/09/24 12:29:55 INFO Signed narinfos id=90 count=16980worker2 # [ 261.034649] nix-grpc-daemon[1723]: time=2026-09-24T12:29:55.583Z level=INFO msg="Upload complete. (207ms)"6981lb # [ 261.062330] niks3-server[1231]: 2026/09/24 12:29:55 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6982lb # [ 261.068844] niks3-server[1231]: 2026/09/24 12:29:55 INFO Received complete push request method=POST path=/api/pushes/90/complete6983worker2 # [ 261.044284] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=0 paths=1 nar_bytes_in=7606984lb # [ 261.073475] niks3-server[1231]: 2026/09/24 12:29:55 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6985worker2 # [ 261.057656] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471966986worker2 # [ 261.063677] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6987worker2 # [ 261.068814] nix-daemon[844]: reaped child process 2080, status = succeeded6988client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6989worker2 # [ 261.080220] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6990client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6991client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6992client # farm-top> LOG-farm-top-drain6993worker2 # [ 261.968204] nix-daemon[844]: reaped child process 2089, status = succeeded6994worker2 # [ 262.069980] nix-daemon[844]: accepted connection from pid 2105, user nix-grpc-daemon (trusted)6995worker2 # [ 262.090008] nix-daemon[844]: reaped child process 2108, status = succeeded6996lb # [ 262.126363] niks3-server[1231]: 2026/09/24 12:29:56 INFO Received push request method=POST path=/api/pushes6997worker2 # [ 262.108486] nix-grpc-daemon[1723]: time=2026-09-24T12:29:56.655Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6998worker2 # [ 262.110298] nix-grpc-daemon[1723]: time=2026-09-24T12:29:56.655Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6999worker2 # [ 262.150063] nix-grpc-daemon[1723]: time=2026-09-24T12:29:56.698Z level=INFO msg="Uploading 1 narinfos"7000lb # [ 262.169123] niks3-server[1231]: 2026/09/24 12:29:56 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7001lb # [ 262.182579] niks3-server[1231]: 2026/09/24 12:29:56 INFO Signed narinfos id=91 count=17002lb # [ 262.190803] niks3-server[1231]: 2026/09/24 12:29:56 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7003lb # [ 262.203134] niks3-server[1231]: 2026/09/24 12:29:56 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7004lb # [ 262.210889] niks3-server[1231]: 2026/09/24 12:29:56 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7005worker2 # [ 262.183212] nix-grpc-daemon[1723]: time=2026-09-24T12:29:56.731Z level=INFO msg="Upload complete. (294ms)"7006lb # [ 262.214209] niks3-server[1231]: 2026/09/24 12:29:56 INFO Received complete push request method=POST path=/api/pushes/91/complete7007lb # [ 262.218836] niks3-server[1231]: 2026/09/24 12:29:56 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7008worker2 # [ 262.197500] nix-grpc-daemon[1627]: ts=2026-09-24T12:29:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47196 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07009worker2 # [ 262.218724] nix-daemon[844]: reaped child process 2087, status = succeeded7010client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.27 seconds)7011client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7012client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds)7013worker1: must succeed: rm /nix/.rw-store/fill7014worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7015worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7016worker1 # [ 263.611282] nix-grpc-daemon[1701]: ts=2026-09-24T12:29:58Z level=info event=healthy reason=min_free7017worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7018(finished: subtest: low disk drains a worker and builds go to the other, in 9.09 seconds)7019(finished: run the VM test script, in 264.68 seconds)7020test script finished in 264.94s7021cleanup7022kill QemuMachine (pid 49)7023client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7024client # [2026-09-24T12:29:58Z INFO virtiofsd] Client disconnected, shutting down7025client # [2026-09-24T12:29:58Z INFO virtiofsd] Client disconnected, shutting down7026client # [2026-09-24T12:29:58Z INFO virtiofsd] Client disconnected, shutting down7027kill QemuMachine (pid 51)7028lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7029lb # [2026-09-24T12:29:58Z INFO virtiofsd] Client disconnected, shutting down7030lb # [2026-09-24T12:29:58Z INFO virtiofsd] Client disconnected, shutting down7031lb # [2026-09-24T12:29:58Z INFO virtiofsd] Client disconnected, shutting down7032kill QemuMachine (pid 50)7033worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7034worker1 # [2026-09-24T12:29:59Z INFO virtiofsd] Client disconnected, shutting down7035worker1 # [2026-09-24T12:29:59Z INFO virtiofsd] Client disconnected, shutting down7036worker1 # [2026-09-24T12:29:59Z INFO virtiofsd] Client disconnected, shutting down7037kill QemuMachine (pid 52)7038worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7039worker2 # [2026-09-24T12:29:59Z INFO virtiofsd] Client disconnected, shutting down7040worker2 # [2026-09-24T12:29:59Z INFO virtiofsd] Client disconnected, shutting down7041worker2 # [2026-09-24T12:29:59Z INFO virtiofsd] Client disconnected, shutting down7042(finished: cleanup, in 1.04 seconds)7043additionally exposed symbols:7044 client, lb, worker1, worker2,7045 vlan1,7046 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh