vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #478
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13worker1 # Disk image does not exist, creating the virtualisation disk image...14lb: QEMU running (pid 51)15worker1: QEMU running (pid 50)16worker2 # Disk image does not exist, creating the virtualisation disk image...17lb # Formatting '/build/vm-state-lb/tmp.pqMcCdx4Qt', fmt=raw size=107374182418client: QEMU running (pid 49)19worker1 # Formatting '/build/vm-state-worker1/tmp.NOgEfWg5LU', fmt=raw size=107374182420worker2: QEMU running (pid 52)21client # Disk image does not exist, creating the virtualisation disk image...22worker1 # mke2fs 1.47.4 (6-Mar-2025)23worker2 # Formatting '/build/vm-state-worker2/tmp.RzLmHsxsUr', fmt=raw size=107374182424lb # mke2fs 1.47.4 (6-Mar-2025)25client # Formatting '/build/vm-state-client/tmp.LLleIHii58', fmt=raw size=107374182426(finished: start all VMs, in 0.48 seconds)27lb # Discarding device blocks: 0/262144 done28client # mke2fs 1.47.4 (6-Mar-2025)29client: waiting for unit network-addresses-eth1.service30lb # Creating filesystem with 262144 4k blocks and 65536 inodes31client # Discarding device blocks: 0/262144 done32client: waiting for the VM to finish booting33lb # Filesystem UUID: 786c49ae-8289-420c-bba5-9774400483cb34client # Creating filesystem with 262144 4k blocks and 65536 inodes35worker2 # mke2fs 1.47.4 (6-Mar-2025)36worker1 # Discarding device blocks: 0/262144 done37lb # Superblock backups stored on blocks:38client # Filesystem UUID: 3d2ad251-c0e5-441f-9f9c-e6b65b15088c39worker2 # Discarding device blocks: 0/262144 done40worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes41lb # 32768, 98304, 163840, 22937642client # Superblock backups stored on blocks:43worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes44worker1 # Filesystem UUID: e2269f16-9a6d-40cf-a9ea-5496e61e819e45lb # 46client # 32768, 98304, 163840, 22937647worker2 # Filesystem UUID: fb6f5575-0a40-4c04-a7b8-b0c52e6f357648worker1 # Superblock backups stored on blocks:49lb # Allocating group tables: 0/8 done50client # 51worker2 # Superblock backups stored on blocks:52worker1 # 32768, 98304, 163840, 22937653lb # Writing inode tables: 0/8 done54client # Allocating group tables: 0/8 done55worker2 # 32768, 98304, 163840, 22937656worker1 # 57lb # Creating journal (8192 blocks): done58client # Writing inode tables: 0/8 done59worker2 # 60worker1 # Allocating group tables: 0/8 done61lb # Writing superblocks and filesystem accounting information: 0/8 done62client # Creating journal (8192 blocks): done63worker2 # Allocating group tables: 0/8 done64worker1 # Writing inode tables: 0/8 done65lb # 66client # Writing superblocks and filesystem accounting information: 0/8 done67worker2 # Writing inode tables: 0/8 done68worker1 # Creating journal (8192 blocks): done69lb # Virtualisation disk image created.70client # 71worker2 # Creating journal (8192 blocks): done72client # Virtualisation disk image created.73worker2 # Writing superblocks and filesystem accounting information: 0/8 done74worker1 # Writing superblocks and filesystem accounting information: 0/8 done75worker2 # 76worker1 # 77worker2 # Virtualisation disk image created.78worker1 # Virtualisation disk image created.79lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681lb # [ 0.000000] KASLR enabled82lb # [ 0.000000] random: crng init done83lb # [ 0.000000] Machine model: linux,dummy-virt84lb # [ 0.000000] efi: UEFI not found.85lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88lb # [ 0.000000] Zone ranges:89lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90lb # [ 0.000000] DMA32 empty91lb # [ 0.000000] Normal empty92lb # [ 0.000000] Device empty93lb # [ 0.000000] Movable zone start for each node94lb # [ 0.000000] Early memory node ranges95lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098lb # [ 0.000000] psci: probing for conduit method from DT.99lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.100lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101lb # [ 0.000000] psci: Trusted OS migration not required102lb # [ 0.000000] psci: SMC Calling Convention v1.1103lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105lb # [ 0.000000] Detected PIPT I-cache on CPU0106lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107lb # [ 0.000000] CPU features: detected: GICv3 CPU interface108lb # [ 0.000000] CPU features: detected: Spectre-v4109lb # [ 0.000000] CPU features: detected: Spectre-BHB110lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112lb # [ 0.000000] alternatives: applying boot alternatives113worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]114worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]115client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]116worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026117worker1 # [ 0.000000] KASLR enabled118worker1 # [ 0.000000] random: crng init done119worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026120worker2 # [ 0.000000] KASLR enabled121worker1 # [ 0.000000] Machine model: linux,dummy-virt122lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hf80c6d06lwc040q6hd4338fw0f5x8bc-nixos-system-lb-test/init regInfo=/nix/store/j7xhxpc23qa8jm0xmjyqz7vk57vrzvr2-closure-info/registration console=ttyAMA0,115200n8 console=tty0123worker2 # [ 0.000000] random: crng init done124client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026125worker1 # [ 0.000000] efi: UEFI not found.126client # [ 0.000000] KASLR enabled127worker2 # [ 0.000000] Machine model: linux,dummy-virt128client # [ 0.000000] random: crng init done129worker2 # [ 0.000000] efi: UEFI not found.130worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT131client # [ 0.000000] Machine model: linux,dummy-virt132lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/j7xhxpc23qa8jm0xmjyqz7vk57vrzvr2-closure-info/registration", will be passed to user space.133worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT134client # [ 0.000000] efi: UEFI not found.135worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]136lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes137client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]139worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]140worker1 # [ 0.000000] Zone ranges:141worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]142lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)143client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]144worker2 # [ 0.000000] Zone ranges:145worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]146worker1 # [ 0.000000] DMA32 empty147client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]148lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)149worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]150worker1 # [ 0.000000] Normal empty151client # [ 0.000000] Zone ranges:152worker2 # [ 0.000000] DMA32 empty153worker1 # [ 0.000000] Device empty154worker2 # [ 0.000000] Normal empty155lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB156client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]157worker1 # [ 0.000000] Movable zone start for each node158worker2 # [ 0.000000] Device empty159lb # [ 0.000000] software IO TLB: area num 1.160client # [ 0.000000] DMA32 empty161worker1 # [ 0.000000] Early memory node ranges162client # [ 0.000000] Normal empty163worker2 # [ 0.000000] Movable zone start for each node164lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB165client # [ 0.000000] Device empty166worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]167worker2 # [ 0.000000] Early memory node ranges168client # [ 0.000000] Movable zone start for each node169worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]170lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)171worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]172client # [ 0.000000] Early memory node ranges173lb # [ 0.000000] Fallback order for Node 0: 0174worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000175worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]176client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]177worker1 # [ 0.000000] psci: probing for conduit method from DT.178lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216179worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000180lb # [ 0.000000] Policy zone: DMA181client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]182worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.183worker2 # [ 0.000000] psci: probing for conduit method from DT.184client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000185lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off186worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.187worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs188client # [ 0.000000] psci: probing for conduit method from DT.189worker1 # [ 0.000000] psci: Trusted OS migration not required190worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs191lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1192client # [ 0.000000] psci: PSCIv1.3 detected in firmware.193worker1 # [ 0.000000] psci: SMC Calling Convention v1.1194worker2 # [ 0.000000] psci: Trusted OS migration not required195lb # [ 0.000000] allocated 3145728 bytes of page_ext196client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs197worker2 # [ 0.000000] psci: SMC Calling Convention v1.1198lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages199worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)200client # [ 0.000000] psci: Trusted OS migration not required201lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups202client # [ 0.000000] psci: SMC Calling Convention v1.1203worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)204worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296205lb # [ 0.000000] rcu: Hierarchical RCU implementation.206worker1 # [ 0.000000] Detected PIPT I-cache on CPU0207worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296208lb # [ 0.000000] rcu: RCU event tracing is enabled.209client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)210worker2 # [ 0.000000] Detected PIPT I-cache on CPU0211worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)212lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.213client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296214client # [ 0.000000] Detected PIPT I-cache on CPU0215worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface216worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)217lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.218worker1 # [ 0.000000] CPU features: detected: Spectre-v4219lb # [ 0.000000] Rude variant of Tasks RCU enabled.220worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface221client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)222worker1 # [ 0.000000] CPU features: detected: Spectre-BHB223lb # [ 0.000000] Tracing variant of Tasks RCU enabled.224worker2 # [ 0.000000] CPU features: detected: Spectre-v4225client # [ 0.000000] CPU features: detected: GICv3 CPU interface226worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38227worker2 # [ 0.000000] CPU features: detected: Spectre-BHB228lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.229client # [ 0.000000] CPU features: detected: Spectre-v4230worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23231worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38232client # [ 0.000000] CPU features: detected: Spectre-BHB233lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1234worker1 # [ 0.000000] alternatives: applying boot alternatives235worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23236client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38237worker2 # [ 0.000000] alternatives: applying boot alternatives238lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.239client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23240client # [ 0.000000] alternatives: applying boot alternatives241lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.242lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.243lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0244worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/dh5phwhcmbkhljl79ynwgb5vwa5cwbjb-nixos-system-worker1-test/init regInfo=/nix/store/017mb1sp5dzsmhgicnlfi2m1dqfaydlz-closure-info/registration console=ttyAMA0,115200n8 console=tty0245lb # [ 0.000000] GICv3: 256 SPIs implemented246worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/h62cvxqbccaqvhayh56ps1gx34979y26-nixos-system-worker2-test/init regInfo=/nix/store/iwqp1884c321zy59dm9fmxhmb9bkh32y-closure-info/registration console=ttyAMA0,115200n8 console=tty0247lb # [ 0.000000] GICv3: 0 Extended SPIs implemented248lb # [ 0.000000] Root IRQ handler: gic_handle_irq249worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/017mb1sp5dzsmhgicnlfi2m1dqfaydlz-closure-info/registration", will be passed to user space.250client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test/init regInfo=/nix/store/sbcz10zhdp5zkwimhpbkjc9fxm11nv35-closure-info/registration console=ttyAMA0,115200n8 console=tty0251lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI252lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0253worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/iwqp1884c321zy59dm9fmxhmb9bkh32y-closure-info/registration", will be passed to user space.254worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes255lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000256worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes257client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/sbcz10zhdp5zkwimhpbkjc9fxm11nv35-closure-info/registration", will be passed to user space.258lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]259worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)260client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes261worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)262lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)263worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)264worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)265client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)266worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB267lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)268client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)269worker1 # [ 0.000000] software IO TLB: area num 1.270worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB271lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000272worker2 # [ 0.000000] software IO TLB: area num 1.273worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB274client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB275lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000276worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB277worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)278client # [ 0.000000] software IO TLB: area num 1.279lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.280worker1 # [ 0.000000] Fallback order for Node 0: 0281worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)282client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB283worker2 # [ 0.000000] Fallback order for Node 0: 0284worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216285worker1 # [ 0.000000] Policy zone: DMA286lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns287client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)288worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216289client # [ 0.000000] Fallback order for Node 0: 0290lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).291worker2 # [ 0.000000] Policy zone: DMA292worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off293client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216294worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off295worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1296client # [ 0.000000] Policy zone: DMA297lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns298worker1 # [ 0.000000] allocated 3145728 bytes of page_ext299worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1300client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off301worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages302worker2 # [ 0.000000] allocated 3145728 bytes of page_ext303lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns304client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1305worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups306lb # [ 0.000032] arm-pv: using stolen time PV307worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages308client # [ 0.000000] allocated 3145728 bytes of page_ext309worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.310worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups311worker1 # [ 0.000000] rcu: RCU event tracing is enabled.312client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages313worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.314lb # [ 0.000646] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)315client # [ 0.000000] ftrace: allocated 294 pages with 4 groups316worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.317worker2 # [ 0.000000] rcu: RCU event tracing is enabled.318lb # [ 0.000837] Console: colour dummy device 80x25319client # [ 0.000000] rcu: Hierarchical RCU implementation.320lb # [ 0.000844] printk: legacy console [tty0] enabled321worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.322worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.323client # [ 0.000000] rcu: RCU event tracing is enabled.324worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.325worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.326worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.327lb # [ 0.001059] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)328client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.329worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.330lb # [ 0.001065] pid_max: default: 32768 minimum: 301331client # [ 0.000000] Trampoline variant of Tasks RCU enabled.332worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.333worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.334client # [ 0.000000] Rude variant of Tasks RCU enabled.335lb # [ 0.001141] LSM: initializing lsm=capability,landlock,yama,bpf,ima336worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1337worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.338lb # [ 0.001307] landlock: Up and running.339client # [ 0.000000] Tracing variant of Tasks RCU enabled.340lb # [ 0.001310] Yama: becoming mindful.341worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1342client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.343worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.344lb # [ 0.001954] LSM support for eBPF active345client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1346lb # [ 0.002124] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)347worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.348worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.349client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.350lb # [ 0.002158] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)351worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.352worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.353lb # [ 0.003290] cacheinfo: Unable to detect cache hierarchy for CPU 0354client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.355worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0356worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.357lb # [ 0.004059] rcu: Hierarchical SRCU implementation.358worker1 # [ 0.000000] GICv3: 256 SPIs implemented359worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0360lb # [ 0.004063] rcu: Max phase no-delay instances is 1000.361worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented362client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.363worker2 # [ 0.000000] GICv3: 256 SPIs implemented364lb # [ 0.005223] fsl-mc MSI: its@8080000 domain created365worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq366client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0367worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented368lb # [ 0.005320] EFI services will not be available.369worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI370client # [ 0.000000] GICv3: 256 SPIs implemented371worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq372lb # [ 0.005449] smp: Bringing up secondary CPUs ...373worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0374client # [ 0.000000] GICv3: 0 Extended SPIs implemented375worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI376lb # [ 0.005457] smp: Brought up 1 node, 1 CPU377client # [ 0.000000] Root IRQ handler: gic_handle_irq378worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000379worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0380lb # [ 0.005460] SMP: Total of 1 processors activated.381client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI382lb # [ 0.005466] CPU: All CPU(s) started at EL1383worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]384worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000385client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0386lb # [ 0.005477] CPU features: detected: Branch Target Identification387worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]388worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)389client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000390lb # [ 0.005481] CPU features: detected: ARMv8.4 Translation Table Level391client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]392worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)393worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)394lb # [ 0.005484] CPU features: detected: Instruction cache invalidation not required for I/D coherence395client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)396worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000397worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)398lb # [ 0.005488] CPU features: detected: Data cache clean to the PoU not required for I/D coherence399worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000400worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000401client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)402lb # [ 0.005491] CPU features: detected: Common not Private translations403worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.404worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000405client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000406lb # [ 0.005494] CPU features: detected: CRC32 instructions407worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.408client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000409worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns410lb # [ 0.005497] CPU features: detected: Data cache clean to Point of Deep Persistence411client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.412worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).413worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns414lb # [ 0.005501] CPU features: detected: Data cache clean to Point of Persistence415worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).416lb # [ 0.005504] CPU features: detected: Data independent timing control (DIT)417client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns418worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns419lb # [ 0.005508] CPU features: detected: E0PD420client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).421worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns422worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns423lb # [ 0.005510] CPU features: detected: Enhanced Counter Virtualization424worker1 # [ 0.000028] arm-pv: using stolen time PV425lb # [ 0.005513] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)426client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns427worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns428worker2 # [ 0.000029] arm-pv: using stolen time PV429lb # [ 0.005516] CPU features: detected: Enhanced Virtualization Traps430worker1 # [ 0.000390] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)431client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns432lb # [ 0.005520] CPU features: detected: Fine Grained Traps433worker1 # [ 0.000572] Console: colour dummy device 80x25434client # [ 0.000028] arm-pv: using stolen time PV435worker2 # [ 0.000403] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)436worker1 # [ 0.000580] printk: legacy console [tty0] enabled437worker2 # [ 0.000601] Console: colour dummy device 80x25438lb # [ 0.005524] CPU features: detected: Generic authentication (architected QARMA5 algorithm)439client # [ 0.000399] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)440worker2 # [ 0.000609] printk: legacy console [tty0] enabled441lb # [ 0.005529] CPU features: detected: RCpc load-acquire (LDAPR)442client # [ 0.000556] Console: colour dummy device 80x25443worker1 # [ 0.000795] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)444lb # [ 0.005532] CPU features: detected: LSE atomic instructions445worker1 # [ 0.000802] pid_max: default: 32768 minimum: 301446client # [ 0.000563] printk: legacy console [tty0] enabled447worker2 # [ 0.000828] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)448lb # [ 0.005535] CPU features: detected: Privileged Access Never449worker2 # [ 0.000835] pid_max: default: 32768 minimum: 301450worker1 # [ 0.000872] LSM: initializing lsm=capability,landlock,yama,bpf,ima451lb # [ 0.005537] CPU features: detected: PMUv3452client # [ 0.000759] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)453worker1 # [ 0.001007] landlock: Up and running.454worker2 # [ 0.000903] LSM: initializing lsm=capability,landlock,yama,bpf,ima455worker1 # [ 0.001010] Yama: becoming mindful.456client # [ 0.000766] pid_max: default: 32768 minimum: 301457lb # [ 0.005540] CPU features: detected: RAS Extension Support458worker2 # [ 0.001037] landlock: Up and running.459worker1 # [ 0.001454] LSM support for eBPF active460worker2 # [ 0.001040] Yama: becoming mindful.461client # [ 0.000841] LSM: initializing lsm=capability,landlock,yama,bpf,ima462worker2 # [ 0.001495] LSM support for eBPF active463client # [ 0.001000] landlock: Up and running.464lb # [ 0.005542] CPU features: detected: RASv1p1 Extension Support465worker1 # [ 0.001609] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)466client # [ 0.001002] Yama: becoming mindful.467lb # [ 0.005545] CPU features: detected: Random Number Generator468worker2 # [ 0.001646] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)469worker1 # [ 0.001644] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)470client # [ 0.001447] LSM support for eBPF active471lb # [ 0.005548] CPU features: detected: Speculation barrier (SB)472worker1 # [ 0.002803] cacheinfo: Unable to detect cache hierarchy for CPU 0473client # [ 0.001608] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)474worker2 # [ 0.001683] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)475lb # [ 0.005551] CPU features: detected: Stage-2 Force Write-Back476worker1 # [ 0.003588] rcu: Hierarchical SRCU implementation.477worker2 # [ 0.002908] cacheinfo: Unable to detect cache hierarchy for CPU 0478client # [ 0.001643] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)479worker1 # [ 0.003592] rcu: Max phase no-delay instances is 1000.480lb # [ 0.005554] CPU features: detected: TLB range maintenance instructions481worker2 # [ 0.003690] rcu: Hierarchical SRCU implementation.482worker1 # [ 0.004817] fsl-mc MSI: its@8080000 domain created483client # [ 0.002805] cacheinfo: Unable to detect cache hierarchy for CPU 0484worker2 # [ 0.003694] rcu: Max phase no-delay instances is 1000.485lb # [ 0.005558] CPU features: detected: Speculative Store Bypassing Safe (SSBS)486worker1 # [ 0.004905] EFI services will not be available.487client # [ 0.003565] rcu: Hierarchical SRCU implementation.488worker2 # [ 0.004885] fsl-mc MSI: its@8080000 domain created489lb # [ 0.005599] alternatives: applying system-wide alternatives490worker1 # [ 0.004972] smp: Bringing up secondary CPUs ...491client # [ 0.003569] rcu: Max phase no-delay instances is 1000.492worker2 # [ 0.004974] EFI services will not be available.493worker1 # [ 0.004980] smp: Brought up 1 node, 1 CPU494client # [ 0.004742] fsl-mc MSI: its@8080000 domain created495worker2 # [ 0.005048] smp: Bringing up secondary CPUs ...496lb # [ 0.008554] CPU features: detected: BBM Level 2 without TLB conflict abort497worker1 # [ 0.004983] SMP: Total of 1 processors activated.498client # [ 0.004831] EFI services will not be available.499worker2 # [ 0.005056] smp: Brought up 1 node, 1 CPU500worker1 # [ 0.004988] CPU: All CPU(s) started at EL1501client # [ 0.004894] smp: Bringing up secondary CPUs ...502worker2 # [ 0.005059] SMP: Total of 1 processors activated.503client # [ 0.004902] smp: Brought up 1 node, 1 CPU504worker1 # [ 0.004999] CPU features: detected: Branch Target Identification505worker2 # [ 0.005064] CPU: All CPU(s) started at EL1506lb # [ 0.008769] Memory: 1406564K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)507lb # [ 0.009189] devtmpfs: initialized508client # [ 0.004906] SMP: Total of 1 processors activated.509worker2 # [ 0.005075] CPU features: detected: Branch Target Identification510worker1 # [ 0.005004] CPU features: detected: ARMv8.4 Translation Table Level511client # [ 0.004911] CPU: All CPU(s) started at EL1512lb # [ 0.011021] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)513worker2 # [ 0.005080] CPU features: detected: ARMv8.4 Translation Table Level514client # [ 0.004921] CPU features: detected: Branch Target Identification515worker1 # [ 0.005007] CPU features: detected: Instruction cache invalidation not required for I/D coherence516client # [ 0.004926] CPU features: detected: ARMv8.4 Translation Table Level517lb # [ 0.011051] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).518worker2 # [ 0.005083] CPU features: detected: Instruction cache invalidation not required for I/D coherence519worker1 # [ 0.005011] CPU features: detected: Data cache clean to the PoU not required for I/D coherence520lb # [ 0.011213] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL521client # [ 0.004929] CPU features: detected: Instruction cache invalidation not required for I/D coherence522worker1 # [ 0.005014] CPU features: detected: Common not Private translations523worker2 # [ 0.005087] CPU features: detected: Data cache clean to the PoU not required for I/D coherence524lb # [ 0.011218] 0 pages in range for non-PLT usage525worker1 # [ 0.005017] CPU features: detected: CRC32 instructions526lb # [ 0.011219] 508288 pages in range for PLT usage527worker2 # [ 0.005090] CPU features: detected: Common not Private translations528client # [ 0.004933] CPU features: detected: Data cache clean to the PoU not required for I/D coherence529lb # [ 0.011327] pinctrl core: initialized pinctrl subsystem530worker2 # [ 0.005093] CPU features: detected: CRC32 instructions531worker1 # [ 0.005020] CPU features: detected: Data cache clean to Point of Deep Persistence532lb # [ 0.012087] DMI not present or invalid.533client # [ 0.004936] CPU features: detected: Common not Private translations534worker2 # [ 0.005096] CPU features: detected: Data cache clean to Point of Deep Persistence535worker1 # [ 0.005024] CPU features: detected: Data cache clean to Point of Persistence536client # [ 0.004952] CPU features: detected: CRC32 instructions537lb # [ 0.015448] NET: Registered PF_NETLINK/PF_ROUTE protocol family538worker2 # [ 0.005100] CPU features: detected: Data cache clean to Point of Persistence539worker1 # [ 0.005027] CPU features: detected: Data independent timing control (DIT)540lb # [ 0.018060] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations541client # [ 0.004955] CPU features: detected: Data cache clean to Point of Deep Persistence542worker1 # [ 0.005030] CPU features: detected: E0PD543worker2 # [ 0.005103] CPU features: detected: Data independent timing control (DIT)544client # [ 0.004959] CPU features: detected: Data cache clean to Point of Persistence545lb # [ 0.018251] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations546worker2 # [ 0.005106] CPU features: detected: E0PD547worker1 # [ 0.005033] CPU features: detected: Enhanced Counter Virtualization548client # [ 0.004962] CPU features: detected: Data independent timing control (DIT)549worker2 # [ 0.005108] CPU features: detected: Enhanced Counter Virtualization550lb # [ 0.018444] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations551worker1 # [ 0.005036] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)552client # [ 0.004965] CPU features: detected: E0PD553lb # [ 0.018466] audit: initializing netlink subsys (disabled)554worker2 # [ 0.005111] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)555worker1 # [ 0.005039] CPU features: detected: Enhanced Virtualization Traps556client # [ 0.004968] CPU features: detected: Enhanced Counter Virtualization557lb # [ 0.019056] thermal_sys: Registered thermal governor 'fair_share'558worker1 # [ 0.005042] CPU features: detected: Fine Grained Traps559worker2 # [ 0.005114] CPU features: detected: Enhanced Virtualization Traps560client # [ 0.004971] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)561lb # [ 0.019058] thermal_sys: Registered thermal governor 'bang_bang'562worker2 # [ 0.005117] CPU features: detected: Fine Grained Traps563worker1 # [ 0.005046] CPU features: detected: Generic authentication (architected QARMA5 algorithm)564client # [ 0.004974] CPU features: detected: Enhanced Virtualization Traps565lb # [ 0.019061] thermal_sys: Registered thermal governor 'step_wise'566worker1 # [ 0.005051] CPU features: detected: RCpc load-acquire (LDAPR)567client # [ 0.004977] CPU features: detected: Fine Grained Traps568worker2 # [ 0.005121] CPU features: detected: Generic authentication (architected QARMA5 algorithm)569lb # [ 0.019066] thermal_sys: Registered thermal governor 'user_space'570worker1 # [ 0.005054] CPU features: detected: LSE atomic instructions571worker2 # [ 0.005126] CPU features: detected: RCpc load-acquire (LDAPR)572client # [ 0.004980] CPU features: detected: Generic authentication (architected QARMA5 algorithm)573lb # [ 0.019069] thermal_sys: Registered thermal governor 'power_allocator'574worker1 # [ 0.005057] CPU features: detected: Privileged Access Never575worker2 # [ 0.005129] CPU features: detected: LSE atomic instructions576worker1 # [ 0.005060] CPU features: detected: PMUv3577worker2 # [ 0.005132] CPU features: detected: Privileged Access Never578lb # [ 0.019093] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1579worker1 # [ 0.005063] CPU features: detected: RAS Extension Support580worker2 # [ 0.005135] CPU features: detected: PMUv3581lb # [ 0.019101] cpuidle: using governor ladder582lb # [ 0.019107] cpuidle: using governor menu583worker1 # [ 0.005065] CPU features: detected: RASv1p1 Extension Support584worker2 # [ 0.005138] CPU features: detected: RAS Extension Support585worker1 # [ 0.005068] CPU features: detected: Random Number Generator586worker2 # [ 0.005140] CPU features: detected: RASv1p1 Extension Support587lb # [ 0.019295] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.588worker1 # [ 0.005070] CPU features: detected: Speculation barrier (SB)589worker2 # [ 0.005143] CPU features: detected: Random Number Generator590lb # [ 0.019310] ASID allocator initialised with 65536 entries591lb # [ 0.020446] Serial: AMBA PL011 UART driver592worker2 # [ 0.005146] CPU features: detected: Speculation barrier (SB)593worker1 # [ 0.005073] CPU features: detected: Stage-2 Force Write-Back594worker2 # [ 0.005149] CPU features: detected: Stage-2 Force Write-Back595lb # [ 0.025709] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1596worker2 # [ 0.005152] CPU features: detected: TLB range maintenance instructions597lb # [ 0.025865] printk: console [ttyAMA0] enabled598worker2 # [ 0.005156] CPU features: detected: Speculative Store Bypassing Safe (SSBS)599worker2 # [ 0.005194] alternatives: applying system-wide alternatives600worker2 # [ 0.008204] CPU features: detected: BBM Level 2 without TLB conflict abort601client # [ 0.004985] CPU features: detected: RCpc load-acquire (LDAPR)602client # [ 0.004988] CPU features: detected: LSE atomic instructions603worker2 # [ 0.008441] Memory: 1406536K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)604worker2 # [ 0.008815] devtmpfs: initialized605client # [ 0.004992] CPU features: detected: Privileged Access Never606client # [ 0.004994] CPU features: detected: PMUv3607worker2 # [ 0.010644] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)608client # [ 0.004997] CPU features: detected: RAS Extension Support609worker2 # [ 0.010667] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).610client # [ 0.004999] CPU features: detected: RASv1p1 Extension Support611lb # [ 0.143095] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages612worker2 # [ 0.010860] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL613client # [ 0.005002] CPU features: detected: Random Number Generator614worker1 # [ 0.005076] CPU features: detected: TLB range maintenance instructions615lb # [ 0.143111] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page616worker2 # [ 0.010864] 0 pages in range for non-PLT usage617client # [ 0.005005] CPU features: detected: Speculation barrier (SB)618worker1 # [ 0.005081] CPU features: detected: Speculative Store Bypassing Safe (SSBS)619lb # [ 0.143116] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages620worker2 # [ 0.010865] 508288 pages in range for PLT usage621client # [ 0.005008] CPU features: detected: Stage-2 Force Write-Back622worker1 # [ 0.005117] alternatives: applying system-wide alternatives623lb # [ 0.143120] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page624client # [ 0.005011] CPU features: detected: TLB range maintenance instructions625worker1 # [ 0.008125] CPU features: detected: BBM Level 2 without TLB conflict abort626lb # [ 0.143125] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages627client # [ 0.005015] CPU features: detected: Speculative Store Bypassing Safe (SSBS)628lb # [ 0.143129] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page629client # [ 0.005051] alternatives: applying system-wide alternatives630worker1 # [ 0.008280] Memory: 1406532K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)631lb # [ 0.143133] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages632client # [ 0.007889] CPU features: detected: BBM Level 2 without TLB conflict abort633worker1 # [ 0.008735] devtmpfs: initialized634lb # [ 0.143137] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page635worker1 # [ 0.010492] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)636client # [ 0.008098] Memory: 1406532K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)637worker1 # [ 0.010515] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).638client # [ 0.008458] devtmpfs: initialized639worker1 # [ 0.010694] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL640lb # [ 0.150360] fbcon: Taking over console641client # [ 0.010265] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)642lb # [ 0.150375] ACPI: Interpreter disabled.643worker1 # [ 0.010699] 0 pages in range for non-PLT usage644worker1 # [ 0.010700] 508288 pages in range for PLT usage645client # [ 0.010288] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).646worker1 # [ 0.010808] pinctrl core: initialized pinctrl subsystem647client # [ 0.010457] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL648worker1 # [ 0.011596] DMI not present or invalid.649worker2 # [ 0.010974] pinctrl core: initialized pinctrl subsystem650client # [ 0.010462] 0 pages in range for non-PLT usage651worker1 # [ 0.014817] NET: Registered PF_NETLINK/PF_ROUTE protocol family652worker2 # [ 0.011677] DMI not present or invalid.653client # [ 0.010463] 508288 pages in range for PLT usage654client # [ 0.010572] pinctrl core: initialized pinctrl subsystem655worker2 # [ 0.014893] NET: Registered PF_NETLINK/PF_ROUTE protocol family656worker1 # [ 0.017013] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations657client # [ 0.011325] DMI not present or invalid.658worker2 # [ 0.017086] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations659worker1 # [ 0.017209] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations660client # [ 0.014561] NET: Registered PF_NETLINK/PF_ROUTE protocol family661worker2 # [ 0.017263] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations662worker1 # [ 0.017407] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations663client # [ 0.016706] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations664worker1 # [ 0.017429] audit: initializing netlink subsys (disabled)665worker2 # [ 0.017463] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations666client # [ 0.016891] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations667worker1 # [ 0.018016] thermal_sys: Registered thermal governor 'fair_share'668worker2 # [ 0.017482] audit: initializing netlink subsys (disabled)669client # [ 0.017105] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations670worker1 # [ 0.018018] thermal_sys: Registered thermal governor 'bang_bang'671worker2 # [ 0.018048] thermal_sys: Registered thermal governor 'fair_share'672client # [ 0.017125] audit: initializing netlink subsys (disabled)673worker1 # [ 0.018021] thermal_sys: Registered thermal governor 'step_wise'674worker2 # [ 0.018050] thermal_sys: Registered thermal governor 'bang_bang'675lb # [ 0.159338] iommu: Default domain type: Translated676client # [ 0.017673] thermal_sys: Registered thermal governor 'fair_share'677worker1 # [ 0.018026] thermal_sys: Registered thermal governor 'user_space'678worker2 # [ 0.018053] thermal_sys: Registered thermal governor 'step_wise'679lb # [ 0.159348] iommu: DMA domain TLB invalidation policy: strict mode680client # [ 0.017676] thermal_sys: Registered thermal governor 'bang_bang'681worker1 # [ 0.018029] thermal_sys: Registered thermal governor 'power_allocator'682worker2 # [ 0.018059] thermal_sys: Registered thermal governor 'user_space'683lb # [ 0.159683] SCSI subsystem initialized684client # [ 0.017679] thermal_sys: Registered thermal governor 'step_wise'685worker2 # [ 0.018061] thermal_sys: Registered thermal governor 'power_allocator'686worker1 # [ 0.018052] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1687client # [ 0.017684] thermal_sys: Registered thermal governor 'user_space'688worker1 # [ 0.018060] cpuidle: using governor ladder689lb # [ 0.161567] usbcore: registered new interface driver usbfs690worker2 # [ 0.018086] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1691worker1 # [ 0.018066] cpuidle: using governor menu692client # [ 0.017687] thermal_sys: Registered thermal governor 'power_allocator'693lb # [ 0.161600] usbcore: registered new interface driver hub694worker2 # [ 0.018094] cpuidle: using governor ladder695worker1 # [ 0.018250] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.696worker2 # [ 0.018100] cpuidle: using governor menu697lb # [ 0.161617] usbcore: registered new device driver usb698client # [ 0.017710] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1699worker1 # [ 0.018264] ASID allocator initialised with 65536 entries700client # [ 0.017720] cpuidle: using governor ladder701lb # [ 0.161907] pps_core: LinuxPPS API ver. 1 registered702worker2 # [ 0.018283] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.703worker1 # [ 0.019401] Serial: AMBA PL011 UART driver704client # [ 0.017725] cpuidle: using governor menu705worker2 # [ 0.018317] ASID allocator initialised with 65536 entries706lb # [ 0.161913] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>707client # [ 0.017911] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.708worker2 # [ 0.019491] Serial: AMBA PL011 UART driver709worker1 # [ 0.024550] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1710lb # [ 0.161922] PTP clock support registered711lb # [ 0.161970] EDAC MC: Ver: 3.0.0712client # [ 0.017925] ASID allocator initialised with 65536 entries713worker1 # [ 0.024669] printk: console [ttyAMA0] enabled714worker2 # [ 0.024636] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1715client # [ 0.019102] Serial: AMBA PL011 UART driver716lb # [ 0.166396] scmi_core: SCMI protocol bus registered717worker2 # [ 0.024751] printk: console [ttyAMA0] enabled718client # [ 0.024157] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1719worker1 # [ 0.144064] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages720lb # [ 0.167350] FPGA manager framework721client # [ 0.024272] printk: console [ttyAMA0] enabled722worker1 # [ 0.144083] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page723worker2 # [ 0.144219] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages724lb # [ 0.168237] vgaarb: loaded725worker2 # [ 0.144235] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page726worker1 # [ 0.144088] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages727lb # [ 0.168841] clocksource: Switched to clocksource arch_sys_counter728client # [ 0.144025] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages729worker1 # [ 0.144092] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page730worker2 # [ 0.144241] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages731client # [ 0.144041] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page732worker2 # [ 0.144245] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page733worker1 # [ 0.144096] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages734client # [ 0.144046] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages735worker1 # [ 0.144101] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page736worker2 # [ 0.144250] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages737client # [ 0.144050] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page738worker2 # [ 0.144254] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page739worker1 # [ 0.144105] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages740client # [ 0.144055] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages741worker1 # [ 0.144109] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page742worker2 # [ 0.144258] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages743client # [ 0.144059] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page744worker2 # [ 0.144262] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page745client # [ 0.144064] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages746worker1 # [ 0.151288] fbcon: Taking over console747worker1 # [ 0.151302] ACPI: Interpreter disabled.748client # [ 0.144068] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page749lb # [ 0.174749] VFS: Disk quotas dquot_6.6.0750lb # [ 0.174788] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)751worker2 # [ 0.151519] fbcon: Taking over console752worker2 # [ 0.151531] ACPI: Interpreter disabled.753lb # [ 0.176399] netfs: FS-Cache loaded754client # [ 0.151267] fbcon: Taking over console755worker2 # [ 0.153356] iommu: Default domain type: Translated756lb # [ 0.176530] pnp: PnP ACPI: disabled757client # [ 0.151280] ACPI: Interpreter disabled.758worker2 # [ 0.153366] iommu: DMA domain TLB invalidation policy: strict mode759worker2 # [ 0.155050] SCSI subsystem initialized760lb # [ 0.180268] NET: Registered PF_INET protocol family761lb # [ 0.180532] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)762worker1 # [ 0.159364] iommu: Default domain type: Translated763worker1 # [ 0.159373] iommu: DMA domain TLB invalidation policy: strict mode764client # [ 0.158590] iommu: Default domain type: Translated765worker1 # [ 0.159727] SCSI subsystem initialized766client # [ 0.158598] iommu: DMA domain TLB invalidation policy: strict mode767client # [ 0.160283] SCSI subsystem initialized768worker1 # [ 0.161699] usbcore: registered new interface driver usbfs769worker1 # [ 0.161731] usbcore: registered new interface driver hub770client # [ 0.160948] usbcore: registered new interface driver usbfs771worker1 # [ 0.161755] usbcore: registered new device driver usb772worker2 # [ 0.162487] usbcore: registered new interface driver usbfs773worker1 # [ 0.162019] pps_core: LinuxPPS API ver. 1 registered774client # [ 0.160979] usbcore: registered new interface driver hub775worker2 # [ 0.162519] usbcore: registered new interface driver hub776client # [ 0.160997] usbcore: registered new device driver usb777worker2 # [ 0.162537] usbcore: registered new device driver usb778worker1 # [ 0.162025] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>779client # [ 0.161299] pps_core: LinuxPPS API ver. 1 registered780worker2 # [ 0.162798] pps_core: LinuxPPS API ver. 1 registered781worker1 # [ 0.162035] PTP clock support registered782worker1 # [ 0.162082] EDAC MC: Ver: 3.0.0783client # [ 0.161305] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>784worker2 # [ 0.162804] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>785client # [ 0.161321] PTP clock support registered786worker1 # [ 0.166599] scmi_core: SCMI protocol bus registered787worker2 # [ 0.162814] PTP clock support registered788client # [ 0.161382] EDAC MC: Ver: 3.0.0789worker2 # [ 0.162861] EDAC MC: Ver: 3.0.0790worker1 # [ 0.167565] FPGA manager framework791client # [ 0.165845] scmi_core: SCMI protocol bus registered792worker2 # [ 0.167413] scmi_core: SCMI protocol bus registered793worker1 # [ 0.168499] vgaarb: loaded794client # [ 0.166792] FPGA manager framework795worker2 # [ 0.168370] FPGA manager framework796worker1 # [ 0.169123] clocksource: Switched to clocksource arch_sys_counter797client # [ 0.167700] vgaarb: loaded798worker2 # [ 0.169265] vgaarb: loaded799client # [ 0.168307] clocksource: Switched to clocksource arch_sys_counter800worker1 # [ 0.170973] VFS: Disk quotas dquot_6.6.0801worker2 # [ 0.169898] clocksource: Switched to clocksource arch_sys_counter802worker2 # [ 0.170473] VFS: Disk quotas dquot_6.6.0803worker1 # [ 0.171009] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)804worker2 # [ 0.170498] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)805worker2 # [ 0.172832] netfs: FS-Cache loaded806worker2 # [ 0.172943] pnp: PnP ACPI: disabled807worker1 # [ 0.174716] netfs: FS-Cache loaded808client # [ 0.172831] VFS: Disk quotas dquot_6.6.0809worker1 # [ 0.174839] pnp: PnP ACPI: disabled810client # [ 0.172873] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)811client # [ 0.174545] netfs: FS-Cache loaded812client # [ 0.174664] pnp: PnP ACPI: disabled813worker1 # [ 0.178653] NET: Registered PF_INET protocol family814worker1 # [ 0.178926] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)815client # [ 0.178434] NET: Registered PF_INET protocol family816worker2 # [ 0.178799] NET: Registered PF_INET protocol family817client # [ 0.178708] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)818worker2 # [ 0.179067] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)819lb # [ 0.208413] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)820lb # [ 0.208452] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)821lb # [ 0.208480] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)822lb # [ 0.208549] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)823lb # [ 0.208691] TCP: Hash tables configured (established 16384 bind 16384)824lb # [ 0.208762] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)825lb # [ 0.208794] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)826lb # [ 0.208829] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)827lb # [ 0.216101] NET: Registered PF_UNIX/PF_LOCAL protocol family828lb # [ 0.216128] NET: Registered PF_XDP protocol family829lb # [ 0.216144] PCI: CLS 0 bytes, default 64830lb # [ 0.216376] Trying to unpack rootfs image as initramfs...831lb # [ 0.227245] kvm [1]: HYP mode not available832worker2 # [ 0.207779] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833worker2 # [ 0.207819] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834worker2 # [ 0.207845] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)835worker1 # [ 0.208020] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)836worker2 # [ 0.207916] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)837worker2 # [ 0.208059] TCP: Hash tables configured (established 16384 bind 16384)838worker1 # [ 0.208062] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)839client # [ 0.207008] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)840worker2 # [ 0.208145] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)841worker1 # [ 0.208088] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)842client # [ 0.207048] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)843worker2 # [ 0.208178] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)844worker1 # [ 0.208161] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)845client # [ 0.207076] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)846worker2 # [ 0.208213] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)847worker1 # [ 0.208304] TCP: Hash tables configured (established 16384 bind 16384)848client # [ 0.207153] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)849worker2 # [ 0.208298] NET: Registered PF_UNIX/PF_LOCAL protocol family850worker1 # [ 0.208381] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)851client # [ 0.207295] TCP: Hash tables configured (established 16384 bind 16384)852worker2 # [ 0.208342] NET: Registered PF_XDP protocol family853worker2 # [ 0.208359] PCI: CLS 0 bytes, default 64854worker1 # [ 0.208417] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)855client # [ 0.207369] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)856worker2 # [ 0.208589] Trying to unpack rootfs image as initramfs...857worker1 # [ 0.208452] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)858client # [ 0.207402] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)859worker1 # [ 0.208547] NET: Registered PF_UNIX/PF_LOCAL protocol family860client # [ 0.207438] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)861worker1 # [ 0.208603] NET: Registered PF_XDP protocol family862client # [ 0.207523] NET: Registered PF_UNIX/PF_LOCAL protocol family863worker1 # [ 0.208619] PCI: CLS 0 bytes, default 64864client # [ 0.207575] NET: Registered PF_XDP protocol family865worker1 # [ 0.208842] Trying to unpack rootfs image as initramfs...866client # [ 0.207592] PCI: CLS 0 bytes, default 64867client # [ 0.207805] Trying to unpack rootfs image as initramfs...868worker2 # [ 0.223531] kvm [1]: HYP mode not available869worker1 # [ 0.226725] kvm [1]: HYP mode not available870client # [ 0.225747] kvm [1]: HYP mode not available871lb # [ 0.310763] Initialise system trusted keyrings872lb # [ 0.311501] workingset: timestamp_bits=42 max_order=19 bucket_order=0873lb # [ 0.312757] squashfs: version 4.0 (2009/01/31) Phillip Lougher874lb # [ 0.313589] 9p: Installing v9fs 9p2000 file system support875client # [ 0.309413] Initialise system trusted keyrings876client # [ 0.310138] workingset: timestamp_bits=42 max_order=19 bucket_order=0877client # [ 0.311380] squashfs: version 4.0 (2009/01/31) Phillip Lougher878client # [ 0.312133] 9p: Installing v9fs 9p2000 file system support879worker2 # [ 0.314448] Initialise system trusted keyrings880worker2 # [ 0.315194] workingset: timestamp_bits=42 max_order=19 bucket_order=0881worker2 # [ 0.316454] squashfs: version 4.0 (2009/01/31) Phillip Lougher882worker1 # [ 0.317674] Initialise system trusted keyrings883worker2 # [ 0.317222] 9p: Installing v9fs 9p2000 file system support884worker1 # [ 0.318419] workingset: timestamp_bits=42 max_order=19 bucket_order=0885lb # [ 0.341563] Key type asymmetric registered886lb # [ 0.341589] Asymmetric key parser 'x509' registered887worker1 # [ 0.319719] squashfs: version 4.0 (2009/01/31) Phillip Lougher888lb # [ 0.341656] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)889lb # [ 0.343766] io scheduler mq-deadline registered890worker1 # [ 0.320491] 9p: Installing v9fs 9p2000 file system support891lb # [ 0.343778] io scheduler kyber registered892lb # [ 0.352958] pl061_gpio 9030000.pl061: PL061 GPIO chip registered893worker2 # [ 0.337873] Key type asymmetric registered894lb # [ 0.353548] ledtrig-cpu: registered to indicate activity on CPUs895lb # [ 0.353915] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:896lb # [ 0.353932] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000897lb # [ 0.353943] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000898client # [ 0.340690] Key type asymmetric registered899lb # [ 0.353951] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000900client # [ 0.340714] Asymmetric key parser 'x509' registered901lb # [ 0.353973] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits902client # [ 0.340781] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)903client # [ 0.342893] io scheduler mq-deadline registered904lb # [ 0.353995] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]905client # [ 0.342903] io scheduler kyber registered906lb # [ 0.354078] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00907worker2 # [ 0.345926] Asymmetric key parser 'x509' registered908lb # [ 0.354088] pci_bus 0000:00: root bus resource [bus 00-ff]909lb # [ 0.354094] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]910worker2 # [ 0.346003] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)911lb # [ 0.354099] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]912worker2 # [ 0.347611] io scheduler mq-deadline registered913worker2 # [ 0.347621] io scheduler kyber registered914lb # [ 0.354104] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]915worker1 # [ 0.349185] Key type asymmetric registered916worker1 # [ 0.349209] Asymmetric key parser 'x509' registered917lb # [ 0.354183] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint918worker1 # [ 0.349277] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)919lb # [ 0.354615] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint920worker1 # [ 0.351413] io scheduler mq-deadline registered921lb # [ 0.354798] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]922worker1 # [ 0.351423] io scheduler kyber registered923lb # [ 0.354815] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]924lb # [ 0.354843] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]925lb # [ 0.354859] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]926lb # [ 0.355312] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint927client # [ 0.352451] pl061_gpio 9030000.pl061: PL061 GPIO chip registered928lb # [ 0.355493] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]929lb # [ 0.355508] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]930lb # [ 0.355536] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]931lb # [ 0.355973] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint932client # [ 0.353788] ledtrig-cpu: registered to indicate activity on CPUs933lb # [ 0.356153] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]934client # [ 0.354188] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:935worker2 # [ 0.358012] pl061_gpio 9030000.pl061: PL061 GPIO chip registered936lb # [ 0.356169] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]937client # [ 0.354205] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000938lb # [ 0.356198] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]939lb # [ 0.356638] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint940client # [ 0.354217] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000941worker1 # [ 0.361240] pl061_gpio 9030000.pl061: PL061 GPIO chip registered942lb # [ 0.356817] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]943client # [ 0.354225] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000944worker2 # [ 0.359332] ledtrig-cpu: registered to indicate activity on CPUs945lb # [ 0.356832] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]946client # [ 0.354246] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits947lb # [ 0.356882] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]948worker2 # [ 0.359704] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:949lb # [ 0.357328] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint950worker2 # [ 0.359721] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000951client # [ 0.354268] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]952lb # [ 0.357508] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]953worker1 # [ 0.362563] ledtrig-cpu: registered to indicate activity on CPUs954client # [ 0.354337] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00955worker2 # [ 0.359733] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000956lb # [ 0.357523] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]957client # [ 0.354346] pci_bus 0000:00: root bus resource [bus 00-ff]958worker1 # [ 0.362969] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:959lb # [ 0.357552] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]960worker2 # [ 0.359741] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000961client # [ 0.354352] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]962worker1 # [ 0.362986] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000963lb # [ 0.358000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint964client # [ 0.354357] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]965worker2 # [ 0.359761] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits966lb # [ 0.358181] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]967worker1 # [ 0.362998] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000968client # [ 0.354362] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]969worker2 # [ 0.359787] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]970lb # [ 0.358196] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]971worker1 # [ 0.363006] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000972client # [ 0.354418] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint973worker2 # [ 0.359868] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00974lb # [ 0.358224] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]975worker2 # [ 0.359878] pci_bus 0000:00: root bus resource [bus 00-ff]976worker1 # [ 0.363028] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits977lb # [ 0.358700] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint978client # [ 0.354850] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint979worker2 # [ 0.359884] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]980client # [ 0.355036] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]981lb # [ 0.358880] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]982worker1 # [ 0.363049] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]983worker2 # [ 0.359890] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]984client # [ 0.355052] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]985lb # [ 0.358895] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]986worker1 # [ 0.363122] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00987client # [ 0.355082] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]988worker2 # [ 0.359895] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]989lb # [ 0.358924] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]990worker1 # [ 0.363131] pci_bus 0000:00: root bus resource [bus 00-ff]991client # [ 0.355098] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]992lb # [ 0.358939] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]993worker1 # [ 0.363137] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]994worker2 # [ 0.359951] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint995client # [ 0.355548] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint996worker1 # [ 0.363142] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]997lb # [ 0.359385] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint998worker2 # [ 0.360396] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint999client # [ 0.355732] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1000lb # [ 0.359569] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1001worker2 # [ 0.360582] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1002worker1 # [ 0.363147] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1003client # [ 0.355748] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1004worker2 # [ 0.360598] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1005lb # [ 0.359598] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006worker1 # [ 0.363236] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1007client # [ 0.355777] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1008worker2 # [ 0.360628] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1009lb # [ 0.360035] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1010worker1 # [ 0.363674] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1011client # [ 0.356226] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1012worker2 # [ 0.360644] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1013lb # [ 0.360218] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1014worker1 # [ 0.363860] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1015worker2 # [ 0.361104] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1016lb # [ 0.360247] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1017worker1 # [ 0.363876] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1018client # [ 0.375842] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1019worker2 # [ 0.361287] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1020client # [ 0.375862] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1021worker1 # [ 0.363906] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1022lb # [ 0.360640] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1023worker2 # [ 0.361303] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1024client # [ 0.375891] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025worker1 # [ 0.363921] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1026lb # [ 0.360818] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1027worker2 # [ 0.361332] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1028worker1 # [ 0.364375] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1029worker2 # [ 0.361790] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1030worker1 # [ 0.364557] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1031worker1 # [ 0.364573] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1032worker2 # [ 0.381281] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1033worker1 # [ 0.364602] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1034worker2 # [ 0.381301] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1035worker1 # [ 0.365058] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1036worker2 # [ 0.381330] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1037worker2 # [ 0.381795] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1038worker1 # [ 0.384615] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1039worker1 # [ 0.384634] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1040worker2 # [ 0.385150] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1041worker1 # [ 0.384664] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042worker2 # [ 0.385170] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1043worker1 # [ 0.385121] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1044worker2 # [ 0.385201] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1045worker2 # [ 0.385663] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1046worker1 # [ 0.388491] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1047worker2 # [ 0.385846] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1048worker1 # [ 0.388509] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1049worker2 # [ 0.385861] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1050lb # [ 0.409107] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1051worker1 # [ 0.388541] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1052lb # [ 0.409376] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1053worker2 # [ 0.385890] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054worker1 # [ 0.388996] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1055lb # [ 0.409394] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1056lb # [ 0.409423] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057lb # [ 0.409885] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1058lb # [ 0.410071] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1059client # [ 0.384384] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1060lb # [ 0.410086] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1061client # [ 0.384576] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1062lb # [ 0.410115] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1063client # [ 0.384592] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1064lb # [ 0.410693] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1065client # [ 0.384625] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1066lb # [ 0.410704] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1067client # [ 0.385105] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1068client # [ 0.385291] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1069lb # [ 0.410710] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1070client # [ 0.385306] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1071lb # [ 0.410755] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1072client # [ 0.385336] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1073lb # [ 0.410802] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1074client # [ 0.385791] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1075worker1 # [ 0.397208] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1076lb # [ 0.410849] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1077client # [ 0.385975] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1078worker1 # [ 0.397231] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1079client # [ 0.385991] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1080lb # [ 0.410896] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1081worker1 # [ 0.397261] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1082client # [ 0.386020] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1083lb # [ 0.410942] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1084worker1 # [ 0.397734] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1085worker1 # [ 0.397919] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1086client # [ 0.386469] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1087lb # [ 0.410990] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1088client # [ 0.386653] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1089worker1 # [ 0.397934] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1090lb # [ 0.411038] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1091client # [ 0.386669] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1092worker1 # [ 0.397963] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1093lb # [ 0.411086] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1094client # [ 0.386698] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1095worker1 # [ 0.398414] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1096worker2 # [ 0.398426] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1097client # [ 0.386713] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1098lb # [ 0.411134] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1099worker1 # [ 0.398597] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1100worker2 # [ 0.398619] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1101worker1 # [ 0.398612] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1102client # [ 0.387195] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1103lb # [ 0.411201] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1104worker2 # [ 0.398635] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1105client # [ 0.387383] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1106worker1 # [ 0.398642] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1107lb # [ 0.411248] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1108worker2 # [ 0.398664] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1109client # [ 0.387413] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1110worker1 # [ 0.398658] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1111lb # [ 0.411269] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1112worker2 # [ 0.399127] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1113lb # [ 0.411291] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1114worker2 # [ 0.399311] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1115worker1 # [ 0.399151] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1116client # [ 0.387864] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1117client # [ 0.388052] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1118worker1 # [ 0.399339] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1119worker2 # [ 0.399326] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1120lb # [ 0.411312] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1121client # [ 0.388082] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122worker1 # [ 0.399368] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123worker2 # [ 0.399355] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124lb # [ 0.411336] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1125worker2 # [ 0.399371] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1126client # [ 0.388483] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1127lb # [ 0.411358] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1128worker1 # [ 0.399825] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1129client # [ 0.388667] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1130worker1 # [ 0.400012] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1131worker2 # [ 0.399838] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1132lb # [ 0.411380] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1133worker2 # [ 0.400025] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1134lb # [ 0.411402] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1135client # [ 0.388911] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1136worker1 # [ 0.400043] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137client # [ 0.389164] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1138lb # [ 0.411424] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1139worker2 # [ 0.400054] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1140worker1 # [ 0.400439] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1141client # [ 0.389181] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1142worker1 # [ 0.400619] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1143lb # [ 0.411445] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1144worker2 # [ 0.400509] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1145client # [ 0.389210] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1146lb # [ 0.411468] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1147worker2 # [ 0.400696] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1148worker1 # [ 0.400869] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1149lb # [ 0.411490] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1150client # [ 0.389698] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1151worker1 # [ 0.401122] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1152worker2 # [ 0.400725] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1153client # [ 0.389882] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1154lb # [ 0.411511] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1155worker2 # [ 0.401116] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1156client # [ 0.389897] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1157lb # [ 0.411532] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1158worker2 # [ 0.401297] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1159lb # [ 0.411553] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1160client # [ 0.389926] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1161worker2 # [ 0.401547] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1162lb # [ 0.411574] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1163client # [ 0.390485] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1164worker2 # [ 0.401825] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1165lb # [ 0.411595] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1166worker1 # [ 0.418152] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1167client # [ 0.390496] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1168worker2 # [ 0.401842] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1169lb # [ 0.411617] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1170worker1 # [ 0.418184] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171client # [ 0.390502] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1172worker2 # [ 0.401872] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1173lb # [ 0.411638] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1174worker1 # [ 0.418657] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1175client # [ 0.390546] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1176lb # [ 0.411660] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1177worker2 # [ 0.402386] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1178worker1 # [ 0.418844] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1179lb # [ 0.411681] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1180worker2 # [ 0.402569] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1181client # [ 0.390592] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1182worker1 # [ 0.418859] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1183lb # [ 0.411708] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1184worker2 # [ 0.402584] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1185worker1 # [ 0.418889] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1186client # [ 0.390637] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1187lb # [ 0.411718] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1188worker2 # [ 0.402613] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1189worker1 # [ 0.419455] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1190client # [ 0.390683] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1191lb # [ 0.411723] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1192worker2 # [ 0.403194] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1193worker1 # [ 0.419466] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1194client # [ 0.390728] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1195worker2 # [ 0.403206] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1196worker1 # [ 0.419472] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1197client # [ 0.390774] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1198worker2 # [ 0.403211] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1199worker1 # [ 0.419516] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1200client # [ 0.390820] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1201worker2 # [ 0.403255] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1202worker1 # [ 0.419563] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1203client # [ 0.390866] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1204worker2 # [ 0.403301] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1205worker1 # [ 0.419609] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1206client # [ 0.390913] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1207worker2 # [ 0.403347] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1208worker1 # [ 0.419655] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1209client # [ 0.390980] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1210worker2 # [ 0.403393] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1211worker1 # [ 0.419701] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1212client # [ 0.391026] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1213worker2 # [ 0.403439] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1214client # [ 0.391048] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1215worker1 # [ 0.419747] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1216worker2 # [ 0.403484] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1217client # [ 0.391070] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1218worker1 # [ 0.419795] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1219worker2 # [ 0.403530] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1220client # [ 0.391091] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1221worker1 # [ 0.419842] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1222client # [ 0.391115] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1223worker2 # [ 0.403576] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1224lb # [ 0.456923] pci 0000:00:0a.0: enabling device (0000 -> 0002)1225worker1 # [ 0.419888] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1226client # [ 0.391136] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1227worker2 # [ 0.403622] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1228client # [ 0.391158] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1229worker1 # [ 0.419964] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1230worker2 # [ 0.403693] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1231client # [ 0.391179] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1232worker1 # [ 0.420009] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1233worker2 # [ 0.403738] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1234client # [ 0.391201] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1235worker1 # [ 0.420031] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1236worker2 # [ 0.403760] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1237client # [ 0.391223] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1238worker1 # [ 0.420052] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1239worker2 # [ 0.403781] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1240client # [ 0.391245] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1241worker1 # [ 0.420073] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1242worker2 # [ 0.403802] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1243client # [ 0.391268] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1244worker1 # [ 0.420097] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1245worker2 # [ 0.403825] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1246client # [ 0.391289] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1247worker1 # [ 0.420119] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1248worker2 # [ 0.403846] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1249client # [ 0.391311] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1250worker1 # [ 0.420140] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1251worker2 # [ 0.403868] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1252client # [ 0.391332] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1253worker1 # [ 0.420162] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1254client # [ 0.391354] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1255worker2 # [ 0.403889] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1256worker1 # [ 0.420183] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1257client # [ 0.391375] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1258worker2 # [ 0.403911] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1259worker1 # [ 0.420205] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1260client # [ 0.391396] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1261worker2 # [ 0.403933] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1262worker1 # [ 0.420227] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1263client # [ 0.391417] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1264worker2 # [ 0.403955] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1265client # [ 0.391439] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1266worker1 # [ 0.420249] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1267worker2 # [ 0.403977] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1268client # [ 0.391461] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1269worker1 # [ 0.420270] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1270worker2 # [ 0.403997] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1271client # [ 0.391488] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1272worker1 # [ 0.420291] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1273worker2 # [ 0.404018] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1274client # [ 0.391497] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1275worker1 # [ 0.420312] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1276worker2 # [ 0.404039] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1277client # [ 0.391501] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1278worker1 # [ 0.420333] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1279worker2 # [ 0.404060] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1280worker1 # [ 0.420354] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1281worker2 # [ 0.404081] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1282worker1 # [ 0.420375] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1283worker2 # [ 0.404102] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1284worker1 # [ 0.420396] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1285worker2 # [ 0.404123] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1286worker1 # [ 0.420418] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1287worker2 # [ 0.404145] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1288worker1 # [ 0.420439] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1289worker2 # [ 0.404166] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1290worker1 # [ 0.420464] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1291worker2 # [ 0.404192] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1292worker1 # [ 0.420473] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1293worker2 # [ 0.404201] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1294worker1 # [ 0.420478] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1295worker2 # [ 0.404206] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1296client # [ 0.452698] pci 0000:00:0a.0: enabling device (0000 -> 0002)1297worker2 # [ 0.405364] pci 0000:00:0a.0: enabling device (0000 -> 0002)1298lb # [ 0.477174] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1299lb # [ 0.479325] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1300worker1 # [ 0.461721] pci 0000:00:0a.0: enabling device (0000 -> 0002)1301lb # [ 0.489039] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1302lb # [ 0.491423] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1303client # [ 0.478440] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1304worker2 # [ 0.480985] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1305worker1 # [ 0.482347] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1306lb # [ 0.500988] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1307worker1 # [ 0.484383] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1308client # [ 0.481516] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1309lb # [ 0.503959] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1310client # [ 0.483409] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1311lb # [ 0.505914] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1312lb # [ 0.508204] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1313worker2 # [ 0.487300] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1314worker2 # [ 0.489192] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1315client # [ 0.490316] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1316worker1 # [ 0.495630] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1317lb # [ 0.518513] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1318lb # [ 0.520250] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1319worker2 # [ 0.499256] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1320worker2 # [ 0.501199] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1321worker1 # [ 0.501722] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1322worker1 # [ 0.503692] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1323client # [ 0.501498] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1324client # [ 0.504186] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1325lb # [ 0.532121] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1326worker2 # [ 0.512700] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1327worker1 # [ 0.513668] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1328worker1 # [ 0.515730] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1329worker2 # [ 0.515639] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1330client # [ 0.513568] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1331worker2 # [ 0.517627] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1332lb # [ 0.544677] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1333lb # [ 0.547129] msm_serial: driver initialized1334lb # [ 0.547278] SuperH (H)SCI(F) driver initialized1335lb # [ 0.547333] STM32 USART driver initialized1336worker2 # [ 0.527577] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1337worker2 # [ 0.529561] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1338worker1 # [ 0.525694] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1339client # [ 0.524850] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1340worker1 # [ 0.528677] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1341client # [ 0.526982] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1342worker1 # [ 0.530465] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1343client # [ 0.528961] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1344worker1 # [ 0.532801] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1345client # [ 0.531036] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1346worker2 # [ 0.541136] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1347client # [ 0.549503] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1348worker2 # [ 0.551147] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1349worker1 # [ 0.553301] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1350client # [ 0.551984] msm_serial: driver initialized1351client # [ 0.552132] SuperH (H)SCI(F) driver initialized1352client # [ 0.552186] STM32 USART driver initialized1353worker1 # [ 0.555757] msm_serial: driver initialized1354worker1 # [ 0.555899] SuperH (H)SCI(F) driver initialized1355worker1 # [ 0.555952] STM32 USART driver initialized1356worker2 # [ 0.554093] msm_serial: driver initialized1357worker2 # [ 0.554226] SuperH (H)SCI(F) driver initialized1358worker2 # [ 0.554285] STM32 USART driver initialized1359lb # [ 0.581451] loop: module loaded1360lb # [ 0.581614] virtio_blk virtio5: 1/0/0 default/read/poll queues1361lb # [ 0.582382] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1362lb # [ 0.593443] megasas: 07.734.00.00-rc11363lb # [ 0.594174] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1364lb # [ 0.596306] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001365lb # [ 0.596356] Intel/Sharp Extended Query Table at 0x00311366lb # [ 0.597967] Using buffer write method1367lb # [ 0.598034] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1368lb # [ 0.599752] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001369lb # [ 0.599781] Intel/Sharp Extended Query Table at 0x00311370client # [ 0.586355] loop: module loaded1371client # [ 0.586522] virtio_blk virtio5: 1/0/0 default/read/poll queues1372worker2 # [ 0.588827] loop: module loaded1373worker1 # [ 0.589711] loop: module loaded1374client # [ 0.587253] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1375worker2 # [ 0.588996] virtio_blk virtio5: 1/0/0 default/read/poll queues1376worker1 # [ 0.589898] virtio_blk virtio5: 1/0/0 default/read/poll queues1377worker2 # [ 0.589709] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1378worker1 # [ 0.590617] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1379lb # [ 0.620742] Using buffer write method1380client # [ 0.592857] megasas: 07.734.00.00-rc11381lb # [ 0.620775] Concatenating MTD devices:1382worker2 # [ 0.594497] megasas: 07.734.00.00-rc11383lb # [ 0.620779] (0): "0.flash"1384lb # [ 0.620783] (1): "0.flash"1385client # [ 0.593631] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1386lb # [ 0.620787] into device "0.flash"1387worker2 # [ 0.595248] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1388client # [ 0.595876] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001389worker2 # [ 0.597457] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001390client # [ 0.595900] Intel/Sharp Extended Query Table at 0x00311391worker2 # [ 0.597477] Intel/Sharp Extended Query Table at 0x00311392worker1 # [ 0.597749] megasas: 07.734.00.00-rc11393worker1 # [ 0.598451] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1394worker1 # [ 0.600671] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001395worker1 # [ 0.600699] Intel/Sharp Extended Query Table at 0x00311396worker1 # [ 0.602272] Using buffer write method1397worker1 # [ 0.602350] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1398client # [ 0.605257] Using buffer write method1399worker2 # [ 0.606917] Using buffer write method1400worker1 # [ 0.604106] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001401client # [ 0.605319] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1402worker2 # [ 0.606990] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1403worker1 # [ 0.604164] Intel/Sharp Extended Query Table at 0x00311404client # [ 0.608250] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001405worker2 # [ 0.609839] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406client # [ 0.608276] Intel/Sharp Extended Query Table at 0x00311407worker2 # [ 0.609871] Intel/Sharp Extended Query Table at 0x00311408worker1 # [ 0.623088] Using buffer write method1409worker1 # [ 0.623122] Concatenating MTD devices:1410worker1 # [ 0.623127] (0): "0.flash"1411client # [ 0.622241] Using buffer write method1412worker1 # [ 0.623131] (1): "0.flash"1413worker2 # [ 0.623758] Using buffer write method1414worker1 # [ 0.623134] into device "0.flash"1415client # [ 0.622274] Concatenating MTD devices:1416worker2 # [ 0.623792] Concatenating MTD devices:1417client # [ 0.622278] (0): "0.flash"1418worker2 # [ 0.623796] (0): "0.flash"1419client # [ 0.622282] (1): "0.flash"1420worker2 # [ 0.623800] (1): "0.flash"1421client # [ 0.622285] into device "0.flash"1422worker2 # [ 0.623803] into device "0.flash"1423lb # [ 0.836874] Freeing initrd memory: 26140K1424lb # [ 0.842758] tun: Universal TUN/TAP device driver, 1.61425lb # [ 0.846511] thunder_xcv, ver 1.01426lb # [ 0.846554] thunder_bgx, ver 1.01427lb # [ 0.846577] nicpf, ver 1.01428lb # [ 0.847114] e1000: Intel(R) PRO/1000 Network Driver1429lb # [ 0.847122] e1000: Copyright (c) 1999-2006 Intel Corporation.1430lb # [ 0.847146] e1000e: Intel(R) PRO/1000 Network Driver1431lb # [ 0.847154] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432lb # [ 0.847184] igb: Intel(R) Gigabit Ethernet Network Driver1433lb # [ 0.847189] igb: Copyright (c) 2007-2014 Intel Corporation.1434lb # [ 0.847217] igbvf: Intel(R) Gigabit Virtual Function Network Driver1435lb # [ 0.847223] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1436lb # [ 0.847357] sky2: driver version 1.301437lb # [ 0.849502] ehci-pci 0000:00:0a.0: EHCI Host Controller1438lb # [ 0.849527] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11439client # [ 0.833684] Freeing initrd memory: 26160K1440lb # [ 0.849722] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001441lb # [ 0.859202] usbcore: registered new interface driver usb-storage1442lb # [ 0.859295] usbcore: registered new interface driver usbserial_generic1443lb # [ 0.859310] usbserial: USB Serial support registered for generic1444lb # [ 0.859903] hv_vmbus: registering driver hyperv_keyboard1445lb # [ 0.863437] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001446client # [ 0.839580] tun: Universal TUN/TAP device driver, 1.61447lb # [ 0.863782] hub 1-0:1.0: USB hub found1448lb # [ 0.863802] hub 1-0:1.0: 6 ports detected1449lb # [ 0.866053] rtc-pl031 9010000.pl031: registered as rtc01450worker1 # [ 0.843802] Freeing initrd memory: 26148K1451lb # [ 0.866079] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T13:53:41 UTC (1789998821)1452lb # [ 0.866407] i2c_dev: i2c /dev entries driver1453client # [ 0.843163] thunder_xcv, ver 1.01454client # [ 0.843208] thunder_bgx, ver 1.01455client # [ 0.843231] nicpf, ver 1.01456client # [ 0.843786] e1000: Intel(R) PRO/1000 Network Driver1457client # [ 0.843792] e1000: Copyright (c) 1999-2006 Intel Corporation.1458worker2 # [ 0.846166] Freeing initrd memory: 26148K1459client # [ 0.843817] e1000e: Intel(R) PRO/1000 Network Driver1460client # [ 0.843824] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1461worker1 # [ 0.849841] tun: Universal TUN/TAP device driver, 1.61462client # [ 0.843854] igb: Intel(R) Gigabit Ethernet Network Driver1463lb # [ 0.871365] sdhci: Secure Digital Host Controller Interface driver1464lb # [ 0.871378] sdhci: Copyright(c) Pierre Ossman1465client # [ 0.843859] igb: Copyright (c) 2007-2014 Intel Corporation.1466lb # [ 0.871642] Synopsys Designware Multimedia Card Interface Driver1467client # [ 0.843883] igbvf: Intel(R) Gigabit Virtual Function Network Driver1468lb # [ 0.872007] sdhci-pltfm: SDHCI platform and OF driver helper1469client # [ 0.843889] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1470client # [ 0.844020] sky2: driver version 1.301471worker2 # [ 0.852134] tun: Universal TUN/TAP device driver, 1.61472lb # [ 0.876303] hid: raw HID events driver (C) Jiri Kosina1473lb # [ 0.876545] usbcore: registered new interface driver usbhid1474lb # [ 0.876552] usbhid: USB HID core driver1475client # [ 0.852990] ehci-pci 0000:00:0a.0: EHCI Host Controller1476client # [ 0.853018] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11477client # [ 0.853241] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001478worker1 # [ 0.853526] thunder_xcv, ver 1.01479worker1 # [ 0.853563] thunder_bgx, ver 1.01480client # [ 0.855920] usbcore: registered new interface driver usb-storage1481lb # [ 0.879482] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1482worker1 # [ 0.853585] nicpf, ver 1.01483client # [ 0.856058] usbcore: registered new interface driver usbserial_generic1484worker2 # [ 0.855911] thunder_xcv, ver 1.01485worker1 # [ 0.854136] e1000: Intel(R) PRO/1000 Network Driver1486worker2 # [ 0.855957] thunder_bgx, ver 1.01487lb # [ 0.882062] drop_monitor: Initializing network drop monitor service1488client # [ 0.856074] usbserial: USB Serial support registered for generic1489worker2 # [ 0.855980] nicpf, ver 1.01490worker1 # [ 0.854143] e1000: Copyright (c) 1999-2006 Intel Corporation.1491lb # [ 0.882241] NET: Registered PF_INET6 protocol family1492worker2 # [ 0.856531] e1000: Intel(R) PRO/1000 Network Driver1493worker1 # [ 0.854167] e1000e: Intel(R) PRO/1000 Network Driver1494worker2 # [ 0.856538] e1000: Copyright (c) 1999-2006 Intel Corporation.1495lb # [ 0.884116] Segment Routing with IPv61496worker1 # [ 0.854175] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1497client # [ 0.858942] hv_vmbus: registering driver hyperv_keyboard1498lb # [ 0.884135] In-situ OAM (IOAM) with IPv61499worker2 # [ 0.856562] e1000e: Intel(R) PRO/1000 Network Driver1500worker1 # [ 0.854204] igb: Intel(R) Gigabit Ethernet Network Driver1501lb # [ 0.884164] NET: Registered PF_PACKET protocol family1502worker2 # [ 0.856570] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1503worker1 # [ 0.854210] igb: Copyright (c) 2007-2014 Intel Corporation.1504worker2 # [ 0.856600] igb: Intel(R) Gigabit Ethernet Network Driver1505worker1 # [ 0.854236] igbvf: Intel(R) Gigabit Virtual Function Network Driver1506client # [ 0.860451] rtc-pl031 9010000.pl031: registered as rtc01507worker2 # [ 0.856605] igb: Copyright (c) 2007-2014 Intel Corporation.1508worker1 # [ 0.854242] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1509worker1 # [ 0.854372] sky2: driver version 1.301510client # [ 0.860475] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T13:53:41 UTC (1789998821)1511worker2 # [ 0.856629] igbvf: Intel(R) Gigabit Virtual Function Network Driver1512client # [ 0.860771] i2c_dev: i2c /dev entries driver1513worker1 # [ 0.855946] usbcore: registered new interface driver usb-storage1514worker2 # [ 0.856635] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1515worker2 # [ 0.856766] sky2: driver version 1.301516worker1 # [ 0.856030] usbcore: registered new interface driver usbserial_generic1517lb # [ 0.885955] 9pnet: Installing 9P2000 support1518worker2 # [ 0.858608] ehci-pci 0000:00:0a.0: EHCI Host Controller1519lb # [ 0.887943] Key type dns_resolver registered1520worker1 # [ 0.856045] usbserial: USB Serial support registered for generic1521worker2 # [ 0.858654] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11522worker1 # [ 0.856622] hv_vmbus: registering driver hyperv_keyboard1523worker1 # [ 0.857833] ehci-pci 0000:00:0a.0: EHCI Host Controller1524worker2 # [ 0.858862] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001525worker1 # [ 0.857859] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11526worker1 # [ 0.858125] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001527client # [ 0.865570] sdhci: Secure Digital Host Controller Interface driver1528worker2 # [ 0.868511] usbcore: registered new interface driver usb-storage1529worker1 # [ 0.869515] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001530client # [ 0.865580] sdhci: Copyright(c) Pierre Ossman1531worker2 # [ 0.868596] usbcore: registered new interface driver usbserial_generic1532worker1 # [ 0.869830] hub 1-0:1.0: USB hub found1533client # [ 0.865841] Synopsys Designware Multimedia Card Interface Driver1534worker1 # [ 0.869845] hub 1-0:1.0: 6 ports detected1535worker2 # [ 0.868614] usbserial: USB Serial support registered for generic1536client # [ 0.866211] sdhci-pltfm: SDHCI platform and OF driver helper1537worker2 # [ 0.869201] hv_vmbus: registering driver hyperv_keyboard1538client # [ 0.867854] hid: raw HID events driver (C) Jiri Kosina1539worker1 # [ 0.872771] rtc-pl031 9010000.pl031: registered as rtc01540client # [ 0.868105] usbcore: registered new interface driver usbhid1541client # [ 0.868111] usbhid: USB HID core driver1542worker2 # [ 0.872703] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001543worker1 # [ 0.872799] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T13:53:41 UTC (1789998821)1544lb # [ 0.894893] registered taskstats version 11545worker2 # [ 0.873008] hub 1-0:1.0: USB hub found1546lb # [ 0.895048] Loading compiled-in X.509 certificates1547worker2 # [ 0.873027] hub 1-0:1.0: 6 ports detected1548client # [ 0.872698] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001549worker1 # [ 0.874838] i2c_dev: i2c /dev entries driver1550client # [ 0.873003] hub 1-0:1.0: USB hub found1551client # [ 0.873033] hub 1-0:1.0: 6 ports detected1552worker2 # [ 0.875269] rtc-pl031 9010000.pl031: registered as rtc01553worker2 # [ 0.875294] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T13:53:41 UTC (1789998821)1554worker2 # [ 0.875604] i2c_dev: i2c /dev entries driver1555client # [ 0.875878] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1556lb # [ 0.903287] Demotion targets for Node 0: null1557client # [ 0.878567] drop_monitor: Initializing network drop monitor service1558worker1 # [ 0.878030] sdhci: Secure Digital Host Controller Interface driver1559lb # [ 0.903387] Key type .fscrypt registered1560client # [ 0.878685] NET: Registered PF_INET6 protocol family1561worker1 # [ 0.878039] sdhci: Copyright(c) Pierre Ossman1562lb # [ 0.903393] Key type fscrypt-provisioning registered1563client # [ 0.880583] Segment Routing with IPv61564worker1 # [ 0.878298] Synopsys Designware Multimedia Card Interface Driver1565lb # [ 0.903480] ima: No TPM chip found, activating TPM-bypass!1566worker2 # [ 0.880464] sdhci: Secure Digital Host Controller Interface driver1567client # [ 0.880599] In-situ OAM (IOAM) with IPv61568worker2 # [ 0.880473] sdhci: Copyright(c) Pierre Ossman1569worker1 # [ 0.878665] sdhci-pltfm: SDHCI platform and OF driver helper1570lb # [ 0.903499] ima: Allocated hash algorithm: sha11571client # [ 0.880627] NET: Registered PF_PACKET protocol family1572lb # [ 0.903518] ima: No architecture policies found1573worker1 # [ 0.880306] hid: raw HID events driver (C) Jiri Kosina1574worker2 # [ 0.880738] Synopsys Designware Multimedia Card Interface Driver1575worker1 # [ 0.880545] usbcore: registered new interface driver usbhid1576worker2 # [ 0.881101] sdhci-pltfm: SDHCI platform and OF driver helper1577worker1 # [ 0.880552] usbhid: USB HID core driver1578lb # [ 0.907383] input: gpio-keys as /devices/platform/gpio-keys/input/input01579client # [ 0.882179] 9pnet: Installing 9P2000 support1580worker2 # [ 0.885363] hid: raw HID events driver (C) Jiri Kosina1581client # [ 0.884219] Key type dns_resolver registered1582worker2 # [ 0.885626] usbcore: registered new interface driver usbhid1583worker2 # [ 0.885632] usbhid: USB HID core driver1584worker1 # [ 0.885982] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1585worker1 # [ 0.887467] drop_monitor: Initializing network drop monitor service1586worker1 # [ 0.887631] NET: Registered PF_INET6 protocol family1587worker1 # [ 0.890646] Segment Routing with IPv61588worker2 # [ 0.888401] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1589worker1 # [ 0.890664] In-situ OAM (IOAM) with IPv61590worker2 # [ 0.889877] drop_monitor: Initializing network drop monitor service1591worker1 # [ 0.890693] NET: Registered PF_PACKET protocol family1592worker2 # [ 0.891850] NET: Registered PF_INET6 protocol family1593worker1 # [ 0.892361] 9pnet: Installing 9P2000 support1594worker2 # [ 0.892993] Segment Routing with IPv61595worker2 # [ 0.893012] In-situ OAM (IOAM) with IPv61596client # [ 0.891189] registered taskstats version 11597worker2 # [ 0.893041] NET: Registered PF_PACKET protocol family1598client # [ 0.891343] Loading compiled-in X.509 certificates1599worker1 # [ 0.895224] Key type dns_resolver registered1600worker2 # [ 0.894896] 9pnet: Installing 9P2000 support1601worker2 # [ 0.896925] Key type dns_resolver registered1602lb # [ 0.924630] clk: Disabling unused clocks1603client # [ 0.899961] Demotion targets for Node 0: null1604lb # [ 0.924654] PM: genpd: Disabling unused power domains1605client # [ 0.900059] Key type .fscrypt registered1606client # [ 0.900066] Key type fscrypt-provisioning registered1607client # [ 0.900153] ima: No TPM chip found, activating TPM-bypass!1608client # [ 0.900172] ima: Allocated hash algorithm: sha11609client # [ 0.900192] ima: No architecture policies found1610worker1 # [ 0.901551] registered taskstats version 11611worker1 # [ 0.901706] Loading compiled-in X.509 certificates1612lb # [ 0.928862] Freeing unused kernel memory: 4736K1613client # [ 0.904231] input: gpio-keys as /devices/platform/gpio-keys/input/input01614worker2 # [ 0.903758] registered taskstats version 11615lb # [ 0.929031] Run /init as init process1616worker2 # [ 0.903916] Loading compiled-in X.509 certificates1617worker1 # [ 0.910067] Demotion targets for Node 0: null1618worker1 # [ 0.910167] Key type .fscrypt registered1619worker1 # [ 0.910174] Key type fscrypt-provisioning registered1620worker1 # [ 0.910271] ima: No TPM chip found, activating TPM-bypass!1621worker1 # [ 0.910290] ima: Allocated hash algorithm: sha11622worker2 # [ 0.912237] Demotion targets for Node 0: null1623worker1 # [ 0.910312] ima: No architecture policies found1624worker2 # [ 0.912337] Key type .fscrypt registered1625worker2 # [ 0.912343] Key type fscrypt-provisioning registered1626worker1 # [ 0.914267] input: gpio-keys as /devices/platform/gpio-keys/input/input01627worker2 # [ 0.912434] ima: No TPM chip found, activating TPM-bypass!1628worker2 # [ 0.912453] ima: Allocated hash algorithm: sha11629worker2 # [ 0.912474] ima: No architecture policies found1630worker2 # [ 0.916500] input: gpio-keys as /devices/platform/gpio-keys/input/input01631lb # [ 0.943028] systemd[1]: Successfully made /usr/ read-only.1632client # [ 0.921263] clk: Disabling unused clocks1633client # [ 0.921287] PM: genpd: Disabling unused power domains1634client # [ 0.925449] Freeing unused kernel memory: 4736K1635client # [ 0.925651] Run /init as init process1636worker1 # [ 0.932815] clk: Disabling unused clocks1637worker1 # [ 0.932844] PM: genpd: Disabling unused power domains1638worker1 # [ 0.937072] Freeing unused kernel memory: 4736K1639worker2 # [ 0.934722] clk: Disabling unused clocks1640worker1 # [ 0.937849] Run /init as init process1641worker2 # [ 0.934750] PM: genpd: Disabling unused power domains1642worker2 # [ 0.939044] Freeing unused kernel memory: 4736K1643worker2 # [ 0.939249] Run /init as init process1644client # [ 0.942490] systemd[1]: Successfully made /usr/ read-only.1645worker1 # [ 0.951409] systemd[1]: Successfully made /usr/ read-only.1646worker2 # [ 0.956038] systemd[1]: Successfully made /usr/ read-only.1647lb # [ 1.108918] usb 1-1: new high-speed USB device number 2 using ehci-pci1648worker1 # [ 1.117204] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker2 # [ 1.117981] usb 1-1: new high-speed USB device number 2 using ehci-pci1650client # [ 1.120384] usb 1-1: new high-speed USB device number 2 using ehci-pci1651lb # [ 1.261229] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652lb # [ 1.277705] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1653lb # [ 1.289262] systemd[1]: Detected virtualization qemu.1654lb # [ 1.291262] systemd[1]: Detected architecture arm64.1655lb # [ 1.293108] systemd[1]: Running in initrd.1656worker1 # [ 1.269499] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11657lb # [ 1.295575] systemd[1]: Initializing machine ID from random generator.1658worker2 # [ 1.270206] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11659lb # [ 1.298417] systemd[1]: Hostname set to <lb>.1660client # [ 1.272607] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11661client # [ 1.278414] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1662client # [ 1.278470] systemd[1]: Detected virtualization qemu.1663client # [ 1.278536] systemd[1]: Detected architecture arm64.1664worker1 # [ 1.286202] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1665client # [ 1.278558] systemd[1]: Running in initrd.1666worker1 # [ 1.298012] systemd[1]: Detected virtualization qemu.1667client # [ 1.279401] systemd[1]: Initializing machine ID from random generator.1668worker1 # [ 1.299973] systemd[1]: Detected architecture arm64.1669worker2 # [ 1.290819] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1670client # [ 1.279658] systemd[1]: Hostname set to <client>.1671worker1 # [ 1.301872] systemd[1]: Running in initrd.1672worker2 # [ 1.302671] systemd[1]: Detected virtualization qemu.1673worker1 # [ 1.304389] systemd[1]: Initializing machine ID from random generator.1674worker2 # [ 1.304552] systemd[1]: Detected architecture arm64.1675worker2 # [ 1.306416] systemd[1]: Running in initrd.1676worker1 # [ 1.307217] systemd[1]: Hostname set to <worker1>.1677worker2 # [ 1.308740] systemd[1]: Initializing machine ID from random generator.1678worker2 # [ 1.311530] systemd[1]: Hostname set to <worker2>.1679lb # [ 1.345159] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01680worker1 # [ 1.353443] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681worker2 # [ 1.354188] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682client # [ 1.360529] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683lb # [ 1.468884] usb 1-2: new high-speed USB device number 3 using ehci-pci1684worker2 # [ 1.473999] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker1 # [ 1.477212] usb 1-2: new high-speed USB device number 3 using ehci-pci1686client # [ 1.480394] usb 1-2: new high-speed USB device number 3 using ehci-pci1687lb # [ 1.610598] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688client # [ 1.599822] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689lb # [ 1.625059] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21690lb # [ 1.630516] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01691worker1 # [ 1.610752] systemd[1]: bpf-restrict-fs: LSM BPF program attached1692worker2 # [ 1.628938] systemd[1]: bpf-restrict-fs: LSM BPF program attached1693client # [ 1.635624] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21694worker1 # [ 1.636787] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21695worker2 # [ 1.637102] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21696client # [ 1.640408] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01697worker1 # [ 1.642528] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01698worker2 # [ 1.644226] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01699lb # [ 1.721967] systemd[1]: Queued start job for default target Initrd Default Target.1700lb # [ 1.731269] systemd[1]: Created slice Slice /system/modprobe.1701lb # [ 1.732440] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1702client # [ 1.708137] systemd[1]: Queued start job for default target Initrd Default Target.1703lb # [ 1.733871] systemd[1]: Expecting device /dev/disk/by-label/nixos...1704lb # [ 1.734925] systemd[1]: Reached target Path Units.1705lb # [ 1.735729] systemd[1]: Reached target Slice Units.1706lb # [ 1.736550] systemd[1]: Reached target Swaps.1707lb # [ 1.737347] systemd[1]: Reached target Timer Units.1708lb # [ 1.738366] systemd[1]: Listening on D-Bus System Message Bus Socket.1709lb # [ 1.739556] systemd[1]: Listening on Journal Socket (/dev/log).1710lb # [ 1.740652] systemd[1]: Listening on Journal Sockets.1711lb # [ 1.741702] systemd[1]: Listening on udev Control Socket.1712lb # [ 1.742702] systemd[1]: Listening on udev Kernel Socket.1713client # [ 1.718584] systemd[1]: Created slice Slice /system/modprobe.1714lb # [ 1.743669] systemd[1]: Reached target Socket Units.1715client # [ 1.719768] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1716lb # [ 1.746118] systemd[1]: Starting Create List of Static Device Nodes...1717client # [ 1.721207] systemd[1]: Expecting device /dev/disk/by-label/nixos...1718client # [ 1.721242] systemd[1]: Reached target Path Units.1719client # [ 1.721261] systemd[1]: Reached target Slice Units.1720client # [ 1.721280] systemd[1]: Reached target Swaps.1721client # [ 1.721300] systemd[1]: Reached target Timer Units.1722client # [ 1.721480] systemd[1]: Listening on D-Bus System Message Bus Socket.1723worker1 # [ 1.729428] systemd[1]: Queued start job for default target Initrd Default Target.1724client # [ 1.721636] systemd[1]: Listening on Journal Socket (/dev/log).1725client # [ 1.721769] systemd[1]: Listening on Journal Sockets.1726client # [ 1.721898] systemd[1]: Listening on udev Control Socket.1727client # [ 1.722006] systemd[1]: Listening on udev Kernel Socket.1728client # [ 1.722028] systemd[1]: Reached target Socket Units.1729lb # [ 1.756417] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1730client # [ 1.732861] systemd[1]: Starting Create List of Static Device Nodes...1731lb # [ 1.757662] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1732worker2 # [ 1.736487] systemd[1]: Queued start job for default target Initrd Default Target.1733worker1 # [ 1.739786] systemd[1]: Created slice Slice /system/modprobe.1734worker1 # [ 1.741057] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1735client # [ 1.740466] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1736worker1 # [ 1.742594] systemd[1]: Expecting device /dev/disk/by-label/nixos...1737worker1 # [ 1.743848] systemd[1]: Reached target Path Units.1738client # [ 1.741667] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1739worker1 # [ 1.744741] systemd[1]: Reached target Slice Units.1740worker1 # [ 1.745701] systemd[1]: Reached target Swaps.1741lb # [ 1.766292] systemd[1]: Mounting Kernel Configuration File System...1742worker1 # [ 1.746519] systemd[1]: Reached target Timer Units.1743worker1 # [ 1.747712] systemd[1]: Listening on D-Bus System Message Bus Socket.1744worker2 # [ 1.748360] systemd[1]: Created slice Slice /system/modprobe.1745worker1 # [ 1.749282] systemd[1]: Listening on Journal Socket (/dev/log).1746client # [ 1.747791] systemd[1]: Mounting Kernel Configuration File System...1747worker2 # [ 1.749533] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1748worker1 # [ 1.750712] systemd[1]: Listening on Journal Sockets.1749worker2 # [ 1.751059] systemd[1]: Expecting device /dev/disk/by-label/nixos...1750worker1 # [ 1.751982] systemd[1]: Listening on udev Control Socket.1751worker2 # [ 1.752084] systemd[1]: Reached target Path Units.1752worker2 # [ 1.752869] systemd[1]: Reached target Slice Units.1753worker1 # [ 1.753298] systemd[1]: Listening on udev Kernel Socket.1754worker2 # [ 1.753679] systemd[1]: Reached target Swaps.1755worker1 # [ 1.754426] systemd[1]: Reached target Socket Units.1756worker2 # [ 1.754494] systemd[1]: Reached target Timer Units.1757worker2 # [ 1.755451] systemd[1]: Listening on D-Bus System Message Bus Socket.1758worker2 # [ 1.756730] systemd[1]: Listening on Journal Socket (/dev/log).1759worker1 # [ 1.757495] systemd[1]: Starting Create List of Static Device Nodes...1760worker2 # [ 1.757816] systemd[1]: Listening on Journal Sockets.1761worker2 # [ 1.758859] systemd[1]: Listening on udev Control Socket.1762worker2 # [ 1.758977] systemd[1]: Listening on udev Kernel Socket.1763worker2 # [ 1.759001] systemd[1]: Reached target Socket Units.1764worker2 # [ 1.763018] systemd[1]: Starting Create List of Static Device Nodes...1765lb # [ 1.786837] systemd[1]: Starting Journal Service...1766worker1 # [ 1.768898] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1767client # [ 1.766982] systemd[1]: Starting Journal Service...1768worker1 # [ 1.770435] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1769worker2 # [ 1.773214] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1770lb # [ 1.797202] systemd[1]: Starting Load Kernel Modules...1771worker2 # [ 1.774525] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1772lb # [ 1.797311] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1773client # [ 1.780045] systemd[1]: Starting Load Kernel Modules...1774client # [ 1.781139] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1775worker1 # [ 1.782807] systemd[1]: Mounting Kernel Configuration File System...1776worker2 # [ 1.787471] systemd[1]: Mounting Kernel Configuration File System...1777lb # [ 1.813410] systemd[1]: Starting Coldplug All udev Devices...1778client # [ 1.789493] systemd[1]: Starting Coldplug All udev Devices...1779worker1 # [ 1.802278] systemd[1]: Starting Journal Service...1780worker1 # [ 1.809369] systemd[1]: Starting Load Kernel Modules...1781worker2 # [ 1.810150] systemd[1]: Starting Journal Service...1782worker1 # [ 1.810499] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1783worker2 # [ 1.814068] systemd[1]: Starting Load Kernel Modules...1784lb # [ 1.837673] systemd-journald[73]: Collecting audit messages is disabled.1785worker2 # [ 1.814993] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1786client # [ 1.812523] systemd[1]: Finished Create List of Static Device Nodes.1787client # [ 1.813546] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1788client # [ 1.813817] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1789client # [ 1.814032] systemd[1]: Mounted Kernel Configuration File System.1790worker2 # [ 1.819130] systemd[1]: Starting Coldplug All udev Devices...1791client # [ 1.820659] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1792lb # [ 1.844966] systemd[1]: Finished Create List of Static Device Nodes.1793lb # [ 1.845950] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1794lb # [ 1.846244] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1795worker2 # [ 1.830200] systemd[1]: Finished Create List of Static Device Nodes.1796worker2 # [ 1.831947] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1797lb # [ 1.857330] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1798worker1 # [ 1.841352] systemd[1]: Starting Coldplug All udev Devices...1799client # [ 1.838093] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1800worker2 # [ 1.846204] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1801worker1 # [ 1.849251] systemd[1]: Finished Create List of Static Device Nodes.1802worker1 # [ 1.850154] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1803worker1 # [ 1.850456] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1804client # [ 1.843503] systemd-journald[73]: Collecting audit messages is disabled.1805client # [ 1.856389] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1806lb # [ 1.881868] systemd[1]: Mounted Kernel Configuration File System.1807worker1 # [ 1.859512] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1808worker2 # [ 1.866115] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1809worker2 # [ 1.867578] systemd[1]: Mounted Kernel Configuration File System.1810worker1 # [ 1.870384] systemd-journald[73]: Collecting audit messages is disabled.1811client # [ 1.872624] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01812client # [ 1.872852] [drm] features: -virgl +edid -resource_blob -host_visible1813client # [ 1.872861] [drm] features: -context_init1814client # [ 1.873546] [drm] number of scanouts: 11815lb # [ 1.899662] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1816client # [ 1.873564] [drm] number of cap sets: 01817worker2 # [ 1.876405] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1818lb # [ 1.902029] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1819client # [ 1.877418] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1820worker1 # [ 1.873697] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1821lb # [ 1.904461] systemd[1]: Starting Create Static Device Nodes in /dev...1822client # [ 1.879790] systemd[1]: Starting Create Static Device Nodes in /dev...1823worker1 # [ 1.878325] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01824worker1 # [ 1.878557] [drm] features: -virgl +edid -resource_blob -host_visible1825worker1 # [ 1.878567] [drm] features: -context_init1826worker1 # [ 1.879268] [drm] number of scanouts: 11827worker1 # [ 1.879286] [drm] number of cap sets: 01828worker2 # [ 1.884962] systemd-journald[73]: Collecting audit messages is disabled.1829worker1 # [ 1.889363] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1830client # [ 1.892701] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1831client # [ 1.892719] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01832lb # [ 1.916967] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1833lb # [ 1.921426] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01834lb # [ 1.921673] [drm] features: -virgl +edid -resource_blob -host_visible1835lb # [ 1.921683] [drm] features: -context_init1836worker2 # [ 1.897967] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1837lb # [ 1.922380] [drm] number of scanouts: 11838worker1 # [ 1.901556] systemd[1]: Mounted Kernel Configuration File System.1839worker2 # [ 1.902498] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01840lb # [ 1.922399] [drm] number of cap sets: 01841worker2 # [ 1.902728] [drm] features: -virgl +edid -resource_blob -host_visible1842worker2 # [ 1.902738] [drm] features: -context_init1843worker2 # [ 1.903422] [drm] number of scanouts: 11844worker2 # [ 1.903440] [drm] number of cap sets: 01845worker1 # [ 1.906295] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1846worker1 # [ 1.906316] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01847worker1 # [ 1.917736] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1848worker2 # [ 1.922555] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1849worker1 # [ 1.925719] systemd[1]: Starting Create Static Device Nodes in /dev...1850lb # [ 1.944981] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1851lb # [ 1.945000] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01852client # [ 1.920585] Console: switching to colour frame buffer device 160x501853worker2 # [ 1.924975] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1854client # [ 1.927332] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1855worker2 # [ 1.924996] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01856worker2 # [ 1.932024] systemd[1]: Starting Create Static Device Nodes in /dev...1857lb # [ 1.956958] systemd[1]: Finished Create Static Device Nodes in /dev.1858lb # [ 1.957130] systemd[1]: Reached target Preparation for Local File Systems.1859lb # [ 1.957158] systemd[1]: Reached target Local File Systems.1860client # [ 1.936956] systemd[1]: Finished Create Static Device Nodes in /dev.1861client # [ 1.937146] systemd[1]: Reached target Preparation for Local File Systems.1862client # [ 1.937179] systemd[1]: Reached target Local File Systems.1863worker1 # [ 1.937404] Console: switching to colour frame buffer device 160x501864lb # [ 1.965295] systemd[1]: Starting Rule-based Manager for Device Events and Files...1865client # [ 1.944661] systemd[1]: Starting Rule-based Manager for Device Events and Files...1866lb # [ 1.973064] Console: switching to colour frame buffer device 160x501867worker2 # [ 1.954177] Console: switching to colour frame buffer device 160x501868client # [ 1.960515] systemd[1]: Finished Load Kernel Modules.1869worker2 # [ 1.960490] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1870worker1 # [ 1.965703] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1871client # [ 1.972528] systemd[1]: Starting Apply Kernel Variables...1872worker1 # [ 1.977245] systemd[1]: Finished Load Kernel Modules.1873lb # [ 2.001490] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1874worker1 # [ 1.980216] systemd[1]: Starting Apply Kernel Variables...1875worker2 # [ 1.979695] systemd[1]: Finished Create Static Device Nodes in /dev.1876worker2 # [ 1.979895] systemd[1]: Reached target Preparation for Local File Systems.1877worker2 # [ 1.979930] systemd[1]: Reached target Local File Systems.1878worker2 # [ 1.984805] systemd[1]: Starting Rule-based Manager for Device Events and Files...1879lb # [ 2.017481] systemd[1]: Finished Load Kernel Modules.1880lb # [ 2.020415] systemd[1]: Starting Apply Kernel Variables...1881worker2 # [ 1.998425] systemd[1]: Finished Load Kernel Modules.1882worker1 # [ 2.001450] systemd[1]: Finished Create Static Device Nodes in /dev.1883worker1 # [ 2.001844] systemd[1]: Reached target Preparation for Local File Systems.1884worker1 # [ 2.001881] systemd[1]: Reached target Local File Systems.1885worker1 # [ 2.006915] systemd[1]: Starting Rule-based Manager for Device Events and Files...1886client # [ 2.008989] systemd[1]: Finished Apply Kernel Variables.1887worker2 # [ 2.010308] systemd[1]: Starting Apply Kernel Variables...1888lb # [ 2.037069] systemd[1]: Started Journal Service.1889lb # [ 2.026994] systemd-modules-load[74]: Inserted module 'dm_mod'1890lb # [ 2.028075] systemd-modules-load[74]: Module 'virtio_balloon' is built in1891lb # [ 2.032371] systemd-modules-load[74]: Module 'virtio_console' is built in1892lb # [ 2.036377] systemd-modules-load[74]: Inserted module 'virtio_gpu'1893client # [ 2.012243] systemd-modules-load[74]: Inserted module 'dm_mod'1894lb # [ 2.041305] systemd-modules-load[74]: Module 'virtio_rng' is built in1895client # [ 2.029595] systemd[1]: Started Journal Service.1896client # [ 2.016356] systemd-modules-load[74]: Module 'virtio_balloon' is built in1897lb # [ 2.049242] systemd[1]: Starting Create System Files and Directories...1898client # [ 2.023896] systemd-modules-load[74]: Module 'virtio_console' is built in1899client # [ 2.028375] systemd-modules-load[74]: Inserted module 'virtio_gpu'1900worker1 # [ 2.045433] systemd[1]: Finished Apply Kernel Variables.1901worker2 # [ 2.046505] systemd[1]: Finished Apply Kernel Variables.1902client # [ 2.032271] systemd-modules-load[74]: Module 'virtio_rng' is built in1903lb # [ 2.061313] systemd-udevd[79]: Using default interface naming scheme 'v261'.1904worker1 # [ 2.050041] systemd[1]: Started Journal Service.1905client # [ 2.036153] systemd[1]: Starting Create System Files and Directories...1906worker1 # [ 2.042988] systemd-modules-load[74]: Inserted module 'dm_mod'1907worker1 # [ 2.044266] systemd-modules-load[74]: Module 'virtio_balloon' is built in1908client # [ 2.044552] systemd-udevd[79]: Using default interface naming scheme 'v261'.1909worker1 # [ 2.052281] systemd-modules-load[74]: Module 'virtio_console' is built in1910worker2 # [ 2.065178] systemd[1]: Started Journal Service.1911worker1 # [ 2.054392] systemd-modules-load[74]: Inserted module 'virtio_gpu'1912lb # [ 2.081125] systemd[1]: Finished Apply Kernel Variables.1913worker2 # [ 2.057161] systemd-modules-load[74]: Inserted module 'dm_mod'1914worker1 # [ 2.064327] systemd-modules-load[74]: Module 'virtio_rng' is built in1915worker2 # [ 2.063151] systemd-modules-load[74]: Module 'virtio_balloon' is built in1916client # [ 2.061534] systemd[1]: Finished Create System Files and Directories.1917worker1 # [ 2.069409] systemd[1]: Starting Create System Files and Directories...1918worker2 # [ 2.068326] systemd-modules-load[74]: Module 'virtio_console' is built in1919worker2 # [ 2.069404] systemd-modules-load[74]: Inserted module 'virtio_gpu'1920lb # [ 2.094211] systemd[1]: Finished Create System Files and Directories.1921worker2 # [ 2.070367] systemd-modules-load[74]: Module 'virtio_rng' is built in1922worker2 # [ 2.071372] systemd[1]: Starting Create System Files and Directories...1923lb # [ 2.104678] systemd[1]: Started Rule-based Manager for Device Events and Files.1924client # [ 2.079221] systemd[1]: Started Rule-based Manager for Device Events and Files.1925worker2 # [ 2.085687] systemd-udevd[80]: Using default interface naming scheme 'v261'.1926worker1 # [ 2.089052] systemd[1]: Finished Create System Files and Directories.1927worker1 # [ 2.098968] systemd-udevd[81]: Using default interface naming scheme 'v261'.1928worker2 # [ 2.113429] systemd[1]: Finished Create System Files and Directories.1929worker2 # [ 2.125207] systemd[1]: Started Rule-based Manager for Device Events and Files.1930worker1 # [ 2.128826] systemd[1]: Started Rule-based Manager for Device Events and Files.1931lb # [ 2.157009] systemd[1]: Starting Virtual Console Setup...1932client # [ 2.135925] systemd[1]: Starting Virtual Console Setup...1933worker2 # [ 2.176100] systemd[1]: Starting Virtual Console Setup...1934worker1 # [ 2.183744] systemd[1]: Starting Virtual Console Setup...1935lb # [ 2.213863] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1936lb # [ 2.215486] systemd[1]: Finished Virtual Console Setup.1937client # [ 2.193164] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1938client # [ 2.195661] systemd[1]: Finished Virtual Console Setup.1939worker2 # [ 2.224433] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940worker2 # [ 2.227795] systemd[1]: Finished Virtual Console Setup.1941worker1 # [ 2.232507] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942worker1 # [ 2.235709] systemd[1]: Finished Virtual Console Setup.1943client # [ 2.766088] systemd[1]: Finished Coldplug All udev Devices.1944client # [ 2.767047] systemd[1]: Reached target System Initialization.1945client # [ 2.767846] systemd[1]: Reached target Basic System.1946lb # [ 2.811056] systemd[1]: Finished Coldplug All udev Devices.1947lb # [ 2.812035] systemd[1]: Reached target System Initialization.1948lb # [ 2.812872] systemd[1]: Reached target Basic System.1949worker2 # [ 2.831388] systemd[1]: Finished Coldplug All udev Devices.1950worker2 # [ 2.832666] systemd[1]: Reached target System Initialization.1951worker2 # [ 2.833488] systemd[1]: Reached target Basic System.1952worker1 # [ 2.854587] systemd[1]: Finished Coldplug All udev Devices.1953worker1 # [ 2.856144] systemd[1]: Reached target System Initialization.1954worker1 # [ 2.860456] systemd[1]: Reached target Basic System.1955client # [ 2.895770] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1956client # [ 2.925790] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1957client # [ 2.935357] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1958lb # [ 2.973866] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1959lb # [ 2.981439] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960lb # [ 2.985405] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1961worker2 # [ 2.982477] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1962worker2 # [ 2.995091] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1963worker2 # [ 3.002875] (udev-worker)[107]: Network interface NamePolicy= disabled on kernel command line.1964lb # [ 3.037564] systemd[1]: Found device /dev/disk/by-label/nixos.1965lb # [ 3.040551] systemd[1]: Reached target Initrd Root Device.1966lb # [ 3.043882] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1967client # [ 3.016271] systemd[1]: Found device /dev/disk/by-label/nixos.1968worker1 # [ 3.024630] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1969client # [ 3.021410] systemd[1]: Reached target Initrd Root Device.1970client # [ 3.025556] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1971worker1 # [ 3.036137] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972worker1 # [ 3.066015] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1973worker2 # [ 3.067932] systemd[1]: Found device /dev/disk/by-label/nixos.1974client # [ 3.067356] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1975worker2 # [ 3.072121] systemd[1]: Reached target Initrd Root Device.1976lb # [ 3.097447] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1977worker2 # [ 3.077268] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1978lb # [ 3.102943] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1979client # [ 3.074871] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1980client # [ 3.077029] systemd[1]: Mounting /sysroot...1981lb # [ 3.116290] systemd[1]: Mounting /sysroot...1982worker1 # [ 3.100692] systemd[1]: Found device /dev/disk/by-label/nixos.1983worker1 # [ 3.103762] systemd[1]: Reached target Initrd Root Device.1984worker1 # [ 3.106084] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985client # [ 3.124649] EXT4-fs (vda): mounted filesystem 3d2ad251-c0e5-441f-9f9c-e6b65b15088c r/w with ordered data mode. Quota mode: none.1986lb # [ 3.151307] EXT4-fs (vda): mounted filesystem 786c49ae-8289-420c-bba5-9774400483cb r/w with ordered data mode. Quota mode: none.1987client # [ 3.113923] systemd[1]: Mounted /sysroot.1988lb # [ 3.143557] systemd[1]: Mounted /sysroot.1989client # [ 3.116085] systemd[1]: Reached target Initrd Root File System.1990lb # [ 3.145566] systemd[1]: Reached target Initrd Root File System.1991lb # [ 3.148110] systemd[1]: Starting Mountpoints Configured in the Real Root...1992client # [ 3.120902] systemd[1]: Starting Mountpoints Configured in the Real Root...1993worker2 # [ 3.125871] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1994worker2 # [ 3.134438] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1995worker2 # [ 3.137447] systemd[1]: Mounting /sysroot...1996lb # [ 3.171548] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1997client # [ 3.148297] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1998client # [ 3.151543] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1999lb # [ 3.180117] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...2000worker1 # [ 3.160665] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks
loading 1,915 hidden lines…
3916worker2 # [ 34.326098] nix-grpc-daemon[778]: ts=2026-09-21T13:54:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3917worker1 # [ 34.373003] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513918worker1 # [ 34.378046] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3919worker2 # [ 34.527028] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513920worker2 # [ 34.532229] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3921worker1 # [ 34.580592] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513922worker1 # [ 34.584574] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3923worker2 # [ 34.734973] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513924worker2 # [ 34.738977] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3925worker1 # [ 34.781114] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513926worker1 # [ 34.785883] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3927worker2 # [ 34.935562] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513928worker2 # [ 34.939913] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3929worker1 # [ 34.986681] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513930worker1 # [ 34.991701] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3931lb: must succeed: curl -sf localhost:9901/clusters3932lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3933worker2 # [ 35.141102] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513934worker2 # [ 35.146020] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3935worker1 # [ 35.192581] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513936worker1 # [ 35.197050] nix-grpc-daemon[772]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3937worker2 # [ 35.346928] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_connected addr=lb:500513938worker2 # [ 35.352960] nix-grpc-daemon[778]: ts=2026-09-21T13:54:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3939worker1 # [ 35.397673] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_connected addr=lb:500513940worker1 # [ 35.402538] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3941worker2 # [ 35.554085] nix-grpc-daemon[778]: ts=2026-09-21T13:54:16Z level=info event=scheduler_connected addr=lb:500513942worker2 # [ 35.559069] nix-grpc-daemon[778]: ts=2026-09-21T13:54:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3943worker1 # [ 35.605099] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_connected addr=lb:500513944worker1 # [ 35.609026] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3945worker2 # [ 35.761783] nix-grpc-daemon[778]: ts=2026-09-21T13:54:16Z level=info event=scheduler_connected addr=lb:500513946worker2 # [ 35.765863] nix-grpc-daemon[778]: ts=2026-09-21T13:54:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3947worker1 # [ 35.805467] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_connected addr=lb:500513948worker1 # [ 35.810234] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3949worker2 # [ 35.962644] nix-grpc-daemon[778]: ts=2026-09-21T13:54:16Z level=info event=scheduler_connected addr=lb:500513950worker2 # [ 35.968220] nix-grpc-daemon[778]: ts=2026-09-21T13:54:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3951worker1 # [ 36.011182] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_connected addr=lb:500513952worker1 # [ 36.016282] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3953lb: must succeed: curl -sf localhost:9901/clusters3954worker2 # [ 36.169140] nix-grpc-daemon[778]: ts=2026-09-21T13:54:16Z level=info event=scheduler_connected addr=lb:500513955worker2 # [ 36.174102] nix-grpc-daemon[778]: ts=2026-09-21T13:54:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3956lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3957worker1 # [ 36.217003] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_connected addr=lb:500513958worker1 # [ 36.221993] nix-grpc-daemon[772]: ts=2026-09-21T13:54:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3959worker2 # [ 36.374966] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513960worker2 # [ 36.380101] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3961worker1 # [ 36.423093] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513962worker1 # [ 36.428643] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3963worker2 # [ 36.581090] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513964worker2 # [ 36.586115] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3965worker1 # [ 36.630755] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513966worker1 # [ 36.634877] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3967worker2 # [ 36.788988] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513968worker2 # [ 36.793041] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3969worker1 # [ 36.831282] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513970worker1 # [ 36.836499] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3971worker2 # [ 36.989545] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513972worker2 # [ 36.994541] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3973worker1 # [ 37.037226] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513974worker1 # [ 37.042167] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3975worker2 # [ 37.195729] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513976worker2 # [ 37.200951] nix-grpc-daemon[778]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3977lb: must succeed: curl -sf localhost:9901/clusters3978worker1 # [ 37.242881] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_connected addr=lb:500513979worker1 # [ 37.247930] nix-grpc-daemon[772]: ts=2026-09-21T13:54:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3980lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3981worker2 # [ 37.401775] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500513982worker2 # [ 37.406729] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3983worker1 # [ 37.448711] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500513984worker1 # [ 37.453745] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3985worker2 # [ 37.607742] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500513986worker2 # [ 37.612804] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3987worker1 # [ 37.656310] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500513988worker1 # [ 37.660315] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3989worker2 # [ 37.815490] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500513990worker2 # [ 37.819649] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3991worker1 # [ 37.856558] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500513992worker1 # [ 37.861635] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3993worker2 # [ 38.017327] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500513994worker2 # [ 38.021449] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3995worker1 # [ 38.062322] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500513996worker1 # [ 38.067555] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3997worker2 # [ 38.218298] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500513998worker2 # [ 38.221108] nix-grpc-daemon[778]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3999worker1 # [ 38.268203] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_connected addr=lb:500514000worker1 # [ 38.273070] nix-grpc-daemon[772]: ts=2026-09-21T13:54:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4001lb: must succeed: curl -sf localhost:9901/clusters4002lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4003worker2 # [ 38.422278] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514004worker2 # [ 38.427306] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4005worker1 # [ 38.473777] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514006worker1 # [ 38.478288] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4007worker2 # [ 38.628329] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514008worker2 # [ 38.633270] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4009worker1 # [ 38.680682] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514010worker1 # [ 38.684759] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4011worker2 # [ 38.835959] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514012worker2 # [ 38.839977] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4013worker1 # [ 38.881200] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514014worker1 # [ 38.886349] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4015worker2 # [ 39.036496] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514016worker2 # [ 39.041502] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4017worker1 # [ 39.087007] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514018worker1 # [ 39.092177] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4019worker2 # [ 39.242471] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514020worker2 # [ 39.247417] nix-grpc-daemon[778]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4021worker1 # [ 39.292870] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_connected addr=lb:500514022worker1 # [ 39.297957] nix-grpc-daemon[772]: ts=2026-09-21T13:54:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4023lb: must succeed: curl -sf localhost:9901/clusters4024worker2 # [ 39.448372] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514025worker2 # [ 39.453116] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4026lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4027worker1 # [ 39.498624] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514028worker1 # [ 39.503455] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4029worker2 # [ 39.654181] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514030worker2 # [ 39.658975] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4031worker1 # [ 39.706101] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514032worker1 # [ 39.710149] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4033worker2 # [ 39.861587] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514034worker2 # [ 39.865474] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4035worker1 # [ 39.906410] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514036worker1 # [ 39.911360] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4037worker2 # [ 40.062264] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514038worker2 # [ 40.067000] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4039worker1 # [ 40.112198] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514040worker1 # [ 40.117192] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4041worker2 # [ 40.268078] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514042worker2 # [ 40.272991] nix-grpc-daemon[778]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4043worker1 # [ 40.317866] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_connected addr=lb:500514044worker1 # [ 40.322966] nix-grpc-daemon[772]: ts=2026-09-21T13:54:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4045worker2 # [ 40.416992] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_take_over4046lb # [ 40.442156] niks3-server[1013]: 2026/09/21 13:54:21 INFO lead: acquired remote=192.168.1.4:438124047worker2 # [ 40.473866] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514048lb: must succeed: curl -sf localhost:9901/clusters4049worker2 # [ 40.478727] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4050worker1 # [ 40.523856] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514051worker1 # [ 40.528988] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4052lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4053worker2 # [ 40.679788] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514054worker2 # [ 40.684994] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4055worker1 # [ 40.731419] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514056worker1 # [ 40.735567] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4057worker2 # [ 40.887662] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514058worker2 # [ 40.891841] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4059worker1 # [ 40.931925] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514060worker1 # [ 40.937067] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4061worker2 # [ 41.088532] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514062worker2 # [ 41.093567] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4063worker1 # [ 41.137730] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514064worker1 # [ 41.142881] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4065worker2 # [ 41.294472] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514066worker2 # [ 41.299566] nix-grpc-daemon[778]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4067worker1 # [ 41.343397] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_connected addr=lb:500514068worker1 # [ 41.348371] nix-grpc-daemon[772]: ts=2026-09-21T13:54:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4069worker2 # [ 41.500967] nix-grpc-daemon[778]: ts=2026-09-21T13:54:22Z level=info event=scheduler_connected addr=lb:500514070worker2 # [ 41.524543] nix-grpc-daemon[778]: ts=2026-09-21T13:54:22Z level=info event=oidc_jwks provider=mock keys=14071worker2 # [ 41.527587] nix-grpc-daemon[778]: ts=2026-09-21T13:54:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:531724072worker2 # [ 41.532271] nix-grpc-daemon[778]: ts=2026-09-21T13:54:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04073worker2 # [ 41.549161] nix-grpc-daemon[778]: ts=2026-09-21T13:54:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:531724074worker1 # [ 41.549068] nix-grpc-daemon[772]: ts=2026-09-21T13:54:22Z level=info event=scheduler_connected addr=lb:500514075worker2 # [ 41.551695] nix-grpc-daemon[778]: ts=2026-09-21T13:54:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04076lb: must succeed: curl -sf localhost:9901/clusters4077lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4078worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4079worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4080lb: must succeed: curl -sf localhost:9901/clusters4081lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4082lb: must succeed: curl -sf localhost:9901/clusters4083lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4084lb: must succeed: curl -sf localhost:9901/clusters4085lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4086lb: must succeed: curl -sf localhost:9901/clusters4087lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4088worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4089worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)4090worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4091worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4092lb: must succeed: curl -sf localhost:9901/clusters4093lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4094lb: must succeed: curl -sf localhost:9901/clusters4095lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4096worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4097worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)4098worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4099worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4100worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4101worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4102worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4103worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4104(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.65 seconds)4105subtest: balancer auth: client cert, bearer token, nothing, unknown CN4106worker1 # [ 42.847948] nix-grpc-daemon[772]: ts=2026-09-21T13:54:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:404184107worker1 # [ 42.864912] systemd[1]: Started Nix Daemon.4108worker1 # [ 43.122570] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4109worker1 # [ 43.145176] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)4110worker1 # [ 43.181343] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 2/5)4111worker1 # [ 43.310076] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 234 ms (attempt 3/5)4112worker1 # [ 43.551644] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 221 ms (attempt 4/5)4113worker1 # [ 43.782297] nix-grpc-daemon[772]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4114worker1 # [ 43.788233] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4115worker2 # [ 43.821323] nix-grpc-daemon[778]: ts=2026-09-21T13:54:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:531824116worker2 # [ 43.840003] systemd[1]: Started Nix Daemon.4117worker2 # [ 44.109351] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4118worker2 # [ 44.131279] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)4119worker2 # [ 44.161956] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 2/5)4120worker2 # [ 44.296355] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 383 ms (attempt 3/5)4121worker2 # [ 44.686894] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 693 ms (attempt 4/5)4122worker2 # [ 45.390529] nix-grpc-daemon[778]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4123worker2 # [ 45.395888] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4124worker2 # [ 45.413072] nix-grpc-daemon[778]: ts=2026-09-21T13:54:26Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4125worker1 # [ 45.425141] nix-grpc-daemon[772]: ts=2026-09-21T13:54:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:404184126worker1 # [ 45.440148] nix-grpc-daemon[772]: ts=2026-09-21T13:54:26Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4127client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4128client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)4129client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4130client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)4131worker1 # [ 45.876267] nix-grpc-daemon[772]: ts=2026-09-21T13:54:26Z level=info event=oidc_jwks provider=mock keys=14132worker1 # [ 45.883571] nix-grpc-daemon[772]: ts=2026-09-21T13:54:26Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:404184133worker2 # [ 45.895006] nix-grpc-daemon[778]: ts=2026-09-21T13:54:26Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:531824134worker2 # [ 45.907434] nix-grpc-daemon[778]: ts=2026-09-21T13:54:26Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4135worker2 # [ 45.921952] nix-grpc-daemon[778]: ts=2026-09-21T13:54:26Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:531824136worker2 # [ 45.929994] nix-grpc-daemon[778]: ts=2026-09-21T13:54:26Z level=debug event=path_info_miss path=5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test error="error: path '/nix/store/5dwn502h5kcmk6n9yvgspbhv8xp80kb2-nixos-system-client-test' is required, but there is no substituter that can build it"4137client: must succeed: rm -r /var/lib/nix-grpc-store4138client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4139worker1 # [ 46.231082] nix-grpc-daemon[772]: ts=2026-09-21T13:54:26Z level=info event=denied method=QueryMissing cn=- role=none4140worker1 # [ 46.486096] nix-grpc-daemon[772]: ts=2026-09-21T13:54:27Z level=info event=denied method=QueryMissing cn=stranger role=none4141(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.04 seconds)4142subtest: DAG build is scheduled across workers and lands in the cache4143client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24144worker2 # [ 46.739323] nix-grpc-daemon[778]: ts=2026-09-21T13:54:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531824145client # don't know how to build these paths:4146client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4147worker1 # [ 46.753506] nix-grpc-daemon[772]: ts=2026-09-21T13:54:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404184148worker2 # [ 46.787973] nix-grpc-daemon[778]: ts=2026-09-21T13:54:27Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04149worker2 # [ 46.795222] nix-grpc-daemon[778]: ts=2026-09-21T13:54:27Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=14150worker2 # [ 46.847371] nix-grpc-daemon[778]: ts=2026-09-21T13:54:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824151worker2 # [ 46.869318] nix-grpc-daemon[778]: ts=2026-09-21T13:54:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824152client # copying 1 paths to 'lb:50051'...4153worker2 # [ 46.884097] nix-grpc-daemon[778]: ts=2026-09-21T13:54:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531824154client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4155worker2 # [ 46.913946] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4156worker2 # [ 46.945607] nix-grpc-daemon[778]: ts=2026-09-21T13:54:27Z level=info event=niks3_push_started pid=8794157worker2 # [ 47.223647] nix-grpc-daemon[879]: time=2026-09-21T13:54:27.860Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4158worker2 # [ 47.430958] nix-daemon[864]: accepted connection from pid 892, user nix-grpc-daemon (trusted)4159worker2 # [ 47.454552] nix-daemon[864]: reaped child process 895, status = succeeded4160worker2 # [ 47.691124] nix-daemon[864]: accepted connection from pid 897, user nix-grpc-daemon (trusted)4161worker2 # [ 47.716099] nix-daemon[864]: reaped child process 900, status = succeeded4162lb # [ 47.753352] niks3-server[1013]: 2026/09/21 13:54:28 INFO Received uploads request method=POST path=/api/pending_closures4163worker2 # [ 47.765983] nix-grpc-daemon[879]: time=2026-09-21T13:54:28.404Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4164worker2 # [ 47.770899] nix-grpc-daemon[879]: time=2026-09-21T13:54:28.406Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4165lb # [ 47.853458] niks3-server[1013]: 2026/09/21 13:54:28 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4166lb # [ 47.866062] niks3-server[1013]: 2026/09/21 13:54:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4167lb # [ 47.878463] niks3-server[1013]: 2026/09/21 13:54:28 INFO Signed narinfos id=1 count=14168worker2 # [ 47.858797] nix-grpc-daemon[879]: time=2026-09-21T13:54:28.495Z level=INFO msg="Uploading 1 narinfos"4169lb # [ 47.927267] niks3-server[1013]: 2026/09/21 13:54:28 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4170lb # [ 47.937854] niks3-server[1013]: 2026/09/21 13:54:28 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4171lb # [ 47.942690] niks3-server[1013]: 2026/09/21 13:54:28 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4172lb # [ 47.950901] niks3-server[1013]: 2026/09/21 13:54:28 INFO Completed upload id=14173worker2 # [ 47.927008] nix-grpc-daemon[879]: time=2026-09-21T13:54:28.565Z level=INFO msg="Upload complete. (695ms)"4174worker2 # [ 47.943175] nix-grpc-daemon[778]: ts=2026-09-21T13:54:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=1 paths=1 nar_bytes_in=5044175worker2 # [ 47.956877] nix-grpc-daemon[778]: ts=2026-09-21T13:54:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824176worker2 # [ 47.974203] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4177worker2 # [ 47.984799] nix-daemon[864]: reaped child process 877, status = succeeded4178client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4179client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4180worker2 # [ 48.000421] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4181client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4182client # farm-a> LOG-farm-a-t14183worker2 # [ 49.447754] nix-daemon[864]: reaped child process 904, status = succeeded4184worker2 # [ 49.653742] nix-daemon[864]: accepted connection from pid 921, user nix-grpc-daemon (trusted)4185worker2 # [ 49.671110] nix-daemon[864]: reaped child process 924, status = succeeded4186lb # [ 49.708789] niks3-server[1013]: 2026/09/21 13:54:30 INFO Received uploads request method=POST path=/api/pending_closures4187worker2 # [ 49.701617] nix-grpc-daemon[879]: time=2026-09-21T13:54:30.338Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4188worker2 # [ 49.703430] nix-grpc-daemon[879]: time=2026-09-21T13:54:30.338Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4189lb # [ 49.768692] niks3-server[1013]: 2026/09/21 13:54:30 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4190lb # [ 49.778571] niks3-server[1013]: 2026/09/21 13:54:30 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4191lb # [ 49.789886] niks3-server[1013]: 2026/09/21 13:54:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4192lb # [ 49.797426] niks3-server[1013]: 2026/09/21 13:54:30 INFO Signed narinfos id=2 count=14193worker2 # [ 49.772110] nix-grpc-daemon[879]: time=2026-09-21T13:54:30.409Z level=INFO msg="Uploading 1 narinfos"4194lb # [ 49.799988] niks3-server[1013]: 2026/09/21 13:54:30 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4195lb # [ 49.810537] niks3-server[1013]: 2026/09/21 13:54:30 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4196lb # [ 49.817520] niks3-server[1013]: 2026/09/21 13:54:30 INFO Completed upload id=24197worker2 # [ 49.792456] nix-grpc-daemon[879]: time=2026-09-21T13:54:30.430Z level=INFO msg="Upload complete. (376ms)"4198lb # [ 49.820139] niks3-server[1013]: 2026/09/21 13:54:30 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4199worker2 # [ 49.817374] nix-grpc-daemon[778]: ts=2026-09-21T13:54:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14200worker2 # [ 49.832350] nix-grpc-daemon[778]: ts=2026-09-21T13:54:30Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04201worker2 # [ 49.835540] nix-grpc-daemon[778]: ts=2026-09-21T13:54:30Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=14202worker2 # [ 49.839517] nix-daemon[864]: reaped child process 902, status = succeeded4203worker2 # [ 49.862450] nix-grpc-daemon[778]: ts=2026-09-21T13:54:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824204worker2 # [ 49.878069] nix-grpc-daemon[778]: ts=2026-09-21T13:54:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824205worker2 # [ 49.893847] nix-grpc-daemon[879]: time=2026-09-21T13:54:30.532Z level=INFO msg="All 1 paths already cached"4206client # copying 1 paths to 'lb:50051'...4207worker2 # [ 49.902710] nix-grpc-daemon[778]: ts=2026-09-21T13:54:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531824208client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4209worker2 # [ 49.926358] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4210worker2 # [ 50.134082] nix-daemon[864]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4211worker2 # [ 50.150910] nix-daemon[864]: reaped child process 933, status = succeeded4212worker2 # [ 50.321975] nix-daemon[864]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4213worker2 # [ 50.338781] nix-daemon[864]: reaped child process 938, status = succeeded4214lb # [ 50.374484] niks3-server[1013]: 2026/09/21 13:54:31 INFO Received uploads request method=POST path=/api/pending_closures4215worker2 # [ 50.360104] nix-grpc-daemon[879]: time=2026-09-21T13:54:30.998Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4216worker2 # [ 50.362201] nix-grpc-daemon[879]: time=2026-09-21T13:54:31.000Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4217worker2 # [ 50.396258] nix-grpc-daemon[879]: time=2026-09-21T13:54:31.034Z level=INFO msg="Uploading 1 narinfos"4218lb # [ 50.408738] niks3-server[1013]: 2026/09/21 13:54:31 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4219lb # [ 50.426442] niks3-server[1013]: 2026/09/21 13:54:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4220lb # [ 50.441010] niks3-server[1013]: 2026/09/21 13:54:31 INFO Signed narinfos id=3 count=14221lb # [ 50.456477] niks3-server[1013]: 2026/09/21 13:54:31 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4222worker2 # [ 50.439330] nix-grpc-daemon[879]: time=2026-09-21T13:54:31.077Z level=INFO msg="Upload complete. (495ms)"4223lb # [ 50.464920] niks3-server[1013]: 2026/09/21 13:54:31 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4224lb # [ 50.474308] niks3-server[1013]: 2026/09/21 13:54:31 INFO Completed upload id=34225worker2 # [ 50.448605] nix-grpc-daemon[778]: ts=2026-09-21T13:54:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_in=6244226lb # [ 50.477352] niks3-server[1013]: 2026/09/21 13:54:31 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4227worker2 # [ 50.463431] nix-grpc-daemon[778]: ts=2026-09-21T13:54:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824228worker2 # [ 50.470504] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4229worker2 # [ 50.475678] nix-daemon[864]: reaped child process 928, status = succeeded4230client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4231client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4232worker2 # [ 50.490324] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4233client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4234lb # [ 51.883275] envoy[903]: conn peer=[2001:db8:1::1]:55366 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=26554235lb # [ 51.888468] envoy[903]: conn peer=[2001:db8:1::1]:55378 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=774236lb # [ 51.894378] envoy[903]: conn peer=[2001:db8:1::1]:55388 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134237lb # [ 51.899003] envoy[903]: conn peer=[2001:db8:1::1]:55394 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=94238client # farm-b> LOG-farm-b-t14239worker2 # [ 51.908903] nix-daemon[864]: reaped child process 942, status = succeeded4240worker2 # [ 52.071197] nix-daemon[864]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4241worker2 # [ 52.086594] nix-daemon[864]: reaped child process 961, status = succeeded4242lb # [ 52.127397] niks3-server[1013]: 2026/09/21 13:54:32 INFO Received uploads request method=POST path=/api/pending_closures4243worker2 # [ 52.120463] nix-grpc-daemon[879]: time=2026-09-21T13:54:32.757Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4244worker2 # [ 52.121933] nix-grpc-daemon[879]: time=2026-09-21T13:54:32.757Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4245lb # [ 52.176194] niks3-server[1013]: 2026/09/21 13:54:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4246worker2 # [ 52.159991] nix-grpc-daemon[879]: time=2026-09-21T13:54:32.798Z level=INFO msg="Uploading 1 narinfos"4247lb # [ 52.193702] niks3-server[1013]: 2026/09/21 13:54:32 INFO Signed narinfos id=4 count=14248lb # [ 52.205017] niks3-server[1013]: 2026/09/21 13:54:32 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4249lb # [ 52.216226] niks3-server[1013]: 2026/09/21 13:54:32 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4250lb # [ 52.221414] niks3-server[1013]: 2026/09/21 13:54:32 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4251lb # [ 52.225245] niks3-server[1013]: 2026/09/21 13:54:32 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4252lb # [ 52.229718] niks3-server[1013]: 2026/09/21 13:54:32 INFO Completed upload id=44253worker2 # [ 52.205578] nix-grpc-daemon[879]: time=2026-09-21T13:54:32.844Z level=INFO msg="Upload complete. (305ms)"4254lb # [ 52.232463] niks3-server[1013]: 2026/09/21 13:54:32 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4255worker2 # [ 52.222529] nix-grpc-daemon[778]: ts=2026-09-21T13:54:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14256worker2 # [ 52.237925] nix-grpc-daemon[778]: ts=2026-09-21T13:54:32Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04257worker2 # [ 52.242299] nix-grpc-daemon[778]: ts=2026-09-21T13:54:32Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=14258worker2 # [ 52.246791] nix-daemon[864]: reaped child process 940, status = succeeded4259worker2 # [ 52.269232] nix-grpc-daemon[778]: ts=2026-09-21T13:54:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824260worker2 # [ 52.278512] nix-grpc-daemon[778]: ts=2026-09-21T13:54:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824261worker2 # [ 52.290763] nix-grpc-daemon[879]: time=2026-09-21T13:54:32.929Z level=INFO msg="All 2 paths already cached"4262client # copying 1 paths to 'lb:50051'...4263worker2 # [ 52.300094] nix-grpc-daemon[778]: ts=2026-09-21T13:54:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531824264client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4265worker2 # [ 52.328888] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4266worker2 # [ 52.528673] nix-daemon[864]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4267worker2 # [ 52.545765] nix-daemon[864]: reaped child process 970, status = succeeded4268worker2 # [ 52.716610] nix-daemon[864]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4269worker2 # [ 52.733042] nix-daemon[864]: reaped child process 975, status = succeeded4270lb # [ 52.769581] niks3-server[1013]: 2026/09/21 13:54:33 INFO Received uploads request method=POST path=/api/pending_closures4271worker2 # [ 52.756736] nix-grpc-daemon[879]: time=2026-09-21T13:54:33.394Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4272worker2 # [ 52.758209] nix-grpc-daemon[879]: time=2026-09-21T13:54:33.395Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4273worker2 # [ 52.784448] nix-grpc-daemon[879]: time=2026-09-21T13:54:33.422Z level=INFO msg="Uploading 1 narinfos"4274lb # [ 52.808463] niks3-server[1013]: 2026/09/21 13:54:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4275lb # [ 52.811816] niks3-server[1013]: 2026/09/21 13:54:33 INFO Signed narinfos id=5 count=14276lb # [ 52.821129] niks3-server[1013]: 2026/09/21 13:54:33 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4277lb # [ 52.831430] niks3-server[1013]: 2026/09/21 13:54:33 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4278worker2 # [ 52.813084] nix-grpc-daemon[879]: time=2026-09-21T13:54:33.451Z level=INFO msg="Upload complete. (466ms)"4279lb # [ 52.839363] niks3-server[1013]: 2026/09/21 13:54:33 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4280worker2 # [ 52.820330] nix-grpc-daemon[778]: ts=2026-09-21T13:54:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_in=7524281lb # [ 52.847470] niks3-server[1013]: 2026/09/21 13:54:33 INFO Completed upload id=54282lb # [ 52.850301] niks3-server[1013]: 2026/09/21 13:54:33 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4283worker2 # [ 52.828552] nix-grpc-daemon[778]: ts=2026-09-21T13:54:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824284worker2 # [ 52.840161] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4285worker2 # [ 52.842438] nix-daemon[864]: reaped child process 965, status = succeeded4286client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4287client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4288worker2 # [ 52.854710] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4289client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4290client # farm-top> LOG-farm-top-t14291worker2 # [ 54.274604] nix-daemon[864]: reaped child process 979, status = succeeded4292worker2 # [ 54.428826] nix-daemon[864]: accepted connection from pid 995, user nix-grpc-daemon (trusted)4293worker2 # [ 54.446432] nix-daemon[864]: reaped child process 998, status = succeeded4294lb # [ 54.483553] niks3-server[1013]: 2026/09/21 13:54:35 INFO Received uploads request method=POST path=/api/pending_closures4295worker2 # [ 54.470569] nix-grpc-daemon[879]: time=2026-09-21T13:54:35.107Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4296worker2 # [ 54.472086] nix-grpc-daemon[879]: time=2026-09-21T13:54:35.107Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4297lb # [ 54.519808] niks3-server[1013]: 2026/09/21 13:54:35 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4298lb # [ 54.531342] niks3-server[1013]: 2026/09/21 13:54:35 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4299lb # [ 54.543973] niks3-server[1013]: 2026/09/21 13:54:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4300worker2 # [ 54.524779] nix-grpc-daemon[879]: time=2026-09-21T13:54:35.162Z level=INFO msg="Uploading 1 narinfos"4301lb # [ 54.552030] niks3-server[1013]: 2026/09/21 13:54:35 INFO Signed narinfos id=6 count=14302lb # [ 54.554646] niks3-server[1013]: 2026/09/21 13:54:35 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4303lb # [ 54.565823] niks3-server[1013]: 2026/09/21 13:54:35 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4304worker2 # [ 54.546823] nix-grpc-daemon[879]: time=2026-09-21T13:54:35.185Z level=INFO msg="Upload complete. (283ms)"4305lb # [ 54.573147] niks3-server[1013]: 2026/09/21 13:54:35 INFO Completed upload id=64306lb # [ 54.577120] niks3-server[1013]: 2026/09/21 13:54:35 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4307worker2 # [ 54.556949] nix-grpc-daemon[778]: ts=2026-09-21T13:54:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14308worker2 # [ 54.572216] nix-daemon[864]: reaped child process 977, status = succeeded4309client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 8.13 seconds)4310client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4311client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.24 seconds)4312client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4313client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4314client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4315client # copying 3 paths...4316client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4317client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4318client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4319client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4320(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.81 seconds)4321subtest: repeat is answered Cached by the scheduler without building4322worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4323worker1 # finding garbage collector roots...4324worker1 # deleting specified paths...4325worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4326worker1 # deleting unused links...4327worker1 # note: hard linking is currently saving 0.0 KiB4328worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds)4329worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4330worker2 # finding garbage collector roots...4331worker2 # deleting specified paths...4332worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4333worker2 # deleting unused links...4334worker2 # note: hard linking is currently saving 0.0 KiB4335worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.27 seconds)4336client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24337worker1 # [ 56.163153] nix-grpc-daemon[772]: ts=2026-09-21T13:54:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:404184338client # don't know how to build these paths:4339client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4340worker2 # [ 56.174200] nix-grpc-daemon[778]: ts=2026-09-21T13:54:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824341worker2 # [ 56.185482] nix-grpc-daemon[879]: time=2026-09-21T13:54:36.823Z level=INFO msg="All 2 paths already cached"4342worker2 # [ 56.202339] nix-grpc-daemon[778]: ts=2026-09-21T13:54:36Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14343client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.35 seconds)4344client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4345client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4346lb: must succeed: curl -sf localhost:9901/clusters4347lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4348lb: must succeed: curl -sf localhost:9901/clusters4349lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4350worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4351worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4352(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.47 seconds)4353subtest: inputs already in the cache are not sent to the scheduler4354lb: must succeed: curl -sf localhost:9901/clusters4355lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4356lb: must succeed: curl -sf localhost:9901/clusters4357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4358worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4359worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4360client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24361worker2 # [ 57.237293] nix-grpc-daemon[778]: ts=2026-09-21T13:54:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531824362client # don't know how to build these paths:4363client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4364worker1 # [ 57.253860] nix-grpc-daemon[772]: ts=2026-09-21T13:54:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404184365worker2 # [ 57.282148] nix-grpc-daemon[778]: ts=2026-09-21T13:54:37Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04366worker2 # [ 57.289805] nix-grpc-daemon[778]: ts=2026-09-21T13:54:37Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14367worker2 # [ 57.328131] nix-grpc-daemon[778]: ts=2026-09-21T13:54:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824368worker2 # [ 57.343601] nix-grpc-daemon[778]: ts=2026-09-21T13:54:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824369client # copying 1 paths to 'lb:50051'...4370worker2 # [ 57.359698] nix-grpc-daemon[879]: time=2026-09-21T13:54:37.996Z level=INFO msg="All 2 paths already cached"4371client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4372worker2 # [ 57.365161] nix-grpc-daemon[778]: ts=2026-09-21T13:54:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531824373worker2 # [ 57.383009] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4374worker2 # [ 57.592750] nix-daemon[864]: accepted connection from pid 1027, user nix-grpc-daemon (trusted)4375worker2 # [ 57.611535] nix-daemon[864]: reaped child process 1030, status = succeeded4376worker2 # [ 57.784264] nix-daemon[864]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4377worker2 # [ 57.801738] nix-daemon[864]: reaped child process 1035, status = succeeded4378lb # [ 57.838502] niks3-server[1013]: 2026/09/21 13:54:38 INFO Received uploads request method=POST path=/api/pending_closures4379worker2 # [ 57.823627] nix-grpc-daemon[879]: time=2026-09-21T13:54:38.461Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4380worker2 # [ 57.825293] nix-grpc-daemon[879]: time=2026-09-21T13:54:38.461Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4381lb # [ 57.864863] niks3-server[1013]: 2026/09/21 13:54:38 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4382lb # [ 57.878628] niks3-server[1013]: 2026/09/21 13:54:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4383worker2 # [ 57.861542] nix-grpc-daemon[879]: time=2026-09-21T13:54:38.499Z level=INFO msg="Uploading 1 narinfos"4384lb # [ 57.889333] niks3-server[1013]: 2026/09/21 13:54:38 INFO Signed narinfos id=7 count=14385lb # [ 57.896220] niks3-server[1013]: 2026/09/21 13:54:38 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4386lb # [ 57.904335] niks3-server[1013]: 2026/09/21 13:54:38 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4387worker2 # [ 57.885440] nix-grpc-daemon[879]: time=2026-09-21T13:54:38.523Z level=INFO msg="Upload complete. (484ms)"4388lb # [ 57.910315] niks3-server[1013]: 2026/09/21 13:54:38 INFO Completed upload id=74389lb # [ 57.912821] niks3-server[1013]: 2026/09/21 13:54:38 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4390worker2 # [ 57.892131] nix-grpc-daemon[778]: ts=2026-09-21T13:54:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_in=7524391worker2 # [ 57.900883] nix-grpc-daemon[778]: ts=2026-09-21T13:54:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824392worker2 # [ 57.910807] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4393worker2 # [ 57.915526] nix-daemon[864]: reaped child process 1025, status = succeeded4394client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4395client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4396worker2 # [ 57.928657] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4397client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4398client # farm-top> LOG-farm-top-t1b4399worker2 # [ 59.347493] nix-daemon[864]: reaped child process 1039, status = succeeded4400worker2 # [ 59.508425] nix-daemon[864]: accepted connection from pid 1055, user nix-grpc-daemon (trusted)4401worker2 # [ 59.526510] nix-daemon[864]: reaped child process 1058, status = succeeded4402lb # [ 59.563488] niks3-server[1013]: 2026/09/21 13:54:40 INFO Received uploads request method=POST path=/api/pending_closures4403worker2 # [ 59.550973] nix-grpc-daemon[879]: time=2026-09-21T13:54:40.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4404worker2 # [ 59.552512] nix-grpc-daemon[879]: time=2026-09-21T13:54:40.186Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4405worker2 # [ 59.587278] nix-grpc-daemon[879]: time=2026-09-21T13:54:40.225Z level=INFO msg="Uploading 1 narinfos"4406lb # [ 59.602221] niks3-server[1013]: 2026/09/21 13:54:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4407lb # [ 59.616984] niks3-server[1013]: 2026/09/21 13:54:40 INFO Signed narinfos id=8 count=14408lb # [ 59.624833] niks3-server[1013]: 2026/09/21 13:54:40 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4409lb # [ 59.635182] niks3-server[1013]: 2026/09/21 13:54:40 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4410worker2 # [ 59.615385] nix-grpc-daemon[879]: time=2026-09-21T13:54:40.253Z level=INFO msg="Upload complete. (277ms)"4411lb # [ 59.640670] niks3-server[1013]: 2026/09/21 13:54:40 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4412lb # [ 59.646861] niks3-server[1013]: 2026/09/21 13:54:40 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4413lb # [ 59.651095] niks3-server[1013]: 2026/09/21 13:54:40 INFO Completed upload id=84414worker2 # [ 59.624854] nix-grpc-daemon[778]: ts=2026-09-21T13:54:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14415lb # [ 59.654010] niks3-server[1013]: 2026/09/21 13:54:40 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4416worker2 # [ 59.638103] nix-daemon[864]: reaped child process 1037, status = succeeded4417client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.70 seconds)4418client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4419client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds)4420lb: must succeed: curl -sf localhost:9901/clusters4421lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4422lb: must succeed: curl -sf localhost:9901/clusters4423lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4424worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4425worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4426(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.32 seconds)4427subtest: two clients wanting the same drv build it once4428client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4429client # [ 60.167607] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4430client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4431client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4432client # Running as unit: dedup1.service; invocation ID: a170b4f13631493ab9ef27abd12f00064433client # [ 60.431540] nix[938]: don't know how to build these paths:4434worker2 # [ 60.431911] nix-grpc-daemon[778]: ts=2026-09-21T13:54:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531824435client # [ 60.433713] nix[938]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4436worker2 # [ 60.446769] nix-grpc-daemon[778]: ts=2026-09-21T13:54:41Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04437worker2 # [ 60.452764] nix-grpc-daemon[778]: ts=2026-09-21T13:54:41Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14438worker1 # [ 60.496959] nix-grpc-daemon[772]: ts=2026-09-21T13:54:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404184439worker1 # [ 60.515103] nix-grpc-daemon[772]: ts=2026-09-21T13:54:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404184440client # [ 60.524197] nix[938]: copying 1 paths to 'lb:50051'...4441worker1 # [ 60.529494] nix-grpc-daemon[772]: ts=2026-09-21T13:54:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:404184442client # [ 60.526529] nix[938]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4443worker1 # [ 60.543490] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4444worker1 # [ 60.576552] nix-grpc-daemon[772]: ts=2026-09-21T13:54:41Z level=info event=niks3_push_started pid=8654445worker1 # [ 60.840910] nix-grpc-daemon[865]: time=2026-09-21T13:54:41.479Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""4446worker1 # [ 61.030058] nix-daemon[838]: accepted connection from pid 878, user nix-grpc-daemon (trusted)4447worker1 # [ 61.048006] nix-daemon[838]: reaped child process 881, status = succeeded4448worker1 # [ 61.218071] nix-daemon[838]: accepted connection from pid 883, user nix-grpc-daemon (trusted)4449worker1 # [ 61.234816] nix-daemon[838]: reaped child process 886, status = succeeded4450lb # [ 61.271078] niks3-server[1013]: 2026/09/21 13:54:41 INFO Received uploads request method=POST path=/api/pending_closures4451worker1 # [ 61.259613] nix-grpc-daemon[865]: time=2026-09-21T13:54:41.899Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4452worker1 # [ 61.262883] nix-grpc-daemon[865]: time=2026-09-21T13:54:41.901Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4453lb # [ 61.309041] niks3-server[1013]: 2026/09/21 13:54:41 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4454client # [ 61.282161] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4455lb # [ 61.320133] niks3-server[1013]: 2026/09/21 13:54:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4456client # Running as unit: dedup2.service; invocation ID: f51f319524c3479c845f4f50a312ab9d4457worker1 # [ 61.304969] nix-grpc-daemon[865]: time=2026-09-21T13:54:41.944Z level=INFO msg="Uploading 1 narinfos"4458lb # [ 61.333093] niks3-server[1013]: 2026/09/21 13:54:41 INFO Signed narinfos id=9 count=14459lb # [ 61.340609] niks3-server[1013]: 2026/09/21 13:54:41 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4460client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4461lb # [ 61.354281] niks3-server[1013]: 2026/09/21 13:54:41 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4462lb # [ 61.360405] niks3-server[1013]: 2026/09/21 13:54:42 INFO Completed upload id=94463worker1 # [ 61.338752] nix-grpc-daemon[865]: time=2026-09-21T13:54:41.977Z level=INFO msg="Upload complete. (489ms)"4464lb # [ 61.364637] niks3-server[1013]: 2026/09/21 13:54:42 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4465worker1 # [ 61.348072] nix-grpc-daemon[772]: ts=2026-09-21T13:54:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=0 paths=1 nar_bytes_in=4324466worker1 # [ 61.351471] nix-grpc-daemon[772]: ts=2026-09-21T13:54:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404184467worker1 # [ 61.367627] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4468worker1 # [ 61.373160] nix-daemon[838]: reaped child process 863, status = succeeded4469client # [ 61.380258] nix[938]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4470worker1 # [ 61.388489] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4471worker2 # [ 61.667631] nix-grpc-daemon[778]: ts=2026-09-21T13:54:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531824472client # [ 61.672051] nix[954]: don't know how to build these paths:4473client # [ 61.673020] nix[954]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4474worker2 # [ 61.684142] nix-grpc-daemon[778]: ts=2026-09-21T13:54:42Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04475worker1 # [ 61.693043] nix-grpc-daemon[772]: ts=2026-09-21T13:54:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404184476client # [ 61.694918] nix[954]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4477lb # [ 61.886793] envoy[903]: conn peer=[2001:db8:1::1]:55410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9592 tx=4122 ms=78554478lb # [ 61.892506] envoy[903]: conn peer=[2001:db8:1::1]:51888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=481 ms=644479lb # [ 61.898181] envoy[903]: conn peer=[2001:db8:1::1]:51898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4215 tx=1827 ms=24034480client: must succeed: sleep 24481client: (finished: must succeed: sleep 2, in 2.06 seconds)4482client: waiting for success: ! systemctl is-active dedup1 dedup24483worker1 # [ 65.037050] nix-daemon[838]: reaped child process 894, status = succeeded4484worker1 # [ 65.212747] nix-daemon[838]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4485worker1 # [ 65.228951] nix-daemon[838]: reaped child process 923, status = succeeded4486lb # [ 65.266652] niks3-server[1013]: 2026/09/21 13:54:45 INFO Received uploads request method=POST path=/api/pending_closures4487worker1 # [ 65.252553] nix-grpc-daemon[865]: time=2026-09-21T13:54:45.891Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4488worker1 # [ 65.254419] nix-grpc-daemon[865]: time=2026-09-21T13:54:45.891Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4489lb # [ 65.317550] niks3-server[1013]: 2026/09/21 13:54:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4490worker1 # [ 65.306367] nix-grpc-daemon[865]: time=2026-09-21T13:54:45.945Z level=INFO msg="Uploading 1 narinfos"4491lb # [ 65.330502] niks3-server[1013]: 2026/09/21 13:54:45 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4492lb # [ 65.337990] niks3-server[1013]: 2026/09/21 13:54:45 INFO Signed narinfos id=10 count=14493lb # [ 65.342821] niks3-server[1013]: 2026/09/21 13:54:45 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4494worker1 # [ 65.328485] nix-grpc-daemon[865]: time=2026-09-21T13:54:45.968Z level=INFO msg="Upload complete. (327ms)"4495lb # [ 65.352782] niks3-server[1013]: 2026/09/21 13:54:45 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4496lb # [ 65.358574] niks3-server[1013]: 2026/09/21 13:54:45 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4497lb # [ 65.363164] niks3-server[1013]: 2026/09/21 13:54:45 INFO Completed upload id=104498worker1 # [ 65.340803] nix-grpc-daemon[772]: ts=2026-09-21T13:54:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14499lb # [ 65.365958] niks3-server[1013]: 2026/09/21 13:54:46 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4500worker1 # [ 65.351921] nix-grpc-daemon[772]: ts=2026-09-21T13:54:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14501worker1 # [ 65.357870] nix-daemon[838]: reaped child process 892, status = succeeded4502client # [ 65.379354] systemd[1]: dedup2.service: Deactivated successfully.4503client # [ 65.386084] systemd[1]: dedup2.service: Consumed 227ms CPU time over 4.101s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4504client # [ 65.430769] systemd[1]: dedup1.service: Deactivated successfully.4505client # [ 65.433256] systemd[1]: dedup1.service: Consumed 173ms CPU time over 5.265s wall clock time, 41.4M memory peak, 7.4K incoming IP traffic, 8.7K outgoing IP traffic.4506client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4507client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4508client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4509worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4510worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4511worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4512worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4513worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4514worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4515worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4516worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4517(finished: subtest: two clients wanting the same drv build it once, in 6.29 seconds)4518subtest: build hook: nix-daemon with builders = grpc://lb4519client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4520client # [ 66.613049] systemd[1]: Started Nix Daemon.4521client # [ 66.796565] nix-daemon[996]: accepted connection from pid 993, user root (trusted)4522worker1 # [ 67.017791] nix-grpc-daemon[772]: ts=2026-09-21T13:54:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:404184523worker1 # [ 67.024821] nix-grpc-daemon[772]: ts=2026-09-21T13:54:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404184524worker2 # [ 67.035837] nix-grpc-daemon[778]: ts=2026-09-21T13:54:47Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04525worker2 # [ 67.038561] nix-grpc-daemon[778]: ts=2026-09-21T13:54:47Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14526worker2 # [ 67.079953] nix-grpc-daemon[778]: ts=2026-09-21T13:54:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824527worker2 # [ 67.095435] nix-grpc-daemon[778]: ts=2026-09-21T13:54:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824528worker2 # [ 67.109836] nix-grpc-daemon[778]: ts=2026-09-21T13:54:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531824529worker2 # [ 67.125026] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4530worker2 # [ 67.327999] nix-daemon[864]: accepted connection from pid 1094, user nix-grpc-daemon (trusted)4531worker2 # [ 67.347558] nix-daemon[864]: reaped child process 1097, status = succeeded4532worker2 # [ 67.519319] nix-daemon[864]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)4533worker2 # [ 67.535741] nix-daemon[864]: reaped child process 1102, status = succeeded4534lb # [ 67.573150] niks3-server[1013]: 2026/09/21 13:54:48 INFO Received uploads request method=POST path=/api/pending_closures4535worker2 # [ 67.558753] nix-grpc-daemon[879]: time=2026-09-21T13:54:48.196Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4536worker2 # [ 67.560569] nix-grpc-daemon[879]: time=2026-09-21T13:54:48.196Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4537lb # [ 67.603311] niks3-server[1013]: 2026/09/21 13:54:48 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4538worker2 # [ 67.589958] nix-grpc-daemon[879]: time=2026-09-21T13:54:48.228Z level=INFO msg="Uploading 1 narinfos"4539lb # [ 67.617480] niks3-server[1013]: 2026/09/21 13:54:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4540lb # [ 67.627143] niks3-server[1013]: 2026/09/21 13:54:48 INFO Signed narinfos id=11 count=14541lb # [ 67.630653] niks3-server[1013]: 2026/09/21 13:54:48 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4542lb # [ 67.637741] niks3-server[1013]: 2026/09/21 13:54:48 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4543worker2 # [ 67.618118] nix-grpc-daemon[879]: time=2026-09-21T13:54:48.256Z level=INFO msg="Upload complete. (476ms)"4544lb # [ 67.643604] niks3-server[1013]: 2026/09/21 13:54:48 INFO Completed upload id=114545lb # [ 67.647482] niks3-server[1013]: 2026/09/21 13:54:48 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4546worker2 # [ 67.624326] nix-grpc-daemon[778]: ts=2026-09-21T13:54:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_in=5124547worker2 # [ 67.635881] nix-grpc-daemon[778]: ts=2026-09-21T13:54:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824548worker2 # [ 67.639324] nix-daemon[864]: reaped child process 1092, status = succeeded4549worker2 # [ 67.647083] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4550worker2 # [ 67.658468] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4551worker2 # [ 68.905270] nix-daemon[864]: reaped child process 1106, status = succeeded4552worker2 # [ 69.084204] nix-daemon[864]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4553worker2 # [ 69.102081] nix-daemon[864]: reaped child process 1125, status = succeeded4554lb # [ 69.138785] niks3-server[1013]: 2026/09/21 13:54:49 INFO Received uploads request method=POST path=/api/pending_closures4555worker2 # [ 69.124077] nix-grpc-daemon[879]: time=2026-09-21T13:54:49.762Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4556worker2 # [ 69.127062] nix-grpc-daemon[879]: time=2026-09-21T13:54:49.764Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4557lb # [ 69.175456] niks3-server[1013]: 2026/09/21 13:54:49 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4558lb # [ 69.188951] niks3-server[1013]: 2026/09/21 13:54:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4559worker2 # [ 69.169834] nix-grpc-daemon[879]: time=2026-09-21T13:54:49.808Z level=INFO msg="Uploading 1 narinfos"4560lb # [ 69.200279] niks3-server[1013]: 2026/09/21 13:54:49 INFO Signed narinfos id=12 count=14561lb # [ 69.202776] niks3-server[1013]: 2026/09/21 13:54:49 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4562lb # [ 69.210400] niks3-server[1013]: 2026/09/21 13:54:49 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4563lb # [ 69.217199] niks3-server[1013]: 2026/09/21 13:54:49 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4564worker2 # [ 69.193973] nix-grpc-daemon[879]: time=2026-09-21T13:54:49.832Z level=INFO msg="Upload complete. (322ms)"4565lb # [ 69.221779] niks3-server[1013]: 2026/09/21 13:54:49 INFO Completed upload id=124566lb # [ 69.224673] niks3-server[1013]: 2026/09/21 13:54:49 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4567worker2 # [ 69.205718] nix-grpc-daemon[778]: ts=2026-09-21T13:54:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14568worker2 # [ 69.223260] nix-grpc-daemon[778]: ts=2026-09-21T13:54:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:531824569worker2 # [ 69.227867] nix-daemon[864]: reaped child process 1104, status = succeeded4570worker2 # [ 69.241950] nix-grpc-daemon[778]: ts=2026-09-21T13:54:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:531824571worker2 # [ 69.247572] nix-grpc-daemon[778]: ts=2026-09-21T13:54:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_out=1284572worker2 # [ 69.556805] nix-grpc-daemon[778]: ts=2026-09-21T13:54:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:531824573worker2 # [ 69.564094] nix-grpc-daemon[778]: ts=2026-09-21T13:54:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824574worker2 # [ 69.571337] nix-grpc-daemon[879]: time=2026-09-21T13:54:50.208Z level=INFO msg="All 1 paths already cached"4575worker2 # [ 69.582531] nix-grpc-daemon[778]: ts=2026-09-21T13:54:50Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04576worker2 # [ 69.589488] nix-grpc-daemon[778]: ts=2026-09-21T13:54:50Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14577worker2 # [ 69.598352] nix-grpc-daemon[778]: ts=2026-09-21T13:54:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824578worker2 # [ 69.612588] nix-grpc-daemon[778]: ts=2026-09-21T13:54:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824579worker2 # [ 69.626524] nix-grpc-daemon[879]: time=2026-09-21T13:54:50.264Z level=INFO msg="All 1 paths already cached"4580worker2 # [ 69.634620] nix-grpc-daemon[778]: ts=2026-09-21T13:54:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531824581worker2 # [ 69.648466] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4582worker2 # [ 69.848288] nix-daemon[864]: accepted connection from pid 1131, user nix-grpc-daemon (trusted)4583worker2 # [ 69.865590] nix-daemon[864]: reaped child process 1134, status = succeeded4584worker2 # [ 70.033661] nix-daemon[864]: accepted connection from pid 1136, user nix-grpc-daemon (trusted)4585worker2 # [ 70.048928] nix-daemon[864]: reaped child process 1139, status = succeeded4586lb # [ 70.084383] niks3-server[1013]: 2026/09/21 13:54:50 INFO Received uploads request method=POST path=/api/pending_closures4587worker2 # [ 70.069357] nix-grpc-daemon[879]: time=2026-09-21T13:54:50.707Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4588worker2 # [ 70.070807] nix-grpc-daemon[879]: time=2026-09-21T13:54:50.707Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4589lb # [ 70.113359] niks3-server[1013]: 2026/09/21 13:54:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4590worker2 # [ 70.095701] nix-grpc-daemon[879]: time=2026-09-21T13:54:50.734Z level=INFO msg="Uploading 1 narinfos"4591lb # [ 70.127485] niks3-server[1013]: 2026/09/21 13:54:50 INFO Signed narinfos id=13 count=14592lb # [ 70.135184] niks3-server[1013]: 2026/09/21 13:54:50 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4593worker2 # [ 70.125296] nix-grpc-daemon[879]: time=2026-09-21T13:54:50.763Z level=INFO msg="Upload complete. (460ms)"4594lb # [ 70.150540] niks3-server[1013]: 2026/09/21 13:54:50 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4595lb # [ 70.155094] niks3-server[1013]: 2026/09/21 13:54:50 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4596worker2 # [ 70.131496] nix-grpc-daemon[778]: ts=2026-09-21T13:54:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_in=6244597lb # [ 70.160246] niks3-server[1013]: 2026/09/21 13:54:50 INFO Completed upload id=134598lb # [ 70.162411] niks3-server[1013]: 2026/09/21 13:54:50 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4599worker2 # [ 70.142637] nix-grpc-daemon[778]: ts=2026-09-21T13:54:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824600worker2 # [ 70.147184] nix-daemon[864]: reaped child process 1129, status = succeeded4601worker2 # [ 70.153467] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4602worker2 # [ 70.165330] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4603worker2 # [ 71.604774] nix-daemon[864]: reaped child process 1143, status = succeeded4604worker2 # [ 71.784524] nix-daemon[864]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)4605worker2 # [ 71.803110] nix-daemon[864]: reaped child process 1162, status = succeeded4606lb # [ 71.839777] niks3-server[1013]: 2026/09/21 13:54:52 INFO Received uploads request method=POST path=/api/pending_closures4607worker2 # [ 71.825824] nix-grpc-daemon[879]: time=2026-09-21T13:54:52.463Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4608worker2 # [ 71.827518] nix-grpc-daemon[879]: time=2026-09-21T13:54:52.463Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4609lb # [ 71.876785] niks3-server[1013]: 2026/09/21 13:54:52 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4610lb # [ 71.890964] niks3-server[1013]: 2026/09/21 13:54:52 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4611worker2 # [ 71.876893] nix-grpc-daemon[879]: time=2026-09-21T13:54:52.514Z level=INFO msg="Uploading 1 narinfos"4612lb # [ 71.904575] niks3-server[1013]: 2026/09/21 13:54:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4613lb # [ 71.911496] niks3-server[1013]: 2026/09/21 13:54:52 INFO Signed narinfos id=14 count=14614lb # [ 71.918315] niks3-server[1013]: 2026/09/21 13:54:52 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4615worker2 # [ 71.897614] nix-grpc-daemon[879]: time=2026-09-21T13:54:52.536Z level=INFO msg="Upload complete. (322ms)"4616lb # [ 71.923700] envoy[903]: conn peer=[2001:db8:1::1]:54402 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1082 ms=36934617lb # [ 71.928206] envoy[903]: conn peer=[2001:db8:1::1]:51908 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3035 tx=1468 ms=49754618lb # [ 71.931873] envoy[903]: conn peer=[2001:db8:1::1]:54414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3413 tx=1742 ms=22894619worker2 # [ 71.909135] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14620lb # [ 71.936240] envoy[903]: conn peer=[2001:db8:1::1]:54424 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=664621lb # [ 71.941461] niks3-server[1013]: 2026/09/21 13:54:52 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4622lb # [ 71.945032] niks3-server[1013]: 2026/09/21 13:54:52 INFO Completed upload id=144623lb # [ 71.947608] niks3-server[1013]: 2026/09/21 13:54:52 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4624worker2 # [ 71.927139] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:531824625worker2 # [ 71.932616] nix-daemon[864]: reaped child process 1141, status = succeeded4626worker2 # [ 71.947224] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:531824627worker2 # [ 71.951592] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_out=1764628worker2 # [ 72.268853] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:531824629worker1 # [ 72.275669] nix-grpc-daemon[772]: ts=2026-09-21T13:54:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404184630worker2 # [ 72.311697] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04631worker2 # [ 72.321100] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14632worker2 # [ 72.329779] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824633worker2 # [ 72.338666] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824634worker2 # [ 72.349666] nix-grpc-daemon[879]: time=2026-09-21T13:54:52.988Z level=INFO msg="All 2 paths already cached"4635worker2 # [ 72.358932] nix-grpc-daemon[778]: ts=2026-09-21T13:54:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531824636worker2 # [ 72.373497] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4637worker2 # [ 72.566440] nix-daemon[864]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)4638worker2 # [ 72.585622] nix-daemon[864]: reaped child process 1171, status = succeeded4639worker2 # [ 72.758408] nix-daemon[864]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)4640worker2 # [ 72.774035] nix-daemon[864]: reaped child process 1176, status = succeeded4641lb # [ 72.814072] niks3-server[1013]: 2026/09/21 13:54:53 INFO Received uploads request method=POST path=/api/pending_closures4642worker2 # [ 72.799084] nix-grpc-daemon[879]: time=2026-09-21T13:54:53.437Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4643worker2 # [ 72.800718] nix-grpc-daemon[879]: time=2026-09-21T13:54:53.437Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4644lb # [ 72.841049] niks3-server[1013]: 2026/09/21 13:54:53 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4645worker2 # [ 72.829843] nix-grpc-daemon[879]: time=2026-09-21T13:54:53.467Z level=INFO msg="Uploading 1 narinfos"4646lb # [ 72.857065] niks3-server[1013]: 2026/09/21 13:54:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4647lb # [ 72.868575] niks3-server[1013]: 2026/09/21 13:54:53 INFO Signed narinfos id=15 count=14648lb # [ 72.871323] niks3-server[1013]: 2026/09/21 13:54:53 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4649lb # [ 72.878531] niks3-server[1013]: 2026/09/21 13:54:53 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4650lb # [ 72.884283] niks3-server[1013]: 2026/09/21 13:54:53 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4651lb # [ 72.889288] niks3-server[1013]: 2026/09/21 13:54:53 INFO Completed upload id=154652worker2 # [ 72.865242] nix-grpc-daemon[879]: time=2026-09-21T13:54:53.503Z level=INFO msg="Upload complete. (476ms)"4653worker2 # [ 72.873009] nix-grpc-daemon[778]: ts=2026-09-21T13:54:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_in=7524654worker2 # [ 72.885279] nix-grpc-daemon[778]: ts=2026-09-21T13:54:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824655worker2 # [ 72.891201] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4656worker2 # [ 72.895845] nix-daemon[864]: reaped child process 1166, status = succeeded4657worker2 # [ 72.908238] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4658worker2 # [ 73.954422] nix-daemon[864]: reaped child process 1180, status = succeeded4659worker2 # [ 74.135389] nix-daemon[864]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)4660worker2 # [ 74.151666] nix-daemon[864]: reaped child process 1199, status = succeeded4661lb # [ 74.191664] niks3-server[1013]: 2026/09/21 13:54:54 INFO Received uploads request method=POST path=/api/pending_closures4662worker2 # [ 74.180445] nix-grpc-daemon[879]: time=2026-09-21T13:54:54.815Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4663worker2 # [ 74.182925] nix-grpc-daemon[879]: time=2026-09-21T13:54:54.815Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4664lb # [ 74.228457] niks3-server[1013]: 2026/09/21 13:54:54 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4665lb # [ 74.240279] niks3-server[1013]: 2026/09/21 13:54:54 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4666lb # [ 74.251703] niks3-server[1013]: 2026/09/21 13:54:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4667worker2 # [ 74.234339] nix-grpc-daemon[879]: time=2026-09-21T13:54:54.872Z level=INFO msg="Uploading 1 narinfos"4668lb # [ 74.258568] niks3-server[1013]: 2026/09/21 13:54:54 INFO Signed narinfos id=16 count=14669lb # [ 74.263162] niks3-server[1013]: 2026/09/21 13:54:54 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4670lb # [ 74.278193] niks3-server[1013]: 2026/09/21 13:54:54 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4671lb # [ 74.283331] niks3-server[1013]: 2026/09/21 13:54:54 INFO Completed upload id=164672worker2 # [ 74.259286] nix-grpc-daemon[879]: time=2026-09-21T13:54:54.897Z level=INFO msg="Upload complete. (340ms)"4673lb # [ 74.286306] niks3-server[1013]: 2026/09/21 13:54:54 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4674worker2 # [ 74.273083] nix-grpc-daemon[778]: ts=2026-09-21T13:54:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14675worker2 # [ 74.287555] nix-daemon[864]: reaped child process 1178, status = succeeded4676worker1 # [ 74.318853] nix-grpc-daemon[772]: ts=2026-09-21T13:54:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:404184677worker1 # [ 74.335813] nix-grpc-daemon[772]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4678worker1 # [ 74.356602] nix-grpc-daemon[772]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4679worker1 # [ 74.370845] nix-grpc-daemon[772]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4680worker1 # [ 74.408105] nix-grpc-daemon[772]: ts=2026-09-21T13:54:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:404184681worker1 # [ 74.414855] nix-grpc-daemon[772]: ts=2026-09-21T13:54:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=0 paths=1 nar_bytes_out=2324682client # [ 74.541073] nix-daemon[996]: reaped child process 998, status = succeeded4683client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.17 seconds)4684client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4685client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4686client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4687client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.06 seconds)4688client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4689client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4690(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.31 seconds)4691subtest: an input only one worker has still reaches the builder4692client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4693client # this derivation will be built:4694client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4695client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4696client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.43 seconds)4697client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4698client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4699worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4700worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4701worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4702worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4703client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24704client # [ 75.824436] nix-daemon[996]: accepted connection from pid 1079, user root (trusted)4705client # this derivation will be built:4706client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4707client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4708worker1 # [ 76.042201] nix-grpc-daemon[772]: ts=2026-09-21T13:54:56Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:404184709worker2 # [ 76.044938] nix-grpc-daemon[778]: ts=2026-09-21T13:54:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824710client # copying 1 paths...4711client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4712worker1 # [ 76.062383] nix-grpc-daemon[772]: ts=2026-09-21T13:54:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:404184713worker1 # [ 76.077442] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4714worker1 # [ 76.283052] nix-daemon[838]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4715worker1 # [ 76.300535] nix-daemon[838]: reaped child process 947, status = succeeded4716lb # [ 76.341578] niks3-server[1013]: 2026/09/21 13:54:56 INFO Received uploads request method=POST path=/api/pending_closures4717worker1 # [ 76.328122] nix-grpc-daemon[865]: time=2026-09-21T13:54:56.968Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4718worker1 # [ 76.330046] nix-grpc-daemon[865]: time=2026-09-21T13:54:56.970Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4719lb # [ 76.375439] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4720lb # [ 76.388374] niks3-server[1013]: 2026/09/21 13:54:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4721worker1 # [ 76.372539] nix-grpc-daemon[865]: time=2026-09-21T13:54:57.012Z level=INFO msg="Uploading 1 narinfos"4722lb # [ 76.399634] niks3-server[1013]: 2026/09/21 13:54:57 INFO Signed narinfos id=17 count=14723lb # [ 76.406457] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4724lb # [ 76.413837] niks3-server[1013]: 2026/09/21 13:54:57 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4725worker1 # [ 76.399577] nix-grpc-daemon[865]: time=2026-09-21T13:54:57.039Z level=INFO msg="Upload complete. (307ms)"4726lb # [ 76.422305] niks3-server[1013]: 2026/09/21 13:54:57 INFO Completed upload id=174727lb # [ 76.426483] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4728worker1 # [ 76.408338] nix-grpc-daemon[772]: ts=2026-09-21T13:54:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=0 paths=1 nar_bytes_in=1204729worker1 # [ 76.422896] nix-daemon[838]: reaped child process 942, status = succeeded4730worker2 # [ 76.424430] nix-grpc-daemon[778]: ts=2026-09-21T13:54:57Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04731worker2 # [ 76.430931] nix-grpc-daemon[778]: ts=2026-09-21T13:54:57Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14732worker1 # [ 76.472903] nix-grpc-daemon[772]: ts=2026-09-21T13:54:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404184733worker1 # [ 76.488347] nix-grpc-daemon[772]: ts=2026-09-21T13:54:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404184734client # copying 2 paths to 'lb:50051'...4735client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4736worker1 # [ 76.507478] nix-grpc-daemon[772]: ts=2026-09-21T13:54:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:404184737client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4738worker1 # [ 76.521439] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4739worker1 # [ 76.726733] nix-daemon[838]: accepted connection from pid 951, user nix-grpc-daemon (trusted)4740worker1 # [ 76.744318] nix-daemon[838]: reaped child process 954, status = succeeded4741worker1 # [ 76.916172] nix-daemon[838]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4742worker1 # [ 76.933517] nix-daemon[838]: reaped child process 959, status = succeeded4743lb # [ 76.968147] niks3-server[1013]: 2026/09/21 13:54:57 INFO Received uploads request method=POST path=/api/pending_closures4744lb # [ 76.979009] niks3-server[1013]: 2026/09/21 13:54:57 INFO Received uploads request method=POST path=/api/pending_closures4745worker1 # [ 76.964789] nix-grpc-daemon[865]: time=2026-09-21T13:54:57.604Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4746worker1 # [ 76.966272] nix-grpc-daemon[865]: time=2026-09-21T13:54:57.604Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4747worker1 # [ 76.967959] nix-grpc-daemon[865]: time=2026-09-21T13:54:57.604Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4748lb # [ 77.007972] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4749lb # [ 77.029288] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4750lb # [ 77.039474] niks3-server[1013]: 2026/09/21 13:54:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4751worker1 # [ 77.022352] nix-grpc-daemon[865]: time=2026-09-21T13:54:57.661Z level=INFO msg="Uploading 3 narinfos"4752lb # [ 77.048599] niks3-server[1013]: 2026/09/21 13:54:57 INFO Signed narinfos id=18 count=14753lb # [ 77.050443] niks3-server[1013]: 2026/09/21 13:54:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4754lb # [ 77.058834] niks3-server[1013]: 2026/09/21 13:54:57 INFO Signed narinfos id=19 count=24755lb # [ 77.062725] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4756lb # [ 77.068258] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4757lb # [ 77.076764] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4758lb # [ 77.084694] niks3-server[1013]: 2026/09/21 13:54:57 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4759lb # [ 77.088806] niks3-server[1013]: 2026/09/21 13:54:57 INFO Completed upload id=184760lb # [ 77.090412] niks3-server[1013]: 2026/09/21 13:54:57 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4761lb # [ 77.094115] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4762lb # [ 77.097310] niks3-server[1013]: 2026/09/21 13:54:57 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4763worker1 # [ 77.074413] nix-grpc-daemon[865]: time=2026-09-21T13:54:57.714Z level=INFO msg="Upload complete. (534ms)"4764lb # [ 77.099832] niks3-server[1013]: 2026/09/21 13:54:57 INFO Completed upload id=194765worker1 # [ 77.085276] nix-grpc-daemon[772]: ts=2026-09-21T13:54:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=0 paths=2 nar_bytes_in=9844766worker1 # [ 77.100819] nix-grpc-daemon[772]: ts=2026-09-21T13:54:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404184767worker1 # [ 77.114208] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4768worker1 # [ 77.122900] nix-daemon[838]: reaped child process 949, status = succeeded4769client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4770worker1 # [ 77.130998] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4771client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4772client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4773worker1 # [ 78.767922] nix-daemon[838]: reaped child process 963, status = succeeded4774worker1 # [ 78.938559] nix-daemon[838]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4775worker1 # [ 78.954766] nix-daemon[838]: reaped child process 982, status = succeeded4776lb # [ 78.990792] niks3-server[1013]: 2026/09/21 13:54:59 INFO Received uploads request method=POST path=/api/pending_closures4777worker1 # [ 78.978636] nix-grpc-daemon[865]: time=2026-09-21T13:54:59.617Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4778lb # [ 79.024720] niks3-server[1013]: 2026/09/21 13:54:59 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4779worker1 # [ 79.012085] nix-grpc-daemon[865]: time=2026-09-21T13:54:59.652Z level=INFO msg="Uploading 1 narinfos"4780lb # [ 79.038599] niks3-server[1013]: 2026/09/21 13:54:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4781lb # [ 79.048846] niks3-server[1013]: 2026/09/21 13:54:59 INFO Signed narinfos id=20 count=14782worker1 # [ 79.033640] nix-grpc-daemon[865]: time=2026-09-21T13:54:59.673Z level=INFO msg="Upload complete. (292ms)"4783lb # [ 79.057933] niks3-server[1013]: 2026/09/21 13:54:59 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4784lb # [ 79.064507] niks3-server[1013]: 2026/09/21 13:54:59 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4785lb # [ 79.068431] niks3-server[1013]: 2026/09/21 13:54:59 INFO Completed upload id=204786worker1 # [ 79.044920] nix-grpc-daemon[772]: ts=2026-09-21T13:54:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14787client # copying 1 paths...4788lb # [ 79.071154] niks3-server[1013]: 2026/09/21 13:54:59 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4789worker1 # [ 79.058864] nix-grpc-daemon[772]: ts=2026-09-21T13:54:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:404184790client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4791worker1 # [ 79.062772] nix-daemon[838]: reaped child process 961, status = succeeded4792worker2 # [ 79.086242] nix-grpc-daemon[778]: ts=2026-09-21T13:54:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:531824793worker2 # [ 79.103281] nix-grpc-daemon[778]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4794worker2 # [ 79.125972] nix-grpc-daemon[778]: ts=2026-09-21T13:54:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_out=1204795client # [ 79.256872] nix-daemon[996]: reaped child process 1082, status = succeeded4796client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.66 seconds)4797client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24798client # [ 79.470873] nix-daemon[996]: accepted connection from pid 1101, user root (trusted)4799client # this derivation will be built:4800client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4801worker2 # [ 79.688814] nix-grpc-daemon[778]: ts=2026-09-21T13:55:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:531824802client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4803worker2 # [ 79.696316] nix-grpc-daemon[778]: ts=2026-09-21T13:55:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824804worker2 # [ 79.715516] nix-grpc-daemon[778]: ts=2026-09-21T13:55:00Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04805worker2 # [ 79.723220] nix-grpc-daemon[778]: ts=2026-09-21T13:55:00Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14806worker2 # [ 79.732105] nix-grpc-daemon[778]: ts=2026-09-21T13:55:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824807worker2 # [ 79.742580] nix-grpc-daemon[778]: ts=2026-09-21T13:55:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824808client # copying 1 paths to 'lb:50051'...4809client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4810worker2 # [ 79.763243] nix-grpc-daemon[778]: ts=2026-09-21T13:55:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531824811worker2 # [ 79.778032] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4812worker2 # [ 79.798980] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)4813worker2 # [ 79.824456] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 2/5)4814worker2 # [ 79.858803] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 187 ms (attempt 3/5)4815worker2 # [ 80.053361] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 4/5)4816worker2 # [ 80.113477] nix-grpc-daemon[778]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4817worker2 # [ 80.120284] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4818worker2 # [ 80.127537] nix-grpc-daemon[778]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4819worker2 # [ 80.356152] nix-daemon[864]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)4820worker2 # [ 80.375297] nix-daemon[864]: reaped child process 1219, status = succeeded4821worker2 # [ 80.547169] nix-daemon[864]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)4822worker2 # [ 80.563588] nix-daemon[864]: reaped child process 1224, status = succeeded4823lb # [ 80.601853] niks3-server[1013]: 2026/09/21 13:55:01 INFO Received uploads request method=POST path=/api/pending_closures4824worker2 # [ 80.586159] nix-grpc-daemon[879]: time=2026-09-21T13:55:01.224Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4825worker2 # [ 80.587609] nix-grpc-daemon[879]: time=2026-09-21T13:55:01.224Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4826lb # [ 80.629090] niks3-server[1013]: 2026/09/21 13:55:01 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4827worker2 # [ 80.617075] nix-grpc-daemon[879]: time=2026-09-21T13:55:01.255Z level=INFO msg="Uploading 1 narinfos"4828lb # [ 80.645617] niks3-server[1013]: 2026/09/21 13:55:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4829lb # [ 80.656283] niks3-server[1013]: 2026/09/21 13:55:01 INFO Signed narinfos id=21 count=14830lb # [ 80.658783] niks3-server[1013]: 2026/09/21 13:55:01 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4831lb # [ 80.666475] niks3-server[1013]: 2026/09/21 13:55:01 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4832worker2 # [ 80.647285] nix-grpc-daemon[879]: time=2026-09-21T13:55:01.285Z level=INFO msg="Upload complete. (492ms)"4833lb # [ 80.671867] niks3-server[1013]: 2026/09/21 13:55:01 INFO Completed upload id=214834lb # [ 80.675499] niks3-server[1013]: 2026/09/21 13:55:01 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4835worker2 # [ 80.656626] nix-grpc-daemon[778]: ts=2026-09-21T13:55:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=0 paths=1 nar_bytes_in=5684836worker2 # [ 80.661495] nix-grpc-daemon[778]: ts=2026-09-21T13:55:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531824837worker2 # [ 80.672667] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4838worker2 # [ 80.677394] nix-daemon[864]: reaped child process 1209, status = succeeded4839worker2 # [ 80.685530] nix-grpc-daemon[778]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4840worker2 # [ 80.700203] nix-daemon[864]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4841client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4842client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4843client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4844worker2 # [ 81.743454] nix-daemon[864]: reaped child process 1229, status = succeeded4845lb # [ 81.901443] envoy[903]: conn peer=[2001:db8:1::1]:55760 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=604846lb # [ 81.907219] envoy[903]: conn peer=[2001:db8:1::1]:54428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3886 tx=1933 ms=24474847lb # [ 81.912277] envoy[903]: conn peer=[2001:db8:1::1]:55764 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4369 tx=2321 ms=22064848lb # [ 81.917095] envoy[903]: conn peer=[2001:db8:1::1]:55778 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=684849lb # [ 81.921750] envoy[903]: conn peer=[2001:db8:1::1]:55790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4243 tx=1867 ms=31444850lb # [ 81.926600] envoy[903]: conn peer=[2001:db8:1::1]:55804 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1044851worker2 # [ 81.927852] nix-daemon[864]: accepted connection from pid 1245, user nix-grpc-daemon (trusted)4852worker2 # [ 81.943171] nix-daemon[864]: reaped child process 1248, status = succeeded4853lb # [ 81.983403] niks3-server[1013]: 2026/09/21 13:55:02 INFO Received uploads request method=POST path=/api/pending_closures4854worker2 # [ 81.969987] nix-grpc-daemon[879]: time=2026-09-21T13:55:02.607Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4855lb # [ 82.012988] niks3-server[1013]: 2026/09/21 13:55:02 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4856lb # [ 82.023435] niks3-server[1013]: 2026/09/21 13:55:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4857worker2 # [ 82.005363] nix-grpc-daemon[879]: time=2026-09-21T13:55:02.643Z level=INFO msg="Uploading 1 narinfos"4858lb # [ 82.033711] niks3-server[1013]: 2026/09/21 13:55:02 INFO Signed narinfos id=22 count=14859lb # [ 82.039690] niks3-server[1013]: 2026/09/21 13:55:02 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4860lb # [ 82.048435] niks3-server[1013]: 2026/09/21 13:55:02 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4861lb # [ 82.053097] niks3-server[1013]: 2026/09/21 13:55:02 INFO Completed upload id=224862worker2 # [ 82.028921] nix-grpc-daemon[879]: time=2026-09-21T13:55:02.667Z level=INFO msg="Upload complete. (316ms)"4863lb # [ 82.056145] niks3-server[1013]: 2026/09/21 13:55:02 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4864client # copying 1 paths...4865worker2 # [ 82.039496] nix-grpc-daemon[778]: ts=2026-09-21T13:55:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53182 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14866worker1 # [ 82.047714] nix-grpc-daemon[772]: ts=2026-09-21T13:55:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:404184867worker2 # [ 82.052308] nix-daemon[864]: reaped child process 1226, status = succeeded4868worker1 # [ 82.059961] nix-grpc-daemon[772]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4869client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4870worker1 # [ 82.104173] nix-grpc-daemon[772]: ts=2026-09-21T13:55:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:404184871worker1 # [ 82.111069] nix-grpc-daemon[772]: ts=2026-09-21T13:55:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=0 paths=1 nar_bytes_out=1204872client # [ 82.244434] nix-daemon[996]: reaped child process 1104, status = succeeded4873client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.99 seconds)4874client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24875client # [ 82.471314] nix-daemon[996]: accepted connection from pid 1123, user root (trusted)4876client # this derivation will be built:4877client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4878worker2 # [ 82.731925] nix-grpc-daemon[778]: ts=2026-09-21T13:55:03Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:531824879client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4880worker2 # [ 82.739679] nix-grpc-daemon[778]: ts=2026-09-21T13:55:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531824881worker2 # [ 82.746805] nix-grpc-daemon[879]: time=2026-09-21T13:55:03.384Z level=INFO msg="All 1 paths already cached"4882worker2 # [ 82.760215] nix-grpc-daemon[778]: ts=2026-09-21T13:55:03Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04883worker2 # [ 82.766698] nix-grpc-daemon[778]: ts=2026-09-21T13:55:03Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14884worker1 # [ 82.815144] nix-grpc-daemon[772]: ts=2026-09-21T13:55:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404184885worker1 # [ 82.832986] nix-grpc-daemon[772]: ts=2026-09-21T13:55:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404184886worker1 # [ 82.851087] nix-grpc-daemon[865]: time=2026-09-21T13:55:03.490Z level=INFO msg="All 1 paths already cached"4887client # copying 1 paths to 'lb:50051'...4888client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4889worker1 # [ 82.862350] nix-grpc-daemon[772]: ts=2026-09-21T13:55:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:404184890worker1 # [ 82.878233] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4891worker1 # [ 83.112179] nix-daemon[838]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4892worker1 # [ 83.132132] nix-daemon[838]: reaped child process 990, status = succeeded4893worker1 # [ 83.324435] nix-daemon[838]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4894worker1 # [ 83.345954] nix-daemon[838]: reaped child process 995, status = succeeded4895lb # [ 83.383046] niks3-server[1013]: 2026/09/21 13:55:04 INFO Received uploads request method=POST path=/api/pending_closures4896worker1 # [ 83.370282] nix-grpc-daemon[865]: time=2026-09-21T13:55:04.009Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4897worker1 # [ 83.371752] nix-grpc-daemon[865]: time=2026-09-21T13:55:04.009Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4898lb # [ 83.410229] niks3-server[1013]: 2026/09/21 13:55:04 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4899lb # [ 83.427000] niks3-server[1013]: 2026/09/21 13:55:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4900worker1 # [ 83.410048] nix-grpc-daemon[865]: time=2026-09-21T13:55:04.050Z level=INFO msg="Uploading 1 narinfos"4901lb # [ 83.439149] niks3-server[1013]: 2026/09/21 13:55:04 INFO Signed narinfos id=23 count=14902lb # [ 83.445938] niks3-server[1013]: 2026/09/21 13:55:04 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4903lb # [ 83.454953] niks3-server[1013]: 2026/09/21 13:55:04 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4904lb # [ 83.460172] niks3-server[1013]: 2026/09/21 13:55:04 INFO Completed upload id=234905worker1 # [ 83.437400] nix-grpc-daemon[865]: time=2026-09-21T13:55:04.077Z level=INFO msg="Upload complete. (539ms)"4906lb # [ 83.463430] niks3-server[1013]: 2026/09/21 13:55:04 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4907worker1 # [ 83.444815] nix-grpc-daemon[772]: ts=2026-09-21T13:55:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=0 paths=1 nar_bytes_in=5684908worker1 # [ 83.459514] nix-grpc-daemon[772]: ts=2026-09-21T13:55:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404184909worker1 # [ 83.467261] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4910worker1 # [ 83.472872] nix-daemon[838]: reaped child process 985, status = succeeded4911client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4912worker1 # [ 83.486691] nix-daemon[838]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4913client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4914client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4915worker1 # [ 84.730618] nix-daemon[838]: reaped child process 999, status = succeeded4916worker1 # [ 84.915972] nix-daemon[838]: accepted connection from pid 1015, user nix-grpc-daemon (trusted)4917worker1 # [ 84.931577] nix-daemon[838]: reaped child process 1018, status = succeeded4918lb # [ 84.970262] niks3-server[1013]: 2026/09/21 13:55:05 INFO Received uploads request method=POST path=/api/pending_closures4919worker1 # [ 84.957662] nix-grpc-daemon[865]: time=2026-09-21T13:55:05.596Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4920lb # [ 85.005390] niks3-server[1013]: 2026/09/21 13:55:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4921worker1 # [ 84.990021] nix-grpc-daemon[865]: time=2026-09-21T13:55:05.629Z level=INFO msg="Uploading 1 narinfos"4922lb # [ 85.017539] niks3-server[1013]: 2026/09/21 13:55:05 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4923lb # [ 85.028382] niks3-server[1013]: 2026/09/21 13:55:05 INFO Signed narinfos id=24 count=14924lb # [ 85.030815] niks3-server[1013]: 2026/09/21 13:55:05 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4925lb # [ 85.039401] niks3-server[1013]: 2026/09/21 13:55:05 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4926lb # [ 85.046412] niks3-server[1013]: 2026/09/21 13:55:05 INFO Completed upload id=244927worker1 # [ 85.023191] nix-grpc-daemon[865]: time=2026-09-21T13:55:05.663Z level=INFO msg="Upload complete. (326ms)"4928lb # [ 85.049025] niks3-server[1013]: 2026/09/21 13:55:05 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4929client # copying 1 paths...4930worker1 # [ 85.035631] nix-grpc-daemon[772]: ts=2026-09-21T13:55:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14931worker2 # [ 85.038884] nix-grpc-daemon[778]: ts=2026-09-21T13:55:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:531824932worker1 # [ 85.051747] nix-daemon[838]: reaped child process 997, status = succeeded4933worker2 # [ 85.051604] nix-grpc-daemon[778]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4934client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4935worker1 # [ 85.097682] nix-grpc-daemon[772]: ts=2026-09-21T13:55:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:404184936worker1 # [ 85.104289] nix-grpc-daemon[772]: ts=2026-09-21T13:55:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40418 duration_s=0 paths=1 nar_bytes_out=1204937client # [ 85.230144] nix-daemon[996]: reaped child process 1126, status = succeeded4938client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.98 seconds)4939(finished: subtest: an input only one worker has still reaches the builder, in 10.54 seconds)4940subtest: upload whose reference the worker lacks is completed from the cache4941client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4942client # this derivation will be built:4943client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4944client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4945client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.45 seconds)4946client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4947client # this derivation will be built:4948client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4949client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4950client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds)4951client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4952client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4953worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4954worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4955worker2 # [ 86.655344] systemd[1]: Stopping Nix worker-protocol over gRPC...4956worker2 # [ 87.345653] nix-grpc-daemon[778]: ts=2026-09-21T13:55:07Z level=info event=signal_exit4957lb # [ 87.434250] niks3-server[1013]: 2026/09/21 13:55:08 INFO lead: released remote=192.168.1.4:438124958worker2 # [ 88.406988] nix-grpc-daemon[778]: ts=2026-09-21T13:55:09Z level=info event=scheduler_restarting4959worker1 # [ 88.410136] nix-grpc-daemon[772]: ts=2026-09-21T13:55:09Z level=info event=scheduler_restarting addr=lb:500514960worker2 # [ 88.419023] nix-grpc-daemon[778]: ts=2026-09-21T13:55:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:531724961worker2 # [ 88.425034] nix-grpc-daemon[778]: ts=2026-09-21T13:55:09Z level=info event=worker_gone addr=192.168.1.3:500514962worker2 # [ 88.429513] nix-grpc-daemon[778]: ts=2026-09-21T13:55:09Z level=info event=scheduler_restarting addr=lb:500514963worker2 # [ 88.434114] nix-grpc-daemon[778]: ts=2026-09-21T13:55:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:531724964worker2 # [ 88.438911] nix-grpc-daemon[778]: ts=2026-09-21T13:55:09Z level=info event=worker_gone addr=192.168.1.4:500514965worker2 # [ 88.451262] nix-grpc-daemon[778]: E0000 00:00:1789998909.089404 823 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4966worker2 # [ 88.472342] nix-grpc-daemon[778]: E0000 00:00:1789998909.110701 823 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true4967worker2 # [ 88.480743] nix-daemon[864]: reaped child process 866, status = succeeded4968worker1 # [ 88.610403] nix-grpc-daemon[772]: ts=2026-09-21T13:55:09Z level=info event=scheduler_connected addr=lb:500514969worker1 # [ 88.615872] nix-grpc-daemon[772]: ts=2026-09-21T13:55:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4970worker2 # [ 88.641718] nix-grpc-daemon[778]: ts=2026-09-21T13:55:09Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4971worker1 # [ 88.816421] nix-grpc-daemon[772]: ts=2026-09-21T13:55:09Z level=info event=scheduler_connected addr=lb:500514972worker1 # [ 88.821763] nix-grpc-daemon[772]: ts=2026-09-21T13:55:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4973worker1 # [ 89.022617] nix-grpc-daemon[772]: ts=2026-09-21T13:55:09Z level=info event=scheduler_connected addr=lb:500514974worker1 # [ 89.027879] nix-grpc-daemon[772]: ts=2026-09-21T13:55:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4975worker1 # [ 89.228711] nix-grpc-daemon[772]: ts=2026-09-21T13:55:09Z level=info event=scheduler_connected addr=lb:500514976worker1 # [ 89.233762] nix-grpc-daemon[772]: ts=2026-09-21T13:55:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4977worker1 # [ 89.436302] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_connected addr=lb:500514978worker1 # [ 89.440445] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4979worker1 # [ 89.636817] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_connected addr=lb:500514980worker1 # [ 89.642029] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4981worker1 # [ 89.842853] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_connected addr=lb:500514982worker1 # [ 89.848205] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4983worker1 # [ 90.049030] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_connected addr=lb:500514984worker1 # [ 90.054253] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4985worker1 # [ 90.255110] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_connected addr=lb:500514986worker1 # [ 90.260744] nix-grpc-daemon[772]: ts=2026-09-21T13:55:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4987worker1 # [ 90.416761] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_take_over4988lb # [ 90.440571] niks3-server[1013]: 2026/09/21 13:55:11 INFO lead: acquired remote=192.168.1.3:605904989worker2 # [ 90.447639] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4990worker2 # [ 90.453155] systemd[1]: Stopped Nix worker-protocol over gRPC.4991worker2 # [ 90.455719] systemd[1]: nix-grpc-daemon.service: Consumed 3.653s CPU time over 1min 8.435s wall clock time, 62.9M memory peak, 99.5K incoming IP traffic, 268.1K outgoing IP traffic.4992worker1 # [ 90.463224] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_connected addr=lb:500514993worker1 # [ 90.467779] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4994worker2 # [ 90.479499] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4995worker2 # [ 90.485206] systemd[1]: Closed Nix worker-protocol over gRPC.4996lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4997lb: must succeed: curl -sf localhost:9901/clusters4998worker2 # [ 90.489457] systemd[1]: nix-grpc-daemon.socket: Consumed 773us CPU time over 1min 16.609s wall clock time, 256K memory peak, 97.3K incoming IP traffic, 77.7K outgoing IP traffic.4999lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5000lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5001client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25002worker1 # [ 90.663923] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_connected addr=lb:500515003worker1 # [ 90.669881] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5004worker1 # [ 90.834082] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:404185005worker1 # [ 90.843225] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:404185006client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5007worker1 # [ 90.869919] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_connected addr=lb:500515008client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.31 seconds)5009worker1 # [ 90.875144] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5010worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5011worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)5012worker2 # [ 90.982800] systemd[1]: Listening on Nix worker-protocol over gRPC.5013worker1 # [ 91.062401] systemd[1]: Stopping Nix worker-protocol over gRPC...5014worker1 # [ 91.077056] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_connected addr=lb:500515015worker1 # [ 91.081768] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5016worker1 # [ 91.204618] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=signal_exit5017worker1 # [ 91.282544] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_connected addr=lb:500515018worker1 # [ 91.287736] nix-grpc-daemon[772]: ts=2026-09-21T13:55:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5019worker2 # [ 91.433616] systemd[1]: Starting scheduler-token.service...5020lb # [ 91.500986] niks3-server[1013]: 2026/09/21 13:55:12 INFO lead: released remote=192.168.1.3:605905021worker1 # [ 91.490368] nix-grpc-daemon[772]: ts=2026-09-21T13:55:12Z level=info event=scheduler_connected addr=lb:500515022worker1 # [ 91.494506] nix-grpc-daemon[772]: ts=2026-09-21T13:55:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5023worker2 # [ 91.532238] systemd[1]: scheduler-token.service: Deactivated successfully.5024worker2 # [ 91.539936] systemd[1]: Finished scheduler-token.service.5025worker2 # [ 91.545023] systemd[1]: scheduler-token.service: Consumed 48ms CPU time over 102ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5026worker2 # [ 91.550458] systemd[1]: Starting Nix worker-protocol over gRPC...5027worker1 # [ 91.691054] nix-grpc-daemon[772]: ts=2026-09-21T13:55:12Z level=info event=scheduler_connected addr=lb:500515028worker1 # [ 91.696196] nix-grpc-daemon[772]: ts=2026-09-21T13:55:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5029worker2 # [ 91.736481] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5030worker2 # [ 91.739219] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5031worker2 # [ 91.743364] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15032worker2 # [ 91.752066] nix-grpc-daemon[1275]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5033worker2 # [ 91.753502] nix-grpc-daemon[1275]: E0000 00:00:1789998912.388480 1275 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5034worker2 # [ 91.764964] nix-grpc-daemon[1275]: E0000 00:00:1789998912.394951 1275 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5035worker2 # [ 91.769799] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5036worker2 # [ 91.773109] systemd[1]: Started Nix worker-protocol over gRPC.5037worker2 # [ 91.774275] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=scheduler_take_over5038lb # [ 91.798407] niks3-server[1013]: 2026/09/21 13:55:12 INFO lead: acquired remote=192.168.1.4:593505039worker2 # [ 91.778527] nix-grpc-daemon[1275]: E0000 00:00:1789998912.416749 1314 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5040worker2 # [ 91.808151] nix-grpc-daemon[1275]: E0000 00:00:1789998912.435558 1314 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5041worker2 # [ 91.810397] nix-grpc-daemon[1275]: E0000 00:00:1789998912.435703 1314 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5042worker2 # [ 91.817271] nix-grpc-daemon[1275]: E0000 00:00:1789998912.455032 1314 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5043worker2 # [ 91.829609] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=scheduler_connected addr=lb:500515044worker2 # [ 91.837083] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=oidc_jwks provider=mock keys=15045worker2 # [ 91.838389] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:560385046worker2 # [ 91.840254] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05047lb # [ 91.904981] envoy[903]: conn peer=[2001:db8:1::1]:55816 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3739 tx=2015 ms=24795048lb # [ 91.910635] envoy[903]: conn peer=[2001:db8:1::1]:42638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=705049lb # [ 91.915542] envoy[903]: conn peer=[2001:db8:1::1]:42652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=715050worker1 # [ 91.897044] nix-grpc-daemon[772]: ts=2026-09-21T13:55:12Z level=info event=scheduler_connected addr=lb:500515051worker2 # [ 91.897626] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:560385052lb # [ 91.921624] envoy[903]: conn peer=[2001:db8:1::1]:42648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3755 tx=2005 ms=24335053worker2 # [ 91.900611] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05054lb # [ 91.926563] envoy[903]: conn peer=[2001:db8:1::4]:56962 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=73451 tx=3929 ms=662365055lb # [ 91.931846] envoy[903]: conn peer=[2001:db8:1::1]:42666 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=305056worker1 # [ 92.470405] nix-grpc-daemon[772]: ts=2026-09-21T13:55:13Z level=info event=scheduler_restarting5057worker2 # [ 92.479655] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:560385058worker2 # [ 92.484295] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:13Z level=info event=worker_gone addr=192.168.1.3:500515059worker1 # [ 92.513318] nix-grpc-daemon[772]: ts=2026-09-21T13:55:13Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5060worker1 # [ 92.528533] nix-daemon[838]: reaped child process 840, status = succeeded5061worker1 # [ 93.311175] nix-grpc-daemon[772]: E0000 00:00:1789998913.951098 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5062worker1 # [ 93.340200] nix-grpc-daemon[772]: E0000 00:00:1789998913.980223 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5063worker1 # [ 94.379120] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5064worker1 # [ 94.385611] systemd[1]: Stopped Nix worker-protocol over gRPC.5065worker1 # [ 94.390954] systemd[1]: nix-grpc-daemon.service: Consumed 1.996s CPU time over 1min 12.513s wall clock time, 51.8M memory peak, 63.3K incoming IP traffic, 114.8K outgoing IP traffic.5066worker1 # [ 94.405451] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5067worker1 # [ 94.407379] systemd[1]: Closed Nix worker-protocol over gRPC.5068worker1 # [ 94.409651] systemd[1]: nix-grpc-daemon.socket: Consumed 616us CPU time over 1min 20.458s wall clock time, 256K memory peak, 66.9K incoming IP traffic, 58.6K outgoing IP traffic.5069lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5070lb: must succeed: curl -sf localhost:9901/clusters5071lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5072lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5073client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25074worker2 # [ 94.762232] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560525075worker2 # [ 94.777907] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5076worker2 # [ 94.811323] nix-grpc-daemon[1275]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 1/5)5077worker2 # [ 94.829464] nix-grpc-daemon[1275]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 2/5)5078worker2 # [ 94.923875] nix-grpc-daemon[1275]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 3/5)5079worker2 # [ 94.950455] nix-grpc-daemon[1275]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 279 ms (attempt 4/5)5080worker2 # [ 95.237269] nix-grpc-daemon[1275]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5081worker2 # [ 95.243095] nix-grpc-daemon[1275]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5082client # copying 2 paths...5083client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5084worker2 # [ 95.267788] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:560525085client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5086worker2 # [ 95.284668] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5087worker2 # [ 95.314484] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:15Z level=info event=niks3_push_started pid=13355088worker2 # [ 95.528569] nix-grpc-daemon[1335]: time=2026-09-21T13:55:16.166Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5089worker2 # [ 95.699189] nix-daemon[864]: accepted connection from pid 1348, user nix-grpc-daemon (trusted)5090worker2 # [ 95.715295] nix-daemon[864]: reaped child process 1351, status = succeeded5091lb # [ 95.757490] niks3-server[1013]: 2026/09/21 13:55:16 INFO Received uploads request method=POST path=/api/pending_closures5092lb # [ 95.766333] niks3-server[1013]: 2026/09/21 13:55:16 INFO Received uploads request method=POST path=/api/pending_closures5093worker2 # [ 95.752068] nix-grpc-daemon[1335]: time=2026-09-21T13:55:16.389Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5094worker2 # [ 95.755351] nix-grpc-daemon[1335]: time=2026-09-21T13:55:16.389Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5095worker2 # [ 95.757384] nix-grpc-daemon[1335]: time=2026-09-21T13:55:16.389Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5096lb # [ 95.819632] niks3-server[1013]: 2026/09/21 13:55:16 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5097lb # [ 95.853135] niks3-server[1013]: 2026/09/21 13:55:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5098lb # [ 95.862370] niks3-server[1013]: 2026/09/21 13:55:16 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5099lb # [ 95.869823] niks3-server[1013]: 2026/09/21 13:55:16 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5100lb # [ 95.873650] niks3-server[1013]: 2026/09/21 13:55:16 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5101lb # [ 95.878700] niks3-server[1013]: 2026/09/21 13:55:16 INFO Signed narinfos id=26 count=25102worker2 # [ 95.852119] nix-grpc-daemon[1335]: time=2026-09-21T13:55:16.488Z level=INFO msg="Uploading 3 narinfos"5103lb # [ 95.881386] niks3-server[1013]: 2026/09/21 13:55:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5104lb # [ 95.883509] niks3-server[1013]: 2026/09/21 13:55:16 INFO Signed narinfos id=25 count=15105lb # [ 95.903118] niks3-server[1013]: 2026/09/21 13:55:16 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5106lb # [ 95.912379] niks3-server[1013]: 2026/09/21 13:55:16 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5107lb # [ 95.917140] niks3-server[1013]: 2026/09/21 13:55:16 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5108lb # [ 95.920809] niks3-server[1013]: 2026/09/21 13:55:16 INFO Completed upload id=255109lb # [ 95.922113] niks3-server[1013]: 2026/09/21 13:55:16 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5110lb # [ 95.925449] niks3-server[1013]: 2026/09/21 13:55:16 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5111lb # [ 95.927686] niks3-server[1013]: 2026/09/21 13:55:16 INFO Completed upload id=265112worker2 # [ 95.903740] nix-grpc-daemon[1335]: time=2026-09-21T13:55:16.542Z level=INFO msg="Upload complete. (371ms)"5113worker2 # [ 95.913158] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56052 duration_s=0 paths=2 nar_bytes_in=3125114worker2 # [ 95.931557] nix-daemon[864]: reaped child process 1333, status = succeeded5115client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.46 seconds)5116worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5117worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5118worker1 # [ 96.076861] systemd[1]: Listening on Nix worker-protocol over gRPC.5119worker1 # [ 96.085133] systemd[1]: Starting Nix worker-protocol over gRPC...5120worker1 # [ 96.287835] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5121worker1 # [ 96.290514] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5122worker1 # [ 96.294840] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15123worker1 # [ 96.302298] nix-grpc-daemon[1041]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5124worker1 # [ 96.303725] nix-grpc-daemon[1041]: E0000 00:00:1789998916.941916 1041 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5125worker1 # [ 96.315158] nix-grpc-daemon[1041]: E0000 00:00:1789998916.948267 1041 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5126worker1 # [ 96.318250] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5127worker1 # [ 96.322336] systemd[1]: Started Nix worker-protocol over gRPC.5128worker1 # [ 96.336126] nix-grpc-daemon[1041]: E0000 00:00:1789998916.964941 1083 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5129worker1 # [ 96.355665] nix-grpc-daemon[1041]: E0000 00:00:1789998916.982609 1083 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5130lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5131lb: must succeed: curl -sf localhost:9901/clusters5132worker1 # [ 96.365059] nix-grpc-daemon[1041]: E0000 00:00:1789998916.982776 1083 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5133worker1 # [ 96.367523] nix-grpc-daemon[1041]: E0000 00:00:1789998917.003251 1083 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5134worker1 # [ 96.396093] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:17Z level=info event=scheduler_connected addr=lb:500515135worker2 # [ 96.398547] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:17Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:560385136worker2 # [ 96.404243] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05137lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.12 seconds)5138lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.12 seconds)5139(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.24 seconds)5140subtest: a niks3 restart does not cost the leader its role5141lb: must succeed: curl -sf localhost:9901/clusters5142lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5143lb: must succeed: curl -sf localhost:9901/clusters5144lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5145lb # [ 96.642890] niks3-server[1013]: 2026/09/21 13:55:17 INFO Shutdown signal received, draining in-flight requests timeout=10s5146worker1 # [ 96.623888] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:17Z level=info event=scheduler_lock_lost status=2005147worker2 # [ 96.623347] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:17Z level=info event=scheduler_lock_lost status=2005148lb # [ 96.646857] systemd[1]: Stopping niks3 server...5149lb # [ 96.647114] niks3-server[1013]: 2026/09/21 13:55:17 INFO lead: released remote=192.168.1.4:593505150lb # [ 96.667860] systemd[1]: niks3.service: Deactivated successfully.5151lb # [ 96.673250] systemd[1]: Stopped niks3 server.5152lb # [ 96.673943] systemd[1]: niks3.service: Consumed 507ms CPU time over 1min 6.567s wall clock time, 22.1M memory peak, 74.2K incoming IP traffic, 105.2K outgoing IP traffic.5153lb # [ 96.682545] systemd[1]: Starting niks3 server...5154lb # [ 96.756658] niks3-server[1149]: 2026/09/21 13:55:17 goose: no migrations to run. current version: 202609200000005155lb # [ 96.763316] niks3-server[1149]: 2026/09/21 13:55:17 OK 1_commit_pending_closure.sql (7.22ms)5156lb # [ 96.767057] niks3-server[1149]: 2026/09/21 13:55:17 OK 2_object_stats_trigger.sql (3.39ms)5157lb # [ 96.768624] niks3-server[1149]: 2026/09/21 13:55:17 goose: up to current file version: 25158lb # [ 96.770475] niks3-server[1149]: 2026/09/21 13:55:17 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key5159lb # [ 96.782034] niks3-server[1149]: 2026/09/21 13:55:17 INFO Read proxy enabled — serving cache objects from S35160lb # [ 96.783359] niks3-server[1149]: 2026/09/21 13:55:17 INFO Using socket-activated listener address=0.0.0.0:57515161lb # [ 96.787460] niks3-server[1149]: 2026/09/21 13:55:17 INFO systemd watchdog enabled interval=15s5162lb # [ 96.788843] niks3-server[1149]: 2026/09/21 13:55:17 INFO Starting HTTP server address=0.0.0.0:57515163lb # [ 96.790007] systemd[1]: Started niks3 server.5164lb: waiting for unit niks3.service5165lb: (finished: waiting for unit niks3.service, in 0.04 seconds)5166lb: must succeed: curl -sf localhost:9901/clusters5167lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5168worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5169worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5170lb: must succeed: curl -sf localhost:9901/clusters5171lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5172lb: must succeed: curl -sf localhost:9901/clusters5173lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5174worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5175worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5176client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&25177worker2 # [ 97.373690] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:560525178client # don't know how to build these paths:5179client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5180worker1 # [ 97.390438] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333925181worker1 # [ 97.405923] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5182worker1 # [ 97.437401] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 1/5)5183worker1 # [ 97.520039] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 178 ms (attempt 2/5)5184lb # [ 97.653801] niks3-server[1149]: 2026/09/21 13:55:18 INFO lead: acquired remote=192.168.1.4:593965185worker1 # [ 97.705465] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 3/5)5186worker1 # [ 97.855975] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 4/5)5187worker1 # [ 97.926266] nix-grpc-daemon[1041]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5188worker1 # [ 97.932825] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5189worker2 # [ 97.977248] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:18Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05190worker2 # [ 97.985696] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:18Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15191worker2 # [ 98.024858] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560525192worker2 # [ 98.042651] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560525193client # copying 1 paths to 'lb:50051'...5194client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5195worker2 # [ 98.057323] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:560525196worker2 # [ 98.073855] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5197worker2 # [ 98.277003] nix-daemon[864]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)5198worker2 # [ 98.295706] nix-daemon[864]: reaped child process 1373, status = succeeded5199worker2 # [ 98.464535] nix-daemon[864]: accepted connection from pid 1375, user nix-grpc-daemon (trusted)5200worker2 # [ 98.481936] nix-daemon[864]: reaped child process 1378, status = succeeded5201lb # [ 98.519470] niks3-server[1149]: 2026/09/21 13:55:19 INFO Received uploads request method=POST path=/api/pending_closures5202worker2 # [ 98.510070] nix-grpc-daemon[1335]: time=2026-09-21T13:55:19.148Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5203worker2 # [ 98.511697] nix-grpc-daemon[1335]: time=2026-09-21T13:55:19.148Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5204lb # [ 98.563482] niks3-server[1149]: 2026/09/21 13:55:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5205lb # [ 98.580147] niks3-server[1149]: 2026/09/21 13:55:19 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5206lb # [ 98.605430] niks3-server[1149]: 2026/09/21 13:55:19 INFO Signed narinfos id=27 count=15207worker2 # [ 98.584157] nix-grpc-daemon[1335]: time=2026-09-21T13:55:19.222Z level=INFO msg="Uploading 1 narinfos"5208lb # [ 98.619729] niks3-server[1149]: 2026/09/21 13:55:19 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5209lb # [ 98.625209] niks3-server[1149]: 2026/09/21 13:55:19 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5210lb # [ 98.631425] niks3-server[1149]: 2026/09/21 13:55:19 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5211lb # [ 98.636404] niks3-server[1149]: 2026/09/21 13:55:19 INFO Completed upload id=275212worker2 # [ 98.612357] nix-grpc-daemon[1335]: time=2026-09-21T13:55:19.250Z level=INFO msg="Upload complete. (523ms)"5213worker2 # [ 98.625158] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56052 duration_s=0 paths=1 nar_bytes_in=5445214worker2 # [ 98.640465] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560525215worker2 # [ 98.649034] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5216worker2 # [ 98.654543] nix-daemon[864]: reaped child process 1368, status = succeeded5217client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5218worker2 # [ 98.666720] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5219client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5220client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5221client # farm-a> LOG-farm-a-after-niks3-restart5222worker2 # [ 100.116164] nix-daemon[864]: reaped child process 1382, status = succeeded5223worker2 # [ 100.288959] nix-daemon[864]: accepted connection from pid 1398, user nix-grpc-daemon (trusted)5224worker2 # [ 100.305987] nix-daemon[864]: reaped child process 1401, status = succeeded5225lb # [ 100.342043] niks3-server[1149]: 2026/09/21 13:55:20 INFO Received uploads request method=POST path=/api/pending_closures5226worker2 # [ 100.332438] nix-grpc-daemon[1335]: time=2026-09-21T13:55:20.969Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5227worker2 # [ 100.334278] nix-grpc-daemon[1335]: time=2026-09-21T13:55:20.969Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5228lb # [ 100.384397] niks3-server[1149]: 2026/09/21 13:55:21 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5229lb # [ 100.399465] niks3-server[1149]: 2026/09/21 13:55:21 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5230lb # [ 100.410647] niks3-server[1149]: 2026/09/21 13:55:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5231worker2 # [ 100.392189] nix-grpc-daemon[1335]: time=2026-09-21T13:55:21.029Z level=INFO msg="Uploading 1 narinfos"5232lb # [ 100.417823] niks3-server[1149]: 2026/09/21 13:55:21 INFO Signed narinfos id=28 count=15233lb # [ 100.421806] niks3-server[1149]: 2026/09/21 13:55:21 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5234lb # [ 100.431387] niks3-server[1149]: 2026/09/21 13:55:21 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5235lb # [ 100.444821] niks3-server[1149]: 2026/09/21 13:55:21 INFO Completed upload id=285236worker2 # [ 100.420656] nix-grpc-daemon[1335]: time=2026-09-21T13:55:21.059Z level=INFO msg="Upload complete. (338ms)"5237lb # [ 100.447609] niks3-server[1149]: 2026/09/21 13:55:21 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5238worker2 # [ 100.438968] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56052 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15239worker2 # [ 100.460812] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:21Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05240worker2 # [ 100.465748] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:21Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15241worker2 # [ 100.470670] nix-daemon[864]: reaped child process 1380, status = succeeded5242worker2 # [ 100.500342] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560525243worker2 # [ 100.515024] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560525244worker2 # [ 100.529354] nix-grpc-daemon[1335]: time=2026-09-21T13:55:21.167Z level=INFO msg="All 1 paths already cached"5245client # copying 1 paths to 'lb:50051'...5246client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5247worker2 # [ 100.539381] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:560525248worker2 # [ 100.553728] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5249worker2 # [ 100.753780] nix-daemon[864]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5250worker2 # [ 100.773579] nix-daemon[864]: reaped child process 1410, status = succeeded5251worker2 # [ 100.944082] nix-daemon[864]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5252worker2 # [ 100.960805] nix-daemon[864]: reaped child process 1415, status = succeeded5253lb # [ 100.995898] niks3-server[1149]: 2026/09/21 13:55:21 INFO Received uploads request method=POST path=/api/pending_closures5254worker2 # [ 100.981220] nix-grpc-daemon[1335]: time=2026-09-21T13:55:21.619Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5255worker2 # [ 100.982723] nix-grpc-daemon[1335]: time=2026-09-21T13:55:21.619Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5256lb # [ 101.023192] niks3-server[1149]: 2026/09/21 13:55:21 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5257worker2 # [ 101.011440] nix-grpc-daemon[1335]: time=2026-09-21T13:55:21.649Z level=INFO msg="Uploading 1 narinfos"5258lb # [ 101.038440] niks3-server[1149]: 2026/09/21 13:55:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5259lb # [ 101.050337] niks3-server[1149]: 2026/09/21 13:55:21 INFO Signed narinfos id=29 count=15260lb # [ 101.055136] niks3-server[1149]: 2026/09/21 13:55:21 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5261lb # [ 101.063521] niks3-server[1149]: 2026/09/21 13:55:21 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5262lb # [ 101.068088] niks3-server[1149]: 2026/09/21 13:55:21 INFO Completed upload id=295263worker2 # [ 101.043620] nix-grpc-daemon[1335]: time=2026-09-21T13:55:21.682Z level=INFO msg="Upload complete. (473ms)"5264lb # [ 101.070733] niks3-server[1149]: 2026/09/21 13:55:21 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5265worker2 # [ 101.052351] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56052 duration_s=0 paths=1 nar_bytes_in=6565266worker2 # [ 101.063402] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560525267worker2 # [ 101.072815] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5268worker2 # [ 101.075516] nix-daemon[864]: reaped child process 1405, status = succeeded5269client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5270worker2 # [ 101.088173] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5271client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5272client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5273client # farm-b> LOG-farm-b-after-niks3-restart5274worker2 # [ 101.709549] nix-daemon[864]: reaped child process 1419, status = succeeded5275worker2 # [ 101.872096] nix-daemon[864]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5276lb # [ 101.905458] envoy[903]: conn peer=[2001:db8:1::3]:33404 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=16114 tx=4120 ms=702355277worker2 # [ 101.890323] nix-daemon[864]: reaped child process 1438, status = succeeded5278lb # [ 101.911325] envoy[903]: conn peer=[2001:db8:1::1]:40842 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=305 ms=11675279lb # [ 101.927587] niks3-server[1149]: 2026/09/21 13:55:22 INFO Received uploads request method=POST path=/api/pending_closures5280worker2 # [ 101.913438] nix-grpc-daemon[1335]: time=2026-09-21T13:55:22.550Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5281worker2 # [ 101.915146] nix-grpc-daemon[1335]: time=2026-09-21T13:55:22.551Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5282lb # [ 101.967187] niks3-server[1149]: 2026/09/21 13:55:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5283worker2 # [ 101.950327] nix-grpc-daemon[1335]: time=2026-09-21T13:55:22.588Z level=INFO msg="Uploading 1 narinfos"5284lb # [ 101.983152] niks3-server[1149]: 2026/09/21 13:55:22 INFO Signed narinfos id=30 count=15285lb # [ 101.989035] niks3-server[1149]: 2026/09/21 13:55:22 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5286lb # [ 102.001798] niks3-server[1149]: 2026/09/21 13:55:22 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5287lb # [ 102.006892] niks3-server[1149]: 2026/09/21 13:55:22 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5288worker2 # [ 101.983530] nix-grpc-daemon[1335]: time=2026-09-21T13:55:22.620Z level=INFO msg="Upload complete. (285ms)"5289lb # [ 102.011540] niks3-server[1149]: 2026/09/21 13:55:22 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5290lb # [ 102.015357] niks3-server[1149]: 2026/09/21 13:55:22 INFO Completed upload id=305291lb # [ 102.017239] niks3-server[1149]: 2026/09/21 13:55:22 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5292worker2 # [ 101.993846] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56052 duration_s=0 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15293worker2 # [ 102.009062] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:22Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05294worker2 # [ 102.014139] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:22Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15295worker2 # [ 102.018436] nix-daemon[864]: reaped child process 1417, status = succeeded5296worker2 # [ 102.056380] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560525297worker2 # [ 102.070451] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560525298worker2 # [ 102.085734] nix-grpc-daemon[1335]: time=2026-09-21T13:55:22.724Z level=INFO msg="All 2 paths already cached"5299client # copying 1 paths to 'lb:50051'...5300client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5301worker2 # [ 102.095771] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:560525302worker2 # [ 102.110081] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5303worker2 # [ 102.308109] nix-daemon[864]: accepted connection from pid 1444, user nix-grpc-daemon (trusted)5304worker2 # [ 102.326690] nix-daemon[864]: reaped child process 1447, status = succeeded5305worker2 # [ 102.493358] nix-daemon[864]: accepted connection from pid 1449, user nix-grpc-daemon (trusted)5306worker2 # [ 102.510787] nix-daemon[864]: reaped child process 1452, status = succeeded5307lb # [ 102.546277] niks3-server[1149]: 2026/09/21 13:55:23 INFO Received uploads request method=POST path=/api/pending_closures5308worker2 # [ 102.532621] nix-grpc-daemon[1335]: time=2026-09-21T13:55:23.170Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5309worker2 # [ 102.534761] nix-grpc-daemon[1335]: time=2026-09-21T13:55:23.170Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5310lb # [ 102.577958] niks3-server[1149]: 2026/09/21 13:55:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5311worker2 # [ 102.560466] nix-grpc-daemon[1335]: time=2026-09-21T13:55:23.198Z level=INFO msg="Uploading 1 narinfos"5312lb # [ 102.592267] niks3-server[1149]: 2026/09/21 13:55:23 INFO Signed narinfos id=31 count=15313lb # [ 102.595415] niks3-server[1149]: 2026/09/21 13:55:23 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5314lb # [ 102.610197] niks3-server[1149]: 2026/09/21 13:55:23 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5315worker2 # [ 102.590321] nix-grpc-daemon[1335]: time=2026-09-21T13:55:23.228Z level=INFO msg="Upload complete. (462ms)"5316lb # [ 102.615247] niks3-server[1149]: 2026/09/21 13:55:23 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5317lb # [ 102.619796] niks3-server[1149]: 2026/09/21 13:55:23 INFO Completed upload id=315318worker2 # [ 102.596202] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56052 duration_s=0 paths=1 nar_bytes_in=7845319lb # [ 102.622974] niks3-server[1149]: 2026/09/21 13:55:23 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5320worker2 # [ 102.604643] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560525321worker2 # [ 102.614044] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5322worker2 # [ 102.618147] nix-daemon[864]: reaped child process 1442, status = succeeded5323client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5324worker2 # [ 102.628721] nix-daemon[864]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5325client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5326client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5327client # farm-top> LOG-farm-top-after-niks3-restart5328worker2 # [ 103.877257] nix-daemon[864]: reaped child process 1456, status = succeeded5329worker2 # [ 104.057524] nix-daemon[864]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5330worker2 # [ 104.074142] nix-daemon[864]: reaped child process 1475, status = succeeded5331lb # [ 104.110925] niks3-server[1149]: 2026/09/21 13:55:24 INFO Received uploads request method=POST path=/api/pending_closures5332worker2 # [ 104.097993] nix-grpc-daemon[1335]: time=2026-09-21T13:55:24.734Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5333worker2 # [ 104.099433] nix-grpc-daemon[1335]: time=2026-09-21T13:55:24.734Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5334lb # [ 104.142809] niks3-server[1149]: 2026/09/21 13:55:24 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5335lb # [ 104.158871] niks3-server[1149]: 2026/09/21 13:55:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5336worker2 # [ 104.140810] nix-grpc-daemon[1335]: time=2026-09-21T13:55:24.778Z level=INFO msg="Uploading 1 narinfos"5337lb # [ 104.169582] niks3-server[1149]: 2026/09/21 13:55:24 INFO Signed narinfos id=32 count=15338lb # [ 104.178056] niks3-server[1149]: 2026/09/21 13:55:24 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5339lb # [ 104.184435] niks3-server[1149]: 2026/09/21 13:55:24 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5340lb # [ 104.191909] niks3-server[1149]: 2026/09/21 13:55:24 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5341lb # [ 104.197746] niks3-server[1149]: 2026/09/21 13:55:24 INFO Completed upload id=325342worker2 # [ 104.172911] nix-grpc-daemon[1335]: time=2026-09-21T13:55:24.811Z level=INFO msg="Upload complete. (332ms)"5343lb # [ 104.200793] niks3-server[1149]: 2026/09/21 13:55:24 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5344worker2 # [ 104.189149] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56052 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15345worker2 # [ 104.203242] nix-daemon[864]: reaped child process 1454, status = succeeded5346client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&2, in 7.14 seconds)5347client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath5348client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath, in 0.23 seconds)5349(finished: subtest: a niks3 restart does not cost the leader its role, in 8.00 seconds)5350subtest: scheduler down: the other node takes the lock and keeps it5351lb: must succeed: curl -sf localhost:9901/clusters5352lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5353lb: must succeed: curl -sf localhost:9901/clusters5354lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5355lb: must succeed: curl -sf localhost:9901/clusters5356lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5357lb: must succeed: curl -sf localhost:9901/clusters5358lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5359worker2 # [ 104.777655] systemd[1]: Stopping Nix worker-protocol over gRPC...5360worker2 # [ 105.769343] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:26Z level=info event=signal_exit5361lb # [ 105.863850] niks3-server[1149]: 2026/09/21 13:55:26 INFO lead: released remote=192.168.1.4:593965362worker1 # [ 106.841898] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:27Z level=info event=scheduler_restarting addr=lb:500515363worker2 # [ 106.841716] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:27Z level=info event=scheduler_restarting5364worker2 # [ 106.842145] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:27Z level=info event=scheduler_restarting addr=lb:500515365worker2 # [ 106.842285] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:560385366worker2 # [ 106.842412] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:27Z level=info event=worker_gone addr=192.168.1.4:500515367worker2 # [ 106.842537] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:560385368worker2 # [ 106.842648] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:27Z level=info event=worker_gone addr=192.168.1.3:500515369worker2 # [ 106.892728] nix-daemon[864]: reaped child process 1327, status = succeeded5370worker1 # [ 107.041616] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:27Z level=info event=scheduler_connected addr=lb:500515371worker1 # [ 107.047004] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5372worker2 # [ 107.060634] nix-grpc-daemon[1275]: ts=2026-09-21T13:55:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5373worker1 # [ 107.247700] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:27Z level=info event=scheduler_connected addr=lb:500515374worker1 # [ 107.252674] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5375worker1 # [ 107.453374] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_connected addr=lb:500515376worker1 # [ 107.458451] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5377lb # [ 107.654737] niks3-server[1149]: 2026/09/21 13:55:28 INFO lead: acquired remote=192.168.1.3:509405378worker1 # [ 107.636299] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_take_over5379worker1 # [ 107.659134] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_connected addr=lb:500515380worker1 # [ 107.664273] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5381worker2 # [ 107.808104] nix-grpc-daemon[1275]: E0000 00:00:1789998928.446226 1320 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5382worker2 # [ 107.808516] nix-grpc-daemon[1275]: E0000 00:00:1789998928.446514 1320 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5383worker1 # [ 107.866696] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_connected addr=lb:500515384worker1 # [ 107.870693] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5385worker1 # [ 108.067433] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_connected addr=lb:500515386worker1 # [ 108.072742] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5387worker1 # [ 108.273507] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_connected addr=lb:500515388worker1 # [ 108.278370] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5389worker1 # [ 108.479258] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:29Z level=info event=scheduler_connected addr=lb:500515390worker1 # [ 108.489057] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:353805391worker1 # [ 108.493952] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05392worker2 # [ 108.983069] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5393worker2 # [ 108.989067] systemd[1]: Stopped Nix worker-protocol over gRPC.5394worker2 # [ 108.991613] systemd[1]: nix-grpc-daemon.service: Consumed 1.512s CPU time over 17.437s wall clock time, 36.1M memory peak, 46K incoming IP traffic, 77K outgoing IP traffic.5395worker2 # [ 109.008533] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5396worker2 # [ 109.010738] systemd[1]: Closed Nix worker-protocol over gRPC.5397worker2 # [ 109.013026] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 18.036s wall clock time, 256K memory peak, 38.9K incoming IP traffic, 28.5K outgoing IP traffic.5398worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5399worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5400worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5401worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5402client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25403worker1 # [ 109.423522] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333925404client # don't know how to build these paths:5405client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5406worker1 # [ 109.434171] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333925407worker1 # [ 109.461270] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05408worker1 # [ 109.468250] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15409worker1 # [ 109.509133] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333925410worker1 # [ 109.523087] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333925411client # copying 1 paths to 'lb:50051'...5412client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5413worker1 # [ 109.537823] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333925414worker1 # [ 109.552564] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5415worker1 # [ 109.577979] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=info event=niks3_push_started pid=11145416worker1 # [ 109.799601] nix-grpc-daemon[1114]: time=2026-09-21T13:55:30.439Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5417worker1 # [ 109.972259] nix-daemon[838]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)5418worker1 # [ 109.991200] nix-daemon[838]: reaped child process 1130, status = succeeded5419worker1 # [ 110.164264] nix-daemon[838]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)5420worker1 # [ 110.181798] nix-daemon[838]: reaped child process 1135, status = succeeded5421lb # [ 110.218258] niks3-server[1149]: 2026/09/21 13:55:30 INFO Received uploads request method=POST path=/api/pending_closures5422worker1 # [ 110.207092] nix-grpc-daemon[1114]: time=2026-09-21T13:55:30.846Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5423worker1 # [ 110.208903] nix-grpc-daemon[1114]: time=2026-09-21T13:55:30.846Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5424lb # [ 110.254273] niks3-server[1149]: 2026/09/21 13:55:30 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5425worker1 # [ 110.247169] nix-grpc-daemon[1114]: time=2026-09-21T13:55:30.886Z level=INFO msg="Uploading 1 narinfos"5426lb # [ 110.263568] niks3-server[1149]: 2026/09/21 13:55:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5427lb # [ 110.277909] niks3-server[1149]: 2026/09/21 13:55:30 INFO Signed narinfos id=33 count=15428lb # [ 110.284414] niks3-server[1149]: 2026/09/21 13:55:30 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5429lb # [ 110.292515] niks3-server[1149]: 2026/09/21 13:55:30 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5430lb # [ 110.304308] niks3-server[1149]: 2026/09/21 13:55:30 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5431lb # [ 110.309136] niks3-server[1149]: 2026/09/21 13:55:30 INFO Completed upload id=335432worker1 # [ 110.286425] nix-grpc-daemon[1114]: time=2026-09-21T13:55:30.926Z level=INFO msg="Upload complete. (481ms)"5433worker1 # [ 110.295295] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33392 duration_s=0 paths=1 nar_bytes_in=5205434worker1 # [ 110.308980] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333925435worker1 # [ 110.318184] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5436worker1 # [ 110.321238] nix-daemon[838]: reaped child process 1112, status = succeeded5437client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5438worker1 # [ 110.332407] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5439client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5440client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5441client # farm-a> LOG-farm-a-on-standby5442worker1 # [ 111.379672] nix-daemon[838]: reaped child process 1139, status = succeeded5443worker1 # [ 111.557882] nix-daemon[838]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5444worker1 # [ 111.574775] nix-daemon[838]: reaped child process 1158, status = succeeded5445lb # [ 111.614103] niks3-server[1149]: 2026/09/21 13:55:32 INFO Received uploads request method=POST path=/api/pending_closures5446worker1 # [ 111.603203] nix-grpc-daemon[1114]: time=2026-09-21T13:55:32.243Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5447worker1 # [ 111.608815] nix-grpc-daemon[1114]: time=2026-09-21T13:55:32.245Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5448lb # [ 111.654450] niks3-server[1149]: 2026/09/21 13:55:32 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5449lb # [ 111.669389] niks3-server[1149]: 2026/09/21 13:55:32 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5450lb # [ 111.680451] niks3-server[1149]: 2026/09/21 13:55:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5451worker1 # [ 111.665182] nix-grpc-daemon[1114]: time=2026-09-21T13:55:32.304Z level=INFO msg="Uploading 1 narinfos"5452lb # [ 111.688006] niks3-server[1149]: 2026/09/21 13:55:32 INFO Signed narinfos id=34 count=15453lb # [ 111.696348] niks3-server[1149]: 2026/09/21 13:55:32 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5454lb # [ 111.704279] niks3-server[1149]: 2026/09/21 13:55:32 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5455lb # [ 111.711807] niks3-server[1149]: 2026/09/21 13:55:32 INFO Completed upload id=345456worker1 # [ 111.688857] nix-grpc-daemon[1114]: time=2026-09-21T13:55:32.329Z level=INFO msg="Upload complete. (343ms)"5457lb # [ 111.714517] niks3-server[1149]: 2026/09/21 13:55:32 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5458worker1 # [ 111.711378] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33392 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15459worker1 # [ 111.727520] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:32Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05460worker1 # [ 111.731255] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:32Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15461worker1 # [ 111.735201] nix-daemon[838]: reaped child process 1137, status = succeeded5462worker1 # [ 111.754508] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333925463worker1 # [ 111.763750] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333925464worker1 # [ 111.776993] nix-grpc-daemon[1114]: time=2026-09-21T13:55:32.417Z level=INFO msg="All 1 paths already cached"5465client # copying 1 paths to 'lb:50051'...5466client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5467worker1 # [ 111.785464] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333925468worker1 # [ 111.798672] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5469lb # [ 111.900354] envoy[903]: conn peer=[2001:db8:1::1]:40852 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9740 tx=4180 ms=68265470lb # [ 111.905726] envoy[903]: conn peer=[2001:db8:1::4]:47738 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1691 tx=425 ms=152215471worker1 # [ 111.997344] nix-daemon[838]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)5472worker1 # [ 112.014078] nix-daemon[838]: reaped child process 1167, status = succeeded5473worker1 # [ 112.185325] nix-daemon[838]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5474worker1 # [ 112.201382] nix-daemon[838]: reaped child process 1172, status = succeeded5475lb # [ 112.236079] niks3-server[1149]: 2026/09/21 13:55:32 INFO Received uploads request method=POST path=/api/pending_closures5476worker1 # [ 112.223999] nix-grpc-daemon[1114]: time=2026-09-21T13:55:32.863Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5477worker1 # [ 112.225525] nix-grpc-daemon[1114]: time=2026-09-21T13:55:32.863Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5478lb # [ 112.266636] niks3-server[1149]: 2026/09/21 13:55:32 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5479worker1 # [ 112.260857] nix-grpc-daemon[1114]: time=2026-09-21T13:55:32.900Z level=INFO msg="Uploading 1 narinfos"5480lb # [ 112.277376] niks3-server[1149]: 2026/09/21 13:55:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5481lb # [ 112.289105] niks3-server[1149]: 2026/09/21 13:55:32 INFO Signed narinfos id=35 count=15482lb # [ 112.292000] niks3-server[1149]: 2026/09/21 13:55:32 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5483lb # [ 112.305581] niks3-server[1149]: 2026/09/21 13:55:32 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5484worker1 # [ 112.288159] nix-grpc-daemon[1114]: time=2026-09-21T13:55:32.928Z level=INFO msg="Upload complete. (474ms)"5485lb # [ 112.312108] niks3-server[1149]: 2026/09/21 13:55:32 INFO Completed upload id=355486lb # [ 112.316145] niks3-server[1149]: 2026/09/21 13:55:32 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5487worker1 # [ 112.296305] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33392 duration_s=0 paths=1 nar_bytes_in=6405488worker1 # [ 112.309080] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333925489worker1 # [ 112.318569] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5490worker1 # [ 112.321237] nix-daemon[838]: reaped child process 1162, status = succeeded5491client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5492client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5493worker1 # [ 112.334095] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5494client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5495client # farm-b> LOG-farm-b-on-standby5496worker1 # [ 113.778418] nix-daemon[838]: reaped child process 1176, status = succeeded5497worker1 # [ 113.963662] nix-daemon[838]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)5498worker1 # [ 113.979599] nix-daemon[838]: reaped child process 1195, status = succeeded5499lb # [ 114.018478] niks3-server[1149]: 2026/09/21 13:55:34 INFO Received uploads request method=POST path=/api/pending_closures5500worker1 # [ 114.006380] nix-grpc-daemon[1114]: time=2026-09-21T13:55:34.645Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5501worker1 # [ 114.007897] nix-grpc-daemon[1114]: time=2026-09-21T13:55:34.645Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5502lb # [ 114.048623] niks3-server[1149]: 2026/09/21 13:55:34 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5503lb # [ 114.063442] niks3-server[1149]: 2026/09/21 13:55:34 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5504lb # [ 114.074344] niks3-server[1149]: 2026/09/21 13:55:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5505worker1 # [ 114.057754] nix-grpc-daemon[1114]: time=2026-09-21T13:55:34.697Z level=INFO msg="Uploading 1 narinfos"5506lb # [ 114.082919] niks3-server[1149]: 2026/09/21 13:55:34 INFO Signed narinfos id=36 count=15507lb # [ 114.088254] niks3-server[1149]: 2026/09/21 13:55:34 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5508lb # [ 114.095637] niks3-server[1149]: 2026/09/21 13:55:34 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5509lb # [ 114.102106] niks3-server[1149]: 2026/09/21 13:55:34 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5510lb # [ 114.105790] niks3-server[1149]: 2026/09/21 13:55:34 INFO Completed upload id=365511worker1 # [ 114.083053] nix-grpc-daemon[1114]: time=2026-09-21T13:55:34.723Z level=INFO msg="Upload complete. (338ms)"5512worker1 # [ 114.101103] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33392 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15513worker1 # [ 114.113472] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:34Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05514worker1 # [ 114.117020] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:34Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15515worker1 # [ 114.120707] nix-daemon[838]: reaped child process 1174, status = succeeded5516worker1 # [ 114.141365] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333925517worker1 # [ 114.155500] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333925518worker1 # [ 114.170065] nix-grpc-daemon[1114]: time=2026-09-21T13:55:34.810Z level=INFO msg="All 2 paths already cached"5519client # copying 1 paths to 'lb:50051'...5520client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5521worker1 # [ 114.178953] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333925522worker1 # [ 114.193673] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5523worker1 # [ 114.392300] nix-daemon[838]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5524worker1 # [ 114.408202] nix-daemon[838]: reaped child process 1204, status = succeeded5525worker1 # [ 114.584169] nix-daemon[838]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5526worker1 # [ 114.599401] nix-daemon[838]: reaped child process 1209, status = succeeded5527lb # [ 114.637313] niks3-server[1149]: 2026/09/21 13:55:35 INFO Received uploads request method=POST path=/api/pending_closures5528worker1 # [ 114.623977] nix-grpc-daemon[1114]: time=2026-09-21T13:55:35.263Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5529worker1 # [ 114.625618] nix-grpc-daemon[1114]: time=2026-09-21T13:55:35.263Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5530lb # [ 114.663425] niks3-server[1149]: 2026/09/21 13:55:35 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5531worker1 # [ 114.653360] nix-grpc-daemon[1114]: time=2026-09-21T13:55:35.293Z level=INFO msg="Uploading 1 narinfos"5532lb # [ 114.680244] niks3-server[1149]: 2026/09/21 13:55:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5533lb # [ 114.690377] niks3-server[1149]: 2026/09/21 13:55:35 INFO Signed narinfos id=37 count=15534lb # [ 114.695306] niks3-server[1149]: 2026/09/21 13:55:35 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5535worker1 # [ 114.682041] nix-grpc-daemon[1114]: time=2026-09-21T13:55:35.322Z level=INFO msg="Upload complete. (471ms)"5536lb # [ 114.704950] niks3-server[1149]: 2026/09/21 13:55:35 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5537lb # [ 114.710084] niks3-server[1149]: 2026/09/21 13:55:35 INFO Completed upload id=375538lb # [ 114.713158] niks3-server[1149]: 2026/09/21 13:55:35 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5539worker1 # [ 114.690854] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33392 duration_s=0 paths=1 nar_bytes_in=7685540worker1 # [ 114.701307] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333925541worker1 # [ 114.706737] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5542worker1 # [ 114.710745] nix-daemon[838]: reaped child process 1199, status = succeeded5543client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5544worker1 # [ 114.723060] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5545client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5546client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5547client # farm-top> LOG-farm-top-on-standby5548worker1 # [ 115.766427] nix-daemon[838]: reaped child process 1213, status = succeeded5549worker1 # [ 115.944509] nix-daemon[838]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5550worker1 # [ 115.962941] nix-daemon[838]: reaped child process 1232, status = succeeded5551worker1 # [ 115.990078] nix-grpc-daemon[1114]: time=2026-09-21T13:55:36.628Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5552worker1 # [ 115.991703] nix-grpc-daemon[1114]: time=2026-09-21T13:55:36.628Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5553lb # [ 115.999914] niks3-server[1149]: 2026/09/21 13:55:36 INFO Received uploads request method=POST path=/api/pending_closures5554lb # [ 116.036566] niks3-server[1149]: 2026/09/21 13:55:36 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5555lb # [ 116.049305] niks3-server[1149]: 2026/09/21 13:55:36 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5556lb # [ 116.057812] niks3-server[1149]: 2026/09/21 13:55:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5557worker1 # [ 116.040863] nix-grpc-daemon[1114]: time=2026-09-21T13:55:36.680Z level=INFO msg="Uploading 1 narinfos"5558lb # [ 116.065476] niks3-server[1149]: 2026/09/21 13:55:36 INFO Signed narinfos id=38 count=15559lb # [ 116.071764] niks3-server[1149]: 2026/09/21 13:55:36 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5560lb # [ 116.081199] niks3-server[1149]: 2026/09/21 13:55:36 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5561lb # [ 116.089432] niks3-server[1149]: 2026/09/21 13:55:36 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5562lb # [ 116.094303] niks3-server[1149]: 2026/09/21 13:55:36 INFO Completed upload id=385563worker1 # [ 116.071609] nix-grpc-daemon[1114]: time=2026-09-21T13:55:36.711Z level=INFO msg="Upload complete. (338ms)"5564worker1 # [ 116.085144] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33392 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15565worker1 # [ 116.101423] nix-daemon[838]: reaped child process 1211, status = succeeded5566client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 6.98 seconds)5567client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5568client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5569worker2 # [ 116.461090] systemd[1]: Listening on Nix worker-protocol over gRPC.5570worker2 # [ 116.468881] systemd[1]: Starting scheduler-token.service...5571worker2 # [ 116.559954] systemd[1]: scheduler-token.service: Deactivated successfully.5572worker2 # [ 116.564272] systemd[1]: Finished scheduler-token.service.5573worker2 # [ 116.565368] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5574worker2 # [ 116.571529] systemd[1]: Starting Nix worker-protocol over gRPC...5575worker2 # [ 116.745177] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5576worker2 # [ 116.747725] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5577worker2 # [ 116.751778] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15578worker2 # [ 116.759280] nix-grpc-daemon[1496]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5579worker2 # [ 116.767428] nix-grpc-daemon[1496]: E0000 00:00:1789998937.396932 1496 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5580worker2 # [ 116.772273] nix-grpc-daemon[1496]: E0000 00:00:1789998937.400538 1496 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5581worker2 # [ 116.776562] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5582worker2 # [ 116.778545] systemd[1]: Started Nix worker-protocol over gRPC.5583lb: must succeed: curl -sf localhost:9901/clusters5584worker2 # [ 116.784533] nix-grpc-daemon[1496]: E0000 00:00:1789998937.418977 1536 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5585worker2 # [ 116.808066] nix-grpc-daemon[1496]: E0000 00:00:1789998937.436940 1536 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5586worker2 # [ 116.810369] nix-grpc-daemon[1496]: E0000 00:00:1789998937.437096 1536 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5587worker2 # [ 116.818901] nix-grpc-daemon[1496]: E0000 00:00:1789998937.456581 1536 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5588worker2 # [ 116.834054] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:37Z level=info event=scheduler_connected addr=lb:500515589worker1 # [ 116.844153] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:37Z level=info event=oidc_jwks provider=mock keys=15590worker1 # [ 116.849710] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:353805591worker1 # [ 116.855060] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05592lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5593worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5594worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5595lb: must succeed: curl -sf localhost:9901/clusters5596lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5597lb: must succeed: curl -sf localhost:9901/clusters5598lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5599worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5600worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5601(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.70 seconds)5602subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5603client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5604client # [ 117.240245] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5605client # Running as unit: rs.service; invocation ID: 9bb8a5290e6f4a349696d522b933259b5606client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5607worker2 # [ 117.502198] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:458085608worker2 # [ 117.517771] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)5609client # [ 117.531145] nix[1249]: don't know how to build these paths:5610client # [ 117.532250] nix[1249]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5611worker1 # [ 117.549367] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:38Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05612worker1 # [ 117.556122] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:38Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15613worker1 # [ 117.602934] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333925614worker1 # [ 117.619637] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333925615client # [ 117.626986] nix[1249]: copying 1 paths to 'lb:50051'...5616client # [ 117.632346] nix[1249]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5617worker1 # [ 117.637486] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333925618worker1 # [ 117.652917] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5619worker1 # [ 117.860088] nix-daemon[838]: accepted connection from pid 1245, user nix-grpc-daemon (trusted)5620worker1 # [ 117.880354] nix-daemon[838]: reaped child process 1248, status = succeeded5621worker1 # [ 118.054987] nix-daemon[838]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)5622worker1 # [ 118.071057] nix-daemon[838]: reaped child process 1253, status = succeeded5623lb # [ 118.107263] niks3-server[1149]: 2026/09/21 13:55:38 INFO Received uploads request method=POST path=/api/pending_closures5624worker1 # [ 118.094770] nix-grpc-daemon[1114]: time=2026-09-21T13:55:38.734Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5625worker1 # [ 118.096314] nix-grpc-daemon[1114]: time=2026-09-21T13:55:38.734Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5626lb # [ 118.133728] niks3-server[1149]: 2026/09/21 13:55:38 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5627lb # [ 118.148120] niks3-server[1149]: 2026/09/21 13:55:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5628worker1 # [ 118.131398] nix-grpc-daemon[1114]: time=2026-09-21T13:55:38.771Z level=INFO msg="Uploading 1 narinfos"5629lb # [ 118.159832] niks3-server[1149]: 2026/09/21 13:55:38 INFO Signed narinfos id=39 count=15630lb # [ 118.165743] niks3-server[1149]: 2026/09/21 13:55:38 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5631lb # [ 118.174838] niks3-server[1149]: 2026/09/21 13:55:38 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5632worker1 # [ 118.155830] nix-grpc-daemon[1114]: time=2026-09-21T13:55:38.796Z level=INFO msg="Upload complete. (486ms)"5633lb # [ 118.179181] niks3-server[1149]: 2026/09/21 13:55:38 INFO Completed upload id=395634lb # [ 118.182543] niks3-server[1149]: 2026/09/21 13:55:38 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5635worker1 # [ 118.164533] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33392 duration_s=0 paths=1 nar_bytes_in=4165636worker1 # [ 118.167969] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333925637worker1 # [ 118.178794] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5638worker1 # [ 118.183201] nix-daemon[838]: reaped child process 1243, status = succeeded5639worker1 # [ 118.193594] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5640client # [ 118.187209] nix[1249]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5641lb: must succeed: curl -sf localhost:9901/clusters5642lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5643lb: must succeed: curl -sf localhost:9901/clusters5644lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5645lb: must succeed: curl -sf localhost:9901/clusters5646lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5647lb: must succeed: curl -sf localhost:9901/clusters5648lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5649worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5650worker2 # [ 120.190272] systemd-journald[274]: Received client request to rotate journal, rotating.5651worker2 # [ 120.209406] systemd-journald[274]: Vacuuming done, freed 0B of archived journals from /var/log/journal/3e887629c8544a7bb12de2a04edf3aed.5652worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5653worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5654worker1 # [ 120.270433] systemd[1]: Stopping Nix worker-protocol over gRPC...5655worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5656client: must succeed: sleep 5.05657worker1 # [ 120.320122] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:40Z level=info event=draining5658lb # [ 121.902749] envoy[903]: conn peer=[2001:db8:1::1]:49820 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9606 tx=4072 ms=66715659client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5660worker1 # [ 125.624703] nix-daemon[838]: reaped child process 1257, status = succeeded5661worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5662worker1 # [ 125.782273] nix-daemon[838]: accepted connection from pid 1298, user nix-grpc-daemon (trusted)5663worker1 # [ 125.800486] nix-daemon[838]: reaped child process 1301, status = succeeded5664lb # [ 125.838022] niks3-server[1149]: 2026/09/21 13:55:46 INFO Received uploads request method=POST path=/api/pending_closures5665worker1 # [ 125.823228] nix-grpc-daemon[1114]: time=2026-09-21T13:55:46.463Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5666worker1 # [ 125.827631] nix-grpc-daemon[1114]: time=2026-09-21T13:55:46.465Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5667lb # [ 125.867890] niks3-server[1149]: 2026/09/21 13:55:46 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5668lb # [ 125.885465] niks3-server[1149]: 2026/09/21 13:55:46 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5669lb # [ 125.900655] niks3-server[1149]: 2026/09/21 13:55:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5670worker1 # [ 125.883474] nix-grpc-daemon[1114]: time=2026-09-21T13:55:46.523Z level=INFO msg="Uploading 1 narinfos"5671lb # [ 125.911416] niks3-server[1149]: 2026/09/21 13:55:46 INFO Signed narinfos id=40 count=15672lb # [ 125.918232] niks3-server[1149]: 2026/09/21 13:55:46 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5673lb # [ 125.927381] niks3-server[1149]: 2026/09/21 13:55:46 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5674lb # [ 125.933468] niks3-server[1149]: 2026/09/21 13:55:46 INFO Completed upload id=405675lb # [ 125.936051] niks3-server[1149]: 2026/09/21 13:55:46 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5676worker1 # [ 125.909886] nix-grpc-daemon[1114]: time=2026-09-21T13:55:46.550Z level=INFO msg="Upload complete. (297ms)"5677worker1 # [ 125.920835] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33392 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15678worker1 # [ 125.936923] nix-daemon[838]: reaped child process 1255, status = succeeded5679client # [ 125.959013] systemd[1]: rs.service: Deactivated successfully.5680client # [ 125.964099] systemd[1]: rs.service: Consumed 179ms CPU time over 8.720s wall clock time, 39.2M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.5681worker1 # [ 126.324117] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:46Z level=info event=signal_exit5682lb # [ 126.885264] niks3-server[1149]: 2026/09/21 13:55:47 INFO lead: released remote=192.168.1.3:509405683worker2 # [ 127.861803] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:48Z level=info event=scheduler_restarting addr=lb:500515684worker1 # [ 127.867058] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:48Z level=info event=scheduler_restarting5685worker1 # [ 127.872329] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:48Z level=info event=scheduler_restarting addr=lb:500515686worker1 # [ 127.876868] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:353805687worker1 # [ 127.882030] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:48Z level=info event=worker_gone addr=192.168.1.3:500515688worker1 # [ 127.887336] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:353805689worker1 # [ 127.896534] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:48Z level=info event=worker_gone addr=192.168.1.4:500515690worker1 # [ 127.914094] nix-daemon[838]: reaped child process 1093, status = succeeded5691worker2 # [ 128.061927] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:48Z level=info event=scheduler_connected addr=lb:500515692worker2 # [ 128.067349] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5693worker1 # [ 128.083758] nix-grpc-daemon[1041]: ts=2026-09-21T13:55:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5694worker2 # [ 128.268499] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:48Z level=info event=scheduler_connected addr=lb:500515695worker2 # [ 128.273691] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5696worker1 # [ 128.457176] nix-grpc-daemon[1041]: E0000 00:00:1789998949.097130 1086 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5697worker1 # [ 128.463750] nix-grpc-daemon[1041]: E0000 00:00:1789998949.097348 1086 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5698worker2 # [ 128.474603] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_connected addr=lb:500515699worker2 # [ 128.479739] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5700worker2 # [ 128.680870] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_connected addr=lb:500515701worker2 # [ 128.686000] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5702worker2 # [ 128.891298] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_connected addr=lb:500515703worker2 # [ 128.895797] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5704worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.29 seconds)5705client: waiting for success: ! systemctl is-active rs5706worker1 # [ 129.002723] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5707client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5708client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5709worker1 # [ 129.008771] systemd[1]: Stopped Nix worker-protocol over gRPC.5710worker1 # [ 129.011130] systemd[1]: nix-grpc-daemon.service: Consumed 1.824s CPU time over 32.913s wall clock time, 37.1M memory peak, 46.2K incoming IP traffic, 87.9K outgoing IP traffic.5711worker1 # [ 129.025240] systemd[1]: Starting Nix worker-protocol over gRPC...5712client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5713client: must fail: journalctl -u rs | grep -q 'reconnecting'5714worker2 # [ 129.090624] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_connected addr=lb:500515715client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5716lb: must succeed: curl -sf localhost:9901/clusters5717worker2 # [ 129.095879] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5718lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5719worker1 # [ 129.219253] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5720worker1 # [ 129.221870] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5721worker1 # [ 129.225958] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15722worker1 # [ 129.235828] nix-grpc-daemon[1308]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5723worker1 # [ 129.241144] nix-grpc-daemon[1308]: E0000 00:00:1789998949.873830 1308 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5724worker1 # [ 129.245789] nix-grpc-daemon[1308]: E0000 00:00:1789998949.875927 1308 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5725worker1 # [ 129.248850] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5726lb # [ 129.271256] niks3-server[1149]: 2026/09/21 13:55:49 INFO lead: acquired remote=192.168.1.3:569965727worker1 # [ 129.250795] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:49Z level=info event=scheduler_take_over5728worker1 # [ 129.252194] systemd[1]: Started Nix worker-protocol over gRPC.5729worker1 # [ 129.256065] nix-grpc-daemon[1308]: E0000 00:00:1789998949.896019 1348 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5730worker1 # [ 129.288054] nix-grpc-daemon[1308]: E0000 00:00:1789998949.914584 1348 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5731worker1 # [ 129.290389] nix-grpc-daemon[1308]: E0000 00:00:1789998949.914719 1348 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5732worker1 # [ 129.294657] nix-grpc-daemon[1308]: E0000 00:00:1789998949.934001 1348 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5733worker2 # [ 129.297197] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_connected addr=lb:500515734worker1 # [ 129.307001] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:49Z level=info event=scheduler_connected addr=lb:500515735worker2 # [ 129.302554] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5736worker1 # [ 129.308660] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5737worker2 # [ 129.504267] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:50Z level=info event=scheduler_connected addr=lb:500515738worker1 # [ 129.509985] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=scheduler_connected addr=lb:500515739worker2 # [ 129.510181] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5740worker1 # [ 129.514939] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5741worker2 # [ 129.711422] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:50Z level=info event=scheduler_connected addr=lb:500515742worker1 # [ 129.715666] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=scheduler_connected addr=lb:500515743worker2 # [ 129.717089] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5744worker1 # [ 129.721102] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5745worker2 # [ 129.920445] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:50Z level=info event=scheduler_connected addr=lb:500515746worker1 # [ 129.922007] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=scheduler_connected addr=lb:500515747worker2 # [ 129.924733] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5748worker1 # [ 129.927028] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5749worker2 # [ 130.121093] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:50Z level=info event=scheduler_connected addr=lb:500515750worker1 # [ 130.128850] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=scheduler_connected addr=lb:500515751worker1 # [ 130.134582] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:554585752worker1 # [ 130.140940] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05753worker1 # [ 130.150668] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=oidc_jwks provider=mock keys=15754worker1 # [ 130.154818] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:554585755worker1 # [ 130.160124] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:50Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05756lb: must succeed: curl -sf localhost:9901/clusters5757lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5758worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5759worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5760(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.16 seconds)5761subtest: scheduler restart mid-build: build finishes, no double build5762client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5763client # [ 130.394260] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5764client # Running as unit: mid.service; invocation ID: 3ad05abf839645269151f98505180c6a5765client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5766worker1 # [ 130.655182] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:554745767worker1 # [ 130.669653] nix-daemon[838]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5768client # [ 130.682341] nix[1285]: don't know how to build these paths:5769client # [ 130.683578] nix[1285]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5770worker1 # [ 130.696542] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:51Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05771worker1 # [ 130.703421] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:51Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15772worker1 # [ 130.749609] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554745773worker1 # [ 130.766320] nix-grpc-daemon[1308]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5)5774worker1 # [ 130.777591] nix-grpc-daemon[1308]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 2/5)5775worker1 # [ 130.843827] nix-grpc-daemon[1308]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 168 ms (attempt 3/5)5776worker1 # [ 131.019631] nix-grpc-daemon[1308]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 579 ms (attempt 4/5)5777worker1 # [ 131.599999] nix-grpc-daemon[1308]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5778worker1 # [ 131.610632] nix-grpc-daemon[1308]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5779worker1 # [ 131.629747] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:554745780client # [ 131.635243] nix[1285]: copying 1 paths to 'lb:50051'...5781worker1 # [ 131.640710] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:554745782client # [ 131.635571] nix[1285]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5783worker1 # [ 131.657063] nix-daemon[838]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5784worker1 # [ 131.685657] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:52Z level=info event=niks3_push_started pid=13815785worker1 # [ 131.882830] nix-grpc-daemon[1381]: time=2026-09-21T13:55:52.522Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5786lb # [ 131.906014] envoy[903]: conn peer=[2001:db8:1::1]:40278 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=1429 ms=84345787lb # [ 131.911040] envoy[903]: conn peer=[2001:db8:1::3]:43734 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2413 tx=836 ms=316975788worker1 # [ 132.060081] nix-daemon[838]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5789worker1 # [ 132.075191] nix-daemon[838]: reaped child process 1397, status = succeeded5790worker1 # [ 132.255178] nix-daemon[838]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5791worker1 # [ 132.274100] nix-daemon[838]: reaped child process 1402, status = succeeded5792lb # [ 132.309535] niks3-server[1149]: 2026/09/21 13:55:52 INFO Received uploads request method=POST path=/api/pending_closures5793worker1 # [ 132.296070] nix-grpc-daemon[1381]: time=2026-09-21T13:55:52.936Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5794worker1 # [ 132.297641] nix-grpc-daemon[1381]: time=2026-09-21T13:55:52.936Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5795lb # [ 132.345052] niks3-server[1149]: 2026/09/21 13:55:52 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5796worker1 # [ 132.334175] nix-grpc-daemon[1381]: time=2026-09-21T13:55:52.973Z level=INFO msg="Uploading 1 narinfos"5797lb # [ 132.356750] niks3-server[1149]: 2026/09/21 13:55:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5798lb # [ 132.366797] niks3-server[1149]: 2026/09/21 13:55:53 INFO Signed narinfos id=41 count=15799lb # [ 132.376188] niks3-server[1149]: 2026/09/21 13:55:53 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5800lb # [ 132.384096] niks3-server[1149]: 2026/09/21 13:55:53 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5801worker1 # [ 132.368533] nix-grpc-daemon[1381]: time=2026-09-21T13:55:53.008Z level=INFO msg="Upload complete. (480ms)"5802lb # [ 132.392245] niks3-server[1149]: 2026/09/21 13:55:53 INFO Completed upload id=415803worker1 # [ 132.376143] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55474 duration_s=0 paths=1 nar_bytes_in=4245804worker1 # [ 132.386042] nix-grpc-daemon[1308]: ts=2026-09-21T13:55:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554745805worker1 # [ 132.395009] nix-grpc-daemon[1381]: time=2026-09-21T13:55:53.025Z level=WARN msg="Failed to register uploaded object" key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo error="server returned 404: object is not pending upload\n"5806worker1 # [ 132.406814] nix-daemon[838]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5807worker1 # [ 132.409992] nix-daemon[838]: reaped child process 1379, status = succeeded5808client # [ 132.407044] nix[1285]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5809worker1 # [ 132.415657] nix-daemon[838]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5810lb: must succeed: curl -sf localhost:9901/clusters5811lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5812lb: must succeed: curl -sf localhost:9901/clusters5813lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5814worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5815worker1 # [ 134.458737] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1308 (nix-grpc-daemon) on client request.5816worker1 # [ 134.460690] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5817lb # [ 134.497138] niks3-server[1149]: 2026/09/21 13:55:55 INFO lead: released remote=192.168.1.3:569965818client # [ 134.481661] nix[1285]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5819worker2 # [ 134.485584] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:55Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5820worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)5821client: must succeed: sleep 5.05822client # [ 134.488456] nix[1285]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5823worker1 # [ 134.503981] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5824worker1 # [ 134.505689] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5825worker1 # [ 134.506930] systemd[1]: nix-grpc-daemon.service: Consumed 520ms CPU time over 5.478s wall clock time, 34.5M memory peak, 14.6K incoming IP traffic, 18.7K outgoing IP traffic.5826worker1 # [ 134.521189] nix-daemon[838]: reaped child process 1404, status = succeeded5827worker1 # [ 134.522897] nix-daemon[838]: reaped child process 1367, status = succeeded5828worker1 # [ 134.532646] nix-daemon[1406]: unexpected Nix daemon error: error: interrupted by the user5829worker1 # [ 134.537555] nix-daemon[838]: reaped child process 1406, status = failed with exit code 15830worker2 # [ 134.685677] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:55Z level=info event=scheduler_connected addr=lb:500515831worker1 # [ 134.692037] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5832worker1 # [ 134.697088] systemd[1]: Starting Nix worker-protocol over gRPC...5833worker1 # [ 134.887453] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5834worker1 # [ 134.890017] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5835worker1 # [ 134.894118] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15836worker1 # [ 134.902662] nix-grpc-daemon[1443]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5837worker1 # [ 134.910391] nix-grpc-daemon[1443]: E0000 00:00:1789998955.540771 1443 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5838worker1 # [ 134.915445] nix-grpc-daemon[1443]: E0000 00:00:1789998955.542768 1443 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5839worker1 # [ 134.918939] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5840lb # [ 134.940467] niks3-server[1149]: 2026/09/21 13:55:55 INFO lead: acquired remote=192.168.1.3:383545841worker1 # [ 134.921876] systemd[1]: Started Nix worker-protocol over gRPC.5842worker1 # [ 134.923307] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=scheduler_take_over5843worker2 # [ 134.921755] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5844worker1 # [ 134.928075] nix-grpc-daemon[1443]: E0000 00:00:1789998955.565908 1483 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5845worker1 # [ 134.956048] nix-grpc-daemon[1443]: E0000 00:00:1789998955.583358 1483 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5846worker1 # [ 134.961054] nix-grpc-daemon[1443]: E0000 00:00:1789998955.583511 1483 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5847worker1 # [ 134.963828] nix-grpc-daemon[1443]: E0000 00:00:1789998955.600574 1483 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5848worker1 # [ 134.973247] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=scheduler_connected addr=lb:500515849worker1 # [ 134.975190] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:397965850worker1 # [ 134.976849] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05851worker1 # [ 135.000050] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05852worker1 # [ 135.001964] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15853worker1 # [ 135.051346] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398085854worker1 # [ 135.059465] nix-daemon[838]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5855worker1 # [ 135.069699] nix-daemon[838]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5856worker1 # [ 135.080610] nix-daemon[838]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5857client # [ 135.074752] nix[1285]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5858worker2 # [ 135.123042] nix-grpc-daemon[1496]: ts=2026-09-21T13:55:55Z level=info event=scheduler_connected addr=lb:500515859worker1 # [ 135.131698] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=oidc_jwks provider=mock keys=15860worker1 # [ 135.135210] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:397965861worker1 # [ 135.139313] nix-grpc-daemon[1443]: ts=2026-09-21T13:55:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05862client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5863client: waiting for success: ! systemctl is-active mid5864worker1 # [ 139.902127] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:00Z level=info event=niks3_push_started pid=15265865worker1 # [ 139.942559] nix-daemon[838]: reaped child process 1501, status = succeeded5866worker1 # [ 140.134421] nix-grpc-daemon[1526]: time=2026-09-21T13:56:00.773Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5867worker1 # [ 140.311795] nix-daemon[838]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)5868worker1 # [ 140.328570] nix-daemon[838]: reaped child process 1542, status = succeeded5869lb # [ 140.367240] niks3-server[1149]: 2026/09/21 13:56:01 INFO Received uploads request method=POST path=/api/pending_closures5870worker1 # [ 140.355475] nix-grpc-daemon[1526]: time=2026-09-21T13:56:00.993Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5871worker1 # [ 140.358285] nix-grpc-daemon[1526]: time=2026-09-21T13:56:00.993Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5872lb # [ 140.417690] niks3-server[1149]: 2026/09/21 13:56:01 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5873worker1 # [ 140.410557] nix-grpc-daemon[1526]: time=2026-09-21T13:56:01.049Z level=INFO msg="Uploading 1 narinfos"5874lb # [ 140.432621] niks3-server[1149]: 2026/09/21 13:56:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5875lb # [ 140.443478] niks3-server[1149]: 2026/09/21 13:56:01 INFO Signed narinfos id=42 count=15876lb # [ 140.458416] niks3-server[1149]: 2026/09/21 13:56:01 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5877lb # [ 140.472388] niks3-server[1149]: 2026/09/21 13:56:01 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5878lb # [ 140.483849] niks3-server[1149]: 2026/09/21 13:56:01 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5879worker1 # [ 140.461320] nix-grpc-daemon[1526]: time=2026-09-21T13:56:01.101Z level=INFO msg="Upload complete. (323ms)"5880lb # [ 140.489486] niks3-server[1149]: 2026/09/21 13:56:01 INFO Completed upload id=425881worker1 # [ 140.480367] nix-grpc-daemon[1526]: time=2026-09-21T13:56:01.107Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo error="server returned 404: object is not pending upload\n"5882worker1 # [ 140.492310] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39808 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15883worker1 # [ 140.499155] nix-daemon[838]: reaped child process 1499, status = succeeded5884client # [ 140.512905] systemd[1]: mid.service: Deactivated successfully.5885client # [ 140.518356] systemd[1]: mid.service: Consumed 186ms CPU time over 10.122s wall clock time, 41.2M memory peak, 8.4K incoming IP traffic, 10.4K outgoing IP traffic.5886client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5887client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5888client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5889(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.72 seconds)5890subtest: interrupting the client stops the build on the worker5891client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5892client # [ 141.089819] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5893client # Running as unit: intr.service; invocation ID: 7eb78e3059114f949c83665b73b52eb65894client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.05 seconds)5895worker2 # [ 141.333653] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:458085896client # [ 141.339062] nix[1321]: don't know how to build these paths:5897client # [ 141.340291] nix[1321]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5898worker1 # [ 141.358336] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:01Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05899worker1 # [ 141.364187] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:01Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15900worker2 # [ 141.404221] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458085901worker2 # [ 141.421700] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 1/5)5902worker2 # [ 141.431902] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 2/5)5903worker2 # [ 141.523301] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 3/5)5904worker2 # [ 141.621036] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 436 ms (attempt 4/5)5905lb # [ 141.903253] envoy[903]: conn peer=[2001:db8:1::3]:39064 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1015 tx=434 ms=51865906lb # [ 141.908636] envoy[903]: conn peer=[2001:db8:1::1]:60862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1826 ms=98395907worker2 # [ 142.065009] nix-grpc-daemon[1496]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5908worker2 # [ 142.070701] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5909worker2 # [ 142.089644] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458085910client # [ 142.100433] nix[1321]: copying 1 paths to 'lb:50051'...5911worker2 # [ 142.105866] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:458085912client # [ 142.103510] nix[1321]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5913worker2 # [ 142.123790] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)5914worker2 # [ 142.150125] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:02Z level=info event=niks3_push_started pid=16445915worker2 # [ 142.376931] nix-grpc-daemon[1644]: time=2026-09-21T13:56:03.013Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5916worker2 # [ 142.557467] nix-daemon[864]: accepted connection from pid 1661, user nix-grpc-daemon (trusted)5917worker2 # [ 142.576957] nix-daemon[864]: reaped child process 1664, status = succeeded5918worker2 # [ 142.757402] nix-daemon[864]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)5919worker2 # [ 142.776234] nix-daemon[864]: reaped child process 1669, status = succeeded5920lb # [ 142.813207] niks3-server[1149]: 2026/09/21 13:56:03 INFO Received uploads request method=POST path=/api/pending_closures5921worker2 # [ 142.806409] nix-grpc-daemon[1644]: time=2026-09-21T13:56:03.443Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5922worker2 # [ 142.807890] nix-grpc-daemon[1644]: time=2026-09-21T13:56:03.443Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5923lb # [ 142.867761] niks3-server[1149]: 2026/09/21 13:56:03 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5924worker2 # [ 142.855102] nix-grpc-daemon[1644]: time=2026-09-21T13:56:03.493Z level=INFO msg="Uploading 1 narinfos"5925lb # [ 142.883106] niks3-server[1149]: 2026/09/21 13:56:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5926lb # [ 142.893731] niks3-server[1149]: 2026/09/21 13:56:03 INFO Signed narinfos id=43 count=15927lb # [ 142.901805] niks3-server[1149]: 2026/09/21 13:56:03 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5928lb # [ 142.909882] niks3-server[1149]: 2026/09/21 13:56:03 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5929lb # [ 142.915319] niks3-server[1149]: 2026/09/21 13:56:03 INFO Completed upload id=435930worker2 # [ 142.890365] nix-grpc-daemon[1644]: time=2026-09-21T13:56:03.528Z level=INFO msg="Upload complete. (510ms)"5931lb # [ 142.918421] niks3-server[1149]: 2026/09/21 13:56:03 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5932worker2 # [ 142.900924] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=0 paths=1 nar_bytes_in=4245933worker2 # [ 142.915859] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458085934worker2 # [ 142.923652] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)5935worker2 # [ 142.929485] nix-daemon[864]: reaped child process 1642, status = succeeded5936client # [ 142.937443] nix[1321]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5937worker2 # [ 142.944445] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)5938client: must succeed: systemctl kill -s INT intr5939client # [ 144.790591] systemd[1]: intr.service: Sent signal SIGINT to main process 1321 (nix) on client request.5940worker2 # [ 144.798185] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:05Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5941client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5942client # [ 144.797136] nix[1321]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5943client # [ 144.801372] nix[1321]: error: interrupted by the user5944client # [ 144.841058] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5945client # [ 144.843705] systemd[1]: intr.service: Failed with result 'exit-code'.5946client # [ 144.847453] systemd[1]: intr.service: Consumed 158ms CPU time over 3.756s wall clock time, 41.4M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.5947worker2 # [ 144.975050] nix-daemon[1673]: unexpected Nix daemon error: error: interrupted by the user5948worker2 # [ 144.981019] nix-daemon[864]: reaped child process 1671, status = succeeded5949worker2 # [ 144.985078] nix-daemon[864]: reaped child process 1673, status = failed with exit code 15950(finished: subtest: interrupting the client stops the build on the worker, in 5.06 seconds)5951subtest: a deploy mid-build drains: the build finishes, the new generation takes over5952client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5953client # [ 146.180669] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5954client # Running as unit: sw.service; invocation ID: d14f08d281e14836ba189e1b93ffaaac5955client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5956client # [ 146.437608] nix[1341]: don't know how to build these paths:5957client # [ 146.438623] nix[1341]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5958worker2 # [ 146.440921] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:458085959worker1 # [ 146.458267] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:07Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05960worker1 # [ 146.458676] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:07Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15961worker1 # [ 146.501960] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398085962worker1 # [ 146.518521] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 1/5)5963worker1 # [ 146.527964] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 157 ms (attempt 2/5)5964worker1 # [ 146.685543] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 155 ms (attempt 3/5)5965worker1 # [ 146.848201] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 161 ms (attempt 4/5)5966worker1 # [ 147.017006] nix-grpc-daemon[1443]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5967worker1 # [ 147.023495] nix-grpc-daemon[1443]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5968worker1 # [ 147.043293] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:398085969client # [ 147.052206] nix[1341]: copying 1 paths to 'lb:50051'...5970worker1 # [ 147.058648] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:398085971client # [ 147.055596] nix[1341]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5972worker1 # [ 147.074244] nix-daemon[838]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5973worker1 # [ 147.286141] nix-daemon[838]: accepted connection from pid 1582, user nix-grpc-daemon (trusted)5974worker1 # [ 147.302950] nix-daemon[838]: reaped child process 1585, status = succeeded5975worker1 # [ 147.492145] nix-daemon[838]: accepted connection from pid 1587, user nix-grpc-daemon (trusted)5976worker1 # [ 147.512606] nix-daemon[838]: reaped child process 1594, status = succeeded5977lb # [ 147.547153] niks3-server[1149]: 2026/09/21 13:56:08 INFO Received uploads request method=POST path=/api/pending_closures5978worker1 # [ 147.533634] nix-grpc-daemon[1526]: time=2026-09-21T13:56:08.173Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5979worker1 # [ 147.535127] nix-grpc-daemon[1526]: time=2026-09-21T13:56:08.173Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5980lb # [ 147.572543] niks3-server[1149]: 2026/09/21 13:56:08 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5981lb # [ 147.591937] niks3-server[1149]: 2026/09/21 13:56:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5982worker1 # [ 147.574268] nix-grpc-daemon[1526]: time=2026-09-21T13:56:08.214Z level=INFO msg="Uploading 1 narinfos"5983lb # [ 147.602897] niks3-server[1149]: 2026/09/21 13:56:08 INFO Signed narinfos id=44 count=15984lb # [ 147.610083] niks3-server[1149]: 2026/09/21 13:56:08 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5985lb # [ 147.621295] niks3-server[1149]: 2026/09/21 13:56:08 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5986worker1 # [ 147.603268] nix-grpc-daemon[1526]: time=2026-09-21T13:56:08.243Z level=INFO msg="Upload complete. (508ms)"5987lb # [ 147.628496] niks3-server[1149]: 2026/09/21 13:56:08 INFO Completed upload id=445988lb # [ 147.631796] niks3-server[1149]: 2026/09/21 13:56:08 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5989worker1 # [ 147.612943] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39808 duration_s=0 paths=1 nar_bytes_in=4165990worker1 # [ 147.625144] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398085991worker1 # [ 147.634522] nix-daemon[838]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5992worker1 # [ 147.637846] nix-daemon[838]: reaped child process 1580, status = succeeded5993client # [ 147.643896] nix[1341]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5994worker1 # [ 147.651738] nix-daemon[838]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5995worker1: must succeed: systemctl reload nix-grpc-daemon.service5996worker1 # [ 148.867447] systemd[1]: Reloading Nix worker-protocol over gRPC...5997worker1 # [ 148.914827] systemd[1]: Reloaded Nix worker-protocol over gRPC.5998worker1 # [ 148.919633] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:09Z level=info event=draining5999worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.10 seconds)6000worker1: must succeed: pgrep -f 'read -t [9]0 x'6001worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)6002client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&26003client # don't know how to build these paths:6004client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6005worker2 # [ 149.245060] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:458086006worker2 # [ 149.252151] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458086007worker1 # [ 149.275718] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:09Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06008worker1 # [ 149.278074] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:09Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16009worker2 # [ 149.323817] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086010worker2 # [ 149.338836] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458086011client # copying 1 paths to 'lb:50051'...6012worker2 # [ 149.354632] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:458086013client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6014worker2 # [ 149.379029] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6015worker2 # [ 149.591992] nix-daemon[864]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6016worker2 # [ 149.610389] nix-daemon[864]: reaped child process 1727, status = succeeded6017worker2 # [ 149.785442] nix-daemon[864]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6018worker2 # [ 149.800968] nix-daemon[864]: reaped child process 1732, status = succeeded6019lb # [ 149.837485] niks3-server[1149]: 2026/09/21 13:56:10 INFO Received uploads request method=POST path=/api/pending_closures6020worker2 # [ 149.822652] nix-grpc-daemon[1644]: time=2026-09-21T13:56:10.460Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6021worker2 # [ 149.824170] nix-grpc-daemon[1644]: time=2026-09-21T13:56:10.460Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6022lb # [ 149.866784] niks3-server[1149]: 2026/09/21 13:56:10 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6023worker2 # [ 149.860738] nix-grpc-daemon[1644]: time=2026-09-21T13:56:10.498Z level=INFO msg="Uploading 1 narinfos"6024lb # [ 149.876904] niks3-server[1149]: 2026/09/21 13:56:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6025lb # [ 149.891582] niks3-server[1149]: 2026/09/21 13:56:10 INFO Signed narinfos id=45 count=16026lb # [ 149.898943] niks3-server[1149]: 2026/09/21 13:56:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6027lb # [ 149.908832] niks3-server[1149]: 2026/09/21 13:56:10 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6028lb # [ 149.915652] niks3-server[1149]: 2026/09/21 13:56:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6029lb # [ 149.921743] niks3-server[1149]: 2026/09/21 13:56:10 INFO Completed upload id=456030worker2 # [ 149.897679] nix-grpc-daemon[1644]: time=2026-09-21T13:56:10.536Z level=INFO msg="Upload complete. (501ms)"6031worker2 # [ 149.905731] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=0 paths=1 nar_bytes_in=5286032worker2 # [ 149.919920] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086033worker2 # [ 149.926029] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6034worker2 # [ 149.930567] nix-daemon[864]: reaped child process 1722, status = succeeded6035client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6036client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6037worker2 # [ 149.942837] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6038client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6039client # farm-a> LOG-farm-a-during-drain6040worker2 # [ 151.181720] nix-daemon[864]: reaped child process 1736, status = succeeded6041worker2 # [ 151.365751] nix-daemon[864]: accepted connection from pid 1752, user nix-grpc-daemon (trusted)6042worker2 # [ 151.382058] nix-daemon[864]: reaped child process 1755, status = succeeded6043lb # [ 151.418878] niks3-server[1149]: 2026/09/21 13:56:12 INFO Received uploads request method=POST path=/api/pending_closures6044worker2 # [ 151.407782] nix-grpc-daemon[1644]: time=2026-09-21T13:56:12.042Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6045worker2 # [ 151.409963] nix-grpc-daemon[1644]: time=2026-09-21T13:56:12.042Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6046lb # [ 151.450745] niks3-server[1149]: 2026/09/21 13:56:12 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6047lb # [ 151.472110] niks3-server[1149]: 2026/09/21 13:56:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6048worker2 # [ 151.457525] nix-grpc-daemon[1644]: time=2026-09-21T13:56:12.094Z level=INFO msg="Uploading 1 narinfos"6049lb # [ 151.486325] niks3-server[1149]: 2026/09/21 13:56:12 INFO Signed narinfos id=46 count=16050lb # [ 151.492688] niks3-server[1149]: 2026/09/21 13:56:12 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6051lb # [ 151.498444] niks3-server[1149]: 2026/09/21 13:56:12 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6052lb # [ 151.508883] niks3-server[1149]: 2026/09/21 13:56:12 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6053lb # [ 151.515250] niks3-server[1149]: 2026/09/21 13:56:12 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6054lb # [ 151.519560] niks3-server[1149]: 2026/09/21 13:56:12 INFO Completed upload id=466055worker2 # [ 151.495204] nix-grpc-daemon[1644]: time=2026-09-21T13:56:12.133Z level=INFO msg="Upload complete. (342ms)"6056worker1 # [ 151.510364] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:12Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06057worker1 # [ 151.512285] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:12Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16058worker2 # [ 151.510309] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16059worker2 # [ 151.524720] nix-daemon[864]: reaped child process 1734, status = succeeded6060worker2 # [ 151.557535] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086061worker2 # [ 151.571445] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458086062worker2 # [ 151.586088] nix-grpc-daemon[1644]: time=2026-09-21T13:56:12.224Z level=INFO msg="All 1 paths already cached"6063client # copying 1 paths to 'lb:50051'...6064worker2 # [ 151.596740] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:458086065client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6066worker2 # [ 151.620191] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6067worker2 # [ 151.822959] nix-daemon[864]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6068worker2 # [ 151.842727] nix-daemon[864]: reaped child process 1762, status = succeeded6069lb # [ 151.906817] envoy[903]: conn peer=[2001:db8:1::1]:51102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=852 ms=34996070worker2 # [ 152.019679] nix-daemon[864]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6071worker2 # [ 152.036387] nix-daemon[864]: reaped child process 1767, status = succeeded6072lb # [ 152.075908] niks3-server[1149]: 2026/09/21 13:56:12 INFO Received uploads request method=POST path=/api/pending_closures6073worker2 # [ 152.064249] nix-grpc-daemon[1644]: time=2026-09-21T13:56:12.702Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6074worker2 # [ 152.066367] nix-grpc-daemon[1644]: time=2026-09-21T13:56:12.702Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6075lb # [ 152.117031] niks3-server[1149]: 2026/09/21 13:56:12 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6076lb # [ 152.133297] niks3-server[1149]: 2026/09/21 13:56:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6077worker2 # [ 152.115506] nix-grpc-daemon[1644]: time=2026-09-21T13:56:12.753Z level=INFO msg="Uploading 1 narinfos"6078lb # [ 152.146573] niks3-server[1149]: 2026/09/21 13:56:12 INFO Signed narinfos id=47 count=16079lb # [ 152.152531] niks3-server[1149]: 2026/09/21 13:56:12 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6080lb # [ 152.160252] niks3-server[1149]: 2026/09/21 13:56:12 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6081lb # [ 152.166903] niks3-server[1149]: 2026/09/21 13:56:12 INFO Completed upload id=476082worker2 # [ 152.142819] nix-grpc-daemon[1644]: time=2026-09-21T13:56:12.781Z level=INFO msg="Upload complete. (505ms)"6083lb # [ 152.169553] niks3-server[1149]: 2026/09/21 13:56:12 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6084worker2 # [ 152.152430] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=0 paths=1 nar_bytes_in=6406085worker2 # [ 152.167210] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086086worker2 # [ 152.173916] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6087worker2 # [ 152.179753] nix-daemon[864]: reaped child process 1757, status = succeeded6088client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6089worker2 # [ 152.193010] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6090client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6091client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6092client # farm-b> LOG-farm-b-during-drain6093worker2 # [ 153.034015] nix-daemon[864]: reaped child process 1771, status = succeeded6094worker2 # [ 153.217357] nix-daemon[864]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6095worker2 # [ 153.235037] nix-daemon[864]: reaped child process 1790, status = succeeded6096lb # [ 153.271865] niks3-server[1149]: 2026/09/21 13:56:13 INFO Received uploads request method=POST path=/api/pending_closures6097worker2 # [ 153.257898] nix-grpc-daemon[1644]: time=2026-09-21T13:56:13.895Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6098worker2 # [ 153.259618] nix-grpc-daemon[1644]: time=2026-09-21T13:56:13.895Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6099lb # [ 153.306041] niks3-server[1149]: 2026/09/21 13:56:13 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6100lb # [ 153.320234] niks3-server[1149]: 2026/09/21 13:56:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6101worker2 # [ 153.302296] nix-grpc-daemon[1644]: time=2026-09-21T13:56:13.940Z level=INFO msg="Uploading 1 narinfos"6102lb # [ 153.331942] niks3-server[1149]: 2026/09/21 13:56:13 INFO Signed narinfos id=48 count=16103lb # [ 153.337788] niks3-server[1149]: 2026/09/21 13:56:13 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6104lb # [ 153.342732] niks3-server[1149]: 2026/09/21 13:56:13 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6105lb # [ 153.352224] niks3-server[1149]: 2026/09/21 13:56:13 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6106lb # [ 153.357552] niks3-server[1149]: 2026/09/21 13:56:14 INFO Completed upload id=486107worker2 # [ 153.333503] nix-grpc-daemon[1644]: time=2026-09-21T13:56:13.971Z level=INFO msg="Upload complete. (327ms)"6108lb # [ 153.360822] niks3-server[1149]: 2026/09/21 13:56:14 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6109worker1 # [ 153.349694] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:13Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06110worker1 # [ 153.351548] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:13Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16111worker2 # [ 153.345176] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16112worker2 # [ 153.365170] nix-daemon[864]: reaped child process 1769, status = succeeded6113worker2 # [ 153.396261] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086114worker2 # [ 153.412496] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458086115worker2 # [ 153.426938] nix-grpc-daemon[1644]: time=2026-09-21T13:56:14.065Z level=INFO msg="All 2 paths already cached"6116client # copying 1 paths to 'lb:50051'...6117client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6118worker2 # [ 153.437230] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:458086119worker2 # [ 153.455917] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6120worker2 # [ 153.659175] nix-daemon[864]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6121worker2 # [ 153.675873] nix-daemon[864]: reaped child process 1797, status = succeeded6122worker2 # [ 153.846297] nix-daemon[864]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6123worker2 # [ 153.861833] nix-daemon[864]: reaped child process 1802, status = succeeded6124lb # [ 153.897412] niks3-server[1149]: 2026/09/21 13:56:14 INFO Received uploads request method=POST path=/api/pending_closures6125worker2 # [ 153.882637] nix-grpc-daemon[1644]: time=2026-09-21T13:56:14.520Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6126worker2 # [ 153.884113] nix-grpc-daemon[1644]: time=2026-09-21T13:56:14.520Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6127lb # [ 153.924597] niks3-server[1149]: 2026/09/21 13:56:14 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6128worker2 # [ 153.914831] nix-grpc-daemon[1644]: time=2026-09-21T13:56:14.553Z level=INFO msg="Uploading 1 narinfos"6129lb # [ 153.939952] niks3-server[1149]: 2026/09/21 13:56:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6130lb # [ 153.949437] niks3-server[1149]: 2026/09/21 13:56:14 INFO Signed narinfos id=49 count=16131lb # [ 153.951971] niks3-server[1149]: 2026/09/21 13:56:14 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6132lb # [ 153.960501] niks3-server[1149]: 2026/09/21 13:56:14 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6133lb # [ 153.967264] niks3-server[1149]: 2026/09/21 13:56:14 INFO Completed upload id=496134worker2 # [ 153.942880] nix-grpc-daemon[1644]: time=2026-09-21T13:56:14.581Z level=INFO msg="Upload complete. (469ms)"6135lb # [ 153.969748] niks3-server[1149]: 2026/09/21 13:56:14 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6136worker2 # [ 153.952066] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=0 paths=1 nar_bytes_in=7686137worker2 # [ 153.961767] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086138worker2 # [ 153.968154] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6139worker2 # [ 153.972188] nix-daemon[864]: reaped child process 1792, status = succeeded6140client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6141worker2 # [ 153.983875] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6142client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6143client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6144client # farm-top> LOG-farm-top-during-drain6145worker2 # [ 155.003692] nix-daemon[864]: reaped child process 1806, status = succeeded6146worker2 # [ 155.161329] nix-daemon[864]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)6147worker2 # [ 155.180265] nix-daemon[864]: reaped child process 1825, status = succeeded6148lb # [ 155.217003] niks3-server[1149]: 2026/09/21 13:56:15 INFO Received uploads request method=POST path=/api/pending_closures6149worker2 # [ 155.204382] nix-grpc-daemon[1644]: time=2026-09-21T13:56:15.841Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6150worker2 # [ 155.205876] nix-grpc-daemon[1644]: time=2026-09-21T13:56:15.841Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6151lb # [ 155.253769] niks3-server[1149]: 2026/09/21 13:56:15 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6152lb # [ 155.266683] niks3-server[1149]: 2026/09/21 13:56:15 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6153worker2 # [ 155.257488] nix-grpc-daemon[1644]: time=2026-09-21T13:56:15.895Z level=INFO msg="Uploading 1 narinfos"6154lb # [ 155.275775] niks3-server[1149]: 2026/09/21 13:56:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6155lb # [ 155.286344] niks3-server[1149]: 2026/09/21 13:56:15 INFO Signed narinfos id=50 count=16156lb # [ 155.292697] niks3-server[1149]: 2026/09/21 13:56:15 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6157lb # [ 155.301901] niks3-server[1149]: 2026/09/21 13:56:15 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6158lb # [ 155.307552] niks3-server[1149]: 2026/09/21 13:56:15 INFO Completed upload id=506159worker2 # [ 155.283373] nix-grpc-daemon[1644]: time=2026-09-21T13:56:15.921Z level=INFO msg="Upload complete. (290ms)"6160lb # [ 155.310937] niks3-server[1149]: 2026/09/21 13:56:15 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6161worker2 # [ 155.293052] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16162worker2 # [ 155.310757] nix-daemon[864]: reaped child process 1804, status = succeeded6163client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.37 seconds)6164client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath6165client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)6166worker1: must succeed: pkill -f 'read -t [9]0 x'6167worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6168client: waiting for success: ! systemctl is-active sw6169worker1 # [ 155.677089] nix-daemon[838]: reaped child process 1598, status = succeeded6170worker1 # [ 155.839843] nix-daemon[838]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)6171worker1 # [ 155.856206] nix-daemon[838]: reaped child process 1644, status = succeeded6172lb # [ 155.890337] niks3-server[1149]: 2026/09/21 13:56:16 INFO Received uploads request method=POST path=/api/pending_closures6173worker1 # [ 155.877397] nix-grpc-daemon[1526]: time=2026-09-21T13:56:16.516Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6174worker1 # [ 155.878855] nix-grpc-daemon[1526]: time=2026-09-21T13:56:16.516Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6175lb # [ 155.925893] niks3-server[1149]: 2026/09/21 13:56:16 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6176worker1 # [ 155.917873] nix-grpc-daemon[1526]: time=2026-09-21T13:56:16.557Z level=INFO msg="Uploading 1 narinfos"6177lb # [ 155.943317] niks3-server[1149]: 2026/09/21 13:56:16 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6178lb # [ 155.952689] niks3-server[1149]: 2026/09/21 13:56:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6179lb # [ 155.957025] niks3-server[1149]: 2026/09/21 13:56:16 INFO Signed narinfos id=51 count=16180lb # [ 155.963715] niks3-server[1149]: 2026/09/21 13:56:16 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6181worker1 # [ 155.944578] nix-grpc-daemon[1526]: time=2026-09-21T13:56:16.584Z level=INFO msg="Upload complete. (278ms)"6182lb # [ 155.967952] niks3-server[1149]: 2026/09/21 13:56:16 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6183lb # [ 155.973146] niks3-server[1149]: 2026/09/21 13:56:16 INFO Completed upload id=516184lb # [ 155.975729] niks3-server[1149]: 2026/09/21 13:56:16 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6185worker1 # [ 155.953849] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39808 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16186worker1 # [ 155.967630] nix-daemon[838]: reaped child process 1596, status = succeeded6187client # [ 155.993585] systemd[1]: sw.service: Deactivated successfully.6188client # [ 155.998704] systemd[1]: sw.service: Consumed 178ms CPU time over 9.816s wall clock time, 41.4M memory peak, 7.4K incoming IP traffic, 8.4K outgoing IP traffic.6189client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds)6190client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6191client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)6192worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6193worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.07 seconds)6194lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6195lb: must succeed: curl -sf localhost:9901/clusters6196worker1 # [ 156.921655] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:17Z level=info event=signal_exit6197lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6198lb # [ 157.169308] niks3-server[1149]: 2026/09/21 13:56:17 INFO lead: released remote=192.168.1.3:383546199lb: must succeed: curl -sf localhost:9901/clusters6200lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6201worker2 # [ 158.144590] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:18Z level=info event=scheduler_restarting addr=lb:500516202worker1 # [ 158.149145] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:18Z level=info event=scheduler_restarting6203worker1 # [ 158.155654] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:397966204worker1 # [ 158.161060] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:18Z level=info event=worker_gone addr=192.168.1.4:500516205worker1 # [ 158.165506] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:18Z level=info event=scheduler_restarting addr=lb:500516206worker1 # [ 158.170346] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:397966207worker1 # [ 158.175343] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:18Z level=info event=worker_gone addr=192.168.1.3:500516208worker1 # [ 158.198856] nix-daemon[838]: reaped child process 1497, status = succeeded6209worker2 # [ 158.344681] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:18Z level=info event=scheduler_connected addr=lb:500516210worker2 # [ 158.349930] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6211worker1 # [ 158.367539] nix-grpc-daemon[1443]: ts=2026-09-21T13:56:19Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6212worker2 # [ 158.550949] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:19Z level=info event=scheduler_connected addr=lb:500516213worker2 # [ 158.555817] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6214worker2 # [ 158.756899] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:19Z level=info event=scheduler_connected addr=lb:500516215worker2 # [ 158.761792] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6216worker1 # [ 158.956887] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6217worker2 # [ 158.963061] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:19Z level=info event=scheduler_connected addr=lb:500516218worker1 # [ 158.963173] systemd[1]: nix-grpc-daemon.service: Consumed 876ms CPU time over 24.458s wall clock time, 35.1M memory peak, 25.3K incoming IP traffic, 31.7K outgoing IP traffic.6219worker2 # [ 158.968353] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6220worker1 # [ 159.062890] systemd[1]: Starting Nix worker-protocol over gRPC...6221lb: must succeed: curl -sf localhost:9901/clusters6222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6223worker2 # [ 159.170956] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:19Z level=info event=scheduler_connected addr=lb:500516224worker2 # [ 159.175031] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6225worker1 # [ 159.271828] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6226worker1 # [ 159.274412] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6227worker1 # [ 159.278574] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16228worker1 # [ 159.287281] nix-grpc-daemon[1656]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6229worker1 # [ 159.298532] nix-grpc-daemon[1656]: E0000 00:00:1789998979.925472 1656 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6230worker1 # [ 159.302299] nix-grpc-daemon[1656]: E0000 00:00:1789998979.927385 1656 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6231worker1 # [ 159.305133] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6232lb # [ 159.326060] niks3-server[1149]: 2026/09/21 13:56:19 INFO lead: acquired remote=192.168.1.3:542446233worker1 # [ 159.307687] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:19Z level=info event=scheduler_take_over6234worker1 # [ 159.309167] systemd[1]: Started Nix worker-protocol over gRPC.6235worker1 # [ 159.312411] nix-grpc-daemon[1656]: E0000 00:00:1789998979.952480 1697 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6236worker1 # [ 159.340055] nix-grpc-daemon[1656]: E0000 00:00:1789998979.970399 1697 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6237worker1 # [ 159.342333] nix-grpc-daemon[1656]: E0000 00:00:1789998979.970529 1697 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6238worker1 # [ 159.350135] nix-grpc-daemon[1656]: E0000 00:00:1789998979.989658 1697 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6239worker1 # [ 159.362118] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:20Z level=info event=scheduler_connected addr=lb:500516240worker1 # [ 159.365606] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:20Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:561306241worker1 # [ 159.367311] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:20Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06242worker2 # [ 159.371846] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:20Z level=info event=scheduler_connected addr=lb:500516243worker1 # [ 159.378059] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:20Z level=info event=oidc_jwks provider=mock keys=16244worker1 # [ 159.379540] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:20Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:561306245worker1 # [ 159.381339] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:20Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06246lb: must succeed: curl -sf localhost:9901/clusters6247lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6248lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 3.32 seconds)6249lb: must succeed: curl -sf localhost:9901/clusters6250lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6251worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6252worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6253(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.25 seconds)6254subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6255client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16256worker2 # [ 160.632554] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:458086257worker2 # [ 160.640505] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458086258worker1 # [ 160.688189] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:21Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06259worker1 # [ 160.688527] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:21Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16260worker1 # [ 160.688722] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:21Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06261worker1 # [ 160.688842] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:21Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06262worker1 # [ 160.737448] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386263worker1 # [ 160.751835] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6264worker1 # [ 160.775052] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 1/5)6265worker1 # [ 160.800560] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 2/5)6266worker1 # [ 160.882864] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 208 ms (attempt 3/5)6267worker1 # [ 161.098356] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 529 ms (attempt 4/5)6268worker1 # [ 161.635276] nix-grpc-daemon[1656]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6269worker1 # [ 161.640943] nix-grpc-daemon[1656]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6270worker1 # [ 161.659170] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386271worker1 # [ 161.674666] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561386272worker1 # [ 161.689823] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6273worker1 # [ 161.724872] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:22Z level=info event=niks3_push_started pid=17216274lb # [ 161.912303] envoy[903]: conn peer=[2001:db8:1::1]:51118 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9651 tx=4135 ms=60696275lb # [ 161.912560] envoy[903]: conn peer=[2001:db8:1::1]:51112 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=1471 ms=95306276lb # [ 161.912670] envoy[903]: conn peer=[2001:db8:1::3]:35052 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=894 tx=344 ms=233846277worker1 # [ 161.934395] nix-grpc-daemon[1721]: time=2026-09-21T13:56:22.573Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""6278worker1 # [ 162.104075] nix-daemon[838]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6279worker1 # [ 162.118712] nix-daemon[838]: reaped child process 1737, status = succeeded6280worker1 # [ 162.296589] nix-daemon[838]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6281worker1 # [ 162.311601] nix-daemon[838]: reaped child process 1742, status = succeeded6282lb # [ 162.349105] niks3-server[1149]: 2026/09/21 13:56:22 INFO Received uploads request method=POST path=/api/pending_closures6283worker1 # [ 162.339710] nix-grpc-daemon[1721]: time=2026-09-21T13:56:22.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6284worker1 # [ 162.341466] nix-grpc-daemon[1721]: time=2026-09-21T13:56:22.978Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6285lb # [ 162.390671] niks3-server[1149]: 2026/09/21 13:56:23 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6286worker1 # [ 162.376516] nix-grpc-daemon[1721]: time=2026-09-21T13:56:23.016Z level=INFO msg="Uploading 1 narinfos"6287lb # [ 162.405519] niks3-server[1149]: 2026/09/21 13:56:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6288lb # [ 162.416300] niks3-server[1149]: 2026/09/21 13:56:23 INFO Signed narinfos id=52 count=16289lb # [ 162.419199] niks3-server[1149]: 2026/09/21 13:56:23 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6290lb # [ 162.426983] niks3-server[1149]: 2026/09/21 13:56:23 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6291lb # [ 162.434388] niks3-server[1149]: 2026/09/21 13:56:23 INFO Completed upload id=526292worker1 # [ 162.411587] nix-grpc-daemon[1721]: time=2026-09-21T13:56:23.051Z level=INFO msg="Upload complete. (473ms)"6293lb # [ 162.437311] niks3-server[1149]: 2026/09/21 13:56:23 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6294worker1 # [ 162.420256] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=0 paths=1 nar_bytes_in=5126295worker1 # [ 162.431262] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386296worker1 # [ 162.437717] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6297worker1 # [ 162.442768] nix-daemon[838]: reaped child process 1719, status = succeeded6298worker1 # [ 162.455573] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6299worker1 # [ 163.892470] nix-daemon[838]: reaped child process 1746, status = succeeded6300worker1 # [ 164.065234] nix-daemon[838]: accepted connection from pid 1762, user nix-grpc-daemon (trusted)6301worker1 # [ 164.082901] nix-daemon[838]: reaped child process 1765, status = succeeded6302lb # [ 164.117941] niks3-server[1149]: 2026/09/21 13:56:24 INFO Received uploads request method=POST path=/api/pending_closures6303worker1 # [ 164.108421] nix-grpc-daemon[1721]: time=2026-09-21T13:56:24.746Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6304worker1 # [ 164.109904] nix-grpc-daemon[1721]: time=2026-09-21T13:56:24.746Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6305lb # [ 164.156268] niks3-server[1149]: 2026/09/21 13:56:24 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6306lb # [ 164.168123] niks3-server[1149]: 2026/09/21 13:56:24 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6307lb # [ 164.181432] niks3-server[1149]: 2026/09/21 13:56:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6308worker1 # [ 164.164907] nix-grpc-daemon[1721]: time=2026-09-21T13:56:24.803Z level=INFO msg="Uploading 1 narinfos"6309lb # [ 164.187375] niks3-server[1149]: 2026/09/21 13:56:24 INFO Signed narinfos id=53 count=16310lb # [ 164.191653] niks3-server[1149]: 2026/09/21 13:56:24 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6311lb # [ 164.203372] niks3-server[1149]: 2026/09/21 13:56:24 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6312lb # [ 164.210404] niks3-server[1149]: 2026/09/21 13:56:24 INFO Completed upload id=536313worker1 # [ 164.187182] nix-grpc-daemon[1721]: time=2026-09-21T13:56:24.827Z level=INFO msg="Upload complete. (321ms)"6314lb # [ 164.212870] niks3-server[1149]: 2026/09/21 13:56:24 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6315worker1 # [ 164.209013] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=16316worker1 # [ 164.220724] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:24Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16317worker1 # [ 164.224583] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:24Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06318worker1 # [ 164.227577] nix-daemon[838]: reaped child process 1744, status = succeeded6319worker1 # [ 164.246566] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386320worker1 # [ 164.255486] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386321worker1 # [ 164.267585] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561386322worker1 # [ 164.281154] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6323worker1 # [ 164.481277] nix-daemon[838]: accepted connection from pid 1771, user nix-grpc-daemon (trusted)6324worker1 # [ 164.499990] nix-daemon[838]: reaped child process 1774, status = succeeded6325worker1 # [ 164.673086] nix-daemon[838]: accepted connection from pid 1776, user nix-grpc-daemon (trusted)6326worker1 # [ 164.688560] nix-daemon[838]: reaped child process 1779, status = succeeded6327lb # [ 164.725745] niks3-server[1149]: 2026/09/21 13:56:25 INFO Received uploads request method=POST path=/api/pending_closures6328worker1 # [ 164.712102] nix-grpc-daemon[1721]: time=2026-09-21T13:56:25.351Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6329worker1 # [ 164.713601] nix-grpc-daemon[1721]: time=2026-09-21T13:56:25.351Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6330lb # [ 164.751976] niks3-server[1149]: 2026/09/21 13:56:25 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6331worker1 # [ 164.741755] nix-grpc-daemon[1721]: time=2026-09-21T13:56:25.381Z level=INFO msg="Uploading 1 narinfos"6332lb # [ 164.765745] niks3-server[1149]: 2026/09/21 13:56:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6333lb # [ 164.778158] niks3-server[1149]: 2026/09/21 13:56:25 INFO Signed narinfos id=54 count=16334lb # [ 164.784753] niks3-server[1149]: 2026/09/21 13:56:25 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6335worker1 # [ 164.770707] nix-grpc-daemon[1721]: time=2026-09-21T13:56:25.410Z level=INFO msg="Upload complete. (475ms)"6336lb # [ 164.795702] niks3-server[1149]: 2026/09/21 13:56:25 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6337worker1 # [ 164.776195] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=0 paths=1 nar_bytes_in=5126338lb # [ 164.802555] niks3-server[1149]: 2026/09/21 13:56:25 INFO Completed upload id=546339worker1 # [ 164.784363] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386340lb # [ 164.806813] niks3-server[1149]: 2026/09/21 13:56:25 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6341worker1 # [ 164.796248] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6342worker1 # [ 164.798664] nix-daemon[838]: reaped child process 1769, status = succeeded6343worker1 # [ 164.808404] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6344worker1 # [ 166.052777] nix-daemon[838]: reaped child process 1783, status = succeeded6345worker1 # [ 166.239108] nix-daemon[838]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6346worker1 # [ 166.255493] nix-daemon[838]: reaped child process 1802, status = succeeded6347lb # [ 166.290715] niks3-server[1149]: 2026/09/21 13:56:26 INFO Received uploads request method=POST path=/api/pending_closures6348worker1 # [ 166.278563] nix-grpc-daemon[1721]: time=2026-09-21T13:56:26.917Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6349worker1 # [ 166.280314] nix-grpc-daemon[1721]: time=2026-09-21T13:56:26.917Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6350lb # [ 166.324594] niks3-server[1149]: 2026/09/21 13:56:26 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6351lb # [ 166.335920] niks3-server[1149]: 2026/09/21 13:56:26 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6352worker1 # [ 166.323082] nix-grpc-daemon[1721]: time=2026-09-21T13:56:26.962Z level=INFO msg="Uploading 1 narinfos"6353lb # [ 166.349330] niks3-server[1149]: 2026/09/21 13:56:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6354lb # [ 166.357448] niks3-server[1149]: 2026/09/21 13:56:26 INFO Signed narinfos id=55 count=16355lb # [ 166.359644] niks3-server[1149]: 2026/09/21 13:56:27 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6356lb # [ 166.368501] niks3-server[1149]: 2026/09/21 13:56:27 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6357lb # [ 166.372413] niks3-server[1149]: 2026/09/21 13:56:27 INFO Completed upload id=556358worker1 # [ 166.349664] nix-grpc-daemon[1721]: time=2026-09-21T13:56:26.989Z level=INFO msg="Upload complete. (330ms)"6359lb # [ 166.374544] niks3-server[1149]: 2026/09/21 13:56:27 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6360worker1 # [ 166.366619] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=16361worker1 # [ 166.380723] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:26Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16362worker1 # [ 166.387092] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386363worker1 # [ 166.391571] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:27Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06364worker1 # [ 166.395747] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386365worker1 # [ 166.400516] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561386366worker1 # [ 166.413896] nix-daemon[838]: reaped child process 1781, status = succeeded6367worker1 # [ 166.416970] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6368worker1 # [ 166.591571] nix-daemon[838]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6369worker1 # [ 166.610675] nix-daemon[838]: reaped child process 1811, status = succeeded6370worker1 # [ 166.781715] nix-daemon[838]: accepted connection from pid 1813, user nix-grpc-daemon (trusted)6371worker1 # [ 166.798970] nix-daemon[838]: reaped child process 1816, status = succeeded6372lb # [ 166.836629] niks3-server[1149]: 2026/09/21 13:56:27 INFO Received uploads request method=POST path=/api/pending_closures6373worker1 # [ 166.822524] nix-grpc-daemon[1721]: time=2026-09-21T13:56:27.462Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6374worker1 # [ 166.824111] nix-grpc-daemon[1721]: time=2026-09-21T13:56:27.462Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6375lb # [ 166.863213] niks3-server[1149]: 2026/09/21 13:56:27 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6376lb # [ 166.876411] niks3-server[1149]: 2026/09/21 13:56:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6377worker1 # [ 166.860321] nix-grpc-daemon[1721]: time=2026-09-21T13:56:27.500Z level=INFO msg="Uploading 1 narinfos"6378lb # [ 166.888032] niks3-server[1149]: 2026/09/21 13:56:27 INFO Signed narinfos id=56 count=16379lb # [ 166.894101] niks3-server[1149]: 2026/09/21 13:56:27 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6380lb # [ 166.900410] niks3-server[1149]: 2026/09/21 13:56:27 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6381lb # [ 166.907898] niks3-server[1149]: 2026/09/21 13:56:27 INFO Completed upload id=566382worker1 # [ 166.885034] nix-grpc-daemon[1721]: time=2026-09-21T13:56:27.525Z level=INFO msg="Upload complete. (464ms)"6383lb # [ 166.910712] niks3-server[1149]: 2026/09/21 13:56:27 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6384worker1 # [ 166.892134] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=0 paths=1 nar_bytes_in=5126385worker1 # [ 166.903751] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386386worker1 # [ 166.911926] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6387worker1 # [ 166.914925] nix-daemon[838]: reaped child process 1806, status = succeeded6388worker1 # [ 166.927247] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6389worker1 # [ 167.969698] nix-daemon[838]: reaped child process 1820, status = succeeded6390worker1 # [ 168.161815] nix-daemon[838]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)6391worker1 # [ 168.177699] nix-daemon[838]: reaped child process 1839, status = succeeded6392lb # [ 168.214104] niks3-server[1149]: 2026/09/21 13:56:28 INFO Received uploads request method=POST path=/api/pending_closures6393worker1 # [ 168.200441] nix-grpc-daemon[1721]: time=2026-09-21T13:56:28.840Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6394worker1 # [ 168.201909] nix-grpc-daemon[1721]: time=2026-09-21T13:56:28.840Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6395lb # [ 168.260462] niks3-server[1149]: 2026/09/21 13:56:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6396lb # [ 168.271100] niks3-server[1149]: 2026/09/21 13:56:28 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6397lb # [ 168.283968] niks3-server[1149]: 2026/09/21 13:56:28 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6398lb # [ 168.287375] niks3-server[1149]: 2026/09/21 13:56:28 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6399lb # [ 168.293679] niks3-server[1149]: 2026/09/21 13:56:28 INFO Signed narinfos id=57 count=16400worker1 # [ 168.271266] nix-grpc-daemon[1721]: time=2026-09-21T13:56:28.910Z level=INFO msg="Uploading 1 narinfos"6401lb # [ 168.302533] niks3-server[1149]: 2026/09/21 13:56:28 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6402lb # [ 168.309894] niks3-server[1149]: 2026/09/21 13:56:28 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6403lb # [ 168.321409] niks3-server[1149]: 2026/09/21 13:56:28 INFO Completed upload id=576404worker1 # [ 168.298785] nix-grpc-daemon[1721]: time=2026-09-21T13:56:28.938Z level=INFO msg="Upload complete. (361ms)"6405worker1 # [ 168.314850] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=16406worker1 # [ 168.340437] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:28Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16407worker1 # [ 168.346836] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386408worker1 # [ 168.352316] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:28Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06409worker1 # [ 168.357829] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386410worker1 # [ 168.363100] nix-daemon[838]: reaped child process 1818, status = succeeded6411worker1 # [ 168.365516] nix-grpc-daemon[1721]: time=2026-09-21T13:56:29.002Z level=INFO msg="All 1 paths already cached"6412worker1 # [ 168.376113] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561386413worker1 # [ 168.380936] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6414worker1 # [ 168.570842] nix-daemon[838]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)6415worker1 # [ 168.587505] nix-daemon[838]: reaped child process 1848, status = succeeded6416worker1 # [ 168.761924] nix-daemon[838]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6417worker1 # [ 168.778103] nix-daemon[838]: reaped child process 1853, status = succeeded6418lb # [ 168.812254] niks3-server[1149]: 2026/09/21 13:56:29 INFO Received uploads request method=POST path=/api/pending_closures6419worker1 # [ 168.800572] nix-grpc-daemon[1721]: time=2026-09-21T13:56:29.440Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6420worker1 # [ 168.802033] nix-grpc-daemon[1721]: time=2026-09-21T13:56:29.440Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6421lb # [ 168.848262] niks3-server[1149]: 2026/09/21 13:56:29 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6422worker1 # [ 168.834437] nix-grpc-daemon[1721]: time=2026-09-21T13:56:29.474Z level=INFO msg="Uploading 1 narinfos"6423lb # [ 168.861555] niks3-server[1149]: 2026/09/21 13:56:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6424lb # [ 168.871674] niks3-server[1149]: 2026/09/21 13:56:29 INFO Signed narinfos id=58 count=16425lb # [ 168.879005] niks3-server[1149]: 2026/09/21 13:56:29 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6426lb # [ 168.883373] niks3-server[1149]: 2026/09/21 13:56:29 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6427worker1 # [ 168.864547] nix-grpc-daemon[1721]: time=2026-09-21T13:56:29.504Z level=INFO msg="Upload complete. (473ms)"6428lb # [ 168.888105] niks3-server[1149]: 2026/09/21 13:56:29 INFO Completed upload id=586429lb # [ 168.891171] niks3-server[1149]: 2026/09/21 13:56:29 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6430worker1 # [ 168.872177] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=0 paths=1 nar_bytes_in=6246431worker1 # [ 168.885959] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386432worker1 # [ 168.891814] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6433worker1 # [ 168.897005] nix-daemon[838]: reaped child process 1843, status = succeeded6434worker1 # [ 168.910447] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6435worker1 # [ 169.744878] nix-daemon[838]: reaped child process 1857, status = succeeded6436worker1 # [ 169.915862] nix-daemon[838]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6437worker1 # [ 169.931176] nix-daemon[838]: reaped child process 1876, status = succeeded6438lb # [ 169.970438] niks3-server[1149]: 2026/09/21 13:56:30 INFO Received uploads request method=POST path=/api/pending_closures6439worker1 # [ 169.959717] nix-grpc-daemon[1721]: time=2026-09-21T13:56:30.597Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6440worker1 # [ 169.961408] nix-grpc-daemon[1721]: time=2026-09-21T13:56:30.597Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6441lb # [ 170.007145] niks3-server[1149]: 2026/09/21 13:56:30 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6442lb # [ 170.021380] niks3-server[1149]: 2026/09/21 13:56:30 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6443lb # [ 170.030326] niks3-server[1149]: 2026/09/21 13:56:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6444lb # [ 170.036522] niks3-server[1149]: 2026/09/21 13:56:30 INFO Signed narinfos id=59 count=16445worker1 # [ 170.013776] nix-grpc-daemon[1721]: time=2026-09-21T13:56:30.653Z level=INFO msg="Uploading 1 narinfos"6446lb # [ 170.040812] niks3-server[1149]: 2026/09/21 13:56:30 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6447lb # [ 170.051922] niks3-server[1149]: 2026/09/21 13:56:30 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6448lb # [ 170.058337] niks3-server[1149]: 2026/09/21 13:56:30 INFO Completed upload id=596449worker1 # [ 170.035001] nix-grpc-daemon[1721]: time=2026-09-21T13:56:30.675Z level=INFO msg="Upload complete. (315ms)"6450lb # [ 170.061467] niks3-server[1149]: 2026/09/21 13:56:30 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6451worker1 # [ 170.051261] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=16452worker1 # [ 170.067508] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:30Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16453worker1 # [ 170.072623] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:30Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06454worker1 # [ 170.076539] nix-daemon[838]: reaped child process 1855, status = succeeded6455worker1 # [ 170.093295] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386456worker1 # [ 170.106334] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386457worker1 # [ 170.119419] nix-grpc-daemon[1721]: time=2026-09-21T13:56:30.759Z level=INFO msg="All 1 paths already cached"6458worker1 # [ 170.127668] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561386459worker1 # [ 170.140497] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6460worker1 # [ 170.333688] nix-daemon[838]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)6461worker1 # [ 170.349951] nix-daemon[838]: reaped child process 1885, status = succeeded6462worker1 # [ 170.522661] nix-daemon[838]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6463worker1 # [ 170.539427] nix-daemon[838]: reaped child process 1890, status = succeeded6464lb # [ 170.573804] niks3-server[1149]: 2026/09/21 13:56:31 INFO Received uploads request method=POST path=/api/pending_closures6465worker1 # [ 170.559704] nix-grpc-daemon[1721]: time=2026-09-21T13:56:31.199Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6466worker1 # [ 170.561332] nix-grpc-daemon[1721]: time=2026-09-21T13:56:31.199Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6467lb # [ 170.603367] niks3-server[1149]: 2026/09/21 13:56:31 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6468worker1 # [ 170.589526] nix-grpc-daemon[1721]: time=2026-09-21T13:56:31.229Z level=INFO msg="Uploading 1 narinfos"6469lb # [ 170.615548] niks3-server[1149]: 2026/09/21 13:56:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6470lb # [ 170.629198] niks3-server[1149]: 2026/09/21 13:56:31 INFO Signed narinfos id=60 count=16471lb # [ 170.631952] niks3-server[1149]: 2026/09/21 13:56:31 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6472worker1 # [ 170.615695] nix-grpc-daemon[1721]: time=2026-09-21T13:56:31.255Z level=INFO msg="Upload complete. (460ms)"6473lb # [ 170.639174] niks3-server[1149]: 2026/09/21 13:56:31 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6474lb # [ 170.644080] niks3-server[1149]: 2026/09/21 13:56:31 INFO Completed upload id=606475worker1 # [ 170.621146] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=0 paths=1 nar_bytes_in=6246476lb # [ 170.646856] niks3-server[1149]: 2026/09/21 13:56:31 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6477worker1 # [ 170.633975] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386478worker1 # [ 170.639555] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6479worker1 # [ 170.643784] nix-daemon[838]: reaped child process 1880, status = succeeded6480worker1 # [ 170.656643] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6481worker1 # [ 171.495766] nix-daemon[838]: reaped child process 1894, status = succeeded6482worker1 # [ 171.680795] nix-daemon[838]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6483worker1 # [ 171.698412] nix-daemon[838]: reaped child process 1913, status = succeeded6484lb # [ 171.734524] niks3-server[1149]: 2026/09/21 13:56:32 INFO Received uploads request method=POST path=/api/pending_closures6485worker1 # [ 171.722164] nix-grpc-daemon[1721]: time=2026-09-21T13:56:32.360Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6486worker1 # [ 171.723670] nix-grpc-daemon[1721]: time=2026-09-21T13:56:32.360Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6487lb # [ 171.772300] niks3-server[1149]: 2026/09/21 13:56:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6488worker1 # [ 171.758471] nix-grpc-daemon[1721]: time=2026-09-21T13:56:32.398Z level=INFO msg="Uploading 1 narinfos"6489lb # [ 171.786786] niks3-server[1149]: 2026/09/21 13:56:32 INFO Signed narinfos id=61 count=16490lb # [ 171.793612] niks3-server[1149]: 2026/09/21 13:56:32 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6491lb # [ 171.801922] niks3-server[1149]: 2026/09/21 13:56:32 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6492lb # [ 171.808304] niks3-server[1149]: 2026/09/21 13:56:32 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6493lb # [ 171.813571] niks3-server[1149]: 2026/09/21 13:56:32 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6494lb # [ 171.817963] niks3-server[1149]: 2026/09/21 13:56:32 INFO Completed upload id=616495worker1 # [ 171.795113] nix-grpc-daemon[1721]: time=2026-09-21T13:56:32.435Z level=INFO msg="Upload complete. (328ms)"6496lb # [ 171.820503] niks3-server[1149]: 2026/09/21 13:56:32 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6497worker1 # [ 171.813051] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16498worker1 # [ 171.825228] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:32Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16499worker1 # [ 171.834728] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386500worker1 # [ 171.838642] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:32Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06501worker1 # [ 171.842655] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386502worker1 # [ 171.847004] nix-daemon[838]: reaped child process 1892, status = succeeded6503worker1 # [ 171.858890] nix-grpc-daemon[1721]: time=2026-09-21T13:56:32.478Z level=INFO msg="All 1 paths already cached"6504worker1 # [ 171.862910] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6505worker1 # [ 171.866683] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561386506worker1 # [ 172.045986] nix-daemon[838]: accepted connection from pid 1919, user nix-grpc-daemon (trusted)6507worker1 # [ 172.064189] nix-daemon[838]: reaped child process 1922, status = succeeded6508worker1 # [ 172.237304] nix-daemon[838]: accepted connection from pid 1924, user nix-grpc-daemon (trusted)6509worker1 # [ 172.253295] nix-daemon[838]: reaped child process 1927, status = succeeded6510lb # [ 172.287677] niks3-server[1149]: 2026/09/21 13:56:32 INFO Received uploads request method=POST path=/api/pending_closures6511worker1 # [ 172.273276] nix-grpc-daemon[1721]: time=2026-09-21T13:56:32.913Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6512worker1 # [ 172.274753] nix-grpc-daemon[1721]: time=2026-09-21T13:56:32.913Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6513lb # [ 172.312614] niks3-server[1149]: 2026/09/21 13:56:32 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6514worker1 # [ 172.303823] nix-grpc-daemon[1721]: time=2026-09-21T13:56:32.943Z level=INFO msg="Uploading 1 narinfos"6515lb # [ 172.328474] niks3-server[1149]: 2026/09/21 13:56:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6516lb # [ 172.338953] niks3-server[1149]: 2026/09/21 13:56:32 INFO Signed narinfos id=62 count=16517lb # [ 172.349821] niks3-server[1149]: 2026/09/21 13:56:32 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6518worker1 # [ 172.332246] nix-grpc-daemon[1721]: time=2026-09-21T13:56:32.972Z level=INFO msg="Upload complete. (462ms)"6519lb # [ 172.355933] niks3-server[1149]: 2026/09/21 13:56:32 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6520lb # [ 172.360830] niks3-server[1149]: 2026/09/21 13:56:33 INFO Completed upload id=626521worker1 # [ 172.340253] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=0 paths=1 nar_bytes_in=6246522lb # [ 172.363941] niks3-server[1149]: 2026/09/21 13:56:33 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6523worker1 # [ 172.350309] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386524worker1 # [ 172.359497] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6525worker1 # [ 172.362133] nix-daemon[838]: reaped child process 1917, status = succeeded6526worker1 # [ 172.374428] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6527worker1 # [ 173.813214] nix-daemon[838]: reaped child process 1931, status = succeeded6528worker1 # [ 173.987418] nix-daemon[838]: accepted connection from pid 1947, user nix-grpc-daemon (trusted)6529worker1 # [ 174.005361] nix-daemon[838]: reaped child process 1950, status = succeeded6530lb # [ 174.040863] niks3-server[1149]: 2026/09/21 13:56:34 INFO Received uploads request method=POST path=/api/pending_closures6531worker1 # [ 174.028390] nix-grpc-daemon[1721]: time=2026-09-21T13:56:34.667Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6532worker1 # [ 174.029848] nix-grpc-daemon[1721]: time=2026-09-21T13:56:34.667Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6533lb # [ 174.079706] niks3-server[1149]: 2026/09/21 13:56:34 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6534lb # [ 174.092311] niks3-server[1149]: 2026/09/21 13:56:34 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6535worker1 # [ 174.079730] nix-grpc-daemon[1721]: time=2026-09-21T13:56:34.719Z level=INFO msg="Uploading 1 narinfos"6536lb # [ 174.103980] niks3-server[1149]: 2026/09/21 13:56:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6537lb # [ 174.112375] niks3-server[1149]: 2026/09/21 13:56:34 INFO Signed narinfos id=63 count=16538lb # [ 174.114513] niks3-server[1149]: 2026/09/21 13:56:34 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6539lb # [ 174.120399] niks3-server[1149]: 2026/09/21 13:56:34 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6540lb # [ 174.127687] niks3-server[1149]: 2026/09/21 13:56:34 INFO Completed upload id=636541worker1 # [ 174.104887] nix-grpc-daemon[1721]: time=2026-09-21T13:56:34.745Z level=INFO msg="Upload complete. (320ms)"6542lb # [ 174.130261] niks3-server[1149]: 2026/09/21 13:56:34 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6543worker1 # [ 174.119243] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16544worker1 # [ 174.138686] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:34Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16545worker1 # [ 174.142924] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386546worker1 # [ 174.147651] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:34Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06547worker1 # [ 174.152517] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386548worker1 # [ 174.156995] nix-daemon[838]: reaped child process 1929, status = succeeded6549worker1 # [ 174.159004] nix-grpc-daemon[1721]: time=2026-09-21T13:56:34.796Z level=INFO msg="All 2 paths already cached"6550worker1 # [ 174.163384] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561386551worker1 # [ 174.173725] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6552worker1 # [ 174.357971] nix-daemon[838]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)6553worker1 # [ 174.374902] nix-daemon[838]: reaped child process 1959, status = succeeded6554worker1 # [ 174.545879] nix-daemon[838]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)6555worker1 # [ 174.561999] nix-daemon[838]: reaped child process 1964, status = succeeded6556lb # [ 174.596544] niks3-server[1149]: 2026/09/21 13:56:35 INFO Received uploads request method=POST path=/api/pending_closures6557worker1 # [ 174.581978] nix-grpc-daemon[1721]: time=2026-09-21T13:56:35.221Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6558worker1 # [ 174.583466] nix-grpc-daemon[1721]: time=2026-09-21T13:56:35.222Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6559lb # [ 174.621989] niks3-server[1149]: 2026/09/21 13:56:35 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6560worker1 # [ 174.610707] nix-grpc-daemon[1721]: time=2026-09-21T13:56:35.250Z level=INFO msg="Uploading 1 narinfos"6561lb # [ 174.636181] niks3-server[1149]: 2026/09/21 13:56:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6562lb # [ 174.645755] niks3-server[1149]: 2026/09/21 13:56:35 INFO Signed narinfos id=64 count=16563lb # [ 174.655947] niks3-server[1149]: 2026/09/21 13:56:35 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6564worker1 # [ 174.641207] nix-grpc-daemon[1721]: time=2026-09-21T13:56:35.281Z level=INFO msg="Upload complete. (459ms)"6565lb # [ 174.664200] niks3-server[1149]: 2026/09/21 13:56:35 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6566lb # [ 174.669217] niks3-server[1149]: 2026/09/21 13:56:35 INFO Completed upload id=646567lb # [ 174.672456] niks3-server[1149]: 2026/09/21 13:56:35 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6568worker1 # [ 174.650217] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=0 paths=1 nar_bytes_in=7526569worker1 # [ 174.661258] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386570worker1 # [ 174.668507] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6571worker1 # [ 174.671443] nix-daemon[838]: reaped child process 1954, status = succeeded6572worker1 # [ 174.681281] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6573worker1 # [ 176.126968] nix-daemon[838]: reaped child process 1968, status = succeeded6574worker1 # [ 176.311425] nix-daemon[838]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)6575worker1 # [ 176.327814] nix-daemon[838]: reaped child process 1987, status = succeeded6576lb # [ 176.363922] niks3-server[1149]: 2026/09/21 13:56:37 INFO Received uploads request method=POST path=/api/pending_closures6577worker1 # [ 176.351947] nix-grpc-daemon[1721]: time=2026-09-21T13:56:36.990Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6578worker1 # [ 176.354242] nix-grpc-daemon[1721]: time=2026-09-21T13:56:36.990Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6579lb # [ 176.404198] niks3-server[1149]: 2026/09/21 13:56:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6580worker1 # [ 176.392916] nix-grpc-daemon[1721]: time=2026-09-21T13:56:37.032Z level=INFO msg="Uploading 1 narinfos"6581lb # [ 176.418620] niks3-server[1149]: 2026/09/21 13:56:37 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6582lb # [ 176.428301] niks3-server[1149]: 2026/09/21 13:56:37 INFO Signed narinfos id=65 count=16583lb # [ 176.434243] niks3-server[1149]: 2026/09/21 13:56:37 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6584lb # [ 176.437758] niks3-server[1149]: 2026/09/21 13:56:37 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6585lb # [ 176.442110] niks3-server[1149]: 2026/09/21 13:56:37 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6586lb # [ 176.446143] niks3-server[1149]: 2026/09/21 13:56:37 INFO Completed upload id=656587lb # [ 176.450245] niks3-server[1149]: 2026/09/21 13:56:37 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6588worker1 # [ 176.418859] nix-grpc-daemon[1721]: time=2026-09-21T13:56:37.059Z level=INFO msg="Upload complete. (325ms)"6589worker1 # [ 176.433777] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16590worker1 # [ 176.446008] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:37Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16591worker1 # [ 176.450305] nix-daemon[838]: reaped child process 1966, status = succeeded6592worker1 # [ 176.477591] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386593worker1 # [ 176.493569] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386594worker1 # [ 176.506641] nix-grpc-daemon[1721]: time=2026-09-21T13:56:37.146Z level=INFO msg="All 2 paths already cached"6595worker1 # [ 176.515043] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561386596worker1 # [ 176.528296] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6597worker1 # [ 176.732166] nix-daemon[838]: accepted connection from pid 1991, user nix-grpc-daemon (trusted)6598worker1 # [ 176.748487] nix-daemon[838]: reaped child process 1994, status = succeeded6599worker1 # [ 176.922761] nix-daemon[838]: accepted connection from pid 1996, user nix-grpc-daemon (trusted)6600worker1 # [ 176.939499] nix-daemon[838]: reaped child process 1999, status = succeeded6601lb # [ 176.973881] niks3-server[1149]: 2026/09/21 13:56:37 INFO Received uploads request method=POST path=/api/pending_closures6602worker1 # [ 176.960100] nix-grpc-daemon[1721]: time=2026-09-21T13:56:37.600Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6603worker1 # [ 176.961560] nix-grpc-daemon[1721]: time=2026-09-21T13:56:37.600Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6604lb # [ 177.000942] niks3-server[1149]: 2026/09/21 13:56:37 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6605lb # [ 177.012541] niks3-server[1149]: 2026/09/21 13:56:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6606worker1 # [ 176.995017] nix-grpc-daemon[1721]: time=2026-09-21T13:56:37.635Z level=INFO msg="Uploading 1 narinfos"6607lb # [ 177.023492] niks3-server[1149]: 2026/09/21 13:56:37 INFO Signed narinfos id=66 count=16608lb # [ 177.030180] niks3-server[1149]: 2026/09/21 13:56:37 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6609lb # [ 177.039553] niks3-server[1149]: 2026/09/21 13:56:37 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6610lb # [ 177.046258] niks3-server[1149]: 2026/09/21 13:56:37 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6611lb # [ 177.051364] niks3-server[1149]: 2026/09/21 13:56:37 INFO Completed upload id=666612worker1 # [ 177.028583] nix-grpc-daemon[1721]: time=2026-09-21T13:56:37.668Z level=INFO msg="Upload complete. (484ms)"6613worker1 # [ 177.036413] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=0 paths=1 nar_bytes_in=7526614worker1 # [ 177.049591] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386615worker1 # [ 177.055390] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6616worker1 # [ 177.059942] nix-daemon[838]: reaped child process 1989, status = succeeded6617worker1 # [ 177.072311] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6618worker1 # [ 177.893603] nix-daemon[838]: reaped child process 2003, status = succeeded6619worker1 # [ 178.053137] nix-daemon[838]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6620worker1 # [ 178.070947] nix-daemon[838]: reaped child process 2022, status = succeeded6621lb # [ 178.106861] niks3-server[1149]: 2026/09/21 13:56:38 INFO Received uploads request method=POST path=/api/pending_closures6622worker1 # [ 178.095210] nix-grpc-daemon[1721]: time=2026-09-21T13:56:38.733Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6623worker1 # [ 178.096860] nix-grpc-daemon[1721]: time=2026-09-21T13:56:38.733Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6624lb # [ 178.143343] niks3-server[1149]: 2026/09/21 13:56:38 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6625lb # [ 178.153649] niks3-server[1149]: 2026/09/21 13:56:38 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6626lb # [ 178.164737] niks3-server[1149]: 2026/09/21 13:56:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6627worker1 # [ 178.147212] nix-grpc-daemon[1721]: time=2026-09-21T13:56:38.787Z level=INFO msg="Uploading 1 narinfos"6628lb # [ 178.170073] niks3-server[1149]: 2026/09/21 13:56:38 INFO Signed narinfos id=67 count=16629lb # [ 178.174945] niks3-server[1149]: 2026/09/21 13:56:38 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6630lb # [ 178.183228] niks3-server[1149]: 2026/09/21 13:56:38 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6631lb # [ 178.188582] niks3-server[1149]: 2026/09/21 13:56:38 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6632lb # [ 178.193145] niks3-server[1149]: 2026/09/21 13:56:38 INFO Completed upload id=676633worker1 # [ 178.170410] nix-grpc-daemon[1721]: time=2026-09-21T13:56:38.810Z level=INFO msg="Upload complete. (288ms)"6634worker1 # [ 178.180986] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=16635worker1 # [ 178.195230] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:38Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16636worker1 # [ 178.199400] nix-daemon[838]: reaped child process 2001, status = succeeded6637worker1 # [ 178.221226] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386638worker1 # [ 178.234991] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386639worker1 # [ 178.248375] nix-grpc-daemon[1721]: time=2026-09-21T13:56:38.888Z level=INFO msg="All 2 paths already cached"6640worker1 # [ 178.256690] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561386641worker1 # [ 178.269117] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6642worker1 # [ 178.464438] nix-daemon[838]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)6643worker1 # [ 178.483256] nix-daemon[838]: reaped child process 2029, status = succeeded6644worker1 # [ 178.656921] nix-daemon[838]: accepted connection from pid 2031, user nix-grpc-daemon (trusted)6645worker1 # [ 178.673710] nix-daemon[838]: reaped child process 2034, status = succeeded6646lb # [ 178.707962] niks3-server[1149]: 2026/09/21 13:56:39 INFO Received uploads request method=POST path=/api/pending_closures6647worker1 # [ 178.695249] nix-grpc-daemon[1721]: time=2026-09-21T13:56:39.335Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6648worker1 # [ 178.697120] nix-grpc-daemon[1721]: time=2026-09-21T13:56:39.335Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6649lb # [ 178.734982] niks3-server[1149]: 2026/09/21 13:56:39 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6650worker1 # [ 178.724809] nix-grpc-daemon[1721]: time=2026-09-21T13:56:39.364Z level=INFO msg="Uploading 1 narinfos"6651lb # [ 178.749799] niks3-server[1149]: 2026/09/21 13:56:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6652lb # [ 178.761261] niks3-server[1149]: 2026/09/21 13:56:39 INFO Signed narinfos id=68 count=16653lb # [ 178.766602] niks3-server[1149]: 2026/09/21 13:56:39 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6654worker1 # [ 178.752508] nix-grpc-daemon[1721]: time=2026-09-21T13:56:39.392Z level=INFO msg="Upload complete. (468ms)"6655lb # [ 178.776064] niks3-server[1149]: 2026/09/21 13:56:39 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6656lb # [ 178.781495] niks3-server[1149]: 2026/09/21 13:56:39 INFO Completed upload id=686657worker1 # [ 178.758515] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=0 paths=1 nar_bytes_in=7526658lb # [ 178.785138] niks3-server[1149]: 2026/09/21 13:56:39 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6659worker1 # [ 178.771453] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561386660worker1 # [ 178.777305] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6661worker1 # [ 178.782186] nix-daemon[838]: reaped child process 2024, status = succeeded6662worker1 # [ 178.794189] nix-daemon[838]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6663worker1 # [ 180.240918] nix-daemon[838]: reaped child process 2038, status = succeeded6664worker1 # [ 180.422735] nix-daemon[838]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)6665worker1 # [ 180.440146] nix-daemon[838]: reaped child process 2057, status = succeeded6666lb # [ 180.479302] niks3-server[1149]: 2026/09/21 13:56:41 INFO Received uploads request method=POST path=/api/pending_closures6667worker1 # [ 180.465828] nix-grpc-daemon[1721]: time=2026-09-21T13:56:41.105Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6668worker1 # [ 180.471274] nix-grpc-daemon[1721]: time=2026-09-21T13:56:41.107Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6669lb # [ 180.516852] niks3-server[1149]: 2026/09/21 13:56:41 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6670lb # [ 180.532237] niks3-server[1149]: 2026/09/21 13:56:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6671worker1 # [ 180.514786] nix-grpc-daemon[1721]: time=2026-09-21T13:56:41.154Z level=INFO msg="Uploading 1 narinfos"6672lb # [ 180.542932] niks3-server[1149]: 2026/09/21 13:56:41 INFO Signed narinfos id=69 count=16673lb # [ 180.549101] niks3-server[1149]: 2026/09/21 13:56:41 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6674lb # [ 180.554739] niks3-server[1149]: 2026/09/21 13:56:41 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6675lb # [ 180.561852] niks3-server[1149]: 2026/09/21 13:56:41 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6676lb # [ 180.565912] niks3-server[1149]: 2026/09/21 13:56:41 INFO Completed upload id=696677worker1 # [ 180.542599] nix-grpc-daemon[1721]: time=2026-09-21T13:56:41.182Z level=INFO msg="Upload complete. (333ms)"6678lb # [ 180.568373] niks3-server[1149]: 2026/09/21 13:56:41 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6679worker1 # [ 180.557014] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56138 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=16680worker1 # [ 180.570316] nix-daemon[838]: reaped child process 2036, status = succeeded6681client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 20.26 seconds)6682worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6683worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6684client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6685client # [ 180.773769] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6686client # Running as unit: gpu.service; invocation ID: 5e887e46e5ea49569f12dd151d3fcb3d6687client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6688lb: must succeed: curl -sf localhost:9901/clusters6689lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6690lb: must succeed: curl -sf localhost:9901/clusters6691lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6692worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6693worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6694worker1 # [ 181.023830] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:561386695client # [ 181.023985] nix[1421]: don't know how to build these paths:6696client # [ 181.025111] nix[1421]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6697worker2 # [ 181.033913] nix-grpc-daemon[1496]: ts=2026-09-21T13:56:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458086698client # [ 181.053675] nix[1421]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6699worker1 # [ 181.058064] nix-grpc-daemon[1656]: ts=2026-09-21T13:56:41Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06700lb # [ 181.910536] envoy[903]: conn peer=[2001:db8:1::1]:37392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27174 tx=11559 ms=199396701lb: must succeed: curl -sf localhost:9901/clusters6702lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6703lb: must succeed: curl -sf localhost:9901/clusters6704lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6705worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6706worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6707client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6708client # [ 211.055176] nix[1421]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6709client # [ 211.064749] nix[1421]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6710client # [ 211.112824] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6711client # [ 211.118909] systemd[1]: gpu.service: Failed with result 'exit-code'.6712client # [ 211.122254] systemd[1]: gpu.service: Consumed 179ms CPU time over 30.338s wall clock time, 35.5M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6713client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.24 seconds)6714client: must succeed: journalctl -u gpu -o cat6715client: (finished: must succeed: journalctl -u gpu -o cat, in 0.04 seconds)6716lb: must succeed: curl -sf localhost:9901/clusters6717lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6718lb: must succeed: curl -sf localhost:9901/clusters6719lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6720worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6721worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6722(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.35 seconds)6723subtest: low disk drains a worker and builds go to the other6724worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6725lb # [ 211.913621] envoy[903]: conn peer=[2001:db8:1::1]:54794 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300606726worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.10 seconds)6727worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6728worker1 # [ 213.328198] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:13Z level=info event=unhealthy reason=min_free6729worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.13 seconds)6730client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26731client # don't know how to build these paths:6732client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6733worker1 # [ 214.200661] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:561386734worker1 # [ 214.207256] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561386735worker1 # [ 214.233446] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:14Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06736worker1 # [ 214.239375] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:14Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16737worker2 # [ 214.282177] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086738worker2 # [ 214.298411] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458086739client # copying 1 paths to 'lb:50051'...6740client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6741worker2 # [ 214.314291] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:458086742worker2 # [ 214.330279] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6743worker2 # [ 214.543263] nix-daemon[864]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6744worker2 # [ 214.562641] nix-daemon[864]: reaped child process 1832, status = succeeded6745worker2 # [ 214.736130] nix-daemon[864]: accepted connection from pid 1834, user nix-grpc-daemon (trusted)6746worker2 # [ 214.753622] nix-daemon[864]: reaped child process 1837, status = succeeded6747lb # [ 214.790015] niks3-server[1149]: 2026/09/21 13:57:15 INFO Received uploads request method=POST path=/api/pending_closures6748worker2 # [ 214.775542] nix-grpc-daemon[1644]: time=2026-09-21T13:57:15.413Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6749worker2 # [ 214.777158] nix-grpc-daemon[1644]: time=2026-09-21T13:57:15.413Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6750lb # [ 214.818759] niks3-server[1149]: 2026/09/21 13:57:15 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6751lb # [ 214.838307] niks3-server[1149]: 2026/09/21 13:57:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6752worker2 # [ 214.820618] nix-grpc-daemon[1644]: time=2026-09-21T13:57:15.458Z level=INFO msg="Uploading 1 narinfos"6753lb # [ 214.850440] niks3-server[1149]: 2026/09/21 13:57:15 INFO Signed narinfos id=70 count=16754lb # [ 214.857493] niks3-server[1149]: 2026/09/21 13:57:15 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6755lb # [ 214.868541] niks3-server[1149]: 2026/09/21 13:57:15 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6756worker2 # [ 214.850254] nix-grpc-daemon[1644]: time=2026-09-21T13:57:15.488Z level=INFO msg="Upload complete. (502ms)"6757lb # [ 214.873856] niks3-server[1149]: 2026/09/21 13:57:15 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6758lb # [ 214.877997] niks3-server[1149]: 2026/09/21 13:57:15 INFO Completed upload id=706759worker2 # [ 214.860073] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=0 paths=1 nar_bytes_in=5126760worker2 # [ 214.871538] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086761worker2 # [ 214.881118] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6762worker2 # [ 214.883992] nix-daemon[864]: reaped child process 1827, status = succeeded6763client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6764worker2 # [ 214.895330] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6765client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6766client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6767client # farm-a> LOG-farm-a-drain6768worker2 # [ 216.146187] nix-daemon[864]: reaped child process 1841, status = succeeded6769worker2 # [ 216.324626] nix-daemon[864]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6770worker2 # [ 216.341035] nix-daemon[864]: reaped child process 1860, status = succeeded6771lb # [ 216.379811] niks3-server[1149]: 2026/09/21 13:57:17 INFO Received uploads request method=POST path=/api/pending_closures6772worker2 # [ 216.368264] nix-grpc-daemon[1644]: time=2026-09-21T13:57:17.004Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6773worker2 # [ 216.369742] nix-grpc-daemon[1644]: time=2026-09-21T13:57:17.004Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6774lb # [ 216.410723] niks3-server[1149]: 2026/09/21 13:57:17 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6775lb # [ 216.422582] niks3-server[1149]: 2026/09/21 13:57:17 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6776lb # [ 216.431257] niks3-server[1149]: 2026/09/21 13:57:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6777worker2 # [ 216.411382] nix-grpc-daemon[1644]: time=2026-09-21T13:57:17.049Z level=INFO msg="Uploading 1 narinfos"6778lb # [ 216.438713] niks3-server[1149]: 2026/09/21 13:57:17 INFO Signed narinfos id=71 count=16779lb # [ 216.443603] niks3-server[1149]: 2026/09/21 13:57:17 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6780lb # [ 216.450846] niks3-server[1149]: 2026/09/21 13:57:17 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6781lb # [ 216.457637] niks3-server[1149]: 2026/09/21 13:57:17 INFO Completed upload id=716782worker2 # [ 216.432092] nix-grpc-daemon[1644]: time=2026-09-21T13:57:17.070Z level=INFO msg="Upload complete. (323ms)"6783lb # [ 216.460199] niks3-server[1149]: 2026/09/21 13:57:17 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6784worker2 # [ 216.445129] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=16785worker1 # [ 216.450238] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:17Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06786worker1 # [ 216.456186] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:17Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16787worker2 # [ 216.464116] nix-daemon[864]: reaped child process 1839, status = succeeded6788worker2 # [ 216.495855] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086789worker2 # [ 216.510792] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458086790worker2 # [ 216.524833] nix-grpc-daemon[1644]: time=2026-09-21T13:57:17.163Z level=INFO msg="All 1 paths already cached"6791client # copying 1 paths to 'lb:50051'...6792client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6793worker2 # [ 216.534917] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:458086794worker2 # [ 216.549308] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6795worker2 # [ 216.752215] nix-daemon[864]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6796worker2 # [ 216.771203] nix-daemon[864]: reaped child process 1867, status = succeeded6797worker2 # [ 216.943610] nix-daemon[864]: accepted connection from pid 1869, user nix-grpc-daemon (trusted)6798worker2 # [ 216.959716] nix-daemon[864]: reaped child process 1872, status = succeeded6799lb # [ 216.995972] niks3-server[1149]: 2026/09/21 13:57:17 INFO Received uploads request method=POST path=/api/pending_closures6800worker2 # [ 216.981917] nix-grpc-daemon[1644]: time=2026-09-21T13:57:17.620Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6801worker2 # [ 216.983427] nix-grpc-daemon[1644]: time=2026-09-21T13:57:17.620Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6802lb # [ 217.024828] niks3-server[1149]: 2026/09/21 13:57:17 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6803worker2 # [ 217.012080] nix-grpc-daemon[1644]: time=2026-09-21T13:57:17.650Z level=INFO msg="Uploading 1 narinfos"6804lb # [ 217.040465] niks3-server[1149]: 2026/09/21 13:57:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6805lb # [ 217.051530] niks3-server[1149]: 2026/09/21 13:57:17 INFO Signed narinfos id=72 count=16806lb # [ 217.059659] niks3-server[1149]: 2026/09/21 13:57:17 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6807worker2 # [ 217.041313] nix-grpc-daemon[1644]: time=2026-09-21T13:57:17.679Z level=INFO msg="Upload complete. (475ms)"6808lb # [ 217.066695] niks3-server[1149]: 2026/09/21 13:57:17 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6809lb # [ 217.072794] niks3-server[1149]: 2026/09/21 13:57:17 INFO Completed upload id=726810worker2 # [ 217.048204] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=0 paths=1 nar_bytes_in=6326811lb # [ 217.076520] niks3-server[1149]: 2026/09/21 13:57:17 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6812worker2 # [ 217.058948] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086813worker2 # [ 217.062073] nix-daemon[864]: reaped child process 1862, status = succeeded6814worker2 # [ 217.068943] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6815client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6816worker2 # [ 217.081236] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6817client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6818client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6819client # farm-b> LOG-farm-b-drain6820worker2 # [ 218.706908] nix-daemon[864]: reaped child process 1876, status = succeeded6821worker2 # [ 218.872392] nix-daemon[864]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6822worker2 # [ 218.891374] nix-daemon[864]: reaped child process 1895, status = succeeded6823lb # [ 218.927915] niks3-server[1149]: 2026/09/21 13:57:19 INFO Received uploads request method=POST path=/api/pending_closures6824worker2 # [ 218.916290] nix-grpc-daemon[1644]: time=2026-09-21T13:57:19.551Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6825worker2 # [ 218.917789] nix-grpc-daemon[1644]: time=2026-09-21T13:57:19.551Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6826lb # [ 218.958987] niks3-server[1149]: 2026/09/21 13:57:19 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6827lb # [ 218.976799] niks3-server[1149]: 2026/09/21 13:57:19 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6828lb # [ 218.986664] niks3-server[1149]: 2026/09/21 13:57:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6829worker2 # [ 218.968098] nix-grpc-daemon[1644]: time=2026-09-21T13:57:19.606Z level=INFO msg="Uploading 1 narinfos"6830lb # [ 218.996274] niks3-server[1149]: 2026/09/21 13:57:19 INFO Signed narinfos id=73 count=16831lb # [ 218.999089] niks3-server[1149]: 2026/09/21 13:57:19 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6832lb # [ 219.009554] niks3-server[1149]: 2026/09/21 13:57:19 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6833lb # [ 219.016874] niks3-server[1149]: 2026/09/21 13:57:19 INFO Completed upload id=736834worker2 # [ 218.992722] nix-grpc-daemon[1644]: time=2026-09-21T13:57:19.631Z level=INFO msg="Upload complete. (301ms)"6835lb # [ 219.019697] niks3-server[1149]: 2026/09/21 13:57:19 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6836worker2 # [ 219.005029] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=16837worker1 # [ 219.011093] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:19Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06838worker1 # [ 219.016963] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:19Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16839worker2 # [ 219.020710] nix-daemon[864]: reaped child process 1874, status = succeeded6840worker2 # [ 219.055894] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086841worker2 # [ 219.070727] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:458086842worker2 # [ 219.085067] nix-grpc-daemon[1644]: time=2026-09-21T13:57:19.723Z level=INFO msg="All 2 paths already cached"6843client # copying 1 paths to 'lb:50051'...6844client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6845worker2 # [ 219.095229] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:458086846worker2 # [ 219.109396] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6847worker2 # [ 219.313790] nix-daemon[864]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6848worker2 # [ 219.330531] nix-daemon[864]: reaped child process 1902, status = succeeded6849worker2 # [ 219.502847] nix-daemon[864]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)6850worker2 # [ 219.519140] nix-daemon[864]: reaped child process 1907, status = succeeded6851lb # [ 219.554751] niks3-server[1149]: 2026/09/21 13:57:20 INFO Received uploads request method=POST path=/api/pending_closures6852worker2 # [ 219.539856] nix-grpc-daemon[1644]: time=2026-09-21T13:57:20.178Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6853worker2 # [ 219.541454] nix-grpc-daemon[1644]: time=2026-09-21T13:57:20.178Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6854lb # [ 219.581769] niks3-server[1149]: 2026/09/21 13:57:20 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6855lb # [ 219.595479] niks3-server[1149]: 2026/09/21 13:57:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6856worker2 # [ 219.577559] nix-grpc-daemon[1644]: time=2026-09-21T13:57:20.215Z level=INFO msg="Uploading 1 narinfos"6857lb # [ 219.606206] niks3-server[1149]: 2026/09/21 13:57:20 INFO Signed narinfos id=74 count=16858lb # [ 219.612795] niks3-server[1149]: 2026/09/21 13:57:20 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6859lb # [ 219.620204] niks3-server[1149]: 2026/09/21 13:57:20 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6860lb # [ 219.625844] niks3-server[1149]: 2026/09/21 13:57:20 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6861lb # [ 219.630557] niks3-server[1149]: 2026/09/21 13:57:20 INFO Completed upload id=746862worker2 # [ 219.606367] nix-grpc-daemon[1644]: time=2026-09-21T13:57:20.244Z level=INFO msg="Upload complete. (480ms)"6863worker2 # [ 219.613569] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=0 paths=1 nar_bytes_in=7606864worker2 # [ 219.626601] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:458086865worker2 # [ 219.633316] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6866worker2 # [ 219.638310] nix-daemon[864]: reaped child process 1897, status = succeeded6867client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6868client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6869worker2 # [ 219.651748] nix-daemon[864]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6870client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6871client # farm-top> LOG-farm-top-drain6872worker2 # [ 220.697355] nix-daemon[864]: reaped child process 1911, status = succeeded6873worker2 # [ 220.877027] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6874worker2 # [ 220.894484] nix-daemon[864]: reaped child process 1930, status = succeeded6875lb # [ 220.931269] niks3-server[1149]: 2026/09/21 13:57:21 INFO Received uploads request method=POST path=/api/pending_closures6876worker2 # [ 220.919110] nix-grpc-daemon[1644]: time=2026-09-21T13:57:21.556Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6877worker2 # [ 220.920832] nix-grpc-daemon[1644]: time=2026-09-21T13:57:21.556Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6878lb # [ 220.970027] niks3-server[1149]: 2026/09/21 13:57:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6879worker2 # [ 220.954457] nix-grpc-daemon[1644]: time=2026-09-21T13:57:21.592Z level=INFO msg="Uploading 1 narinfos"6880lb # [ 220.987563] niks3-server[1149]: 2026/09/21 13:57:21 INFO Signed narinfos id=75 count=16881lb # [ 220.999402] niks3-server[1149]: 2026/09/21 13:57:21 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6882lb # [ 221.007841] niks3-server[1149]: 2026/09/21 13:57:21 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6883lb # [ 221.016309] niks3-server[1149]: 2026/09/21 13:57:21 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6884worker2 # [ 220.996477] nix-grpc-daemon[1644]: time=2026-09-21T13:57:21.634Z level=INFO msg="Upload complete. (333ms)"6885lb # [ 221.020746] niks3-server[1149]: 2026/09/21 13:57:21 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6886lb # [ 221.024835] niks3-server[1149]: 2026/09/21 13:57:21 INFO Completed upload id=756887lb # [ 221.028675] niks3-server[1149]: 2026/09/21 13:57:21 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6888worker2 # [ 221.008867] nix-grpc-daemon[1496]: ts=2026-09-21T13:57:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45808 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=16889worker2 # [ 221.022331] nix-daemon[864]: reaped child process 1909, status = succeeded6890client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 7.12 seconds)6891client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6892client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.22 seconds)6893worker1: must succeed: rm /nix/.rw-store/fill6894worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.21 seconds)6895worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6896lb # [ 221.916343] envoy[903]: conn peer=[2001:db8:1::1]:33124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9549 tx=4086 ms=68266897worker1 # [ 222.336575] nix-grpc-daemon[1656]: ts=2026-09-21T13:57:22Z level=info event=healthy reason=min_free6898worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.19 seconds)6899(finished: subtest: low disk drains a worker and builds go to the other, in 11.07 seconds)6900(finished: run the VM test script, in 223.69 seconds)6901test script finished in 224.05s6902cleanup6903kill QemuMachine (pid 49)6904client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6905kill QemuMachine (pid 51)6906lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6907kill QemuMachine (pid 50)6908worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6909kill QemuMachine (pid 52)6910worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6911(finished: cleanup, in 1.06 seconds)6912additionally exposed symbols:6913 client, lb, worker1, worker2,6914 vlan1,6915 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh