vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #431
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13worker2 # Disk image does not exist, creating the virtualisation disk image...14client # Disk image does not exist, creating the virtualisation disk image...15worker2: QEMU running (pid 52)16client: QEMU running (pid 49)17lb # Formatting '/build/vm-state-lb/tmp.wDJYm77xV3', fmt=raw size=107374182418lb: QEMU running (pid 50)19worker1: QEMU running (pid 51)20client # Formatting '/build/vm-state-client/tmp.yrCSPcpXPt', fmt=raw size=107374182421worker1 # Disk image does not exist, creating the virtualisation disk image...22worker2 # Formatting '/build/vm-state-worker2/tmp.MdEG3PCqCw', fmt=raw size=107374182423client # mke2fs 1.47.4 (6-Mar-2025)24lb # mke2fs 1.47.4 (6-Mar-2025)25worker1 # Formatting '/build/vm-state-worker1/tmp.hiTn7oEZ3v', fmt=raw size=107374182426client # Discarding device blocks: 0/262144 done27lb # Discarding device blocks: 0/262144 done28worker1 # mke2fs 1.47.4 (6-Mar-2025)29client # Creating filesystem with 262144 4k blocks and 65536 inodes30lb # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # Discarding device blocks: 0/262144 done32client # Filesystem UUID: 482e7b32-98b3-4298-aeed-cfe3f41a78b733lb # Filesystem UUID: 7f67a60e-a8b8-4e9f-b907-f8646fb6afc734worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes35client # Superblock backups stored on blocks:36lb # Superblock backups stored on blocks:37worker1 # Filesystem UUID: 376254bc-797f-4e24-bd30-4d24e02b296838client # 32768, 98304, 163840, 22937639lb # 32768, 98304, 163840, 22937640worker1 # Superblock backups stored on blocks:41client # 42lb # 43worker1 # 32768, 98304, 163840, 22937644client # Allocating group tables: 0/8 done45lb # Allocating group tables: 0/8 done46worker1 # 47client # Writing inode tables: 0/8 done48lb # Writing inode tables: 0/8 done49worker1 # Allocating group tables: 0/8 done50client # Creating journal (8192 blocks): done51lb # Creating journal (8192 blocks): done52worker1 # Writing inode tables: 0/8 done53client # Writing superblocks and filesystem accounting information: 0/8 done54lb # Writing superblocks and filesystem accounting information: 0/82/8 done55worker1 # Creating journal (8192 blocks): done56client # 57lb # 58worker1 # Writing superblocks and filesystem accounting information: 0/8 done59client # Virtualisation disk image created.60lb # Virtualisation disk image created.61worker1 # 62(finished: start all VMs, in 0.48 seconds)63worker1 # Virtualisation disk image created.64client: waiting for unit network-addresses-eth1.service65client: waiting for the VM to finish booting66worker2 # mke2fs 1.47.4 (6-Mar-2025)67worker2 # Discarding device blocks: 0/262144 done68worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes69worker2 # Filesystem UUID: 077b16bd-bcb2-4770-9b57-70bfa82bc1bd70worker2 # Superblock backups stored on blocks:71worker2 # 32768, 98304, 163840, 22937672worker2 # 73worker2 # Allocating group tables: 0/8 done74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681lb # [ 0.000000] KASLR enabled82lb # [ 0.000000] random: crng init done83lb # [ 0.000000] Machine model: linux,dummy-virt84lb # [ 0.000000] efi: UEFI not found.85lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88lb # [ 0.000000] Zone ranges:89lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90lb # [ 0.000000] DMA32 empty91lb # [ 0.000000] Normal empty92lb # [ 0.000000] Device empty93lb # [ 0.000000] Movable zone start for each node94lb # [ 0.000000] Early memory node ranges95lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098lb # [ 0.000000] psci: probing for conduit method from DT.99lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.100lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101lb # [ 0.000000] psci: Trusted OS migration not required102lb # [ 0.000000] psci: SMC Calling Convention v1.1103lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105lb # [ 0.000000] Detected PIPT I-cache on CPU0106lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107lb # [ 0.000000] CPU features: detected: GICv3 CPU interface108lb # [ 0.000000] CPU features: detected: Spectre-v4109lb # [ 0.000000] CPU features: detected: Spectre-BHB110lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112lb # [ 0.000000] alternatives: applying boot alternatives113lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/px4c3ifawqq4xs8a4zhy519i865fkk09-nixos-system-lb-test/init regInfo=/nix/store/qp69c7d7ks4qa26wilaji9gbrwmdkkah-closure-info/registration console=ttyAMA0,115200n8 console=tty0114lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qp69c7d7ks4qa26wilaji9gbrwmdkkah-closure-info/registration", will be passed to user space.115lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119lb # [ 0.000000] software IO TLB: area num 1.120lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)122worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]123lb # [ 0.000000] Fallback order for Node 0: 0124lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216125lb # [ 0.000000] Policy zone: DMA126worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026127worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]128worker1 # [ 0.000000] KASLR enabled129lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off130worker1 # [ 0.000000] random: crng init done131worker1 # [ 0.000000] Machine model: linux,dummy-virt132lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1133worker1 # [ 0.000000] efi: UEFI not found.134client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]135worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026136lb # [ 0.000000] allocated 3145728 bytes of page_ext137worker2 # [ 0.000000] KASLR enabled138worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT139worker2 # [ 0.000000] random: crng init done140lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages141client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026142worker2 # [ 0.000000] Machine model: linux,dummy-virt143worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]144client # [ 0.000000] KASLR enabled145lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups146worker2 # [ 0.000000] efi: UEFI not found.147client # [ 0.000000] random: crng init done148lb # [ 0.000000] rcu: Hierarchical RCU implementation.149worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]150worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT151worker1 # [ 0.000000] Zone ranges:152lb # [ 0.000000] rcu: RCU event tracing is enabled.153client # [ 0.000000] Machine model: linux,dummy-virt154client # [ 0.000000] efi: UEFI not found.155worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]156worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]157lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.158worker1 # [ 0.000000] DMA32 empty159client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT160lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.161worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]162worker1 # [ 0.000000] Normal empty163worker2 # [ 0.000000] Zone ranges:164worker1 # [ 0.000000] Device empty165lb # [ 0.000000] Rude variant of Tasks RCU enabled.166client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]167worker1 # [ 0.000000] Movable zone start for each node168worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]169lb # [ 0.000000] Tracing variant of Tasks RCU enabled.170client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]171worker1 # [ 0.000000] Early memory node ranges172worker2 # [ 0.000000] DMA32 empty173client # [ 0.000000] Zone ranges:174worker2 # [ 0.000000] Normal empty175worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]176lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.177worker2 # [ 0.000000] Device empty178client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]179worker2 # [ 0.000000] Movable zone start for each node180client # [ 0.000000] DMA32 empty181lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1182worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]183worker2 # [ 0.000000] Early memory node ranges184client # [ 0.000000] Normal empty185client # [ 0.000000] Device empty186worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000187worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]188lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.189client # [ 0.000000] Movable zone start for each node190worker1 # [ 0.000000] psci: probing for conduit method from DT.191client # [ 0.000000] Early memory node ranges192worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]193worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.194lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.195client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]196worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000197worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs198client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]199worker1 # [ 0.000000] psci: Trusted OS migration not required200worker2 # [ 0.000000] psci: probing for conduit method from DT.201lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.202worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.203worker1 # [ 0.000000] psci: SMC Calling Convention v1.1204client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000205lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0206worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs207client # [ 0.000000] psci: probing for conduit method from DT.208lb # [ 0.000000] GICv3: 256 SPIs implemented209worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)210worker2 # [ 0.000000] psci: Trusted OS migration not required211client # [ 0.000000] psci: PSCIv1.3 detected in firmware.212lb # [ 0.000000] GICv3: 0 Extended SPIs implemented213worker2 # [ 0.000000] psci: SMC Calling Convention v1.1214lb # [ 0.000000] Root IRQ handler: gic_handle_irq215client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs216worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296217worker1 # [ 0.000000] Detected PIPT I-cache on CPU0218lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI219client # [ 0.000000] psci: Trusted OS migration not required220worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)221client # [ 0.000000] psci: SMC Calling Convention v1.1222lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0223worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)224worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296225lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000226worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface227worker2 # [ 0.000000] Detected PIPT I-cache on CPU0228client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)229lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]230worker1 # [ 0.000000] CPU features: detected: Spectre-v4231client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296232worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)233worker1 # [ 0.000000] CPU features: detected: Spectre-BHB234client # [ 0.000000] Detected PIPT I-cache on CPU0235lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)236worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface237worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38238worker2 # [ 0.000000] CPU features: detected: Spectre-v4239client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)240worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23241worker2 # [ 0.000000] CPU features: detected: Spectre-BHB242lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)243client # [ 0.000000] CPU features: detected: GICv3 CPU interface244worker1 # [ 0.000000] alternatives: applying boot alternatives245lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000246worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38247client # [ 0.000000] CPU features: detected: Spectre-v4248client # [ 0.000000] CPU features: detected: Spectre-BHB249worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23250lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000251client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38252worker2 # [ 0.000000] alternatives: applying boot alternatives253lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.254client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23255client # [ 0.000000] alternatives: applying boot alternatives256lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns257lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).258worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/q0xpda7nssqkdhnmlkp5q4dxx9dxngjp-nixos-system-worker1-test/init regInfo=/nix/store/fzq8f8i8j8bdnczah2k202f5ss3lk651-closure-info/registration console=ttyAMA0,115200n8 console=tty0259lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns260worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a4g43incxmb90ykmkdsd5cyqwyy4lipw-nixos-system-worker2-test/init regInfo=/nix/store/wdwfc23551pfjsw81kz0k6mfm1w1gfyj-closure-info/registration console=ttyAMA0,115200n8 console=tty0261worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/fzq8f8i8j8bdnczah2k202f5ss3lk651-closure-info/registration", will be passed to user space.262lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns263worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes264lb # [ 0.000039] arm-pv: using stolen time PV265client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/c59dy5901id87cijqm6p1wgx5fgpmqry-nixos-system-client-test/init regInfo=/nix/store/pmfwpcq6npqfsdnqpnd837cg39b7406m-closure-info/registration console=ttyAMA0,115200n8 console=tty0266worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)267worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/wdwfc23551pfjsw81kz0k6mfm1w1gfyj-closure-info/registration", will be passed to user space.268lb # [ 0.000727] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)269client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/pmfwpcq6npqfsdnqpnd837cg39b7406m-closure-info/registration", will be passed to user space.270worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)271lb # [ 0.000899] Console: colour dummy device 80x25272worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes273lb # [ 0.000908] printk: legacy console [tty0] enabled274worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB275client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes276worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)277worker1 # [ 0.000000] software IO TLB: area num 1.278client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)279lb # [ 0.001110] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)280worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)281worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB282lb # [ 0.001118] pid_max: default: 32768 minimum: 301283client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)284worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB285worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)286lb # [ 0.001186] LSM: initializing lsm=capability,landlock,yama,bpf,ima287worker2 # [ 0.000000] software IO TLB: area num 1.288client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB289worker1 # [ 0.000000] Fallback order for Node 0: 0290lb # [ 0.001379] landlock: Up and running.291client # [ 0.000000] software IO TLB: area num 1.292lb # [ 0.001381] Yama: becoming mindful.293worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB294worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216295lb # [ 0.002103] LSM support for eBPF active296worker1 # [ 0.000000] Policy zone: DMA297client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB298worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)299lb # [ 0.002287] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)300worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off301worker2 # [ 0.000000] Fallback order for Node 0: 0302client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)303client # [ 0.000000] Fallback order for Node 0: 0304lb # [ 0.002320] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)305worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1306worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216307worker2 # [ 0.000000] Policy zone: DMA308worker1 # [ 0.000000] allocated 3145728 bytes of page_ext309lb # [ 0.003560] cacheinfo: Unable to detect cache hierarchy for CPU 0310client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216311client # [ 0.000000] Policy zone: DMA312worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off313worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages314lb # [ 0.004297] rcu: Hierarchical SRCU implementation.315client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off316worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups317lb # [ 0.004301] rcu: Max phase no-delay instances is 1000.318worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1319worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.320lb # [ 0.005501] fsl-mc MSI: its@8080000 domain created321worker2 # [ 0.000000] allocated 3145728 bytes of page_ext322client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1323worker1 # [ 0.000000] rcu: RCU event tracing is enabled.324lb # [ 0.005599] EFI services will not be available.325client # [ 0.000000] allocated 3145728 bytes of page_ext326worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages327lb # [ 0.005682] smp: Bringing up secondary CPUs ...328client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages329worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.330worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups331lb # [ 0.005692] smp: Brought up 1 node, 1 CPU332worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.333client # [ 0.000000] ftrace: allocated 294 pages with 4 groups334worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.335lb # [ 0.005695] SMP: Total of 1 processors activated.336client # [ 0.000000] rcu: Hierarchical RCU implementation.337worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.338worker2 # [ 0.000000] rcu: RCU event tracing is enabled.339lb # [ 0.005700] CPU: All CPU(s) started at EL1340client # [ 0.000000] rcu: RCU event tracing is enabled.341worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.342worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.343lb # [ 0.005712] CPU features: detected: Branch Target Identification344client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.345worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.346worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.347lb # [ 0.005717] CPU features: detected: ARMv8.4 Translation Table Level348worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.349client # [ 0.000000] Trampoline variant of Tasks RCU enabled.350worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1351client # [ 0.000000] Rude variant of Tasks RCU enabled.352worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.353lb # [ 0.005720] CPU features: detected: Instruction cache invalidation not required for I/D coherence354client # [ 0.000000] Tracing variant of Tasks RCU enabled.355worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.356worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.357lb # [ 0.005724] CPU features: detected: Data cache clean to the PoU not required for I/D coherence358client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.359worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1360worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.361lb # [ 0.005727] CPU features: detected: Common not Private translations362client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1363worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.364lb # [ 0.005730] CPU features: detected: CRC32 instructions365worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.366client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.367lb # [ 0.005733] CPU features: detected: Data cache clean to Point of Deep Persistence368worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0369worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.370worker1 # [ 0.000000] GICv3: 256 SPIs implemented371client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.372lb # [ 0.005737] CPU features: detected: Data cache clean to Point of Persistence373worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented374worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.375worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq376lb # [ 0.005740] CPU features: detected: Data independent timing control (DIT)377client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.378worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0379lb # [ 0.005743] CPU features: detected: E0PD380worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI381worker2 # [ 0.000000] GICv3: 256 SPIs implemented382client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0383worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0384worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented385lb # [ 0.005745] CPU features: detected: Enhanced Counter Virtualization386client # [ 0.000000] GICv3: 256 SPIs implemented387worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq388worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000389client # [ 0.000000] GICv3: 0 Extended SPIs implemented390lb # [ 0.005748] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)391client # [ 0.000000] Root IRQ handler: gic_handle_irq392worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI393worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]394lb # [ 0.005751] CPU features: detected: Enhanced Virtualization Traps395worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0396client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI397lb # [ 0.005755] CPU features: detected: Fine Grained Traps398client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0399worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)400worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000401worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]402client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000403lb # [ 0.005759] CPU features: detected: Generic authentication (architected QARMA5 algorithm)404client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]405worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)406lb # [ 0.005764] CPU features: detected: RCpc load-acquire (LDAPR)407worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)408worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000409lb # [ 0.005767] CPU features: detected: LSE atomic instructions410client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)411worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000412lb # [ 0.005770] CPU features: detected: Privileged Access Never413worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)414lb # [ 0.005773] CPU features: detected: PMUv3415client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)416worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.417worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000418lb # [ 0.005776] CPU features: detected: RAS Extension Support419client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000420worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000421lb # [ 0.005778] CPU features: detected: RASv1p1 Extension Support422worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns423client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000424lb # [ 0.005781] CPU features: detected: Random Number Generator425worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.426worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).427client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.428lb # [ 0.005783] CPU features: detected: Speculation barrier (SB)429worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns430worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns431lb # [ 0.005787] CPU features: detected: Stage-2 Force Write-Back432client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns433worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).434lb # [ 0.005789] CPU features: detected: TLB range maintenance instructions435client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).436worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns437worker1 # [ 0.000034] arm-pv: using stolen time PV438worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns439lb # [ 0.005794] CPU features: detected: Speculative Store Bypassing Safe (SSBS)440client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns441lb # [ 0.005834] alternatives: applying system-wide alternatives442worker1 # [ 0.000462] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)443worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns444worker2 # [ 0.000031] arm-pv: using stolen time PV445lb # [ 0.009040] CPU features: detected: BBM Level 2 without TLB conflict abort446worker1 # [ 0.000633] Console: colour dummy device 80x25447client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns448client # [ 0.000029] arm-pv: using stolen time PV449worker1 # [ 0.000641] printk: legacy console [tty0] enabled450worker2 # [ 0.000427] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)451lb # [ 0.009245] Memory: 1406580K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)452client # [ 0.000470] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)453worker2 # [ 0.000633] Console: colour dummy device 80x25454worker1 # [ 0.000848] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)455lb # [ 0.009717] devtmpfs: initialized456worker2 # [ 0.000641] printk: legacy console [tty0] enabled457client # [ 0.000637] Console: colour dummy device 80x25458worker1 # [ 0.000859] pid_max: default: 32768 minimum: 301459client # [ 0.000645] printk: legacy console [tty0] enabled460lb # [ 0.011709] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)461worker1 # [ 0.000943] LSM: initializing lsm=capability,landlock,yama,bpf,ima462worker2 # [ 0.000853] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)463worker1 # [ 0.001068] landlock: Up and running.464lb # [ 0.011733] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).465worker2 # [ 0.000862] pid_max: default: 32768 minimum: 301466worker1 # [ 0.001070] Yama: becoming mindful.467client # [ 0.000837] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)468worker1 # [ 0.001576] LSM support for eBPF active469client # [ 0.000844] pid_max: default: 32768 minimum: 301470lb # [ 0.011940] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL471worker2 # [ 0.000944] LSM: initializing lsm=capability,landlock,yama,bpf,ima472worker2 # [ 0.001120] landlock: Up and running.473client # [ 0.000920] LSM: initializing lsm=capability,landlock,yama,bpf,ima474worker1 # [ 0.001762] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)475worker2 # [ 0.001126] Yama: becoming mindful.476client # [ 0.001065] landlock: Up and running.477worker2 # [ 0.001629] LSM support for eBPF active478client # [ 0.001068] Yama: becoming mindful.479worker1 # [ 0.001804] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)480client # [ 0.001545] LSM support for eBPF active481worker2 # [ 0.001765] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)482worker1 # [ 0.002959] cacheinfo: Unable to detect cache hierarchy for CPU 0483client # [ 0.001692] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)484worker1 # [ 0.003790] rcu: Hierarchical SRCU implementation.485worker2 # [ 0.001798] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)486worker1 # [ 0.003796] rcu: Max phase no-delay instances is 1000.487client # [ 0.001725] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)488worker2 # [ 0.003122] cacheinfo: Unable to detect cache hierarchy for CPU 0489worker1 # [ 0.005084] fsl-mc MSI: its@8080000 domain created490worker2 # [ 0.003865] rcu: Hierarchical SRCU implementation.491client # [ 0.002892] cacheinfo: Unable to detect cache hierarchy for CPU 0492worker1 # [ 0.005170] EFI services will not be available.493client # [ 0.003625] rcu: Hierarchical SRCU implementation.494worker2 # [ 0.003869] rcu: Max phase no-delay instances is 1000.495worker1 # [ 0.005250] smp: Bringing up secondary CPUs ...496worker2 # [ 0.005130] fsl-mc MSI: its@8080000 domain created497worker1 # [ 0.005259] smp: Brought up 1 node, 1 CPU498client # [ 0.003629] rcu: Max phase no-delay instances is 1000.499worker2 # [ 0.005220] EFI services will not be available.500client # [ 0.004904] fsl-mc MSI: its@8080000 domain created501worker1 # [ 0.005262] SMP: Total of 1 processors activated.502worker2 # [ 0.005299] smp: Bringing up secondary CPUs ...503worker1 # [ 0.005267] CPU: All CPU(s) started at EL1504client # [ 0.004999] EFI services will not be available.505worker2 # [ 0.005308] smp: Brought up 1 node, 1 CPU506client # [ 0.005121] smp: Bringing up secondary CPUs ...507worker1 # [ 0.005277] CPU features: detected: Branch Target Identification508worker2 # [ 0.005311] SMP: Total of 1 processors activated.509client # [ 0.005131] smp: Brought up 1 node, 1 CPU510worker2 # [ 0.005317] CPU: All CPU(s) started at EL1511worker1 # [ 0.005282] CPU features: detected: ARMv8.4 Translation Table Level512client # [ 0.005134] SMP: Total of 1 processors activated.513lb # [ 0.011945] 0 pages in range for non-PLT usage514client # [ 0.005142] CPU: All CPU(s) started at EL1515worker2 # [ 0.005329] CPU features: detected: Branch Target Identification516worker1 # [ 0.005285] CPU features: detected: Instruction cache invalidation not required for I/D coherence517lb # [ 0.011946] 508288 pages in range for PLT usage518client # [ 0.005153] CPU features: detected: Branch Target Identification519worker2 # [ 0.005333] CPU features: detected: ARMv8.4 Translation Table Level520lb # [ 0.012072] pinctrl core: initialized pinctrl subsystem521worker1 # [ 0.005289] CPU features: detected: Data cache clean to the PoU not required for I/D coherence522client # [ 0.005158] CPU features: detected: ARMv8.4 Translation Table Level523lb # [ 0.012976] DMI not present or invalid.524worker2 # [ 0.005337] CPU features: detected: Instruction cache invalidation not required for I/D coherence525worker1 # [ 0.005293] CPU features: detected: Common not Private translations526lb # [ 0.017076] NET: Registered PF_NETLINK/PF_ROUTE protocol family527client # [ 0.005161] CPU features: detected: Instruction cache invalidation not required for I/D coherence528worker1 # [ 0.005296] CPU features: detected: CRC32 instructions529worker2 # [ 0.005340] CPU features: detected: Data cache clean to the PoU not required for I/D coherence530lb # [ 0.019365] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations531client # [ 0.005165] CPU features: detected: Data cache clean to the PoU not required for I/D coherence532worker2 # [ 0.005344] CPU features: detected: Common not Private translations533worker1 # [ 0.005298] CPU features: detected: Data cache clean to Point of Deep Persistence534lb # [ 0.019550] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations535worker2 # [ 0.005347] CPU features: detected: CRC32 instructions536client # [ 0.005168] CPU features: detected: Common not Private translations537worker1 # [ 0.005302] CPU features: detected: Data cache clean to Point of Persistence538client # [ 0.005171] CPU features: detected: CRC32 instructions539lb # [ 0.019759] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations540worker2 # [ 0.005350] CPU features: detected: Data cache clean to Point of Deep Persistence541worker1 # [ 0.005306] CPU features: detected: Data independent timing control (DIT)542lb # [ 0.019785] audit: initializing netlink subsys (disabled)543client # [ 0.005174] CPU features: detected: Data cache clean to Point of Deep Persistence544worker1 # [ 0.005309] CPU features: detected: E0PD545worker2 # [ 0.005354] CPU features: detected: Data cache clean to Point of Persistence546lb # [ 0.020396] thermal_sys: Registered thermal governor 'fair_share'547worker1 # [ 0.005311] CPU features: detected: Enhanced Counter Virtualization548client # [ 0.005178] CPU features: detected: Data cache clean to Point of Persistence549worker2 # [ 0.005357] CPU features: detected: Data independent timing control (DIT)550lb # [ 0.020398] thermal_sys: Registered thermal governor 'bang_bang'551worker2 # [ 0.005360] CPU features: detected: E0PD552worker1 # [ 0.005314] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)553client # [ 0.005182] CPU features: detected: Data independent timing control (DIT)554lb # [ 0.020402] thermal_sys: Registered thermal governor 'step_wise'555client # [ 0.005184] CPU features: detected: E0PD556worker2 # [ 0.005363] CPU features: detected: Enhanced Counter Virtualization557worker1 # [ 0.005318] CPU features: detected: Enhanced Virtualization Traps558lb # [ 0.020407] thermal_sys: Registered thermal governor 'user_space'559client # [ 0.005187] CPU features: detected: Enhanced Counter Virtualization560worker1 # [ 0.005321] CPU features: detected: Fine Grained Traps561worker2 # [ 0.005366] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)562lb # [ 0.020410] thermal_sys: Registered thermal governor 'power_allocator'563client # [ 0.005190] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)564worker2 # [ 0.005369] CPU features: detected: Enhanced Virtualization Traps565worker1 # [ 0.005324] CPU features: detected: Generic authentication (architected QARMA5 algorithm)566lb # [ 0.020434] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1567client # [ 0.005193] CPU features: detected: Enhanced Virtualization Traps568worker2 # [ 0.005372] CPU features: detected: Fine Grained Traps569worker1 # [ 0.005328] CPU features: detected: RCpc load-acquire (LDAPR)570lb # [ 0.020445] cpuidle: using governor ladder571client # [ 0.005196] CPU features: detected: Fine Grained Traps572lb # [ 0.020450] cpuidle: using governor menu573worker1 # [ 0.005331] CPU features: detected: LSE atomic instructions574worker2 # [ 0.005376] CPU features: detected: Generic authentication (architected QARMA5 algorithm)575client # [ 0.005200] CPU features: detected: Generic authentication (architected QARMA5 algorithm)576worker1 # [ 0.005334] CPU features: detected: Privileged Access Never577worker2 # [ 0.005382] CPU features: detected: RCpc load-acquire (LDAPR)578lb # [ 0.020660] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.579worker1 # [ 0.005337] CPU features: detected: PMUv3580client # [ 0.005204] CPU features: detected: RCpc load-acquire (LDAPR)581lb # [ 0.020678] ASID allocator initialised with 65536 entries582worker2 # [ 0.005385] CPU features: detected: LSE atomic instructions583worker1 # [ 0.005340] CPU features: detected: RAS Extension Support584lb # [ 0.022039] Serial: AMBA PL011 UART driver585client # [ 0.005207] CPU features: detected: LSE atomic instructions586worker2 # [ 0.005388] CPU features: detected: Privileged Access Never587worker1 # [ 0.005342] CPU features: detected: RASv1p1 Extension Support588worker2 # [ 0.005391] CPU features: detected: PMUv3589client # [ 0.005210] CPU features: detected: Privileged Access Never590lb # [ 0.027788] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1591client # [ 0.005213] CPU features: detected: PMUv3592worker1 # [ 0.005345] CPU features: detected: Random Number Generator593worker2 # [ 0.005393] CPU features: detected: RAS Extension Support594lb # [ 0.027913] printk: console [ttyAMA0] enabled595client # [ 0.005215] CPU features: detected: RAS Extension Support596worker1 # [ 0.005347] CPU features: detected: Speculation barrier (SB)597worker2 # [ 0.005396] CPU features: detected: RASv1p1 Extension Support598client # [ 0.005218] CPU features: detected: RASv1p1 Extension Support599worker1 # [ 0.005350] CPU features: detected: Stage-2 Force Write-Back600worker2 # [ 0.005398] CPU features: detected: Random Number Generator601lb # [ 0.149264] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages602client # [ 0.005221] CPU features: detected: Random Number Generator603worker1 # [ 0.005353] CPU features: detected: TLB range maintenance instructions604worker2 # [ 0.005401] CPU features: detected: Speculation barrier (SB)605lb # [ 0.149284] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page606client # [ 0.005223] CPU features: detected: Speculation barrier (SB)607worker1 # [ 0.005358] CPU features: detected: Speculative Store Bypassing Safe (SSBS)608worker2 # [ 0.005404] CPU features: detected: Stage-2 Force Write-Back609client # [ 0.005226] CPU features: detected: Stage-2 Force Write-Back610lb # [ 0.149289] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages611worker1 # [ 0.005395] alternatives: applying system-wide alternatives612worker2 # [ 0.005407] CPU features: detected: TLB range maintenance instructions613lb # [ 0.149294] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page614client # [ 0.005229] CPU features: detected: TLB range maintenance instructions615worker1 # [ 0.008557] CPU features: detected: BBM Level 2 without TLB conflict abort616worker2 # [ 0.005412] CPU features: detected: Speculative Store Bypassing Safe (SSBS)617client # [ 0.005233] CPU features: detected: Speculative Store Bypassing Safe (SSBS)618lb # [ 0.149298] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages619worker2 # [ 0.005449] alternatives: applying system-wide alternatives620client # [ 0.005271] alternatives: applying system-wide alternatives621lb # [ 0.149302] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page622worker1 # [ 0.008735] Memory: 1406556K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)623worker2 # [ 0.008488] CPU features: detected: BBM Level 2 without TLB conflict abort624client # [ 0.008314] CPU features: detected: BBM Level 2 without TLB conflict abort625lb # [ 0.149307] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages626worker1 # [ 0.009261] devtmpfs: initialized627lb # [ 0.149311] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page628worker1 # [ 0.011271] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)629worker2 # [ 0.008672] Memory: 1406488K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)630client # [ 0.008548] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)631worker2 # [ 0.009170] devtmpfs: initialized632client # [ 0.009173] devtmpfs: initialized633worker2 # [ 0.010999] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)634worker1 # [ 0.011301] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).635client # [ 0.011106] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)636lb # [ 0.156537] fbcon: Taking over console637worker1 # [ 0.011488] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL638lb # [ 0.156551] ACPI: Interpreter disabled.639client # [ 0.011134] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).640worker1 # [ 0.011494] 0 pages in range for non-PLT usage641lb # [ 0.158482] iommu: Default domain type: Translated642worker1 # [ 0.011495] 508288 pages in range for PLT usage643client # [ 0.011311] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL644lb # [ 0.158492] iommu: DMA domain TLB invalidation policy: strict mode645client # [ 0.011315] 0 pages in range for non-PLT usage646worker1 # [ 0.011615] pinctrl core: initialized pinctrl subsystem647worker1 # [ 0.012441] DMI not present or invalid.648client # [ 0.011316] 508288 pages in range for PLT usage649lb # [ 0.160135] SCSI subsystem initialized650client # [ 0.011451] pinctrl core: initialized pinctrl subsystem651worker1 # [ 0.015809] NET: Registered PF_NETLINK/PF_ROUTE protocol family652client # [ 0.012199] DMI not present or invalid.653worker1 # [ 0.018107] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations654client # [ 0.015523] NET: Registered PF_NETLINK/PF_ROUTE protocol family655worker1 # [ 0.018288] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations656client # [ 0.017763] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations657worker1 # [ 0.018508] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations658client # [ 0.017969] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations659worker1 # [ 0.018536] audit: initializing netlink subsys (disabled)660client # [ 0.018172] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations661worker1 # [ 0.019163] thermal_sys: Registered thermal governor 'fair_share'662client # [ 0.018195] audit: initializing netlink subsys (disabled)663worker1 # [ 0.019166] thermal_sys: Registered thermal governor 'bang_bang'664worker2 # [ 0.011021] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).665client # [ 0.018759] thermal_sys: Registered thermal governor 'fair_share'666worker1 # [ 0.019170] thermal_sys: Registered thermal governor 'step_wise'667client # [ 0.018761] thermal_sys: Registered thermal governor 'bang_bang'668worker2 # [ 0.011218] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL669worker1 # [ 0.019175] thermal_sys: Registered thermal governor 'user_space'670worker2 # [ 0.011224] 0 pages in range for non-PLT usage671client # [ 0.018764] thermal_sys: Registered thermal governor 'step_wise'672worker2 # [ 0.011224] 508288 pages in range for PLT usage673worker1 # [ 0.019178] thermal_sys: Registered thermal governor 'power_allocator'674client # [ 0.018769] thermal_sys: Registered thermal governor 'user_space'675worker2 # [ 0.011355] pinctrl core: initialized pinctrl subsystem676lb # [ 0.167891] usbcore: registered new interface driver usbfs677worker1 # [ 0.019207] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1678client # [ 0.018772] thermal_sys: Registered thermal governor 'power_allocator'679worker2 # [ 0.012127] DMI not present or invalid.680worker1 # [ 0.019215] cpuidle: using governor ladder681lb # [ 0.167917] usbcore: registered new interface driver hub682worker2 # [ 0.015278] NET: Registered PF_NETLINK/PF_ROUTE protocol family683worker1 # [ 0.019221] cpuidle: using governor menu684client # [ 0.018796] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1685lb # [ 0.167944] usbcore: registered new device driver usb686client # [ 0.018804] cpuidle: using governor ladder687worker2 # [ 0.017521] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations688lb # [ 0.168228] pps_core: LinuxPPS API ver. 1 registered689worker1 # [ 0.019412] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.690client # [ 0.018809] cpuidle: using governor menu691worker1 # [ 0.019433] ASID allocator initialised with 65536 entries692worker2 # [ 0.017704] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations693client # [ 0.018990] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.694lb # [ 0.168233] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>695worker1 # [ 0.020724] Serial: AMBA PL011 UART driver696lb # [ 0.168244] PTP clock support registered697client # [ 0.019004] ASID allocator initialised with 65536 entries698worker2 # [ 0.017902] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations699lb # [ 0.168284] EDAC MC: Ver: 3.0.0700client # [ 0.020235] Serial: AMBA PL011 UART driver701worker1 # [ 0.026107] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1702worker2 # [ 0.017924] audit: initializing netlink subsys (disabled)703worker1 # [ 0.026242] printk: console [ttyAMA0] enabled704lb # [ 0.172807] scmi_core: SCMI protocol bus registered705worker2 # [ 0.018534] thermal_sys: Registered thermal governor 'fair_share'706client # [ 0.025643] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1707client # [ 0.025797] printk: console [ttyAMA0] enabled708worker2 # [ 0.018537] thermal_sys: Registered thermal governor 'bang_bang'709lb # [ 0.173804] FPGA manager framework710lb # [ 0.174790] vgaarb: loaded711worker2 # [ 0.018541] thermal_sys: Registered thermal governor 'step_wise'712worker2 # [ 0.018547] thermal_sys: Registered thermal governor 'user_space'713worker2 # [ 0.018550] thermal_sys: Registered thermal governor 'power_allocator'714lb # [ 0.175437] clocksource: Switched to clocksource arch_sys_counter715lb # [ 0.176057] VFS: Disk quotas dquot_6.6.0716worker2 # [ 0.018578] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1717lb # [ 0.176086] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)718worker2 # [ 0.018586] cpuidle: using governor ladder719worker2 # [ 0.018592] cpuidle: using governor menu720lb # [ 0.178523] netfs: FS-Cache loaded721lb # [ 0.178647] pnp: PnP ACPI: disabled722worker2 # [ 0.018785] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.723worker2 # [ 0.018801] ASID allocator initialised with 65536 entries724worker2 # [ 0.020023] Serial: AMBA PL011 UART driver725worker2 # [ 0.025334] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1726worker2 # [ 0.025492] printk: console [ttyAMA0] enabled727worker1 # [ 0.147263] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages728worker2 # [ 0.145318] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages729worker1 # [ 0.147285] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page730client # [ 0.144722] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages731worker2 # [ 0.145340] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page732worker1 # [ 0.147290] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages733client # [ 0.144739] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page734worker2 # [ 0.145345] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages735worker1 # [ 0.147295] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page736client # [ 0.144745] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages737worker2 # [ 0.145349] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page738worker1 # [ 0.147299] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages739client # [ 0.144749] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page740worker2 # [ 0.145354] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages741worker1 # [ 0.147303] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page742client # [ 0.144753] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages743worker2 # [ 0.145358] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page744worker1 # [ 0.147307] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages745client # [ 0.144758] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page746lb # [ 0.184846] NET: Registered PF_INET protocol family747worker2 # [ 0.145362] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages748worker1 # [ 0.147312] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page749client # [ 0.144762] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages750lb # [ 0.185139] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)751worker2 # [ 0.145366] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page752client # [ 0.144766] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page753worker1 # [ 0.154654] fbcon: Taking over console754worker1 # [ 0.154670] ACPI: Interpreter disabled.755worker2 # [ 0.152706] fbcon: Taking over console756client # [ 0.152125] fbcon: Taking over console757worker2 # [ 0.152726] ACPI: Interpreter disabled.758worker1 # [ 0.156616] iommu: Default domain type: Translated759client # [ 0.152140] ACPI: Interpreter disabled.760worker2 # [ 0.154593] iommu: Default domain type: Translated761worker1 # [ 0.156627] iommu: DMA domain TLB invalidation policy: strict mode762client # [ 0.153990] iommu: Default domain type: Translated763worker2 # [ 0.154601] iommu: DMA domain TLB invalidation policy: strict mode764worker1 # [ 0.158403] SCSI subsystem initialized765client # [ 0.153999] iommu: DMA domain TLB invalidation policy: strict mode766worker2 # [ 0.156364] SCSI subsystem initialized767client # [ 0.155757] SCSI subsystem initialized768worker1 # [ 0.163663] usbcore: registered new interface driver usbfs769worker1 # [ 0.163702] usbcore: registered new interface driver hub770client # [ 0.161041] usbcore: registered new interface driver usbfs771worker1 # [ 0.163720] usbcore: registered new device driver usb772client # [ 0.161075] usbcore: registered new interface driver hub773worker2 # [ 0.162759] usbcore: registered new interface driver usbfs774worker1 # [ 0.163993] pps_core: LinuxPPS API ver. 1 registered775client # [ 0.161093] usbcore: registered new device driver usb776worker2 # [ 0.162792] usbcore: registered new interface driver hub777client # [ 0.161364] pps_core: LinuxPPS API ver. 1 registered778worker1 # [ 0.163999] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>779worker2 # [ 0.162810] usbcore: registered new device driver usb780worker1 # [ 0.164009] PTP clock support registered781worker2 # [ 0.163081] pps_core: LinuxPPS API ver. 1 registered782worker1 # [ 0.164066] EDAC MC: Ver: 3.0.0783client # [ 0.161370] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>784client # [ 0.161379] PTP clock support registered785worker2 # [ 0.163086] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>786worker1 # [ 0.168673] scmi_core: SCMI protocol bus registered787client # [ 0.161427] EDAC MC: Ver: 3.0.0788worker2 # [ 0.163096] PTP clock support registered789worker2 # [ 0.163142] EDAC MC: Ver: 3.0.0790client # [ 0.165968] scmi_core: SCMI protocol bus registered791worker1 # [ 0.169646] FPGA manager framework792worker1 # [ 0.170670] vgaarb: loaded793worker2 # [ 0.167647] scmi_core: SCMI protocol bus registered794client # [ 0.167003] FPGA manager framework795client # [ 0.168068] vgaarb: loaded796worker1 # [ 0.171374] clocksource: Switched to clocksource arch_sys_counter797worker2 # [ 0.168648] FPGA manager framework798worker2 # [ 0.169645] vgaarb: loaded799client # [ 0.168738] clocksource: Switched to clocksource arch_sys_counter800worker2 # [ 0.170307] clocksource: Switched to clocksource arch_sys_counter801worker2 # [ 0.173050] VFS: Disk quotas dquot_6.6.0802worker2 # [ 0.173086] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)803client # [ 0.173638] VFS: Disk quotas dquot_6.6.0804worker1 # [ 0.177874] VFS: Disk quotas dquot_6.6.0805client # [ 0.173677] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)806worker1 # [ 0.177918] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)807worker1 # [ 0.179602] netfs: FS-Cache loaded808worker2 # [ 0.177118] netfs: FS-Cache loaded809worker1 # [ 0.179708] pnp: PnP ACPI: disabled810worker2 # [ 0.177275] pnp: PnP ACPI: disabled811client # [ 0.177496] netfs: FS-Cache loaded812client # [ 0.177630] pnp: PnP ACPI: disabled813worker2 # [ 0.181353] NET: Registered PF_INET protocol family814worker2 # [ 0.181637] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)815worker1 # [ 0.183821] NET: Registered PF_INET protocol family816worker1 # [ 0.184111] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)817client # [ 0.181564] NET: Registered PF_INET protocol family818lb # [ 0.214053] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)819client # [ 0.181868] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)820lb # [ 0.214099] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)821lb # [ 0.214128] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)822lb # [ 0.214199] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)823lb # [ 0.214342] TCP: Hash tables configured (established 16384 bind 16384)824lb # [ 0.214427] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)825lb # [ 0.214464] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)826lb # [ 0.214500] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)827lb # [ 0.214617] NET: Registered PF_UNIX/PF_LOCAL protocol family828lb # [ 0.214657] NET: Registered PF_XDP protocol family829lb # [ 0.214674] PCI: CLS 0 bytes, default 64830lb # [ 0.214940] Trying to unpack rootfs image as initramfs...831lb # [ 0.233145] kvm [1]: HYP mode not available832client # [ 0.212642] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833client # [ 0.212696] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834client # [ 0.212729] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)835worker2 # [ 0.210894] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)836worker2 # [ 0.210950] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)837worker2 # [ 0.210980] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)838worker1 # [ 0.214394] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)839worker2 # [ 0.211061] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)840worker1 # [ 0.214446] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)841worker2 # [ 0.211205] TCP: Hash tables configured (established 16384 bind 16384)842worker1 # [ 0.214479] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)843worker2 # [ 0.211312] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)844worker1 # [ 0.214556] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)845worker2 # [ 0.211355] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)846worker1 # [ 0.214701] TCP: Hash tables configured (established 16384 bind 16384)847worker2 # [ 0.211392] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)848worker2 # [ 0.211500] NET: Registered PF_UNIX/PF_LOCAL protocol family849worker1 # [ 0.214787] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)850client # [ 0.215866] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)851worker2 # [ 0.211548] NET: Registered PF_XDP protocol family852worker1 # [ 0.214845] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)853worker2 # [ 0.211568] PCI: CLS 0 bytes, default 64854client # [ 0.216015] TCP: Hash tables configured (established 16384 bind 16384)855worker2 # [ 0.211857] Trying to unpack rootfs image as initramfs...856worker1 # [ 0.214882] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)857client # [ 0.216113] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)858worker1 # [ 0.214995] NET: Registered PF_UNIX/PF_LOCAL protocol family859client # [ 0.216150] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)860worker1 # [ 0.215043] NET: Registered PF_XDP protocol family861worker1 # [ 0.215064] PCI: CLS 0 bytes, default 64862client # [ 0.216189] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)863worker1 # [ 0.215324] Trying to unpack rootfs image as initramfs...864client # [ 0.216281] NET: Registered PF_UNIX/PF_LOCAL protocol family865client # [ 0.216303] NET: Registered PF_XDP protocol family866client # [ 0.216323] PCI: CLS 0 bytes, default 64867client # [ 0.216595] Trying to unpack rootfs image as initramfs...868client # [ 0.231392] kvm [1]: HYP mode not available869worker2 # [ 0.228066] kvm [1]: HYP mode not available870worker1 # [ 0.233300] kvm [1]: HYP mode not available871lb # [ 0.328096] Initialise system trusted keyrings872lb # [ 0.328844] workingset: timestamp_bits=42 max_order=19 bucket_order=0873lb # [ 0.330130] squashfs: version 4.0 (2009/01/31) Phillip Lougher874lb # [ 0.330913] 9p: Installing v9fs 9p2000 file system support875lb # [ 0.359657] Key type asymmetric registered876lb # [ 0.359689] Asymmetric key parser 'x509' registered877worker2 # [ 0.323792] Initialise system trusted keyrings878lb # [ 0.359762] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)879lb # [ 0.361921] io scheduler mq-deadline registered880worker2 # [ 0.324582] workingset: timestamp_bits=42 max_order=19 bucket_order=0881lb # [ 0.361932] io scheduler kyber registered882worker2 # [ 0.325929] squashfs: version 4.0 (2009/01/31) Phillip Lougher883worker1 # [ 0.331679] Initialise system trusted keyrings884client # [ 0.328588] Initialise system trusted keyrings885worker1 # [ 0.332499] workingset: timestamp_bits=42 max_order=19 bucket_order=0886worker1 # [ 0.333858] squashfs: version 4.0 (2009/01/31) Phillip Lougher887worker1 # [ 0.334645] 9p: Installing v9fs 9p2000 file system support888worker2 # [ 0.326727] 9p: Installing v9fs 9p2000 file system support889lb # [ 0.371589] pl061_gpio 9030000.pl061: PL061 GPIO chip registered890client # [ 0.336817] workingset: timestamp_bits=42 max_order=19 bucket_order=0891lb # [ 0.372993] ledtrig-cpu: registered to indicate activity on CPUs892lb # [ 0.373405] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:893client # [ 0.338300] squashfs: version 4.0 (2009/01/31) Phillip Lougher894lb # [ 0.373425] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000895lb # [ 0.373437] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000896lb # [ 0.373446] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000897lb # [ 0.373470] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits898lb # [ 0.373494] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]899lb # [ 0.373572] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00900client # [ 0.339086] 9p: Installing v9fs 9p2000 file system support901lb # [ 0.373581] pci_bus 0000:00: root bus resource [bus 00-ff]902lb # [ 0.373587] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]903worker2 # [ 0.346830] Key type asymmetric registered904worker2 # [ 0.346859] Asymmetric key parser 'x509' registered905lb # [ 0.373592] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]906worker2 # [ 0.346931] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)907lb # [ 0.373597] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]908lb # [ 0.373664] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint909lb # [ 0.374105] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint910lb # [ 0.374291] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]911lb # [ 0.374307] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]912lb # [ 0.374337] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]913lb # [ 0.374353] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]914lb # [ 0.374815] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint915worker2 # [ 0.354385] io scheduler mq-deadline registered916worker2 # [ 0.354405] io scheduler kyber registered917lb # [ 0.374998] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]918lb # [ 0.375014] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]919worker1 # [ 0.358672] Key type asymmetric registered920lb # [ 0.375043] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]921worker1 # [ 0.358709] Asymmetric key parser 'x509' registered922worker1 # [ 0.358793] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)923lb # [ 0.394671] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint924lb # [ 0.394862] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]925lb # [ 0.394878] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]926client # [ 0.359056] Key type asymmetric registered927lb # [ 0.394909] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]928client # [ 0.359077] Asymmetric key parser 'x509' registered929client # [ 0.359165] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)930lb # [ 0.395380] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint931client # [ 0.361340] io scheduler mq-deadline registered932client # [ 0.361348] io scheduler kyber registered933worker2 # [ 0.366493] pl061_gpio 9030000.pl061: PL061 GPIO chip registered934worker1 # [ 0.367430] io scheduler mq-deadline registered935worker1 # [ 0.367450] io scheduler kyber registered936worker2 # [ 0.367891] ledtrig-cpu: registered to indicate activity on CPUs937worker1 # [ 0.372899] pl061_gpio 9030000.pl061: PL061 GPIO chip registered938worker2 # [ 0.368341] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:939worker2 # [ 0.368365] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000940worker1 # [ 0.374238] ledtrig-cpu: registered to indicate activity on CPUs941worker2 # [ 0.368378] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000942worker1 # [ 0.374681] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:943lb # [ 0.403603] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]944client # [ 0.372892] pl061_gpio 9030000.pl061: PL061 GPIO chip registered945worker2 # [ 0.368385] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000946worker1 # [ 0.374701] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000947lb # [ 0.403629] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]948worker2 # [ 0.368407] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits949lb # [ 0.403662] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]950worker1 # [ 0.374719] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000951lb # [ 0.404146] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint952worker2 # [ 0.368432] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]953worker1 # [ 0.374732] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000954lb # [ 0.404329] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]955worker2 # [ 0.368538] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00956client # [ 0.374222] ledtrig-cpu: registered to indicate activity on CPUs957worker1 # [ 0.374753] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits958lb # [ 0.404344] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]959worker2 # [ 0.368554] pci_bus 0000:00: root bus resource [bus 00-ff]960client # [ 0.374618] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:961worker2 # [ 0.368560] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]962lb # [ 0.404373] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]963worker1 # [ 0.374774] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]964worker2 # [ 0.368565] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]965lb # [ 0.404824] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint966client # [ 0.374635] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000967worker1 # [ 0.374853] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00968lb # [ 0.405008] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]969worker1 # [ 0.374870] pci_bus 0000:00: root bus resource [bus 00-ff]970worker2 # [ 0.368570] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]971client # [ 0.374648] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000972lb # [ 0.405024] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]973worker1 # [ 0.374877] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]974worker2 # [ 0.368664] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint975client # [ 0.374656] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000976lb # [ 0.405053] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]977worker1 # [ 0.374882] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]978worker2 # [ 0.369105] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint979client # [ 0.374677] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits980worker1 # [ 0.374887] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]981lb # [ 0.405509] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint982worker2 # [ 0.369292] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]983lb # [ 0.405695] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]984worker1 # [ 0.374946] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint985worker2 # [ 0.369310] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]986client # [ 0.374719] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]987lb # [ 0.405710] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]988worker2 # [ 0.369341] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]989client # [ 0.374797] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00990lb # [ 0.405739] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]991worker2 # [ 0.369357] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]992client # [ 0.374807] pci_bus 0000:00: root bus resource [bus 00-ff]993lb # [ 0.405755] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]994client # [ 0.374813] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]995worker2 # [ 0.369812] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint996lb # [ 0.406214] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint997client # [ 0.374818] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]998worker2 # [ 0.369998] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]999lb # [ 0.406400] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1000worker2 # [ 0.370015] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1001client # [ 0.374823] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1002lb # [ 0.406430] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1003worker2 # [ 0.370045] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1004client # [ 0.374881] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1005lb # [ 0.406887] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1006client # [ 0.375329] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1007lb # [ 0.407076] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1008client # [ 0.375533] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1009lb # [ 0.407106] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1010worker2 # [ 0.389108] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1011client # [ 0.375551] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1012worker2 # [ 0.389301] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1013lb # [ 0.407519] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1014client # [ 0.375581] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1015worker2 # [ 0.389318] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1016lb # [ 0.407701] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1017client # [ 0.375597] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1018worker2 # [ 0.389347] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019lb # [ 0.407955] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1020client # [ 0.376082] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1021worker2 # [ 0.389798] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1022lb # [ 0.408228] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1023client # [ 0.376272] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1024worker2 # [ 0.389981] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1025lb # [ 0.408244] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1026client # [ 0.376288] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1027worker2 # [ 0.389997] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1028lb # [ 0.408274] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1029client # [ 0.376319] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1030worker2 # [ 0.390035] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1031lb # [ 0.408762] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1032lb # [ 0.408947] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1033lb # [ 0.408962] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1034lb # [ 0.408993] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035lb # [ 0.409583] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1036lb # [ 0.409594] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1037lb # [ 0.409600] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1038worker1 # [ 0.395471] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1039worker1 # [ 0.395683] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1040lb # [ 0.409652] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1041worker1 # [ 0.395701] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1042lb # [ 0.409700] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1043worker1 # [ 0.395731] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1044lb # [ 0.409748] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1045worker1 # [ 0.395748] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1046lb # [ 0.409795] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1047worker1 # [ 0.396259] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1048worker1 # [ 0.396447] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1049lb # [ 0.409841] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1050worker1 # [ 0.396463] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1051lb # [ 0.409888] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1052worker1 # [ 0.396494] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1053lb # [ 0.409935] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1054worker1 # [ 0.396965] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1055lb # [ 0.409982] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1056worker1 # [ 0.397156] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1057worker1 # [ 0.397171] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1058lb # [ 0.410030] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1059client # [ 0.400842] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1060worker1 # [ 0.397201] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1061client # [ 0.401042] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1062worker2 # [ 0.402614] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1063lb # [ 0.410096] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1064worker2 # [ 0.402818] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1065client # [ 0.401059] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1066worker1 # [ 0.397667] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1067lb # [ 0.410142] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1068worker2 # [ 0.402836] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1069client # [ 0.401089] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070worker1 # [ 0.397853] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1071lb # [ 0.410163] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1072worker2 # [ 0.402865] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1073worker1 # [ 0.397869] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1074client # [ 0.401594] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1075lb # [ 0.410185] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1076worker2 # [ 0.403330] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1077worker1 # [ 0.397901] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078client # [ 0.401781] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1079lb # [ 0.410207] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1080worker2 # [ 0.403513] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1081client # [ 0.401797] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1082worker1 # [ 0.398371] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1083worker2 # [ 0.403530] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1084lb # [ 0.410233] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1085client # [ 0.401830] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086worker1 # [ 0.398561] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1087lb # [ 0.410255] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1088worker2 # [ 0.403560] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089worker1 # [ 0.398578] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1090client # [ 0.402306] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1091lb # [ 0.410277] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1092worker2 # [ 0.404021] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1093client # [ 0.402493] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1094worker1 # [ 0.398608] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1095lb # [ 0.410299] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1096worker2 # [ 0.404206] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1097client # [ 0.402508] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1098worker1 # [ 0.399076] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1099worker2 # [ 0.404222] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1100lb # [ 0.410321] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1101client # [ 0.402538] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1102worker1 # [ 0.399263] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1103worker2 # [ 0.404251] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1104lb # [ 0.410344] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1105worker1 # [ 0.399279] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1106client # [ 0.403000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1107worker2 # [ 0.404267] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1108lb # [ 0.410366] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1109client # [ 0.403185] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1110worker1 # [ 0.399309] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111lb # [ 0.410389] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1112worker2 # [ 0.404781] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1113client # [ 0.403201] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1114worker1 # [ 0.399821] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1115lb # [ 0.410410] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1116worker2 # [ 0.404970] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1117client # [ 0.403232] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118worker1 # [ 0.400009] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1119lb # [ 0.410431] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1120worker2 # [ 0.405000] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1121worker1 # [ 0.400025] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1122client # [ 0.403701] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1123lb # [ 0.410453] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1124client # [ 0.403888] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1125worker2 # [ 0.405462] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1126worker1 # [ 0.400055] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127lb # [ 0.410475] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1128client # [ 0.403910] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1129worker2 # [ 0.405651] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1130worker1 # [ 0.400072] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1131lb # [ 0.410496] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1132client # [ 0.403939] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133worker2 # [ 0.405680] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1134lb # [ 0.410518] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1135worker1 # [ 0.400544] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1136client # [ 0.403956] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1137worker2 # [ 0.406069] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1138lb # [ 0.410540] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1139worker1 # [ 0.400736] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1140worker2 # [ 0.406251] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1141lb # [ 0.410562] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1142client # [ 0.404471] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1143worker1 # [ 0.400766] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1144client # [ 0.404660] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1145lb # [ 0.410584] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1146worker2 # [ 0.406536] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1147worker1 # [ 0.401231] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1148lb # [ 0.410611] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1149client # [ 0.404691] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1150worker2 # [ 0.406814] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1151worker1 # [ 0.401421] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1152lb # [ 0.410620] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1153worker2 # [ 0.406832] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1154client # [ 0.405167] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1155worker1 # [ 0.401451] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1156lb # [ 0.410625] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1157worker2 # [ 0.406861] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158client # [ 0.405356] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1159worker1 # [ 0.401845] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1160worker2 # [ 0.407330] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1161client # [ 0.405386] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1162worker1 # [ 0.402030] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1163worker2 # [ 0.407521] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1164client # [ 0.405792] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1165worker2 # [ 0.407537] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1166worker1 # [ 0.402278] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1167client # [ 0.405974] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1168worker1 # [ 0.402564] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1169worker2 # [ 0.407567] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1170client # [ 0.406227] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1171worker1 # [ 0.402581] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1172worker2 # [ 0.408143] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1173client # [ 0.406491] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1174worker1 # [ 0.402611] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1175worker2 # [ 0.408154] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1176client # [ 0.406509] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1177worker1 # [ 0.403100] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1178worker2 # [ 0.408161] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1179client # [ 0.406540] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180worker1 # [ 0.403287] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1181worker2 # [ 0.408206] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1182client # [ 0.407020] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1183worker1 # [ 0.403303] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1184client # [ 0.407204] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1185worker2 # [ 0.408254] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1186worker1 # [ 0.403333] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1187client # [ 0.407220] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1188worker2 # [ 0.408302] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1189client # [ 0.407250] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1190worker2 # [ 0.408358] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1191client # [ 0.407835] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1192worker2 # [ 0.408405] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1193client # [ 0.407846] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1194lb # [ 0.475883] pci 0000:00:0a.0: enabling device (0000 -> 0002)1195worker2 # [ 0.408452] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1196client # [ 0.407851] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1197worker1 # [ 0.440585] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1198worker2 # [ 0.408499] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1199client # [ 0.407897] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1200worker1 # [ 0.440605] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1201worker2 # [ 0.408545] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1202client # [ 0.407944] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1203worker1 # [ 0.440611] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1204worker2 # [ 0.408591] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1205client # [ 0.407997] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1206worker1 # [ 0.440657] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1207client # [ 0.408044] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1208worker2 # [ 0.408654] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1209worker1 # [ 0.440706] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1210worker2 # [ 0.408700] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1211client # [ 0.408091] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1212worker1 # [ 0.440754] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1213worker2 # [ 0.408722] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1214client # [ 0.408138] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1215worker1 # [ 0.440801] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1216worker2 # [ 0.408744] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1217client # [ 0.408192] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1218worker2 # [ 0.408765] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1219worker1 # [ 0.440848] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1220client # [ 0.408239] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1221worker2 # [ 0.408790] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1222worker1 # [ 0.440895] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1223client # [ 0.408287] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1224worker2 # [ 0.408811] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1225worker1 # [ 0.440943] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1226worker2 # [ 0.408834] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1227client # [ 0.408359] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1228worker1 # [ 0.440991] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1229worker2 # [ 0.408855] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1230client # [ 0.408405] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1231worker1 # [ 0.441040] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1232worker2 # [ 0.408877] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1233client # [ 0.408427] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1234worker2 # [ 0.408900] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1235worker1 # [ 0.441139] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1236client # [ 0.408448] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1237worker2 # [ 0.408927] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1238worker1 # [ 0.441187] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1239client # [ 0.408470] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1240worker2 # [ 0.408950] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1241client # [ 0.408494] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1242worker1 # [ 0.441209] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1243worker2 # [ 0.408971] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1244client # [ 0.408516] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1245worker1 # [ 0.441231] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1246worker2 # [ 0.408993] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1247client # [ 0.408539] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1248worker1 # [ 0.441253] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1249worker2 # [ 0.409014] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1250client # [ 0.408562] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1251worker1 # [ 0.441278] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1252worker2 # [ 0.409036] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1253client # [ 0.408584] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1254worker1 # [ 0.441300] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1255worker2 # [ 0.409057] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1256client # [ 0.408606] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1257worker1 # [ 0.441323] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1258worker2 # [ 0.409079] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1259worker1 # [ 0.441345] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1260worker2 # [ 0.409107] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1261client # [ 0.408629] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1262worker2 # [ 0.409129] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1263client # [ 0.408652] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1264worker1 # [ 0.441367] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1265worker2 # [ 0.409151] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1266client # [ 0.408674] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1267worker1 # [ 0.441389] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1268worker2 # [ 0.409178] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1269client # [ 0.408696] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1270worker1 # [ 0.441413] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1271worker2 # [ 0.409188] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1272client # [ 0.408717] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1273worker1 # [ 0.441436] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1274lb # [ 0.497294] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1275worker2 # [ 0.409193] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1276worker1 # [ 0.441458] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1277worker1 # [ 0.441479] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1278worker1 # [ 0.441501] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1279worker1 # [ 0.441523] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1280worker1 # [ 0.441545] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1281worker1 # [ 0.441567] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1282worker1 # [ 0.441588] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1283worker1 # [ 0.441611] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1284worker2 # [ 0.466429] pci 0000:00:0a.0: enabling device (0000 -> 0002)1285worker1 # [ 0.441636] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1286client # [ 0.464764] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1287worker1 # [ 0.441665] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1288client # [ 0.464802] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1289worker1 # [ 0.441675] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1290client # [ 0.464825] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1291worker1 # [ 0.441680] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1292client # [ 0.464849] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1293worker1 # [ 0.442873] pci 0000:00:0a.0: enabling device (0000 -> 0002)1294client # [ 0.464871] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1295client # [ 0.464895] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1296client # [ 0.464924] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1297client # [ 0.464934] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1298lb # [ 0.507587] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1299client # [ 0.464938] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1300client # [ 0.466122] pci 0000:00:0a.0: enabling device (0000 -> 0002)1301lb # [ 0.511045] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1302worker1 # [ 0.479456] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 35728 usecs1303lb # [ 0.520822] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1304lb # [ 0.525525] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1305worker2 # [ 0.493035] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1306worker2 # [ 0.496694] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1307lb # [ 0.531595] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1308lb # [ 0.533968] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1309client # [ 0.499552] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1310worker1 # [ 0.506650] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1311worker1 # [ 0.509796] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1312client # [ 0.505889] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1313lb # [ 0.543991] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1314worker2 # [ 0.506503] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1315lb # [ 0.546069] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1316worker2 # [ 0.508469] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1317client # [ 0.508128] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1318worker2 # [ 0.512019] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1319worker2 # [ 0.514174] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1320worker1 # [ 0.516020] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1321client # [ 0.518215] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1322client # [ 0.520339] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1323worker1 # [ 0.523479] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1324worker1 # [ 0.525507] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1325lb # [ 0.556024] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1326lb # [ 0.559146] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1327worker2 # [ 0.527792] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1328worker2 # [ 0.529985] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1329client # [ 0.531756] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1330worker2 # [ 0.533860] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1331lb # [ 0.572740] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1332worker1 # [ 0.535535] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1333worker1 # [ 0.537827] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1334client # [ 0.534870] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1335lb # [ 0.575471] msm_serial: driver initialized1336lb # [ 0.575616] SuperH (H)SCI(F) driver initialized1337lb # [ 0.575669] STM32 USART driver initialized1338worker2 # [ 0.542148] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1339client # [ 0.544980] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1340client # [ 0.547172] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1341worker1 # [ 0.547987] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1342worker1 # [ 0.550159] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1343client # [ 0.550594] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1344worker1 # [ 0.552019] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1345worker2 # [ 0.551702] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1346worker1 # [ 0.554231] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1347client # [ 0.556972] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1348worker2 # [ 0.565714] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1349worker2 # [ 0.568263] msm_serial: driver initialized1350worker1 # [ 0.572722] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1351worker2 # [ 0.568447] SuperH (H)SCI(F) driver initialized1352worker2 # [ 0.568507] STM32 USART driver initialized1353lb # [ 0.609828] loop: module loaded1354lb # [ 0.610029] virtio_blk virtio5: 1/0/0 default/read/poll queues1355lb # [ 0.610799] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1356worker1 # [ 0.575437] msm_serial: driver initialized1357worker1 # [ 0.575651] SuperH (H)SCI(F) driver initialized1358client # [ 0.575674] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1359worker1 # [ 0.575705] STM32 USART driver initialized1360client # [ 0.578328] msm_serial: driver initialized1361client # [ 0.578465] SuperH (H)SCI(F) driver initialized1362lb # [ 0.616059] megasas: 07.734.00.00-rc11363client # [ 0.578519] STM32 USART driver initialized1364lb # [ 0.616793] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1365lb # [ 0.619148] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001366lb # [ 0.619181] Intel/Sharp Extended Query Table at 0x00311367lb # [ 0.628592] Using buffer write method1368lb # [ 0.628663] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1369lb # [ 0.630229] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001370lb # [ 0.630283] Intel/Sharp Extended Query Table at 0x00311371lb # [ 0.644116] Using buffer write method1372lb # [ 0.644151] Concatenating MTD devices:1373lb # [ 0.644155] (0): "0.flash"1374lb # [ 0.644159] (1): "0.flash"1375lb # [ 0.644163] into device "0.flash"1376worker1 # [ 0.610331] loop: module loaded1377worker2 # [ 0.607315] loop: module loaded1378worker1 # [ 0.610540] virtio_blk virtio5: 1/0/0 default/read/poll queues1379worker2 # [ 0.607507] virtio_blk virtio5: 1/0/0 default/read/poll queues1380client # [ 0.607730] loop: module loaded1381client # [ 0.607926] virtio_blk virtio5: 1/0/0 default/read/poll queues1382worker1 # [ 0.611357] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1383worker2 # [ 0.608257] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1384client # [ 0.608731] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1385client # [ 0.621430] megasas: 07.734.00.00-rc11386worker2 # [ 0.619059] megasas: 07.734.00.00-rc11387client # [ 0.622223] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1388worker2 # [ 0.619804] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1389worker2 # [ 0.622121] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001390worker1 # [ 0.624069] megasas: 07.734.00.00-rc11391worker2 # [ 0.622149] Intel/Sharp Extended Query Table at 0x00311392worker2 # [ 0.623806] Using buffer write method1393worker1 # [ 0.624851] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1394worker2 # [ 0.623889] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1395worker1 # [ 0.626803] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001396worker1 # [ 0.626831] Intel/Sharp Extended Query Table at 0x00311397worker2 # [ 0.625552] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001398worker1 # [ 0.628531] Using buffer write method1399worker2 # [ 0.625582] Intel/Sharp Extended Query Table at 0x00311400worker1 # [ 0.628605] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1401worker1 # [ 0.630328] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402worker1 # [ 0.630384] Intel/Sharp Extended Query Table at 0x00311403worker2 # [ 0.639275] Using buffer write method1404worker2 # [ 0.639304] Concatenating MTD devices:1405worker2 # [ 0.639308] (0): "0.flash"1406client # [ 0.632807] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001407worker2 # [ 0.639312] (1): "0.flash"1408worker2 # [ 0.639315] into device "0.flash"1409client # [ 0.632852] Intel/Sharp Extended Query Table at 0x00311410worker1 # [ 0.643959] Using buffer write method1411client # [ 0.634661] Using buffer write method1412worker1 # [ 0.643996] Concatenating MTD devices:1413worker1 # [ 0.644000] (0): "0.flash"1414client # [ 0.634737] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1415worker1 # [ 0.644004] (1): "0.flash"1416worker1 # [ 0.644008] into device "0.flash"1417client # [ 0.636637] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001418client # [ 0.636668] Intel/Sharp Extended Query Table at 0x00311419client # [ 0.638522] Using buffer write method1420client # [ 0.638578] Concatenating MTD devices:1421client # [ 0.638591] (0): "0.flash"1422client # [ 0.638595] (1): "0.flash"1423client # [ 0.638599] into device "0.flash"1424lb # [ 0.901719] Freeing initrd memory: 26140K1425lb # [ 0.908378] tun: Universal TUN/TAP device driver, 1.61426lb # [ 0.912302] thunder_xcv, ver 1.01427lb # [ 0.912346] thunder_bgx, ver 1.01428lb # [ 0.912368] nicpf, ver 1.01429lb # [ 0.912983] e1000: Intel(R) PRO/1000 Network Driver1430lb # [ 0.912993] e1000: Copyright (c) 1999-2006 Intel Corporation.1431lb # [ 0.913017] e1000e: Intel(R) PRO/1000 Network Driver1432lb # [ 0.913024] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1433lb # [ 0.913052] igb: Intel(R) Gigabit Ethernet Network Driver1434lb # [ 0.913058] igb: Copyright (c) 2007-2014 Intel Corporation.1435lb # [ 0.913083] igbvf: Intel(R) Gigabit Virtual Function Network Driver1436lb # [ 0.913088] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1437lb # [ 0.913222] sky2: driver version 1.301438lb # [ 0.914948] usbcore: registered new interface driver usb-storage1439lb # [ 0.915004] usbcore: registered new interface driver usbserial_generic1440lb # [ 0.915019] usbserial: USB Serial support registered for generic1441lb # [ 0.916135] ehci-pci 0000:00:0a.0: EHCI Host Controller1442lb # [ 0.916170] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11443lb # [ 0.916380] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001444lb # [ 0.927999] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001445lb # [ 0.928351] hub 1-0:1.0: USB hub found1446lb # [ 0.928372] hub 1-0:1.0: 6 ports detected1447lb # [ 0.930654] hv_vmbus: registering driver hyperv_keyboard1448lb # [ 0.932315] rtc-pl031 9010000.pl031: registered as rtc01449lb # [ 0.932344] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T14:30:34 UTC (1789914634)1450lb # [ 0.932670] i2c_dev: i2c /dev entries driver1451lb # [ 0.937702] sdhci: Secure Digital Host Controller Interface driver1452lb # [ 0.937716] sdhci: Copyright(c) Pierre Ossman1453lb # [ 0.937987] Synopsys Designware Multimedia Card Interface Driver1454lb # [ 0.938359] sdhci-pltfm: SDHCI platform and OF driver helper1455lb # [ 0.943011] hid: raw HID events driver (C) Jiri Kosina1456lb # [ 0.943300] usbcore: registered new interface driver usbhid1457lb # [ 0.943310] usbhid: USB HID core driver1458worker2 # [ 0.908288] Freeing initrd memory: 26148K1459client # [ 0.907698] Freeing initrd memory: 26160K1460lb # [ 0.946284] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1461lb # [ 0.948955] drop_monitor: Initializing network drop monitor service1462lb # [ 0.949106] NET: Registered PF_INET6 protocol family1463worker1 # [ 0.916105] Freeing initrd memory: 26148K1464lb # [ 0.951134] Segment Routing with IPv61465lb # [ 0.951155] In-situ OAM (IOAM) with IPv61466worker2 # [ 0.914537] tun: Universal TUN/TAP device driver, 1.61467lb # [ 0.951190] NET: Registered PF_PACKET protocol family1468client # [ 0.914209] tun: Universal TUN/TAP device driver, 1.61469lb # [ 0.953155] 9pnet: Installing 9P2000 support1470lb # [ 0.955363] Key type dns_resolver registered1471worker1 # [ 0.922709] tun: Universal TUN/TAP device driver, 1.61472client # [ 0.918156] thunder_xcv, ver 1.01473client # [ 0.918208] thunder_bgx, ver 1.01474client # [ 0.918234] nicpf, ver 1.01475worker2 # [ 0.918409] thunder_xcv, ver 1.01476worker2 # [ 0.918450] thunder_bgx, ver 1.01477client # [ 0.918859] e1000: Intel(R) PRO/1000 Network Driver1478worker2 # [ 0.918476] nicpf, ver 1.01479client # [ 0.918867] e1000: Copyright (c) 1999-2006 Intel Corporation.1480worker1 # [ 0.926565] thunder_xcv, ver 1.01481worker2 # [ 0.919047] e1000: Intel(R) PRO/1000 Network Driver1482worker1 # [ 0.926619] thunder_bgx, ver 1.01483client # [ 0.918891] e1000e: Intel(R) PRO/1000 Network Driver1484worker2 # [ 0.919058] e1000: Copyright (c) 1999-2006 Intel Corporation.1485worker1 # [ 0.926643] nicpf, ver 1.01486client # [ 0.918899] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1487worker2 # [ 0.919094] e1000e: Intel(R) PRO/1000 Network Driver1488worker1 # [ 0.927229] e1000: Intel(R) PRO/1000 Network Driver1489lb # [ 0.962341] registered taskstats version 11490client # [ 0.918932] igb: Intel(R) Gigabit Ethernet Network Driver1491worker1 # [ 0.927236] e1000: Copyright (c) 1999-2006 Intel Corporation.1492worker2 # [ 0.919105] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1493lb # [ 0.962510] Loading compiled-in X.509 certificates1494worker1 # [ 0.927261] e1000e: Intel(R) PRO/1000 Network Driver1495client # [ 0.918946] igb: Copyright (c) 2007-2014 Intel Corporation.1496worker2 # [ 0.919139] igb: Intel(R) Gigabit Ethernet Network Driver1497worker2 # [ 0.919147] igb: Copyright (c) 2007-2014 Intel Corporation.1498client # [ 0.918981] igbvf: Intel(R) Gigabit Virtual Function Network Driver1499worker1 # [ 0.927269] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1500client # [ 0.918987] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1501worker1 # [ 0.927297] igb: Intel(R) Gigabit Ethernet Network Driver1502worker2 # [ 0.919171] igbvf: Intel(R) Gigabit Virtual Function Network Driver1503client # [ 0.919122] sky2: driver version 1.301504worker1 # [ 0.927303] igb: Copyright (c) 2007-2014 Intel Corporation.1505worker2 # [ 0.919177] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1506client # [ 0.921400] ehci-pci 0000:00:0a.0: EHCI Host Controller1507worker2 # [ 0.919323] sky2: driver version 1.301508worker1 # [ 0.927338] igbvf: Intel(R) Gigabit Virtual Function Network Driver1509worker2 # [ 0.921006] usbcore: registered new interface driver usb-storage1510client # [ 0.921437] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11511worker1 # [ 0.927348] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1512client # [ 0.921650] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001513worker2 # [ 0.921126] usbcore: registered new interface driver usbserial_generic1514worker2 # [ 0.921144] usbserial: USB Serial support registered for generic1515worker2 # [ 0.921759] hv_vmbus: registering driver hyperv_keyboard1516worker2 # [ 0.923042] ehci-pci 0000:00:0a.0: EHCI Host Controller1517client # [ 0.930925] usbcore: registered new interface driver usb-storage1518worker1 # [ 0.934064] sky2: driver version 1.301519worker2 # [ 0.923069] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11520lb # [ 0.971127] Demotion targets for Node 0: null1521client # [ 0.931023] usbcore: registered new interface driver usbserial_generic1522worker1 # [ 0.936162] ehci-pci 0000:00:0a.0: EHCI Host Controller1523lb # [ 0.971245] Key type .fscrypt registered1524client # [ 0.931038] usbserial: USB Serial support registered for generic1525worker1 # [ 0.936200] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11526lb # [ 0.971251] Key type fscrypt-provisioning registered1527worker2 # [ 0.923328] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001528client # [ 0.931676] hv_vmbus: registering driver hyperv_keyboard1529worker1 # [ 0.936477] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001530lb # [ 0.971346] ima: No TPM chip found, activating TPM-bypass!1531worker2 # [ 0.934707] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001532lb # [ 0.971365] ima: Allocated hash algorithm: sha11533worker2 # [ 0.935049] hub 1-0:1.0: USB hub found1534client # [ 0.935233] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001535worker2 # [ 0.935064] hub 1-0:1.0: 6 ports detected1536lb # [ 0.971389] ima: No architecture policies found1537client # [ 0.935588] hub 1-0:1.0: USB hub found1538client # [ 0.935634] hub 1-0:1.0: 6 ports detected1539worker2 # [ 0.938005] rtc-pl031 9010000.pl031: registered as rtc01540lb # [ 0.975537] input: gpio-keys as /devices/platform/gpio-keys/input/input01541worker1 # [ 0.939633] usbcore: registered new interface driver usb-storage1542worker1 # [ 0.939757] usbcore: registered new interface driver usbserial_generic1543worker2 # [ 0.938036] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T14:30:34 UTC (1789914634)1544client # [ 0.937901] rtc-pl031 9010000.pl031: registered as rtc01545worker1 # [ 0.939782] usbserial: USB Serial support registered for generic1546worker1 # [ 0.940414] hv_vmbus: registering driver hyperv_keyboard1547worker2 # [ 0.940098] i2c_dev: i2c /dev entries driver1548client # [ 0.937935] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T14:30:34 UTC (1789914634)1549client # [ 0.938277] i2c_dev: i2c /dev entries driver1550worker1 # [ 0.944331] rtc-pl031 9010000.pl031: registered as rtc01551worker1 # [ 0.944365] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T14:30:34 UTC (1789914634)1552worker1 # [ 0.944707] i2c_dev: i2c /dev entries driver1553worker1 # [ 0.947751] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001554client # [ 0.943283] sdhci: Secure Digital Host Controller Interface driver1555worker1 # [ 0.948072] hub 1-0:1.0: USB hub found1556client # [ 0.943296] sdhci: Copyright(c) Pierre Ossman1557worker2 # [ 0.943388] sdhci: Secure Digital Host Controller Interface driver1558worker1 # [ 0.948096] hub 1-0:1.0: 6 ports detected1559worker2 # [ 0.943399] sdhci: Copyright(c) Pierre Ossman1560client # [ 0.943564] Synopsys Designware Multimedia Card Interface Driver1561worker2 # [ 0.943694] Synopsys Designware Multimedia Card Interface Driver1562client # [ 0.943938] sdhci-pltfm: SDHCI platform and OF driver helper1563worker2 # [ 0.944077] sdhci-pltfm: SDHCI platform and OF driver helper1564worker2 # [ 0.945775] hid: raw HID events driver (C) Jiri Kosina1565worker2 # [ 0.946014] usbcore: registered new interface driver usbhid1566client # [ 0.948395] hid: raw HID events driver (C) Jiri Kosina1567worker2 # [ 0.946021] usbhid: USB HID core driver1568client # [ 0.948661] usbcore: registered new interface driver usbhid1569client # [ 0.948671] usbhid: USB HID core driver1570worker1 # [ 0.952194] sdhci: Secure Digital Host Controller Interface driver1571worker1 # [ 0.952211] sdhci: Copyright(c) Pierre Ossman1572worker2 # [ 0.951657] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1573client # [ 0.951673] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1574worker1 # [ 0.952488] Synopsys Designware Multimedia Card Interface Driver1575worker1 # [ 0.952885] sdhci-pltfm: SDHCI platform and OF driver helper1576worker2 # [ 0.953238] drop_monitor: Initializing network drop monitor service1577worker1 # [ 0.954664] hid: raw HID events driver (C) Jiri Kosina1578worker2 # [ 0.953431] NET: Registered PF_INET6 protocol family1579client # [ 0.954337] drop_monitor: Initializing network drop monitor service1580worker1 # [ 0.954925] usbcore: registered new interface driver usbhid1581worker2 # [ 0.956458] Segment Routing with IPv61582client # [ 0.954522] NET: Registered PF_INET6 protocol family1583worker1 # [ 0.954932] usbhid: USB HID core driver1584worker2 # [ 0.956492] In-situ OAM (IOAM) with IPv61585client # [ 0.956530] Segment Routing with IPv61586worker2 # [ 0.956530] NET: Registered PF_PACKET protocol family1587client # [ 0.956563] In-situ OAM (IOAM) with IPv61588lb # [ 0.993683] clk: Disabling unused clocks1589worker2 # [ 0.958164] 9pnet: Installing 9P2000 support1590client # [ 0.956603] NET: Registered PF_PACKET protocol family1591lb # [ 0.993709] PM: genpd: Disabling unused power domains1592worker1 # [ 0.960590] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1593lb # [ 0.998001] Freeing unused kernel memory: 4736K1594worker1 # [ 0.962173] drop_monitor: Initializing network drop monitor service1595lb # [ 0.998212] Run /init as init process1596client # [ 0.958466] 9pnet: Installing 9P2000 support1597worker1 # [ 0.962358] NET: Registered PF_INET6 protocol family1598client # [ 0.960527] Key type dns_resolver registered1599worker2 # [ 0.961044] Key type dns_resolver registered1600worker1 # [ 0.965462] Segment Routing with IPv61601worker1 # [ 0.965494] In-situ OAM (IOAM) with IPv61602worker1 # [ 0.965523] NET: Registered PF_PACKET protocol family1603worker1 # [ 0.967186] 9pnet: Installing 9P2000 support1604worker1 # [ 0.969897] Key type dns_resolver registered1605client # [ 0.967797] registered taskstats version 11606client # [ 0.967987] Loading compiled-in X.509 certificates1607worker2 # [ 0.967387] registered taskstats version 11608worker2 # [ 0.967556] Loading compiled-in X.509 certificates1609lb # [ 1.013750] systemd[1]: Successfully made /usr/ read-only.1610worker2 # [ 0.976420] Demotion targets for Node 0: null1611worker1 # [ 0.976693] registered taskstats version 11612worker2 # [ 0.976538] Key type .fscrypt registered1613worker1 # [ 0.976873] Loading compiled-in X.509 certificates1614client # [ 0.976723] Demotion targets for Node 0: null1615worker2 # [ 0.976544] Key type fscrypt-provisioning registered1616worker2 # [ 0.976643] ima: No TPM chip found, activating TPM-bypass!1617client # [ 0.977562] Key type .fscrypt registered1618worker2 # [ 0.976664] ima: Allocated hash algorithm: sha11619client # [ 0.977571] Key type fscrypt-provisioning registered1620worker2 # [ 0.976684] ima: No architecture policies found1621client # [ 0.977670] ima: No TPM chip found, activating TPM-bypass!1622client # [ 0.977690] ima: Allocated hash algorithm: sha11623worker2 # [ 0.980720] input: gpio-keys as /devices/platform/gpio-keys/input/input01624client # [ 0.977725] ima: No architecture policies found1625client # [ 0.981274] input: gpio-keys as /devices/platform/gpio-keys/input/input01626worker1 # [ 0.985414] Demotion targets for Node 0: null1627worker1 # [ 0.985574] Key type .fscrypt registered1628worker1 # [ 0.985581] Key type fscrypt-provisioning registered1629worker1 # [ 0.985690] ima: No TPM chip found, activating TPM-bypass!1630worker1 # [ 0.985715] ima: Allocated hash algorithm: sha11631worker1 # [ 0.985742] ima: No architecture policies found1632worker1 # [ 0.990073] input: gpio-keys as /devices/platform/gpio-keys/input/input01633worker2 # [ 0.999169] clk: Disabling unused clocks1634client # [ 1.000283] clk: Disabling unused clocks1635worker2 # [ 0.999206] PM: genpd: Disabling unused power domains1636client # [ 1.000311] PM: genpd: Disabling unused power domains1637worker2 # [ 1.003980] Freeing unused kernel memory: 4736K1638worker2 # [ 1.004196] Run /init as init process1639client # [ 1.004853] Freeing unused kernel memory: 4736K1640client # [ 1.005072] Run /init as init process1641worker1 # [ 1.009648] clk: Disabling unused clocks1642worker1 # [ 1.009681] PM: genpd: Disabling unused power domains1643worker1 # [ 1.014520] Freeing unused kernel memory: 4736K1644worker1 # [ 1.014792] Run /init as init process1645worker2 # [ 1.020626] systemd[1]: Successfully made /usr/ read-only.1646client # [ 1.023078] systemd[1]: Successfully made /usr/ read-only.1647worker1 # [ 1.029825] systemd[1]: Successfully made /usr/ read-only.1648lb # [ 1.175540] usb 1-1: new high-speed USB device number 2 using ehci-pci1649client # [ 1.180820] usb 1-1: new high-speed USB device number 2 using ehci-pci1650worker2 # [ 1.182392] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker1 # [ 1.195517] usb 1-1: new high-speed USB device number 2 using ehci-pci1652lb # [ 1.330125] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653lb # [ 1.348644] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1654lb # [ 1.360825] systemd[1]: Detected virtualization qemu.1655lb # [ 1.362894] systemd[1]: Detected architecture arm64.1656lb # [ 1.364911] systemd[1]: Running in initrd.1657lb # [ 1.367823] systemd[1]: Initializing machine ID from random generator.1658lb # [ 1.370978] systemd[1]: Hostname set to <lb>.1659client # [ 1.333186] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660worker2 # [ 1.336907] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11661worker1 # [ 1.348080] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11662worker2 # [ 1.355689] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1663worker2 # [ 1.368207] systemd[1]: Detected virtualization qemu.1664client # [ 1.358117] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1665worker2 # [ 1.370343] systemd[1]: Detected architecture arm64.1666client # [ 1.370767] systemd[1]: Detected virtualization qemu.1667worker2 # [ 1.372283] systemd[1]: Running in initrd.1668worker1 # [ 1.365168] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1669client # [ 1.373004] systemd[1]: Detected architecture arm64.1670worker1 # [ 1.377589] systemd[1]: Detected virtualization qemu.1671client # [ 1.374979] systemd[1]: Running in initrd.1672worker2 # [ 1.375090] systemd[1]: Initializing machine ID from random generator.1673worker1 # [ 1.379827] systemd[1]: Detected architecture arm64.1674worker2 # [ 1.378075] systemd[1]: Hostname set to <worker2>.1675worker1 # [ 1.381791] systemd[1]: Running in initrd.1676client # [ 1.378019] systemd[1]: Initializing machine ID from random generator.1677client # [ 1.381346] systemd[1]: Hostname set to <client>.1678worker1 # [ 1.384778] systemd[1]: Initializing machine ID from random generator.1679worker1 # [ 1.387912] systemd[1]: Hostname set to <worker1>.1680lb # [ 1.423822] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681worker2 # [ 1.422699] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682client # [ 1.429118] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker1 # [ 1.435599] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01684lb # [ 1.543493] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker2 # [ 1.546374] usb 1-2: new high-speed USB device number 3 using ehci-pci1686worker1 # [ 1.555443] usb 1-2: new high-speed USB device number 3 using ehci-pci1687client # [ 1.552824] usb 1-2: new high-speed USB device number 3 using ehci-pci1688lb # [ 1.677636] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689worker1 # [ 1.652399] systemd[1]: bpf-restrict-fs: LSM BPF program attached1690lb # [ 1.703636] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21691lb # [ 1.705748] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01692worker2 # [ 1.699974] systemd[1]: bpf-restrict-fs: LSM BPF program attached1693lb # [ 1.746009] systemd[1]: Queued start job for default target Initrd Default Target.1694worker1 # [ 1.714425] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21695lb # [ 1.753407] systemd[1]: Created slice Slice /system/modprobe.1696worker1 # [ 1.717301] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01697lb # [ 1.754383] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1698lb # [ 1.755489] systemd[1]: Expecting device /dev/disk/by-label/nixos...1699lb # [ 1.756331] systemd[1]: Reached target Path Units.1700lb # [ 1.757034] systemd[1]: Reached target Slice Units.1701client # [ 1.714413] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21702lb # [ 1.757690] systemd[1]: Reached target Swaps.1703worker2 # [ 1.716479] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21704lb # [ 1.758279] systemd[1]: Reached target Timer Units.1705lb # [ 1.759079] systemd[1]: Listening on D-Bus System Message Bus Socket.1706lb # [ 1.760128] systemd[1]: Listening on Journal Socket (/dev/log).1707lb # [ 1.761014] systemd[1]: Listening on Journal Sockets.1708lb # [ 1.761798] systemd[1]: Listening on udev Control Socket.1709client # [ 1.719970] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01710lb # [ 1.762595] systemd[1]: Listening on udev Kernel Socket.1711lb # [ 1.763286] systemd[1]: Reached target Socket Units.1712worker2 # [ 1.722717] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01713lb # [ 1.765388] systemd[1]: Starting Create List of Static Device Nodes...1714client # [ 1.730897] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715lb # [ 1.774185] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1716lb # [ 1.775176] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1717worker1 # [ 1.750346] systemd[1]: Queued start job for default target Initrd Default Target.1718lb # [ 1.784723] systemd[1]: Mounting Kernel Configuration File System...1719worker1 # [ 1.762139] systemd[1]: Created slice Slice /system/modprobe.1720lb # [ 1.799355] systemd[1]: Starting Journal Service...1721worker1 # [ 1.763505] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1722worker1 # [ 1.763568] systemd[1]: Expecting device /dev/disk/by-label/nixos...1723worker1 # [ 1.763605] systemd[1]: Reached target Path Units.1724worker1 # [ 1.763627] systemd[1]: Reached target Slice Units.1725lb # [ 1.803603] systemd[1]: Starting Load Kernel Modules...1726worker1 # [ 1.763648] systemd[1]: Reached target Swaps.1727worker1 # [ 1.763668] systemd[1]: Reached target Timer Units.1728lb # [ 1.803713] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1729worker1 # [ 1.763891] systemd[1]: Listening on D-Bus System Message Bus Socket.1730worker1 # [ 1.764093] systemd[1]: Listening on Journal Socket (/dev/log).1731worker1 # [ 1.764272] systemd[1]: Listening on Journal Sockets.1732worker1 # [ 1.764460] systemd[1]: Listening on udev Control Socket.1733worker1 # [ 1.764620] systemd[1]: Listening on udev Kernel Socket.1734worker1 # [ 1.764658] systemd[1]: Reached target Socket Units.1735worker1 # [ 1.777895] systemd[1]: Starting Create List of Static Device Nodes...1736lb # [ 1.819654] systemd[1]: Starting Coldplug All udev Devices...1737worker1 # [ 1.785055] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1738worker1 # [ 1.786222] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1739lb # [ 1.823321] systemd[1]: Finished Create List of Static Device Nodes.1740lb # [ 1.829081] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1741worker1 # [ 1.795592] systemd[1]: Mounting Kernel Configuration File System...1742worker1 # [ 1.803567] systemd[1]: Starting Journal Service...1743lb # [ 1.843693] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1744lb # [ 1.849122] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1745lb # [ 1.850347] systemd[1]: Mounted Kernel Configuration File System.1746worker2 # [ 1.814460] systemd[1]: Queued start job for default target Initrd Default Target.1747worker2 # [ 1.824900] systemd[1]: Created slice Slice /system/modprobe.1748worker2 # [ 1.826002] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1749worker2 # [ 1.827325] systemd[1]: Expecting device /dev/disk/by-label/nixos...1750worker2 # [ 1.828333] systemd[1]: Reached target Path Units.1751worker2 # [ 1.829088] systemd[1]: Reached target Slice Units.1752worker2 # [ 1.829856] systemd[1]: Reached target Swaps.1753worker1 # [ 1.831616] systemd[1]: Starting Load Kernel Modules...1754worker2 # [ 1.830592] systemd[1]: Reached target Timer Units.1755worker1 # [ 1.831744] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1756worker2 # [ 1.831532] systemd[1]: Listening on D-Bus System Message Bus Socket.1757worker1 # [ 1.834879] systemd[1]: Starting Coldplug All udev Devices...1758worker2 # [ 1.832685] systemd[1]: Listening on Journal Socket (/dev/log).1759worker2 # [ 1.833740] systemd[1]: Listening on Journal Sockets.1760lb # [ 1.871971] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1761worker2 # [ 1.834683] systemd[1]: Listening on udev Control Socket.1762client # [ 1.834959] systemd[1]: Queued start job for default target Initrd Default Target.1763worker2 # [ 1.834797] systemd[1]: Listening on udev Kernel Socket.1764worker2 # [ 1.834819] systemd[1]: Reached target Socket Units.1765lb # [ 1.878636] systemd-journald[73]: Collecting audit messages is disabled.1766worker2 # [ 1.842538] systemd[1]: Starting Create List of Static Device Nodes...1767client # [ 1.843110] systemd[1]: Created slice Slice /system/modprobe.1768worker2 # [ 1.845188] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1769client # [ 1.844271] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1770worker2 # [ 1.846232] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1771client # [ 1.845647] systemd[1]: Expecting device /dev/disk/by-label/nixos...1772client # [ 1.845683] systemd[1]: Reached target Path Units.1773client # [ 1.845701] systemd[1]: Reached target Slice Units.1774client # [ 1.845720] systemd[1]: Reached target Swaps.1775client # [ 1.845737] systemd[1]: Reached target Timer Units.1776client # [ 1.845925] systemd[1]: Listening on D-Bus System Message Bus Socket.1777client # [ 1.846070] systemd[1]: Listening on Journal Socket (/dev/log).1778client # [ 1.846202] systemd[1]: Listening on Journal Sockets.1779lb # [ 1.891559] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1780client # [ 1.846332] systemd[1]: Listening on udev Control Socket.1781client # [ 1.846441] systemd[1]: Listening on udev Kernel Socket.1782client # [ 1.846462] systemd[1]: Reached target Socket Units.1783worker2 # [ 1.854495] systemd[1]: Mounting Kernel Configuration File System...1784worker1 # [ 1.854696] systemd-journald[73]: Collecting audit messages is disabled.1785worker1 # [ 1.854823] systemd[1]: Finished Create List of Static Device Nodes.1786client # [ 1.856888] systemd[1]: Starting Create List of Static Device Nodes...1787worker1 # [ 1.855813] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1788worker1 # [ 1.864330] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1789worker2 # [ 1.862503] systemd[1]: Starting Journal Service...1790worker1 # [ 1.865713] systemd[1]: Mounted Kernel Configuration File System.1791client # [ 1.864923] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1792client # [ 1.866035] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1793lb # [ 1.908224] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01794lb # [ 1.908465] [drm] features: -virgl +edid -resource_blob -host_visible1795lb # [ 1.908476] [drm] features: -context_init1796client # [ 1.872235] systemd[1]: Mounting Kernel Configuration File System...1797lb # [ 1.909185] [drm] number of scanouts: 11798lb # [ 1.909202] [drm] number of cap sets: 01799lb # [ 1.913141] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1800worker1 # [ 1.879823] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1801lb # [ 1.919810] systemd[1]: Starting Create Static Device Nodes in /dev...1802worker2 # [ 1.887569] systemd[1]: Starting Load Kernel Modules...1803client # [ 1.887774] systemd[1]: Starting Journal Service...1804worker2 # [ 1.890578] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1805lb # [ 1.927723] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1806lb # [ 1.927740] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01807client # [ 1.909136] systemd[1]: Starting Load Kernel Modules...1808worker2 # [ 1.906491] systemd[1]: Starting Coldplug All udev Devices...1809worker2 # [ 1.908016] systemd[1]: Finished Create List of Static Device Nodes.1810client # [ 1.909973] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1811worker2 # [ 1.908840] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1812worker2 # [ 1.909149] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1813worker2 # [ 1.909374] systemd[1]: Mounted Kernel Configuration File System.1814worker1 # [ 1.912020] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1815worker2 # [ 1.915216] systemd-journald[73]: Collecting audit messages is disabled.1816worker1 # [ 1.916031] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1817worker1 # [ 1.920045] systemd[1]: Starting Create Static Device Nodes in /dev...1818lb # [ 1.956079] Console: switching to colour frame buffer device 160x501819lb # [ 1.962697] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1820client # [ 1.932709] systemd-journald[73]: Collecting audit messages is disabled.1821worker2 # [ 1.939089] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1822worker1 # [ 1.939484] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1823worker1 # [ 1.944077] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01824client # [ 1.941164] systemd[1]: Starting Coldplug All udev Devices...1825worker1 # [ 1.944309] [drm] features: -virgl +edid -resource_blob -host_visible1826worker1 # [ 1.944319] [drm] features: -context_init1827worker1 # [ 1.945046] [drm] number of scanouts: 11828worker1 # [ 1.945065] [drm] number of cap sets: 01829lb # [ 1.984054] systemd[1]: Finished Load Kernel Modules.1830lb # [ 1.987242] systemd[1]: Starting Apply Kernel Variables...1831client # [ 1.952995] systemd[1]: Finished Create List of Static Device Nodes.1832client # [ 1.953905] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1833client # [ 1.954222] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1834worker2 # [ 1.956762] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1835worker2 # [ 1.959396] systemd[1]: Starting Create Static Device Nodes in /dev...1836worker1 # [ 1.964063] systemd[1]: Finished Create Static Device Nodes in /dev.1837worker1 # [ 1.964305] systemd[1]: Reached target Preparation for Local File Systems.1838worker1 # [ 1.964333] systemd[1]: Reached target Local File Systems.1839worker1 # [ 1.968351] systemd[1]: Starting Rule-based Manager for Device Events and Files...1840lb # [ 2.007611] systemd[1]: Finished Create Static Device Nodes in /dev.1841worker1 # [ 1.972736] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1842worker1 # [ 1.972765] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01843client # [ 1.972901] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1844lb # [ 2.009909] systemd[1]: Reached target Preparation for Local File Systems.1845lb # [ 2.009954] systemd[1]: Reached target Local File Systems.1846worker2 # [ 1.981547] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1847lb # [ 2.019856] systemd[1]: Starting Rule-based Manager for Device Events and Files...1848client # [ 1.988866] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1849client # [ 1.993436] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01850client # [ 1.993670] [drm] features: -virgl +edid -resource_blob -host_visible1851client # [ 1.993680] [drm] features: -context_init1852client # [ 1.994447] [drm] number of scanouts: 11853client # [ 1.994467] [drm] number of cap sets: 01854worker2 # [ 1.994730] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1855worker2 # [ 1.999390] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01856client # [ 2.000951] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1857worker2 # [ 1.999625] [drm] features: -virgl +edid -resource_blob -host_visible1858worker2 # [ 1.999634] [drm] features: -context_init1859worker2 # [ 2.000375] [drm] number of scanouts: 11860lb # [ 2.039577] systemd[1]: Finished Apply Kernel Variables.1861worker2 # [ 2.000393] [drm] number of cap sets: 01862worker1 # [ 2.003842] Console: switching to colour frame buffer device 160x501863client # [ 2.013115] systemd[1]: Mounted Kernel Configuration File System.1864worker2 # [ 2.018925] systemd[1]: Finished Create Static Device Nodes in /dev.1865worker2 # [ 2.019317] systemd[1]: Reached target Preparation for Local File Systems.1866client # [ 2.017156] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1867worker2 # [ 2.019347] systemd[1]: Reached target Local File Systems.1868client # [ 2.017173] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01869worker2 # [ 2.023382] systemd[1]: Starting Rule-based Manager for Device Events and Files...1870worker2 # [ 2.027683] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1871worker1 # [ 2.032009] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1872worker2 # [ 2.027704] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01873lb # [ 2.067637] systemd[1]: Started Journal Service.1874lb # [ 2.049661] systemd-modules-load[74]: Inserted module 'dm_mod'1875lb # [ 2.052979] systemd-modules-load[74]: Module 'virtio_balloon' is built in1876worker1 # [ 2.040891] systemd[1]: Finished Load Kernel Modules.1877client # [ 2.038838] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1878lb # [ 2.060391] systemd-modules-load[74]: Module 'virtio_console' is built in1879lb # [ 2.061475] systemd-modules-load[74]: Inserted module 'virtio_gpu'1880lb # [ 2.062447] systemd-modules-load[74]: Module 'virtio_rng' is built in1881lb # [ 2.063569] systemd[1]: Starting Create System Files and Directories...1882client # [ 2.045122] systemd[1]: Starting Create Static Device Nodes in /dev...1883worker1 # [ 2.055836] systemd[1]: Starting Apply Kernel Variables...1884client # [ 2.053091] Console: switching to colour frame buffer device 160x501885worker2 # [ 2.058611] Console: switching to colour frame buffer device 160x501886worker2 # [ 2.065378] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1887worker1 # [ 2.059671] systemd-modules-load[74]: Inserted module 'dm_mod'1888worker1 # [ 2.060896] systemd-modules-load[74]: Module 'virtio_balloon' is built in1889worker1 # [ 2.062004] systemd-modules-load[74]: Module 'virtio_console' is built in1890worker1 # [ 2.063061] systemd-modules-load[74]: Inserted module 'virtio_gpu'1891lb # [ 2.096531] systemd-udevd[81]: Using default interface naming scheme 'v261'.1892worker1 # [ 2.079664] systemd[1]: Started Journal Service.1893lb # [ 2.104473] systemd[1]: Finished Create System Files and Directories.1894client # [ 2.085460] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1895client # [ 2.089669] systemd[1]: Finished Create Static Device Nodes in /dev.1896client # [ 2.091586] systemd[1]: Reached target Preparation for Local File Systems.1897worker1 # [ 2.080984] systemd-modules-load[74]: Module 'virtio_rng' is built in1898client # [ 2.093395] systemd[1]: Reached target Local File Systems.1899worker1 # [ 2.084625] systemd-udevd[80]: Using default interface naming scheme 'v261'.1900worker2 # [ 2.098962] systemd[1]: Finished Load Kernel Modules.1901worker1 # [ 2.089301] systemd[1]: Starting Create System Files and Directories...1902client # [ 2.101283] systemd[1]: Starting Rule-based Manager for Device Events and Files...1903worker2 # [ 2.093009] systemd-modules-load[75]: Inserted module 'dm_mod'1904worker1 # [ 2.096472] systemd[1]: Finished Apply Kernel Variables.1905worker2 # [ 2.108685] systemd[1]: Starting Apply Kernel Variables...1906worker2 # [ 2.108860] systemd[1]: Started Journal Service.1907client # [ 2.108973] systemd[1]: Finished Load Kernel Modules.1908lb # [ 2.130153] systemd[1]: Started Rule-based Manager for Device Events and Files.1909worker2 # [ 2.098351] systemd-modules-load[75]: Module 'virtio_balloon' is built in1910worker2 # [ 2.099491] systemd-modules-load[75]: Module 'virtio_console' is built in1911worker2 # [ 2.108426] systemd-modules-load[75]: Inserted module 'virtio_gpu'1912worker2 # [ 2.109477] systemd-modules-load[75]: Module 'virtio_rng' is built in1913client # [ 2.128877] systemd[1]: Starting Apply Kernel Variables...1914worker2 # [ 2.121357] systemd-udevd[79]: Using default interface naming scheme 'v261'.1915client # [ 2.126222] systemd-modules-load[74]: Inserted module 'dm_mod'1916client # [ 2.140090] systemd[1]: Started Journal Service.1917worker2 # [ 2.129844] systemd[1]: Starting Create System Files and Directories...1918client # [ 2.132258] systemd-modules-load[74]: Module 'virtio_balloon' is built in1919worker1 # [ 2.133427] systemd[1]: Finished Create System Files and Directories.1920worker1 # [ 2.135269] systemd[1]: Started Rule-based Manager for Device Events and Files.1921client # [ 2.144299] systemd-modules-load[74]: Module 'virtio_console' is built in1922client # [ 2.148337] systemd-modules-load[74]: Inserted module 'virtio_gpu'1923client # [ 2.152363] systemd-modules-load[74]: Module 'virtio_rng' is built in1924client # [ 2.159707] systemd[1]: Starting Create System Files and Directories...1925lb # [ 2.192101] systemd[1]: Starting Virtual Console Setup...1926worker2 # [ 2.162543] systemd[1]: Finished Create System Files and Directories.1927worker2 # [ 2.168535] systemd[1]: Finished Apply Kernel Variables.1928client # [ 2.184157] systemd[1]: Finished Apply Kernel Variables.1929worker2 # [ 2.185439] systemd[1]: Started Rule-based Manager for Device Events and Files.1930worker1 # [ 2.192411] systemd[1]: Starting Virtual Console Setup...1931client # [ 2.196797] systemd-udevd[80]: Using default interface naming scheme 'v261'.1932client # [ 2.198828] systemd[1]: Finished Create System Files and Directories.1933lb # [ 2.240496] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1934lb # [ 2.244087] systemd[1]: Finished Virtual Console Setup.1935client # [ 2.229763] systemd[1]: Started Rule-based Manager for Device Events and Files.1936worker1 # [ 2.244475] systemd-vconsole-setup[104]: Configuration of first virtual console was skipped, ignoring remaining ones.1937worker2 # [ 2.244241] systemd[1]: Starting Virtual Console Setup...1938worker1 # [ 2.248058] systemd[1]: Finished Virtual Console Setup.1939client # [ 2.288168] systemd[1]: Starting Virtual Console Setup...1940worker2 # [ 2.288539] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1941worker2 # [ 2.292309] systemd[1]: Finished Virtual Console Setup.1942client # [ 2.340496] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1943client # [ 2.343975] systemd[1]: Finished Virtual Console Setup.1944lb # [ 2.830369] systemd[1]: Finished Coldplug All udev Devices.1945lb # [ 2.832136] systemd[1]: Reached target System Initialization.1946lb # [ 2.836120] systemd[1]: Reached target Basic System.1947worker1 # [ 2.845631] systemd[1]: Finished Coldplug All udev Devices.1948worker1 # [ 2.847176] systemd[1]: Reached target System Initialization.1949worker1 # [ 2.848144] systemd[1]: Reached target Basic System.1950worker2 # [ 2.913609] systemd[1]: Finished Coldplug All udev Devices.1951worker2 # [ 2.914597] systemd[1]: Reached target System Initialization.1952worker2 # [ 2.915416] systemd[1]: Reached target Basic System.1953client # [ 2.926724] systemd[1]: Finished Coldplug All udev Devices.1954client # [ 2.927680] systemd[1]: Reached target System Initialization.1955client # [ 2.932212] systemd[1]: Reached target Basic System.1956lb # [ 2.979260] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1957lb # [ 3.008061] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1958lb # [ 3.010928] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1959worker1 # [ 2.999131] (udev-worker)[108]: Network interface NamePolicy= disabled on kernel command line.1960worker1 # [ 3.033933] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1961worker1 # [ 3.040386] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1962lb # [ 3.074025] systemd[1]: Found device /dev/disk/by-label/nixos.1963lb # [ 3.079090] systemd[1]: Reached target Initrd Root Device.1964worker2 # [ 3.048235] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1965lb # [ 3.083688] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1966worker2 # [ 3.084996] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1967worker2 # [ 3.098359] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1968client # [ 3.103038] (udev-worker)[102]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969lb # [ 3.137245] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1970client # [ 3.108380] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1971worker1 # [ 3.109995] systemd[1]: Found device /dev/disk/by-label/nixos.1972worker1 # [ 3.114329] systemd[1]: Reached target Initrd Root Device.1973worker1 # [ 3.116119] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1974lb # [ 3.146823] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1975lb # [ 3.148406] systemd[1]: Mounting /sysroot...1976client # [ 3.121282] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1977lb # [ 3.207483] EXT4-fs (vda): mounted filesystem 7f67a60e-a8b8-4e9f-b907-f8646fb6afc7 r/w with ordered data mode. Quota mode: none.1978lb # [ 3.193207] systemd[1]: Mounted /sysroot.1979worker2 # [ 3.164136] systemd[1]: Found device /dev/disk/by-label/nixos.1980worker1 # [ 3.166333] systemd-fsck[113]: nixos: clean, 12/65536 files, 13019/262144 blocks1981lb # [ 3.197337] systemd[1]: Reached target Initrd Root File System.1982worker2 # [ 3.166286] systemd[1]: Reached target Initrd Root Device.1983lb # [ 3.198692] systemd[1]: Starting Mountpoints Configured in the Real Root...1984worker2 # [ 3.169380] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985worker1 # [ 3.173992] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1986worker1 # [ 3.180167] systemd[1]: Mounting /sysroot...1987client # [ 3.188412] systemd[1]: Found device /dev/disk/by-label/nixos.1988lb # [ 3.224124] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1989lb # [ 3.230741] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1990lb # [ 3.235337] systemd[1]: Reloading...1991client # [ 3.204103] systemd[1]: Reached target Initrd Root Device.1992client # [ 3.209187] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1993worker1 # [ 3.230767] EXT4-fs (vda): mounted filesystem 376254bc-797f-4e24-bd30-4d24e02b2968 r/w with ordered data mode. Quota mode: none.1994worker1 # [ 3.220910] systemd[1]: Mounted /sysroot.1995worker1 # [ 3.222881] systemd[1]: Reached target Initrd Root File System.1996worker1 # [ 3.224177] systemd[1]: Starting Mountpoints Configured in the Real Root...1997worker2 # [ 3.223985] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1998worker2 # [ 3.232108] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1999worker2 # [ 3.236238] systemd[1]: Mounting /sysroot...2000worker1 # [ 3.251509] systemd-sysroot-fstab-check[121]: /sysroot should be mounted in the initrd, will request daemon-reload.
loading 1,491 hidden lines…
3492lb # [ 19.976940] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3493lb # [ 19.978903] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection3494lb # [ 19.985899] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3495lb # [ 19.988682] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3496lb # [ 19.991844] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re23497lb # [ 19.993838] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3498lb # [ 20.001380] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3499lb # [ 20.004618] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3500lb # [ 20.007268] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3501lb # [ 20.009840] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit3502lb # [ 20.012699] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3503lb # [ 20.015516] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo3504lb # [ 20.017777] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3505lb # [ 20.022527] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port3506lb # [ 20.031040] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3507lb # [ 20.033803] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3508lb # [ 20.035715] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy3509lb # [ 20.038870] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3510lb # [ 20.041552] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3511lb # [ 20.045965] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3512lb # [ 20.048061] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3513lb # [ 20.049748] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config3514lb # [ 20.053526] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter3515lb # [ 20.055699] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator3516lb # [ 20.058543] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection3517lb # [ 20.060518] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3518lb # [ 20.063037] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3519lb # [ 20.067943] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3520lb # [ 20.070237] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3521lb # [ 20.072273] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3522lb # [ 20.077273] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3523lb # [ 20.080158] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3524lb # [ 20.081911] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3525lb # [ 20.083787] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3526lb # [ 20.086680] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3527lb # [ 20.088612] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default3528lb # [ 20.090355] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme3529lb # [ 20.093916] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr3530lb # [ 20.095829] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case3531lb # [ 20.099629] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3532lb # [ 20.102078] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder3533lb # [ 20.105314] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3534lb # [ 20.111018] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3535lb # [ 20.113356] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3536lb # [ 20.115303] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin3537lb # [ 20.119486] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3538lb # [ 20.124625] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable3539lb # [ 20.126862] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default3540lb # [ 20.129018] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy3541lb # [ 20.143038] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23542lb # [ 20.144986] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3543lb # [ 20.147790] envoy[901]: [2026-09-20 14:30:53.393][901][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3544lb # [ 20.149983] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3545lb # [ 20.151764] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23546lb # [ 20.154756] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13547lb # [ 20.157182] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic3548lb # [ 20.159069] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3549lb # [ 20.163278] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3550lb # [ 20.166486] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3551lb # [ 20.168992] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3552lb # [ 20.171500] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3553lb # [ 20.174841] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3554lb # [ 20.176706] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift3555lb # [ 20.180123] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action3556lb # [ 20.182422] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3557lb # [ 20.184520] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3558lb # [ 20.186714] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3559lb # [ 20.198126] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3560lb # [ 20.199852] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3561lb # [ 20.202835] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal3562lb # [ 20.205099] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3563lb # [ 20.207015] envoy[901]: [2026-09-20 14:30:53.394][901][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3564lb # [ 20.209209] envoy[901]: [2026-09-20 14:30:53.414][901][info][main] [source/server/server.cc:531] HTTP header map info:3565lb # [ 20.210674] envoy[901]: [2026-09-20 14:30:53.416][901][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3566lb # [ 20.226596] envoy[901]: [2026-09-20 14:30:53.416][901][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3567lb # [ 20.228614] envoy[901]: [2026-09-20 14:30:53.416][901][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3568lb # [ 20.237485] envoy[901]: [2026-09-20 14:30:53.416][901][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3569lb # [ 20.239483] envoy[901]: [2026-09-20 14:30:53.667][901][info][main] [source/server/server.cc:968] runtime: {}3570lb # [ 20.240986] envoy[901]: [2026-09-20 14:30:53.691][901][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3571lb # [ 20.242715] envoy[901]: [2026-09-20 14:30:53.691][901][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013572lb # [ 20.244584] envoy[901]: [2026-09-20 14:30:53.692][901][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013573lb # [ 20.246316] envoy[901]: [2026-09-20 14:30:53.702][901][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3574lb # [ 20.247914] envoy[901]: [2026-09-20 14:30:53.702][901][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3575lb # [ 20.249786] envoy[901]: [2026-09-20 14:30:53.702][901][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3576lb # [ 20.271472] envoy[901]: [2026-09-20 14:30:53.858][901][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3577worker2 # [ 20.243973] systemd[1]: Reached target Network is Online.3578lb # [ 20.282035] envoy[901]: [2026-09-20 14:30:53.869][901][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3579lb # [ 20.284839] envoy[901]: [2026-09-20 14:30:53.871][901][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3580worker1 # [ 20.253973] systemd[1]: Starting Nix worker-protocol over gRPC...3581lb # [ 20.288199] envoy[901]: [2026-09-20 14:30:53.871][901][info][main] [source/server/server.cc:1077] starting main dispatch loop3582worker2 # [ 20.255119] systemd[1]: Starting scheduler-token.service...3583worker2 # [ 20.376781] systemd[1]: scheduler-token.service: Deactivated successfully.3584worker2 # [ 20.383329] systemd[1]: Finished scheduler-token.service.3585worker2 # [ 20.385554] systemd[1]: scheduler-token.service: Consumed 53ms CPU time over 124ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3586worker2 # [ 20.391036] systemd[1]: Starting Nix worker-protocol over gRPC...3587worker1 # [ 20.503311] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3588worker1 # [ 20.515528] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3589worker1 # [ 20.518137] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13590worker1 # [ 20.568805] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3591worker1 # [ 20.570524] systemd[1]: Started Nix worker-protocol over gRPC.3592worker2 # [ 20.607250] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3593lb # [ 20.638820] envoy[901]: [2026-09-20 14:30:54.225][901][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3594lb # [ 20.645844] envoy[901]: [2026-09-20 14:30:54.225][901][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3595worker2 # [ 20.620521] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3596lb # [ 20.651396] envoy[901]: [2026-09-20 14:30:54.225][901][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3597worker2 # [ 20.622896] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13598lb # [ 20.656906] envoy[901]: [2026-09-20 14:30:54.225][901][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3599worker1 # [ 20.643291] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=scheduler_connected addr=lb:500513600worker1 # [ 20.647684] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3601worker2 # [ 20.666020] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3602worker2 # [ 20.667695] systemd[1]: Started Nix worker-protocol over gRPC.3603worker2 # [ 20.729053] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=scheduler_connected addr=lb:500513604worker2 # [ 20.730873] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3605worker1 # [ 20.848863] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=scheduler_connected addr=lb:500513606worker1 # [ 20.853927] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3607worker2 # [ 20.932393] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=scheduler_connected addr=lb:500513608worker2 # [ 20.937280] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3609worker1 # [ 21.054762] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=scheduler_connected addr=lb:500513610worker1 # [ 21.059878] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3611worker2 # [ 21.138411] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=scheduler_connected addr=lb:500513612worker2 # [ 21.143421] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3613worker1 # [ 21.260717] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=scheduler_connected addr=lb:500513614worker1 # [ 21.265827] nix-grpc-daemon[775]: ts=2026-09-20T14:30:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3615worker2 # [ 21.344691] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=scheduler_connected addr=lb:500513616worker2 # [ 21.349544] nix-grpc-daemon[778]: ts=2026-09-20T14:30:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3617worker1 # [ 21.466597] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513618worker1 # [ 21.471635] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3619worker2 # [ 21.550708] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513620worker2 # [ 21.555669] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3621worker1 # [ 21.674137] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513622worker1 # [ 21.678264] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3623worker2 # [ 21.758385] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513624worker2 # [ 21.762439] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3625worker1 # [ 21.874582] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513626worker1 # [ 21.879696] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3627worker2 # [ 21.959042] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513628worker2 # [ 21.964178] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3629worker1 # [ 22.080592] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513630worker1 # [ 22.085535] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3631worker2 # [ 22.165291] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513632worker2 # [ 22.170781] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3633worker1 # [ 22.286406] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513634worker1 # [ 22.291496] nix-grpc-daemon[775]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3635worker2 # [ 22.371916] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_connected addr=lb:500513636worker2 # [ 22.376695] nix-grpc-daemon[778]: ts=2026-09-20T14:30:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3637worker1 # [ 22.492323] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513638worker1 # [ 22.501331] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3639worker2 # [ 22.578079] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513640worker2 # [ 22.583326] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3641worker1 # [ 22.699963] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513642worker1 # [ 22.703760] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3643worker2 # [ 22.786040] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513644worker2 # [ 22.790173] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3645worker1 # [ 22.900756] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513646worker1 # [ 22.906280] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3647worker2 # [ 22.987008] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513648worker2 # [ 22.992387] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3649worker1 # [ 23.107145] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513650worker1 # [ 23.112402] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3651worker2 # [ 23.193521] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513652worker2 # [ 23.198818] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3653worker1 # [ 23.313111] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513654worker1 # [ 23.318111] nix-grpc-daemon[775]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3655worker2 # [ 23.399884] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_connected addr=lb:500513656worker2 # [ 23.405169] nix-grpc-daemon[778]: ts=2026-09-20T14:30:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3657worker1 # [ 23.518931] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_connected addr=lb:500513658worker1 # [ 23.524146] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3659worker2 # [ 23.606196] nix-grpc-daemon[778]: ts=2026-09-20T14:30:57Z level=info event=scheduler_connected addr=lb:500513660worker2 # [ 23.611486] nix-grpc-daemon[778]: ts=2026-09-20T14:30:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3661worker1 # [ 23.726008] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_connected addr=lb:500513662worker1 # [ 23.730692] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3663worker2 # [ 23.814267] nix-grpc-daemon[778]: ts=2026-09-20T14:30:57Z level=info event=scheduler_connected addr=lb:500513664worker2 # [ 23.818385] nix-grpc-daemon[778]: ts=2026-09-20T14:30:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3665worker1 # [ 23.927675] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_connected addr=lb:500513666worker1 # [ 23.932863] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3667worker2 # [ 24.015169] nix-grpc-daemon[778]: ts=2026-09-20T14:30:57Z level=info event=scheduler_connected addr=lb:500513668worker2 # [ 24.020448] nix-grpc-daemon[778]: ts=2026-09-20T14:30:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3669worker1 # [ 24.133731] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_connected addr=lb:500513670worker1 # [ 24.138887] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3671worker2 # [ 24.221643] nix-grpc-daemon[778]: ts=2026-09-20T14:30:57Z level=info event=scheduler_connected addr=lb:500513672worker2 # [ 24.226875] nix-grpc-daemon[778]: ts=2026-09-20T14:30:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3673worker1 # [ 24.339749] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_connected addr=lb:500513674worker1 # [ 24.344826] nix-grpc-daemon[775]: ts=2026-09-20T14:30:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3675worker2 # [ 24.427938] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513676worker2 # [ 24.433044] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3677worker1 # [ 24.545625] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513678worker1 # [ 24.550651] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3679worker2 # [ 24.634480] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513680worker2 # [ 24.639669] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3681worker1 # [ 24.752836] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513682worker1 # [ 24.757464] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3683worker2 # [ 24.842751] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513684worker2 # [ 24.846910] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3685worker1 # [ 24.958233] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513686worker1 # [ 24.963383] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3687worker2 # [ 25.043613] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513688worker2 # [ 25.049073] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3689worker1 # [ 25.164105] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513690worker1 # [ 25.169160] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3691worker2 # [ 25.250137] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513692worker2 # [ 25.255497] nix-grpc-daemon[778]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3693worker1 # [ 25.369967] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_connected addr=lb:500513694worker1 # [ 25.374890] nix-grpc-daemon[775]: ts=2026-09-20T14:30:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3695worker2 # [ 25.456699] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513696worker2 # [ 25.461615] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3697worker1 # [ 25.575663] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513698worker1 # [ 25.580801] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3699worker2 # [ 25.662564] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513700worker2 # [ 25.666532] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3701worker1 # [ 25.783320] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513702worker1 # [ 25.787698] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3703worker2 # [ 25.869075] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513704worker2 # [ 25.873089] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3705worker1 # [ 25.983848] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513706worker1 # [ 25.989178] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3707worker2 # [ 26.069855] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513708worker2 # [ 26.074930] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3709worker1 # [ 26.190085] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513710worker1 # [ 26.195230] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3711worker2 # [ 26.276199] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513712worker2 # [ 26.281282] nix-grpc-daemon[778]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3713worker1 # [ 26.396120] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_connected addr=lb:500513714worker1 # [ 26.401279] nix-grpc-daemon[775]: ts=2026-09-20T14:30:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3715worker2 # [ 26.482536] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513716worker2 # [ 26.487743] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3717worker1 # [ 26.602192] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513718worker1 # [ 26.607538] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3719worker2 # [ 26.688869] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513720worker2 # [ 26.693954] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3721worker1 # [ 26.810318] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513722worker1 # [ 26.814486] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3723worker2 # [ 26.896866] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513724worker2 # [ 26.900945] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3725worker1 # [ 27.010980] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513726worker1 # [ 27.016379] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker2 # [ 27.098035] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513728worker2 # [ 27.103129] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3729worker1 # [ 27.217108] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513730worker1 # [ 27.222163] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker2 # [ 27.304122] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513732worker2 # [ 27.309063] nix-grpc-daemon[778]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3733worker1 # [ 27.422933] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_connected addr=lb:500513734worker1 # [ 27.428153] nix-grpc-daemon[775]: ts=2026-09-20T14:31:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker2 # [ 27.510259] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_connected addr=lb:500513736worker2 # [ 27.515320] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3737worker1 # [ 27.628911] nix-grpc-daemon[775]: ts=2026-09-20T14:31:01Z level=info event=scheduler_connected addr=lb:500513738worker1 # [ 27.635519] nix-grpc-daemon[775]: ts=2026-09-20T14:31:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3739worker2 # [ 27.716384] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_connected addr=lb:500513740worker2 # [ 27.721346] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3741worker1 # [ 27.837905] nix-grpc-daemon[775]: ts=2026-09-20T14:31:01Z level=info event=scheduler_connected addr=lb:500513742worker1 # [ 27.842010] nix-grpc-daemon[775]: ts=2026-09-20T14:31:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3743worker2 # [ 27.924480] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_connected addr=lb:500513744worker2 # [ 27.928536] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3745worker1 # [ 28.038693] nix-grpc-daemon[775]: ts=2026-09-20T14:31:01Z level=info event=scheduler_connected addr=lb:500513746worker1 # [ 28.043832] nix-grpc-daemon[775]: ts=2026-09-20T14:31:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3747worker2 # [ 28.125337] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_connected addr=lb:500513748worker2 # [ 28.130621] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3749worker1 # [ 28.244661] nix-grpc-daemon[775]: ts=2026-09-20T14:31:01Z level=info event=scheduler_connected addr=lb:500513750worker1 # [ 28.249708] nix-grpc-daemon[775]: ts=2026-09-20T14:31:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751worker2 # [ 28.332193] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_connected addr=lb:500513752worker2 # [ 28.337262] nix-grpc-daemon[778]: ts=2026-09-20T14:31:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3753worker1 # [ 28.450625] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513754worker1 # [ 28.455790] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3755worker2 # [ 28.538437] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513756worker2 # [ 28.543729] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3757worker1 # [ 28.656642] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513758worker1 # [ 28.661882] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3759worker2 # [ 28.744881] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513760lb # [ 28.782045] rustfs-bucket-start[1000]: mb s3://farm3761worker2 # [ 28.750074] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3762lb # [ 28.792758] systemd[1]: Finished rustfs-bucket.service.3763lb # [ 28.803977] systemd[1]: Starting niks3 server...3764worker1 # [ 28.864528] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513765worker1 # [ 28.868999] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3766worker2 # [ 28.953061] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513767worker2 # [ 28.964187] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3768lb # [ 29.060799] postgres[1017]: [1017] ERROR: relation "goose_db_version" does not exist at character 363769lb # [ 29.062082] postgres[1017]: [1017] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3770lb # [ 29.096971] niks3-server[1010]: 2026/09/20 14:31:02 OK 20241026095416_initial_model.sql (21.43ms)3771worker1 # [ 29.064755] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513772lb # [ 29.100406] niks3-server[1010]: 2026/09/20 14:31:02 OK 20251210153512_drop_unused_gin_index.sql (1.85ms)3773worker1 # [ 29.069867] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774lb # [ 29.106017] niks3-server[1010]: 2026/09/20 14:31:02 OK 20251218171726_add_pins.sql (5.54ms)3775lb # [ 29.110680] niks3-server[1010]: 2026/09/20 14:31:02 OK 20260628120000_add_object_size_and_stats.sql (4.63ms)3776lb # [ 29.115533] niks3-server[1010]: 2026/09/20 14:31:02 OK 20260905000000_add_claims.sql (4.82ms)3777lb # [ 29.120053] niks3-server[1010]: 2026/09/20 14:31:02 OK 20260920000000_drop_claims.sql (4.37ms)3778lb # [ 29.121292] niks3-server[1010]: 2026/09/20 14:31:02 goose: successfully migrated database to version: 202609200000003779lb # [ 29.126510] niks3-server[1010]: 2026/09/20 14:31:02 OK 1_commit_pending_closure.sql (6.46ms)3780lb # [ 29.129141] niks3-server[1010]: 2026/09/20 14:31:02 OK 2_object_stats_trigger.sql (2.51ms)3781lb # [ 29.130552] niks3-server[1010]: 2026/09/20 14:31:02 goose: up to current file version: 23782lb # [ 29.134378] niks3-server[1010]: 2026/09/20 14:31:02 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3783lb # [ 29.179157] niks3-server[1010]: 2026/09/20 14:31:02 INFO Read proxy enabled — serving cache objects from S33784lb # [ 29.181143] niks3-server[1010]: 2026/09/20 14:31:02 INFO Using socket-activated listener address=0.0.0.0:57513785lb # [ 29.185846] niks3-server[1010]: 2026/09/20 14:31:02 INFO systemd watchdog enabled interval=15s3786worker2 # [ 29.153773] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513787lb # [ 29.189176] niks3-server[1010]: 2026/09/20 14:31:02 INFO Starting HTTP server address=0.0.0.0:57513788lb # [ 29.190817] systemd[1]: Started niks3 server.3789lb # [ 29.191501] systemd[1]: Reached target Multi-User System.3790lb # [ 29.192931] systemd[1]: Startup finished in 983ms (kernel) + 5.347s (initrd) + 22.856s (userspace) = 29.187s.3791worker2 # [ 29.159036] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3792lb # [ 29.200130] niks3-server[1010]: 2026/09/20 14:31:02 INFO lead: acquired remote=192.168.1.4:389223793worker2 # [ 29.172610] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_take_over3794worker1 # [ 29.270521] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513795worker1 # [ 29.273802] nix-grpc-daemon[775]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3796lb: (finished: waiting for unit niks3.service, in 15.76 seconds)3797lb: waiting for TCP port 5751 on localhost3798lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3799worker2 # [ 29.360249] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_connected addr=lb:500513800lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)3801lb: waiting for unit envoy.service3802worker2 # [ 29.365245] nix-grpc-daemon[778]: ts=2026-09-20T14:31:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3803lb: (finished: waiting for unit envoy.service, in 0.06 seconds)3804lb: waiting for TCP port 50051 on localhost3805lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3806lb # [ 29.498058] envoy[901]: conn peer=[::1]:58792 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33807lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3808worker1 # [ 29.475693] nix-grpc-daemon[775]: ts=2026-09-20T14:31:03Z level=info event=scheduler_connected addr=lb:500513809worker1 # [ 29.482115] nix-grpc-daemon[775]: ts=2026-09-20T14:31:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3810worker2 # [ 29.566482] nix-grpc-daemon[778]: ts=2026-09-20T14:31:03Z level=info event=scheduler_connected addr=lb:500513811worker2 # [ 29.576169] nix-grpc-daemon[778]: ts=2026-09-20T14:31:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3812worker2 # [ 29.654728] systemd[1]: Starting scheduler-token.service...3813worker1 # [ 29.676555] nix-grpc-daemon[775]: ts=2026-09-20T14:31:03Z level=info event=scheduler_connected addr=lb:500513814worker1 # [ 29.681864] nix-grpc-daemon[775]: ts=2026-09-20T14:31:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3815worker2 # [ 29.746351] systemd[1]: scheduler-token.service: Deactivated successfully.3816worker2 # [ 29.750199] systemd[1]: Finished scheduler-token.service.3817worker2 # [ 29.751326] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3818lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3819lb: must succeed: curl -sf localhost:9901/clusters3820worker2 # [ 29.772792] nix-grpc-daemon[778]: ts=2026-09-20T14:31:03Z level=info event=scheduler_connected addr=lb:500513821worker2 # [ 29.792100] nix-grpc-daemon[778]: ts=2026-09-20T14:31:03Z level=info event=oidc_jwks provider=mock keys=13822worker2 # [ 29.795472] nix-grpc-daemon[778]: ts=2026-09-20T14:31:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:345503823worker2 # [ 29.799336] nix-grpc-daemon[778]: ts=2026-09-20T14:31:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03824lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3825lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3826subtest: one node schedules and both builders hold a WorkerSession on it3827lb: must succeed: curl -sf localhost:9901/clusters3828worker1 # [ 29.883889] nix-grpc-daemon[775]: ts=2026-09-20T14:31:03Z level=info event=scheduler_connected addr=lb:500513829worker2 # [ 29.882805] nix-grpc-daemon[778]: ts=2026-09-20T14:31:03Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:345503830worker2 # [ 29.888405] nix-grpc-daemon[778]: ts=2026-09-20T14:31:03Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03831lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3832worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3833worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)3834lb: must succeed: curl -sf localhost:9901/clusters3835lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3836lb: must succeed: curl -sf localhost:9901/clusters3837lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3838lb: must succeed: curl -sf localhost:9901/clusters3839lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3840lb: must succeed: curl -sf localhost:9901/clusters3841lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)3842worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3843worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)3844worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3845worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3846lb: must succeed: curl -sf localhost:9901/clusters3847lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3848lb: must succeed: curl -sf localhost:9901/clusters3849lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3850worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3851worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3852worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3853worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)3854worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3855worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3856worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3857worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)3858(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.83 seconds)3859subtest: balancer auth: client cert, bearer token, nothing, unknown CN3860worker2 # [ 31.049555] nix-grpc-daemon[778]: ts=2026-09-20T14:31:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:486843861worker2 # [ 31.066687] systemd[1]: Started Nix Daemon.3862worker2 # [ 31.323980] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)3863worker2 # [ 31.361674] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5)3864worker2 # [ 31.408644] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 2/5)3865worker2 # [ 31.449993] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 3/5)3866worker2 # [ 31.626740] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 4/5)3867worker2 # [ 31.691526] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3868worker1 # [ 31.731002] nix-grpc-daemon[775]: ts=2026-09-20T14:31:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:339663869worker1 # [ 31.749150] systemd[1]: Started Nix Daemon.3870worker1 # [ 32.031988] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3871worker1 # [ 32.060504] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)3872worker1 # [ 32.118991] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 2/5)3873worker1 # [ 32.264090] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 295 ms (attempt 3/5)3874worker1 # [ 32.566593] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 634 ms (attempt 4/5)3875worker1 # [ 33.205593] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3876worker1 # [ 33.217215] nix-grpc-daemon[775]: ts=2026-09-20T14:31:06Z level=debug event=path_info_miss path=c59dy5901id87cijqm6p1wgx5fgpmqry-nixos-system-client-test error="error: path '/nix/store/c59dy5901id87cijqm6p1wgx5fgpmqry-nixos-system-client-test' is required, but there is no substituter that can build it"3877worker2 # [ 33.229496] nix-grpc-daemon[778]: ts=2026-09-20T14:31:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:486843878worker2 # [ 33.245140] nix-grpc-daemon[778]: ts=2026-09-20T14:31:06Z level=debug event=path_info_miss path=c59dy5901id87cijqm6p1wgx5fgpmqry-nixos-system-client-test error="error: path '/nix/store/c59dy5901id87cijqm6p1wgx5fgpmqry-nixos-system-client-test' is required, but there is no substituter that can build it"3879client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3880client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)3881client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3882client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)3883worker2 # [ 33.675969] nix-grpc-daemon[778]: ts=2026-09-20T14:31:07Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:486843884worker2 # [ 33.692575] nix-grpc-daemon[778]: ts=2026-09-20T14:31:07Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:486843885worker2 # [ 33.704844] nix-grpc-daemon[778]: ts=2026-09-20T14:31:07Z level=debug event=path_info_miss path=c59dy5901id87cijqm6p1wgx5fgpmqry-nixos-system-client-test error="error: path '/nix/store/c59dy5901id87cijqm6p1wgx5fgpmqry-nixos-system-client-test' is required, but there is no substituter that can build it"3886worker2 # [ 33.719032] nix-grpc-daemon[778]: ts=2026-09-20T14:31:07Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:486843887worker2 # [ 33.726918] nix-grpc-daemon[778]: ts=2026-09-20T14:31:07Z level=debug event=path_info_miss path=c59dy5901id87cijqm6p1wgx5fgpmqry-nixos-system-client-test error="error: path '/nix/store/c59dy5901id87cijqm6p1wgx5fgpmqry-nixos-system-client-test' is required, but there is no substituter that can build it"3888client: must succeed: rm -r /var/lib/nix-grpc-store3889client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)3890worker2 # [ 34.025647] nix-grpc-daemon[778]: ts=2026-09-20T14:31:07Z level=info event=denied method=QueryMissing cn=- role=none3891worker1 # [ 34.286124] nix-grpc-daemon[775]: ts=2026-09-20T14:31:07Z level=info event=denied method=QueryMissing cn=stranger role=none3892(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.61 seconds)3893subtest: DAG build is scheduled across workers and lands in the cache3894client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23895client # don't know how to build these paths:3896client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3897worker2 # [ 34.574238] nix-grpc-daemon[778]: ts=2026-09-20T14:31:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:486843898worker2 # [ 34.581736] nix-grpc-daemon[778]: ts=2026-09-20T14:31:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486843899worker2 # [ 34.634110] nix-grpc-daemon[778]: ts=2026-09-20T14:31:08Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03900worker2 # [ 34.641251] nix-grpc-daemon[778]: ts=2026-09-20T14:31:08Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=13901worker2 # [ 34.701091] nix-grpc-daemon[778]: ts=2026-09-20T14:31:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486843902worker2 # [ 34.724402] nix-grpc-daemon[778]: ts=2026-09-20T14:31:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486843903client # copying 1 paths to 'lb:50051'...3904worker2 # [ 34.744167] nix-grpc-daemon[778]: ts=2026-09-20T14:31:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486843905client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3906worker2 # [ 34.781883] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)3907worker2 # [ 34.820204] nix-grpc-daemon[778]: ts=2026-09-20T14:31:08Z level=info event=niks3_push_started pid=8703908worker2 # [ 35.162705] nix-grpc-daemon[870]: time=2026-09-20T14:31:08.736Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""3909worker2 # [ 35.380135] nix-daemon[858]: accepted connection from pid 883, user nix-grpc-daemon (trusted)3910worker2 # [ 35.399702] nix-daemon[858]: reaped child process 886, status = succeeded3911worker2 # [ 35.624738] nix-daemon[858]: accepted connection from pid 888, user nix-grpc-daemon (trusted)3912worker2 # [ 35.671675] nix-daemon[858]: reaped child process 891, status = succeeded3913lb # [ 35.714529] niks3-server[1010]: 2026/09/20 14:31:09 INFO Received uploads request method=POST path=/api/pending_closures3914worker2 # [ 35.706408] nix-grpc-daemon[870]: time=2026-09-20T14:31:09.282Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3915worker2 # [ 35.711081] nix-grpc-daemon[870]: time=2026-09-20T14:31:09.282Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3916lb # [ 35.784723] niks3-server[1010]: 2026/09/20 14:31:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3917lb # [ 35.789507] niks3-server[1010]: 2026/09/20 14:31:09 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3918lb # [ 35.816435] niks3-server[1010]: 2026/09/20 14:31:09 INFO Signed narinfos id=1 count=13919worker2 # [ 35.786788] nix-grpc-daemon[870]: time=2026-09-20T14:31:09.362Z level=INFO msg="Uploading 1 narinfos"3920lb # [ 35.835149] niks3-server[1010]: 2026/09/20 14:31:09 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3921lb # [ 35.844951] niks3-server[1010]: 2026/09/20 14:31:09 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3922lb # [ 35.852414] niks3-server[1010]: 2026/09/20 14:31:09 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3923lb # [ 35.858974] niks3-server[1010]: 2026/09/20 14:31:09 INFO Completed upload id=13924worker2 # [ 35.827389] nix-grpc-daemon[870]: time=2026-09-20T14:31:09.403Z level=INFO msg="Upload complete. (656ms)"3925worker2 # [ 35.843491] nix-grpc-daemon[778]: ts=2026-09-20T14:31:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 paths=1 nar_bytes_in=5043926worker2 # [ 35.860407] nix-grpc-daemon[778]: ts=2026-09-20T14:31:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486843927worker2 # [ 35.870586] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)3928worker2 # [ 35.876970] nix-daemon[858]: reaped child process 868, status = succeeded3929client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3930client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3931worker2 # [ 35.893682] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)3932client # farm-a> LOG-farm-a-t13933worker2 # [ 37.354504] nix-daemon[858]: reaped child process 896, status = succeeded3934worker2 # [ 37.570743] nix-daemon[858]: accepted connection from pid 911, user nix-grpc-daemon (trusted)3935worker2 # [ 37.590634] nix-daemon[858]: reaped child process 914, status = succeeded3936lb # [ 37.638015] niks3-server[1010]: 2026/09/20 14:31:11 INFO Received uploads request method=POST path=/api/pending_closures3937worker2 # [ 37.626738] nix-grpc-daemon[870]: time=2026-09-20T14:31:11.201Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3938worker2 # [ 37.628878] nix-grpc-daemon[870]: time=2026-09-20T14:31:11.201Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3939lb # [ 37.715203] niks3-server[1010]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3940lb # [ 37.732380] niks3-server[1010]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3941lb # [ 37.743599] niks3-server[1010]: 2026/09/20 14:31:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3942lb # [ 37.751067] niks3-server[1010]: 2026/09/20 14:31:11 INFO Signed narinfos id=2 count=13943worker2 # [ 37.718557] nix-grpc-daemon[870]: time=2026-09-20T14:31:11.292Z level=INFO msg="Uploading 1 narinfos"3944lb # [ 37.754250] niks3-server[1010]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3945lb # [ 37.772106] niks3-server[1010]: 2026/09/20 14:31:11 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3946lb # [ 37.781673] niks3-server[1010]: 2026/09/20 14:31:11 INFO Completed upload id=23947worker2 # [ 37.750082] nix-grpc-daemon[870]: time=2026-09-20T14:31:11.326Z level=INFO msg="Upload complete. (442ms)"3948lb # [ 37.786440] niks3-server[1010]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo3949worker2 # [ 37.788939] nix-grpc-daemon[778]: ts=2026-09-20T14:31:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=13950worker2 # [ 37.804240] nix-grpc-daemon[778]: ts=2026-09-20T14:31:11Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=03951worker2 # [ 37.807438] nix-grpc-daemon[778]: ts=2026-09-20T14:31:11Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=13952worker2 # [ 37.811174] nix-daemon[858]: reaped child process 894, status = succeeded3953worker2 # [ 37.830869] nix-grpc-daemon[778]: ts=2026-09-20T14:31:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486843954worker2 # [ 37.853227] nix-grpc-daemon[778]: ts=2026-09-20T14:31:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486843955worker2 # [ 37.876527] nix-grpc-daemon[870]: time=2026-09-20T14:31:11.452Z level=INFO msg="All 1 paths already cached"3956client # copying 1 paths to 'lb:50051'...3957worker2 # [ 37.889271] nix-grpc-daemon[778]: ts=2026-09-20T14:31:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486843958client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...3959worker2 # [ 37.913581] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)3960worker2 # [ 38.193994] nix-daemon[858]: accepted connection from pid 920, user nix-grpc-daemon (trusted)3961worker2 # [ 38.217205] nix-daemon[858]: reaped child process 923, status = succeeded3962worker2 # [ 38.435681] nix-daemon[858]: accepted connection from pid 925, user nix-grpc-daemon (trusted)3963worker2 # [ 38.457012] nix-daemon[858]: reaped child process 928, status = succeeded3964lb # [ 38.502423] niks3-server[1010]: 2026/09/20 14:31:12 INFO Received uploads request method=POST path=/api/pending_closures3965worker2 # [ 38.492459] nix-grpc-daemon[870]: time=2026-09-20T14:31:12.067Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3966worker2 # [ 38.493987] nix-grpc-daemon[870]: time=2026-09-20T14:31:12.068Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"3967lb # [ 38.575758] niks3-server[1010]: 2026/09/20 14:31:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign3968worker2 # [ 38.553993] nix-grpc-daemon[870]: time=2026-09-20T14:31:12.129Z level=INFO msg="Uploading 1 narinfos"3969lb # [ 38.590383] niks3-server[1010]: 2026/09/20 14:31:12 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst3970lb # [ 38.599307] niks3-server[1010]: 2026/09/20 14:31:12 INFO Signed narinfos id=3 count=13971lb # [ 38.603524] niks3-server[1010]: 2026/09/20 14:31:12 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls3972lb # [ 38.613125] niks3-server[1010]: 2026/09/20 14:31:12 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete3973lb # [ 38.620174] niks3-server[1010]: 2026/09/20 14:31:12 INFO Completed upload id=33974worker2 # [ 38.588305] nix-grpc-daemon[870]: time=2026-09-20T14:31:12.164Z level=INFO msg="Upload complete. (642ms)"3975lb # [ 38.622629] niks3-server[1010]: 2026/09/20 14:31:12 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo3976worker2 # [ 38.600575] nix-grpc-daemon[778]: ts=2026-09-20T14:31:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=0 paths=1 nar_bytes_in=6243977worker2 # [ 38.615591] nix-grpc-daemon[778]: ts=2026-09-20T14:31:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486843978worker2 # [ 38.632431] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)3979worker2 # [ 38.640730] nix-daemon[858]: reaped child process 918, status = succeeded3980client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...3981client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3982worker2 # [ 38.658723] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)3983client # farm-b> LOG-farm-b-t13984worker2 # [ 39.704368] nix-daemon[858]: reaped child process 932, status = succeeded3985worker2 # [ 39.870246] nix-daemon[858]: accepted connection from pid 948, user nix-grpc-daemon (trusted)3986worker2 # [ 39.888882] nix-daemon[858]: reaped child process 951, status = succeeded3987lb # [ 39.938931] niks3-server[1010]: 2026/09/20 14:31:13 INFO Received uploads request method=POST path=/api/pending_closures3988worker2 # [ 39.914965] nix-grpc-daemon[870]: time=2026-09-20T14:31:13.490Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3989worker2 # [ 39.916549] nix-grpc-daemon[870]: time=2026-09-20T14:31:13.490Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"3990lb # [ 39.975635] niks3-server[1010]: 2026/09/20 14:31:13 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3991worker2 # [ 39.957599] nix-grpc-daemon[870]: time=2026-09-20T14:31:13.533Z level=INFO msg="Uploading 1 narinfos"3992lb # [ 39.991761] niks3-server[1010]: 2026/09/20 14:31:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign3993lb # [ 40.000953] niks3-server[1010]: 2026/09/20 14:31:13 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst3994lb # [ 40.009590] niks3-server[1010]: 2026/09/20 14:31:13 INFO Signed narinfos id=4 count=13995lb # [ 40.020362] niks3-server[1010]: 2026/09/20 14:31:13 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls3996lb # [ 40.026946] niks3-server[1010]: 2026/09/20 14:31:13 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete3997lb # [ 40.030899] niks3-server[1010]: 2026/09/20 14:31:13 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo3998lb # [ 40.034974] niks3-server[1010]: 2026/09/20 14:31:13 INFO Completed upload id=43999worker2 # [ 40.002985] nix-grpc-daemon[870]: time=2026-09-20T14:31:13.579Z level=INFO msg="Upload complete. (328ms)"4000worker2 # [ 40.025112] nix-grpc-daemon[778]: ts=2026-09-20T14:31:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14001worker2 # [ 40.040075] nix-grpc-daemon[778]: ts=2026-09-20T14:31:13Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04002worker2 # [ 40.044467] nix-grpc-daemon[778]: ts=2026-09-20T14:31:13Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=14003worker2 # [ 40.048977] nix-daemon[858]: reaped child process 930, status = succeeded4004worker2 # [ 40.066022] nix-grpc-daemon[778]: ts=2026-09-20T14:31:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844005worker2 # [ 40.080647] nix-grpc-daemon[778]: ts=2026-09-20T14:31:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486844006worker2 # [ 40.095790] nix-grpc-daemon[870]: time=2026-09-20T14:31:13.671Z level=INFO msg="All 2 paths already cached"4007client # copying 1 paths to 'lb:50051'...4008worker2 # [ 40.106203] nix-grpc-daemon[778]: ts=2026-09-20T14:31:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486844009client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4010worker2 # [ 40.128410] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4011lb # [ 40.279250] envoy[901]: conn peer=[2001:db8:1::1]:43238 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=22404012lb # [ 40.284154] envoy[901]: conn peer=[2001:db8:1::1]:43246 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=654013lb # [ 40.289534] envoy[901]: conn peer=[2001:db8:1::1]:43248 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114014lb # [ 40.293644] envoy[901]: conn peer=[2001:db8:1::1]:43252 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134015worker2 # [ 40.336449] nix-daemon[858]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4016worker2 # [ 40.352755] nix-daemon[858]: reaped child process 960, status = succeeded4017worker2 # [ 40.526215] nix-daemon[858]: accepted connection from pid 962, user nix-grpc-daemon (trusted)4018worker2 # [ 40.542397] nix-daemon[858]: reaped child process 965, status = succeeded4019lb # [ 40.586049] niks3-server[1010]: 2026/09/20 14:31:14 INFO Received uploads request method=POST path=/api/pending_closures4020worker2 # [ 40.569088] nix-grpc-daemon[870]: time=2026-09-20T14:31:14.144Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4021worker2 # [ 40.570586] nix-grpc-daemon[870]: time=2026-09-20T14:31:14.145Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4022worker2 # [ 40.594725] nix-grpc-daemon[870]: time=2026-09-20T14:31:14.170Z level=INFO msg="Uploading 1 narinfos"4023lb # [ 40.621698] niks3-server[1010]: 2026/09/20 14:31:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4024lb # [ 40.631490] niks3-server[1010]: 2026/09/20 14:31:14 INFO Signed narinfos id=5 count=14025lb # [ 40.637088] niks3-server[1010]: 2026/09/20 14:31:14 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4026lb # [ 40.644704] niks3-server[1010]: 2026/09/20 14:31:14 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4027lb # [ 40.656959] niks3-server[1010]: 2026/09/20 14:31:14 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4028worker2 # [ 40.630686] nix-grpc-daemon[870]: time=2026-09-20T14:31:14.206Z level=INFO msg="Upload complete. (484ms)"4029lb # [ 40.663758] niks3-server[1010]: 2026/09/20 14:31:14 INFO Completed upload id=54030lb # [ 40.669000] niks3-server[1010]: 2026/09/20 14:31:14 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4031worker2 # [ 40.640288] nix-grpc-daemon[778]: ts=2026-09-20T14:31:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=0 paths=1 nar_bytes_in=7524032worker2 # [ 40.651853] nix-grpc-daemon[778]: ts=2026-09-20T14:31:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844033worker2 # [ 40.660389] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4034worker2 # [ 40.664469] nix-daemon[858]: reaped child process 955, status = succeeded4035client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4036worker2 # [ 40.675811] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4037client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4038client # farm-top> LOG-farm-top-t14039worker2 # [ 41.903785] nix-daemon[858]: reaped child process 969, status = succeeded4040worker2 # [ 42.067061] nix-daemon[858]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4041worker2 # [ 42.082971] nix-daemon[858]: reaped child process 987, status = succeeded4042lb # [ 42.130037] niks3-server[1010]: 2026/09/20 14:31:15 INFO Received uploads request method=POST path=/api/pending_closures4043worker2 # [ 42.108261] nix-grpc-daemon[870]: time=2026-09-20T14:31:15.682Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4044worker2 # [ 42.109896] nix-grpc-daemon[870]: time=2026-09-20T14:31:15.682Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4045lb # [ 42.172352] niks3-server[1010]: 2026/09/20 14:31:15 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4046worker2 # [ 42.149254] nix-grpc-daemon[870]: time=2026-09-20T14:31:15.725Z level=INFO msg="Uploading 1 narinfos"4047lb # [ 42.184814] niks3-server[1010]: 2026/09/20 14:31:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4048lb # [ 42.193144] niks3-server[1010]: 2026/09/20 14:31:15 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4049lb # [ 42.197801] niks3-server[1010]: 2026/09/20 14:31:15 INFO Signed narinfos id=6 count=14050lb # [ 42.206050] niks3-server[1010]: 2026/09/20 14:31:15 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4051lb # [ 42.209644] niks3-server[1010]: 2026/09/20 14:31:15 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4052lb # [ 42.213428] niks3-server[1010]: 2026/09/20 14:31:15 INFO Completed upload id=64053worker2 # [ 42.180915] nix-grpc-daemon[870]: time=2026-09-20T14:31:15.757Z level=INFO msg="Upload complete. (292ms)"4054lb # [ 42.215865] niks3-server[1010]: 2026/09/20 14:31:15 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4055worker2 # [ 42.190200] nix-grpc-daemon[778]: ts=2026-09-20T14:31:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14056worker2 # [ 42.203688] nix-daemon[858]: reaped child process 967, status = succeeded4057client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.95 seconds)4058client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4059client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.26 seconds)4060client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4061client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4062client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4063client # copying 3 paths...4064client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4065client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4066client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4067client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4068(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.66 seconds)4069subtest: repeat is answered Cached by the scheduler without building4070worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4071worker1 # finding garbage collector roots...4072worker1 # deleting specified paths...4073worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4074worker1 # deleting unused links...4075worker1 # note: hard linking is currently saving 0.0 KiB4076worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4077worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4078worker2 # finding garbage collector roots...4079worker2 # deleting specified paths...4080worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4081worker2 # deleting unused links...4082worker2 # note: hard linking is currently saving 0.0 KiB4083worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.27 seconds)4084client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24085worker2 # [ 43.806297] nix-grpc-daemon[778]: ts=2026-09-20T14:31:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:486844086client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4087client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4088worker2 # [ 43.827183] nix-grpc-daemon[778]: ts=2026-09-20T14:31:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486844089worker2 # [ 43.838248] nix-grpc-daemon[870]: time=2026-09-20T14:31:17.413Z level=INFO msg="All 2 paths already cached"4090worker2 # [ 43.855916] nix-grpc-daemon[778]: ts=2026-09-20T14:31:17Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14091client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.35 seconds)4092client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4093client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4094lb: must succeed: curl -sf localhost:9901/clusters4095lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4096lb: must succeed: curl -sf localhost:9901/clusters4097lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4098worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4099worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4100(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.45 seconds)4101subtest: inputs already in the cache are not sent to the scheduler4102lb: must succeed: curl -sf localhost:9901/clusters4103lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4104lb: must succeed: curl -sf localhost:9901/clusters4105lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4106worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4107worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4108client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24109worker2 # [ 44.870817] nix-grpc-daemon[778]: ts=2026-09-20T14:31:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:486844110client # don't know how to build these paths:4111client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4112worker1 # [ 44.886318] nix-grpc-daemon[775]: ts=2026-09-20T14:31:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339664113worker2 # [ 44.914115] nix-grpc-daemon[778]: ts=2026-09-20T14:31:18Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04114worker2 # [ 44.922011] nix-grpc-daemon[778]: ts=2026-09-20T14:31:18Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14115worker2 # [ 44.962891] nix-grpc-daemon[778]: ts=2026-09-20T14:31:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844116worker2 # [ 44.978102] nix-grpc-daemon[778]: ts=2026-09-20T14:31:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486844117client # copying 1 paths to 'lb:50051'...4118worker2 # [ 44.994240] nix-grpc-daemon[870]: time=2026-09-20T14:31:18.568Z level=INFO msg="All 2 paths already cached"4119client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4120worker2 # [ 44.999591] nix-grpc-daemon[778]: ts=2026-09-20T14:31:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486844121worker2 # [ 45.015230] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4122worker2 # [ 45.225981] nix-daemon[858]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)4123worker2 # [ 45.246972] nix-daemon[858]: reaped child process 1020, status = succeeded4124worker2 # [ 45.424577] nix-daemon[858]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)4125worker2 # [ 45.443438] nix-daemon[858]: reaped child process 1025, status = succeeded4126lb # [ 45.489959] niks3-server[1010]: 2026/09/20 14:31:19 INFO Received uploads request method=POST path=/api/pending_closures4127worker2 # [ 45.465551] nix-grpc-daemon[870]: time=2026-09-20T14:31:19.041Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4128worker2 # [ 45.467017] nix-grpc-daemon[870]: time=2026-09-20T14:31:19.041Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4129lb # [ 45.517551] niks3-server[1010]: 2026/09/20 14:31:19 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4130lb # [ 45.529292] niks3-server[1010]: 2026/09/20 14:31:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4131worker2 # [ 45.503767] nix-grpc-daemon[870]: time=2026-09-20T14:31:19.079Z level=INFO msg="Uploading 1 narinfos"4132lb # [ 45.540992] niks3-server[1010]: 2026/09/20 14:31:19 INFO Signed narinfos id=7 count=14133lb # [ 45.543759] niks3-server[1010]: 2026/09/20 14:31:19 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4134lb # [ 45.560241] niks3-server[1010]: 2026/09/20 14:31:19 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4135worker2 # [ 45.531795] nix-grpc-daemon[870]: time=2026-09-20T14:31:19.107Z level=INFO msg="Upload complete. (499ms)"4136lb # [ 45.564472] niks3-server[1010]: 2026/09/20 14:31:19 INFO Completed upload id=74137lb # [ 45.571966] niks3-server[1010]: 2026/09/20 14:31:19 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4138worker2 # [ 45.539172] nix-grpc-daemon[778]: ts=2026-09-20T14:31:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=0 paths=1 nar_bytes_in=7524139worker2 # [ 45.553363] nix-grpc-daemon[778]: ts=2026-09-20T14:31:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844140worker2 # [ 45.556919] nix-daemon[858]: reaped child process 1015, status = succeeded4141worker2 # [ 45.569133] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4142client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4143client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4144worker2 # [ 45.598760] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4145client # farm-top> LOG-farm-top-t1b4146worker2 # [ 46.847563] nix-daemon[858]: reaped child process 1029, status = succeeded4147worker2 # [ 47.026721] nix-daemon[858]: accepted connection from pid 1045, user nix-grpc-daemon (trusted)4148worker2 # [ 47.043668] nix-daemon[858]: reaped child process 1048, status = succeeded4149lb # [ 47.089950] niks3-server[1010]: 2026/09/20 14:31:20 INFO Received uploads request method=POST path=/api/pending_closures4150worker2 # [ 47.068332] nix-grpc-daemon[870]: time=2026-09-20T14:31:20.641Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4151worker2 # [ 47.069841] nix-grpc-daemon[870]: time=2026-09-20T14:31:20.641Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4152lb # [ 47.127831] niks3-server[1010]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4153worker2 # [ 47.109521] nix-grpc-daemon[870]: time=2026-09-20T14:31:20.685Z level=INFO msg="Uploading 1 narinfos"4154lb # [ 47.142627] niks3-server[1010]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4155lb # [ 47.151426] niks3-server[1010]: 2026/09/20 14:31:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4156lb # [ 47.161033] niks3-server[1010]: 2026/09/20 14:31:20 INFO Signed narinfos id=8 count=14157lb # [ 47.164216] niks3-server[1010]: 2026/09/20 14:31:20 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4158lb # [ 47.167321] niks3-server[1010]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4159lb # [ 47.171485] niks3-server[1010]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4160lb # [ 47.175272] niks3-server[1010]: 2026/09/20 14:31:20 INFO Completed upload id=84161worker2 # [ 47.143187] nix-grpc-daemon[870]: time=2026-09-20T14:31:20.719Z level=INFO msg="Upload complete. (332ms)"4162worker2 # [ 47.157200] nix-grpc-daemon[778]: ts=2026-09-20T14:31:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14163worker2 # [ 47.172288] nix-daemon[858]: reaped child process 1027, status = succeeded4164client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.60 seconds)4165client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4166client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds)4167lb: must succeed: curl -sf localhost:9901/clusters4168lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4169lb: must succeed: curl -sf localhost:9901/clusters4170lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4171worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4172worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4173(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.20 seconds)4174subtest: two clients wanting the same drv build it once4175client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4176client # [ 47.701066] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4177client # Running as unit: dedup1.service; invocation ID: 4690987ad9c24e48b8d37e70d41d7d414178client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4179client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4180client # [ 47.964001] nix[879]: don't know how to build these paths:4181client # [ 47.965140] nix[879]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4182worker2 # [ 47.962098] nix-grpc-daemon[778]: ts=2026-09-20T14:31:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:486844183worker2 # [ 47.976451] nix-grpc-daemon[778]: ts=2026-09-20T14:31:21Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04184worker1 # [ 47.984944] nix-grpc-daemon[775]: ts=2026-09-20T14:31:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:339664185worker2 # [ 47.982551] nix-grpc-daemon[778]: ts=2026-09-20T14:31:21Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14186worker1 # [ 48.002093] nix-grpc-daemon[775]: ts=2026-09-20T14:31:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339664187client # [ 48.014576] nix[879]: copying 1 paths to 'lb:50051'...4188client # [ 48.016286] nix[879]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4189worker1 # [ 48.016105] nix-grpc-daemon[775]: ts=2026-09-20T14:31:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:339664190worker1 # [ 48.032302] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4191worker1 # [ 48.063402] nix-grpc-daemon[775]: ts=2026-09-20T14:31:21Z level=info event=niks3_push_started pid=8624192worker1 # [ 48.331861] nix-grpc-daemon[862]: time=2026-09-20T14:31:21.900Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""4193worker1 # [ 48.520292] nix-daemon[835]: accepted connection from pid 875, user nix-grpc-daemon (trusted)4194worker1 # [ 48.541192] nix-daemon[835]: reaped child process 878, status = succeeded4195worker1 # [ 48.716496] nix-daemon[835]: accepted connection from pid 880, user nix-grpc-daemon (trusted)4196worker1 # [ 48.732274] nix-daemon[835]: reaped child process 883, status = succeeded4197lb # [ 48.778294] niks3-server[1010]: 2026/09/20 14:31:22 INFO Received uploads request method=POST path=/api/pending_closures4198worker1 # [ 48.760314] nix-grpc-daemon[862]: time=2026-09-20T14:31:22.331Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4199worker1 # [ 48.763550] nix-grpc-daemon[862]: time=2026-09-20T14:31:22.332Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4200lb # [ 48.819967] niks3-server[1010]: 2026/09/20 14:31:22 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4201worker1 # [ 48.805901] nix-grpc-daemon[862]: time=2026-09-20T14:31:22.376Z level=INFO msg="Uploading 1 narinfos"4202lb # [ 48.829535] niks3-server[1010]: 2026/09/20 14:31:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4203lb # [ 48.843560] niks3-server[1010]: 2026/09/20 14:31:22 INFO Signed narinfos id=9 count=14204lb # [ 48.850938] niks3-server[1010]: 2026/09/20 14:31:22 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4205client # [ 48.818121] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4206lb # [ 48.858605] niks3-server[1010]: 2026/09/20 14:31:22 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4207client # Running as unit: dedup2.service; invocation ID: 138869ea6001497da071d6d2acdd0bd94208lb # [ 48.867769] niks3-server[1010]: 2026/09/20 14:31:22 INFO Completed upload id=94209worker1 # [ 48.837447] nix-grpc-daemon[862]: time=2026-09-20T14:31:22.408Z level=INFO msg="Upload complete. (498ms)"4210lb # [ 48.870680] niks3-server[1010]: 2026/09/20 14:31:22 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4211client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4212worker1 # [ 48.844258] nix-grpc-daemon[775]: ts=2026-09-20T14:31:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=0 paths=1 nar_bytes_in=4324213worker1 # [ 48.858987] nix-grpc-daemon[775]: ts=2026-09-20T14:31:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:339664214worker1 # [ 48.869637] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4215worker1 # [ 48.875877] nix-daemon[835]: reaped child process 860, status = succeeded4216client # [ 48.889547] nix[879]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4217worker1 # [ 48.894815] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4218client # [ 49.171871] nix[895]: don't know how to build these paths:4219worker2 # [ 49.170852] nix-grpc-daemon[778]: ts=2026-09-20T14:31:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:486844220client # [ 49.175623] nix[895]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4221worker2 # [ 49.184285] nix-grpc-daemon[778]: ts=2026-09-20T14:31:22Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04222worker1 # [ 49.190269] nix-grpc-daemon[775]: ts=2026-09-20T14:31:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:339664223client # [ 49.196054] nix[895]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4224client: must succeed: sleep 24225lb # [ 50.274889] envoy[901]: conn peer=[2001:db8:1::1]:43262 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9562 tx=3820 ms=76504226lb # [ 50.279960] envoy[901]: conn peer=[2001:db8:1::1]:54886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=724227lb # [ 50.285214] envoy[901]: conn peer=[2001:db8:1::1]:54888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1706 ms=22954228client: (finished: must succeed: sleep 2, in 2.06 seconds)4229client: waiting for success: ! systemctl is-active dedup1 dedup24230worker1 # [ 52.547107] nix-daemon[835]: reaped child process 890, status = succeeded4231worker1 # [ 52.738120] nix-daemon[835]: accepted connection from pid 916, user nix-grpc-daemon (trusted)4232worker1 # [ 52.755522] nix-daemon[835]: reaped child process 919, status = succeeded4233lb # [ 52.799534] niks3-server[1010]: 2026/09/20 14:31:26 INFO Received uploads request method=POST path=/api/pending_closures4234worker1 # [ 52.779441] nix-grpc-daemon[862]: time=2026-09-20T14:31:26.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4235worker1 # [ 52.781741] nix-grpc-daemon[862]: time=2026-09-20T14:31:26.349Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4236lb # [ 52.841680] niks3-server[1010]: 2026/09/20 14:31:26 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4237lb # [ 52.855082] niks3-server[1010]: 2026/09/20 14:31:26 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4238lb # [ 52.868108] niks3-server[1010]: 2026/09/20 14:31:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4239worker1 # [ 52.844325] nix-grpc-daemon[862]: time=2026-09-20T14:31:26.413Z level=INFO msg="Uploading 1 narinfos"4240lb # [ 52.874842] niks3-server[1010]: 2026/09/20 14:31:26 INFO Signed narinfos id=10 count=14241lb # [ 52.880296] niks3-server[1010]: 2026/09/20 14:31:26 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4242lb # [ 52.891350] niks3-server[1010]: 2026/09/20 14:31:26 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4243lb # [ 52.898290] niks3-server[1010]: 2026/09/20 14:31:26 INFO Completed upload id=104244worker1 # [ 52.868058] nix-grpc-daemon[862]: time=2026-09-20T14:31:26.438Z level=INFO msg="Upload complete. (357ms)"4245lb # [ 52.901343] niks3-server[1010]: 2026/09/20 14:31:26 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4246worker1 # [ 52.881128] nix-grpc-daemon[775]: ts=2026-09-20T14:31:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14247worker1 # [ 52.896354] nix-grpc-daemon[775]: ts=2026-09-20T14:31:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14248worker1 # [ 52.903188] nix-daemon[835]: reaped child process 888, status = succeeded4249client # [ 52.945088] systemd[1]: dedup1.service: Deactivated successfully.4250client # [ 52.949761] systemd[1]: dedup1.service: Consumed 170ms CPU time over 5.247s wall clock time, 43.4M memory peak, 7K incoming IP traffic, 8.3K outgoing IP traffic.4251client # [ 52.958334] systemd[1]: dedup2.service: Deactivated successfully.4252client # [ 52.962936] systemd[1]: dedup2.service: Consumed 209ms CPU time over 4.143s wall clock time, 35.2M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4253client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4254client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4255client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4256worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4257worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4258worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4259worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4260worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4261worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4262worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4263worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4264(finished: subtest: two clients wanting the same drv build it once, in 6.29 seconds)4265subtest: build hook: nix-daemon with builders = grpc://lb4266client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4267client # [ 54.130577] systemd[1]: Started Nix Daemon.4268client # [ 54.318885] nix-daemon[937]: accepted connection from pid 934, user root (trusted)4269worker2 # [ 54.532597] nix-grpc-daemon[778]: ts=2026-09-20T14:31:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:486844270worker1 # [ 54.538249] nix-grpc-daemon[775]: ts=2026-09-20T14:31:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339664271worker2 # [ 54.556140] nix-grpc-daemon[778]: ts=2026-09-20T14:31:28Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04272worker2 # [ 54.562714] nix-grpc-daemon[778]: ts=2026-09-20T14:31:28Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14273worker2 # [ 54.608082] nix-grpc-daemon[778]: ts=2026-09-20T14:31:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844274worker2 # [ 54.623858] nix-grpc-daemon[778]: ts=2026-09-20T14:31:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486844275worker2 # [ 54.639034] nix-grpc-daemon[778]: ts=2026-09-20T14:31:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486844276worker2 # [ 54.653930] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4277worker2 # [ 54.858821] nix-daemon[858]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)4278worker2 # [ 54.879290] nix-daemon[858]: reaped child process 1087, status = succeeded4279worker2 # [ 55.051561] nix-daemon[858]: accepted connection from pid 1089, user nix-grpc-daemon (trusted)4280worker2 # [ 55.069259] nix-daemon[858]: reaped child process 1092, status = succeeded4281lb # [ 55.115354] niks3-server[1010]: 2026/09/20 14:31:28 INFO Received uploads request method=POST path=/api/pending_closures4282worker2 # [ 55.093009] nix-grpc-daemon[870]: time=2026-09-20T14:31:28.668Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4283worker2 # [ 55.094463] nix-grpc-daemon[870]: time=2026-09-20T14:31:28.668Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4284lb # [ 55.142336] niks3-server[1010]: 2026/09/20 14:31:28 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4285worker2 # [ 55.123061] nix-grpc-daemon[870]: time=2026-09-20T14:31:28.699Z level=INFO msg="Uploading 1 narinfos"4286lb # [ 55.156168] niks3-server[1010]: 2026/09/20 14:31:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4287lb # [ 55.167045] niks3-server[1010]: 2026/09/20 14:31:28 INFO Signed narinfos id=11 count=14288lb # [ 55.174081] niks3-server[1010]: 2026/09/20 14:31:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4289lb # [ 55.180265] niks3-server[1010]: 2026/09/20 14:31:28 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4290lb # [ 55.188109] niks3-server[1010]: 2026/09/20 14:31:28 INFO Completed upload id=114291worker2 # [ 55.156192] nix-grpc-daemon[870]: time=2026-09-20T14:31:28.732Z level=INFO msg="Upload complete. (485ms)"4292lb # [ 55.192660] niks3-server[1010]: 2026/09/20 14:31:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4293worker2 # [ 55.164160] nix-grpc-daemon[778]: ts=2026-09-20T14:31:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=0 paths=1 nar_bytes_in=5124294worker2 # [ 55.179394] nix-grpc-daemon[778]: ts=2026-09-20T14:31:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844295worker2 # [ 55.183598] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4296worker2 # [ 55.189178] nix-daemon[858]: reaped child process 1082, status = succeeded4297worker2 # [ 55.200883] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4298worker2 # [ 56.243401] nix-daemon[858]: reaped child process 1096, status = succeeded4299worker2 # [ 56.421467] nix-daemon[858]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)4300worker2 # [ 56.438369] nix-daemon[858]: reaped child process 1114, status = succeeded4301lb # [ 56.482686] niks3-server[1010]: 2026/09/20 14:31:30 INFO Received uploads request method=POST path=/api/pending_closures4302worker2 # [ 56.464188] nix-grpc-daemon[870]: time=2026-09-20T14:31:30.036Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4303worker2 # [ 56.465843] nix-grpc-daemon[870]: time=2026-09-20T14:31:30.036Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4304lb # [ 56.524864] niks3-server[1010]: 2026/09/20 14:31:30 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4305worker2 # [ 56.506572] nix-grpc-daemon[870]: time=2026-09-20T14:31:30.082Z level=INFO msg="Uploading 1 narinfos"4306lb # [ 56.535675] niks3-server[1010]: 2026/09/20 14:31:30 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4307lb # [ 56.549335] niks3-server[1010]: 2026/09/20 14:31:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4308lb # [ 56.554437] niks3-server[1010]: 2026/09/20 14:31:30 INFO Signed narinfos id=12 count=14309lb # [ 56.560816] niks3-server[1010]: 2026/09/20 14:31:30 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4310lb # [ 56.565024] niks3-server[1010]: 2026/09/20 14:31:30 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4311worker2 # [ 56.532956] nix-grpc-daemon[870]: time=2026-09-20T14:31:30.109Z level=INFO msg="Upload complete. (322ms)"4312lb # [ 56.568608] niks3-server[1010]: 2026/09/20 14:31:30 INFO Completed upload id=124313lb # [ 56.571524] niks3-server[1010]: 2026/09/20 14:31:30 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4314worker1 # [ 56.551690] nix-grpc-daemon[775]: ts=2026-09-20T14:31:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:339664315worker2 # [ 56.545041] nix-grpc-daemon[778]: ts=2026-09-20T14:31:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14316worker2 # [ 56.564327] nix-daemon[858]: reaped child process 1094, status = succeeded4317worker1 # [ 56.567999] nix-grpc-daemon[775]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4318worker1 # [ 56.616196] nix-grpc-daemon[775]: ts=2026-09-20T14:31:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:339664319worker1 # [ 56.623020] nix-grpc-daemon[775]: ts=2026-09-20T14:31:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=0 paths=1 nar_bytes_out=1284320worker1 # [ 56.938166] nix-grpc-daemon[775]: ts=2026-09-20T14:31:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:339664321worker1 # [ 56.945930] nix-grpc-daemon[775]: ts=2026-09-20T14:31:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339664322worker1 # [ 56.953525] nix-grpc-daemon[862]: time=2026-09-20T14:31:30.522Z level=INFO msg="All 1 paths already cached"4323worker2 # [ 56.963626] nix-grpc-daemon[778]: ts=2026-09-20T14:31:30Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04324worker2 # [ 56.970841] nix-grpc-daemon[778]: ts=2026-09-20T14:31:30Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14325worker2 # [ 56.979793] nix-grpc-daemon[778]: ts=2026-09-20T14:31:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844326worker2 # [ 56.990823] nix-grpc-daemon[778]: ts=2026-09-20T14:31:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486844327worker2 # [ 57.004477] nix-grpc-daemon[870]: time=2026-09-20T14:31:30.580Z level=INFO msg="All 1 paths already cached"4328worker2 # [ 57.013225] nix-grpc-daemon[778]: ts=2026-09-20T14:31:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486844329worker2 # [ 57.026803] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4330worker2 # [ 57.226539] nix-daemon[858]: accepted connection from pid 1120, user nix-grpc-daemon (trusted)4331worker2 # [ 57.245682] nix-daemon[858]: reaped child process 1123, status = succeeded4332worker2 # [ 57.413527] nix-daemon[858]: accepted connection from pid 1125, user nix-grpc-daemon (trusted)4333worker2 # [ 57.429342] nix-daemon[858]: reaped child process 1128, status = succeeded4334lb # [ 57.472218] niks3-server[1010]: 2026/09/20 14:31:31 INFO Received uploads request method=POST path=/api/pending_closures4335worker2 # [ 57.449204] nix-grpc-daemon[870]: time=2026-09-20T14:31:31.025Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4336worker2 # [ 57.450673] nix-grpc-daemon[870]: time=2026-09-20T14:31:31.025Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4337lb # [ 57.501094] niks3-server[1010]: 2026/09/20 14:31:31 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4338worker2 # [ 57.482727] nix-grpc-daemon[870]: time=2026-09-20T14:31:31.058Z level=INFO msg="Uploading 1 narinfos"4339lb # [ 57.518499] niks3-server[1010]: 2026/09/20 14:31:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4340lb # [ 57.527857] niks3-server[1010]: 2026/09/20 14:31:31 INFO Signed narinfos id=13 count=14341lb # [ 57.531075] niks3-server[1010]: 2026/09/20 14:31:31 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4342lb # [ 57.539308] niks3-server[1010]: 2026/09/20 14:31:31 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4343lb # [ 57.544446] niks3-server[1010]: 2026/09/20 14:31:31 INFO Completed upload id=134344worker2 # [ 57.512545] nix-grpc-daemon[870]: time=2026-09-20T14:31:31.088Z level=INFO msg="Upload complete. (470ms)"4345lb # [ 57.547321] niks3-server[1010]: 2026/09/20 14:31:31 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4346worker2 # [ 57.520138] nix-grpc-daemon[778]: ts=2026-09-20T14:31:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=0 paths=1 nar_bytes_in=6244347worker2 # [ 57.532304] nix-grpc-daemon[778]: ts=2026-09-20T14:31:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844348worker2 # [ 57.540350] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4349worker2 # [ 57.545149] nix-daemon[858]: reaped child process 1118, status = succeeded4350worker2 # [ 57.556200] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4351worker2 # [ 58.982377] nix-daemon[858]: reaped child process 1132, status = succeeded4352worker2 # [ 59.141507] nix-daemon[858]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)4353worker2 # [ 59.160205] nix-daemon[858]: reaped child process 1150, status = succeeded4354lb # [ 59.205473] niks3-server[1010]: 2026/09/20 14:31:32 INFO Received uploads request method=POST path=/api/pending_closures4355worker2 # [ 59.184071] nix-grpc-daemon[870]: time=2026-09-20T14:31:32.759Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4356worker2 # [ 59.185509] nix-grpc-daemon[870]: time=2026-09-20T14:31:32.759Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4357lb # [ 59.246330] niks3-server[1010]: 2026/09/20 14:31:32 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4358worker2 # [ 59.233833] nix-grpc-daemon[870]: time=2026-09-20T14:31:32.809Z level=INFO msg="Uploading 1 narinfos"4359lb # [ 59.258326] niks3-server[1010]: 2026/09/20 14:31:32 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4360lb # [ 59.271706] niks3-server[1010]: 2026/09/20 14:31:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4361lb # [ 59.277850] niks3-server[1010]: 2026/09/20 14:31:32 INFO Signed narinfos id=14 count=14362lb # [ 59.279883] niks3-server[1010]: 2026/09/20 14:31:32 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4363lb # [ 59.286488] niks3-server[1010]: 2026/09/20 14:31:32 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4364lb # [ 59.293368] niks3-server[1010]: 2026/09/20 14:31:32 INFO Completed upload id=144365lb # [ 59.296047] niks3-server[1010]: 2026/09/20 14:31:32 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4366worker2 # [ 59.261420] nix-grpc-daemon[870]: time=2026-09-20T14:31:32.837Z level=INFO msg="Upload complete. (292ms)"4367worker2 # [ 59.273058] nix-grpc-daemon[778]: ts=2026-09-20T14:31:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14368worker2 # [ 59.285638] nix-daemon[858]: reaped child process 1130, status = succeeded4369worker2 # [ 59.315824] nix-grpc-daemon[778]: ts=2026-09-20T14:31:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:486844370worker1 # [ 59.342782] nix-grpc-daemon[775]: ts=2026-09-20T14:31:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:339664371worker1 # [ 59.355798] nix-grpc-daemon[775]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4372worker1 # [ 59.373038] nix-grpc-daemon[775]: ts=2026-09-20T14:31:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=0 paths=1 nar_bytes_out=1764373worker2 # [ 59.691754] nix-grpc-daemon[778]: ts=2026-09-20T14:31:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:486844374worker1 # [ 59.698345] nix-grpc-daemon[775]: ts=2026-09-20T14:31:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339664375worker1 # [ 59.708898] nix-grpc-daemon[862]: time=2026-09-20T14:31:33.278Z level=INFO msg="All 2 paths already cached"4376worker2 # [ 59.724849] nix-grpc-daemon[778]: ts=2026-09-20T14:31:33Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04377worker2 # [ 59.731937] nix-grpc-daemon[778]: ts=2026-09-20T14:31:33Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14378worker2 # [ 59.740793] nix-grpc-daemon[778]: ts=2026-09-20T14:31:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844379worker2 # [ 59.751670] nix-grpc-daemon[778]: ts=2026-09-20T14:31:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486844380worker2 # [ 59.765380] nix-grpc-daemon[870]: time=2026-09-20T14:31:33.341Z level=INFO msg="All 2 paths already cached"4381worker2 # [ 59.775397] nix-grpc-daemon[778]: ts=2026-09-20T14:31:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486844382worker2 # [ 59.789274] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4383worker2 # [ 59.987386] nix-daemon[858]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)4384worker2 # [ 60.007852] nix-daemon[858]: reaped child process 1159, status = succeeded4385worker2 # [ 60.183706] nix-daemon[858]: accepted connection from pid 1161, user nix-grpc-daemon (trusted)4386worker2 # [ 60.199862] nix-daemon[858]: reaped child process 1164, status = succeeded4387lb # [ 60.247286] niks3-server[1010]: 2026/09/20 14:31:33 INFO Received uploads request method=POST path=/api/pending_closures4388worker2 # [ 60.225089] nix-grpc-daemon[870]: time=2026-09-20T14:31:33.800Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4389worker2 # [ 60.226517] nix-grpc-daemon[870]: time=2026-09-20T14:31:33.800Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4390lb # [ 60.273389] niks3-server[1010]: 2026/09/20 14:31:33 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4391lb # [ 60.285221] envoy[901]: conn peer=[2001:db8:1::1]:54890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1372 ms=49334392worker2 # [ 60.262920] nix-grpc-daemon[870]: time=2026-09-20T14:31:33.838Z level=INFO msg="Uploading 1 narinfos"4393lb # [ 60.300080] envoy[901]: conn peer=[2001:db8:1::1]:54902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1085 ms=37384394lb # [ 60.311661] envoy[901]: conn peer=[2001:db8:1::1]:58440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=333 ms=704395lb # [ 60.317494] envoy[901]: conn peer=[2001:db8:1::1]:58432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3406 tx=1777 ms=21494396worker2 # [ 60.294563] nix-grpc-daemon[870]: time=2026-09-20T14:31:33.870Z level=INFO msg="Upload complete. (489ms)"4397lb # [ 60.327202] envoy[901]: conn peer=[2001:db8:1::1]:58452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3862 tx=1852 ms=24924398lb # [ 60.332371] envoy[901]: conn peer=[2001:db8:1::1]:58458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=934399lb # [ 60.337341] niks3-server[1010]: 2026/09/20 14:31:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4400worker2 # [ 60.305801] nix-grpc-daemon[778]: ts=2026-09-20T14:31:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=0 paths=1 nar_bytes_in=7524401lb # [ 60.341182] niks3-server[1010]: 2026/09/20 14:31:33 INFO Signed narinfos id=15 count=14402lb # [ 60.343653] niks3-server[1010]: 2026/09/20 14:31:33 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4403lb # [ 60.347385] niks3-server[1010]: 2026/09/20 14:31:33 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4404lb # [ 60.351177] niks3-server[1010]: 2026/09/20 14:31:33 INFO Completed upload id=154405worker2 # [ 60.317944] nix-grpc-daemon[778]: ts=2026-09-20T14:31:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844406lb # [ 60.353402] niks3-server[1010]: 2026/09/20 14:31:33 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4407worker2 # [ 60.325857] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4408worker2 # [ 60.328384] nix-daemon[858]: reaped child process 1154, status = succeeded4409worker2 # [ 60.341707] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4410worker2 # [ 61.166363] nix-daemon[858]: reaped child process 1168, status = succeeded4411worker2 # [ 61.321905] nix-daemon[858]: accepted connection from pid 1183, user nix-grpc-daemon (trusted)4412worker2 # [ 61.338661] nix-daemon[858]: reaped child process 1186, status = succeeded4413lb # [ 61.387008] niks3-server[1010]: 2026/09/20 14:31:34 INFO Received uploads request method=POST path=/api/pending_closures4414worker2 # [ 61.364380] nix-grpc-daemon[870]: time=2026-09-20T14:31:34.939Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4415worker2 # [ 61.365839] nix-grpc-daemon[870]: time=2026-09-20T14:31:34.939Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4416lb # [ 61.423299] niks3-server[1010]: 2026/09/20 14:31:35 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4417lb # [ 61.437256] niks3-server[1010]: 2026/09/20 14:31:35 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4418lb # [ 61.451673] niks3-server[1010]: 2026/09/20 14:31:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4419worker2 # [ 61.425252] nix-grpc-daemon[870]: time=2026-09-20T14:31:35.001Z level=INFO msg="Uploading 1 narinfos"4420lb # [ 61.463368] niks3-server[1010]: 2026/09/20 14:31:35 INFO Signed narinfos id=16 count=14421lb # [ 61.470371] niks3-server[1010]: 2026/09/20 14:31:35 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4422worker2 # [ 61.448571] nix-grpc-daemon[870]: time=2026-09-20T14:31:35.024Z level=INFO msg="Upload complete. (297ms)"4423lb # [ 61.479362] niks3-server[1010]: 2026/09/20 14:31:35 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4424lb # [ 61.485279] niks3-server[1010]: 2026/09/20 14:31:35 INFO Completed upload id=164425lb # [ 61.489037] niks3-server[1010]: 2026/09/20 14:31:35 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4426worker2 # [ 61.461151] nix-grpc-daemon[778]: ts=2026-09-20T14:31:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14427worker2 # [ 61.478546] nix-daemon[858]: reaped child process 1166, status = succeeded4428worker1 # [ 61.481190] nix-grpc-daemon[775]: ts=2026-09-20T14:31:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:339664429worker1 # [ 61.494464] nix-grpc-daemon[775]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4430worker2 # [ 61.532100] nix-grpc-daemon[778]: ts=2026-09-20T14:31:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:486844431worker2 # [ 61.540082] nix-grpc-daemon[778]: ts=2026-09-20T14:31:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=0 paths=1 nar_bytes_out=2324432client # [ 61.633521] nix-daemon[937]: reaped child process 939, status = succeeded4433client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.74 seconds)4434client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4435client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4436client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4437client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4438client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4439client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4440(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.87 seconds)4441subtest: an input only one worker has still reaches the builder4442client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4443client # this derivation will be built:4444client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4445client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4446client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.44 seconds)4447client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4448client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4449worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4450worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.26 seconds)4451worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4452worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4453client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24454client # [ 62.936138] nix-daemon[937]: accepted connection from pid 1020, user root (trusted)4455client # this derivation will be built:4456client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4457client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4458worker2 # [ 63.147006] nix-grpc-daemon[778]: ts=2026-09-20T14:31:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:486844459worker1 # [ 63.153980] nix-grpc-daemon[775]: ts=2026-09-20T14:31:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339664460worker1 # [ 63.368221] nix-daemon[835]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4461worker1 # [ 63.385406] nix-daemon[835]: reaped child process 941, status = succeeded4462lb # [ 63.431711] niks3-server[1010]: 2026/09/20 14:31:37 INFO Received uploads request method=POST path=/api/pending_closures4463worker1 # [ 63.410870] nix-grpc-daemon[862]: time=2026-09-20T14:31:36.981Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4464worker1 # [ 63.412805] nix-grpc-daemon[862]: time=2026-09-20T14:31:36.981Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4465lb # [ 63.467130] niks3-server[1010]: 2026/09/20 14:31:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4466worker1 # [ 63.445529] nix-grpc-daemon[862]: time=2026-09-20T14:31:37.016Z level=INFO msg="Uploading 1 narinfos"4467lb # [ 63.477122] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4468lb # [ 63.491211] niks3-server[1010]: 2026/09/20 14:31:37 INFO Signed narinfos id=17 count=14469lb # [ 63.496851] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4470worker1 # [ 63.475317] nix-grpc-daemon[862]: time=2026-09-20T14:31:37.046Z level=INFO msg="Upload complete. (315ms)"4471lb # [ 63.504617] niks3-server[1010]: 2026/09/20 14:31:37 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4472lb # [ 63.510428] niks3-server[1010]: 2026/09/20 14:31:37 INFO Completed upload id=174473lb # [ 63.513134] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4474worker2 # [ 63.495444] nix-grpc-daemon[778]: ts=2026-09-20T14:31:37Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04475worker2 # [ 63.502080] nix-grpc-daemon[778]: ts=2026-09-20T14:31:37Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14476worker1 # [ 63.507469] nix-grpc-daemon[775]: ts=2026-09-20T14:31:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:339664477worker1 # [ 63.523903] nix-grpc-daemon[775]: ts=2026-09-20T14:31:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339664478client # copying 2 paths to 'lb:50051'...4479worker1 # [ 63.542566] nix-grpc-daemon[775]: ts=2026-09-20T14:31:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:339664480client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4481client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4482worker1 # [ 63.567492] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4483worker1 # [ 63.786909] nix-daemon[835]: accepted connection from pid 945, user nix-grpc-daemon (trusted)4484worker1 # [ 63.804843] nix-daemon[835]: reaped child process 948, status = succeeded4485worker1 # [ 63.979387] nix-daemon[835]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4486worker1 # [ 63.995606] nix-daemon[835]: reaped child process 953, status = succeeded4487lb # [ 64.037628] niks3-server[1010]: 2026/09/20 14:31:37 INFO Received uploads request method=POST path=/api/pending_closures4488lb # [ 64.046816] niks3-server[1010]: 2026/09/20 14:31:37 INFO Received uploads request method=POST path=/api/pending_closures4489worker1 # [ 64.025077] nix-grpc-daemon[862]: time=2026-09-20T14:31:37.595Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4490worker1 # [ 64.026585] nix-grpc-daemon[862]: time=2026-09-20T14:31:37.595Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4491worker1 # [ 64.028355] nix-grpc-daemon[862]: time=2026-09-20T14:31:37.595Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4492lb # [ 64.078828] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4493lb # [ 64.092591] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4494lb # [ 64.104970] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4495lb # [ 64.113918] niks3-server[1010]: 2026/09/20 14:31:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4496worker1 # [ 64.089838] nix-grpc-daemon[862]: time=2026-09-20T14:31:37.659Z level=INFO msg="Uploading 3 narinfos"4497lb # [ 64.121384] niks3-server[1010]: 2026/09/20 14:31:37 INFO Signed narinfos id=18 count=14498lb # [ 64.123640] niks3-server[1010]: 2026/09/20 14:31:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4499lb # [ 64.134433] niks3-server[1010]: 2026/09/20 14:31:37 INFO Signed narinfos id=19 count=24500lb # [ 64.140251] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4501lb # [ 64.151884] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4502lb # [ 64.161208] niks3-server[1010]: 2026/09/20 14:31:37 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4503lb # [ 64.166614] niks3-server[1010]: 2026/09/20 14:31:37 INFO Completed upload id=184504lb # [ 64.168740] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4505lb # [ 64.172589] niks3-server[1010]: 2026/09/20 14:31:37 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4506worker1 # [ 64.142234] nix-grpc-daemon[862]: time=2026-09-20T14:31:37.713Z level=INFO msg="Upload complete. (554ms)"4507lb # [ 64.176056] niks3-server[1010]: 2026/09/20 14:31:37 INFO Completed upload id=194508lb # [ 64.177974] niks3-server[1010]: 2026/09/20 14:31:37 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4509worker1 # [ 64.152510] nix-grpc-daemon[775]: ts=2026-09-20T14:31:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=0 paths=2 nar_bytes_in=9844510worker1 # [ 64.173385] nix-grpc-daemon[775]: ts=2026-09-20T14:31:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:339664511worker1 # [ 64.184431] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4512worker1 # [ 64.188710] nix-daemon[835]: reaped child process 943, status = succeeded4513client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4514worker1 # [ 64.201035] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4515client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4516worker1 # [ 65.626808] nix-daemon[835]: reaped child process 957, status = succeeded4517worker1 # [ 65.792507] nix-daemon[835]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4518worker1 # [ 65.809410] nix-daemon[835]: reaped child process 975, status = succeeded4519lb # [ 65.853226] niks3-server[1010]: 2026/09/20 14:31:39 INFO Received uploads request method=POST path=/api/pending_closures4520worker1 # [ 65.832578] nix-grpc-daemon[862]: time=2026-09-20T14:31:39.403Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4521lb # [ 65.885903] niks3-server[1010]: 2026/09/20 14:31:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4522worker1 # [ 65.868210] nix-grpc-daemon[862]: time=2026-09-20T14:31:39.438Z level=INFO msg="Uploading 1 narinfos"4523lb # [ 65.900776] niks3-server[1010]: 2026/09/20 14:31:39 INFO Signed narinfos id=20 count=14524lb # [ 65.903951] niks3-server[1010]: 2026/09/20 14:31:39 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4525lb # [ 65.917453] niks3-server[1010]: 2026/09/20 14:31:39 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4526lb # [ 65.926030] niks3-server[1010]: 2026/09/20 14:31:39 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4527worker1 # [ 65.903478] nix-grpc-daemon[862]: time=2026-09-20T14:31:39.474Z level=INFO msg="Upload complete. (287ms)"4528lb # [ 65.933764] niks3-server[1010]: 2026/09/20 14:31:39 INFO Completed upload id=204529lb # [ 65.937395] niks3-server[1010]: 2026/09/20 14:31:39 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4530worker1 # [ 65.912871] nix-grpc-daemon[775]: ts=2026-09-20T14:31:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14531worker1 # [ 65.925811] nix-daemon[835]: reaped child process 955, status = succeeded4532client # copying 1 paths...4533worker2 # [ 65.961716] nix-grpc-daemon[778]: ts=2026-09-20T14:31:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:486844534worker2 # [ 65.978365] nix-grpc-daemon[778]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4535client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4536worker1 # [ 66.031053] nix-grpc-daemon[775]: ts=2026-09-20T14:31:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:339664537worker1 # [ 66.037855] nix-grpc-daemon[775]: ts=2026-09-20T14:31:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=0 paths=1 nar_bytes_out=1204538client # [ 66.168105] nix-daemon[937]: reaped child process 1023, status = succeeded4539client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.47 seconds)4540client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24541client # [ 66.382322] nix-daemon[937]: accepted connection from pid 1042, user root (trusted)4542client # this derivation will be built:4543client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4544worker1 # [ 66.597425] nix-grpc-daemon[775]: ts=2026-09-20T14:31:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:339664545client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4546worker2 # [ 66.600388] nix-grpc-daemon[778]: ts=2026-09-20T14:31:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486844547worker2 # [ 66.626171] nix-grpc-daemon[778]: ts=2026-09-20T14:31:40Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04548worker2 # [ 66.633675] nix-grpc-daemon[778]: ts=2026-09-20T14:31:40Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14549worker2 # [ 66.642901] nix-grpc-daemon[778]: ts=2026-09-20T14:31:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844550worker2 # [ 66.653539] nix-grpc-daemon[778]: ts=2026-09-20T14:31:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486844551client # copying 1 paths to 'lb:50051'...4552client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4553worker2 # [ 66.672500] nix-grpc-daemon[778]: ts=2026-09-20T14:31:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486844554worker2 # [ 66.687619] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4555worker2 # [ 66.709275] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)4556worker2 # [ 66.788349] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 2/5)4557worker2 # [ 66.820610] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 112 ms (attempt 3/5)4558worker2 # [ 66.940190] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 458 ms (attempt 4/5)4559worker2 # [ 67.401417] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4560worker2 # [ 67.408371] nix-grpc-daemon[778]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4561worker2 # [ 67.659287] nix-daemon[858]: accepted connection from pid 1202, user nix-grpc-daemon (trusted)4562worker2 # [ 67.677477] nix-daemon[858]: reaped child process 1205, status = succeeded4563worker2 # [ 67.854109] nix-daemon[858]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)4564worker2 # [ 67.870215] nix-daemon[858]: reaped child process 1210, status = succeeded4565lb # [ 67.918893] niks3-server[1010]: 2026/09/20 14:31:41 INFO Received uploads request method=POST path=/api/pending_closures4566worker2 # [ 67.897341] nix-grpc-daemon[870]: time=2026-09-20T14:31:41.472Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4567worker2 # [ 67.898834] nix-grpc-daemon[870]: time=2026-09-20T14:31:41.473Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4568lb # [ 67.949050] niks3-server[1010]: 2026/09/20 14:31:41 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4569lb # [ 67.964818] niks3-server[1010]: 2026/09/20 14:31:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4570worker2 # [ 67.939127] nix-grpc-daemon[870]: time=2026-09-20T14:31:41.515Z level=INFO msg="Uploading 1 narinfos"4571lb # [ 67.977649] niks3-server[1010]: 2026/09/20 14:31:41 INFO Signed narinfos id=21 count=14572lb # [ 67.985002] niks3-server[1010]: 2026/09/20 14:31:41 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4573lb # [ 67.996153] niks3-server[1010]: 2026/09/20 14:31:41 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4574lb # [ 68.001180] niks3-server[1010]: 2026/09/20 14:31:41 INFO Completed upload id=214575worker2 # [ 67.969171] nix-grpc-daemon[870]: time=2026-09-20T14:31:41.545Z level=INFO msg="Upload complete. (527ms)"4576lb # [ 68.003995] niks3-server[1010]: 2026/09/20 14:31:41 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4577worker2 # [ 67.976251] nix-grpc-daemon[778]: ts=2026-09-20T14:31:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 paths=1 nar_bytes_in=5684578worker2 # [ 67.988832] nix-grpc-daemon[778]: ts=2026-09-20T14:31:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486844579worker2 # [ 68.000557] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4580worker2 # [ 68.005473] nix-daemon[858]: reaped child process 1196, status = succeeded4581worker2 # [ 68.014269] nix-grpc-daemon[778]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4582client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4583worker2 # [ 68.030555] nix-daemon[858]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4584client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4585worker2 # [ 69.057524] nix-daemon[858]: reaped child process 1215, status = succeeded4586worker2 # [ 69.215050] nix-daemon[858]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)4587worker2 # [ 69.232670] nix-daemon[858]: reaped child process 1233, status = succeeded4588lb # [ 69.278753] niks3-server[1010]: 2026/09/20 14:31:42 INFO Received uploads request method=POST path=/api/pending_closures4589worker2 # [ 69.257094] nix-grpc-daemon[870]: time=2026-09-20T14:31:42.832Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4590lb # [ 69.307853] niks3-server[1010]: 2026/09/20 14:31:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4591worker2 # [ 69.287234] nix-grpc-daemon[870]: time=2026-09-20T14:31:42.863Z level=INFO msg="Uploading 1 narinfos"4592lb # [ 69.322696] niks3-server[1010]: 2026/09/20 14:31:42 INFO Signed narinfos id=22 count=14593lb # [ 69.329240] niks3-server[1010]: 2026/09/20 14:31:42 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4594lb # [ 69.337139] niks3-server[1010]: 2026/09/20 14:31:42 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4595lb # [ 69.344411] niks3-server[1010]: 2026/09/20 14:31:42 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4596lb # [ 69.348453] niks3-server[1010]: 2026/09/20 14:31:42 INFO Completed upload id=224597worker2 # [ 69.316067] nix-grpc-daemon[870]: time=2026-09-20T14:31:42.892Z level=INFO msg="Upload complete. (272ms)"4598lb # [ 69.351184] niks3-server[1010]: 2026/09/20 14:31:42 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4599worker2 # [ 69.324219] nix-grpc-daemon[778]: ts=2026-09-20T14:31:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14600client # copying 1 paths...4601worker2 # [ 69.336755] nix-daemon[858]: reaped child process 1212, status = succeeded4602worker2 # [ 69.339609] nix-grpc-daemon[778]: ts=2026-09-20T14:31:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:486844603client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4604worker2 # [ 69.368149] nix-grpc-daemon[778]: ts=2026-09-20T14:31:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:486844605worker2 # [ 69.372910] nix-grpc-daemon[778]: ts=2026-09-20T14:31:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=0 paths=1 nar_bytes_out=1204606client # [ 69.507602] nix-daemon[937]: reaped child process 1045, status = succeeded4607client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.34 seconds)4608client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24609client # [ 69.729497] nix-daemon[937]: accepted connection from pid 1064, user root (trusted)4610client # this derivation will be built:4611client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4612client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4613worker1 # [ 69.943229] nix-grpc-daemon[775]: ts=2026-09-20T14:31:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:339664614worker1 # [ 69.950823] nix-grpc-daemon[775]: ts=2026-09-20T14:31:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339664615worker1 # [ 69.958544] nix-grpc-daemon[862]: time=2026-09-20T14:31:43.527Z level=INFO msg="All 1 paths already cached"4616worker2 # [ 69.968154] nix-grpc-daemon[778]: ts=2026-09-20T14:31:43Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04617worker2 # [ 69.974446] nix-grpc-daemon[778]: ts=2026-09-20T14:31:43Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14618worker1 # [ 70.021794] nix-grpc-daemon[775]: ts=2026-09-20T14:31:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:339664619worker1 # [ 70.037393] nix-grpc-daemon[775]: ts=2026-09-20T14:31:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339664620worker1 # [ 70.052688] nix-grpc-daemon[862]: time=2026-09-20T14:31:43.623Z level=INFO msg="All 1 paths already cached"4621client # copying 1 paths to 'lb:50051'...4622client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4623worker1 # [ 70.063526] nix-grpc-daemon[775]: ts=2026-09-20T14:31:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:339664624worker1 # [ 70.078898] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4625lb # [ 70.282424] envoy[901]: conn peer=[2001:db8:1::1]:58464 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4312 tx=2222 ms=18654626lb # [ 70.287541] envoy[901]: conn peer=[2001:db8:1::1]:47904 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=284627lb # [ 70.294385] envoy[901]: conn peer=[2001:db8:1::1]:47914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3852 tx=1878 ms=29464628lb # [ 70.299247] envoy[901]: conn peer=[2001:db8:1::1]:47922 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=684629lb # [ 70.303933] envoy[901]: conn peer=[2001:db8:1::1]:47926 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3700 tx=1800 ms=28364630lb # [ 70.308748] envoy[901]: conn peer=[2001:db8:1::1]:47942 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=694631worker1 # [ 70.289486] nix-daemon[835]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4632worker1 # [ 70.308424] nix-daemon[835]: reaped child process 982, status = succeeded4633worker1 # [ 70.487869] nix-daemon[835]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4634worker1 # [ 70.505963] nix-daemon[835]: reaped child process 987, status = succeeded4635lb # [ 70.550677] niks3-server[1010]: 2026/09/20 14:31:44 INFO Received uploads request method=POST path=/api/pending_closures4636worker1 # [ 70.529106] nix-grpc-daemon[862]: time=2026-09-20T14:31:44.099Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4637worker1 # [ 70.530663] nix-grpc-daemon[862]: time=2026-09-20T14:31:44.099Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4638lb # [ 70.576728] niks3-server[1010]: 2026/09/20 14:31:44 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4639lb # [ 70.590399] niks3-server[1010]: 2026/09/20 14:31:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4640worker1 # [ 70.567485] nix-grpc-daemon[862]: time=2026-09-20T14:31:44.138Z level=INFO msg="Uploading 1 narinfos"4641lb # [ 70.603719] niks3-server[1010]: 2026/09/20 14:31:44 INFO Signed narinfos id=23 count=14642lb # [ 70.609313] niks3-server[1010]: 2026/09/20 14:31:44 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4643worker1 # [ 70.587085] nix-grpc-daemon[862]: time=2026-09-20T14:31:44.157Z level=INFO msg="Upload complete. (490ms)"4644lb # [ 70.617821] niks3-server[1010]: 2026/09/20 14:31:44 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4645lb # [ 70.622906] niks3-server[1010]: 2026/09/20 14:31:44 INFO Completed upload id=234646worker1 # [ 70.594399] nix-grpc-daemon[775]: ts=2026-09-20T14:31:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=0 paths=1 nar_bytes_in=5684647lb # [ 70.626337] niks3-server[1010]: 2026/09/20 14:31:44 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4648worker1 # [ 70.606588] nix-grpc-daemon[775]: ts=2026-09-20T14:31:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:339664649worker1 # [ 70.612859] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4650worker1 # [ 70.617365] nix-daemon[835]: reaped child process 977, status = succeeded4651client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4652worker1 # [ 70.629831] nix-daemon[835]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4653client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4654worker1 # [ 71.257483] nix-daemon[835]: reaped child process 991, status = succeeded4655worker1 # [ 71.417926] nix-daemon[835]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4656worker1 # [ 71.434671] nix-daemon[835]: reaped child process 1009, status = succeeded4657lb # [ 71.476890] niks3-server[1010]: 2026/09/20 14:31:45 INFO Received uploads request method=POST path=/api/pending_closures4658worker1 # [ 71.457388] nix-grpc-daemon[862]: time=2026-09-20T14:31:45.026Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4659lb # [ 71.506752] niks3-server[1010]: 2026/09/20 14:31:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4660worker1 # [ 71.485122] nix-grpc-daemon[862]: time=2026-09-20T14:31:45.055Z level=INFO msg="Uploading 1 narinfos"4661lb # [ 71.521928] niks3-server[1010]: 2026/09/20 14:31:45 INFO Signed narinfos id=24 count=14662lb # [ 71.527633] niks3-server[1010]: 2026/09/20 14:31:45 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4663lb # [ 71.532984] niks3-server[1010]: 2026/09/20 14:31:45 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4664worker1 # [ 71.510906] nix-grpc-daemon[862]: time=2026-09-20T14:31:45.081Z level=INFO msg="Upload complete. (269ms)"4665lb # [ 71.542100] niks3-server[1010]: 2026/09/20 14:31:45 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4666lb # [ 71.546499] niks3-server[1010]: 2026/09/20 14:31:45 INFO Completed upload id=244667lb # [ 71.549213] niks3-server[1010]: 2026/09/20 14:31:45 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4668worker1 # [ 71.520735] nix-grpc-daemon[775]: ts=2026-09-20T14:31:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33966 duration_s=0 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14669worker1 # [ 71.529919] nix-daemon[835]: reaped child process 989, status = succeeded4670client # copying 1 paths...4671worker1 # [ 71.567200] nix-grpc-daemon[775]: ts=2026-09-20T14:31:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:339664672client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4673worker2 # [ 71.595315] nix-grpc-daemon[778]: ts=2026-09-20T14:31:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:486844674worker2 # [ 71.608913] nix-grpc-daemon[778]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4675worker2 # [ 71.627070] nix-grpc-daemon[778]: ts=2026-09-20T14:31:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48684 duration_s=0 paths=1 nar_bytes_out=1204676client # [ 71.758588] nix-daemon[937]: reaped child process 1067, status = succeeded4677client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.25 seconds)4678(finished: subtest: an input only one worker has still reaches the builder, in 9.99 seconds)4679subtest: upload whose reference the worker lacks is completed from the cache4680client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4681client # this derivation will be built:4682client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4683client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4684client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.45 seconds)4685client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4686client # this derivation will be built:4687client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4688client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4689client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds)4690client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4691client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4692worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4693worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4694worker2 # [ 73.173879] systemd[1]: Stopping Nix worker-protocol over gRPC...4695worker2 # [ 73.706185] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=signal_exit4696worker2 # [ 73.938358] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=scheduler_yield4697worker1 # [ 73.944934] nix-grpc-daemon[775]: ts=2026-09-20T14:31:47Z level=info event=scheduler_restarting addr=lb:500514698lb # [ 73.974909] niks3-server[1010]: 2026/09/20 14:31:47 INFO lead: released remote=192.168.1.4:389224699worker2 # [ 73.954153] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:345504700worker2 # [ 73.961711] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=worker_gone addr=192.168.1.3:500514701worker2 # [ 73.965913] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=scheduler_restarting addr=lb:500514702worker2 # [ 73.969833] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:345504703worker2 # [ 73.974227] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=worker_gone addr=192.168.1.4:500514704worker1 # [ 74.145438] nix-grpc-daemon[775]: ts=2026-09-20T14:31:47Z level=info event=scheduler_connected addr=lb:500514705worker1 # [ 74.150577] nix-grpc-daemon[775]: ts=2026-09-20T14:31:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4706lb # [ 74.194233] niks3-server[1010]: 2026/09/20 14:31:47 INFO lead: acquired remote=192.168.1.3:460664707worker2 # [ 74.162362] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=scheduler_connected addr=lb:500514708worker1 # [ 74.169916] nix-grpc-daemon[775]: ts=2026-09-20T14:31:47Z level=info event=scheduler_take_over4709worker2 # [ 74.167763] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4710worker1 # [ 74.351518] nix-grpc-daemon[775]: ts=2026-09-20T14:31:47Z level=info event=scheduler_connected addr=lb:500514711worker1 # [ 74.356890] nix-grpc-daemon[775]: ts=2026-09-20T14:31:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4712worker2 # [ 74.369132] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=scheduler_connected addr=lb:500514713worker2 # [ 74.374190] nix-grpc-daemon[778]: ts=2026-09-20T14:31:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4714worker1 # [ 74.557786] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_connected addr=lb:500514715worker1 # [ 74.563144] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4716worker2 # [ 74.575380] nix-grpc-daemon[778]: ts=2026-09-20T14:31:48Z level=info event=scheduler_connected addr=lb:500514717worker2 # [ 74.581370] nix-grpc-daemon[778]: ts=2026-09-20T14:31:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4718worker1 # [ 74.764371] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_connected addr=lb:500514719worker1 # [ 74.769719] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4720worker2 # [ 74.784146] nix-grpc-daemon[778]: ts=2026-09-20T14:31:48Z level=info event=scheduler_connected addr=lb:500514721worker2 # [ 74.788243] nix-grpc-daemon[778]: ts=2026-09-20T14:31:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4722worker2 # [ 74.943141] nix-grpc-daemon[778]: ts=2026-09-20T14:31:48Z level=info event=scheduler_restarting4723worker1 # [ 74.968948] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_connected addr=lb:500514724worker1 # [ 74.973191] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4725worker2 # [ 74.976908] nix-daemon[858]: reaped child process 860, status = succeeded4726worker2 # [ 75.012310] nix-grpc-daemon[778]: ts=2026-09-20T14:31:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4727worker1 # [ 75.175459] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_connected addr=lb:500514728worker1 # [ 75.180520] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4729worker1 # [ 75.383758] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_connected addr=lb:500514730worker1 # [ 75.389375] nix-grpc-daemon[775]: ts=2026-09-20T14:31:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4731worker1 # [ 75.592618] nix-grpc-daemon[775]: ts=2026-09-20T14:31:49Z level=info event=scheduler_connected addr=lb:500514732worker1 # [ 75.602394] nix-grpc-daemon[775]: ts=2026-09-20T14:31:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4733worker1 # [ 75.795975] nix-grpc-daemon[775]: ts=2026-09-20T14:31:49Z level=info event=scheduler_connected addr=lb:500514734worker1 # [ 75.824876] nix-grpc-daemon[775]: ts=2026-09-20T14:31:49Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:474684735worker1 # [ 75.832866] nix-grpc-daemon[775]: ts=2026-09-20T14:31:49Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04736worker2 # [ 75.843368] nix-grpc-daemon[778]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4737worker2 # [ 75.848507] nix-grpc-daemon[778]: E0000 00:00:1789914709.416221 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4738worker2 # [ 75.855681] nix-grpc-daemon[778]: E0000 00:00:1789914709.416954 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4739worker2 # [ 75.970229] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4740worker2 # [ 75.980516] systemd[1]: Stopped Nix worker-protocol over gRPC.4741worker2 # [ 75.982842] systemd[1]: nix-grpc-daemon.service: Consumed 3.706s CPU time over 55.578s wall clock time, 66M memory peak, 90.3K incoming IP traffic, 222.1K outgoing IP traffic.4742worker2 # [ 76.002348] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4743worker2 # [ 76.007462] systemd[1]: Closed Nix worker-protocol over gRPC.4744worker2 # [ 76.010131] systemd[1]: nix-grpc-daemon.socket: Consumed 553us CPU time over 1min 5.202s wall clock time, 256K memory peak, 90.8K incoming IP traffic, 69.2K outgoing IP traffic.4745lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4746lb: must succeed: curl -sf localhost:9901/clusters4747lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4748lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4749client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24750worker1 # [ 76.382387] nix-grpc-daemon[775]: ts=2026-09-20T14:31:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:339664751client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4752worker1 # [ 76.389881] nix-grpc-daemon[775]: ts=2026-09-20T14:31:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:339664753client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.31 seconds)4754worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4755worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4756worker2 # [ 76.522238] systemd[1]: Listening on Nix worker-protocol over gRPC.4757worker1 # [ 76.607775] systemd[1]: Stopping Nix worker-protocol over gRPC...4758worker2 # [ 76.956161] systemd[1]: Starting scheduler-token.service...4759worker2 # [ 77.060704] systemd[1]: scheduler-token.service: Deactivated successfully.4760worker2 # [ 77.064168] systemd[1]: Finished scheduler-token.service.4761worker2 # [ 77.068571] systemd[1]: scheduler-token.service: Consumed 53ms CPU time over 107ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4762worker2 # [ 77.078337] systemd[1]: Starting Nix worker-protocol over gRPC...4763worker2 # [ 77.266694] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4764worker2 # [ 77.269152] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4765worker2 # [ 77.273215] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14766worker2 # [ 77.282044] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4767worker2 # [ 77.283719] systemd[1]: Started Nix worker-protocol over gRPC.4768worker2 # [ 77.327165] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:50Z level=info event=scheduler_connected addr=lb:500514769worker1 # [ 77.342203] nix-grpc-daemon[775]: ts=2026-09-20T14:31:50Z level=info event=oidc_jwks provider=mock keys=14770worker1 # [ 77.342660] nix-grpc-daemon[775]: ts=2026-09-20T14:31:50Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:474684771worker1 # [ 77.351962] nix-grpc-daemon[775]: ts=2026-09-20T14:31:50Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04772worker1 # [ 77.606037] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=signal_exit4773worker1 # [ 77.851615] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=scheduler_yield4774lb # [ 77.885627] niks3-server[1010]: 2026/09/20 14:31:51 INFO lead: released remote=192.168.1.3:460664775worker2 # [ 77.855770] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:51Z level=info event=scheduler_restarting addr=lb:500514776worker1 # [ 77.867665] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:474684777worker1 # [ 77.874667] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=worker_gone addr=192.168.1.4:500514778worker1 # [ 77.878715] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=scheduler_restarting addr=lb:500514779worker1 # [ 77.882696] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:474684780worker1 # [ 77.888277] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=worker_gone addr=192.168.1.3:500514781worker2 # [ 78.057131] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:51Z level=info event=scheduler_connected addr=lb:500514782worker2 # [ 78.062363] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4783worker1 # [ 78.074357] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=scheduler_connected addr=lb:500514784worker1 # [ 78.079591] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4785worker2 # [ 78.263450] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:51Z level=info event=scheduler_connected addr=lb:500514786worker2 # [ 78.269696] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4787worker1 # [ 78.280428] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=scheduler_connected addr=lb:500514788worker1 # [ 78.285448] nix-grpc-daemon[775]: ts=2026-09-20T14:31:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4789worker2 # [ 78.469577] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_connected addr=lb:500514790worker2 # [ 78.474548] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4791worker1 # [ 78.486149] nix-grpc-daemon[775]: ts=2026-09-20T14:31:52Z level=info event=scheduler_connected addr=lb:500514792worker1 # [ 78.490926] nix-grpc-daemon[775]: ts=2026-09-20T14:31:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4793worker2 # [ 78.675524] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_connected addr=lb:500514794worker2 # [ 78.680587] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4795worker1 # [ 78.693097] nix-grpc-daemon[775]: ts=2026-09-20T14:31:52Z level=info event=scheduler_connected addr=lb:500514796worker1 # [ 78.698001] nix-grpc-daemon[775]: ts=2026-09-20T14:31:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4797worker1 # [ 78.857143] nix-grpc-daemon[775]: ts=2026-09-20T14:31:52Z level=info event=scheduler_restarting4798worker2 # [ 78.882415] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_connected addr=lb:500514799worker2 # [ 78.887377] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4800worker1 # [ 78.893801] nix-daemon[835]: reaped child process 837, status = succeeded4801worker1 # [ 78.921768] nix-grpc-daemon[775]: ts=2026-09-20T14:31:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4802worker2 # [ 79.088573] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_connected addr=lb:500514803worker2 # [ 79.093673] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4804worker2 # [ 79.296369] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_connected addr=lb:500514805worker2 # [ 79.300416] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4806worker2 # [ 79.497157] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_connected addr=lb:500514807worker2 # [ 79.502343] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4808worker2 # [ 79.703303] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_connected addr=lb:500514809worker2 # [ 79.708475] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4810worker1 # [ 79.737063] nix-grpc-daemon[775]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4811worker1 # [ 79.742174] nix-grpc-daemon[775]: E0000 00:00:1789914713.306562 818 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true4812worker1 # [ 79.748640] nix-grpc-daemon[775]: E0000 00:00:1789914713.312938 818 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4813worker1 # [ 79.897566] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4814worker1 # [ 79.905160] systemd[1]: Stopped Nix worker-protocol over gRPC.4815worker1 # [ 79.907640] systemd[1]: nix-grpc-daemon.service: Consumed 1.882s CPU time over 59.642s wall clock time, 50.2M memory peak, 54.1K incoming IP traffic, 95K outgoing IP traffic.4816worker2 # [ 79.909539] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_connected addr=lb:500514817worker2 # [ 79.914576] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4818worker1 # [ 79.926854] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4819worker1 # [ 79.931192] systemd[1]: Closed Nix worker-protocol over gRPC.4820worker1 # [ 79.934462] systemd[1]: nix-grpc-daemon.socket: Consumed 555us CPU time over 1min 8.563s wall clock time, 256K memory peak, 63.5K incoming IP traffic, 57.4K outgoing IP traffic.4821lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4822lb: must succeed: curl -sf localhost:9901/clusters4823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4824lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4825client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24826worker2 # [ 80.115701] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_connected addr=lb:500514827worker2 # [ 80.120994] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4828lb # [ 80.284327] envoy[901]: conn peer=[2001:db8:1::1]:47952 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3731 tx=1791 ms=17434829lb # [ 80.284701] envoy[901]: conn peer=[2001:db8:1::1]:53618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=954830lb # [ 80.284865] envoy[901]: conn peer=[2001:db8:1::4]:42650 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=39206 tx=2485 ms=542674831lb # [ 80.284983] envoy[901]: conn peer=[2001:db8:1::1]:53620 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=244832lb # [ 80.285104] envoy[901]: conn peer=[2001:db8:1::3]:37652 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=8976 tx=2432 ms=582664833worker2 # [ 80.294537] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:512104834worker2 # [ 80.313077] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)4835worker2 # [ 80.328875] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_connected addr=lb:500514836worker2 # [ 80.334613] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4837worker2 # [ 80.355230] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)4838worker2 # [ 80.462293] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)4839worker2 # [ 80.505743] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 3/5)4840worker2 # [ 80.524165] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_connected addr=lb:500514841worker2 # [ 80.529230] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4842worker2 # [ 80.539193] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 292 ms (attempt 4/5)4843worker2 # [ 80.730189] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_connected addr=lb:500514844worker2 # [ 80.735351] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4845worker2 # [ 80.839319] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4846client # copying 2 paths...4847client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4848worker2 # [ 80.865775] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:512104849client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4850worker2 # [ 80.883245] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)4851worker2 # [ 80.917566] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=niks3_push_started pid=13164852worker2 # [ 80.938300] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_connected addr=lb:500514853worker2 # [ 80.943628] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4854worker2 # [ 81.140135] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_connected addr=lb:500514855worker2 # [ 81.142200] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4856worker2 # [ 81.152882] nix-grpc-daemon[1316]: time=2026-09-20T14:31:54.728Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4857worker2 # [ 81.330079] nix-daemon[858]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)4858worker2 # [ 81.343153] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_connected addr=lb:500514859worker2 # [ 81.346170] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4860worker2 # [ 81.352499] nix-daemon[858]: reaped child process 1332, status = succeeded4861lb # [ 81.398289] niks3-server[1010]: 2026/09/20 14:31:54 INFO Received uploads request method=POST path=/api/pending_closures4862lb # [ 81.407702] niks3-server[1010]: 2026/09/20 14:31:54 INFO Received uploads request method=POST path=/api/pending_closures4863worker2 # [ 81.386729] nix-grpc-daemon[1316]: time=2026-09-20T14:31:54.960Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4864worker2 # [ 81.389185] nix-grpc-daemon[1316]: time=2026-09-20T14:31:54.960Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4865worker2 # [ 81.391009] nix-grpc-daemon[1316]: time=2026-09-20T14:31:54.961Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4866lb # [ 81.456746] niks3-server[1010]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4867lb # [ 81.471138] niks3-server[1010]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4868lb # [ 81.479048] niks3-server[1010]: 2026/09/20 14:31:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4869worker2 # [ 81.452258] nix-grpc-daemon[1316]: time=2026-09-20T14:31:55.028Z level=INFO msg="Uploading 3 narinfos"4870lb # [ 81.486922] niks3-server[1010]: 2026/09/20 14:31:55 INFO Signed narinfos id=25 count=14871lb # [ 81.492344] niks3-server[1010]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4872lb # [ 81.499714] niks3-server[1010]: 2026/09/20 14:31:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4873lb # [ 81.507431] niks3-server[1010]: 2026/09/20 14:31:55 INFO Signed narinfos id=26 count=24874lb # [ 81.511948] niks3-server[1010]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4875lb # [ 81.520421] niks3-server[1010]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4876lb # [ 81.530601] niks3-server[1010]: 2026/09/20 14:31:55 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4877lb # [ 81.536610] niks3-server[1010]: 2026/09/20 14:31:55 INFO Completed upload id=254878worker2 # [ 81.504666] nix-grpc-daemon[1316]: time=2026-09-20T14:31:55.080Z level=INFO msg="Upload complete. (347ms)"4879lb # [ 81.539220] niks3-server[1010]: 2026/09/20 14:31:55 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4880lb # [ 81.544250] niks3-server[1010]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4881lb # [ 81.550016] niks3-server[1010]: 2026/09/20 14:31:55 INFO Completed upload id=264882worker2 # [ 81.516442] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51210 duration_s=0 paths=2 nar_bytes_in=3124883lb # [ 81.557347] niks3-server[1010]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4884worker2 # [ 81.531809] nix-daemon[858]: reaped child process 1314, status = succeeded4885worker2 # [ 81.546051] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:55Z level=info event=scheduler_connected addr=lb:500514886worker2 # [ 81.550713] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4887client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.54 seconds)4888worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4889worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)4890worker1 # [ 81.677214] systemd[1]: Listening on Nix worker-protocol over gRPC.4891worker1 # [ 81.687551] systemd[1]: Starting Nix worker-protocol over gRPC...4892worker2 # [ 81.752106] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:55Z level=info event=scheduler_connected addr=lb:500514893worker2 # [ 81.757227] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4894worker1 # [ 81.892519] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4895worker1 # [ 81.895155] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4896worker1 # [ 81.899560] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14897worker1 # [ 81.908838] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4898worker1 # [ 81.910537] systemd[1]: Started Nix worker-protocol over gRPC.4899lb # [ 81.957518] niks3-server[1010]: 2026/09/20 14:31:55 INFO lead: acquired remote=192.168.1.3:331764900lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4901lb: must succeed: curl -sf localhost:9901/clusters4902worker1 # [ 81.932285] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=scheduler_take_over4903worker1 # [ 81.956272] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=scheduler_connected addr=lb:500514904worker1 # [ 81.959449] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:327804905worker2 # [ 81.958336] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:55Z level=info event=scheduler_connected addr=lb:500514906worker1 # [ 81.961615] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04907worker1 # [ 81.969045] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=oidc_jwks provider=mock keys=14908worker1 # [ 81.970324] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:327804909worker1 # [ 81.972594] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04910lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4911lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.07 seconds)4912(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.23 seconds)4913subtest: scheduler down: the other node takes the lock and keeps it4914lb: must succeed: curl -sf localhost:9901/clusters4915lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4916lb: must succeed: curl -sf localhost:9901/clusters4917lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4918lb: must succeed: curl -sf localhost:9901/clusters4919lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4920lb: must succeed: curl -sf localhost:9901/clusters4921lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4922worker1 # [ 82.256597] systemd[1]: Stopping Nix worker-protocol over gRPC...4923worker1 # [ 82.916893] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:56Z level=info event=signal_exit4924worker1 # [ 83.126313] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:56Z level=info event=scheduler_yield4925lb # [ 83.166760] niks3-server[1010]: 2026/09/20 14:31:56 INFO lead: released remote=192.168.1.3:331764926worker2 # [ 83.135688] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:56Z level=info event=scheduler_restarting addr=lb:500514927worker1 # [ 83.137973] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:327804928worker1 # [ 83.145454] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:56Z level=info event=worker_gone addr=192.168.1.4:500514929worker1 # [ 83.149661] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:56Z level=info event=scheduler_restarting addr=lb:500514930worker1 # [ 83.153975] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:327804931worker1 # [ 83.158426] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:56Z level=info event=worker_gone addr=192.168.1.3:500514932worker2 # [ 83.335850] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:56Z level=info event=scheduler_connected addr=lb:500514933worker2 # [ 83.341201] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4934worker1 # [ 83.344907] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:56Z level=info event=scheduler_connected addr=lb:500514935worker1 # [ 83.349847] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4936worker2 # [ 83.542236] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_connected addr=lb:500514937worker2 # [ 83.547499] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4938worker1 # [ 83.550811] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:57Z level=info event=scheduler_connected addr=lb:500514939worker1 # [ 83.556328] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4940worker2 # [ 83.748705] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_connected addr=lb:500514941worker2 # [ 83.753966] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4942worker1 # [ 83.757146] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:57Z level=info event=scheduler_connected addr=lb:500514943worker1 # [ 83.762109] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4944worker2 # [ 83.955177] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_connected addr=lb:500514945worker2 # [ 83.960651] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4946worker1 # [ 83.964707] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:57Z level=info event=scheduler_connected addr=lb:500514947worker1 # [ 83.968855] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4948worker1 # [ 84.131488] nix-grpc-daemon[1032]: ts=2026-09-20T14:31:57Z level=info event=scheduler_restarting4949worker2 # [ 84.163770] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_connected addr=lb:500514950worker2 # [ 84.168141] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4951worker2 # [ 84.364431] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_connected addr=lb:500514952worker2 # [ 84.369680] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4953worker2 # [ 84.570845] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_connected addr=lb:500514954worker2 # [ 84.576224] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4955worker2 # [ 84.777309] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_connected addr=lb:500514956worker2 # [ 84.782390] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4957worker1 # [ 84.947319] nix-grpc-daemon[1032]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4958worker1 # [ 84.947771] nix-grpc-daemon[1032]: E0000 00:00:1789914718.517928 1075 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4959worker1 # [ 84.947924] nix-grpc-daemon[1032]: E0000 00:00:1789914718.518167 1075 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4960worker2 # [ 84.983575] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_connected addr=lb:500514961worker2 # [ 84.988766] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4962worker2 # [ 85.191636] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_connected addr=lb:500514963worker2 # [ 85.195845] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4964worker2 # [ 85.392348] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_connected addr=lb:500514965worker2 # [ 85.397313] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4966worker2 # [ 85.598465] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:59Z level=info event=scheduler_connected addr=lb:500514967worker2 # [ 85.603605] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4968worker2 # [ 85.804660] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:59Z level=info event=scheduler_connected addr=lb:500514969worker2 # [ 85.809621] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4970worker1 # [ 85.944908] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4971worker1 # [ 85.947953] systemd[1]: Stopped Nix worker-protocol over gRPC.4972worker1 # [ 85.953640] systemd[1]: nix-grpc-daemon.service: Consumed 198ms CPU time over 4.254s wall clock time, 13.6M memory peak, 9.2K incoming IP traffic, 7.9K outgoing IP traffic.4973worker1 # [ 85.970881] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4974worker1 # [ 85.973270] systemd[1]: Closed Nix worker-protocol over gRPC.4975worker1 # [ 85.976471] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 4.302s wall clock time, 256K memory peak, 13.4K incoming IP traffic, 10.2K outgoing IP traffic.4976worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over4977worker2 # [ 86.011400] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:59Z level=info event=scheduler_connected addr=lb:500514978worker2 # [ 86.018729] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4979worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)4980worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4981worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4982worker2 # [ 86.219552] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:59Z level=info event=scheduler_connected addr=lb:500514983worker2 # [ 86.223593] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4984worker2 # [ 86.420204] nix-grpc-daemon[1260]: ts=2026-09-20T14:31:59Z level=info event=scheduler_connected addr=lb:500514985worker2 # [ 86.425190] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4986worker2 # [ 86.626309] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_connected addr=lb:500514987worker2 # [ 86.631527] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4988worker2 # [ 86.832619] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_connected addr=lb:500514989worker2 # [ 86.837662] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4990worker2 # [ 87.038789] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_connected addr=lb:500514991worker2 # [ 87.043909] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4992worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4993worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4994worker2 # [ 87.246697] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_connected addr=lb:500514995worker2 # [ 87.250631] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4996lb # [ 87.333311] niks3-server[1010]: 2026/09/20 14:32:00 INFO lead: acquired remote=192.168.1.4:577324997worker2 # [ 87.302053] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:00Z level=info event=scheduler_take_over4998worker2 # [ 87.447464] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:01Z level=info event=scheduler_connected addr=lb:500514999worker2 # [ 87.474213] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:01Z level=info event=oidc_jwks provider=mock keys=15000worker2 # [ 87.478711] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:01Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:443825001worker2 # [ 87.484000] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:01Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05002worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5003worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5004client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25005worker2 # [ 88.552600] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:512105006client # don't know how to build these paths:5007client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5008worker2 # [ 88.563325] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:512105009worker2 # [ 88.587782] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:02Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05010worker2 # [ 88.594845] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:02Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15011worker2 # [ 88.631718] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:512105012worker2 # [ 88.646947] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:512105013client # copying 1 paths to 'lb:50051'...5014worker2 # [ 88.662254] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:512105015client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5016worker2 # [ 88.680535] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5017worker2 # [ 88.891599] nix-daemon[858]: accepted connection from pid 1356, user nix-grpc-daemon (trusted)5018worker2 # [ 88.908791] nix-daemon[858]: reaped child process 1359, status = succeeded5019worker2 # [ 89.082197] nix-daemon[858]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5020worker2 # [ 89.097821] nix-daemon[858]: reaped child process 1364, status = succeeded5021lb # [ 89.141676] niks3-server[1010]: 2026/09/20 14:32:02 INFO Received uploads request method=POST path=/api/pending_closures5022worker2 # [ 89.120208] nix-grpc-daemon[1316]: time=2026-09-20T14:32:02.695Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5023worker2 # [ 89.121686] nix-grpc-daemon[1316]: time=2026-09-20T14:32:02.695Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5024lb # [ 89.167899] niks3-server[1010]: 2026/09/20 14:32:02 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5025lb # [ 89.182471] niks3-server[1010]: 2026/09/20 14:32:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5026worker2 # [ 89.155690] nix-grpc-daemon[1316]: time=2026-09-20T14:32:02.731Z level=INFO msg="Uploading 1 narinfos"5027lb # [ 89.197556] niks3-server[1010]: 2026/09/20 14:32:02 INFO Signed narinfos id=27 count=15028lb # [ 89.214393] niks3-server[1010]: 2026/09/20 14:32:02 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5029worker2 # [ 89.188504] nix-grpc-daemon[1316]: time=2026-09-20T14:32:02.764Z level=INFO msg="Upload complete. (490ms)"5030lb # [ 89.224893] niks3-server[1010]: 2026/09/20 14:32:02 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5031lb # [ 89.227719] niks3-server[1010]: 2026/09/20 14:32:02 INFO Completed upload id=275032worker2 # [ 89.196164] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51210 duration_s=0 paths=1 nar_bytes_in=5205033worker2 # [ 89.211632] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:512105034worker2 # [ 89.215660] nix-grpc-daemon[1316]: time=2026-09-20T14:32:02.781Z level=WARN msg="Failed to register uploaded object" key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo error="server returned 404: object is not pending upload\n"5035worker2 # [ 89.225214] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5036worker2 # [ 89.228603] nix-daemon[858]: reaped child process 1354, status = succeeded5037client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5038client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5039worker2 # [ 89.240183] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5040client # farm-a> LOG-farm-a-on-standby5041lb # [ 90.286203] envoy[901]: conn peer=[2001:db8:1::1]:53624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1011 tx=282 ms=12385042worker2 # [ 90.271573] nix-daemon[858]: reaped child process 1368, status = succeeded5043worker2 # [ 90.468796] nix-daemon[858]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)5044worker2 # [ 90.486938] nix-daemon[858]: reaped child process 1386, status = succeeded5045lb # [ 90.293799] envoy[901]: conn peer=[2001:db8:1::3]:39630 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1028 tx=349 ms=22175046lb # [ 90.530706] niks3-server[1010]: 2026/09/20 14:32:04 INFO Received uploads request method=POST path=/api/pending_closures5047worker2 # [ 90.520210] nix-grpc-daemon[1316]: time=2026-09-20T14:32:04.093Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5048worker2 # [ 90.522315] nix-grpc-daemon[1316]: time=2026-09-20T14:32:04.093Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5049worker2 # [ 90.565834] nix-grpc-daemon[1316]: time=2026-09-20T14:32:04.140Z level=INFO msg="Uploading 1 narinfos"5050lb # [ 90.589255] niks3-server[1010]: 2026/09/20 14:32:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5051lb # [ 90.605049] niks3-server[1010]: 2026/09/20 14:32:04 INFO Signed narinfos id=28 count=15052lb # [ 90.607721] niks3-server[1010]: 2026/09/20 14:32:04 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5053lb # [ 90.618482] niks3-server[1010]: 2026/09/20 14:32:04 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5054lb # [ 90.632445] niks3-server[1010]: 2026/09/20 14:32:04 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5055lb # [ 90.638501] niks3-server[1010]: 2026/09/20 14:32:04 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5056lb # [ 90.642093] niks3-server[1010]: 2026/09/20 14:32:04 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5057lb # [ 90.645830] niks3-server[1010]: 2026/09/20 14:32:04 INFO Completed upload id=285058worker2 # [ 90.613797] nix-grpc-daemon[1316]: time=2026-09-20T14:32:04.189Z level=INFO msg="Upload complete. (362ms)"5059worker2 # [ 90.636223] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51210 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15060worker2 # [ 90.652357] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:04Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05061worker2 # [ 90.656901] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:04Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15062worker2 # [ 90.661242] nix-daemon[858]: reaped child process 1366, status = succeeded5063worker2 # [ 90.680998] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:512105064worker2 # [ 90.696428] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:512105065worker2 # [ 90.709971] nix-grpc-daemon[1316]: time=2026-09-20T14:32:04.286Z level=INFO msg="All 1 paths already cached"5066client # copying 1 paths to 'lb:50051'...5067client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5068worker2 # [ 90.719633] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:512105069worker2 # [ 90.734058] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5070worker2 # [ 90.938592] nix-daemon[858]: accepted connection from pid 1392, user nix-grpc-daemon (trusted)5071worker2 # [ 90.956256] nix-daemon[858]: reaped child process 1395, status = succeeded5072worker2 # [ 91.128667] nix-daemon[858]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)5073worker2 # [ 91.145404] nix-daemon[858]: reaped child process 1400, status = succeeded5074lb # [ 91.190237] niks3-server[1010]: 2026/09/20 14:32:04 INFO Received uploads request method=POST path=/api/pending_closures5075worker2 # [ 91.167880] nix-grpc-daemon[1316]: time=2026-09-20T14:32:04.743Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5076worker2 # [ 91.169564] nix-grpc-daemon[1316]: time=2026-09-20T14:32:04.743Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5077lb # [ 91.217875] niks3-server[1010]: 2026/09/20 14:32:04 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5078worker2 # [ 91.198007] nix-grpc-daemon[1316]: time=2026-09-20T14:32:04.773Z level=INFO msg="Uploading 1 narinfos"5079lb # [ 91.234465] niks3-server[1010]: 2026/09/20 14:32:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5080lb # [ 91.245681] niks3-server[1010]: 2026/09/20 14:32:04 INFO Signed narinfos id=29 count=15081worker2 # [ 91.224779] nix-grpc-daemon[1316]: time=2026-09-20T14:32:04.800Z level=INFO msg="Upload complete. (473ms)"5082lb # [ 91.255957] niks3-server[1010]: 2026/09/20 14:32:04 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5083worker2 # [ 91.230692] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51210 duration_s=0 paths=1 nar_bytes_in=6405084lb # [ 91.263088] niks3-server[1010]: 2026/09/20 14:32:04 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5085lb # [ 91.269001] niks3-server[1010]: 2026/09/20 14:32:04 INFO Completed upload id=295086worker2 # [ 91.240447] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:512105087lb # [ 91.271713] niks3-server[1010]: 2026/09/20 14:32:04 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5088worker2 # [ 91.249464] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5089worker2 # [ 91.251877] nix-daemon[858]: reaped child process 1390, status = succeeded5090client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5091worker2 # [ 91.263230] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5092client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5093client # farm-b> LOG-farm-b-on-standby5094worker2 # [ 92.308344] nix-daemon[858]: reaped child process 1404, status = succeeded5095worker2 # [ 92.479110] nix-daemon[858]: accepted connection from pid 1419, user nix-grpc-daemon (trusted)5096worker2 # [ 92.495705] nix-daemon[858]: reaped child process 1422, status = succeeded5097lb # [ 92.540651] niks3-server[1010]: 2026/09/20 14:32:06 INFO Received uploads request method=POST path=/api/pending_closures5098worker2 # [ 92.519871] nix-grpc-daemon[1316]: time=2026-09-20T14:32:06.095Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5099worker2 # [ 92.521504] nix-grpc-daemon[1316]: time=2026-09-20T14:32:06.095Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5100lb # [ 92.580800] niks3-server[1010]: 2026/09/20 14:32:06 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5101worker2 # [ 92.561649] nix-grpc-daemon[1316]: time=2026-09-20T14:32:06.137Z level=INFO msg="Uploading 1 narinfos"5102lb # [ 92.594000] niks3-server[1010]: 2026/09/20 14:32:06 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5103lb # [ 92.605295] niks3-server[1010]: 2026/09/20 14:32:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5104lb # [ 92.610600] niks3-server[1010]: 2026/09/20 14:32:06 INFO Signed narinfos id=30 count=15105worker2 # [ 92.585414] nix-grpc-daemon[1316]: time=2026-09-20T14:32:06.161Z level=INFO msg="Upload complete. (313ms)"5106lb # [ 92.617543] niks3-server[1010]: 2026/09/20 14:32:06 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5107lb # [ 92.621205] niks3-server[1010]: 2026/09/20 14:32:06 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5108lb # [ 92.625590] niks3-server[1010]: 2026/09/20 14:32:06 INFO Completed upload id=305109lb # [ 92.628260] niks3-server[1010]: 2026/09/20 14:32:06 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5110worker2 # [ 92.594744] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51210 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15111worker2 # [ 92.615488] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:06Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05112worker2 # [ 92.619150] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:06Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15113worker2 # [ 92.623175] nix-daemon[858]: reaped child process 1402, status = succeeded5114worker2 # [ 92.655885] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:512105115worker2 # [ 92.670965] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:512105116worker2 # [ 92.685336] nix-grpc-daemon[1316]: time=2026-09-20T14:32:06.261Z level=INFO msg="All 2 paths already cached"5117client # copying 1 paths to 'lb:50051'...5118client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5119worker2 # [ 92.695141] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:512105120worker2 # [ 92.711920] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5121worker2 # [ 92.911025] nix-daemon[858]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)5122worker2 # [ 92.929861] nix-daemon[858]: reaped child process 1431, status = succeeded5123worker2 # [ 93.099531] nix-daemon[858]: accepted connection from pid 1433, user nix-grpc-daemon (trusted)5124worker2 # [ 93.114654] nix-daemon[858]: reaped child process 1436, status = succeeded5125lb # [ 93.161886] niks3-server[1010]: 2026/09/20 14:32:06 INFO Received uploads request method=POST path=/api/pending_closures5126worker2 # [ 93.141577] nix-grpc-daemon[1316]: time=2026-09-20T14:32:06.717Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5127worker2 # [ 93.143047] nix-grpc-daemon[1316]: time=2026-09-20T14:32:06.717Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5128lb # [ 93.191327] niks3-server[1010]: 2026/09/20 14:32:06 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5129worker2 # [ 93.173035] nix-grpc-daemon[1316]: time=2026-09-20T14:32:06.748Z level=INFO msg="Uploading 1 narinfos"5130lb # [ 93.208304] niks3-server[1010]: 2026/09/20 14:32:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5131lb # [ 93.218790] niks3-server[1010]: 2026/09/20 14:32:06 INFO Signed narinfos id=31 count=15132lb # [ 93.224701] niks3-server[1010]: 2026/09/20 14:32:06 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5133lb # [ 93.234670] niks3-server[1010]: 2026/09/20 14:32:06 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5134lb # [ 93.241371] niks3-server[1010]: 2026/09/20 14:32:06 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5135lb # [ 93.246235] niks3-server[1010]: 2026/09/20 14:32:06 INFO Completed upload id=315136worker2 # [ 93.214265] nix-grpc-daemon[1316]: time=2026-09-20T14:32:06.790Z level=INFO msg="Upload complete. (486ms)"5137worker2 # [ 93.222134] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51210 duration_s=0 paths=1 nar_bytes_in=7685138worker2 # [ 93.236405] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:512105139worker2 # [ 93.246933] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5140worker2 # [ 93.249527] nix-daemon[858]: reaped child process 1426, status = succeeded5141client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5142client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5143worker2 # [ 93.264366] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5144client # farm-top> LOG-farm-top-on-standby5145worker2 # [ 94.910928] nix-daemon[858]: reaped child process 1440, status = succeeded5146worker2 # [ 95.085557] nix-daemon[858]: accepted connection from pid 1455, user nix-grpc-daemon (trusted)5147worker2 # [ 95.102444] nix-daemon[858]: reaped child process 1458, status = succeeded5148lb # [ 95.147125] niks3-server[1010]: 2026/09/20 14:32:08 INFO Received uploads request method=POST path=/api/pending_closures5149worker2 # [ 95.126909] nix-grpc-daemon[1316]: time=2026-09-20T14:32:08.701Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5150worker2 # [ 95.128904] nix-grpc-daemon[1316]: time=2026-09-20T14:32:08.701Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5151lb # [ 95.182733] niks3-server[1010]: 2026/09/20 14:32:08 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5152worker2 # [ 95.164879] nix-grpc-daemon[1316]: time=2026-09-20T14:32:08.740Z level=INFO msg="Uploading 1 narinfos"5153lb # [ 95.197174] niks3-server[1010]: 2026/09/20 14:32:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5154lb # [ 95.205642] niks3-server[1010]: 2026/09/20 14:32:08 INFO Signed narinfos id=32 count=15155lb # [ 95.211568] niks3-server[1010]: 2026/09/20 14:32:08 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5156lb # [ 95.224140] niks3-server[1010]: 2026/09/20 14:32:08 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5157lb # [ 95.227611] niks3-server[1010]: 2026/09/20 14:32:08 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5158lb # [ 95.232540] niks3-server[1010]: 2026/09/20 14:32:08 INFO Completed upload id=325159worker2 # [ 95.200696] nix-grpc-daemon[1316]: time=2026-09-20T14:32:08.776Z level=INFO msg="Upload complete. (324ms)"5160lb # [ 95.236512] niks3-server[1010]: 2026/09/20 14:32:08 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5161worker2 # [ 95.213086] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51210 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15162worker2 # [ 95.232362] nix-daemon[858]: reaped child process 1438, status = succeeded5163client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 6.98 seconds)5164client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5165client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5166worker1 # [ 95.576854] systemd[1]: Listening on Nix worker-protocol over gRPC.5167worker1 # [ 95.583438] systemd[1]: Starting Nix worker-protocol over gRPC...5168worker1 # [ 95.775990] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5169worker1 # [ 95.778574] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5170worker1 # [ 95.782595] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15171worker1 # [ 95.790080] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5172worker1 # [ 95.791832] systemd[1]: Started Nix worker-protocol over gRPC.5173lb: must succeed: curl -sf localhost:9901/clusters5174worker1 # [ 95.832855] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:09Z level=info event=scheduler_connected addr=lb:500515175worker2 # [ 95.833928] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:443825176worker2 # [ 95.840138] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05177lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5178worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5179worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5180lb: must succeed: curl -sf localhost:9901/clusters5181lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5182lb: must succeed: curl -sf localhost:9901/clusters5183lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5184worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5185worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5186(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.18 seconds)5187subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5188client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5189client # [ 96.248734] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5190client # Running as unit: rs.service; invocation ID: 7c72fe4936d54ac59fc95d11a6b33aec5191client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5192worker1 # [ 96.499384] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:591205193worker1 # [ 96.514700] nix-daemon[835]: accepted connection from pid 1096, user nix-grpc-daemon (trusted)5194client # [ 96.530631] nix[1168]: don't know how to build these paths:5195client # [ 96.531860] nix[1168]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5196worker2 # [ 96.545150] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:10Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05197worker2 # [ 96.545489] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:10Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15198worker2 # [ 96.591578] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:512105199worker2 # [ 96.606853] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:512105200client # [ 96.620374] nix[1168]: copying 1 paths to 'lb:50051'...5201worker2 # [ 96.621070] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:512105202client # [ 96.624153] nix[1168]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5203worker2 # [ 96.637911] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5204worker2 # [ 96.862768] nix-daemon[858]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5205worker2 # [ 96.881464] nix-daemon[858]: reaped child process 1474, status = succeeded5206worker2 # [ 97.055465] nix-daemon[858]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)5207worker2 # [ 97.071925] nix-daemon[858]: reaped child process 1479, status = succeeded5208lb # [ 97.116691] niks3-server[1010]: 2026/09/20 14:32:10 INFO Received uploads request method=POST path=/api/pending_closures5209worker2 # [ 97.098193] nix-grpc-daemon[1316]: time=2026-09-20T14:32:10.673Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5210worker2 # [ 97.099632] nix-grpc-daemon[1316]: time=2026-09-20T14:32:10.673Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5211lb # [ 97.148520] niks3-server[1010]: 2026/09/20 14:32:10 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5212worker2 # [ 97.131144] nix-grpc-daemon[1316]: time=2026-09-20T14:32:10.707Z level=INFO msg="Uploading 1 narinfos"5213lb # [ 97.165339] niks3-server[1010]: 2026/09/20 14:32:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5214lb # [ 97.175060] niks3-server[1010]: 2026/09/20 14:32:10 INFO Signed narinfos id=33 count=15215lb # [ 97.179603] niks3-server[1010]: 2026/09/20 14:32:10 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5216lb # [ 97.187897] niks3-server[1010]: 2026/09/20 14:32:10 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5217lb # [ 97.193764] niks3-server[1010]: 2026/09/20 14:32:10 INFO Completed upload id=335218worker2 # [ 97.161518] nix-grpc-daemon[1316]: time=2026-09-20T14:32:10.737Z level=INFO msg="Upload complete. (507ms)"5219lb # [ 97.196550] niks3-server[1010]: 2026/09/20 14:32:10 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5220worker2 # [ 97.168243] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51210 duration_s=0 paths=1 nar_bytes_in=4165221worker2 # [ 97.180328] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:512105222worker2 # [ 97.190096] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5223worker2 # [ 97.194601] nix-daemon[858]: reaped child process 1469, status = succeeded5224client # [ 97.203892] nix[1168]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5225worker2 # [ 97.206585] nix-daemon[858]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5226lb: must succeed: curl -sf localhost:9901/clusters5227lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5228lb: must succeed: curl -sf localhost:9901/clusters5229lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5230lb: must succeed: curl -sf localhost:9901/clusters5231lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5232lb: must succeed: curl -sf localhost:9901/clusters5233lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5234worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5235worker1 # [ 99.190409] systemd-journald[275]: Received client request to rotate journal, rotating.5236worker1 # [ 99.214092] systemd-journald[275]: Vacuuming done, freed 0B of archived journals from /var/log/journal/52fb374f0fb2438e8f37f413c929ec86.5237worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5238worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5239worker2 # [ 99.267406] systemd[1]: Stopping Nix worker-protocol over gRPC...5240worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5241client: must succeed: sleep 5.05242worker2 # [ 99.295907] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:12Z level=info event=draining5243lb # [ 100.289073] envoy[901]: conn peer=[2001:db8:1::1]:59490 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9649 tx=3886 ms=66765244client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5245worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5246worker2 # [ 104.856670] nix-daemon[858]: reaped child process 1483, status = succeeded5247worker2 # [ 105.038652] nix-daemon[858]: accepted connection from pid 1523, user nix-grpc-daemon (trusted)5248worker2 # [ 105.055213] nix-daemon[858]: reaped child process 1526, status = succeeded5249lb # [ 105.099739] niks3-server[1010]: 2026/09/20 14:32:18 INFO Received uploads request method=POST path=/api/pending_closures5250worker2 # [ 105.083293] nix-grpc-daemon[1316]: time=2026-09-20T14:32:18.655Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5251worker2 # [ 105.085682] nix-grpc-daemon[1316]: time=2026-09-20T14:32:18.655Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5252lb # [ 105.143476] niks3-server[1010]: 2026/09/20 14:32:18 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5253lb # [ 105.154482] niks3-server[1010]: 2026/09/20 14:32:18 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5254lb # [ 105.163393] niks3-server[1010]: 2026/09/20 14:32:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5255worker2 # [ 105.136553] nix-grpc-daemon[1316]: time=2026-09-20T14:32:18.712Z level=INFO msg="Uploading 1 narinfos"5256lb # [ 105.170911] niks3-server[1010]: 2026/09/20 14:32:18 INFO Signed narinfos id=34 count=15257lb # [ 105.175209] niks3-server[1010]: 2026/09/20 14:32:18 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5258lb # [ 105.183630] niks3-server[1010]: 2026/09/20 14:32:18 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5259lb # [ 105.187369] niks3-server[1010]: 2026/09/20 14:32:18 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5260lb # [ 105.191602] niks3-server[1010]: 2026/09/20 14:32:18 INFO Completed upload id=345261worker2 # [ 105.159784] nix-grpc-daemon[1316]: time=2026-09-20T14:32:18.735Z level=INFO msg="Upload complete. (334ms)"5262worker2 # [ 105.173224] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51210 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15263worker2 # [ 105.191463] nix-daemon[858]: reaped child process 1481, status = succeeded5264client # [ 105.212901] systemd[1]: rs.service: Deactivated successfully.5265client # [ 105.217267] systemd[1]: rs.service: Consumed 175ms CPU time over 8.968s wall clock time, 43.2M memory peak, 6.9K incoming IP traffic, 8.3K outgoing IP traffic.5266worker2 # [ 105.298305] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:18Z level=info event=signal_exit5267worker2 # [ 105.523792] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_yield5268lb # [ 105.564383] niks3-server[1010]: 2026/09/20 14:32:19 INFO lead: released remote=192.168.1.4:577325269worker2 # [ 105.537742] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_restarting addr=lb:500515270worker1 # [ 105.541330] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_restarting addr=lb:500515271worker2 # [ 105.544135] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:443825272worker2 # [ 105.548436] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=worker_gone addr=192.168.1.4:500515273worker2 # [ 105.552313] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:443825274worker2 # [ 105.556640] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=worker_gone addr=192.168.1.3:500515275worker2 # [ 105.739281] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_connected addr=lb:500515276worker1 # [ 105.740971] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_connected addr=lb:500515277worker2 # [ 105.744696] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5278worker1 # [ 105.746885] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5279worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.16 seconds)5280client: waiting for success: ! systemctl is-active rs5281lb # [ 105.833301] niks3-server[1010]: 2026/09/20 14:32:19 INFO lead: acquired remote=192.168.1.3:512545282worker1 # [ 105.806948] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_take_over5283client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5284client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5285client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds)5286client: must fail: journalctl -u rs | grep -q 'reconnecting'5287worker2 # [ 105.945692] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_connected addr=lb:500515288worker1 # [ 105.947385] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_connected addr=lb:500515289worker2 # [ 105.950847] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5290client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5291worker1 # [ 105.952913] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5292lb: must succeed: curl -sf localhost:9901/clusters5293lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5294worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5295worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5296worker2 # [ 106.151846] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_connected addr=lb:500515297worker1 # [ 106.153915] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_connected addr=lb:500515298worker2 # [ 106.157078] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5299worker1 # [ 106.159218] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5300worker1 # [ 106.359776] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_connected addr=lb:500515301worker2 # [ 106.359488] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_connected addr=lb:500515302worker1 # [ 106.365095] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5303worker2 # [ 106.363748] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5304worker2 # [ 106.529340] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:20Z level=info event=scheduler_restarting5305worker1 # [ 106.567584] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:20Z level=info event=scheduler_connected addr=lb:500515306worker1 # [ 106.571796] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5307worker2 # [ 106.581991] nix-daemon[858]: reaped child process 1309, status = succeeded5308worker2 # [ 106.584841] nix-grpc-daemon[1260]: ts=2026-09-20T14:32:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5309worker1 # [ 106.767922] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:20Z level=info event=scheduler_connected addr=lb:500515310worker1 # [ 106.773023] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5311worker1 # [ 106.973595] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:20Z level=info event=scheduler_connected addr=lb:500515312worker1 # [ 106.982841] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:20Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:516625313worker1 # [ 106.988225] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:20Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05314lb: must succeed: curl -sf localhost:9901/clusters5315lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5316worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5317worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5318worker2 # [ 107.359983] nix-grpc-daemon[1260]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5319worker2 # [ 107.360364] nix-grpc-daemon[1260]: E0000 00:00:1789914740.934607 1303 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5320worker2 # [ 107.360487] nix-grpc-daemon[1260]: E0000 00:00:1789914740.935013 1303 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5321worker2 # [ 108.024899] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5322worker2 # [ 108.028871] systemd[1]: Stopped Nix worker-protocol over gRPC.5323worker2 # [ 108.031401] systemd[1]: nix-grpc-daemon.service: Consumed 2.037s CPU time over 30.942s wall clock time, 36.8M memory peak, 63.1K incoming IP traffic, 161.2K outgoing IP traffic.5324worker2 # [ 108.049832] systemd[1]: Starting scheduler-token.service...5325worker2 # [ 108.146023] systemd[1]: scheduler-token.service: Deactivated successfully.5326worker2 # [ 108.152878] systemd[1]: Finished scheduler-token.service.5327worker2 # [ 108.156422] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 99ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5328worker2 # [ 108.161303] systemd[1]: Starting Nix worker-protocol over gRPC...5329lb: must succeed: curl -sf localhost:9901/clusters5330worker2 # [ 108.343723] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5332worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5333worker2 # [ 108.346367] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5334worker2 # [ 108.350305] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15335worker2 # [ 108.358186] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5336worker2 # [ 108.359908] systemd[1]: Started Nix worker-protocol over gRPC.5337worker2 # [ 108.394566] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:21Z level=info event=scheduler_connected addr=lb:500515338worker1 # [ 108.408660] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:21Z level=info event=oidc_jwks provider=mock keys=15339worker1 # [ 108.412645] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:516625340worker1 # [ 108.417862] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05341worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)5342(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.27 seconds)5343subtest: scheduler restart mid-build: build finishes, no double build5344client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5345client # [ 108.514483] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5346client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5347worker1 # [ 108.775606] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:591205348client # Running as unit: mid.service; invocation ID: b536d2a53ef34343a26d68d1117e124d5349client # [ 108.783333] nix[1204]: don't know how to build these paths:5350client # [ 108.785406] nix[1204]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5351worker1 # [ 108.793678] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:22Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05352worker1 # [ 108.801428] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:22Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15353worker1 # [ 108.841733] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591205354worker1 # [ 108.862700] nix-grpc-daemon[1096]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)5355worker1 # [ 108.924390] nix-grpc-daemon[1096]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 144 ms (attempt 2/5)5356worker1 # [ 109.068974] nix-grpc-daemon[1096]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 3/5)5357worker1 # [ 109.264327] nix-grpc-daemon[1096]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 715 ms (attempt 4/5)5358worker1 # [ 109.981502] nix-grpc-daemon[1096]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5359worker1 # [ 110.001532] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591205360client # [ 110.016854] nix[1204]: copying 1 paths to 'lb:50051'...5361worker1 # [ 110.016329] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591205362client # [ 110.017099] nix[1204]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5363worker1 # [ 110.031396] nix-daemon[835]: accepted connection from pid 1096, user nix-grpc-daemon (trusted)5364worker1 # [ 110.057751] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:23Z level=info event=niks3_push_started pid=12155365lb # [ 110.289003] envoy[901]: conn peer=[2001:db8:1::1]:53546 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=1328 ms=86885366lb # [ 110.293986] envoy[901]: conn peer=[2001:db8:1::4]:45346 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=36252 tx=2077 ms=292425367worker1 # [ 110.281750] nix-grpc-daemon[1215]: time=2026-09-20T14:32:23.852Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5368worker1 # [ 110.453858] nix-daemon[835]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5369worker1 # [ 110.470343] nix-daemon[835]: reaped child process 1231, status = succeeded5370worker1 # [ 110.643620] nix-daemon[835]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)5371worker1 # [ 110.657952] nix-daemon[835]: reaped child process 1236, status = succeeded5372lb # [ 110.704909] niks3-server[1010]: 2026/09/20 14:32:24 INFO Received uploads request method=POST path=/api/pending_closures5373worker1 # [ 110.685495] nix-grpc-daemon[1215]: time=2026-09-20T14:32:24.255Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5374worker1 # [ 110.686968] nix-grpc-daemon[1215]: time=2026-09-20T14:32:24.255Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5375lb # [ 110.738165] niks3-server[1010]: 2026/09/20 14:32:24 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5376worker1 # [ 110.720761] nix-grpc-daemon[1215]: time=2026-09-20T14:32:24.291Z level=INFO msg="Uploading 1 narinfos"5377lb # [ 110.750810] niks3-server[1010]: 2026/09/20 14:32:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5378lb # [ 110.758163] niks3-server[1010]: 2026/09/20 14:32:24 INFO Signed narinfos id=35 count=15379lb # [ 110.770951] niks3-server[1010]: 2026/09/20 14:32:24 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5380lb # [ 110.785699] niks3-server[1010]: 2026/09/20 14:32:24 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5381worker1 # [ 110.760078] nix-grpc-daemon[1215]: time=2026-09-20T14:32:24.330Z level=INFO msg="Upload complete. (473ms)"5382lb # [ 110.790706] niks3-server[1010]: 2026/09/20 14:32:24 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5383lb # [ 110.795903] niks3-server[1010]: 2026/09/20 14:32:24 INFO Completed upload id=355384worker1 # [ 110.768137] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59120 duration_s=0 paths=1 nar_bytes_in=4245385worker1 # [ 110.781489] nix-grpc-daemon[1096]: ts=2026-09-20T14:32:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591205386worker1 # [ 110.787622] nix-daemon[835]: accepted connection from pid 1096, user nix-grpc-daemon (trusted)5387worker1 # [ 110.792651] nix-daemon[835]: reaped child process 1213, status = succeeded5388worker1 # [ 110.803794] nix-daemon[835]: accepted connection from pid 1096, user nix-grpc-daemon (trusted)5389client # [ 110.801180] nix[1204]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5390lb: must succeed: curl -sf localhost:9901/clusters5391lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5392lb: must succeed: curl -sf localhost:9901/clusters5393lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5394worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5395worker1 # [ 112.485916] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1096 (nix-grpc-daemon) on client request.5396worker1 # [ 112.488778] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5397lb # [ 112.522055] niks3-server[1010]: 2026/09/20 14:32:26 INFO lead: released remote=192.168.1.3:512545398client # [ 112.503418] nix[1204]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5399worker2 # [ 112.502494] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:26Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5400worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)5401client: must succeed: sleep 5.05402client # [ 112.509682] nix[1204]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5403worker1 # [ 112.524257] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5404worker1 # [ 112.525741] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5405worker1 # [ 112.526958] systemd[1]: nix-grpc-daemon.service: Consumed 569ms CPU time over 16.939s wall clock time, 34M memory peak, 17K incoming IP traffic, 27.5K outgoing IP traffic.5406worker1 # [ 112.539256] nix-daemon[835]: reaped child process 1238, status = succeeded5407worker1 # [ 112.540944] nix-daemon[835]: reaped child process 1152, status = succeeded5408worker1 # [ 112.548532] nix-daemon[1240]: unexpected Nix daemon error: error: interrupted by the user5409worker1 # [ 112.555260] nix-daemon[835]: reaped child process 1240, status = failed with exit code 15410worker1 # [ 112.675514] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5411worker1 # [ 112.680345] systemd[1]: Starting Nix worker-protocol over gRPC...5412worker2 # [ 112.712071] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:26Z level=info event=scheduler_connected addr=lb:500515413worker1 # [ 112.881677] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5414worker1 # [ 112.884252] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5415worker1 # [ 112.888262] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15416worker1 # [ 112.895644] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5417worker1 # [ 112.903593] systemd[1]: Started Nix worker-protocol over gRPC.5418lb # [ 112.939996] niks3-server[1010]: 2026/09/20 14:32:26 INFO lead: acquired remote=192.168.1.3:385545419worker1 # [ 112.914246] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=scheduler_take_over5420worker1 # [ 112.927151] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=oidc_jwks provider=mock keys=15421worker1 # [ 112.928756] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:536905422worker1 # [ 112.930836] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05423worker1 # [ 112.943863] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=scheduler_connected addr=lb:500515424worker1 # [ 112.945815] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:536905425worker1 # [ 112.947363] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05426worker1 # [ 113.012980] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05427worker1 # [ 113.016832] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:26Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15428worker2 # [ 113.069543] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377225429worker2 # [ 113.084272] nix-daemon[858]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5430worker2 # [ 113.107665] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 1/5)5431worker2 # [ 113.146260] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 2/5)5432worker2 # [ 113.291799] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 314 ms (attempt 3/5)5433worker2 # [ 113.614453] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 4/5)5434worker2 # [ 113.761630] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5435worker2 # [ 113.778896] nix-grpc-daemon[1536]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5436worker2 # [ 113.806429] nix-daemon[858]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5437client # [ 113.824598] nix[1204]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5438worker2 # [ 113.824522] nix-daemon[858]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5439client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5440client: waiting for success: ! systemctl is-active mid5441worker2 # [ 118.055358] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:31Z level=info event=niks3_push_started pid=16375442worker2 # [ 118.101518] nix-daemon[858]: reaped child process 1613, status = succeeded5443worker2 # [ 118.309349] nix-grpc-daemon[1637]: time=2026-09-20T14:32:31.884Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5444worker2 # [ 118.480178] nix-daemon[858]: accepted connection from pid 1650, user nix-grpc-daemon (trusted)5445worker2 # [ 118.496481] nix-daemon[858]: reaped child process 1653, status = succeeded5446lb # [ 118.543913] niks3-server[1010]: 2026/09/20 14:32:32 INFO Received uploads request method=POST path=/api/pending_closures5447worker2 # [ 118.522970] nix-grpc-daemon[1637]: time=2026-09-20T14:32:32.097Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5448worker2 # [ 118.525750] nix-grpc-daemon[1637]: time=2026-09-20T14:32:32.097Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5449lb # [ 118.588792] niks3-server[1010]: 2026/09/20 14:32:32 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5450lb # [ 118.602058] niks3-server[1010]: 2026/09/20 14:32:32 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5451lb # [ 118.614062] niks3-server[1010]: 2026/09/20 14:32:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5452lb # [ 118.623147] niks3-server[1010]: 2026/09/20 14:32:32 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5453lb # [ 118.634260] niks3-server[1010]: 2026/09/20 14:32:32 INFO Signed narinfos id=36 count=15454worker2 # [ 118.604189] nix-grpc-daemon[1637]: time=2026-09-20T14:32:32.178Z level=INFO msg="Uploading 1 narinfos"5455lb # [ 118.648086] niks3-server[1010]: 2026/09/20 14:32:32 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5456lb # [ 118.656851] niks3-server[1010]: 2026/09/20 14:32:32 INFO Completed upload id=365457worker2 # [ 118.623717] nix-grpc-daemon[1637]: time=2026-09-20T14:32:32.199Z level=INFO msg="Upload complete. (310ms)"5458lb # [ 118.659490] niks3-server[1010]: 2026/09/20 14:32:32 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5459worker2 # [ 118.636337] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37722 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15460worker2 # [ 118.652536] nix-daemon[858]: reaped child process 1611, status = succeeded5461client # [ 118.685227] systemd[1]: mid.service: Deactivated successfully.5462client # [ 118.691211] systemd[1]: mid.service: Consumed 187ms CPU time over 10.174s wall clock time, 39.2M memory peak, 8.2K incoming IP traffic, 9.9K outgoing IP traffic.5463client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5464client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5465client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5466(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.62 seconds)5467subtest: interrupting the client stops the build on the worker5468client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5469client # [ 119.106490] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5470client # Running as unit: intr.service; invocation ID: cd11dba5baa7420699e3b7e9dd81d5805471client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.05 seconds)5472client # [ 119.349467] nix[1240]: don't know how to build these paths:5473client # [ 119.351218] nix[1240]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5474worker2 # [ 119.348400] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:377225475worker1 # [ 119.363902] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:32Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05476worker1 # [ 119.369037] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:32Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15477worker1 # [ 119.424121] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125478worker1 # [ 119.438648] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5479worker1 # [ 119.461709] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 1/5)5480worker1 # [ 119.544686] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 2/5)5481worker1 # [ 119.589015] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 129 ms (attempt 3/5)5482worker1 # [ 119.725446] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 561 ms (attempt 4/5)5483lb # [ 120.291761] envoy[901]: conn peer=[2001:db8:1::3]:50080 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1476 tx=539 ms=166815484lb # [ 120.296955] envoy[901]: conn peer=[2001:db8:1::1]:43652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3871 tx=1700 ms=98825485worker1 # [ 120.295356] nix-grpc-daemon[1276]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5486worker1 # [ 120.316209] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537125487client # [ 120.324259] nix[1240]: copying 1 paths to 'lb:50051'...5488client # [ 120.329438] nix[1240]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5489worker1 # [ 120.331215] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537125490worker1 # [ 120.349201] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5491worker1 # [ 120.381856] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:33Z level=info event=niks3_push_started pid=13525492worker1 # [ 120.588089] nix-grpc-daemon[1352]: time=2026-09-20T14:32:34.158Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5493worker1 # [ 120.760727] nix-daemon[835]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5494worker1 # [ 120.777131] nix-daemon[835]: reaped child process 1368, status = succeeded5495worker1 # [ 120.948563] nix-daemon[835]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)5496worker1 # [ 120.966829] nix-daemon[835]: reaped child process 1373, status = succeeded5497lb # [ 121.010480] niks3-server[1010]: 2026/09/20 14:32:34 INFO Received uploads request method=POST path=/api/pending_closures5498worker1 # [ 120.990608] nix-grpc-daemon[1352]: time=2026-09-20T14:32:34.560Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5499worker1 # [ 120.992550] nix-grpc-daemon[1352]: time=2026-09-20T14:32:34.560Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5500lb # [ 121.044911] niks3-server[1010]: 2026/09/20 14:32:34 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5501worker1 # [ 121.033981] nix-grpc-daemon[1352]: time=2026-09-20T14:32:34.604Z level=INFO msg="Uploading 1 narinfos"5502lb # [ 121.057893] niks3-server[1010]: 2026/09/20 14:32:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5503lb # [ 121.067434] niks3-server[1010]: 2026/09/20 14:32:34 INFO Signed narinfos id=37 count=15504lb # [ 121.077062] niks3-server[1010]: 2026/09/20 14:32:34 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5505lb # [ 121.086431] niks3-server[1010]: 2026/09/20 14:32:34 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5506worker1 # [ 121.064578] nix-grpc-daemon[1352]: time=2026-09-20T14:32:34.635Z level=INFO msg="Upload complete. (473ms)"5507lb # [ 121.097104] niks3-server[1010]: 2026/09/20 14:32:34 INFO Completed upload id=375508lb # [ 121.101319] niks3-server[1010]: 2026/09/20 14:32:34 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5509worker1 # [ 121.072140] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=4245510worker1 # [ 121.080686] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125511worker1 # [ 121.091126] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5512worker1 # [ 121.095690] nix-daemon[835]: reaped child process 1349, status = succeeded5513client # [ 121.105271] nix[1240]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5514worker1 # [ 121.108584] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5515client: must succeed: systemctl kill -s INT intr5516client # [ 122.886598] systemd[1]: intr.service: Sent signal SIGINT to main process 1240 (nix) on client request.5517worker1 # [ 122.889993] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:36Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5518client # [ 122.892884] nix[1240]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5519client # [ 122.904133] nix[1240]: error: interrupted by the user5520client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5521worker1 # [ 122.944920] nix-daemon[1377]: unexpected Nix daemon error: error: interrupted by the user5522client # [ 122.945354] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5523client # [ 122.947943] systemd[1]: intr.service: Failed with result 'exit-code'.5524worker1 # [ 122.950670] nix-daemon[835]: reaped child process 1375, status = succeeded5525client # [ 122.951985] systemd[1]: intr.service: Consumed 161ms CPU time over 3.838s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 7.9K outgoing IP traffic.5526worker1 # [ 122.955600] nix-daemon[835]: reaped child process 1377, status = failed with exit code 15527(finished: subtest: interrupting the client stops the build on the worker, in 4.00 seconds)5528subtest: a deploy mid-build drains: the build finishes, the new generation takes over5529client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5530client # [ 123.127272] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5531client # Running as unit: sw.service; invocation ID: e74945243083463c967fb52f663d32a05532client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5533client # [ 123.389182] nix[1260]: don't know how to build these paths:5534client # [ 123.390970] nix[1260]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5535worker2 # [ 123.388613] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:377225536worker1 # [ 123.400783] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:36Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05537worker1 # [ 123.407398] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:36Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15538worker2 # [ 123.452107] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377225539worker2 # [ 123.467745] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377225540client # [ 123.482958] nix[1260]: copying 1 paths to 'lb:50051'...5541worker2 # [ 123.482194] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377225542client # [ 123.485566] nix[1260]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5543worker2 # [ 123.497109] nix-daemon[858]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5544worker2 # [ 123.699560] nix-daemon[858]: accepted connection from pid 1681, user nix-grpc-daemon (trusted)5545worker2 # [ 123.714875] nix-daemon[858]: reaped child process 1684, status = succeeded5546worker2 # [ 123.888085] nix-daemon[858]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)5547worker2 # [ 123.905964] nix-daemon[858]: reaped child process 1689, status = succeeded5548lb # [ 123.950191] niks3-server[1010]: 2026/09/20 14:32:37 INFO Received uploads request method=POST path=/api/pending_closures5549worker2 # [ 123.929204] nix-grpc-daemon[1637]: time=2026-09-20T14:32:37.504Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5550worker2 # [ 123.930689] nix-grpc-daemon[1637]: time=2026-09-20T14:32:37.504Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5551lb # [ 123.979607] niks3-server[1010]: 2026/09/20 14:32:37 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5552worker2 # [ 123.961729] nix-grpc-daemon[1637]: time=2026-09-20T14:32:37.537Z level=INFO msg="Uploading 1 narinfos"5553lb # [ 123.993488] niks3-server[1010]: 2026/09/20 14:32:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5554lb # [ 124.007060] niks3-server[1010]: 2026/09/20 14:32:37 INFO Signed narinfos id=38 count=15555lb # [ 124.017868] niks3-server[1010]: 2026/09/20 14:32:37 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5556lb # [ 124.033013] niks3-server[1010]: 2026/09/20 14:32:37 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5557worker2 # [ 124.007488] nix-grpc-daemon[1637]: time=2026-09-20T14:32:37.583Z level=INFO msg="Upload complete. (493ms)"5558lb # [ 124.043408] niks3-server[1010]: 2026/09/20 14:32:37 INFO Completed upload id=385559worker2 # [ 124.016322] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37722 duration_s=0 paths=1 nar_bytes_in=4165560worker2 # [ 124.032782] nix-grpc-daemon[1637]: time=2026-09-20T14:32:37.598Z level=WARN msg="Failed to register uploaded object" key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo error="server returned 404: object is not pending upload\n"5561worker2 # [ 124.044819] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377225562worker2 # [ 124.056314] nix-daemon[858]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5563worker2 # [ 124.061138] nix-daemon[858]: reaped child process 1679, status = succeeded5564client # [ 124.062312] nix[1260]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5565worker2 # [ 124.065296] nix-daemon[858]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5566worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25567worker2 # Checking switch inhibitors... done5568worker2 # [ 126.578704] nixos[1724]: switching to system configuration /nix/store/lipj73azsya8lp370bv5zwh9743lgssg-nixos-system-worker2-test5569worker2 # [ 126.585148] systemd[1]: Stopped target Local File Systems.5570worker2 # [ 126.585942] systemd[1]: Stopped target Remote File Systems.5571worker2 # activating the configuration...5572worker2 # setting up /etc...5573worker2 # [ 127.345123] systemd[1]: Reload requested from client PID 1724 ('.switch-to-conf') (unit backdoor.service)...5574worker2 # [ 127.346910] systemd[1]: Reloading...5575worker2 # [ 128.224485] systemd[1]: Reloading finished in 872 ms.5576worker2 # restarting sysinit-reactivation.target5577worker2 # [ 128.302154] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5578worker2 # [ 128.449150] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5579worker2 # [ 128.456552] systemd[1]: Reached target Reactivate sysinit units.5580worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5581worker2 # [ 128.461489] systemd[1]: Reloading D-Bus System Message Bus...5582worker2 # [ 128.463844] dbus-broker-launch[495]: Caught SIGHUP, trigger reload.5583worker2 # [ 128.472944] systemd[1]: Reloading Nix worker-protocol over gRPC...5584worker2 # [ 128.492956] dbus-broker-launch[495]: Looking up NSS user entry for 'systemd-timesync'...5585worker2 # [ 128.502561] dbus-broker-launch[495]: NSS returned no entry for 'systemd-timesync'5586worker2 # [ 128.504788] dbus-broker-launch[495]: Invalid user-name in /nix/store/c8a4kacp5lnfwrcxrjmwczbl8ba32m4a-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5587worker2 # [ 128.517827] systemd[1]: Reloaded Nix worker-protocol over gRPC.5588worker2 # [ 128.548478] systemd[1]: Reloaded D-Bus System Message Bus.5589worker2 # [ 128.553424] systemd[1]: Reached target Local File Systems.5590worker2 # [ 128.557176] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5591worker2 # [ 128.562760] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5592worker2 # [ 128.568285] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5593worker2 # [ 128.571618] systemd[1]: Starting Create SUID/SGID Wrappers...5594worker2 # [ 128.574709] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5595worker2 # [ 128.575913] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675596worker2 # [ 128.588692] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5597worker2 # [ 128.592986] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5598worker2 # [ 128.596209] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5599worker2 # [ 128.600560] systemd[1]: Reached target Remote File Systems.5600worker2 # [ 129.083748] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5601worker2 # [ 129.088076] systemd[1]: Finished Create SUID/SGID Wrappers.5602worker2 # [ 129.089180] systemd[1]: Starting linger-users.service...5603worker2 # [ 129.144672] systemd[1]: linger-users.service: Deactivated successfully.5604worker2 # [ 129.152158] systemd[1]: Finished linger-users.service.5605worker2 # [ 129.370933] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:42Z level=info event=draining5606worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5607worker2 # [ 129.402486] nixos[1724]: finished switching to system configuration /nix/store/lipj73azsya8lp370bv5zwh9743lgssg-nixos-system-worker2-test5608worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.65 seconds)5609worker2: must succeed: pgrep -f 'read -t [9]0 x'5610worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5611client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25612client # don't know how to build these paths:5613client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5614worker1 # [ 129.756491] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537125615worker1 # [ 129.762926] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537125616worker1 # [ 129.790808] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:43Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05617worker1 # [ 129.797658] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:43Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15618worker1 # [ 129.841750] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125619worker1 # [ 129.856660] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537125620client # copying 1 paths to 'lb:50051'...5621worker1 # [ 129.872473] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537125622client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5623worker1 # [ 129.896147] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5624worker1 # [ 130.105641] nix-daemon[835]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)5625worker1 # [ 130.122851] nix-daemon[835]: reaped child process 1430, status = succeeded5626lb # [ 130.297415] envoy[901]: conn peer=[2001:db8:1::1]:57458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2944 tx=753 ms=35685627worker1 # [ 130.293206] nix-daemon[835]: accepted connection from pid 1432, user nix-grpc-daemon (trusted)5628worker1 # [ 130.309204] nix-daemon[835]: reaped child process 1435, status = succeeded5629lb # [ 130.352959] niks3-server[1010]: 2026/09/20 14:32:43 INFO Received uploads request method=POST path=/api/pending_closures5630worker1 # [ 130.337437] nix-grpc-daemon[1352]: time=2026-09-20T14:32:43.907Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5631worker1 # [ 130.338888] nix-grpc-daemon[1352]: time=2026-09-20T14:32:43.907Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5632lb # [ 130.401993] niks3-server[1010]: 2026/09/20 14:32:43 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5633worker1 # [ 130.386663] nix-grpc-daemon[1352]: time=2026-09-20T14:32:43.957Z level=INFO msg="Uploading 1 narinfos"5634lb # [ 130.411991] niks3-server[1010]: 2026/09/20 14:32:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5635lb # [ 130.421277] niks3-server[1010]: 2026/09/20 14:32:44 INFO Signed narinfos id=39 count=15636lb # [ 130.423265] niks3-server[1010]: 2026/09/20 14:32:44 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5637lb # [ 130.431257] niks3-server[1010]: 2026/09/20 14:32:44 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5638lb # [ 130.436962] niks3-server[1010]: 2026/09/20 14:32:44 INFO Completed upload id=395639worker1 # [ 130.406778] nix-grpc-daemon[1352]: time=2026-09-20T14:32:43.977Z level=INFO msg="Upload complete. (492ms)"5640lb # [ 130.440117] niks3-server[1010]: 2026/09/20 14:32:44 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5641worker1 # [ 130.416623] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=5285642worker1 # [ 130.430291] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125643worker1 # [ 130.436874] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5644worker1 # [ 130.441837] nix-daemon[835]: reaped child process 1425, status = succeeded5645client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5646worker1 # [ 130.452838] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5647client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5648client # farm-a> LOG-farm-a-during-drain5649worker1 # [ 131.292007] nix-daemon[835]: reaped child process 1439, status = succeeded5650worker1 # [ 131.473809] nix-daemon[835]: accepted connection from pid 1454, user nix-grpc-daemon (trusted)5651worker1 # [ 131.491228] nix-daemon[835]: reaped child process 1457, status = succeeded5652lb # [ 131.533948] niks3-server[1010]: 2026/09/20 14:32:45 INFO Received uploads request method=POST path=/api/pending_closures5653worker1 # [ 131.514658] nix-grpc-daemon[1352]: time=2026-09-20T14:32:45.085Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5654worker1 # [ 131.519751] nix-grpc-daemon[1352]: time=2026-09-20T14:32:45.087Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5655lb # [ 131.574616] niks3-server[1010]: 2026/09/20 14:32:45 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5656lb # [ 131.585793] niks3-server[1010]: 2026/09/20 14:32:45 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5657lb # [ 131.600654] niks3-server[1010]: 2026/09/20 14:32:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5658worker1 # [ 131.577347] nix-grpc-daemon[1352]: time=2026-09-20T14:32:45.147Z level=INFO msg="Uploading 1 narinfos"5659lb # [ 131.612827] niks3-server[1010]: 2026/09/20 14:32:45 INFO Signed narinfos id=40 count=15660lb # [ 131.615588] niks3-server[1010]: 2026/09/20 14:32:45 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5661lb # [ 131.623956] niks3-server[1010]: 2026/09/20 14:32:45 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5662lb # [ 131.634715] niks3-server[1010]: 2026/09/20 14:32:45 INFO Completed upload id=405663worker1 # [ 131.604441] nix-grpc-daemon[1352]: time=2026-09-20T14:32:45.175Z level=INFO msg="Upload complete. (339ms)"5664lb # [ 131.637291] niks3-server[1010]: 2026/09/20 14:32:45 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5665worker1 # [ 131.620980] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15666worker1 # [ 131.636354] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:45Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05667worker1 # [ 131.639944] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:45Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15668worker1 # [ 131.644715] nix-daemon[835]: reaped child process 1437, status = succeeded5669worker1 # [ 131.665722] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125670worker1 # [ 131.685983] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537125671client # copying 1 paths to 'lb:50051'...5672worker1 # [ 131.701528] nix-grpc-daemon[1352]: time=2026-09-20T14:32:45.271Z level=INFO msg="All 1 paths already cached"5673worker1 # [ 131.706882] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537125674client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5675worker1 # [ 131.729615] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5676worker1 # [ 131.932782] nix-daemon[835]: accepted connection from pid 1463, user nix-grpc-daemon (trusted)5677worker1 # [ 131.952630] nix-daemon[835]: reaped child process 1466, status = succeeded5678worker1 # [ 132.127869] nix-daemon[835]: accepted connection from pid 1468, user nix-grpc-daemon (trusted)5679worker1 # [ 132.147091] nix-daemon[835]: reaped child process 1471, status = succeeded5680lb # [ 132.190199] niks3-server[1010]: 2026/09/20 14:32:45 INFO Received uploads request method=POST path=/api/pending_closures5681worker1 # [ 132.169442] nix-grpc-daemon[1352]: time=2026-09-20T14:32:45.739Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5682worker1 # [ 132.170980] nix-grpc-daemon[1352]: time=2026-09-20T14:32:45.739Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5683lb # [ 132.217007] niks3-server[1010]: 2026/09/20 14:32:45 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5684worker1 # [ 132.199598] nix-grpc-daemon[1352]: time=2026-09-20T14:32:45.770Z level=INFO msg="Uploading 1 narinfos"5685lb # [ 132.233331] niks3-server[1010]: 2026/09/20 14:32:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5686lb # [ 132.244167] niks3-server[1010]: 2026/09/20 14:32:45 INFO Signed narinfos id=41 count=15687lb # [ 132.250530] niks3-server[1010]: 2026/09/20 14:32:45 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5688worker1 # [ 132.230932] nix-grpc-daemon[1352]: time=2026-09-20T14:32:45.801Z level=INFO msg="Upload complete. (483ms)"5689lb # [ 132.260690] niks3-server[1010]: 2026/09/20 14:32:45 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5690lb # [ 132.267310] niks3-server[1010]: 2026/09/20 14:32:45 INFO Completed upload id=415691worker1 # [ 132.237044] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=6405692lb # [ 132.270693] niks3-server[1010]: 2026/09/20 14:32:45 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5693worker1 # [ 132.249363] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125694worker1 # [ 132.256304] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5695worker1 # [ 132.261160] nix-daemon[835]: reaped child process 1461, status = succeeded5696client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5697client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5698worker1 # [ 132.274426] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5699client # farm-b> LOG-farm-b-during-drain5700worker1 # [ 133.514788] nix-daemon[835]: reaped child process 1475, status = succeeded5701worker1 # [ 133.700473] nix-daemon[835]: accepted connection from pid 1490, user nix-grpc-daemon (trusted)5702worker1 # [ 133.716418] nix-daemon[835]: reaped child process 1493, status = succeeded5703lb # [ 133.761799] niks3-server[1010]: 2026/09/20 14:32:47 INFO Received uploads request method=POST path=/api/pending_closures5704worker1 # [ 133.742102] nix-grpc-daemon[1352]: time=2026-09-20T14:32:47.312Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5705worker1 # [ 133.743580] nix-grpc-daemon[1352]: time=2026-09-20T14:32:47.312Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5706lb # [ 133.792733] niks3-server[1010]: 2026/09/20 14:32:47 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5707lb # [ 133.809251] niks3-server[1010]: 2026/09/20 14:32:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5708worker1 # [ 133.785212] nix-grpc-daemon[1352]: time=2026-09-20T14:32:47.355Z level=INFO msg="Uploading 1 narinfos"5709lb # [ 133.822811] niks3-server[1010]: 2026/09/20 14:32:47 INFO Signed narinfos id=42 count=15710lb # [ 133.830404] niks3-server[1010]: 2026/09/20 14:32:47 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5711lb # [ 133.843006] niks3-server[1010]: 2026/09/20 14:32:47 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5712worker1 # [ 133.816831] nix-grpc-daemon[1352]: time=2026-09-20T14:32:47.387Z level=INFO msg="Upload complete. (332ms)"5713lb # [ 133.847715] niks3-server[1010]: 2026/09/20 14:32:47 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5714lb # [ 133.851864] niks3-server[1010]: 2026/09/20 14:32:47 INFO Completed upload id=425715lb # [ 133.855133] niks3-server[1010]: 2026/09/20 14:32:47 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5716worker1 # [ 133.832133] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15717worker1 # [ 133.848900] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:47Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05718worker1 # [ 133.853983] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:47Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=15719worker1 # [ 133.858455] nix-daemon[835]: reaped child process 1473, status = succeeded5720worker1 # [ 133.885982] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125721worker1 # [ 133.900991] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537125722worker1 # [ 133.915979] nix-grpc-daemon[1352]: time=2026-09-20T14:32:47.486Z level=INFO msg="All 2 paths already cached"5723client # copying 1 paths to 'lb:50051'...5724worker1 # [ 133.926107] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537125725client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5726worker1 # [ 133.948874] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5727worker1 # [ 134.161440] nix-daemon[835]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5728worker1 # [ 134.178499] nix-daemon[835]: reaped child process 1502, status = succeeded5729worker1 # [ 134.351014] nix-daemon[835]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5730worker1 # [ 134.367494] nix-daemon[835]: reaped child process 1507, status = succeeded5731lb # [ 134.409554] niks3-server[1010]: 2026/09/20 14:32:47 INFO Received uploads request method=POST path=/api/pending_closures5732worker1 # [ 134.388704] nix-grpc-daemon[1352]: time=2026-09-20T14:32:47.959Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5733worker1 # [ 134.390181] nix-grpc-daemon[1352]: time=2026-09-20T14:32:47.959Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5734lb # [ 134.436911] niks3-server[1010]: 2026/09/20 14:32:48 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5735worker1 # [ 134.418920] nix-grpc-daemon[1352]: time=2026-09-20T14:32:47.989Z level=INFO msg="Uploading 1 narinfos"5736lb # [ 134.452879] niks3-server[1010]: 2026/09/20 14:32:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5737lb # [ 134.463421] niks3-server[1010]: 2026/09/20 14:32:48 INFO Signed narinfos id=43 count=15738lb # [ 134.467591] niks3-server[1010]: 2026/09/20 14:32:48 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5739lb # [ 134.475722] niks3-server[1010]: 2026/09/20 14:32:48 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5740worker1 # [ 134.449350] nix-grpc-daemon[1352]: time=2026-09-20T14:32:48.020Z level=INFO msg="Upload complete. (483ms)"5741lb # [ 134.480362] niks3-server[1010]: 2026/09/20 14:32:48 INFO Completed upload id=435742lb # [ 134.483724] niks3-server[1010]: 2026/09/20 14:32:48 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5743worker1 # [ 134.456286] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=7685744worker1 # [ 134.466096] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125745worker1 # [ 134.471353] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5746worker1 # [ 134.475631] nix-daemon[835]: reaped child process 1497, status = succeeded5747client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5748worker1 # [ 134.487374] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5749client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5750client # farm-top> LOG-farm-top-during-drain5751worker1 # [ 135.534044] nix-daemon[835]: reaped child process 1511, status = succeeded5752worker1 # [ 135.711676] nix-daemon[835]: accepted connection from pid 1526, user nix-grpc-daemon (trusted)5753worker1 # [ 135.730980] nix-daemon[835]: reaped child process 1529, status = succeeded5754lb # [ 135.773892] niks3-server[1010]: 2026/09/20 14:32:49 INFO Received uploads request method=POST path=/api/pending_closures5755worker1 # [ 135.754696] nix-grpc-daemon[1352]: time=2026-09-20T14:32:49.323Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5756worker1 # [ 135.756356] nix-grpc-daemon[1352]: time=2026-09-20T14:32:49.323Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5757lb # [ 135.805036] niks3-server[1010]: 2026/09/20 14:32:49 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5758lb # [ 135.818449] niks3-server[1010]: 2026/09/20 14:32:49 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5759lb # [ 135.831347] niks3-server[1010]: 2026/09/20 14:32:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5760worker1 # [ 135.806453] nix-grpc-daemon[1352]: time=2026-09-20T14:32:49.376Z level=INFO msg="Uploading 1 narinfos"5761lb # [ 135.841869] niks3-server[1010]: 2026/09/20 14:32:49 INFO Signed narinfos id=44 count=15762lb # [ 135.847453] niks3-server[1010]: 2026/09/20 14:32:49 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5763lb # [ 135.856687] niks3-server[1010]: 2026/09/20 14:32:49 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5764lb # [ 135.861819] niks3-server[1010]: 2026/09/20 14:32:49 INFO Completed upload id=445765worker1 # [ 135.831213] nix-grpc-daemon[1352]: time=2026-09-20T14:32:49.402Z level=INFO msg="Upload complete. (334ms)"5766lb # [ 135.864969] niks3-server[1010]: 2026/09/20 14:32:49 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5767worker1 # [ 135.845191] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15768worker1 # [ 135.862031] nix-daemon[835]: reaped child process 1509, status = succeeded5769client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.42 seconds)5770client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5771client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5772worker2: must succeed: pkill -f 'read -t [9]0 x'5773worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5774client: waiting for success: ! systemctl is-active sw5775worker2 # [ 136.325756] nix-daemon[858]: reaped child process 1693, status = succeeded5776worker2 # [ 136.501277] nix-daemon[858]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)5777worker2 # [ 136.519122] nix-daemon[858]: reaped child process 1867, status = succeeded5778lb # [ 136.563415] niks3-server[1010]: 2026/09/20 14:32:50 INFO Received uploads request method=POST path=/api/pending_closures5779worker2 # [ 136.542043] nix-grpc-daemon[1637]: time=2026-09-20T14:32:50.117Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5780worker2 # [ 136.543604] nix-grpc-daemon[1637]: time=2026-09-20T14:32:50.117Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5781lb # [ 136.599644] niks3-server[1010]: 2026/09/20 14:32:50 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5782lb # [ 136.609951] niks3-server[1010]: 2026/09/20 14:32:50 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5783worker2 # [ 136.590350] nix-grpc-daemon[1637]: time=2026-09-20T14:32:50.165Z level=INFO msg="Uploading 1 narinfos"5784lb # [ 136.621928] niks3-server[1010]: 2026/09/20 14:32:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5785lb # [ 136.626924] niks3-server[1010]: 2026/09/20 14:32:50 INFO Signed narinfos id=45 count=15786lb # [ 136.631486] niks3-server[1010]: 2026/09/20 14:32:50 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5787lb # [ 136.639491] niks3-server[1010]: 2026/09/20 14:32:50 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5788lb # [ 136.647627] niks3-server[1010]: 2026/09/20 14:32:50 INFO Completed upload id=455789worker2 # [ 136.615358] nix-grpc-daemon[1637]: time=2026-09-20T14:32:50.191Z level=INFO msg="Upload complete. (323ms)"5790lb # [ 136.650497] niks3-server[1010]: 2026/09/20 14:32:50 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5791worker2 # [ 136.629122] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37722 duration_s=12 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15792worker2 # [ 136.642238] nix-daemon[858]: reaped child process 1691, status = succeeded5793client # [ 136.674067] systemd[1]: sw.service: Deactivated successfully.5794client # [ 136.678634] systemd[1]: sw.service: Consumed 181ms CPU time over 13.549s wall clock time, 39.2M memory peak, 6.8K incoming IP traffic, 8.2K outgoing IP traffic.5795client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds)5796client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5797client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)5798worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5799worker2 # [ 137.378006] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:50Z level=info event=signal_exit5800worker2 # [ 137.439426] systemd[1]: Starting scheduler-token.service...5801worker2 # [ 137.526190] systemd[1]: scheduler-token.service: Deactivated successfully.5802worker2 # [ 137.529616] systemd[1]: Finished scheduler-token.service.5803worker2 # [ 137.530936] systemd[1]: scheduler-token.service: Consumed 31ms CPU time over 87ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5804worker1 # [ 138.610774] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:536905805worker2 # [ 138.612884] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:52Z level=info event=scheduler_restarting5806worker1 # [ 138.615349] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:52Z level=info event=worker_gone addr=192.168.1.4:500515807worker2 # [ 138.631258] nix-grpc-daemon[1536]: ts=2026-09-20T14:32:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5808worker2 # [ 138.654080] nix-daemon[858]: reaped child process 1605, status = succeeded5809worker2 # [ 139.085422] nix-grpc-daemon[1536]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5810worker2 # [ 139.089845] nix-grpc-daemon[1536]: E0000 00:00:1789914772.661368 1579 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5811worker2 # [ 139.114493] nix-grpc-daemon[1536]: E0000 00:00:1789914772.690540 1579 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5812lb # [ 140.294042] envoy[901]: conn peer=[2001:db8:1::1]:41332 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9625 tx=3866 ms=61065813lb # [ 140.299138] envoy[901]: conn peer=[2001:db8:1::1]:41322 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2935 tx=1335 ms=132545814lb # [ 140.304483] envoy[901]: conn peer=[2001:db8:1::4]:36872 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2238 tx=425 ms=302315815worker2 # [ 140.425518] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5816worker2 # [ 140.428409] systemd[1]: nix-grpc-daemon.service: Consumed 885ms CPU time over 32.266s wall clock time, 33.3M memory peak, 20.9K incoming IP traffic, 28.4K outgoing IP traffic.5817worker2 # [ 140.449849] systemd[1]: Starting Nix worker-protocol over gRPC...5818worker2 # [ 140.642052] nix-grpc-daemon[1884]: ts=2026-09-20T14:32:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5819worker2 # [ 140.644857] nix-grpc-daemon[1884]: ts=2026-09-20T14:32:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5820worker2 # [ 140.648990] nix-grpc-daemon[1884]: ts=2026-09-20T14:32:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15821worker2 # [ 140.656861] nix-grpc-daemon[1884]: ts=2026-09-20T14:32:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5822worker2 # [ 140.658609] systemd[1]: Started Nix worker-protocol over gRPC.5823worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.33 seconds)5824lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5825lb: must succeed: curl -sf localhost:9901/clusters5826worker2 # [ 140.702013] nix-grpc-daemon[1884]: ts=2026-09-20T14:32:54Z level=info event=scheduler_connected addr=lb:500515827worker1 # [ 140.707863] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:536905828worker1 # [ 140.713918] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05829lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5830lb: must succeed: curl -sf localhost:9901/clusters5831lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5832lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5833lb: must succeed: curl -sf localhost:9901/clusters5834lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5835worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5836worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5837(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 18.93 seconds)5838subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5839client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15840worker2 # [ 142.255699] nix-grpc-daemon[1884]: ts=2026-09-20T14:32:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:546865841worker2 # [ 142.269961] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)5842worker2 # [ 142.291546] nix-grpc-daemon[1884]: ts=2026-09-20T14:32:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546865843worker2 # [ 142.307217] nix-grpc-daemon[1884]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 1/5)5844worker2 # [ 142.381939] nix-grpc-daemon[1884]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 2/5)5845worker2 # [ 142.530639] nix-grpc-daemon[1884]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 344 ms (attempt 3/5)5846worker2 # [ 142.882487] nix-grpc-daemon[1884]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 4/5)5847worker2 # [ 143.023539] nix-grpc-daemon[1884]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5848worker1 # [ 143.084676] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:56Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05849worker1 # [ 143.090039] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:56Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15850worker1 # [ 143.095905] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:56Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05851worker1 # [ 143.101287] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:56Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05852worker1 # [ 143.130683] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125853worker1 # [ 143.147470] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537125854worker1 # [ 143.163517] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537125855worker1 # [ 143.179598] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5856worker1 # [ 143.386662] nix-daemon[835]: accepted connection from pid 1538, user nix-grpc-daemon (trusted)5857worker1 # [ 143.404884] nix-daemon[835]: reaped child process 1541, status = succeeded5858worker1 # [ 143.576899] nix-daemon[835]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5859worker1 # [ 143.594565] nix-daemon[835]: reaped child process 1546, status = succeeded5860lb # [ 143.637754] niks3-server[1010]: 2026/09/20 14:32:57 INFO Received uploads request method=POST path=/api/pending_closures5861worker1 # [ 143.617262] nix-grpc-daemon[1352]: time=2026-09-20T14:32:57.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5862worker1 # [ 143.618729] nix-grpc-daemon[1352]: time=2026-09-20T14:32:57.187Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5863lb # [ 143.668439] niks3-server[1010]: 2026/09/20 14:32:57 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5864worker1 # [ 143.652362] nix-grpc-daemon[1352]: time=2026-09-20T14:32:57.222Z level=INFO msg="Uploading 1 narinfos"5865lb # [ 143.682810] niks3-server[1010]: 2026/09/20 14:32:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5866lb # [ 143.693469] niks3-server[1010]: 2026/09/20 14:32:57 INFO Signed narinfos id=46 count=15867lb # [ 143.698185] niks3-server[1010]: 2026/09/20 14:32:57 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5868worker1 # [ 143.675480] nix-grpc-daemon[1352]: time=2026-09-20T14:32:57.246Z level=INFO msg="Upload complete. (479ms)"5869lb # [ 143.706516] niks3-server[1010]: 2026/09/20 14:32:57 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5870lb # [ 143.711337] niks3-server[1010]: 2026/09/20 14:32:57 INFO Completed upload id=465871lb # [ 143.714523] niks3-server[1010]: 2026/09/20 14:32:57 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5872worker1 # [ 143.684673] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=5125873worker1 # [ 143.693639] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125874worker1 # [ 143.699551] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5875worker1 # [ 143.703657] nix-daemon[835]: reaped child process 1536, status = succeeded5876worker1 # [ 143.715790] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5877worker1 # [ 145.147861] nix-daemon[835]: reaped child process 1550, status = succeeded5878worker1 # [ 145.331504] nix-daemon[835]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5879worker1 # [ 145.346289] nix-daemon[835]: reaped child process 1568, status = succeeded5880lb # [ 145.393571] niks3-server[1010]: 2026/09/20 14:32:58 INFO Received uploads request method=POST path=/api/pending_closures5881worker1 # [ 145.372523] nix-grpc-daemon[1352]: time=2026-09-20T14:32:58.943Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5882worker1 # [ 145.377152] nix-grpc-daemon[1352]: time=2026-09-20T14:32:58.945Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5883worker1 # [ 145.411194] nix-grpc-daemon[1352]: time=2026-09-20T14:32:58.981Z level=INFO msg="Uploading 1 narinfos"5884lb # [ 145.434945] niks3-server[1010]: 2026/09/20 14:32:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5885lb # [ 145.451888] niks3-server[1010]: 2026/09/20 14:32:59 INFO Signed narinfos id=47 count=15886lb # [ 145.462504] niks3-server[1010]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5887lb # [ 145.470405] niks3-server[1010]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5888lb # [ 145.479153] niks3-server[1010]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5889worker1 # [ 145.453156] nix-grpc-daemon[1352]: time=2026-09-20T14:32:59.024Z level=INFO msg="Upload complete. (326ms)"5890lb # [ 145.484249] niks3-server[1010]: 2026/09/20 14:32:59 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5891lb # [ 145.489881] niks3-server[1010]: 2026/09/20 14:32:59 INFO Completed upload id=475892lb # [ 145.493195] niks3-server[1010]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5893worker1 # [ 145.470640] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=15894worker1 # [ 145.484493] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:59Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15895worker1 # [ 145.495017] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125896worker1 # [ 145.499515] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:59Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05897worker1 # [ 145.504247] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537125898worker1 # [ 145.508502] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537125899worker1 # [ 145.513109] nix-daemon[835]: reaped child process 1548, status = succeeded5900worker1 # [ 145.524277] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5901worker1 # [ 145.705153] nix-daemon[835]: accepted connection from pid 1574, user nix-grpc-daemon (trusted)5902worker1 # [ 145.722985] nix-daemon[835]: reaped child process 1577, status = succeeded5903worker1 # [ 145.896116] nix-daemon[835]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5904worker1 # [ 145.913040] nix-daemon[835]: reaped child process 1582, status = succeeded5905lb # [ 145.954773] niks3-server[1010]: 2026/09/20 14:32:59 INFO Received uploads request method=POST path=/api/pending_closures5906worker1 # [ 145.932725] nix-grpc-daemon[1352]: time=2026-09-20T14:32:59.503Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5907worker1 # [ 145.934317] nix-grpc-daemon[1352]: time=2026-09-20T14:32:59.503Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5908lb # [ 145.979606] niks3-server[1010]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5909worker1 # [ 145.962072] nix-grpc-daemon[1352]: time=2026-09-20T14:32:59.532Z level=INFO msg="Uploading 1 narinfos"5910lb # [ 145.994905] niks3-server[1010]: 2026/09/20 14:32:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5911lb # [ 146.006086] niks3-server[1010]: 2026/09/20 14:32:59 INFO Signed narinfos id=48 count=15912lb # [ 146.010530] niks3-server[1010]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5913worker1 # [ 145.989767] nix-grpc-daemon[1352]: time=2026-09-20T14:32:59.560Z level=INFO msg="Upload complete. (460ms)"5914lb # [ 146.020742] niks3-server[1010]: 2026/09/20 14:32:59 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5915lb # [ 146.025769] niks3-server[1010]: 2026/09/20 14:32:59 INFO Completed upload id=485916worker1 # [ 145.995048] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=5125917lb # [ 146.028966] niks3-server[1010]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5918worker1 # [ 146.006146] nix-grpc-daemon[1276]: ts=2026-09-20T14:32:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125919worker1 # [ 146.011242] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5920worker1 # [ 146.015545] nix-daemon[835]: reaped child process 1572, status = succeeded5921worker1 # [ 146.026891] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5922worker1 # [ 146.854056] nix-daemon[835]: reaped child process 1586, status = succeeded5923worker1 # [ 147.018890] nix-daemon[835]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)5924worker1 # [ 147.037061] nix-daemon[835]: reaped child process 1604, status = succeeded5925lb # [ 147.085086] niks3-server[1010]: 2026/09/20 14:33:00 INFO Received uploads request method=POST path=/api/pending_closures5926worker1 # [ 147.064390] nix-grpc-daemon[1352]: time=2026-09-20T14:33:00.634Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5927worker1 # [ 147.065910] nix-grpc-daemon[1352]: time=2026-09-20T14:33:00.634Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5928lb # [ 147.131266] niks3-server[1010]: 2026/09/20 14:33:00 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5929worker1 # [ 147.112070] nix-grpc-daemon[1352]: time=2026-09-20T14:33:00.682Z level=INFO msg="Uploading 1 narinfos"5930lb # [ 147.144181] niks3-server[1010]: 2026/09/20 14:33:00 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5931lb # [ 147.153147] niks3-server[1010]: 2026/09/20 14:33:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5932lb # [ 147.162943] niks3-server[1010]: 2026/09/20 14:33:00 INFO Signed narinfos id=49 count=15933worker1 # [ 147.136146] nix-grpc-daemon[1352]: time=2026-09-20T14:33:00.707Z level=INFO msg="Upload complete. (299ms)"5934lb # [ 147.166813] niks3-server[1010]: 2026/09/20 14:33:00 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5935lb # [ 147.171809] niks3-server[1010]: 2026/09/20 14:33:00 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5936lb # [ 147.176476] niks3-server[1010]: 2026/09/20 14:33:00 INFO Completed upload id=495937lb # [ 147.180411] niks3-server[1010]: 2026/09/20 14:33:00 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5938worker1 # [ 147.149475] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=15939worker1 # [ 147.166711] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:00Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15940worker1 # [ 147.171470] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125941worker1 # [ 147.176686] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:00Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05942worker1 # [ 147.181797] nix-daemon[835]: reaped child process 1584, status = succeeded5943worker1 # [ 147.183620] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537125944worker1 # [ 147.191896] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537125945worker1 # [ 147.201136] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5946worker1 # [ 147.392268] nix-daemon[835]: accepted connection from pid 1610, user nix-grpc-daemon (trusted)5947worker1 # [ 147.410568] nix-daemon[835]: reaped child process 1613, status = succeeded5948worker1 # [ 147.581552] nix-daemon[835]: accepted connection from pid 1615, user nix-grpc-daemon (trusted)5949worker1 # [ 147.597884] nix-daemon[835]: reaped child process 1618, status = succeeded5950lb # [ 147.639556] niks3-server[1010]: 2026/09/20 14:33:01 INFO Received uploads request method=POST path=/api/pending_closures5951worker1 # [ 147.618109] nix-grpc-daemon[1352]: time=2026-09-20T14:33:01.188Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5952worker1 # [ 147.619577] nix-grpc-daemon[1352]: time=2026-09-20T14:33:01.188Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"5953lb # [ 147.666808] niks3-server[1010]: 2026/09/20 14:33:01 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst5954lb # [ 147.679799] niks3-server[1010]: 2026/09/20 14:33:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5955worker1 # [ 147.656291] nix-grpc-daemon[1352]: time=2026-09-20T14:33:01.227Z level=INFO msg="Uploading 1 narinfos"5956lb # [ 147.692258] niks3-server[1010]: 2026/09/20 14:33:01 INFO Signed narinfos id=50 count=15957lb # [ 147.697633] niks3-server[1010]: 2026/09/20 14:33:01 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls5958lb # [ 147.704576] niks3-server[1010]: 2026/09/20 14:33:01 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5959worker1 # [ 147.678522] nix-grpc-daemon[1352]: time=2026-09-20T14:33:01.249Z level=INFO msg="Upload complete. (466ms)"5960lb # [ 147.710457] niks3-server[1010]: 2026/09/20 14:33:01 INFO Completed upload id=505961lb # [ 147.714338] niks3-server[1010]: 2026/09/20 14:33:01 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo5962worker1 # [ 147.691190] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=5125963worker1 # [ 147.695895] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125964worker1 # [ 147.699961] nix-daemon[835]: reaped child process 1608, status = succeeded5965worker1 # [ 147.706068] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5966worker1 # [ 147.716225] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5967worker1 # [ 148.942713] nix-daemon[835]: reaped child process 1622, status = succeeded5968worker1 # [ 149.106263] nix-daemon[835]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)5969worker1 # [ 149.124832] nix-daemon[835]: reaped child process 1640, status = succeeded5970lb # [ 149.169894] niks3-server[1010]: 2026/09/20 14:33:02 INFO Received uploads request method=POST path=/api/pending_closures5971worker1 # [ 149.158358] nix-grpc-daemon[1352]: time=2026-09-20T14:33:02.727Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5972worker1 # [ 149.160105] nix-grpc-daemon[1352]: time=2026-09-20T14:33:02.728Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"5973lb # [ 149.214190] niks3-server[1010]: 2026/09/20 14:33:02 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst5974worker1 # [ 149.196292] nix-grpc-daemon[1352]: time=2026-09-20T14:33:02.766Z level=INFO msg="Uploading 1 narinfos"5975lb # [ 149.227771] niks3-server[1010]: 2026/09/20 14:33:02 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv5976lb # [ 149.237506] niks3-server[1010]: 2026/09/20 14:33:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5977lb # [ 149.242991] niks3-server[1010]: 2026/09/20 14:33:02 INFO Signed narinfos id=51 count=15978lb # [ 149.248708] niks3-server[1010]: 2026/09/20 14:33:02 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls5979lb # [ 149.251916] niks3-server[1010]: 2026/09/20 14:33:02 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5980lb # [ 149.256258] niks3-server[1010]: 2026/09/20 14:33:02 INFO Completed upload id=515981lb # [ 149.258775] niks3-server[1010]: 2026/09/20 14:33:02 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo5982worker1 # [ 149.225180] nix-grpc-daemon[1352]: time=2026-09-20T14:33:02.796Z level=INFO msg="Upload complete. (298ms)"5983worker1 # [ 149.244671] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=15984worker1 # [ 149.256362] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:02Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=15985worker1 # [ 149.265324] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537125986worker1 # [ 149.270080] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:02Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=05987worker1 # [ 149.275181] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537125988worker1 # [ 149.279893] nix-daemon[835]: reaped child process 1620, status = succeeded5989worker1 # [ 149.281995] nix-grpc-daemon[1352]: time=2026-09-20T14:33:02.850Z level=INFO msg="All 1 paths already cached"5990worker1 # [ 149.285602] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537125991worker1 # [ 149.296734] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5992worker1 # [ 149.479450] nix-daemon[835]: accepted connection from pid 1646, user nix-grpc-daemon (trusted)5993worker1 # [ 149.494739] nix-daemon[835]: reaped child process 1649, status = succeeded5994worker1 # [ 149.671308] nix-daemon[835]: accepted connection from pid 1651, user nix-grpc-daemon (trusted)5995worker1 # [ 149.687603] nix-daemon[835]: reaped child process 1654, status = succeeded5996lb # [ 149.729296] niks3-server[1010]: 2026/09/20 14:33:03 INFO Received uploads request method=POST path=/api/pending_closures5997worker1 # [ 149.707971] nix-grpc-daemon[1352]: time=2026-09-20T14:33:03.278Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5998worker1 # [ 149.709649] nix-grpc-daemon[1352]: time=2026-09-20T14:33:03.278Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"5999lb # [ 149.755437] niks3-server[1010]: 2026/09/20 14:33:03 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6000worker1 # [ 149.738584] nix-grpc-daemon[1352]: time=2026-09-20T14:33:03.309Z level=INFO msg="Uploading 1 narinfos"6001lb # [ 149.771253] niks3-server[1010]: 2026/09/20 14:33:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6002lb # [ 149.781093] niks3-server[1010]: 2026/09/20 14:33:03 INFO Signed narinfos id=52 count=16003lb # [ 149.790030] niks3-server[1010]: 2026/09/20 14:33:03 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6004lb # [ 149.796928] niks3-server[1010]: 2026/09/20 14:33:03 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6005lb # [ 149.802019] niks3-server[1010]: 2026/09/20 14:33:03 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6006lb # [ 149.806463] niks3-server[1010]: 2026/09/20 14:33:03 INFO Completed upload id=526007worker1 # [ 149.775792] nix-grpc-daemon[1352]: time=2026-09-20T14:33:03.346Z level=INFO msg="Upload complete. (468ms)"6008worker1 # [ 149.783320] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=6246009worker1 # [ 149.796924] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126010worker1 # [ 149.802809] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6011worker1 # [ 149.807257] nix-daemon[835]: reaped child process 1644, status = succeeded6012worker1 # [ 149.818026] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6013worker1 # [ 150.463144] nix-daemon[835]: reaped child process 1658, status = succeeded6014worker1 # [ 150.642284] nix-daemon[835]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6015worker1 # [ 150.658988] nix-daemon[835]: reaped child process 1676, status = succeeded6016lb # [ 150.702350] niks3-server[1010]: 2026/09/20 14:33:04 INFO Received uploads request method=POST path=/api/pending_closures6017worker1 # [ 150.684393] nix-grpc-daemon[1352]: time=2026-09-20T14:33:04.252Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6018worker1 # [ 150.685926] nix-grpc-daemon[1352]: time=2026-09-20T14:33:04.252Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6019lb # [ 150.748237] niks3-server[1010]: 2026/09/20 14:33:04 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6020worker1 # [ 150.727638] nix-grpc-daemon[1352]: time=2026-09-20T14:33:04.298Z level=INFO msg="Uploading 1 narinfos"6021lb # [ 150.762118] niks3-server[1010]: 2026/09/20 14:33:04 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6022lb # [ 150.772211] niks3-server[1010]: 2026/09/20 14:33:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6023lb # [ 150.775861] niks3-server[1010]: 2026/09/20 14:33:04 INFO Signed narinfos id=53 count=16024lb # [ 150.781762] niks3-server[1010]: 2026/09/20 14:33:04 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6025lb # [ 150.785787] niks3-server[1010]: 2026/09/20 14:33:04 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6026worker1 # [ 150.755492] nix-grpc-daemon[1352]: time=2026-09-20T14:33:04.326Z level=INFO msg="Upload complete. (323ms)"6027lb # [ 150.789727] niks3-server[1010]: 2026/09/20 14:33:04 INFO Completed upload id=536028lb # [ 150.792369] niks3-server[1010]: 2026/09/20 14:33:04 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6029worker1 # [ 150.768845] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16030worker1 # [ 150.781287] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:04Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16031worker1 # [ 150.785527] nix-daemon[835]: reaped child process 1656, status = succeeded6032worker1 # [ 150.809235] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126033worker1 # [ 150.816673] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:04Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06034worker1 # [ 150.827243] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537126035worker1 # [ 150.840640] nix-grpc-daemon[1352]: time=2026-09-20T14:33:04.411Z level=INFO msg="All 1 paths already cached"6036worker1 # [ 150.848790] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537126037worker1 # [ 150.861267] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6038worker1 # [ 151.060169] nix-daemon[835]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6039worker1 # [ 151.077275] nix-daemon[835]: reaped child process 1685, status = succeeded6040worker1 # [ 151.255574] nix-daemon[835]: accepted connection from pid 1687, user nix-grpc-daemon (trusted)6041worker1 # [ 151.271930] nix-daemon[835]: reaped child process 1690, status = succeeded6042lb # [ 151.313787] niks3-server[1010]: 2026/09/20 14:33:04 INFO Received uploads request method=POST path=/api/pending_closures6043worker1 # [ 151.294075] nix-grpc-daemon[1352]: time=2026-09-20T14:33:04.864Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6044worker1 # [ 151.295536] nix-grpc-daemon[1352]: time=2026-09-20T14:33:04.864Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6045lb # [ 151.349465] niks3-server[1010]: 2026/09/20 14:33:04 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6046worker1 # [ 151.328604] nix-grpc-daemon[1352]: time=2026-09-20T14:33:04.899Z level=INFO msg="Uploading 1 narinfos"6047lb # [ 151.362173] niks3-server[1010]: 2026/09/20 14:33:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6048lb # [ 151.372281] niks3-server[1010]: 2026/09/20 14:33:04 INFO Signed narinfos id=54 count=16049lb # [ 151.380045] niks3-server[1010]: 2026/09/20 14:33:04 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6050lb # [ 151.383454] niks3-server[1010]: 2026/09/20 14:33:04 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6051worker1 # [ 151.357284] nix-grpc-daemon[1352]: time=2026-09-20T14:33:04.928Z level=INFO msg="Upload complete. (482ms)"6052lb # [ 151.388446] niks3-server[1010]: 2026/09/20 14:33:04 INFO Completed upload id=546053lb # [ 151.390865] niks3-server[1010]: 2026/09/20 14:33:04 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6054worker1 # [ 151.364119] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=6246055worker1 # [ 151.374710] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126056worker1 # [ 151.382901] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6057worker1 # [ 151.385312] nix-daemon[835]: reaped child process 1680, status = succeeded6058worker1 # [ 151.398236] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6059worker1 # [ 152.217688] nix-daemon[835]: reaped child process 1694, status = succeeded6060worker1 # [ 152.373629] nix-daemon[835]: accepted connection from pid 1709, user nix-grpc-daemon (trusted)6061worker1 # [ 152.390895] nix-daemon[835]: reaped child process 1712, status = succeeded6062lb # [ 152.433884] niks3-server[1010]: 2026/09/20 14:33:06 INFO Received uploads request method=POST path=/api/pending_closures6063worker1 # [ 152.413450] nix-grpc-daemon[1352]: time=2026-09-20T14:33:05.983Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6064worker1 # [ 152.414907] nix-grpc-daemon[1352]: time=2026-09-20T14:33:05.983Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6065lb # [ 152.466031] niks3-server[1010]: 2026/09/20 14:33:06 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6066lb # [ 152.482164] niks3-server[1010]: 2026/09/20 14:33:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6067worker1 # [ 152.456712] nix-grpc-daemon[1352]: time=2026-09-20T14:33:06.027Z level=INFO msg="Uploading 1 narinfos"6068lb # [ 152.492835] niks3-server[1010]: 2026/09/20 14:33:06 INFO Signed narinfos id=55 count=16069lb # [ 152.495167] niks3-server[1010]: 2026/09/20 14:33:06 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6070lb # [ 152.503283] niks3-server[1010]: 2026/09/20 14:33:06 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6071lb # [ 152.509780] niks3-server[1010]: 2026/09/20 14:33:06 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6072lb # [ 152.513960] niks3-server[1010]: 2026/09/20 14:33:06 INFO Completed upload id=556073worker1 # [ 152.483714] nix-grpc-daemon[1352]: time=2026-09-20T14:33:06.054Z level=INFO msg="Upload complete. (276ms)"6074lb # [ 152.516354] niks3-server[1010]: 2026/09/20 14:33:06 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6075worker1 # [ 152.498358] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16076worker1 # [ 152.511788] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:06Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16077worker1 # [ 152.516982] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126078worker1 # [ 152.520541] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:06Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06079worker1 # [ 152.524828] nix-daemon[835]: reaped child process 1692, status = succeeded6080worker1 # [ 152.528170] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537126081worker1 # [ 152.537805] nix-grpc-daemon[1352]: time=2026-09-20T14:33:06.108Z level=INFO msg="All 1 paths already cached"6082worker1 # [ 152.544153] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537126083worker1 # [ 152.554104] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6084worker1 # [ 152.735045] nix-daemon[835]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)6085worker1 # [ 152.751831] nix-daemon[835]: reaped child process 1721, status = succeeded6086worker1 # [ 152.926892] nix-daemon[835]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6087worker1 # [ 152.942897] nix-daemon[835]: reaped child process 1726, status = succeeded6088lb # [ 152.984223] niks3-server[1010]: 2026/09/20 14:33:06 INFO Received uploads request method=POST path=/api/pending_closures6089worker1 # [ 152.962963] nix-grpc-daemon[1352]: time=2026-09-20T14:33:06.533Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6090worker1 # [ 152.964589] nix-grpc-daemon[1352]: time=2026-09-20T14:33:06.533Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6091lb # [ 153.009130] niks3-server[1010]: 2026/09/20 14:33:06 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6092lb # [ 153.021297] niks3-server[1010]: 2026/09/20 14:33:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6093worker1 # [ 152.998177] nix-grpc-daemon[1352]: time=2026-09-20T14:33:06.568Z level=INFO msg="Uploading 1 narinfos"6094lb # [ 153.031461] niks3-server[1010]: 2026/09/20 14:33:06 INFO Signed narinfos id=56 count=16095lb # [ 153.038665] niks3-server[1010]: 2026/09/20 14:33:06 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6096lb # [ 153.046478] niks3-server[1010]: 2026/09/20 14:33:06 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6097lb # [ 153.053899] niks3-server[1010]: 2026/09/20 14:33:06 INFO Completed upload id=566098worker1 # [ 153.023553] nix-grpc-daemon[1352]: time=2026-09-20T14:33:06.594Z level=INFO msg="Upload complete. (458ms)"6099lb # [ 153.056689] niks3-server[1010]: 2026/09/20 14:33:06 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6100worker1 # [ 153.032972] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=6246101worker1 # [ 153.043244] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126102worker1 # [ 153.051617] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6103worker1 # [ 153.053872] nix-daemon[835]: reaped child process 1716, status = succeeded6104worker1 # [ 153.067334] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6105worker1 # [ 153.896955] nix-daemon[835]: reaped child process 1730, status = succeeded6106worker1 # [ 154.068291] nix-daemon[835]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6107worker1 # [ 154.087296] nix-daemon[835]: reaped child process 1748, status = succeeded6108lb # [ 154.130210] niks3-server[1010]: 2026/09/20 14:33:07 INFO Received uploads request method=POST path=/api/pending_closures6109worker1 # [ 154.110174] nix-grpc-daemon[1352]: time=2026-09-20T14:33:07.679Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6110worker1 # [ 154.111643] nix-grpc-daemon[1352]: time=2026-09-20T14:33:07.679Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6111lb # [ 154.165708] niks3-server[1010]: 2026/09/20 14:33:07 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6112lb # [ 154.178048] niks3-server[1010]: 2026/09/20 14:33:07 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6113worker1 # [ 154.159112] nix-grpc-daemon[1352]: time=2026-09-20T14:33:07.729Z level=INFO msg="Uploading 1 narinfos"6114lb # [ 154.190645] niks3-server[1010]: 2026/09/20 14:33:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6115lb # [ 154.199673] niks3-server[1010]: 2026/09/20 14:33:07 INFO Signed narinfos id=57 count=16116lb # [ 154.203964] niks3-server[1010]: 2026/09/20 14:33:07 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6117worker1 # [ 154.181712] nix-grpc-daemon[1352]: time=2026-09-20T14:33:07.752Z level=INFO msg="Upload complete. (305ms)"6118lb # [ 154.212255] niks3-server[1010]: 2026/09/20 14:33:07 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6119lb # [ 154.215993] niks3-server[1010]: 2026/09/20 14:33:07 INFO Completed upload id=576120lb # [ 154.221174] niks3-server[1010]: 2026/09/20 14:33:07 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6121worker1 # [ 154.197656] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16122worker1 # [ 154.212730] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:07Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16123worker1 # [ 154.216495] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126124worker1 # [ 154.219266] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:07Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06125worker1 # [ 154.223355] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537126126worker1 # [ 154.227210] nix-daemon[835]: reaped child process 1728, status = succeeded6127worker1 # [ 154.230265] nix-grpc-daemon[1352]: time=2026-09-20T14:33:07.801Z level=INFO msg="All 2 paths already cached"6128worker1 # [ 154.236932] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537126129worker1 # [ 154.245899] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6130worker1 # [ 154.425156] nix-daemon[835]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)6131worker1 # [ 154.443898] nix-daemon[835]: reaped child process 1757, status = succeeded6132worker1 # [ 154.613773] nix-daemon[835]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6133worker1 # [ 154.629485] nix-daemon[835]: reaped child process 1762, status = succeeded6134lb # [ 154.671393] niks3-server[1010]: 2026/09/20 14:33:08 INFO Received uploads request method=POST path=/api/pending_closures6135worker1 # [ 154.649511] nix-grpc-daemon[1352]: time=2026-09-20T14:33:08.220Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6136worker1 # [ 154.651001] nix-grpc-daemon[1352]: time=2026-09-20T14:33:08.220Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6137lb # [ 154.696072] niks3-server[1010]: 2026/09/20 14:33:08 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6138worker1 # [ 154.680451] nix-grpc-daemon[1352]: time=2026-09-20T14:33:08.251Z level=INFO msg="Uploading 1 narinfos"6139lb # [ 154.709687] niks3-server[1010]: 2026/09/20 14:33:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6140lb # [ 154.723254] niks3-server[1010]: 2026/09/20 14:33:08 INFO Signed narinfos id=58 count=16141lb # [ 154.732630] niks3-server[1010]: 2026/09/20 14:33:08 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6142worker1 # [ 154.708788] nix-grpc-daemon[1352]: time=2026-09-20T14:33:08.279Z level=INFO msg="Upload complete. (452ms)"6143lb # [ 154.739971] niks3-server[1010]: 2026/09/20 14:33:08 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6144lb # [ 154.745706] niks3-server[1010]: 2026/09/20 14:33:08 INFO Completed upload id=586145worker1 # [ 154.715031] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=7526146lb # [ 154.749216] niks3-server[1010]: 2026/09/20 14:33:08 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6147worker1 # [ 154.727177] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126148worker1 # [ 154.733101] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6149worker1 # [ 154.737255] nix-daemon[835]: reaped child process 1752, status = succeeded6150worker1 # [ 154.749772] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6151worker1 # [ 155.969775] nix-daemon[835]: reaped child process 1766, status = succeeded6152worker1 # [ 156.131837] nix-daemon[835]: accepted connection from pid 1781, user nix-grpc-daemon (trusted)6153worker1 # [ 156.150817] nix-daemon[835]: reaped child process 1784, status = succeeded6154lb # [ 156.194052] niks3-server[1010]: 2026/09/20 14:33:09 INFO Received uploads request method=POST path=/api/pending_closures6155worker1 # [ 156.174908] nix-grpc-daemon[1352]: time=2026-09-20T14:33:09.744Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6156worker1 # [ 156.176409] nix-grpc-daemon[1352]: time=2026-09-20T14:33:09.744Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6157lb # [ 156.224999] niks3-server[1010]: 2026/09/20 14:33:09 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6158lb # [ 156.238628] niks3-server[1010]: 2026/09/20 14:33:09 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6159lb # [ 156.251051] niks3-server[1010]: 2026/09/20 14:33:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6160worker1 # [ 156.226654] nix-grpc-daemon[1352]: time=2026-09-20T14:33:09.797Z level=INFO msg="Uploading 1 narinfos"6161lb # [ 156.261411] niks3-server[1010]: 2026/09/20 14:33:09 INFO Signed narinfos id=59 count=16162lb # [ 156.266949] niks3-server[1010]: 2026/09/20 14:33:09 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6163lb # [ 156.274294] niks3-server[1010]: 2026/09/20 14:33:09 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6164lb # [ 156.280332] niks3-server[1010]: 2026/09/20 14:33:09 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6165lb # [ 156.285253] niks3-server[1010]: 2026/09/20 14:33:09 INFO Completed upload id=596166worker1 # [ 156.255012] nix-grpc-daemon[1352]: time=2026-09-20T14:33:09.825Z level=INFO msg="Upload complete. (296ms)"6167worker1 # [ 156.269012] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16168worker1 # [ 156.280279] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:09Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16169worker1 # [ 156.286330] nix-daemon[835]: reaped child process 1764, status = succeeded6170worker1 # [ 156.309593] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126171worker1 # [ 156.323978] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537126172worker1 # [ 156.337278] nix-grpc-daemon[1352]: time=2026-09-20T14:33:09.908Z level=INFO msg="All 2 paths already cached"6173worker1 # [ 156.345758] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537126174worker1 # [ 156.359634] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6175worker1 # [ 156.569377] nix-daemon[835]: accepted connection from pid 1788, user nix-grpc-daemon (trusted)6176worker1 # [ 156.587858] nix-daemon[835]: reaped child process 1791, status = succeeded6177worker1 # [ 156.760118] nix-daemon[835]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6178worker1 # [ 156.776587] nix-daemon[835]: reaped child process 1796, status = succeeded6179lb # [ 156.818172] niks3-server[1010]: 2026/09/20 14:33:10 INFO Received uploads request method=POST path=/api/pending_closures6180worker1 # [ 156.796573] nix-grpc-daemon[1352]: time=2026-09-20T14:33:10.367Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6181worker1 # [ 156.798034] nix-grpc-daemon[1352]: time=2026-09-20T14:33:10.367Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6182lb # [ 156.845804] niks3-server[1010]: 2026/09/20 14:33:10 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6183worker1 # [ 156.826276] nix-grpc-daemon[1352]: time=2026-09-20T14:33:10.397Z level=INFO msg="Uploading 1 narinfos"6184lb # [ 156.860628] niks3-server[1010]: 2026/09/20 14:33:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6185lb # [ 156.871714] niks3-server[1010]: 2026/09/20 14:33:10 INFO Signed narinfos id=60 count=16186worker1 # [ 156.850321] nix-grpc-daemon[1352]: time=2026-09-20T14:33:10.421Z level=INFO msg="Upload complete. (474ms)"6187lb # [ 156.880762] niks3-server[1010]: 2026/09/20 14:33:10 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6188lb # [ 156.886079] niks3-server[1010]: 2026/09/20 14:33:10 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6189worker1 # [ 156.856239] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=7526190lb # [ 156.890954] niks3-server[1010]: 2026/09/20 14:33:10 INFO Completed upload id=606191lb # [ 156.893622] niks3-server[1010]: 2026/09/20 14:33:10 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6192worker1 # [ 156.868159] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126193worker1 # [ 156.871163] nix-daemon[835]: reaped child process 1786, status = succeeded6194worker1 # [ 156.875695] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6195worker1 # [ 156.889094] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6196worker1 # [ 158.129585] nix-daemon[835]: reaped child process 1800, status = succeeded6197worker1 # [ 158.306915] nix-daemon[835]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6198worker1 # [ 158.324274] nix-daemon[835]: reaped child process 1818, status = succeeded6199lb # [ 158.370538] niks3-server[1010]: 2026/09/20 14:33:11 INFO Received uploads request method=POST path=/api/pending_closures6200worker1 # [ 158.352472] nix-grpc-daemon[1352]: time=2026-09-20T14:33:11.922Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6201worker1 # [ 158.354384] nix-grpc-daemon[1352]: time=2026-09-20T14:33:11.922Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6202lb # [ 158.416580] niks3-server[1010]: 2026/09/20 14:33:12 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6203lb # [ 158.429428] niks3-server[1010]: 2026/09/20 14:33:12 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6204worker1 # [ 158.406703] nix-grpc-daemon[1352]: time=2026-09-20T14:33:11.977Z level=INFO msg="Uploading 1 narinfos"6205lb # [ 158.439228] niks3-server[1010]: 2026/09/20 14:33:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6206lb # [ 158.449891] niks3-server[1010]: 2026/09/20 14:33:12 INFO Signed narinfos id=61 count=16207lb # [ 158.454576] niks3-server[1010]: 2026/09/20 14:33:12 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6208lb # [ 158.464518] niks3-server[1010]: 2026/09/20 14:33:12 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6209lb # [ 158.469794] niks3-server[1010]: 2026/09/20 14:33:12 INFO Completed upload id=616210worker1 # [ 158.438936] nix-grpc-daemon[1352]: time=2026-09-20T14:33:12.009Z level=INFO msg="Upload complete. (337ms)"6211lb # [ 158.472773] niks3-server[1010]: 2026/09/20 14:33:12 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6212worker1 # [ 158.453106] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16213worker1 # [ 158.470450] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:12Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16214worker1 # [ 158.477675] nix-daemon[835]: reaped child process 1798, status = succeeded6215worker1 # [ 158.494187] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126216worker1 # [ 158.508457] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537126217worker1 # [ 158.523000] nix-grpc-daemon[1352]: time=2026-09-20T14:33:12.093Z level=INFO msg="All 2 paths already cached"6218worker1 # [ 158.531776] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537126219worker1 # [ 158.545095] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6220worker1 # [ 158.744511] nix-daemon[835]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)6221worker1 # [ 158.761655] nix-daemon[835]: reaped child process 1825, status = succeeded6222worker1 # [ 158.934819] nix-daemon[835]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)6223worker1 # [ 158.950727] nix-daemon[835]: reaped child process 1830, status = succeeded6224lb # [ 158.992774] niks3-server[1010]: 2026/09/20 14:33:12 INFO Received uploads request method=POST path=/api/pending_closures6225worker1 # [ 158.972070] nix-grpc-daemon[1352]: time=2026-09-20T14:33:12.543Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6226worker1 # [ 158.973987] nix-grpc-daemon[1352]: time=2026-09-20T14:33:12.544Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6227lb # [ 159.023934] niks3-server[1010]: 2026/09/20 14:33:12 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6228lb # [ 159.035873] niks3-server[1010]: 2026/09/20 14:33:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6229worker1 # [ 159.012775] nix-grpc-daemon[1352]: time=2026-09-20T14:33:12.583Z level=INFO msg="Uploading 1 narinfos"6230lb # [ 159.044721] niks3-server[1010]: 2026/09/20 14:33:12 INFO Signed narinfos id=62 count=16231lb # [ 159.047969] niks3-server[1010]: 2026/09/20 14:33:12 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6232lb # [ 159.063263] niks3-server[1010]: 2026/09/20 14:33:12 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6233worker1 # [ 159.039326] nix-grpc-daemon[1352]: time=2026-09-20T14:33:12.610Z level=INFO msg="Upload complete. (479ms)"6234lb # [ 159.069622] niks3-server[1010]: 2026/09/20 14:33:12 INFO Completed upload id=626235lb # [ 159.074472] niks3-server[1010]: 2026/09/20 14:33:12 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6236worker1 # [ 159.048295] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=0 paths=1 nar_bytes_in=7526237worker1 # [ 159.061728] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537126238worker1 # [ 159.071413] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6239worker1 # [ 159.073879] nix-daemon[835]: reaped child process 1820, status = succeeded6240worker1 # [ 159.088330] nix-daemon[835]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)6241worker1 # [ 160.333958] nix-daemon[835]: reaped child process 1834, status = succeeded6242worker1 # [ 160.512839] nix-daemon[835]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6243worker1 # [ 160.531682] nix-daemon[835]: reaped child process 1852, status = succeeded6244lb # [ 160.574875] niks3-server[1010]: 2026/09/20 14:33:14 INFO Received uploads request method=POST path=/api/pending_closures6245worker1 # [ 160.554801] nix-grpc-daemon[1352]: time=2026-09-20T14:33:14.124Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6246worker1 # [ 160.556607] nix-grpc-daemon[1352]: time=2026-09-20T14:33:14.124Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6247lb # [ 160.611065] niks3-server[1010]: 2026/09/20 14:33:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6248worker1 # [ 160.592615] nix-grpc-daemon[1352]: time=2026-09-20T14:33:14.163Z level=INFO msg="Uploading 1 narinfos"6249lb # [ 160.627665] niks3-server[1010]: 2026/09/20 14:33:14 INFO Signed narinfos id=63 count=16250lb # [ 160.634122] niks3-server[1010]: 2026/09/20 14:33:14 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6251lb # [ 160.642153] niks3-server[1010]: 2026/09/20 14:33:14 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6252lb # [ 160.650335] niks3-server[1010]: 2026/09/20 14:33:14 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6253lb # [ 160.655159] niks3-server[1010]: 2026/09/20 14:33:14 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6254lb # [ 160.658470] niks3-server[1010]: 2026/09/20 14:33:14 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6255lb # [ 160.662153] niks3-server[1010]: 2026/09/20 14:33:14 INFO Completed upload id=636256worker1 # [ 160.631602] nix-grpc-daemon[1352]: time=2026-09-20T14:33:14.202Z level=INFO msg="Upload complete. (332ms)"6257worker1 # [ 160.645120] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53712 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16258worker1 # [ 160.661785] nix-daemon[835]: reaped child process 1832, status = succeeded6259client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.72 seconds)6260worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6261worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6262client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6263client # [ 160.857805] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6264client # Running as unit: gpu.service; invocation ID: ba34550e2c7947fd8443e901bc02c9006265client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6266lb: must succeed: curl -sf localhost:9901/clusters6267lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6268lb: must succeed: curl -sf localhost:9901/clusters6269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6270worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6271worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6272worker2 # [ 161.101312] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:546866273client # [ 161.109865] nix[1339]: don't know how to build these paths:6274client # [ 161.110876] nix[1339]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6275worker2 # [ 161.113699] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546866276client # [ 161.138208] nix[1339]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6277worker1 # [ 161.138312] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:14Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06278lb: must succeed: curl -sf localhost:9901/clusters6279lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6280lb: must succeed: curl -sf localhost:9901/clusters6281lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6282worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6283worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6284client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6285lb # [ 170.294330] envoy[901]: conn peer=[2001:db8:1::1]:33298 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27092 tx=10726 ms=184076286client # [ 191.145667] nix[1339]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6287client # [ 191.157602] nix[1339]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6288client # [ 191.197583] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6289client # [ 191.205394] systemd[1]: gpu.service: Failed with result 'exit-code'.6290client # [ 191.209212] systemd[1]: gpu.service: Consumed 171ms CPU time over 30.345s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6291client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.25 seconds)6292client: must succeed: journalctl -u gpu -o cat6293client: (finished: must succeed: journalctl -u gpu -o cat, in 0.04 seconds)6294lb: must succeed: curl -sf localhost:9901/clusters6295lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6296lb: must succeed: curl -sf localhost:9901/clusters6297lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6298worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6299worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6300(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.81 seconds)6301subtest: low disk drains a worker and builds go to the other6302worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6303worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.09 seconds)6304worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6305worker1 # [ 192.941279] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:46Z level=info event=unhealthy reason=min_free6306worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6307client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26308client # don't know how to build these paths:6309client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6310worker1 # [ 194.273414] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537126311worker1 # [ 194.281362] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537126312worker1 # [ 194.301358] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:47Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06313worker1 # [ 194.306616] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:47Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16314worker2 # [ 194.346055] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546866315worker2 # [ 194.361715] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546866316client # copying 1 paths to 'lb:50051'...6317client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6318worker2 # [ 194.377373] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546866319worker2 # [ 194.392880] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6320worker2 # [ 194.420204] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:47Z level=info event=niks3_push_started pid=19406321worker2 # [ 194.630773] nix-grpc-daemon[1940]: time=2026-09-20T14:33:48.206Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""6322worker2 # [ 194.800691] nix-daemon[858]: accepted connection from pid 1953, user nix-grpc-daemon (trusted)6323worker2 # [ 194.817483] nix-daemon[858]: reaped child process 1956, status = succeeded6324worker2 # [ 194.988253] nix-daemon[858]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6325worker2 # [ 195.004627] nix-daemon[858]: reaped child process 1961, status = succeeded6326lb # [ 195.051321] niks3-server[1010]: 2026/09/20 14:33:48 INFO Received uploads request method=POST path=/api/pending_closures6327worker2 # [ 195.032066] nix-grpc-daemon[1940]: time=2026-09-20T14:33:48.607Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6328worker2 # [ 195.033556] nix-grpc-daemon[1940]: time=2026-09-20T14:33:48.607Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6329worker2 # [ 195.076044] nix-grpc-daemon[1940]: time=2026-09-20T14:33:48.651Z level=INFO msg="Uploading 1 narinfos"6330lb # [ 195.098516] niks3-server[1010]: 2026/09/20 14:33:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6331lb # [ 195.112997] niks3-server[1010]: 2026/09/20 14:33:48 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6332lb # [ 195.122623] niks3-server[1010]: 2026/09/20 14:33:48 INFO Signed narinfos id=64 count=16333lb # [ 195.129389] niks3-server[1010]: 2026/09/20 14:33:48 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6334worker2 # [ 195.101581] nix-grpc-daemon[1940]: time=2026-09-20T14:33:48.677Z level=INFO msg="Upload complete. (467ms)"6335lb # [ 195.134250] niks3-server[1010]: 2026/09/20 14:33:48 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6336lb # [ 195.138645] niks3-server[1010]: 2026/09/20 14:33:48 INFO Completed upload id=646337worker2 # [ 195.108141] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54686 duration_s=0 paths=1 nar_bytes_in=5126338lb # [ 195.141808] niks3-server[1010]: 2026/09/20 14:33:48 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6339worker2 # [ 195.117845] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546866340worker2 # [ 195.128460] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6341worker2 # [ 195.133176] nix-daemon[858]: reaped child process 1938, status = succeeded6342client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6343client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6344worker2 # [ 195.146089] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6345client # farm-a> LOG-farm-a-drain6346worker2 # [ 196.592194] nix-daemon[858]: reaped child process 1965, status = succeeded6347worker2 # [ 196.771647] nix-daemon[858]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)6348worker2 # [ 196.789254] nix-daemon[858]: reaped child process 1983, status = succeeded6349lb # [ 196.838064] niks3-server[1010]: 2026/09/20 14:33:50 INFO Received uploads request method=POST path=/api/pending_closures6350worker2 # [ 196.816442] nix-grpc-daemon[1940]: time=2026-09-20T14:33:50.390Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6351worker2 # [ 196.818001] nix-grpc-daemon[1940]: time=2026-09-20T14:33:50.390Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6352lb # [ 196.881083] niks3-server[1010]: 2026/09/20 14:33:50 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6353lb # [ 196.894081] niks3-server[1010]: 2026/09/20 14:33:50 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6354lb # [ 196.902113] niks3-server[1010]: 2026/09/20 14:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6355lb # [ 196.908139] niks3-server[1010]: 2026/09/20 14:33:50 INFO Signed narinfos id=65 count=16356worker2 # [ 196.876194] nix-grpc-daemon[1940]: time=2026-09-20T14:33:50.451Z level=INFO msg="Uploading 1 narinfos"6357lb # [ 196.911441] niks3-server[1010]: 2026/09/20 14:33:50 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6358lb # [ 196.923856] niks3-server[1010]: 2026/09/20 14:33:50 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6359lb # [ 196.931564] niks3-server[1010]: 2026/09/20 14:33:50 INFO Completed upload id=656360worker2 # [ 196.899687] nix-grpc-daemon[1940]: time=2026-09-20T14:33:50.475Z level=INFO msg="Upload complete. (342ms)"6361lb # [ 196.934718] niks3-server[1010]: 2026/09/20 14:33:50 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6362worker2 # [ 196.913199] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54686 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16363worker1 # [ 196.919432] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:50Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06364worker1 # [ 196.925404] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:50Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16365worker2 # [ 196.930377] nix-daemon[858]: reaped child process 1963, status = succeeded6366worker2 # [ 196.964706] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546866367worker2 # [ 196.985628] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546866368worker2 # [ 197.006115] nix-grpc-daemon[1940]: time=2026-09-20T14:33:50.582Z level=INFO msg="All 1 paths already cached"6369client # copying 1 paths to 'lb:50051'...6370client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6371worker2 # [ 197.016564] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546866372worker2 # [ 197.030663] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6373worker2 # [ 197.238124] nix-daemon[858]: accepted connection from pid 1987, user nix-grpc-daemon (trusted)6374worker2 # [ 197.258986] nix-daemon[858]: reaped child process 1990, status = succeeded6375worker2 # [ 197.436796] nix-daemon[858]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6376worker2 # [ 197.454177] nix-daemon[858]: reaped child process 1995, status = succeeded6377lb # [ 197.498244] niks3-server[1010]: 2026/09/20 14:33:51 INFO Received uploads request method=POST path=/api/pending_closures6378worker2 # [ 197.476136] nix-grpc-daemon[1940]: time=2026-09-20T14:33:51.051Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6379worker2 # [ 197.477643] nix-grpc-daemon[1940]: time=2026-09-20T14:33:51.051Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6380lb # [ 197.529575] niks3-server[1010]: 2026/09/20 14:33:51 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6381worker2 # [ 197.506467] nix-grpc-daemon[1940]: time=2026-09-20T14:33:51.082Z level=INFO msg="Uploading 1 narinfos"6382lb # [ 197.544342] niks3-server[1010]: 2026/09/20 14:33:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6383lb # [ 197.554541] niks3-server[1010]: 2026/09/20 14:33:51 INFO Signed narinfos id=66 count=16384lb # [ 197.561739] niks3-server[1010]: 2026/09/20 14:33:51 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6385worker2 # [ 197.534139] nix-grpc-daemon[1940]: time=2026-09-20T14:33:51.110Z level=INFO msg="Upload complete. (486ms)"6386lb # [ 197.567228] niks3-server[1010]: 2026/09/20 14:33:51 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6387lb # [ 197.572247] niks3-server[1010]: 2026/09/20 14:33:51 INFO Completed upload id=666388worker2 # [ 197.544265] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54686 duration_s=0 paths=1 nar_bytes_in=6326389lb # [ 197.576770] niks3-server[1010]: 2026/09/20 14:33:51 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6390worker2 # [ 197.548565] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546866391worker2 # [ 197.560878] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6392worker2 # [ 197.563537] nix-daemon[858]: reaped child process 1985, status = succeeded6393client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6394worker2 # [ 197.573830] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6395client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6396client # farm-b> LOG-farm-b-drain6397worker2 # [ 199.015697] nix-daemon[858]: reaped child process 1999, status = succeeded6398worker2 # [ 199.197032] nix-daemon[858]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6399worker2 # [ 199.213148] nix-daemon[858]: reaped child process 2017, status = succeeded6400lb # [ 199.259442] niks3-server[1010]: 2026/09/20 14:33:52 INFO Received uploads request method=POST path=/api/pending_closures6401worker2 # [ 199.238394] nix-grpc-daemon[1940]: time=2026-09-20T14:33:52.813Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6402worker2 # [ 199.239876] nix-grpc-daemon[1940]: time=2026-09-20T14:33:52.813Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6403lb # [ 199.287648] niks3-server[1010]: 2026/09/20 14:33:52 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6404lb # [ 199.301264] niks3-server[1010]: 2026/09/20 14:33:52 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6405lb # [ 199.314756] niks3-server[1010]: 2026/09/20 14:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6406worker2 # [ 199.287242] nix-grpc-daemon[1940]: time=2026-09-20T14:33:52.863Z level=INFO msg="Uploading 1 narinfos"6407lb # [ 199.324209] niks3-server[1010]: 2026/09/20 14:33:52 INFO Signed narinfos id=67 count=16408lb # [ 199.326782] niks3-server[1010]: 2026/09/20 14:33:52 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6409lb # [ 199.336397] niks3-server[1010]: 2026/09/20 14:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6410lb # [ 199.343852] niks3-server[1010]: 2026/09/20 14:33:52 INFO Completed upload id=676411worker2 # [ 199.311193] nix-grpc-daemon[1940]: time=2026-09-20T14:33:52.887Z level=INFO msg="Upload complete. (324ms)"6412lb # [ 199.347010] niks3-server[1010]: 2026/09/20 14:33:52 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6413worker2 # [ 199.324566] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54686 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16414worker1 # [ 199.331665] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:52Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06415worker1 # [ 199.337755] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:52Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16416worker2 # [ 199.340963] nix-daemon[858]: reaped child process 1997, status = succeeded6417worker2 # [ 199.376735] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546866418worker2 # [ 199.393503] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546866419worker2 # [ 199.407532] nix-grpc-daemon[1940]: time=2026-09-20T14:33:52.983Z level=INFO msg="All 2 paths already cached"6420client # copying 1 paths to 'lb:50051'...6421client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6422worker2 # [ 199.418047] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546866423worker2 # [ 199.431151] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6424worker2 # [ 199.625062] nix-daemon[858]: accepted connection from pid 2021, user nix-grpc-daemon (trusted)6425worker2 # [ 199.643912] nix-daemon[858]: reaped child process 2024, status = succeeded6426worker2 # [ 199.815677] nix-daemon[858]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)6427worker2 # [ 199.830545] nix-daemon[858]: reaped child process 2029, status = succeeded6428lb # [ 199.878305] niks3-server[1010]: 2026/09/20 14:33:53 INFO Received uploads request method=POST path=/api/pending_closures6429worker2 # [ 199.855173] nix-grpc-daemon[1940]: time=2026-09-20T14:33:53.431Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6430worker2 # [ 199.856769] nix-grpc-daemon[1940]: time=2026-09-20T14:33:53.431Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6431lb # [ 199.906986] niks3-server[1010]: 2026/09/20 14:33:53 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6432worker2 # [ 199.884880] nix-grpc-daemon[1940]: time=2026-09-20T14:33:53.460Z level=INFO msg="Uploading 1 narinfos"6433lb # [ 199.920245] niks3-server[1010]: 2026/09/20 14:33:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6434lb # [ 199.931327] niks3-server[1010]: 2026/09/20 14:33:53 INFO Signed narinfos id=68 count=16435lb # [ 199.939359] niks3-server[1010]: 2026/09/20 14:33:53 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6436worker2 # [ 199.911746] nix-grpc-daemon[1940]: time=2026-09-20T14:33:53.487Z level=INFO msg="Upload complete. (463ms)"6437lb # [ 199.944504] niks3-server[1010]: 2026/09/20 14:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6438lb # [ 199.949460] niks3-server[1010]: 2026/09/20 14:33:53 INFO Completed upload id=686439lb # [ 199.952241] niks3-server[1010]: 2026/09/20 14:33:53 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6440worker2 # [ 199.921090] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54686 duration_s=0 paths=1 nar_bytes_in=7606441worker2 # [ 199.929947] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546866442worker2 # [ 199.936135] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6443worker2 # [ 199.939993] nix-daemon[858]: reaped child process 2019, status = succeeded6444client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6445worker2 # [ 199.951777] nix-daemon[858]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6446client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6447lb # [ 200.296241] envoy[901]: conn peer=[2001:db8:1::1]:37168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300656448client # farm-top> LOG-farm-top-drain6449worker2 # [ 201.579660] nix-daemon[858]: reaped child process 2033, status = succeeded6450worker2 # [ 201.739956] nix-daemon[858]: accepted connection from pid 2048, user nix-grpc-daemon (trusted)6451worker2 # [ 201.756109] nix-daemon[858]: reaped child process 2051, status = succeeded6452lb # [ 201.804541] niks3-server[1010]: 2026/09/20 14:33:55 INFO Received uploads request method=POST path=/api/pending_closures6453worker2 # [ 201.784349] nix-grpc-daemon[1940]: time=2026-09-20T14:33:55.358Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6454worker2 # [ 201.786024] nix-grpc-daemon[1940]: time=2026-09-20T14:33:55.358Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6455lb # [ 201.833421] niks3-server[1010]: 2026/09/20 14:33:55 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6456lb # [ 201.849412] niks3-server[1010]: 2026/09/20 14:33:55 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6457lb # [ 201.860926] niks3-server[1010]: 2026/09/20 14:33:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6458worker2 # [ 201.833127] nix-grpc-daemon[1940]: time=2026-09-20T14:33:55.409Z level=INFO msg="Uploading 1 narinfos"6459lb # [ 201.867419] niks3-server[1010]: 2026/09/20 14:33:55 INFO Signed narinfos id=69 count=16460lb # [ 201.873741] niks3-server[1010]: 2026/09/20 14:33:55 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6461lb # [ 201.883008] niks3-server[1010]: 2026/09/20 14:33:55 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6462lb # [ 201.888822] niks3-server[1010]: 2026/09/20 14:33:55 INFO Completed upload id=696463worker2 # [ 201.856389] nix-grpc-daemon[1940]: time=2026-09-20T14:33:55.432Z level=INFO msg="Upload complete. (295ms)"6464lb # [ 201.891944] niks3-server[1010]: 2026/09/20 14:33:55 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6465worker2 # [ 201.868846] nix-grpc-daemon[1884]: ts=2026-09-20T14:33:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54686 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16466worker2 # [ 201.880095] nix-daemon[858]: reaped child process 2031, status = succeeded6467client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 7.91 seconds)6468client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6469client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6470worker1: must succeed: rm /nix/.rw-store/fill6471worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6472worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6473worker1 # [ 202.948604] nix-grpc-daemon[1276]: ts=2026-09-20T14:33:56Z level=info event=healthy reason=min_free6474worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6475(finished: subtest: low disk drains a worker and builds go to the other, in 11.73 seconds)6476(finished: run the VM test script, in 204.47 seconds)6477test script finished in 204.76s6478cleanup6479kill QemuMachine (pid 49)6480client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6481kill QemuMachine (pid 50)6482lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6483kill QemuMachine (pid 51)6484worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6485kill QemuMachine (pid 52)6486worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6487(finished: cleanup, in 1.09 seconds)6488additionally exposed symbols:6489 client, lb, worker1, worker2,6490 vlan1,6491 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh