vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #467
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13client # Disk image does not exist, creating the virtualisation disk image...14worker1 # Disk image does not exist, creating the virtualisation disk image...15client: QEMU running (pid 49)16lb # Disk image does not exist, creating the virtualisation disk image...17client # Formatting '/build/vm-state-client/tmp.2TAJQAmMXA', fmt=raw size=107374182418worker2 # Formatting '/build/vm-state-worker2/tmp.LnX4PrHZI6', fmt=raw size=107374182419worker2: QEMU running (pid 51)20lb # Formatting '/build/vm-state-lb/tmp.BUAcV3XL43', fmt=raw size=107374182421client # mke2fs 1.47.4 (6-Mar-2025)22worker2 # mke2fs 1.47.4 (6-Mar-2025)23lb: QEMU running (pid 50)24worker1 # Formatting '/build/vm-state-worker1/tmp.zpcdo71ni4', fmt=raw size=107374182425worker1: QEMU running (pid 52)26lb # mke2fs 1.47.4 (6-Mar-2025)27worker1 # mke2fs 1.47.4 (6-Mar-2025)28client # Discarding device blocks: 0/262144 done29worker2 # Discarding device blocks: 0/262144 done30(finished: start all VMs, in 0.47 seconds)31client # Creating filesystem with 262144 4k blocks and 65536 inodes32worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes33client: waiting for unit network-addresses-eth1.service34client # Filesystem UUID: b33be005-ca40-4a05-a1fe-ff0acc2ff65a35worker2 # Filesystem UUID: fb96e099-efc4-4c5b-9589-f5efd947905336client: waiting for the VM to finish booting37client # Superblock backups stored on blocks:38worker2 # Superblock backups stored on blocks:39worker1 # Discarding device blocks: 0/262144 done40lb # Discarding device blocks: 0/262144 done41client # 32768, 98304, 163840, 22937642worker2 # 32768, 98304, 163840, 22937643worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes44lb # Creating filesystem with 262144 4k blocks and 65536 inodes45client # 46worker2 # 47worker1 # Filesystem UUID: 8c8dabfc-891b-4801-962e-21c7a7e4239d48lb # Filesystem UUID: c2eb829f-f19e-4e6f-ae63-b26aa4c7dcce49client # Allocating group tables: 0/8 done50worker2 # Allocating group tables: 0/8 done51worker1 # Superblock backups stored on blocks:52lb # Superblock backups stored on blocks:53client # Writing inode tables: 0/8 done54worker2 # Writing inode tables: 0/8 done55worker1 # 32768, 98304, 163840, 22937656lb # 32768, 98304, 163840, 22937657client # Creating journal (8192 blocks): done58worker2 # Creating journal (8192 blocks): done59worker1 # 60lb # 61client # Writing superblocks and filesystem accounting information: 0/8 done62worker2 # Writing superblocks and filesystem accounting information: 0/8 done63worker1 # Allocating group tables: 0/8 done64lb # Allocating group tables: 0/8 done65client # 66worker2 # 67worker1 # Writing inode tables: 0/8 done68lb # Writing inode tables: 0/8 done69client # Virtualisation disk image created.70worker2 # Virtualisation disk image created.71worker1 # Creating journal (8192 blocks): done72lb # Creating journal (8192 blocks): done73worker1 # Writing superblocks and filesystem accounting information: 0/8 done74lb # Writing superblocks and filesystem accounting information: 0/8 done75worker1 # 76lb # 77worker1 # Virtualisation disk image created.78lb # Virtualisation disk image created.79worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker2 # [ 0.000000] KASLR enabled82worker2 # [ 0.000000] random: crng init done83worker2 # [ 0.000000] Machine model: linux,dummy-virt84worker2 # [ 0.000000] efi: UEFI not found.85worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88worker2 # [ 0.000000] Zone ranges:89worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90worker2 # [ 0.000000] DMA32 empty91worker2 # [ 0.000000] Normal empty92worker2 # [ 0.000000] Device empty93worker2 # [ 0.000000] Movable zone start for each node94worker2 # [ 0.000000] Early memory node ranges95worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098worker2 # [ 0.000000] psci: probing for conduit method from DT.99worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.100worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101worker2 # [ 0.000000] psci: Trusted OS migration not required102worker2 # [ 0.000000] psci: SMC Calling Convention v1.1103worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105worker2 # [ 0.000000] Detected PIPT I-cache on CPU0106worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface108worker2 # [ 0.000000] CPU features: detected: Spectre-v4109worker2 # [ 0.000000] CPU features: detected: Spectre-BHB110worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112worker2 # [ 0.000000] alternatives: applying boot alternatives113worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7125nj8p93qa3r29n9673xywybzmdlgw-nixos-system-worker2-test/init regInfo=/nix/store/2xzyszdr99wfxpqj1a8llwgvj56qhxgy-closure-info/registration console=ttyAMA0,115200n8 console=tty0114worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/2xzyszdr99wfxpqj1a8llwgvj56qhxgy-closure-info/registration", will be passed to user space.115worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119worker2 # [ 0.000000] software IO TLB: area num 1.120worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)122worker2 # [ 0.000000] Fallback order for Node 0: 0123worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216124worker2 # [ 0.000000] Policy zone: DMA125worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off126worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1127worker2 # [ 0.000000] allocated 3145728 bytes of page_ext128worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages129client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]130worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups131worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.132client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026133worker2 # [ 0.000000] rcu: RCU event tracing is enabled.134client # [ 0.000000] KASLR enabled135client # [ 0.000000] random: crng init done136worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.137client # [ 0.000000] Machine model: linux,dummy-virt138worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.139client # [ 0.000000] efi: UEFI not found.140worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.141client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT142worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.143client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]144worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.145client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]146lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]147worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1148client # [ 0.000000] Zone ranges:149client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]150worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.151client # [ 0.000000] DMA32 empty152lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026153client # [ 0.000000] Normal empty154lb # [ 0.000000] KASLR enabled155client # [ 0.000000] Device empty156worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.157lb # [ 0.000000] random: crng init done158client # [ 0.000000] Movable zone start for each node159lb # [ 0.000000] Machine model: linux,dummy-virt160client # [ 0.000000] Early memory node ranges161worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]162lb # [ 0.000000] efi: UEFI not found.163worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.164client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]165worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0166lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT167worker2 # [ 0.000000] GICv3: 256 SPIs implemented168client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]169worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026170lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]171worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented172worker1 # [ 0.000000] KASLR enabled173client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000174worker1 # [ 0.000000] random: crng init done175worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq176lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]177client # [ 0.000000] psci: probing for conduit method from DT.178lb # [ 0.000000] Zone ranges:179worker1 # [ 0.000000] Machine model: linux,dummy-virt180worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI181worker1 # [ 0.000000] efi: UEFI not found.182client # [ 0.000000] psci: PSCIv1.3 detected in firmware.183worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0184lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]185lb # [ 0.000000] DMA32 empty186client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs187worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT188lb # [ 0.000000] Normal empty189client # [ 0.000000] psci: Trusted OS migration not required190worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000191lb # [ 0.000000] Device empty192worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]193worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]194client # [ 0.000000] psci: SMC Calling Convention v1.1195lb # [ 0.000000] Movable zone start for each node196lb # [ 0.000000] Early memory node ranges197worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]198worker1 # [ 0.000000] Zone ranges:199client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)200worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)201lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]202worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]203client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296204worker1 # [ 0.000000] DMA32 empty205lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]206worker1 # [ 0.000000] Normal empty207worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)208client # [ 0.000000] Detected PIPT I-cache on CPU0209worker1 # [ 0.000000] Device empty210lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000211worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000212worker1 # [ 0.000000] Movable zone start for each node213lb # [ 0.000000] psci: probing for conduit method from DT.214client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)215worker1 # [ 0.000000] Early memory node ranges216worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000217lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.218client # [ 0.000000] CPU features: detected: GICv3 CPU interface219worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]220lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs221client # [ 0.000000] CPU features: detected: Spectre-v4222worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.223client # [ 0.000000] CPU features: detected: Spectre-BHB224lb # [ 0.000000] psci: Trusted OS migration not required225worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]226lb # [ 0.000000] psci: SMC Calling Convention v1.1227worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000228client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38229worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns230worker1 # [ 0.000000] psci: probing for conduit method from DT.231client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23232worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).233lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)234worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.235client # [ 0.000000] alternatives: applying boot alternatives236lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296237worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs238lb # [ 0.000000] Detected PIPT I-cache on CPU0239worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns240worker1 # [ 0.000000] psci: Trusted OS migration not required241worker1 # [ 0.000000] psci: SMC Calling Convention v1.1242worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns243lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)244worker2 # [ 0.000037] arm-pv: using stolen time PV245lb # [ 0.000000] CPU features: detected: GICv3 CPU interface246worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)247lb # [ 0.000000] CPU features: detected: Spectre-v4248worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296249worker2 # [ 0.000655] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)250lb # [ 0.000000] CPU features: detected: Spectre-BHB251client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kn580alm8i65l8xnw33vjbcfrdhrbapf-nixos-system-client-test/init regInfo=/nix/store/pjzrk30p45r569g66fnvv3qqq9mz2m7v-closure-info/registration console=ttyAMA0,115200n8 console=tty0252worker1 # [ 0.000000] Detected PIPT I-cache on CPU0253worker2 # [ 0.000831] Console: colour dummy device 80x25254lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38255worker2 # [ 0.000839] printk: legacy console [tty0] enabled256worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)257lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23258client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/pjzrk30p45r569g66fnvv3qqq9mz2m7v-closure-info/registration", will be passed to user space.259lb # [ 0.000000] alternatives: applying boot alternatives260worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface261worker2 # [ 0.001031] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)262worker1 # [ 0.000000] CPU features: detected: Spectre-v4263client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes264worker2 # [ 0.001038] pid_max: default: 32768 minimum: 301265worker1 # [ 0.000000] CPU features: detected: Spectre-BHB266worker2 # [ 0.001108] LSM: initializing lsm=capability,landlock,yama,bpf,ima267client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)268worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38269worker2 # [ 0.001280] landlock: Up and running.270worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23271worker2 # [ 0.001283] Yama: becoming mindful.272client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)273worker2 # [ 0.001943] LSM support for eBPF active274worker1 # [ 0.000000] alternatives: applying boot alternatives275client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB276lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/sy3pchz5vg22ylc3whvkp4s8qngi8wmd-nixos-system-lb-test/init regInfo=/nix/store/4q693lxjjy3z9gghvdrv4s7qcr63j8a1-closure-info/registration console=ttyAMA0,115200n8 console=tty0277client # [ 0.000000] software IO TLB: area num 1.278worker2 # [ 0.002119] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)279client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB280worker2 # [ 0.002152] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)281lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4q693lxjjy3z9gghvdrv4s7qcr63j8a1-closure-info/registration", will be passed to user space.282worker2 # [ 0.003280] cacheinfo: Unable to detect cache hierarchy for CPU 0283client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)284client # [ 0.000000] Fallback order for Node 0: 0285worker2 # [ 0.004111] rcu: Hierarchical SRCU implementation.286worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vniqsik90d4pd2jhgb9p4y85sl28s16w-nixos-system-worker1-test/init regInfo=/nix/store/sm9ih6p81j3iq8b7cs9xw4s50lcfgml0-closure-info/registration console=ttyAMA0,115200n8 console=tty0287worker2 # [ 0.004116] rcu: Max phase no-delay instances is 1000.288lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes289client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216290worker2 # [ 0.005328] fsl-mc MSI: its@8080000 domain created291lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)292client # [ 0.000000] Policy zone: DMA293worker2 # [ 0.005418] EFI services will not be available.294worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/sm9ih6p81j3iq8b7cs9xw4s50lcfgml0-closure-info/registration", will be passed to user space.295worker2 # [ 0.005531] smp: Bringing up secondary CPUs ...296lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)297client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off298worker2 # [ 0.005539] smp: Brought up 1 node, 1 CPU299lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB300client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1301worker2 # [ 0.005542] SMP: Total of 1 processors activated.302lb # [ 0.000000] software IO TLB: area num 1.303worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes304client # [ 0.000000] allocated 3145728 bytes of page_ext305worker2 # [ 0.005547] CPU: All CPU(s) started at EL1306lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB307client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages308worker2 # [ 0.005559] CPU features: detected: Branch Target Identification309worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)310client # [ 0.000000] ftrace: allocated 294 pages with 4 groups311worker2 # [ 0.005564] CPU features: detected: ARMv8.4 Translation Table Level312lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)313worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)314client # [ 0.000000] rcu: Hierarchical RCU implementation.315lb # [ 0.000000] Fallback order for Node 0: 0316client # [ 0.000000] rcu: RCU event tracing is enabled.317worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB318worker2 # [ 0.005567] CPU features: detected: Instruction cache invalidation not required for I/D coherence319lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216320worker1 # [ 0.000000] software IO TLB: area num 1.321lb # [ 0.000000] Policy zone: DMA322client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.323worker2 # [ 0.005570] CPU features: detected: Data cache clean to the PoU not required for I/D coherence324client # [ 0.000000] Trampoline variant of Tasks RCU enabled.325worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB326lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off327client # [ 0.000000] Rude variant of Tasks RCU enabled.328worker2 # [ 0.005574] CPU features: detected: Common not Private translations329worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)330lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1331client # [ 0.000000] Tracing variant of Tasks RCU enabled.332worker2 # [ 0.005577] CPU features: detected: CRC32 instructions333worker1 # [ 0.000000] Fallback order for Node 0: 0334lb # [ 0.000000] allocated 3145728 bytes of page_ext335client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.336worker2 # [ 0.005580] CPU features: detected: Data cache clean to Point of Deep Persistence337lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages338worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216339worker1 # [ 0.000000] Policy zone: DMA340lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups341client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1342worker2 # [ 0.005584] CPU features: detected: Data cache clean to Point of Persistence343lb # [ 0.000000] rcu: Hierarchical RCU implementation.344worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off345worker2 # [ 0.005587] CPU features: detected: Data independent timing control (DIT)346lb # [ 0.000000] rcu: RCU event tracing is enabled.347client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.348worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1349worker2 # [ 0.005590] CPU features: detected: E0PD350lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.351worker1 # [ 0.000000] allocated 3145728 bytes of page_ext352client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.353worker2 # [ 0.005592] CPU features: detected: Enhanced Counter Virtualization354lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.355worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages356lb # [ 0.000000] Rude variant of Tasks RCU enabled.357worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups358worker2 # [ 0.005595] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)359client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.360lb # [ 0.000000] Tracing variant of Tasks RCU enabled.361worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.362worker2 # [ 0.005598] CPU features: detected: Enhanced Virtualization Traps363client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0364worker1 # [ 0.000000] rcu: RCU event tracing is enabled.365lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.366client # [ 0.000000] GICv3: 256 SPIs implemented367worker2 # [ 0.005601] CPU features: detected: Fine Grained Traps368worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.369client # [ 0.000000] GICv3: 0 Extended SPIs implemented370lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1371worker2 # [ 0.005605] CPU features: detected: Generic authentication (architected QARMA5 algorithm)372worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.373client # [ 0.000000] Root IRQ handler: gic_handle_irq374client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI375worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.376worker2 # [ 0.005610] CPU features: detected: RCpc load-acquire (LDAPR)377lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.378client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0379worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.380worker2 # [ 0.005612] CPU features: detected: LSE atomic instructions381lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.382client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000383worker2 # [ 0.005615] CPU features: detected: Privileged Access Never384worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.385client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]386worker2 # [ 0.005618] CPU features: detected: PMUv3387worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1388lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.389worker2 # [ 0.005620] CPU features: detected: RAS Extension Support390lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0391client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)392worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.393lb # [ 0.000000] GICv3: 256 SPIs implemented394lb # [ 0.000000] GICv3: 0 Extended SPIs implemented395worker2 # [ 0.005623] CPU features: detected: RASv1p1 Extension Support396client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)397worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.398lb # [ 0.000000] Root IRQ handler: gic_handle_irq399worker2 # [ 0.005626] CPU features: detected: Random Number Generator400client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000401lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI402worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.403worker2 # [ 0.005628] CPU features: detected: Speculation barrier (SB)404lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0405client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000406worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0407worker2 # [ 0.005631] CPU features: detected: Stage-2 Force Write-Back408lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000409worker1 # [ 0.000000] GICv3: 256 SPIs implemented410client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.411lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]412worker2 # [ 0.005634] CPU features: detected: TLB range maintenance instructions413worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented414worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq415worker2 # [ 0.005639] CPU features: detected: Speculative Store Bypassing Safe (SSBS)416client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns417lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)418worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI419worker2 # [ 0.005688] alternatives: applying system-wide alternatives420client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).421worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0422lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)423worker2 # [ 0.008889] CPU features: detected: BBM Level 2 without TLB conflict abort424worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000425lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000426client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns427worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]428lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000429client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns430worker2 # [ 0.009074] Memory: 1406548K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)431worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)432client # [ 0.000030] arm-pv: using stolen time PV433worker2 # [ 0.009534] devtmpfs: initialized434lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.435worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)436worker2 # [ 0.011441] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)437client # [ 0.000378] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)438lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns439worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000440client # [ 0.000530] Console: colour dummy device 80x25441worker2 # [ 0.011464] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).442lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).443client # [ 0.000538] printk: legacy console [tty0] enabled444worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000445worker2 # [ 0.011655] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL446worker2 # [ 0.011660] 0 pages in range for non-PLT usage447worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.448lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns449client # [ 0.000750] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)450worker2 # [ 0.011661] 508288 pages in range for PLT usage451client # [ 0.000757] pid_max: default: 32768 minimum: 301452worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns453worker2 # [ 0.011775] pinctrl core: initialized pinctrl subsystem454lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns455lb # [ 0.000028] arm-pv: using stolen time PV456client # [ 0.000825] LSM: initializing lsm=capability,landlock,yama,bpf,ima457worker2 # [ 0.012575] DMI not present or invalid.458worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).459client # [ 0.000964] landlock: Up and running.460worker2 # [ 0.016531] NET: Registered PF_NETLINK/PF_ROUTE protocol family461client # [ 0.000967] Yama: becoming mindful.462lb # [ 0.000377] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)463client # [ 0.001412] LSM support for eBPF active464lb # [ 0.000526] Console: colour dummy device 80x25465worker2 # [ 0.018936] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations466worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns467client # [ 0.001543] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)468lb # [ 0.000533] printk: legacy console [tty0] enabled469worker2 # [ 0.019128] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations470worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns471worker1 # [ 0.000030] arm-pv: using stolen time PV472client # [ 0.001577] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)473worker2 # [ 0.019338] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations474lb # [ 0.000725] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)475client # [ 0.002624] cacheinfo: Unable to detect cache hierarchy for CPU 0476lb # [ 0.000732] pid_max: default: 32768 minimum: 301477worker1 # [ 0.000425] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)478worker2 # [ 0.019363] audit: initializing netlink subsys (disabled)479client # [ 0.003356] rcu: Hierarchical SRCU implementation.480worker1 # [ 0.000610] Console: colour dummy device 80x25481lb # [ 0.000799] LSM: initializing lsm=capability,landlock,yama,bpf,ima482worker2 # [ 0.019990] thermal_sys: Registered thermal governor 'fair_share'483lb # [ 0.000928] landlock: Up and running.484worker1 # [ 0.000618] printk: legacy console [tty0] enabled485client # [ 0.003360] rcu: Max phase no-delay instances is 1000.486lb # [ 0.000931] Yama: becoming mindful.487worker2 # [ 0.019992] thermal_sys: Registered thermal governor 'bang_bang'488lb # [ 0.001355] LSM support for eBPF active489worker2 # [ 0.019996] thermal_sys: Registered thermal governor 'step_wise'490client # [ 0.004529] fsl-mc MSI: its@8080000 domain created491worker1 # [ 0.000829] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)492lb # [ 0.001509] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)493worker2 # [ 0.020001] thermal_sys: Registered thermal governor 'user_space'494worker1 # [ 0.000835] pid_max: default: 32768 minimum: 301495client # [ 0.004620] EFI services will not be available.496worker1 # [ 0.000907] LSM: initializing lsm=capability,landlock,yama,bpf,ima497worker2 # [ 0.020004] thermal_sys: Registered thermal governor 'power_allocator'498lb # [ 0.001541] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)499client # [ 0.004700] smp: Bringing up secondary CPUs ...500worker1 # [ 0.001042] landlock: Up and running.501lb # [ 0.002613] cacheinfo: Unable to detect cache hierarchy for CPU 0502worker1 # [ 0.001044] Yama: becoming mindful.503client # [ 0.004708] smp: Brought up 1 node, 1 CPU504worker2 # [ 0.020038] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1505worker1 # [ 0.001500] LSM support for eBPF active506lb # [ 0.003352] rcu: Hierarchical SRCU implementation.507client # [ 0.004711] SMP: Total of 1 processors activated.508worker2 # [ 0.020047] cpuidle: using governor ladder509worker2 # [ 0.020052] cpuidle: using governor menu510client # [ 0.004716] CPU: All CPU(s) started at EL1511lb # [ 0.003356] rcu: Max phase no-delay instances is 1000.512worker1 # [ 0.001650] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)513lb # [ 0.004525] fsl-mc MSI: its@8080000 domain created514client # [ 0.004726] CPU features: detected: Branch Target Identification515worker2 # [ 0.020251] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.516lb # [ 0.004624] EFI services will not be available.517worker1 # [ 0.001682] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)518client # [ 0.004731] CPU features: detected: ARMv8.4 Translation Table Level519worker2 # [ 0.020268] ASID allocator initialised with 65536 entries520lb # [ 0.004712] smp: Bringing up secondary CPUs ...521worker1 # [ 0.002826] cacheinfo: Unable to detect cache hierarchy for CPU 0522worker2 # [ 0.021576] Serial: AMBA PL011 UART driver523lb # [ 0.004720] smp: Brought up 1 node, 1 CPU524worker1 # [ 0.003588] rcu: Hierarchical SRCU implementation.525client # [ 0.004734] CPU features: detected: Instruction cache invalidation not required for I/D coherence526lb # [ 0.004724] SMP: Total of 1 processors activated.527worker2 # [ 0.027043] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1528worker1 # [ 0.003592] rcu: Max phase no-delay instances is 1000.529lb # [ 0.004729] CPU: All CPU(s) started at EL1530worker2 # [ 0.027216] printk: console [ttyAMA0] enabled531worker1 # [ 0.004861] fsl-mc MSI: its@8080000 domain created532client # [ 0.004738] CPU features: detected: Data cache clean to the PoU not required for I/D coherence533lb # [ 0.004740] CPU features: detected: Branch Target Identification534worker1 # [ 0.004953] EFI services will not be available.535client # [ 0.004741] CPU features: detected: Common not Private translations536worker1 # [ 0.005034] smp: Bringing up secondary CPUs ...537lb # [ 0.004745] CPU features: detected: ARMv8.4 Translation Table Level538client # [ 0.004744] CPU features: detected: CRC32 instructions539worker1 # [ 0.005042] smp: Brought up 1 node, 1 CPU540lb # [ 0.004748] CPU features: detected: Instruction cache invalidation not required for I/D coherence541worker1 # [ 0.005045] SMP: Total of 1 processors activated.542client # [ 0.004747] CPU features: detected: Data cache clean to Point of Deep Persistence543worker1 # [ 0.005050] CPU: All CPU(s) started at EL1544client # [ 0.004751] CPU features: detected: Data cache clean to Point of Persistence545lb # [ 0.004751] CPU features: detected: Data cache clean to the PoU not required for I/D coherence546worker1 # [ 0.005061] CPU features: detected: Branch Target Identification547lb # [ 0.004755] CPU features: detected: Common not Private translations548client # [ 0.004754] CPU features: detected: Data independent timing control (DIT)549worker1 # [ 0.005066] CPU features: detected: ARMv8.4 Translation Table Level550client # [ 0.004757] CPU features: detected: E0PD551lb # [ 0.004758] CPU features: detected: CRC32 instructions552worker1 # [ 0.005069] CPU features: detected: Instruction cache invalidation not required for I/D coherence553client # [ 0.004760] CPU features: detected: Enhanced Counter Virtualization554lb # [ 0.004761] CPU features: detected: Data cache clean to Point of Deep Persistence555client # [ 0.004763] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)556worker1 # [ 0.005073] CPU features: detected: Data cache clean to the PoU not required for I/D coherence557lb # [ 0.004764] CPU features: detected: Data cache clean to Point of Persistence558client # [ 0.004766] CPU features: detected: Enhanced Virtualization Traps559worker1 # [ 0.005077] CPU features: detected: Common not Private translations560lb # [ 0.004768] CPU features: detected: Data independent timing control (DIT)561client # [ 0.004769] CPU features: detected: Fine Grained Traps562lb # [ 0.004771] CPU features: detected: E0PD563worker1 # [ 0.005080] CPU features: detected: CRC32 instructions564lb # [ 0.004773] CPU features: detected: Enhanced Counter Virtualization565worker1 # [ 0.005083] CPU features: detected: Data cache clean to Point of Deep Persistence566client # [ 0.004773] CPU features: detected: Generic authentication (architected QARMA5 algorithm)567client # [ 0.004778] CPU features: detected: RCpc load-acquire (LDAPR)568worker1 # [ 0.005087] CPU features: detected: Data cache clean to Point of Persistence569lb # [ 0.004776] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)570worker2 # [ 0.150462] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages571client # [ 0.004781] CPU features: detected: LSE atomic instructions572worker1 # [ 0.005090] CPU features: detected: Data independent timing control (DIT)573lb # [ 0.004780] CPU features: detected: Enhanced Virtualization Traps574worker2 # [ 0.150482] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page575worker1 # [ 0.005093] CPU features: detected: E0PD576lb # [ 0.004783] CPU features: detected: Fine Grained Traps577client # [ 0.004784] CPU features: detected: Privileged Access Never578worker2 # [ 0.150487] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages579client # [ 0.004787] CPU features: detected: PMUv3580worker1 # [ 0.005096] CPU features: detected: Enhanced Counter Virtualization581lb # [ 0.004786] CPU features: detected: Generic authentication (architected QARMA5 algorithm)582client # [ 0.004789] CPU features: detected: RAS Extension Support583worker2 # [ 0.150491] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page584worker1 # [ 0.005099] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)585lb # [ 0.004791] CPU features: detected: RCpc load-acquire (LDAPR)586client # [ 0.004792] CPU features: detected: RASv1p1 Extension Support587worker2 # [ 0.150495] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages588worker1 # [ 0.005102] CPU features: detected: Enhanced Virtualization Traps589lb # [ 0.004794] CPU features: detected: LSE atomic instructions590client # [ 0.004795] CPU features: detected: Random Number Generator591worker1 # [ 0.005105] CPU features: detected: Fine Grained Traps592lb # [ 0.004797] CPU features: detected: Privileged Access Never593worker2 # [ 0.150499] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page594client # [ 0.004797] CPU features: detected: Speculation barrier (SB)595lb # [ 0.004800] CPU features: detected: PMUv3596worker2 # [ 0.150504] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages597worker1 # [ 0.005109] CPU features: detected: Generic authentication (architected QARMA5 algorithm)598client # [ 0.004800] CPU features: detected: Stage-2 Force Write-Back599lb # [ 0.004803] CPU features: detected: RAS Extension Support600worker2 # [ 0.150508] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page601worker1 # [ 0.005115] CPU features: detected: RCpc load-acquire (LDAPR)602lb # [ 0.004805] CPU features: detected: RASv1p1 Extension Support603client # [ 0.004803] CPU features: detected: TLB range maintenance instructions604worker1 # [ 0.005118] CPU features: detected: LSE atomic instructions605lb # [ 0.004808] CPU features: detected: Random Number Generator606client # [ 0.004808] CPU features: detected: Speculative Store Bypassing Safe (SSBS)607worker1 # [ 0.005121] CPU features: detected: Privileged Access Never608lb # [ 0.004811] CPU features: detected: Speculation barrier (SB)609worker1 # [ 0.005124] CPU features: detected: PMUv3610client # [ 0.004845] alternatives: applying system-wide alternatives611lb # [ 0.004813] CPU features: detected: Stage-2 Force Write-Back612worker2 # [ 0.157764] fbcon: Taking over console613worker1 # [ 0.005127] CPU features: detected: RAS Extension Support614client # [ 0.007763] CPU features: detected: BBM Level 2 without TLB conflict abort615worker2 # [ 0.157781] ACPI: Interpreter disabled.616worker1 # [ 0.005129] CPU features: detected: RASv1p1 Extension Support617worker2 # [ 0.159664] iommu: Default domain type: Translated618worker1 # [ 0.005132] CPU features: detected: Random Number Generator619worker2 # [ 0.159675] iommu: DMA domain TLB invalidation policy: strict mode620worker1 # [ 0.005135] CPU features: detected: Speculation barrier (SB)621client # [ 0.007945] Memory: 1406532K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)622worker1 # [ 0.005138] CPU features: detected: Stage-2 Force Write-Back623worker2 # [ 0.161373] SCSI subsystem initialized624worker1 # [ 0.005141] CPU features: detected: TLB range maintenance instructions625worker1 # [ 0.005146] CPU features: detected: Speculative Store Bypassing Safe (SSBS)626worker1 # [ 0.005184] alternatives: applying system-wide alternatives627worker1 # [ 0.008260] CPU features: detected: BBM Level 2 without TLB conflict abort628worker1 # [ 0.008410] Memory: 1406516K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)629worker1 # [ 0.008802] devtmpfs: initialized630lb # [ 0.004816] CPU features: detected: TLB range maintenance instructions631worker2 # [ 0.166725] usbcore: registered new interface driver usbfs632worker1 # [ 0.010594] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)633worker2 # [ 0.166758] usbcore: registered new interface driver hub634lb # [ 0.004821] CPU features: detected: Speculative Store Bypassing Safe (SSBS)635worker1 # [ 0.010616] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).636worker2 # [ 0.166776] usbcore: registered new device driver usb637lb # [ 0.004859] alternatives: applying system-wide alternatives638worker2 # [ 0.167038] pps_core: LinuxPPS API ver. 1 registered639worker1 # [ 0.010804] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL640client # [ 0.008364] devtmpfs: initialized641lb # [ 0.007835] CPU features: detected: BBM Level 2 without TLB conflict abort642worker1 # [ 0.010808] 0 pages in range for non-PLT usage643client # [ 0.010230] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)644worker1 # [ 0.010809] 508288 pages in range for PLT usage645worker2 # [ 0.167043] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>646worker2 # [ 0.167054] PTP clock support registered647worker1 # [ 0.010905] pinctrl core: initialized pinctrl subsystem648client # [ 0.010254] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).649worker2 # [ 0.167102] EDAC MC: Ver: 3.0.0650lb # [ 0.008103] Memory: 1406580K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)651lb # [ 0.008484] devtmpfs: initialized652client # [ 0.010443] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL653worker2 # [ 0.171611] scmi_core: SCMI protocol bus registered654client # [ 0.010448] 0 pages in range for non-PLT usage655lb # [ 0.010495] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)656client # [ 0.010449] 508288 pages in range for PLT usage657worker2 # [ 0.172578] FPGA manager framework658lb # [ 0.010517] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).659worker2 # [ 0.173568] vgaarb: loaded660client # [ 0.010567] pinctrl core: initialized pinctrl subsystem661client # [ 0.011320] DMI not present or invalid.662lb # [ 0.010715] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL663worker2 # [ 0.174196] clocksource: Switched to clocksource arch_sys_counter664client # [ 0.014886] NET: Registered PF_NETLINK/PF_ROUTE protocol family665lb # [ 0.010719] 0 pages in range for non-PLT usage666lb # [ 0.010720] 508288 pages in range for PLT usage667client # [ 0.017134] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations668lb # [ 0.010815] pinctrl core: initialized pinctrl subsystem669lb # [ 0.011628] DMI not present or invalid.670client # [ 0.017338] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations671lb # [ 0.014587] NET: Registered PF_NETLINK/PF_ROUTE protocol family672client # [ 0.017539] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations673lb # [ 0.016811] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations674client # [ 0.017561] audit: initializing netlink subsys (disabled)675worker2 # [ 0.178113] VFS: Disk quotas dquot_6.6.0676lb # [ 0.016993] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations677client # [ 0.018164] thermal_sys: Registered thermal governor 'fair_share'678worker2 # [ 0.178149] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)679worker1 # [ 0.011736] DMI not present or invalid.680client # [ 0.018166] thermal_sys: Registered thermal governor 'bang_bang'681lb # [ 0.017193] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations682worker1 # [ 0.015025] NET: Registered PF_NETLINK/PF_ROUTE protocol family683client # [ 0.018170] thermal_sys: Registered thermal governor 'step_wise'684lb # [ 0.017216] audit: initializing netlink subsys (disabled)685worker1 # [ 0.017272] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations686client # [ 0.018175] thermal_sys: Registered thermal governor 'user_space'687lb # [ 0.017835] thermal_sys: Registered thermal governor 'fair_share'688lb # [ 0.017837] thermal_sys: Registered thermal governor 'bang_bang'689worker1 # [ 0.017451] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations690client # [ 0.018178] thermal_sys: Registered thermal governor 'power_allocator'691worker2 # [ 0.181712] netfs: FS-Cache loaded692worker2 # [ 0.181849] pnp: PnP ACPI: disabled693lb # [ 0.017840] thermal_sys: Registered thermal governor 'step_wise'694client # [ 0.018203] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1695worker1 # [ 0.017654] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations696client # [ 0.018212] cpuidle: using governor ladder697lb # [ 0.017845] thermal_sys: Registered thermal governor 'user_space'698worker1 # [ 0.017676] audit: initializing netlink subsys (disabled)699client # [ 0.018218] cpuidle: using governor menu700lb # [ 0.017848] thermal_sys: Registered thermal governor 'power_allocator'701worker1 # [ 0.018296] thermal_sys: Registered thermal governor 'fair_share'702client # [ 0.018405] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.703worker1 # [ 0.018298] thermal_sys: Registered thermal governor 'bang_bang'704lb # [ 0.017873] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1705client # [ 0.018421] ASID allocator initialised with 65536 entries706lb # [ 0.017882] cpuidle: using governor ladder707worker1 # [ 0.018301] thermal_sys: Registered thermal governor 'step_wise'708client # [ 0.019670] Serial: AMBA PL011 UART driver709lb # [ 0.017887] cpuidle: using governor menu710worker1 # [ 0.018307] thermal_sys: Registered thermal governor 'user_space'711lb # [ 0.018073] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.712worker2 # [ 0.185695] NET: Registered PF_INET protocol family713client # [ 0.024898] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1714worker1 # [ 0.018310] thermal_sys: Registered thermal governor 'power_allocator'715client # [ 0.025021] printk: console [ttyAMA0] enabled716lb # [ 0.018088] ASID allocator initialised with 65536 entries717worker2 # [ 0.185950] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)718lb # [ 0.019305] Serial: AMBA PL011 UART driver719worker1 # [ 0.018342] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1720worker1 # [ 0.018351] cpuidle: using governor ladder721client # [ 0.148282] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages722lb # [ 0.024511] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1723worker1 # [ 0.018356] cpuidle: using governor menu724lb # [ 0.024627] printk: console [ttyAMA0] enabled725client # [ 0.148305] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page726worker1 # [ 0.018541] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.727client # [ 0.148311] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages728worker1 # [ 0.018557] ASID allocator initialised with 65536 entries729lb # [ 0.143943] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages730worker1 # [ 0.019762] Serial: AMBA PL011 UART driver731client # [ 0.148315] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page732lb # [ 0.143961] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page733worker1 # [ 0.024867] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1734client # [ 0.148319] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages735worker1 # [ 0.024988] printk: console [ttyAMA0] enabled736lb # [ 0.143966] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages737client # [ 0.148324] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page738lb # [ 0.143970] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page739client # [ 0.148328] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages740worker1 # [ 0.144186] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages741lb # [ 0.143975] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages742client # [ 0.148332] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page743worker1 # [ 0.144205] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page744lb # [ 0.143979] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page745worker1 # [ 0.144211] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages746lb # [ 0.143983] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages747worker1 # [ 0.144224] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page748lb # [ 0.143987] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page749client # [ 0.155766] fbcon: Taking over console750worker1 # [ 0.144228] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages751client # [ 0.155784] ACPI: Interpreter disabled.752worker1 # [ 0.144232] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page753client # [ 0.157718] iommu: Default domain type: Translated754worker1 # [ 0.144237] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages755lb # [ 0.151328] fbcon: Taking over console756client # [ 0.157731] iommu: DMA domain TLB invalidation policy: strict mode757lb # [ 0.151344] ACPI: Interpreter disabled.758worker1 # [ 0.144241] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page759client # [ 0.159451] SCSI subsystem initialized760lb # [ 0.153181] iommu: Default domain type: Translated761lb # [ 0.153190] iommu: DMA domain TLB invalidation policy: strict mode762worker1 # [ 0.151484] fbcon: Taking over console763lb # [ 0.154907] SCSI subsystem initialized764worker1 # [ 0.151499] ACPI: Interpreter disabled.765worker1 # [ 0.153339] iommu: Default domain type: Translated766worker1 # [ 0.153348] iommu: DMA domain TLB invalidation policy: strict mode767worker1 # [ 0.154982] SCSI subsystem initialized768lb # [ 0.160206] usbcore: registered new interface driver usbfs769lb # [ 0.160237] usbcore: registered new interface driver hub770client # [ 0.167181] usbcore: registered new interface driver usbfs771lb # [ 0.160255] usbcore: registered new device driver usb772client # [ 0.167215] usbcore: registered new interface driver hub773lb # [ 0.160529] pps_core: LinuxPPS API ver. 1 registered774client # [ 0.167243] usbcore: registered new device driver usb775client # [ 0.167543] pps_core: LinuxPPS API ver. 1 registered776worker1 # [ 0.160468] usbcore: registered new interface driver usbfs777lb # [ 0.160535] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>778lb # [ 0.160545] PTP clock support registered779worker1 # [ 0.160500] usbcore: registered new interface driver hub780client # [ 0.167550] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>781lb # [ 0.160593] EDAC MC: Ver: 3.0.0782worker1 # [ 0.160518] usbcore: registered new device driver usb783client # [ 0.167560] PTP clock support registered784client # [ 0.167615] EDAC MC: Ver: 3.0.0785lb # [ 0.165130] scmi_core: SCMI protocol bus registered786worker1 # [ 0.160785] pps_core: LinuxPPS API ver. 1 registered787client # [ 0.172300] scmi_core: SCMI protocol bus registered788lb # [ 0.166061] FPGA manager framework789worker1 # [ 0.160790] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>790lb # [ 0.166985] vgaarb: loaded791worker1 # [ 0.160800] PTP clock support registered792client # [ 0.173354] FPGA manager framework793worker1 # [ 0.160847] EDAC MC: Ver: 3.0.0794lb # [ 0.167604] clocksource: Switched to clocksource arch_sys_counter795client # [ 0.174367] vgaarb: loaded796worker1 # [ 0.165361] scmi_core: SCMI protocol bus registered797worker1 # [ 0.166280] FPGA manager framework798client # [ 0.175043] clocksource: Switched to clocksource arch_sys_counter799worker1 # [ 0.167297] vgaarb: loaded800client # [ 0.175654] VFS: Disk quotas dquot_6.6.0801worker1 # [ 0.167931] clocksource: Switched to clocksource arch_sys_counter802client # [ 0.175683] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)803client # [ 0.178111] netfs: FS-Cache loaded804client # [ 0.178248] pnp: PnP ACPI: disabled805lb # [ 0.171170] VFS: Disk quotas dquot_6.6.0806lb # [ 0.171204] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)807worker1 # [ 0.171462] VFS: Disk quotas dquot_6.6.0808lb # [ 0.174801] netfs: FS-Cache loaded809worker1 # [ 0.171491] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)810lb # [ 0.174939] pnp: PnP ACPI: disabled811worker2 # [ 0.214681] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)812worker2 # [ 0.214723] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)813worker1 # [ 0.175159] netfs: FS-Cache loaded814worker1 # [ 0.175284] pnp: PnP ACPI: disabled815worker2 # [ 0.214750] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)816worker2 # [ 0.214823] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)817lb # [ 0.178758] NET: Registered PF_INET protocol family818worker2 # [ 0.214967] TCP: Hash tables configured (established 16384 bind 16384)819lb # [ 0.179032] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)820client # [ 0.184517] NET: Registered PF_INET protocol family821worker2 # [ 0.215044] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)822client # [ 0.184812] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)823worker2 # [ 0.215078] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)824worker1 # [ 0.179082] NET: Registered PF_INET protocol family825worker2 # [ 0.215113] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)826worker1 # [ 0.179342] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)827worker2 # [ 0.215220] NET: Registered PF_UNIX/PF_LOCAL protocol family828worker2 # [ 0.215242] NET: Registered PF_XDP protocol family829worker2 # [ 0.215258] PCI: CLS 0 bytes, default 64830worker2 # [ 0.215498] Trying to unpack rootfs image as initramfs...831worker2 # [ 0.232079] kvm [1]: HYP mode not available832client # [ 0.215017] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833lb # [ 0.208273] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)834lb # [ 0.208319] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)835client # [ 0.216458] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)836lb # [ 0.208346] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)837client # [ 0.216507] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)838lb # [ 0.208421] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)839client # [ 0.216593] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)840worker1 # [ 0.208240] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)841lb # [ 0.208566] TCP: Hash tables configured (established 16384 bind 16384)842client # [ 0.216740] TCP: Hash tables configured (established 16384 bind 16384)843worker1 # [ 0.208290] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)844lb # [ 0.208651] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)845client # [ 0.216856] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)846worker1 # [ 0.208316] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)847lb # [ 0.208685] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)848client # [ 0.216902] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)849worker1 # [ 0.208392] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)850lb # [ 0.208720] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)851client # [ 0.216942] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)852worker1 # [ 0.208536] TCP: Hash tables configured (established 16384 bind 16384)853lb # [ 0.208828] NET: Registered PF_UNIX/PF_LOCAL protocol family854client # [ 0.217086] NET: Registered PF_UNIX/PF_LOCAL protocol family855worker1 # [ 0.208619] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)856lb # [ 0.208852] NET: Registered PF_XDP protocol family857client # [ 0.217116] NET: Registered PF_XDP protocol family858lb # [ 0.208871] PCI: CLS 0 bytes, default 64859client # [ 0.217137] PCI: CLS 0 bytes, default 64860worker1 # [ 0.208664] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)861lb # [ 0.209117] Trying to unpack rootfs image as initramfs...862client # [ 0.217424] Trying to unpack rootfs image as initramfs...863worker1 # [ 0.208701] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)864worker1 # [ 0.208795] NET: Registered PF_UNIX/PF_LOCAL protocol family865worker1 # [ 0.208818] NET: Registered PF_XDP protocol family866worker1 # [ 0.208837] PCI: CLS 0 bytes, default 64867worker1 # [ 0.209085] Trying to unpack rootfs image as initramfs...868lb # [ 0.225209] kvm [1]: HYP mode not available869worker1 # [ 0.225422] kvm [1]: HYP mode not available870client # [ 0.239107] kvm [1]: HYP mode not available871worker2 # [ 0.322763] Initialise system trusted keyrings872worker2 # [ 0.323522] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker2 # [ 0.324863] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker2 # [ 0.325655] 9p: Installing v9fs 9p2000 file system support875worker2 # [ 0.354335] Key type asymmetric registered876worker2 # [ 0.354364] Asymmetric key parser 'x509' registered877worker2 # [ 0.354446] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)878worker2 # [ 0.356529] io scheduler mq-deadline registered879worker2 # [ 0.356540] io scheduler kyber registered880lb # [ 0.316157] Initialise system trusted keyrings881lb # [ 0.316890] workingset: timestamp_bits=42 max_order=19 bucket_order=0882lb # [ 0.318196] squashfs: version 4.0 (2009/01/31) Phillip Lougher883lb # [ 0.319005] 9p: Installing v9fs 9p2000 file system support884worker2 # [ 0.366353] pl061_gpio 9030000.pl061: PL061 GPIO chip registered885worker1 # [ 0.321131] Initialise system trusted keyrings886worker2 # [ 0.367738] ledtrig-cpu: registered to indicate activity on CPUs887worker1 # [ 0.321879] workingset: timestamp_bits=42 max_order=19 bucket_order=0888worker2 # [ 0.368138] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:889client # [ 0.331658] Initialise system trusted keyrings890worker1 # [ 0.323183] squashfs: version 4.0 (2009/01/31) Phillip Lougher891client # [ 0.332404] workingset: timestamp_bits=42 max_order=19 bucket_order=0892worker2 # [ 0.368156] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000893client # [ 0.333768] squashfs: version 4.0 (2009/01/31) Phillip Lougher894worker2 # [ 0.368168] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000895worker2 # [ 0.368176] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000896client # [ 0.334564] 9p: Installing v9fs 9p2000 file system support897worker2 # [ 0.368207] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits898worker1 # [ 0.323925] 9p: Installing v9fs 9p2000 file system support899worker2 # [ 0.368233] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]900worker2 # [ 0.368306] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00901worker2 # [ 0.368316] pci_bus 0000:00: root bus resource [bus 00-ff]902worker2 # [ 0.368322] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]903worker2 # [ 0.368327] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]904worker2 # [ 0.368332] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]905worker2 # [ 0.368389] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint906worker2 # [ 0.368837] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint907worker2 # [ 0.369025] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]908worker2 # [ 0.369044] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]909worker2 # [ 0.369076] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]910worker2 # [ 0.369098] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]911worker2 # [ 0.369558] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint912worker2 # [ 0.369748] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]913worker2 # [ 0.369766] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]914worker2 # [ 0.369799] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]915worker2 # [ 0.389528] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint916worker2 # [ 0.389716] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]917worker2 # [ 0.389735] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]918worker1 # [ 0.343832] Key type asymmetric registered919worker2 # [ 0.389767] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]920worker1 # [ 0.343857] Asymmetric key parser 'x509' registered921lb # [ 0.347696] Key type asymmetric registered922worker1 # [ 0.343926] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)923lb # [ 0.347722] Asymmetric key parser 'x509' registered924worker2 # [ 0.393412] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint925lb # [ 0.347782] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)926worker2 # [ 0.393603] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]927lb # [ 0.349892] io scheduler mq-deadline registered928worker2 # [ 0.393622] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]929lb # [ 0.349903] io scheduler kyber registered930worker2 # [ 0.393657] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]931worker2 # [ 0.394120] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint932worker1 # [ 0.352087] io scheduler mq-deadline registered933worker1 # [ 0.352095] io scheduler kyber registered934client # [ 0.363253] Key type asymmetric registered935client # [ 0.363286] Asymmetric key parser 'x509' registered936client # [ 0.363352] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)937client # [ 0.365500] io scheduler mq-deadline registered938client # [ 0.365511] io scheduler kyber registered939lb # [ 0.359753] pl061_gpio 9030000.pl061: PL061 GPIO chip registered940worker1 # [ 0.360069] pl061_gpio 9030000.pl061: PL061 GPIO chip registered941lb # [ 0.361151] ledtrig-cpu: registered to indicate activity on CPUs942lb # [ 0.361556] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:943lb # [ 0.361572] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000944worker1 # [ 0.361445] ledtrig-cpu: registered to indicate activity on CPUs945lb # [ 0.361584] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000946worker1 # [ 0.361811] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:947lb # [ 0.361592] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000948worker2 # [ 0.406340] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]949worker1 # [ 0.361829] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000950lb # [ 0.361612] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits951worker2 # [ 0.406366] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]952client # [ 0.375163] pl061_gpio 9030000.pl061: PL061 GPIO chip registered953worker1 # [ 0.361841] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000954lb # [ 0.361636] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]955worker2 # [ 0.406396] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]956lb # [ 0.361709] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00957worker1 # [ 0.361849] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000958worker2 # [ 0.406877] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint959lb # [ 0.361718] pci_bus 0000:00: root bus resource [bus 00-ff]960worker2 # [ 0.407065] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]961worker1 # [ 0.361880] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits962lb # [ 0.361732] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]963worker2 # [ 0.407081] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]964client # [ 0.376505] ledtrig-cpu: registered to indicate activity on CPUs965worker1 # [ 0.361905] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]966lb # [ 0.361739] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]967worker2 # [ 0.407110] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]968worker1 # [ 0.361984] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00969client # [ 0.376894] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:970lb # [ 0.361744] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]971worker1 # [ 0.361993] pci_bus 0000:00: root bus resource [bus 00-ff]972worker2 # [ 0.407551] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint973client # [ 0.376910] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000974lb # [ 0.361799] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint975worker2 # [ 0.407734] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]976worker1 # [ 0.361999] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]977worker2 # [ 0.407749] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]978lb # [ 0.362228] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint979client # [ 0.376923] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000980worker1 # [ 0.362004] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]981lb # [ 0.362411] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]982worker2 # [ 0.407779] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]983worker1 # [ 0.362009] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]984client # [ 0.376932] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000985lb # [ 0.362427] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]986worker2 # [ 0.407795] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]987worker1 # [ 0.362064] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint988client # [ 0.376956] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits989lb # [ 0.362456] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]990worker2 # [ 0.408252] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint991worker1 # [ 0.362495] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint992lb # [ 0.362472] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]993client # [ 0.376979] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]994worker2 # [ 0.408439] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]995worker1 # [ 0.362678] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]996lb # [ 0.362933] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint997client # [ 0.377054] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00998worker2 # [ 0.408469] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]999worker1 # [ 0.362694] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1000lb # [ 0.363115] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1001client # [ 0.377063] pci_bus 0000:00: root bus resource [bus 00-ff]1002worker1 # [ 0.362724] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1003worker2 # [ 0.408929] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1004lb # [ 0.363130] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1005client # [ 0.377069] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1006worker1 # [ 0.362740] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1007worker2 # [ 0.409120] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1008lb # [ 0.363159] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1009client # [ 0.377074] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1010worker2 # [ 0.409152] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1011worker1 # [ 0.363190] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1012lb # [ 0.363605] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1013client # [ 0.377079] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1014worker1 # [ 0.363372] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1015worker2 # [ 0.409554] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1016worker1 # [ 0.363388] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1017client # [ 0.377173] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1018lb # [ 0.383321] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1019worker2 # [ 0.409740] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1020lb # [ 0.383340] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1021worker1 # [ 0.363417] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1022client # [ 0.377631] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1023worker2 # [ 0.409987] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1024lb # [ 0.383369] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025client # [ 0.377820] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1026worker1 # [ 0.363863] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1027worker2 # [ 0.410286] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1028client # [ 0.377836] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1029worker2 # [ 0.410306] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1030client # [ 0.377865] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1031worker2 # [ 0.410338] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1032client # [ 0.377881] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1033worker2 # [ 0.410805] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1034client # [ 0.378351] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1035worker2 # [ 0.410992] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1036client # [ 0.378535] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1037worker2 # [ 0.411011] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1038client # [ 0.378550] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1039worker2 # [ 0.411044] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1040client # [ 0.378580] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1041worker2 # [ 0.411632] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1042worker2 # [ 0.411645] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1043client # [ 0.397673] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1044worker2 # [ 0.411650] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1045client # [ 0.397858] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1046worker2 # [ 0.411695] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1047client # [ 0.397874] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1048worker2 # [ 0.411745] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1049client # [ 0.397904] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1050lb # [ 0.391883] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1051worker2 # [ 0.411796] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1052client # [ 0.398369] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1053lb # [ 0.392076] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1054client # [ 0.398552] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1055worker2 # [ 0.411846] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1056lb # [ 0.392093] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1057client # [ 0.398567] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1058worker2 # [ 0.411896] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1059lb # [ 0.392125] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1060client # [ 0.398599] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1061worker2 # [ 0.411946] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1062lb # [ 0.392589] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1063lb # [ 0.392770] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1064worker2 # [ 0.411996] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1065lb # [ 0.392786] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1066worker2 # [ 0.412046] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1067lb # [ 0.392815] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1068worker2 # [ 0.412097] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1069lb # [ 0.393260] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1070worker1 # [ 0.392107] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1071lb # [ 0.393442] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1072worker2 # [ 0.412161] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1073worker1 # [ 0.392136] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1074lb # [ 0.393457] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1075worker2 # [ 0.412209] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1076worker1 # [ 0.392166] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077lb # [ 0.393486] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078worker2 # [ 0.412234] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1079worker1 # [ 0.392676] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1080worker2 # [ 0.412259] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1081lb # [ 0.393965] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1082worker1 # [ 0.392863] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1083lb # [ 0.394147] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1084worker2 # [ 0.412284] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1085worker1 # [ 0.392879] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1086lb # [ 0.394162] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1087worker2 # [ 0.412312] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1088worker1 # [ 0.392910] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089lb # [ 0.394192] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1090worker2 # [ 0.412336] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1091worker1 # [ 0.393374] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1092lb # [ 0.394207] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1093worker2 # [ 0.412362] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1094worker1 # [ 0.393569] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1095lb # [ 0.394657] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1096worker1 # [ 0.393587] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1097worker2 # [ 0.412386] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1098lb # [ 0.394842] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1099worker1 # [ 0.393618] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100worker2 # [ 0.412412] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1101lb # [ 0.394878] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1102worker2 # [ 0.412437] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1103worker1 # [ 0.394079] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1104lb # [ 0.395328] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1105worker1 # [ 0.394264] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1106worker2 # [ 0.412462] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1107lb # [ 0.395514] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1108worker1 # [ 0.394279] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1109worker2 # [ 0.412488] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1110lb # [ 0.395543] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111worker1 # [ 0.394309] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1112client # [ 0.411106] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1113worker2 # [ 0.412513] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1114client # [ 0.411310] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1115worker1 # [ 0.394787] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1116worker2 # [ 0.412537] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1117client # [ 0.411327] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1118worker1 # [ 0.394970] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1119worker2 # [ 0.412562] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1120client # [ 0.411357] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1121worker1 # [ 0.394986] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1122worker2 # [ 0.412586] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1123worker1 # [ 0.395015] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124worker2 # [ 0.412611] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1125client # [ 0.411851] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1126worker1 # [ 0.395031] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1127client # [ 0.412036] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1128worker2 # [ 0.412635] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1129client # [ 0.412051] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1130worker2 # [ 0.412660] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1131worker1 # [ 0.395487] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1132worker2 # [ 0.412685] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1133client # [ 0.412081] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1134worker1 # [ 0.395674] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1135worker2 # [ 0.412710] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1136worker1 # [ 0.395704] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137client # [ 0.412539] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1138lb # [ 0.414132] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1139worker2 # [ 0.412739] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1140client # [ 0.412723] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1141worker1 # [ 0.396168] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1142lb # [ 0.414316] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1143worker2 # [ 0.412751] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1144client # [ 0.412738] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1145worker1 # [ 0.396362] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1146worker2 # [ 0.412765] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1147lb # [ 0.414573] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1148client # [ 0.412767] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149worker1 # [ 0.396392] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1150worker2 # [ 0.413924] pci 0000:00:0a.0: enabling device (0000 -> 0002)1151lb # [ 0.414826] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1152client # [ 0.412783] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1153lb # [ 0.414843] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1154worker1 # [ 0.396783] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1155client # [ 0.413298] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1156worker1 # [ 0.396971] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1157lb # [ 0.414881] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158client # [ 0.413486] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1159worker1 # [ 0.397220] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1160lb # [ 0.415336] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1161client # [ 0.413515] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1162worker1 # [ 0.397490] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1163lb # [ 0.415520] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1164client # [ 0.413974] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1165worker1 # [ 0.397507] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1166lb # [ 0.415535] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1167client # [ 0.414162] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1168worker1 # [ 0.397536] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1169lb # [ 0.415564] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1170client # [ 0.414192] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171lb # [ 0.416148] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1172worker1 # [ 0.398001] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1173worker1 # [ 0.398183] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1174lb # [ 0.416160] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1175client # [ 0.414586] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1176worker1 # [ 0.398198] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1177client # [ 0.414769] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1178lb # [ 0.416165] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1179worker1 # [ 0.398227] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180worker2 # [ 0.474318] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 58982 usecs1181client # [ 0.415019] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1182lb # [ 0.416209] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1183client # [ 0.415319] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1184worker1 # [ 0.398804] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1185lb # [ 0.416255] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1186client # [ 0.415337] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1187worker1 # [ 0.398815] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1188lb # [ 0.416301] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1189client # [ 0.415366] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1190worker1 # [ 0.398820] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1191lb # [ 0.416346] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1192client # [ 0.415835] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1193worker1 # [ 0.398867] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1194client # [ 0.416021] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1195lb # [ 0.416392] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1196worker1 # [ 0.398915] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1197client # [ 0.416036] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1198lb # [ 0.416439] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1199worker1 # [ 0.398962] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1200client # [ 0.416074] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1201lb # [ 0.416485] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1202client # [ 0.416661] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1203worker1 # [ 0.399010] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1204lb # [ 0.416530] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1205client # [ 0.416673] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1206worker1 # [ 0.399057] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1207lb # [ 0.416577] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1208client # [ 0.416679] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1209worker1 # [ 0.399104] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1210lb # [ 0.416646] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1211client # [ 0.416724] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1212worker1 # [ 0.399154] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1213lb # [ 0.416692] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1214client # [ 0.416771] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1215worker1 # [ 0.399201] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1216lb # [ 0.416714] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1217client # [ 0.416817] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1218worker1 # [ 0.399250] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1219lb # [ 0.416735] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1220client # [ 0.416866] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1221worker1 # [ 0.399332] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1222lb # [ 0.416756] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1223lb # [ 0.416780] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1224worker1 # [ 0.399378] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1225client # [ 0.416913] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1226lb # [ 0.416804] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1227worker1 # [ 0.399399] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1228client # [ 0.416959] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1229worker1 # [ 0.399421] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1230lb # [ 0.416825] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1231client # [ 0.417006] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1232worker1 # [ 0.399442] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1233lb # [ 0.416847] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1234client # [ 0.417052] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1235worker1 # [ 0.399466] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1236lb # [ 0.416869] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1237client # [ 0.417099] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1238lb # [ 0.416890] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1239worker1 # [ 0.399488] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1240lb # [ 0.416912] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1241client # [ 0.417171] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1242worker1 # [ 0.399510] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1243lb # [ 0.416935] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1244worker1 # [ 0.399531] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1245client # [ 0.417217] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1246lb # [ 0.416955] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1247worker1 # [ 0.399553] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1248client # [ 0.417238] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1249lb # [ 0.416977] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1250worker1 # [ 0.399575] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1251client # [ 0.417270] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1252lb # [ 0.416998] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1253client # [ 0.417292] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1254worker1 # [ 0.399597] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1255lb # [ 0.417019] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1256client # [ 0.417317] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1257worker1 # [ 0.399620] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1258lb # [ 0.417040] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1259worker1 # [ 0.399641] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1260client # [ 0.417339] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1261lb # [ 0.417062] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1262worker2 # [ 0.494489] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1263worker1 # [ 0.399662] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1264client # [ 0.417360] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1265lb # [ 0.417083] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1266worker1 # [ 0.399683] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1267worker2 # [ 0.496621] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1268client # [ 0.417383] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1269lb # [ 0.417104] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1270worker1 # [ 0.399704] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1271lb # [ 0.417125] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1272client # [ 0.417407] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1273worker1 # [ 0.399725] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1274lb # [ 0.417151] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1275client # [ 0.417430] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1276worker1 # [ 0.399747] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1277lb # [ 0.417161] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1278client # [ 0.417453] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1279worker1 # [ 0.399768] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1280lb # [ 0.417165] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1281client # [ 0.417475] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1282worker1 # [ 0.399789] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1283lb # [ 0.418319] pci 0000:00:0a.0: enabling device (0000 -> 0002)1284client # [ 0.417496] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1285worker1 # [ 0.399811] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1286worker1 # [ 0.399837] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1287client # [ 0.417517] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1288client # [ 0.417538] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1289worker1 # [ 0.399847] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1290client # [ 0.417559] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1291worker1 # [ 0.399852] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1292client # [ 0.417581] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1293client # [ 0.417602] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1294client # [ 0.417623] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1295client # [ 0.417645] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1296client # [ 0.417666] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1297client # [ 0.417693] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1298worker2 # [ 0.506817] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1299client # [ 0.417703] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1300worker2 # [ 0.508911] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1301client # [ 0.417707] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1302client # [ 0.418884] pci 0000:00:0a.0: enabling device (0000 -> 0002)1303worker1 # [ 0.465071] pci 0000:00:0a.0: enabling device (0000 -> 0002)1304client # [ 0.479206] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 58910 usecs1305worker2 # [ 0.518775] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1306worker2 # [ 0.521552] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1307worker2 # [ 0.523802] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1308worker2 # [ 0.525985] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1309lb # [ 0.482249] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1310worker1 # [ 0.485620] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1311lb # [ 0.488634] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1312worker1 # [ 0.487785] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1313worker2 # [ 0.536059] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1314lb # [ 0.490601] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1315worker2 # [ 0.537909] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1316client # [ 0.499839] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1317client # [ 0.502133] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1318worker1 # [ 0.498785] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1319lb # [ 0.500970] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1320worker2 # [ 0.549354] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1321lb # [ 0.503007] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1322client # [ 0.513099] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1323worker1 # [ 0.505025] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1324worker1 # [ 0.506960] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1325lb # [ 0.514384] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1326worker2 # [ 0.563454] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1327worker2 # [ 0.566080] msm_serial: driver initialized1328lb # [ 0.517382] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1329lb # [ 0.519443] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1330worker1 # [ 0.517290] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1331worker1 # [ 0.519603] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1332client # [ 0.523325] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1333client # [ 0.527329] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1334client # [ 0.529383] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1335worker2 # [ 0.570288] SuperH (H)SCI(F) driver initialized1336worker2 # [ 0.570382] STM32 USART driver initialized1337lb # [ 0.529412] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1338lb # [ 0.531388] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1339worker1 # [ 0.530961] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1340client # [ 0.539227] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1341client # [ 0.542503] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1342worker1 # [ 0.533945] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1343worker1 # [ 0.535736] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1344lb # [ 0.542852] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1345client # [ 0.553975] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1346worker1 # [ 0.545476] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1347worker2 # [ 0.597938] loop: module loaded1348worker2 # [ 0.598110] virtio_blk virtio5: 1/0/0 default/read/poll queues1349client # [ 0.556624] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1350client # [ 0.558820] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1351lb # [ 0.555562] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1352lb # [ 0.558056] msm_serial: driver initialized1353lb # [ 0.558203] SuperH (H)SCI(F) driver initialized1354lb # [ 0.558259] STM32 USART driver initialized1355worker2 # [ 0.607041] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1356worker1 # [ 0.561126] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1357worker1 # [ 0.563706] msm_serial: driver initialized1358worker1 # [ 0.563850] SuperH (H)SCI(F) driver initialized1359worker1 # [ 0.563903] STM32 USART driver initialized1360worker2 # [ 0.609948] megasas: 07.734.00.00-rc11361worker2 # [ 0.610632] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1362client # [ 0.576375] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1363client # [ 0.579174] msm_serial: driver initialized1364client # [ 0.579345] SuperH (H)SCI(F) driver initialized1365client # [ 0.579400] STM32 USART driver initialized1366worker2 # [ 0.622235] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001367worker2 # [ 0.622260] Intel/Sharp Extended Query Table at 0x00311368worker2 # [ 0.623939] Using buffer write method1369worker2 # [ 0.624015] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1370worker2 # [ 0.626904] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001371worker2 # [ 0.626929] Intel/Sharp Extended Query Table at 0x00311372worker2 # [ 0.628337] Using buffer write method1373worker2 # [ 0.628364] Concatenating MTD devices:1374worker2 # [ 0.628368] (0): "0.flash"1375worker2 # [ 0.628371] (1): "0.flash"1376worker2 # [ 0.628375] into device "0.flash"1377lb # [ 0.590341] loop: module loaded1378lb # [ 0.590529] virtio_blk virtio5: 1/0/0 default/read/poll queues1379lb # [ 0.591301] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1380worker1 # [ 0.594129] loop: module loaded1381worker1 # [ 0.594323] virtio_blk virtio5: 1/0/0 default/read/poll queues1382worker1 # [ 0.595058] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1383worker1 # [ 0.600801] megasas: 07.734.00.00-rc11384worker1 # [ 0.601521] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1385client # [ 0.614057] loop: module loaded1386worker1 # [ 0.603452] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001387client # [ 0.614239] virtio_blk virtio5: 1/0/0 default/read/poll queues1388worker1 # [ 0.603477] Intel/Sharp Extended Query Table at 0x00311389client # [ 0.615023] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1390lb # [ 0.604239] megasas: 07.734.00.00-rc11391lb # [ 0.604981] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1392lb # [ 0.607099] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001393lb # [ 0.607152] Intel/Sharp Extended Query Table at 0x00311394lb # [ 0.608698] Using buffer write method1395lb # [ 0.608756] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1396lb # [ 0.610362] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001397lb # [ 0.610388] Intel/Sharp Extended Query Table at 0x00311398worker1 # [ 0.613378] Using buffer write method1399worker1 # [ 0.613448] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1400worker1 # [ 0.614884] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001401worker1 # [ 0.614911] Intel/Sharp Extended Query Table at 0x00311402client # [ 0.619676] megasas: 07.734.00.00-rc11403lb # [ 0.624129] Using buffer write method1404client # [ 0.620391] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1405lb # [ 0.624154] Concatenating MTD devices:1406lb # [ 0.624158] (0): "0.flash"1407lb # [ 0.624163] (1): "0.flash"1408lb # [ 0.624166] into device "0.flash"1409worker1 # [ 0.624675] Using buffer write method1410worker1 # [ 0.624701] Concatenating MTD devices:1411worker1 # [ 0.624705] (0): "0.flash"1412worker1 # [ 0.624710] (1): "0.flash"1413worker1 # [ 0.624713] into device "0.flash"1414client # [ 0.644237] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001415client # [ 0.644291] Intel/Sharp Extended Query Table at 0x00311416client # [ 0.646134] Using buffer write method1417client # [ 0.646210] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1418client # [ 0.657015] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001419client # [ 0.657049] Intel/Sharp Extended Query Table at 0x00311420client # [ 0.658643] Using buffer write method1421client # [ 0.658674] Concatenating MTD devices:1422client # [ 0.658679] (0): "0.flash"1423client # [ 0.658683] (1): "0.flash"1424client # [ 0.658686] into device "0.flash"1425worker2 # [ 0.878508] Freeing initrd memory: 26148K1426worker2 # [ 0.884551] tun: Universal TUN/TAP device driver, 1.61427worker2 # [ 0.888267] thunder_xcv, ver 1.01428worker2 # [ 0.888308] thunder_bgx, ver 1.01429worker2 # [ 0.888333] nicpf, ver 1.01430worker2 # [ 0.888862] e1000: Intel(R) PRO/1000 Network Driver1431worker2 # [ 0.888869] e1000: Copyright (c) 1999-2006 Intel Corporation.1432worker2 # [ 0.888892] e1000e: Intel(R) PRO/1000 Network Driver1433worker2 # [ 0.888901] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1434worker2 # [ 0.888933] igb: Intel(R) Gigabit Ethernet Network Driver1435lb # [ 0.849500] Freeing initrd memory: 26140K1436worker2 # [ 0.888939] igb: Copyright (c) 2007-2014 Intel Corporation.1437worker2 # [ 0.888960] igbvf: Intel(R) Gigabit Virtual Function Network Driver1438worker2 # [ 0.888965] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1439worker2 # [ 0.889094] sky2: driver version 1.301440worker2 # [ 0.890831] ehci-pci 0000:00:0a.0: EHCI Host Controller1441worker2 # [ 0.890881] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11442worker2 # [ 0.891085] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001443lb # [ 0.855487] tun: Universal TUN/TAP device driver, 1.61444worker2 # [ 0.901049] usbcore: registered new interface driver usb-storage1445worker2 # [ 0.901178] usbcore: registered new interface driver usbserial_generic1446worker2 # [ 0.901192] usbserial: USB Serial support registered for generic1447worker2 # [ 0.901764] hv_vmbus: registering driver hyperv_keyboard1448lb # [ 0.859313] thunder_xcv, ver 1.01449lb # [ 0.859359] thunder_bgx, ver 1.01450worker2 # [ 0.902251] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001451lb # [ 0.859381] nicpf, ver 1.01452worker2 # [ 0.902575] hub 1-0:1.0: USB hub found1453worker2 # [ 0.902597] hub 1-0:1.0: 6 ports detected1454worker2 # [ 0.908000] rtc-pl031 9010000.pl031: registered as rtc01455worker1 # [ 0.859127] Freeing initrd memory: 26148K1456worker2 # [ 0.908031] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:34:31 UTC (1789983271)1457lb # [ 0.861146] e1000: Intel(R) PRO/1000 Network Driver1458worker2 # [ 0.908344] i2c_dev: i2c /dev entries driver1459lb # [ 0.861158] e1000: Copyright (c) 1999-2006 Intel Corporation.1460lb # [ 0.861183] e1000e: Intel(R) PRO/1000 Network Driver1461lb # [ 0.861192] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1462lb # [ 0.861221] igb: Intel(R) Gigabit Ethernet Network Driver1463worker1 # [ 0.865168] tun: Universal TUN/TAP device driver, 1.61464lb # [ 0.861227] igb: Copyright (c) 2007-2014 Intel Corporation.1465worker2 # [ 0.913344] sdhci: Secure Digital Host Controller Interface driver1466lb # [ 0.861250] igbvf: Intel(R) Gigabit Virtual Function Network Driver1467worker2 # [ 0.913358] sdhci: Copyright(c) Pierre Ossman1468lb # [ 0.861255] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1469worker2 # [ 0.913612] Synopsys Designware Multimedia Card Interface Driver1470lb # [ 0.861387] sky2: driver version 1.301471worker2 # [ 0.913971] sdhci-pltfm: SDHCI platform and OF driver helper1472lb # [ 0.863067] usbcore: registered new interface driver usb-storage1473lb # [ 0.863121] usbcore: registered new interface driver usbserial_generic1474lb # [ 0.863136] usbserial: USB Serial support registered for generic1475lb # [ 0.864363] ehci-pci 0000:00:0a.0: EHCI Host Controller1476lb # [ 0.864388] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11477worker2 # [ 0.918361] hid: raw HID events driver (C) Jiri Kosina1478lb # [ 0.864703] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001479worker2 # [ 0.918612] usbcore: registered new interface driver usbhid1480worker2 # [ 0.918619] usbhid: USB HID core driver1481worker1 # [ 0.868994] thunder_xcv, ver 1.01482worker1 # [ 0.869040] thunder_bgx, ver 1.01483worker1 # [ 0.869061] nicpf, ver 1.01484lb # [ 0.875163] hv_vmbus: registering driver hyperv_keyboard1485worker2 # [ 0.921536] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1486worker1 # [ 0.869614] e1000: Intel(R) PRO/1000 Network Driver1487lb # [ 0.875643] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001488worker1 # [ 0.869621] e1000: Copyright (c) 1999-2006 Intel Corporation.1489lb # [ 0.875973] hub 1-0:1.0: USB hub found1490lb # [ 0.875994] hub 1-0:1.0: 6 ports detected1491worker1 # [ 0.869645] e1000e: Intel(R) PRO/1000 Network Driver1492worker2 # [ 0.924092] drop_monitor: Initializing network drop monitor service1493worker1 # [ 0.869653] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1494worker2 # [ 0.924253] NET: Registered PF_INET6 protocol family1495worker1 # [ 0.869681] igb: Intel(R) Gigabit Ethernet Network Driver1496worker1 # [ 0.869687] igb: Copyright (c) 2007-2014 Intel Corporation.1497lb # [ 0.878994] rtc-pl031 9010000.pl031: registered as rtc01498worker2 # [ 0.926197] Segment Routing with IPv61499worker1 # [ 0.869710] igbvf: Intel(R) Gigabit Virtual Function Network Driver1500worker2 # [ 0.926242] In-situ OAM (IOAM) with IPv61501lb # [ 0.879021] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:34:31 UTC (1789983271)1502worker2 # [ 0.926276] NET: Registered PF_PACKET protocol family1503worker1 # [ 0.869715] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1504lb # [ 0.879356] i2c_dev: i2c /dev entries driver1505worker1 # [ 0.869846] sky2: driver version 1.301506worker1 # [ 0.871386] usbcore: registered new interface driver usb-storage1507worker1 # [ 0.871489] usbcore: registered new interface driver usbserial_generic1508worker1 # [ 0.871503] usbserial: USB Serial support registered for generic1509worker2 # [ 0.927877] 9pnet: Installing 9P2000 support1510worker1 # [ 0.872722] ehci-pci 0000:00:0a.0: EHCI Host Controller1511worker2 # [ 0.930096] Key type dns_resolver registered1512client # [ 0.891650] Freeing initrd memory: 26160K1513worker1 # [ 0.872754] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11514worker1 # [ 0.873004] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001515lb # [ 0.884165] sdhci: Secure Digital Host Controller Interface driver1516worker1 # [ 0.884638] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001517lb # [ 0.884174] sdhci: Copyright(c) Pierre Ossman1518worker1 # [ 0.884992] hub 1-0:1.0: USB hub found1519worker1 # [ 0.885007] hub 1-0:1.0: 6 ports detected1520lb # [ 0.884445] Synopsys Designware Multimedia Card Interface Driver1521lb # [ 0.884822] sdhci-pltfm: SDHCI platform and OF driver helper1522lb # [ 0.886505] hid: raw HID events driver (C) Jiri Kosina1523worker1 # [ 0.887235] hv_vmbus: registering driver hyperv_keyboard1524lb # [ 0.886774] usbcore: registered new interface driver usbhid1525lb # [ 0.886781] usbhid: USB HID core driver1526client # [ 0.897875] tun: Universal TUN/TAP device driver, 1.61527worker2 # [ 0.937344] registered taskstats version 11528worker1 # [ 0.888864] rtc-pl031 9010000.pl031: registered as rtc01529worker2 # [ 0.937509] Loading compiled-in X.509 certificates1530worker1 # [ 0.888889] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:34:31 UTC (1789983271)1531worker1 # [ 0.889194] i2c_dev: i2c /dev entries driver1532lb # [ 0.892424] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1533lb # [ 0.893942] drop_monitor: Initializing network drop monitor service1534lb # [ 0.894076] NET: Registered PF_INET6 protocol family1535client # [ 0.901599] thunder_xcv, ver 1.01536client # [ 0.901645] thunder_bgx, ver 1.01537lb # [ 0.897080] Segment Routing with IPv61538client # [ 0.901669] nicpf, ver 1.01539lb # [ 0.897110] In-situ OAM (IOAM) with IPv61540client # [ 0.902215] e1000: Intel(R) PRO/1000 Network Driver1541lb # [ 0.897140] NET: Registered PF_PACKET protocol family1542client # [ 0.902222] e1000: Copyright (c) 1999-2006 Intel Corporation.1543worker1 # [ 0.894004] sdhci: Secure Digital Host Controller Interface driver1544client # [ 0.902247] e1000e: Intel(R) PRO/1000 Network Driver1545worker1 # [ 0.894016] sdhci: Copyright(c) Pierre Ossman1546lb # [ 0.898758] 9pnet: Installing 9P2000 support1547client # [ 0.902256] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1548worker1 # [ 0.894280] Synopsys Designware Multimedia Card Interface Driver1549client # [ 0.902285] igb: Intel(R) Gigabit Ethernet Network Driver1550worker2 # [ 0.946013] Demotion targets for Node 0: null1551worker1 # [ 0.894653] sdhci-pltfm: SDHCI platform and OF driver helper1552worker2 # [ 0.946120] Key type .fscrypt registered1553client # [ 0.902291] igb: Copyright (c) 2007-2014 Intel Corporation.1554worker2 # [ 0.946127] Key type fscrypt-provisioning registered1555client # [ 0.902314] igbvf: Intel(R) Gigabit Virtual Function Network Driver1556worker2 # [ 0.948054] ima: No TPM chip found, activating TPM-bypass!1557worker1 # [ 0.899007] hid: raw HID events driver (C) Jiri Kosina1558client # [ 0.902320] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1559worker2 # [ 0.948080] ima: Allocated hash algorithm: sha11560client # [ 0.902452] sky2: driver version 1.301561worker1 # [ 0.899245] usbcore: registered new interface driver usbhid1562lb # [ 0.901629] Key type dns_resolver registered1563worker2 # [ 0.948102] ima: No architecture policies found1564worker1 # [ 0.899252] usbhid: USB HID core driver1565worker2 # [ 0.950496] input: gpio-keys as /devices/platform/gpio-keys/input/input01566client # [ 0.911714] ehci-pci 0000:00:0a.0: EHCI Host Controller1567client # [ 0.911747] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11568worker1 # [ 0.902055] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1569client # [ 0.911984] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001570worker1 # [ 0.903546] drop_monitor: Initializing network drop monitor service1571worker1 # [ 0.903689] NET: Registered PF_INET6 protocol family1572client # [ 0.914394] usbcore: registered new interface driver usb-storage1573worker1 # [ 0.906828] Segment Routing with IPv61574client # [ 0.914450] usbcore: registered new interface driver usbserial_generic1575worker1 # [ 0.906845] In-situ OAM (IOAM) with IPv61576client # [ 0.914465] usbserial: USB Serial support registered for generic1577worker1 # [ 0.906877] NET: Registered PF_PACKET protocol family1578client # [ 0.917350] hv_vmbus: registering driver hyperv_keyboard1579lb # [ 0.907918] registered taskstats version 11580lb # [ 0.908062] Loading compiled-in X.509 certificates1581client # [ 0.918841] rtc-pl031 9010000.pl031: registered as rtc01582client # [ 0.918869] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:34:31 UTC (1789983271)1583worker1 # [ 0.908725] 9pnet: Installing 9P2000 support1584worker1 # [ 0.910826] Key type dns_resolver registered1585client # [ 0.920977] i2c_dev: i2c /dev entries driver1586lb # [ 0.916618] Demotion targets for Node 0: null1587lb # [ 0.916724] Key type .fscrypt registered1588lb # [ 0.916730] Key type fscrypt-provisioning registered1589lb # [ 0.916826] ima: No TPM chip found, activating TPM-bypass!1590lb # [ 0.916845] ima: Allocated hash algorithm: sha11591lb # [ 0.916865] ima: No architecture policies found1592client # [ 0.924185] sdhci: Secure Digital Host Controller Interface driver1593lb # [ 0.920846] input: gpio-keys as /devices/platform/gpio-keys/input/input01594client # [ 0.924194] sdhci: Copyright(c) Pierre Ossman1595client # [ 0.924461] Synopsys Designware Multimedia Card Interface Driver1596worker1 # [ 0.917764] registered taskstats version 11597client # [ 0.924829] sdhci-pltfm: SDHCI platform and OF driver helper1598worker1 # [ 0.917917] Loading compiled-in X.509 certificates1599client # [ 0.926569] hid: raw HID events driver (C) Jiri Kosina1600client # [ 0.926810] usbcore: registered new interface driver usbhid1601worker2 # [ 0.968201] clk: Disabling unused clocks1602client # [ 0.926817] usbhid: USB HID core driver1603worker2 # [ 0.968226] PM: genpd: Disabling unused power domains1604client # [ 0.931546] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001605client # [ 0.932592] hub 1-0:1.0: USB hub found1606client # [ 0.933062] hub 1-0:1.0: 6 ports detected1607worker2 # [ 0.972484] Freeing unused kernel memory: 4736K1608worker2 # [ 0.972677] Run /init as init process1609client # [ 0.934816] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1610worker1 # [ 0.926623] Demotion targets for Node 0: null1611worker1 # [ 0.926736] Key type .fscrypt registered1612worker1 # [ 0.926742] Key type fscrypt-provisioning registered1613worker1 # [ 0.926832] ima: No TPM chip found, activating TPM-bypass!1614client # [ 0.937415] drop_monitor: Initializing network drop monitor service1615worker1 # [ 0.926851] ima: Allocated hash algorithm: sha11616client # [ 0.937565] NET: Registered PF_INET6 protocol family1617worker1 # [ 0.926872] ima: No architecture policies found1618client # [ 0.939498] Segment Routing with IPv61619client # [ 0.939519] In-situ OAM (IOAM) with IPv61620worker1 # [ 0.930972] input: gpio-keys as /devices/platform/gpio-keys/input/input01621client # [ 0.939574] NET: Registered PF_PACKET protocol family1622client # [ 0.941222] 9pnet: Installing 9P2000 support1623lb # [ 0.938499] clk: Disabling unused clocks1624lb # [ 0.938529] PM: genpd: Disabling unused power domains1625client # [ 0.944003] Key type dns_resolver registered1626worker2 # [ 0.988448] systemd[1]: Successfully made /usr/ read-only.1627lb # [ 0.942752] Freeing unused kernel memory: 4736K1628lb # [ 0.942953] Run /init as init process1629client # [ 0.950462] registered taskstats version 11630client # [ 0.950628] Loading compiled-in X.509 certificates1631client # [ 0.959324] Demotion targets for Node 0: null1632client # [ 0.959430] Key type .fscrypt registered1633worker1 # [ 0.949537] clk: Disabling unused clocks1634client # [ 0.959437] Key type fscrypt-provisioning registered1635worker1 # [ 0.949563] PM: genpd: Disabling unused power domains1636client # [ 0.959533] ima: No TPM chip found, activating TPM-bypass!1637client # [ 0.959553] ima: Allocated hash algorithm: sha11638client # [ 0.959574] ima: No architecture policies found1639worker1 # [ 0.953886] Freeing unused kernel memory: 4736K1640worker1 # [ 0.954109] Run /init as init process1641client # [ 0.963657] input: gpio-keys as /devices/platform/gpio-keys/input/input01642lb # [ 0.957780] systemd[1]: Successfully made /usr/ read-only.1643worker1 # [ 0.970194] systemd[1]: Successfully made /usr/ read-only.1644client # [ 0.982027] clk: Disabling unused clocks1645client # [ 0.982055] PM: genpd: Disabling unused power domains1646client # [ 0.986316] Freeing unused kernel memory: 4736K1647client # [ 0.986513] Run /init as init process1648client # [ 1.001308] systemd[1]: Successfully made /usr/ read-only.1649worker2 # [ 1.154260] usb 1-1: new high-speed USB device number 2 using ehci-pci1650lb # [ 1.123722] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker1 # [ 1.132023] usb 1-1: new high-speed USB device number 2 using ehci-pci1652client # [ 1.179120] usb 1-1: new high-speed USB device number 2 using ehci-pci1653worker2 # [ 1.306527] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11654lb # [ 1.275950] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11655worker2 # [ 1.323267] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1656worker2 # [ 1.335488] systemd[1]: Detected virtualization qemu.1657worker1 # [ 1.284289] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11658worker2 # [ 1.337594] systemd[1]: Detected architecture arm64.1659worker2 # [ 1.339589] systemd[1]: Running in initrd.1660worker2 # [ 1.342318] systemd[1]: Initializing machine ID from random generator.1661worker2 # [ 1.345169] systemd[1]: Hostname set to <worker2>.1662lb # [ 1.292643] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1663lb # [ 1.304649] systemd[1]: Detected virtualization qemu.1664lb # [ 1.306625] systemd[1]: Detected architecture arm64.1665lb # [ 1.308610] systemd[1]: Running in initrd.1666lb # [ 1.311200] systemd[1]: Initializing machine ID from random generator.1667lb # [ 1.314134] systemd[1]: Hostname set to <lb>.1668worker1 # [ 1.305057] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1669worker1 # [ 1.316973] systemd[1]: Detected virtualization qemu.1670worker1 # [ 1.318936] systemd[1]: Detected architecture arm64.1671worker1 # [ 1.320895] systemd[1]: Running in initrd.1672worker1 # [ 1.323475] systemd[1]: Initializing machine ID from random generator.1673client # [ 1.331246] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11674worker1 # [ 1.326491] systemd[1]: Hostname set to <worker1>.1675client # [ 1.337257] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1676client # [ 1.349565] systemd[1]: Detected virtualization qemu.1677worker2 # [ 1.390414] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01678client # [ 1.351678] systemd[1]: Detected architecture arm64.1679client # [ 1.353612] systemd[1]: Running in initrd.1680client # [ 1.356237] systemd[1]: Initializing machine ID from random generator.1681client # [ 1.359137] systemd[1]: Hostname set to <client>.1682lb # [ 1.359889] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker1 # [ 1.372221] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01684client # [ 1.415307] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01685worker2 # [ 1.510286] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.479711] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker1 # [ 1.491980] usb 1-2: new high-speed USB device number 3 using ehci-pci1688client # [ 1.535086] usb 1-2: new high-speed USB device number 3 using ehci-pci1689worker2 # [ 1.622944] systemd[1]: bpf-restrict-fs: LSM BPF program attached1690lb # [ 1.611804] systemd[1]: bpf-restrict-fs: LSM BPF program attached1691worker2 # [ 1.669254] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21692worker2 # [ 1.672943] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01693client # [ 1.642223] systemd[1]: bpf-restrict-fs: LSM BPF program attached1694worker1 # [ 1.633430] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695lb # [ 1.638043] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21696lb # [ 1.643856] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01697worker1 # [ 1.661575] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21698worker1 # [ 1.666251] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01699client # [ 1.684667] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21700client # [ 1.686849] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01701worker2 # [ 1.727493] systemd[1]: Queued start job for default target Initrd Default Target.1702worker2 # [ 1.736707] systemd[1]: Created slice Slice /system/modprobe.1703worker2 # [ 1.737985] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1704worker2 # [ 1.739477] systemd[1]: Expecting device /dev/disk/by-label/nixos...1705worker2 # [ 1.740651] systemd[1]: Reached target Path Units.1706worker2 # [ 1.741508] systemd[1]: Reached target Slice Units.1707worker2 # [ 1.742401] systemd[1]: Reached target Swaps.1708worker2 # [ 1.743169] systemd[1]: Reached target Timer Units.1709worker2 # [ 1.744211] systemd[1]: Listening on D-Bus System Message Bus Socket.1710worker2 # [ 1.745500] systemd[1]: Listening on Journal Socket (/dev/log).1711worker2 # [ 1.746751] systemd[1]: Listening on Journal Sockets.1712worker2 # [ 1.746899] systemd[1]: Listening on udev Control Socket.1713worker2 # [ 1.747022] systemd[1]: Listening on udev Kernel Socket.1714worker2 # [ 1.747047] systemd[1]: Reached target Socket Units.1715client # [ 1.711552] systemd[1]: Queued start job for default target Initrd Default Target.1716worker2 # [ 1.752435] systemd[1]: Starting Create List of Static Device Nodes...1717worker2 # [ 1.759543] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1718client # [ 1.721080] systemd[1]: Created slice Slice /system/modprobe.1719worker2 # [ 1.760776] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1720client # [ 1.722094] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1721client # [ 1.723250] systemd[1]: Expecting device /dev/disk/by-label/nixos...1722client # [ 1.724139] systemd[1]: Reached target Path Units.1723client # [ 1.724865] systemd[1]: Reached target Slice Units.1724client # [ 1.725565] systemd[1]: Reached target Swaps.1725client # [ 1.726191] systemd[1]: Reached target Timer Units.1726client # [ 1.727118] systemd[1]: Listening on D-Bus System Message Bus Socket.1727client # [ 1.728160] systemd[1]: Listening on Journal Socket (/dev/log).1728client # [ 1.729127] systemd[1]: Listening on Journal Sockets.1729client # [ 1.729974] systemd[1]: Listening on udev Control Socket.1730client # [ 1.730869] systemd[1]: Listening on udev Kernel Socket.1731lb # [ 1.724788] systemd[1]: Queued start job for default target Initrd Default Target.1732client # [ 1.731711] systemd[1]: Reached target Socket Units.1733worker2 # [ 1.770394] systemd[1]: Mounting Kernel Configuration File System...1734client # [ 1.733874] systemd[1]: Starting Create List of Static Device Nodes...1735worker2 # [ 1.778434] systemd[1]: Starting Journal Service...1736client # [ 1.740242] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1737lb # [ 1.735142] systemd[1]: Created slice Slice /system/modprobe.1738client # [ 1.741206] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1739lb # [ 1.736399] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1740lb # [ 1.737694] systemd[1]: Expecting device /dev/disk/by-label/nixos...1741lb # [ 1.738736] systemd[1]: Reached target Path Units.1742lb # [ 1.739552] systemd[1]: Reached target Slice Units.1743lb # [ 1.740439] systemd[1]: Reached target Swaps.1744lb # [ 1.741182] systemd[1]: Reached target Timer Units.1745lb # [ 1.742169] systemd[1]: Listening on D-Bus System Message Bus Socket.1746lb # [ 1.743398] systemd[1]: Listening on Journal Socket (/dev/log).1747client # [ 1.750646] systemd[1]: Mounting Kernel Configuration File System...1748lb # [ 1.744656] systemd[1]: Listening on Journal Sockets.1749lb # [ 1.745664] systemd[1]: Listening on udev Control Socket.1750lb # [ 1.746696] systemd[1]: Listening on udev Kernel Socket.1751lb # [ 1.747632] systemd[1]: Reached target Socket Units.1752lb # [ 1.750106] systemd[1]: Starting Create List of Static Device Nodes...1753worker1 # [ 1.749261] systemd[1]: Queued start job for default target Initrd Default Target.1754lb # [ 1.756894] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1755client # [ 1.763415] systemd[1]: Starting Journal Service...1756lb # [ 1.758099] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1757worker1 # [ 1.758279] systemd[1]: Created slice Slice /system/modprobe.1758worker1 # [ 1.759461] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1759worker1 # [ 1.760881] systemd[1]: Expecting device /dev/disk/by-label/nixos...1760worker1 # [ 1.761931] systemd[1]: Reached target Path Units.1761worker1 # [ 1.762739] systemd[1]: Reached target Slice Units.1762worker2 # [ 1.810406] systemd[1]: Starting Load Kernel Modules...1763worker1 # [ 1.763566] systemd[1]: Reached target Swaps.1764worker2 # [ 1.810525] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1765worker1 # [ 1.764382] systemd[1]: Reached target Timer Units.1766worker2 # [ 1.813650] systemd[1]: Starting Coldplug All udev Devices...1767lb # [ 1.767767] systemd[1]: Mounting Kernel Configuration File System...1768worker1 # [ 1.765382] systemd[1]: Listening on D-Bus System Message Bus Socket.1769worker1 # [ 1.766591] systemd[1]: Listening on Journal Socket (/dev/log).1770worker1 # [ 1.767756] systemd[1]: Listening on Journal Sockets.1771worker1 # [ 1.768821] systemd[1]: Listening on udev Control Socket.1772worker1 # [ 1.768943] systemd[1]: Listening on udev Kernel Socket.1773worker1 # [ 1.768966] systemd[1]: Reached target Socket Units.1774client # [ 1.781979] systemd[1]: Starting Load Kernel Modules...1775worker1 # [ 1.773197] systemd[1]: Starting Create List of Static Device Nodes...1776client # [ 1.782882] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1777lb # [ 1.777175] systemd[1]: Starting Journal Service...1778worker1 # [ 1.783619] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1779worker2 # [ 1.832166] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1780client # [ 1.792354] systemd[1]: Starting Coldplug All udev Devices...1781worker1 # [ 1.784990] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1782worker2 # [ 1.838322] systemd[1]: Finished Create List of Static Device Nodes.1783worker2 # [ 1.839496] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1784worker2 # [ 1.839781] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1785worker2 # [ 1.840002] systemd[1]: Mounted Kernel Configuration File System.1786lb # [ 1.802905] systemd[1]: Starting Load Kernel Modules...1787lb # [ 1.803013] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1788worker1 # [ 1.797555] systemd[1]: Mounting Kernel Configuration File System...1789client # [ 1.811327] systemd[1]: Finished Create List of Static Device Nodes.1790worker2 # [ 1.850688] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1791client # [ 1.812441] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1792client # [ 1.812738] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1793client # [ 1.812976] systemd[1]: Mounted Kernel Configuration File System.1794worker2 # [ 1.858357] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1795client # [ 1.823866] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1796worker1 # [ 1.816178] systemd[1]: Starting Journal Service...1797lb # [ 1.821391] systemd-journald[73]: Collecting audit messages is disabled.1798lb # [ 1.822393] systemd[1]: Starting Coldplug All udev Devices...1799worker2 # [ 1.872321] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01800worker1 # [ 1.825477] systemd[1]: Starting Load Kernel Modules...1801worker2 # [ 1.872566] [drm] features: -virgl +edid -resource_blob -host_visible1802worker2 # [ 1.872574] [drm] features: -context_init1803worker2 # [ 1.873303] [drm] number of scanouts: 11804client # [ 1.833413] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1805worker1 # [ 1.826390] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1806lb # [ 1.829520] systemd[1]: Finished Create List of Static Device Nodes.1807worker2 # [ 1.873321] [drm] number of cap sets: 01808worker2 # [ 1.880823] systemd-journald[73]: Collecting audit messages is disabled.1809worker2 # [ 1.882009] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1810client # [ 1.842942] systemd-journald[73]: Collecting audit messages is disabled.1811worker2 # [ 1.882028] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01812lb # [ 1.840567] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1813lb # [ 1.847986] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1814lb # [ 1.848389] systemd[1]: Mounted Kernel Configuration File System.1815lb # [ 1.851255] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1816client # [ 1.855131] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1817client # [ 1.859855] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01818client # [ 1.860093] [drm] features: -virgl +edid -resource_blob -host_visible1819client # [ 1.860101] [drm] features: -context_init1820client # [ 1.860797] [drm] number of scanouts: 11821client # [ 1.860814] [drm] number of cap sets: 01822worker1 # [ 1.860431] systemd[1]: Starting Coldplug All udev Devices...1823worker2 # [ 1.910820] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1824worker2 # [ 1.913363] systemd[1]: Starting Create Static Device Nodes in /dev...1825client # [ 1.879140] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1826client # [ 1.879162] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01827worker2 # [ 1.921280] Console: switching to colour frame buffer device 160x501828client # [ 1.891708] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1829worker1 # [ 1.881256] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1830worker1 # [ 1.881568] systemd[1]: Finished Create List of Static Device Nodes.1831worker1 # [ 1.882742] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1832worker1 # [ 1.883067] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1833worker1 # [ 1.888378] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1834worker1 # [ 1.889555] systemd[1]: Mounted Kernel Configuration File System.1835lb # [ 1.892247] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1836client # [ 1.899415] systemd[1]: Starting Create Static Device Nodes in /dev...1837lb # [ 1.898787] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1838worker1 # [ 1.899647] systemd-journald[73]: Collecting audit messages is disabled.1839lb # [ 1.904109] systemd[1]: Starting Create Static Device Nodes in /dev...1840worker2 # [ 1.954794] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1841lb # [ 1.911689] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1842worker1 # [ 1.904002] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1843worker1 # [ 1.908575] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01844worker1 # [ 1.908814] [drm] features: -virgl +edid -resource_blob -host_visible1845worker2 # [ 1.958148] systemd[1]: Finished Create Static Device Nodes in /dev.1846worker1 # [ 1.908823] [drm] features: -context_init1847worker1 # [ 1.909534] [drm] number of scanouts: 11848worker2 # [ 1.958391] systemd[1]: Reached target Preparation for Local File Systems.1849worker1 # [ 1.909552] [drm] number of cap sets: 01850worker2 # [ 1.958425] systemd[1]: Reached target Local File Systems.1851client # [ 1.919423] Console: switching to colour frame buffer device 160x501852client # [ 1.923701] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1853worker2 # [ 1.963997] systemd[1]: Starting Rule-based Manager for Device Events and Files...1854lb # [ 1.923600] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01855worker2 # [ 1.978566] systemd[1]: Finished Load Kernel Modules.1856client # [ 1.939277] systemd[1]: Finished Create Static Device Nodes in /dev.1857client # [ 1.939496] systemd[1]: Reached target Preparation for Local File Systems.1858lb # [ 1.935583] [drm] features: -virgl +edid -resource_blob -host_visible1859client # [ 1.939531] systemd[1]: Reached target Local File Systems.1860lb # [ 1.935606] [drm] features: -context_init1861lb # [ 1.936386] [drm] number of scanouts: 11862lb # [ 1.936404] [drm] number of cap sets: 01863worker1 # [ 1.928265] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1864client # [ 1.944335] systemd[1]: Starting Rule-based Manager for Device Events and Files...1865worker1 # [ 1.928287] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01866lb # [ 1.940235] systemd[1]: Finished Create Static Device Nodes in /dev.1867lb # [ 1.940599] systemd[1]: Reached target Preparation for Local File Systems.1868lb # [ 1.940626] systemd[1]: Reached target Local File Systems.1869client # [ 1.951566] systemd[1]: Finished Load Kernel Modules.1870lb # [ 1.944596] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1871worker2 # [ 1.986347] systemd[1]: Starting Apply Kernel Variables...1872lb # [ 1.944615] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01873lb # [ 1.956059] systemd[1]: Starting Rule-based Manager for Device Events and Files...1874client # [ 1.963556] systemd[1]: Starting Apply Kernel Variables...1875worker1 # [ 1.956618] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1876worker1 # [ 1.959192] systemd[1]: Starting Create Static Device Nodes in /dev...1877worker1 # [ 1.961055] Console: switching to colour frame buffer device 160x501878worker2 # [ 2.026839] systemd[1]: Finished Apply Kernel Variables.1879lb # [ 1.976025] Console: switching to colour frame buffer device 160x501880worker1 # [ 1.984592] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1881client # [ 2.007712] systemd[1]: Finished Apply Kernel Variables.1882worker1 # [ 2.001220] systemd[1]: Finished Load Kernel Modules.1883worker1 # [ 2.003433] systemd[1]: Starting Apply Kernel Variables...1884lb # [ 2.011497] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1885worker2 # [ 2.060267] systemd[1]: Started Journal Service.1886lb # [ 2.016152] systemd[1]: Finished Load Kernel Modules.1887worker2 # [ 2.052273] systemd-modules-load[74]: Inserted module 'dm_mod'1888lb # [ 2.009324] systemd-modules-load[75]: Inserted module 'dm_mod'1889worker1 # [ 2.021336] systemd[1]: Finished Create Static Device Nodes in /dev.1890worker2 # [ 2.053451] systemd-modules-load[74]: Module 'virtio_balloon' is built in1891worker1 # [ 2.021550] systemd[1]: Reached target Preparation for Local File Systems.1892worker1 # [ 2.021587] systemd[1]: Reached target Local File Systems.1893worker2 # [ 2.054540] systemd-modules-load[74]: Module 'virtio_console' is built in1894client # [ 2.031294] systemd[1]: Started Journal Service.1895worker2 # [ 2.055604] systemd-modules-load[74]: Inserted module 'virtio_gpu'1896lb # [ 2.026201] systemd[1]: Starting Apply Kernel Variables...1897lb # [ 2.026379] systemd[1]: Started Journal Service.1898client # [ 2.020310] systemd-modules-load[74]: Inserted module 'dm_mod'1899client # [ 2.021539] systemd-modules-load[74]: Module 'virtio_balloon' is built in1900lb # [ 2.019803] systemd-modules-load[75]: Module 'virtio_balloon' is built in1901lb # [ 2.024336] systemd-modules-load[75]: Module 'virtio_console' is built in1902worker2 # [ 2.068407] systemd-modules-load[74]: Module 'virtio_rng' is built in1903lb # [ 2.025388] systemd-modules-load[75]: Inserted module 'virtio_gpu'1904client # [ 2.032329] systemd-modules-load[74]: Module 'virtio_console' is built in1905worker2 # [ 2.069524] systemd[1]: Starting Create System Files and Directories...1906lb # [ 2.026363] systemd-modules-load[75]: Module 'virtio_rng' is built in1907client # [ 2.033457] systemd-modules-load[74]: Inserted module 'virtio_gpu'1908client # [ 2.034423] systemd-modules-load[74]: Module 'virtio_rng' is built in1909worker2 # [ 2.070864] systemd-udevd[80]: Using default interface naming scheme 'v261'.1910worker1 # [ 2.036331] systemd[1]: Starting Rule-based Manager for Device Events and Files...1911client # [ 2.035464] systemd[1]: Starting Create System Files and Directories...1912lb # [ 2.040129] systemd[1]: Starting Create System Files and Directories...1913client # [ 2.052718] systemd-udevd[80]: Using default interface naming scheme 'v261'.1914worker2 # [ 2.091757] systemd[1]: Finished Create System Files and Directories.1915lb # [ 2.052504] systemd-udevd[80]: Using default interface naming scheme 'v261'.1916worker1 # [ 2.068326] systemd[1]: Finished Apply Kernel Variables.1917lb # [ 2.057405] systemd[1]: Finished Apply Kernel Variables.1918worker1 # [ 2.072908] systemd[1]: Started Journal Service.1919client # [ 2.069976] systemd[1]: Finished Create System Files and Directories.1920worker1 # [ 2.064395] systemd-modules-load[74]: Inserted module 'dm_mod'1921worker2 # [ 2.111398] systemd[1]: Started Rule-based Manager for Device Events and Files.1922worker1 # [ 2.065633] systemd-modules-load[74]: Module 'virtio_balloon' is built in1923worker1 # [ 2.075874] systemd-modules-load[74]: Module 'virtio_console' is built in1924worker1 # [ 2.077199] systemd-modules-load[74]: Inserted module 'virtio_gpu'1925worker1 # [ 2.078153] systemd-modules-load[74]: Module 'virtio_rng' is built in1926client # [ 2.088159] systemd[1]: Started Rule-based Manager for Device Events and Files.1927worker1 # [ 2.079174] systemd[1]: Starting Create System Files and Directories...1928lb # [ 2.094303] systemd[1]: Finished Create System Files and Directories.1929lb # [ 2.103095] systemd[1]: Started Rule-based Manager for Device Events and Files.1930worker1 # [ 2.111424] systemd[1]: Finished Create System Files and Directories.1931worker1 # [ 2.121223] systemd-udevd[81]: Using default interface naming scheme 'v261'.1932worker2 # [ 2.168129] systemd[1]: Starting Virtual Console Setup...1933client # [ 2.144690] systemd[1]: Starting Virtual Console Setup...1934worker1 # [ 2.148867] systemd[1]: Started Rule-based Manager for Device Events and Files.1935lb # [ 2.158500] systemd[1]: Starting Virtual Console Setup...1936worker2 # [ 2.216508] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1937worker2 # [ 2.219972] systemd[1]: Finished Virtual Console Setup.1938client # [ 2.196527] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1939client # [ 2.200176] systemd[1]: Finished Virtual Console Setup.1940lb # [ 2.212492] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1941worker1 # [ 2.207964] systemd[1]: Starting Virtual Console Setup...1942lb # [ 2.216055] systemd[1]: Finished Virtual Console Setup.1943worker1 # [ 2.256481] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1944worker1 # [ 2.259758] systemd[1]: Finished Virtual Console Setup.1945worker2 # [ 2.811805] systemd[1]: Finished Coldplug All udev Devices.1946worker2 # [ 2.812924] systemd[1]: Reached target System Initialization.1947worker2 # [ 2.813742] systemd[1]: Reached target Basic System.1948client # [ 2.813373] systemd[1]: Finished Coldplug All udev Devices.1949lb # [ 2.806946] systemd[1]: Finished Coldplug All udev Devices.1950client # [ 2.814419] systemd[1]: Reached target System Initialization.1951lb # [ 2.807899] systemd[1]: Reached target System Initialization.1952client # [ 2.815261] systemd[1]: Reached target Basic System.1953lb # [ 2.808873] systemd[1]: Reached target Basic System.1954worker1 # [ 2.840858] systemd[1]: Finished Coldplug All udev Devices.1955worker1 # [ 2.843033] systemd[1]: Reached target System Initialization.1956worker1 # [ 2.843918] systemd[1]: Reached target Basic System.1957worker2 # [ 2.952298] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1958worker2 # [ 2.964400] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1959lb # [ 2.940007] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960worker2 # [ 2.990974] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1961client # [ 2.974324] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1962client # [ 2.980193] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1963lb # [ 2.982140] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1964client # [ 2.992864] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1965lb # [ 2.989954] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1966worker1 # [ 2.995387] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1967worker2 # [ 3.048142] systemd[1]: Found device /dev/disk/by-label/nixos.1968worker2 # [ 3.051452] systemd[1]: Reached target Initrd Root Device.1969worker2 # [ 3.056971] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1970worker1 # [ 3.018950] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1971worker1 # [ 3.029026] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1972lb # [ 3.049174] systemd[1]: Found device /dev/disk/by-label/nixos.1973lb # [ 3.053026] systemd[1]: Reached target Initrd Root Device.1974client # [ 3.062302] systemd[1]: Found device /dev/disk/by-label/nixos.1975client # [ 3.066658] systemd[1]: Reached target Initrd Root Device.1976lb # [ 3.060117] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1977worker2 # [ 3.105975] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1978client # [ 3.072557] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1979worker2 # [ 3.114118] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1980worker2 # [ 3.128243] systemd[1]: Mounting /sysroot...1981worker1 # [ 3.100786] systemd[1]: Found device /dev/disk/by-label/nixos.1982lb # [ 3.106977] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1983worker1 # [ 3.108086] systemd[1]: Reached target Initrd Root Device.1984worker1 # [ 3.111303] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985lb # [ 3.114887] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1986worker2 # [ 3.177294] EXT4-fs (vda): mounted filesystem fb96e099-efc4-4c5b-9589-f5efd9479053 r/w with ordered data mode. Quota mode: none.1987client # [ 3.124738] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1988worker2 # [ 3.164925] systemd[1]: Mounted /sysroot.1989worker2 # [ 3.166815] systemd[1]: Reached target Initrd Root File System.1990client # [ 3.130460] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1991worker2 # [ 3.168201] systemd[1]: Starting Mountpoints Configured in the Real Root...1992lb # [ 3.128117] systemd[1]: Mounting /sysroot...1993client # [ 3.136135] systemd[1]: Mounting /sysroot...1994worker2 # [ 3.195676] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1995lb # [ 3.166901] EXT4-fs (vda): mounted filesystem c2eb829f-f19e-4e6f-ae63-b26aa4c7dcce r/w with ordered data mode. Quota mode: none.1996lb # [ 3.158255] systemd[1]: Mounted /sysroot.1997lb # [ 3.159418] systemd[1]: Reached target Initrd Root File System.1998worker1 # [ 3.156710] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1999worker2 # [ 3.204114] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...2000worker2 # [ 3.206473] systemd[1]: Reloading...
loading 1,503 hidden lines…
3504lb # [ 19.586751] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3505lb # [ 19.589456] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3506lb # [ 19.591597] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit3507lb # [ 19.594627] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3508lb # [ 19.597876] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo3509lb # [ 19.599525] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3510client # [ 19.613144] dhcpcd[544]: eth0: leased 10.0.2.15 for 86400 seconds3511client # [ 19.616589] dhcpcd[544]: eth0: adding route to 10.0.2.0/243512lb # [ 19.604406] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port3513client # [ 19.618468] dhcpcd[544]: eth0: adding default route via 10.0.2.23514lb # [ 19.613461] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3515lb # [ 19.615818] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3516lb # [ 19.618012] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy3517lb # [ 19.621317] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3518lb # [ 19.623837] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3519lb # [ 19.628529] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3520lb # [ 19.630837] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3521lb # [ 19.632905] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config3522lb # [ 19.636642] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter3523lb # [ 19.638672] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator3524lb # [ 19.641603] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection3525lb # [ 19.643486] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3526lb # [ 19.646590] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3527lb # [ 19.651543] envoy[917]: [2026-09-21 09:34:50.024][917][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3528lb # [ 19.653777] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3529lb # [ 19.656131] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3530lb # [ 19.661212] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3531lb # [ 19.664238] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3532lb # [ 19.666028] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3533lb # [ 19.667959] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3534lb # [ 19.670976] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3535worker2 # [ 19.715281] dhcpcd[645]: eth0: leased 10.0.2.15 for 86400 seconds3536lb # [ 19.673234] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default3537lb # [ 19.674977] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme3538lb # [ 19.678552] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr3539worker2 # [ 19.722146] dhcpcd[645]: eth0: adding route to 10.0.2.0/243540lb # [ 19.681025] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case3541worker2 # [ 19.725320] dhcpcd[645]: eth0: adding default route via 10.0.2.23542lb # [ 19.683506] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3543lb # [ 19.686133] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder3544lb # [ 19.688550] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3545lb # [ 19.694209] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3546lb # [ 19.698345] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3547lb # [ 19.700272] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin3548lb # [ 19.703738] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3549lb # [ 19.708729] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable3550lb # [ 19.710826] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default3551lb # [ 19.714595] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy3552lb # [ 19.728462] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23553lb # [ 19.730335] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3554lb # [ 19.733658] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3555lb # [ 19.735618] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3556lb # [ 19.737607] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23557lb # [ 19.740226] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13558lb # [ 19.742450] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic3559lb # [ 19.744401] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3560lb # [ 19.747444] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3561lb # [ 19.750876] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3562lb # [ 19.753303] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3563lb # [ 19.755804] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3564lb # [ 19.759606] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3565lb # [ 19.761528] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift3566lb # [ 19.765219] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action3567lb # [ 19.767463] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3568lb # [ 19.769819] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3569client # [ 19.787719] systemd[1]: Started DHCP Client.3570lb # [ 19.771975] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3571client # [ 19.791378] systemd[1]: Reached target Multi-User System.3572lb # [ 19.783665] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3573client # [ 19.793269] systemd[1]: Startup finished in 976ms (kernel) + 5.332s (initrd) + 13.483s (userspace) = 19.792s.3574lb # [ 19.785713] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3575lb # [ 19.788562] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal3576lb # [ 19.790593] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3577lb # [ 19.792716] envoy[917]: [2026-09-21 09:34:50.025][917][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3578lb # [ 19.794747] envoy[917]: [2026-09-21 09:34:50.045][917][info][main] [source/server/server.cc:531] HTTP header map info:3579lb # [ 19.796985] envoy[917]: [2026-09-21 09:34:50.065][917][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3580lb # [ 19.812985] envoy[917]: [2026-09-21 09:34:50.065][917][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3581lb # [ 19.814562] envoy[917]: [2026-09-21 09:34:50.065][917][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3582lb # [ 19.823963] envoy[917]: [2026-09-21 09:34:50.065][917][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3583lb # [ 19.825990] envoy[917]: [2026-09-21 09:34:50.317][917][info][main] [source/server/server.cc:968] runtime: {}3584lb # [ 19.827425] envoy[917]: [2026-09-21 09:34:50.334][917][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3585lb # [ 19.829782] envoy[917]: [2026-09-21 09:34:50.334][917][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013586lb # [ 19.831601] envoy[917]: [2026-09-21 09:34:50.334][917][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013587lb # [ 19.835279] envoy[917]: [2026-09-21 09:34:50.340][917][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3588lb # [ 19.837050] envoy[917]: [2026-09-21 09:34:50.340][917][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3589lb # [ 19.838782] envoy[917]: [2026-09-21 09:34:50.340][917][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3590worker2 # [ 19.907937] systemd[1]: Started DHCP Client.3591lb # [ 19.865486] envoy[917]: [2026-09-21 09:34:50.501][917][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3592worker2 # [ 19.912583] systemd[1]: Reached target Multi-User System.3593worker2 # [ 19.913467] systemd[1]: Startup finished in 959ms (kernel) + 5.386s (initrd) + 13.565s (userspace) = 19.910s.3594worker2 # [ 19.917735] systemd[1]: Reached target Network is Online.3595lb # [ 19.875060] envoy[917]: [2026-09-21 09:34:50.511][917][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3596worker2 # [ 19.920807] systemd[1]: Starting scheduler-token.service...3597lb # [ 19.878002] envoy[917]: [2026-09-21 09:34:50.514][917][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3598lb # [ 19.881563] envoy[917]: [2026-09-21 09:34:50.514][917][info][main] [source/server/server.cc:1077] starting main dispatch loop3599worker1 # [ 19.889467] systemd[1]: Starting Nix worker-protocol over gRPC...3600worker2 # [ 20.020539] systemd[1]: scheduler-token.service: Deactivated successfully.3601worker2 # [ 20.022522] systemd[1]: Finished scheduler-token.service.3602worker2 # [ 20.026555] systemd[1]: scheduler-token.service: Consumed 31ms CPU time over 101ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3603worker2 # [ 20.030058] systemd[1]: Starting Nix worker-protocol over gRPC...3604worker1 # [ 20.169969] nix-grpc-daemon[776]: ts=2026-09-21T09:34:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3605worker1 # [ 20.186829] nix-grpc-daemon[776]: ts=2026-09-21T09:34:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3606worker1 # [ 20.189347] nix-grpc-daemon[776]: ts=2026-09-21T09:34:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13607worker2 # [ 20.264152] nix-grpc-daemon[777]: ts=2026-09-21T09:34:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3608worker1 # [ 20.229333] nix-grpc-daemon[776]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR3609worker1 # [ 20.230923] nix-grpc-daemon[776]: E0000 00:00:1789983290.856100 776 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3610worker2 # [ 20.279239] nix-grpc-daemon[777]: ts=2026-09-21T09:34:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3611worker2 # [ 20.281989] nix-grpc-daemon[777]: ts=2026-09-21T09:34:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13612worker1 # [ 20.245564] nix-grpc-daemon[776]: E0000 00:00:1789983290.872388 776 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3613worker1 # [ 20.250466] nix-grpc-daemon[776]: ts=2026-09-21T09:34:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3614worker1 # [ 20.254630] systemd[1]: Started Nix worker-protocol over gRPC.3615worker2 # [ 20.323279] nix-grpc-daemon[777]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR3616worker2 # [ 20.325202] nix-grpc-daemon[777]: E0000 00:00:1789983290.932920 777 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3617worker1 # [ 20.284091] nix-grpc-daemon[776]: E0000 00:00:1789983290.910090 818 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3618worker2 # [ 20.338872] nix-grpc-daemon[777]: E0000 00:00:1789983290.948539 777 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3619worker2 # [ 20.342773] nix-grpc-daemon[777]: ts=2026-09-21T09:34:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3620lb # [ 20.305247] envoy[917]: [2026-09-21 09:34:50.941][917][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3621worker2 # [ 20.348992] systemd[1]: Started Nix worker-protocol over gRPC.3622lb # [ 20.313988] envoy[917]: [2026-09-21 09:34:50.941][917][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3623worker1 # [ 20.318144] nix-grpc-daemon[776]: E0000 00:00:1789983290.944783 818 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3624lb # [ 20.321256] envoy[917]: [2026-09-21 09:34:50.941][917][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3625worker1 # [ 20.324099] nix-grpc-daemon[776]: E0000 00:00:1789983290.948318 818 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3626worker2 # [ 20.374515] nix-grpc-daemon[777]: E0000 00:00:1789983290.983267 819 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3627lb # [ 20.331832] envoy[917]: [2026-09-21 09:34:50.968][917][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3628worker1 # [ 20.349862] nix-grpc-daemon[776]: E0000 00:00:1789983290.976820 818 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3629worker2 # [ 20.397560] nix-grpc-daemon[777]: E0000 00:00:1789983291.007429 819 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3630worker2 # [ 20.400682] nix-grpc-daemon[777]: E0000 00:00:1789983291.010758 819 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3631worker1 # [ 20.367679] nix-grpc-daemon[776]: ts=2026-09-21T09:34:50Z level=info event=scheduler_connected addr=lb:500513632worker1 # [ 20.371346] nix-grpc-daemon[776]: ts=2026-09-21T09:34:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3633worker2 # [ 20.427001] nix-grpc-daemon[777]: E0000 00:00:1789983291.035976 819 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3634worker2 # [ 20.443869] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_connected addr=lb:500513635worker2 # [ 20.445862] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3636worker1 # [ 20.573349] nix-grpc-daemon[776]: ts=2026-09-21T09:34:51Z level=info event=scheduler_connected addr=lb:500513637worker1 # [ 20.579064] nix-grpc-daemon[776]: ts=2026-09-21T09:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3638worker2 # [ 20.648771] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_connected addr=lb:500513639worker2 # [ 20.654141] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3640worker1 # [ 20.780312] nix-grpc-daemon[776]: ts=2026-09-21T09:34:51Z level=info event=scheduler_connected addr=lb:500513641worker1 # [ 20.786361] nix-grpc-daemon[776]: ts=2026-09-21T09:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3642worker2 # [ 20.855748] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_connected addr=lb:500513643worker2 # [ 20.861526] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3644worker1 # [ 20.987618] nix-grpc-daemon[776]: ts=2026-09-21T09:34:51Z level=info event=scheduler_connected addr=lb:500513645worker1 # [ 20.994462] nix-grpc-daemon[776]: ts=2026-09-21T09:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3646worker2 # [ 21.063186] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_connected addr=lb:500513647worker2 # [ 21.068899] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3648worker1 # [ 21.198815] nix-grpc-daemon[776]: ts=2026-09-21T09:34:51Z level=info event=scheduler_connected addr=lb:500513649worker1 # [ 21.203198] nix-grpc-daemon[776]: ts=2026-09-21T09:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3650worker2 # [ 21.273283] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_connected addr=lb:500513651worker2 # [ 21.277508] nix-grpc-daemon[777]: ts=2026-09-21T09:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3652worker1 # [ 21.399807] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513653worker1 # [ 21.405737] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3654worker2 # [ 21.474553] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513655worker2 # [ 21.480253] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3656worker1 # [ 21.607508] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513657worker1 # [ 21.613422] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3658worker2 # [ 21.681697] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513659worker2 # [ 21.687290] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3660worker1 # [ 21.815000] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513661worker1 # [ 21.820972] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3662worker2 # [ 21.889271] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513663worker2 # [ 21.894664] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker1 # [ 22.022177] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513665worker1 # [ 22.028085] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3666worker2 # [ 22.096723] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513667worker2 # [ 22.102649] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker1 # [ 22.233219] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 22.237619] nix-grpc-daemon[776]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3670worker2 # [ 22.307196] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_connected addr=lb:500513671worker2 # [ 22.311606] nix-grpc-daemon[777]: ts=2026-09-21T09:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker1 # [ 22.433636] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513673worker1 # [ 22.439519] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3674worker2 # [ 22.509028] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513675worker2 # [ 22.516457] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker1 # [ 22.641001] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513677worker1 # [ 22.646465] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3678worker2 # [ 22.718812] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513679worker2 # [ 22.724822] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker1 # [ 22.847435] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513681worker1 # [ 22.852742] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3682worker2 # [ 22.925427] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513683worker2 # [ 22.930223] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker1 # [ 23.053559] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513685worker1 # [ 23.058707] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3686worker2 # [ 23.131788] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513687worker2 # [ 23.137107] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker1 # [ 23.261434] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513689worker1 # [ 23.265538] nix-grpc-daemon[776]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3690worker2 # [ 23.340162] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_connected addr=lb:500513691worker2 # [ 23.344406] nix-grpc-daemon[777]: ts=2026-09-21T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker1 # [ 23.462057] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513693worker1 # [ 23.467245] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3694worker2 # [ 23.540757] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513695worker2 # [ 23.546120] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3696worker1 # [ 23.668101] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513697worker1 # [ 23.673186] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3698worker2 # [ 23.747243] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513699worker2 # [ 23.752679] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker1 # [ 23.874096] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513701worker1 # [ 23.879341] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3702worker2 # [ 23.953997] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513703worker2 # [ 23.959013] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3704worker1 # [ 24.080564] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513705worker1 # [ 24.085792] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3706worker2 # [ 24.160107] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513707worker2 # [ 24.165077] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3708worker1 # [ 24.288761] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513709worker1 # [ 24.293030] nix-grpc-daemon[776]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3710worker2 # [ 24.367696] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_connected addr=lb:500513711worker2 # [ 24.371851] nix-grpc-daemon[777]: ts=2026-09-21T09:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3712worker1 # [ 24.489229] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_connected addr=lb:500513713worker1 # [ 24.494645] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3714worker2 # [ 24.568249] nix-grpc-daemon[777]: ts=2026-09-21T09:34:55Z level=info event=scheduler_connected addr=lb:500513715worker2 # [ 24.573030] nix-grpc-daemon[777]: ts=2026-09-21T09:34:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716worker1 # [ 24.695450] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_connected addr=lb:500513717worker1 # [ 24.700923] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3718worker2 # [ 24.774251] nix-grpc-daemon[777]: ts=2026-09-21T09:34:55Z level=info event=scheduler_connected addr=lb:500513719worker2 # [ 24.779308] nix-grpc-daemon[777]: ts=2026-09-21T09:34:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3720worker1 # [ 24.901841] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_connected addr=lb:500513721worker1 # [ 24.907190] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3722worker2 # [ 24.980404] nix-grpc-daemon[777]: ts=2026-09-21T09:34:55Z level=info event=scheduler_connected addr=lb:500513723worker2 # [ 24.985273] nix-grpc-daemon[777]: ts=2026-09-21T09:34:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3724worker1 # [ 25.108261] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_connected addr=lb:500513725worker1 # [ 25.113601] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3726worker2 # [ 25.186281] nix-grpc-daemon[777]: ts=2026-09-21T09:34:55Z level=info event=scheduler_connected addr=lb:500513727worker2 # [ 25.191367] nix-grpc-daemon[777]: ts=2026-09-21T09:34:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3728worker1 # [ 25.318163] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_connected addr=lb:500513729worker1 # [ 25.322340] nix-grpc-daemon[776]: ts=2026-09-21T09:34:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3730worker2 # [ 25.394203] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513731worker2 # [ 25.398283] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3732worker1 # [ 25.518447] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 25.523751] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3734worker2 # [ 25.594773] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513735worker2 # [ 25.599888] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3736worker1 # [ 25.724682] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513737worker1 # [ 25.729912] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3738worker2 # [ 25.800884] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513739worker2 # [ 25.805814] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3740worker1 # [ 25.930770] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 25.936151] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3742worker2 # [ 26.007176] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513743worker2 # [ 26.012326] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744worker1 # [ 26.136959] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513745worker1 # [ 26.142087] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3746worker2 # [ 26.213270] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513747worker2 # [ 26.218286] nix-grpc-daemon[777]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3748worker1 # [ 26.344709] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_connected addr=lb:500513749worker1 # [ 26.348787] nix-grpc-daemon[776]: ts=2026-09-21T09:34:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3750worker2 # [ 26.423043] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513751worker2 # [ 26.427273] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3752worker1 # [ 26.545075] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513753worker1 # [ 26.550245] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3754worker2 # [ 26.621904] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513755worker2 # [ 26.627026] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3756worker1 # [ 26.751042] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513757worker1 # [ 26.756633] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3758worker2 # [ 26.828140] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513759worker2 # [ 26.833080] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3760worker1 # [ 26.957404] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513761worker1 # [ 26.962444] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3762worker2 # [ 27.033999] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513763worker2 # [ 27.039010] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3764worker1 # [ 27.163358] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513765worker1 # [ 27.168511] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3766worker2 # [ 27.240152] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513767worker2 # [ 27.244962] nix-grpc-daemon[777]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3768worker1 # [ 27.370968] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_connected addr=lb:500513769worker1 # [ 27.375095] nix-grpc-daemon[776]: ts=2026-09-21T09:34:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3770worker2 # [ 27.447342] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_connected addr=lb:500513771worker2 # [ 27.451508] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3772worker1 # [ 27.571409] nix-grpc-daemon[776]: ts=2026-09-21T09:34:58Z level=info event=scheduler_connected addr=lb:500513773worker1 # [ 27.576692] nix-grpc-daemon[776]: ts=2026-09-21T09:34:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774worker2 # [ 27.648177] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_connected addr=lb:500513775worker2 # [ 27.653267] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3776lb # [ 27.723612] systemd[1]: niks3.service: Scheduled restart job, restart counter is at 1.3777lb # [ 27.743822] systemd[1]: Starting niks3 server...3778worker1 # [ 27.777530] nix-grpc-daemon[776]: ts=2026-09-21T09:34:58Z level=info event=scheduler_connected addr=lb:500513779worker1 # [ 27.782572] nix-grpc-daemon[776]: ts=2026-09-21T09:34:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3780worker2 # [ 27.854414] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_connected addr=lb:500513781worker2 # [ 27.859614] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3782lb # [ 27.844451] postgres[979]: [979] ERROR: relation "goose_db_version" does not exist at character 363783lb # [ 27.845753] postgres[979]: [979] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3784lb # [ 27.883146] niks3-server[974]: 2026/09/21 09:34:58 OK 20241026095416_initial_model.sql (22ms)3785lb # [ 27.886266] niks3-server[974]: 2026/09/21 09:34:58 OK 20251210153512_drop_unused_gin_index.sql (3.41ms)3786lb # [ 27.891892] niks3-server[974]: 2026/09/21 09:34:58 OK 20251218171726_add_pins.sql (5.56ms)3787lb # [ 27.896994] niks3-server[974]: 2026/09/21 09:34:58 OK 20260628120000_add_object_size_and_stats.sql (5.11ms)3788lb # [ 27.902044] niks3-server[974]: 2026/09/21 09:34:58 OK 20260905000000_add_claims.sql (3.51ms)3789lb # [ 27.906569] niks3-server[974]: 2026/09/21 09:34:58 OK 20260920000000_drop_claims.sql (4.41ms)3790lb # [ 27.908672] niks3-server[974]: 2026/09/21 09:34:58 goose: successfully migrated database to version: 202609200000003791lb # [ 27.913156] niks3-server[974]: 2026/09/21 09:34:58 OK 1_commit_pending_closure.sql (6.56ms)3792lb # [ 27.915674] niks3-server[974]: 2026/09/21 09:34:58 OK 2_object_stats_trigger.sql (2.4ms)3793lb # [ 27.917166] niks3-server[974]: 2026/09/21 09:34:58 goose: up to current file version: 23794lb # [ 27.920484] niks3-server[974]: 2026/09/21 09:34:58 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3795lb # [ 27.965282] niks3-server[974]: 2026/09/21 09:34:58 INFO Read proxy enabled — serving cache objects from S33796lb # [ 27.966585] niks3-server[974]: 2026/09/21 09:34:58 INFO Using socket-activated listener address=0.0.0.0:57513797lb # [ 27.970327] niks3-server[974]: 2026/09/21 09:34:58 INFO systemd watchdog enabled interval=15s3798lb # [ 27.974295] systemd[1]: Started niks3 server.3799lb # [ 27.974984] niks3-server[974]: 2026/09/21 09:34:58 INFO Starting HTTP server address=0.0.0.0:57513800lb # [ 27.980267] niks3-server[974]: 2026/09/21 09:34:58 INFO lead: acquired remote=192.168.1.4:556623801worker2 # [ 28.025237] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_take_over3802worker1 # [ 27.983290] nix-grpc-daemon[776]: ts=2026-09-21T09:34:58Z level=info event=scheduler_connected addr=lb:500513803worker1 # [ 27.988900] nix-grpc-daemon[776]: ts=2026-09-21T09:34:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3804worker2 # [ 28.060765] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_connected addr=lb:500513805worker2 # [ 28.065782] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3806worker1 # [ 28.189664] nix-grpc-daemon[776]: ts=2026-09-21T09:34:58Z level=info event=scheduler_connected addr=lb:500513807worker1 # [ 28.195133] nix-grpc-daemon[776]: ts=2026-09-21T09:34:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3808worker2 # [ 28.266930] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_connected addr=lb:500513809worker2 # [ 28.271990] nix-grpc-daemon[777]: ts=2026-09-21T09:34:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3810worker1 # [ 28.397126] nix-grpc-daemon[776]: ts=2026-09-21T09:34:59Z level=info event=scheduler_connected addr=lb:500513811worker2 # [ 28.449376] nix-grpc-daemon[777]: ts=2026-09-21T09:34:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:450703812worker2 # [ 28.455129] nix-grpc-daemon[777]: ts=2026-09-21T09:34:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03813worker2 # [ 28.472984] nix-grpc-daemon[777]: ts=2026-09-21T09:34:59Z level=info event=scheduler_connected addr=lb:500513814worker2 # [ 28.492148] nix-grpc-daemon[777]: ts=2026-09-21T09:34:59Z level=info event=oidc_jwks provider=mock keys=13815worker2 # [ 28.497632] nix-grpc-daemon[777]: ts=2026-09-21T09:34:59Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:450703816worker2 # [ 28.502956] nix-grpc-daemon[777]: ts=2026-09-21T09:34:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03817lb: (finished: waiting for unit niks3.service, in 15.48 seconds)3818lb: waiting for TCP port 5751 on localhost3819lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3820lb: (finished: waiting for TCP port 5751 on localhost, in 0.10 seconds)3821lb: waiting for unit envoy.service3822lb: (finished: waiting for unit envoy.service, in 0.07 seconds)3823lb: waiting for TCP port 50051 on localhost3824lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3825lb # [ 29.069258] envoy[917]: conn peer=[::1]:37028 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33826lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3827worker2 # [ 29.291259] systemd[1]: Starting scheduler-token.service...3828worker2 # [ 29.383584] systemd[1]: scheduler-token.service: Deactivated successfully.3829worker2 # [ 29.386548] systemd[1]: Finished scheduler-token.service.3830worker2 # [ 29.387482] systemd[1]: scheduler-token.service: Consumed 31ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3831lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3832lb: must succeed: curl -sf localhost:9901/clusters3833lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)3834lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.09 seconds)3835subtest: one node schedules and both builders hold a WorkerSession on it3836lb: must succeed: curl -sf localhost:9901/clusters3837lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3838worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3839worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)3840lb: must succeed: curl -sf localhost:9901/clusters3841lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3842lb: must succeed: curl -sf localhost:9901/clusters3843lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3844lb: must succeed: curl -sf localhost:9901/clusters3845lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3846lb: must succeed: curl -sf localhost:9901/clusters3847lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)3848worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3849worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)3850worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3851worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3852lb: must succeed: curl -sf localhost:9901/clusters3853lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3854lb: must succeed: curl -sf localhost:9901/clusters3855lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3856worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3857worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3858worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3859worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)3860worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3861worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3862worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3863worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3864(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.85 seconds)3865subtest: balancer auth: client cert, bearer token, nothing, unknown CN3866worker1 # [ 30.679113] nix-grpc-daemon[776]: ts=2026-09-21T09:35:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520163867worker1 # [ 30.698230] systemd[1]: Started Nix Daemon.3868worker1 # [ 30.998250] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3869worker1 # [ 31.023231] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5)3870worker1 # [ 31.029969] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 2/5)3871worker1 # [ 31.073450] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 3/5)3872worker1 # [ 31.414067] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 689 ms (attempt 4/5)3873worker1 # [ 32.106523] nix-grpc-daemon[776]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3874worker1 # [ 32.112284] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3875worker1 # [ 32.140426] nix-grpc-daemon[776]: ts=2026-09-21T09:35:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:520163876worker1 # [ 32.156879] nix-grpc-daemon[776]: ts=2026-09-21T09:35:02Z level=debug event=path_info_miss path=kn580alm8i65l8xnw33vjbcfrdhrbapf-nixos-system-client-test error="error: path '/nix/store/kn580alm8i65l8xnw33vjbcfrdhrbapf-nixos-system-client-test' is required, but there is no substituter that can build it"3877worker2 # [ 32.225340] nix-grpc-daemon[777]: ts=2026-09-21T09:35:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:439603878worker2 # [ 32.244779] systemd[1]: Started Nix Daemon.3879worker2 # [ 32.592574] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3880worker2 # [ 32.627210] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 43 ms (attempt 1/5)3881worker2 # [ 32.670454] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 148 ms (attempt 2/5)3882worker2 # [ 32.818761] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 295 ms (attempt 3/5)3883worker2 # [ 33.121389] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 172 ms (attempt 4/5)3884worker2 # [ 33.303844] nix-grpc-daemon[777]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3885worker2 # [ 33.309954] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3886worker2 # [ 33.324159] nix-grpc-daemon[777]: ts=2026-09-21T09:35:03Z level=debug event=path_info_miss path=kn580alm8i65l8xnw33vjbcfrdhrbapf-nixos-system-client-test error="error: path '/nix/store/kn580alm8i65l8xnw33vjbcfrdhrbapf-nixos-system-client-test' is required, but there is no substituter that can build it"3887client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3888client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)3889client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3890client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)3891worker2 # [ 33.777795] nix-grpc-daemon[777]: ts=2026-09-21T09:35:04Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:439603892worker2 # [ 33.796173] nix-grpc-daemon[777]: ts=2026-09-21T09:35:04Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:439603893worker2 # [ 33.808429] nix-grpc-daemon[777]: ts=2026-09-21T09:35:04Z level=debug event=path_info_miss path=kn580alm8i65l8xnw33vjbcfrdhrbapf-nixos-system-client-test error="error: path '/nix/store/kn580alm8i65l8xnw33vjbcfrdhrbapf-nixos-system-client-test' is required, but there is no substituter that can build it"3894worker1 # [ 33.778795] nix-grpc-daemon[776]: ts=2026-09-21T09:35:04Z level=info event=oidc_jwks provider=mock keys=13895worker1 # [ 33.784410] nix-grpc-daemon[776]: ts=2026-09-21T09:35:04Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:520163896worker1 # [ 33.795448] nix-grpc-daemon[776]: ts=2026-09-21T09:35:04Z level=debug event=path_info_miss path=kn580alm8i65l8xnw33vjbcfrdhrbapf-nixos-system-client-test error="error: path '/nix/store/kn580alm8i65l8xnw33vjbcfrdhrbapf-nixos-system-client-test' is required, but there is no substituter that can build it"3897client: must succeed: rm -r /var/lib/nix-grpc-store3898client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)3899worker1 # [ 34.099142] nix-grpc-daemon[776]: ts=2026-09-21T09:35:04Z level=info event=denied method=QueryMissing cn=- role=none3900worker1 # [ 34.361236] nix-grpc-daemon[776]: ts=2026-09-21T09:35:04Z level=info event=denied method=QueryMissing cn=stranger role=none3901(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.09 seconds)3902subtest: DAG build is scheduled across workers and lands in the cache3903client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23904worker1 # [ 34.619106] nix-grpc-daemon[776]: ts=2026-09-21T09:35:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520163905client # don't know how to build these paths:3906client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3907worker2 # [ 34.679008] nix-grpc-daemon[777]: ts=2026-09-21T09:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:439603908worker2 # [ 34.714297] nix-grpc-daemon[777]: ts=2026-09-21T09:35:05Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03909worker2 # [ 34.720116] nix-grpc-daemon[777]: ts=2026-09-21T09:35:05Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=13910worker1 # [ 34.713598] nix-grpc-daemon[776]: ts=2026-09-21T09:35:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520163911worker1 # [ 34.732908] nix-grpc-daemon[776]: ts=2026-09-21T09:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520163912client # copying 1 paths to 'lb:50051'...3913worker1 # [ 34.748103] nix-grpc-daemon[776]: ts=2026-09-21T09:35:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520163914client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3915worker1 # [ 34.775187] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3916worker1 # [ 34.814971] nix-grpc-daemon[776]: ts=2026-09-21T09:35:05Z level=info event=niks3_push_started pid=8523917worker1 # [ 35.096250] nix-grpc-daemon[852]: time=2026-09-21T09:35:05.721Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""3918worker1 # [ 35.287417] nix-daemon[839]: accepted connection from pid 865, user nix-grpc-daemon (trusted)3919worker1 # [ 35.305454] nix-daemon[839]: reaped child process 868, status = succeeded3920worker1 # [ 35.478022] nix-daemon[839]: accepted connection from pid 870, user nix-grpc-daemon (trusted)3921worker1 # [ 35.493834] nix-daemon[839]: reaped child process 873, status = succeeded3922lb # [ 35.508881] niks3-server[974]: 2026/09/21 09:35:06 INFO Received uploads request method=POST path=/api/pending_closures3923worker1 # [ 35.529175] nix-grpc-daemon[852]: time=2026-09-21T09:35:06.156Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3924worker1 # [ 35.532553] nix-grpc-daemon[852]: time=2026-09-21T09:35:06.157Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3925lb # [ 35.580964] niks3-server[974]: 2026/09/21 09:35:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3926lb # [ 35.594593] niks3-server[974]: 2026/09/21 09:35:06 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3927lb # [ 35.634454] niks3-server[974]: 2026/09/21 09:35:06 INFO Signed narinfos id=1 count=13928worker1 # [ 35.635342] nix-grpc-daemon[852]: time=2026-09-21T09:35:06.261Z level=INFO msg="Uploading 1 narinfos"3929lb # [ 35.650346] niks3-server[974]: 2026/09/21 09:35:06 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3930lb # [ 35.660272] niks3-server[974]: 2026/09/21 09:35:06 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3931lb # [ 35.669912] niks3-server[974]: 2026/09/21 09:35:06 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3932lb # [ 35.677276] niks3-server[974]: 2026/09/21 09:35:06 INFO Completed upload id=13933worker1 # [ 35.674522] nix-grpc-daemon[852]: time=2026-09-21T09:35:06.301Z level=INFO msg="Upload complete. (570ms)"3934worker1 # [ 35.681204] nix-grpc-daemon[776]: ts=2026-09-21T09:35:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_in=5043935worker1 # [ 35.683786] nix-grpc-daemon[776]: ts=2026-09-21T09:35:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520163936worker1 # [ 35.694144] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3937worker1 # [ 35.697799] nix-daemon[839]: reaped child process 850, status = succeeded3938client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3939worker1 # [ 35.707646] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3940client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3941client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds3942client # farm-a> LOG-farm-a-t13943worker1 # [ 36.947199] nix-daemon[839]: reaped child process 878, status = succeeded3944worker1 # [ 37.122750] nix-daemon[839]: accepted connection from pid 894, user nix-grpc-daemon (trusted)3945worker1 # [ 37.140285] nix-daemon[839]: reaped child process 897, status = succeeded3946lb # [ 37.155910] niks3-server[974]: 2026/09/21 09:35:07 INFO Received uploads request method=POST path=/api/pending_closures3947worker1 # [ 37.166271] nix-grpc-daemon[852]: time=2026-09-21T09:35:07.792Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3948worker1 # [ 37.168300] nix-grpc-daemon[852]: time=2026-09-21T09:35:07.792Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3949lb # [ 37.210652] niks3-server[974]: 2026/09/21 09:35:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3950worker1 # [ 37.219192] nix-grpc-daemon[852]: time=2026-09-21T09:35:07.845Z level=INFO msg="Uploading 1 narinfos"3951lb # [ 37.228770] niks3-server[974]: 2026/09/21 09:35:07 INFO Signed narinfos id=2 count=13952lb # [ 37.231637] niks3-server[974]: 2026/09/21 09:35:07 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3953lb # [ 37.252645] niks3-server[974]: 2026/09/21 09:35:07 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3954lb # [ 37.255926] niks3-server[974]: 2026/09/21 09:35:07 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3955lb # [ 37.262012] niks3-server[974]: 2026/09/21 09:35:07 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3956lb # [ 37.266247] niks3-server[974]: 2026/09/21 09:35:07 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo3957worker1 # [ 37.263521] nix-grpc-daemon[852]: time=2026-09-21T09:35:07.890Z level=INFO msg="Upload complete. (347ms)"3958lb # [ 37.268976] niks3-server[974]: 2026/09/21 09:35:07 INFO Completed upload id=23959worker2 # [ 37.333366] nix-grpc-daemon[777]: ts=2026-09-21T09:35:07Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=03960worker1 # [ 37.277100] nix-grpc-daemon[776]: ts=2026-09-21T09:35:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=13961worker1 # [ 37.295295] nix-daemon[839]: reaped child process 876, status = succeeded3962worker2 # [ 37.333680] nix-grpc-daemon[777]: ts=2026-09-21T09:35:07Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=13963worker1 # [ 37.340363] nix-grpc-daemon[776]: ts=2026-09-21T09:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520163964worker1 # [ 37.356348] nix-grpc-daemon[776]: ts=2026-09-21T09:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520163965worker1 # [ 37.370272] nix-grpc-daemon[852]: time=2026-09-21T09:35:07.997Z level=INFO msg="All 1 paths already cached"3966client # copying 1 paths to 'lb:50051'...3967worker1 # [ 37.380397] nix-grpc-daemon[776]: ts=2026-09-21T09:35:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520163968client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...3969worker1 # [ 37.404357] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3970worker1 # [ 37.608262] nix-daemon[839]: accepted connection from pid 901, user nix-grpc-daemon (trusted)3971worker1 # [ 37.627406] nix-daemon[839]: reaped child process 904, status = succeeded3972worker1 # [ 37.799766] nix-daemon[839]: accepted connection from pid 906, user nix-grpc-daemon (trusted)3973worker1 # [ 37.815422] nix-daemon[839]: reaped child process 909, status = succeeded3974lb # [ 37.829488] niks3-server[974]: 2026/09/21 09:35:08 INFO Received uploads request method=POST path=/api/pending_closures3975worker1 # [ 37.837881] nix-grpc-daemon[852]: time=2026-09-21T09:35:08.464Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3976worker1 # [ 37.839323] nix-grpc-daemon[852]: time=2026-09-21T09:35:08.464Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"3977lb # [ 37.862387] niks3-server[974]: 2026/09/21 09:35:08 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst3978worker1 # [ 37.877323] nix-grpc-daemon[852]: time=2026-09-21T09:35:08.504Z level=INFO msg="Uploading 1 narinfos"3979lb # [ 37.872586] niks3-server[974]: 2026/09/21 09:35:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign3980lb # [ 37.886337] niks3-server[974]: 2026/09/21 09:35:08 INFO Signed narinfos id=3 count=13981lb # [ 37.893742] niks3-server[974]: 2026/09/21 09:35:08 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls3982lb # [ 37.905027] niks3-server[974]: 2026/09/21 09:35:08 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete3983worker1 # [ 37.907169] nix-grpc-daemon[852]: time=2026-09-21T09:35:08.534Z level=INFO msg="Upload complete. (485ms)"3984lb # [ 37.910980] niks3-server[974]: 2026/09/21 09:35:08 INFO Completed upload id=33985lb # [ 37.914982] niks3-server[974]: 2026/09/21 09:35:08 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo3986worker1 # [ 37.916231] nix-grpc-daemon[776]: ts=2026-09-21T09:35:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_in=6243987worker1 # [ 37.926072] nix-grpc-daemon[776]: ts=2026-09-21T09:35:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520163988worker1 # [ 37.932976] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3989worker1 # [ 37.937874] nix-daemon[839]: reaped child process 899, status = succeeded3990client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...3991client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3992worker1 # [ 37.952183] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3993client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds3994client # farm-b> LOG-farm-b-t13995worker1 # [ 39.398260] nix-daemon[839]: reaped child process 913, status = succeeded3996worker1 # [ 39.572302] nix-daemon[839]: accepted connection from pid 930, user nix-grpc-daemon (trusted)3997worker1 # [ 39.588392] nix-daemon[839]: reaped child process 933, status = succeeded3998lb # [ 39.605991] niks3-server[974]: 2026/09/21 09:35:10 INFO Received uploads request method=POST path=/api/pending_closures3999worker1 # [ 39.614306] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.240Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4000worker1 # [ 39.615739] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.240Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4001lb # [ 39.637950] niks3-server[974]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4002lb # [ 39.652324] niks3-server[974]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4003lb # [ 39.664979] niks3-server[974]: 2026/09/21 09:35:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4004worker1 # [ 39.666994] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.293Z level=INFO msg="Uploading 1 narinfos"4005lb # [ 39.674605] niks3-server[974]: 2026/09/21 09:35:10 INFO Signed narinfos id=4 count=14006lb # [ 39.679478] niks3-server[974]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4007lb # [ 39.686845] niks3-server[974]: 2026/09/21 09:35:10 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4008lb # [ 39.692627] niks3-server[974]: 2026/09/21 09:35:10 INFO Completed upload id=44009worker1 # [ 39.689776] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.316Z level=INFO msg="Upload complete. (327ms)"4010worker1 # [ 39.700265] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.316Z level=WARN msg="Failed to register uploaded object" key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo error="server returned 404: object is not pending upload\n"4011worker2 # [ 39.752750] nix-grpc-daemon[777]: ts=2026-09-21T09:35:10Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04012worker2 # [ 39.753263] nix-grpc-daemon[777]: ts=2026-09-21T09:35:10Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14013worker1 # [ 39.717964] nix-grpc-daemon[776]: ts=2026-09-21T09:35:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14014worker1 # [ 39.724640] nix-daemon[839]: reaped child process 911, status = succeeded4015worker1 # [ 39.755526] nix-grpc-daemon[776]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164016worker1 # [ 39.770548] nix-grpc-daemon[776]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164017worker1 # [ 39.784575] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.411Z level=INFO msg="All 2 paths already cached"4018client # copying 1 paths to 'lb:50051'...4019worker1 # [ 39.794737] nix-grpc-daemon[776]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520164020client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4021worker1 # [ 39.818476] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4022lb # [ 39.868594] envoy[917]: conn peer=[2001:db8:1::1]:34016 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=26654023lb # [ 39.873464] envoy[917]: conn peer=[2001:db8:1::1]:55444 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=794024lb # [ 39.878694] envoy[917]: conn peer=[2001:db8:1::1]:55448 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=164025lb # [ 39.883198] envoy[917]: conn peer=[2001:db8:1::1]:55458 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=124026worker1 # [ 40.028174] nix-daemon[839]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4027worker1 # [ 40.045366] nix-daemon[839]: reaped child process 940, status = succeeded4028worker1 # [ 40.216773] nix-daemon[839]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4029worker1 # [ 40.232414] nix-daemon[839]: reaped child process 945, status = succeeded4030lb # [ 40.249083] niks3-server[974]: 2026/09/21 09:35:10 INFO Received uploads request method=POST path=/api/pending_closures4031worker1 # [ 40.255527] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.882Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4032worker1 # [ 40.257168] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.882Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4033lb # [ 40.277441] niks3-server[974]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4034lb # [ 40.289378] niks3-server[974]: 2026/09/21 09:35:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4035worker1 # [ 40.293866] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.920Z level=INFO msg="Uploading 1 narinfos"4036lb # [ 40.298038] niks3-server[974]: 2026/09/21 09:35:10 INFO Signed narinfos id=5 count=14037lb # [ 40.304747] niks3-server[974]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4038lb # [ 40.314043] niks3-server[974]: 2026/09/21 09:35:10 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4039worker1 # [ 40.320937] nix-grpc-daemon[852]: time=2026-09-21T09:35:10.947Z level=INFO msg="Upload complete. (484ms)"4040lb # [ 40.324322] niks3-server[974]: 2026/09/21 09:35:10 INFO Completed upload id=54041lb # [ 40.326871] niks3-server[974]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4042worker1 # [ 40.328127] nix-grpc-daemon[776]: ts=2026-09-21T09:35:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_in=7524043worker1 # [ 40.337390] nix-grpc-daemon[776]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164044worker1 # [ 40.347069] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4045worker1 # [ 40.352521] nix-daemon[839]: reaped child process 935, status = succeeded4046client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4047worker1 # [ 40.365279] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4048client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4049client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4050client # farm-top> LOG-farm-top-t14051worker1 # [ 41.994685] nix-daemon[839]: reaped child process 949, status = succeeded4052worker1 # [ 42.160284] nix-daemon[839]: accepted connection from pid 965, user nix-grpc-daemon (trusted)4053worker1 # [ 42.178446] nix-daemon[839]: reaped child process 968, status = succeeded4054lb # [ 42.194253] niks3-server[974]: 2026/09/21 09:35:12 INFO Received uploads request method=POST path=/api/pending_closures4055worker1 # [ 42.203307] nix-grpc-daemon[852]: time=2026-09-21T09:35:12.828Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4056worker1 # [ 42.204957] nix-grpc-daemon[852]: time=2026-09-21T09:35:12.828Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4057lb # [ 42.225993] niks3-server[974]: 2026/09/21 09:35:12 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4058lb # [ 42.241516] niks3-server[974]: 2026/09/21 09:35:12 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4059lb # [ 42.255172] niks3-server[974]: 2026/09/21 09:35:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4060worker1 # [ 42.257693] nix-grpc-daemon[852]: time=2026-09-21T09:35:12.884Z level=INFO msg="Uploading 1 narinfos"4061lb # [ 42.263638] niks3-server[974]: 2026/09/21 09:35:12 INFO Signed narinfos id=6 count=14062lb # [ 42.268151] niks3-server[974]: 2026/09/21 09:35:12 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4063lb # [ 42.276794] niks3-server[974]: 2026/09/21 09:35:12 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4064lb # [ 42.280286] niks3-server[974]: 2026/09/21 09:35:12 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4065lb # [ 42.283799] niks3-server[974]: 2026/09/21 09:35:12 INFO Completed upload id=64066worker1 # [ 42.281302] nix-grpc-daemon[852]: time=2026-09-21T09:35:12.907Z level=INFO msg="Upload complete. (305ms)"4067worker1 # [ 42.288459] nix-grpc-daemon[776]: ts=2026-09-21T09:35:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14068worker1 # [ 42.297629] nix-daemon[839]: reaped child process 947, status = succeeded4069client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.96 seconds)4070client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4071client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.20 seconds)4072client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4073client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4074client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4075client # copying 3 paths...4076client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4077client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4078client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4079client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.29 seconds)4080(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.58 seconds)4081subtest: repeat is answered Cached by the scheduler without building4082worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4083worker1 # finding garbage collector roots...4084worker1 # deleting specified paths...4085worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4086worker1 # deleting unused links...4087worker1 # note: hard linking is currently saving 0.0 KiB4088worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.26 seconds)4089worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4090worker2 # finding garbage collector roots...4091worker2 # deleting specified paths...4092worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4093worker2 # deleting unused links...4094worker2 # note: hard linking is currently saving 0.0 KiB4095worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.29 seconds)4096client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24097client # don't know how to build these paths:4098client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4099worker2 # [ 43.821646] nix-grpc-daemon[777]: ts=2026-09-21T09:35:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:439604100worker2 # [ 43.829431] nix-grpc-daemon[777]: ts=2026-09-21T09:35:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:439604101worker2 # [ 43.857033] nix-grpc-daemon[777]: ts=2026-09-21T09:35:14Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14102client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.33 seconds)4103client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4104client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4105lb: must succeed: curl -sf localhost:9901/clusters4106lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4107lb: must succeed: curl -sf localhost:9901/clusters4108lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4109worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4110worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4111(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.41 seconds)4112subtest: inputs already in the cache are not sent to the scheduler4113lb: must succeed: curl -sf localhost:9901/clusters4114lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4115lb: must succeed: curl -sf localhost:9901/clusters4116lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4117worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4118worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4119client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24120client # don't know how to build these paths:4121client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4122worker1 # [ 44.818930] nix-grpc-daemon[776]: ts=2026-09-21T09:35:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520164123worker1 # [ 44.826298] nix-grpc-daemon[776]: ts=2026-09-21T09:35:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164124worker1 # [ 44.834164] nix-grpc-daemon[852]: time=2026-09-21T09:35:15.458Z level=INFO msg="All 2 paths already cached"4125worker2 # [ 44.891588] nix-grpc-daemon[777]: ts=2026-09-21T09:35:15Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04126worker1 # [ 44.889215] nix-grpc-daemon[776]: ts=2026-09-21T09:35:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164127worker1 # [ 44.904798] nix-grpc-daemon[776]: ts=2026-09-21T09:35:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164128worker1 # [ 44.918273] nix-grpc-daemon[852]: time=2026-09-21T09:35:15.545Z level=INFO msg="All 2 paths already cached"4129client # copying 1 paths to 'lb:50051'...4130client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4131worker1 # [ 44.927651] nix-grpc-daemon[776]: ts=2026-09-21T09:35:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520164132worker1 # [ 44.941252] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4133worker1 # [ 45.144782] nix-daemon[839]: accepted connection from pid 986, user nix-grpc-daemon (trusted)4134worker1 # [ 45.164400] nix-daemon[839]: reaped child process 989, status = succeeded4135worker1 # [ 45.334330] nix-daemon[839]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4136worker1 # [ 45.353288] nix-daemon[839]: reaped child process 994, status = succeeded4137lb # [ 45.368742] niks3-server[974]: 2026/09/21 09:35:16 INFO Received uploads request method=POST path=/api/pending_closures4138worker1 # [ 45.375220] nix-grpc-daemon[852]: time=2026-09-21T09:35:16.001Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4139worker1 # [ 45.376811] nix-grpc-daemon[852]: time=2026-09-21T09:35:16.001Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4140lb # [ 45.394443] niks3-server[974]: 2026/09/21 09:35:16 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4141lb # [ 45.408939] niks3-server[974]: 2026/09/21 09:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4142worker1 # [ 45.413076] nix-grpc-daemon[852]: time=2026-09-21T09:35:16.039Z level=INFO msg="Uploading 1 narinfos"4143lb # [ 45.421898] niks3-server[974]: 2026/09/21 09:35:16 INFO Signed narinfos id=7 count=14144lb # [ 45.428434] niks3-server[974]: 2026/09/21 09:35:16 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4145lb # [ 45.438108] niks3-server[974]: 2026/09/21 09:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4146lb # [ 45.444171] niks3-server[974]: 2026/09/21 09:35:16 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4147lb # [ 45.448934] niks3-server[974]: 2026/09/21 09:35:16 INFO Completed upload id=74148worker1 # [ 45.446224] nix-grpc-daemon[852]: time=2026-09-21T09:35:16.073Z level=INFO msg="Upload complete. (488ms)"4149worker1 # [ 45.454214] nix-grpc-daemon[776]: ts=2026-09-21T09:35:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_in=7524150worker1 # [ 45.467762] nix-grpc-daemon[776]: ts=2026-09-21T09:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164151worker1 # [ 45.474318] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4152worker2 # [ 44.897682] nix-grpc-daemon[777]: ts=2026-09-21T09:35:15Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14153worker1 # [ 45.479045] nix-daemon[839]: reaped child process 984, status = succeeded4154client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4155worker1 # [ 45.490800] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4156client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4157client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4158client # farm-top> LOG-farm-top-t1b4159worker1 # [ 46.717241] nix-daemon[839]: reaped child process 998, status = succeeded4160worker1 # [ 46.870508] nix-daemon[839]: accepted connection from pid 1015, user nix-grpc-daemon (trusted)4161worker1 # [ 46.888216] nix-daemon[839]: reaped child process 1018, status = succeeded4162lb # [ 46.905737] niks3-server[974]: 2026/09/21 09:35:17 INFO Received uploads request method=POST path=/api/pending_closures4163worker1 # [ 46.912439] nix-grpc-daemon[852]: time=2026-09-21T09:35:17.539Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4164worker1 # [ 46.916648] nix-grpc-daemon[852]: time=2026-09-21T09:35:17.539Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4165lb # [ 46.943340] niks3-server[974]: 2026/09/21 09:35:17 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4166lb # [ 46.955096] niks3-server[974]: 2026/09/21 09:35:17 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4167worker1 # [ 46.961534] nix-grpc-daemon[852]: time=2026-09-21T09:35:17.588Z level=INFO msg="Uploading 1 narinfos"4168lb # [ 46.966740] niks3-server[974]: 2026/09/21 09:35:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4169lb # [ 46.975853] niks3-server[974]: 2026/09/21 09:35:17 INFO Signed narinfos id=8 count=14170lb # [ 46.979138] niks3-server[974]: 2026/09/21 09:35:17 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4171lb # [ 46.986091] niks3-server[974]: 2026/09/21 09:35:17 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4172lb # [ 46.990295] niks3-server[974]: 2026/09/21 09:35:17 INFO Completed upload id=84173worker1 # [ 46.987644] nix-grpc-daemon[852]: time=2026-09-21T09:35:17.614Z level=INFO msg="Upload complete. (284ms)"4174lb # [ 46.992707] niks3-server[974]: 2026/09/21 09:35:17 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4175worker1 # [ 46.996918] nix-grpc-daemon[776]: ts=2026-09-21T09:35:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14176worker1 # [ 47.011306] nix-daemon[839]: reaped child process 996, status = succeeded4177client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.48 seconds)4178client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4179client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.22 seconds)4180lb: must succeed: curl -sf localhost:9901/clusters4181lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4182lb: must succeed: curl -sf localhost:9901/clusters4183lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4184worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4185worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4186(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.10 seconds)4187subtest: two clients wanting the same drv build it once4188client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4189client # [ 47.557277] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4190client # Running as unit: dedup1.service; invocation ID: 62b7b24f9ccb4eea81c05671cc912a524191client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4192client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4193client # [ 47.817480] nix[880]: don't know how to build these paths:4194client # [ 47.818644] nix[880]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4195worker2 # [ 47.853382] nix-grpc-daemon[777]: ts=2026-09-21T09:35:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:439604196worker2 # [ 47.866764] nix-grpc-daemon[777]: ts=2026-09-21T09:35:18Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04197worker2 # [ 47.874505] nix-grpc-daemon[777]: ts=2026-09-21T09:35:18Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14198worker2 # [ 47.882257] nix-grpc-daemon[777]: ts=2026-09-21T09:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439604199worker2 # [ 47.899906] nix-grpc-daemon[777]: ts=2026-09-21T09:35:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:439604200client # [ 47.876111] nix[880]: copying 1 paths to 'lb:50051'...4201client # [ 47.878743] nix[880]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4202worker2 # [ 47.914896] nix-grpc-daemon[777]: ts=2026-09-21T09:35:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:439604203worker2 # [ 47.930894] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4204worker2 # [ 47.957207] nix-grpc-daemon[777]: ts=2026-09-21T09:35:18Z level=info event=niks3_push_started pid=9154205worker2 # [ 48.222550] nix-grpc-daemon[915]: time=2026-09-21T09:35:18.831Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4206worker2 # [ 48.414473] nix-daemon[860]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4207worker2 # [ 48.434608] nix-daemon[860]: reaped child process 931, status = succeeded4208worker2 # [ 48.609702] nix-daemon[860]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4209worker2 # [ 48.626483] nix-daemon[860]: reaped child process 936, status = succeeded4210lb # [ 48.596606] niks3-server[974]: 2026/09/21 09:35:19 INFO Received uploads request method=POST path=/api/pending_closures4211worker2 # [ 48.652711] nix-grpc-daemon[915]: time=2026-09-21T09:35:19.262Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4212worker2 # [ 48.656060] nix-grpc-daemon[915]: time=2026-09-21T09:35:19.264Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4213lb # [ 48.634664] niks3-server[974]: 2026/09/21 09:35:19 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4214worker2 # [ 48.700256] nix-grpc-daemon[915]: time=2026-09-21T09:35:19.309Z level=INFO msg="Uploading 1 narinfos"4215lb # [ 48.649569] niks3-server[974]: 2026/09/21 09:35:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4216lb # [ 48.660490] niks3-server[974]: 2026/09/21 09:35:19 INFO Signed narinfos id=9 count=14217client # [ 48.667896] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4218lb # [ 48.669731] niks3-server[974]: 2026/09/21 09:35:19 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4219lb # [ 48.681891] niks3-server[974]: 2026/09/21 09:35:19 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4220worker2 # [ 48.732101] nix-grpc-daemon[915]: time=2026-09-21T09:35:19.342Z level=INFO msg="Upload complete. (501ms)"4221lb # [ 48.688890] niks3-server[974]: 2026/09/21 09:35:19 INFO Completed upload id=94222client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.12 seconds)4223lb # [ 48.692344] niks3-server[974]: 2026/09/21 09:35:19 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4224worker2 # [ 48.740161] nix-grpc-daemon[777]: ts=2026-09-21T09:35:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=0 paths=1 nar_bytes_in=4324225worker2 # [ 48.748406] nix-grpc-daemon[777]: ts=2026-09-21T09:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439604226worker2 # [ 48.762079] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4227worker2 # [ 48.765109] nix-daemon[860]: reaped child process 913, status = succeeded4228client # Running as unit: dedup2.service; invocation ID: 3df2899f1cce494db6e53dfc926844094229client # [ 48.737941] nix[880]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4230worker2 # [ 48.777791] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4231client # [ 49.036840] nix[896]: don't know how to build these paths:4232client # [ 49.037833] nix[896]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4233worker1 # [ 49.026270] nix-grpc-daemon[776]: ts=2026-09-21T09:35:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520164234worker2 # [ 49.087245] nix-grpc-daemon[777]: ts=2026-09-21T09:35:19Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04235client # [ 49.059117] nix[896]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4236worker2 # [ 49.095021] nix-grpc-daemon[777]: ts=2026-09-21T09:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439604237lb # [ 49.865045] envoy[917]: conn peer=[2001:db8:1::1]:55474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9579 tx=4092 ms=76954238lb # [ 49.870084] envoy[917]: conn peer=[2001:db8:1::1]:56698 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=564239lb # [ 49.875055] envoy[917]: conn peer=[2001:db8:1::1]:56712 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1807 ms=21934240client: must succeed: sleep 24241client: (finished: must succeed: sleep 2, in 2.06 seconds)4242client: waiting for success: ! systemctl is-active dedup1 dedup24243worker2 # [ 52.422250] nix-daemon[860]: reaped child process 940, status = succeeded4244worker2 # [ 52.600468] nix-daemon[860]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4245worker2 # [ 52.619237] nix-daemon[860]: reaped child process 975, status = succeeded4246lb # [ 52.589656] niks3-server[974]: 2026/09/21 09:35:23 INFO Received uploads request method=POST path=/api/pending_closures4247worker2 # [ 52.642957] nix-grpc-daemon[915]: time=2026-09-21T09:35:23.251Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4248worker2 # [ 52.644957] nix-grpc-daemon[915]: time=2026-09-21T09:35:23.251Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4249lb # [ 52.635993] niks3-server[974]: 2026/09/21 09:35:23 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4250worker2 # [ 52.691600] nix-grpc-daemon[915]: time=2026-09-21T09:35:23.300Z level=INFO msg="Uploading 1 narinfos"4251lb # [ 52.650025] niks3-server[974]: 2026/09/21 09:35:23 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4252lb # [ 52.658645] niks3-server[974]: 2026/09/21 09:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4253lb # [ 52.666101] niks3-server[974]: 2026/09/21 09:35:23 INFO Signed narinfos id=10 count=14254worker2 # [ 52.716425] nix-grpc-daemon[915]: time=2026-09-21T09:35:23.326Z level=INFO msg="Upload complete. (325ms)"4255lb # [ 52.673478] niks3-server[974]: 2026/09/21 09:35:23 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4256lb # [ 52.680348] niks3-server[974]: 2026/09/21 09:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4257lb # [ 52.685593] niks3-server[974]: 2026/09/21 09:35:23 INFO Completed upload id=104258worker2 # [ 52.728908] nix-grpc-daemon[777]: ts=2026-09-21T09:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14259lb # [ 52.687915] niks3-server[974]: 2026/09/21 09:35:23 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4260worker2 # [ 52.740974] nix-grpc-daemon[777]: ts=2026-09-21T09:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14261worker2 # [ 52.746169] nix-daemon[860]: reaped child process 938, status = succeeded4262client # [ 52.745245] systemd[1]: dedup1.service: Deactivated successfully.4263client # [ 52.753817] systemd[1]: dedup1.service: Consumed 172ms CPU time over 5.190s wall clock time, 43.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.4264client # [ 52.800409] systemd[1]: dedup2.service: Deactivated successfully.4265client # [ 52.804724] systemd[1]: dedup2.service: Consumed 217ms CPU time over 4.135s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4266client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4267client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4268client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4269worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4270worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4271worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4272worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4273worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4274worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4275worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4276worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4277(finished: subtest: two clients wanting the same drv build it once, in 6.25 seconds)4278subtest: build hook: nix-daemon with builders = grpc://lb4279client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4280client # [ 53.959068] systemd[1]: Started Nix Daemon.4281client # [ 54.147438] nix-daemon[938]: accepted connection from pid 935, user root (trusted)4282worker2 # [ 54.401927] nix-grpc-daemon[777]: ts=2026-09-21T09:35:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:439604283worker1 # [ 54.360777] nix-grpc-daemon[776]: ts=2026-09-21T09:35:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164284worker2 # [ 54.418837] nix-grpc-daemon[777]: ts=2026-09-21T09:35:25Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04285worker2 # [ 54.425081] nix-grpc-daemon[777]: ts=2026-09-21T09:35:25Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14286worker1 # [ 54.417109] nix-grpc-daemon[776]: ts=2026-09-21T09:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164287worker1 # [ 54.432485] nix-grpc-daemon[776]: ts=2026-09-21T09:35:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164288worker1 # [ 54.446770] nix-grpc-daemon[776]: ts=2026-09-21T09:35:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520164289worker1 # [ 54.461167] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4290worker1 # [ 54.670476] nix-daemon[839]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)4291worker1 # [ 54.689039] nix-daemon[839]: reaped child process 1046, status = succeeded4292worker1 # [ 54.856597] nix-daemon[839]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)4293worker1 # [ 54.874589] nix-daemon[839]: reaped child process 1051, status = succeeded4294lb # [ 54.889487] niks3-server[974]: 2026/09/21 09:35:25 INFO Received uploads request method=POST path=/api/pending_closures4295worker1 # [ 54.896468] nix-grpc-daemon[852]: time=2026-09-21T09:35:25.523Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4296worker1 # [ 54.898347] nix-grpc-daemon[852]: time=2026-09-21T09:35:25.525Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4297lb # [ 54.920078] niks3-server[974]: 2026/09/21 09:35:25 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4298worker1 # [ 54.932078] nix-grpc-daemon[852]: time=2026-09-21T09:35:25.558Z level=INFO msg="Uploading 1 narinfos"4299lb # [ 54.935534] niks3-server[974]: 2026/09/21 09:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4300lb # [ 54.945847] niks3-server[974]: 2026/09/21 09:35:25 INFO Signed narinfos id=11 count=14301lb # [ 54.952318] niks3-server[974]: 2026/09/21 09:35:25 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4302lb # [ 54.962675] niks3-server[974]: 2026/09/21 09:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4303lb # [ 54.971362] niks3-server[974]: 2026/09/21 09:35:25 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4304lb # [ 54.977106] niks3-server[974]: 2026/09/21 09:35:25 INFO Completed upload id=114305worker1 # [ 54.974249] nix-grpc-daemon[852]: time=2026-09-21T09:35:25.601Z level=INFO msg="Upload complete. (496ms)"4306worker1 # [ 54.983625] nix-grpc-daemon[776]: ts=2026-09-21T09:35:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_in=5124307worker1 # [ 54.998695] nix-grpc-daemon[776]: ts=2026-09-21T09:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164308worker1 # [ 55.005248] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4309worker1 # [ 55.010320] nix-daemon[839]: reaped child process 1041, status = succeeded4310worker1 # [ 55.021315] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4311worker1 # [ 56.666570] nix-daemon[839]: reaped child process 1055, status = succeeded4312worker1 # [ 56.861336] nix-daemon[839]: accepted connection from pid 1071, user nix-grpc-daemon (trusted)4313worker1 # [ 56.881191] nix-daemon[839]: reaped child process 1074, status = succeeded4314lb # [ 56.902822] niks3-server[974]: 2026/09/21 09:35:27 INFO Received uploads request method=POST path=/api/pending_closures4315worker1 # [ 56.916530] nix-grpc-daemon[852]: time=2026-09-21T09:35:27.541Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4316worker1 # [ 56.918392] nix-grpc-daemon[852]: time=2026-09-21T09:35:27.541Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4317lb # [ 56.976801] niks3-server[974]: 2026/09/21 09:35:27 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4318lb # [ 56.993877] niks3-server[974]: 2026/09/21 09:35:27 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4319lb # [ 57.003468] niks3-server[974]: 2026/09/21 09:35:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4320worker1 # [ 57.005882] nix-grpc-daemon[852]: time=2026-09-21T09:35:27.632Z level=INFO msg="Uploading 1 narinfos"4321lb # [ 57.012079] niks3-server[974]: 2026/09/21 09:35:27 INFO Signed narinfos id=12 count=14322lb # [ 57.014256] niks3-server[974]: 2026/09/21 09:35:27 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4323lb # [ 57.024192] niks3-server[974]: 2026/09/21 09:35:27 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4324lb # [ 57.036125] niks3-server[974]: 2026/09/21 09:35:27 INFO Completed upload id=124325worker1 # [ 57.033397] nix-grpc-daemon[852]: time=2026-09-21T09:35:27.660Z level=INFO msg="Upload complete. (398ms)"4326lb # [ 57.039420] niks3-server[974]: 2026/09/21 09:35:27 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4327worker1 # [ 57.045290] nix-grpc-daemon[776]: ts=2026-09-21T09:35:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=2 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14328worker2 # [ 57.097966] nix-grpc-daemon[777]: ts=2026-09-21T09:35:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:439604329worker1 # [ 57.067139] nix-daemon[839]: reaped child process 1053, status = succeeded4330worker2 # [ 57.119146] nix-grpc-daemon[777]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4331worker1 # [ 57.126491] nix-grpc-daemon[776]: ts=2026-09-21T09:35:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:520164332worker1 # [ 57.135474] nix-grpc-daemon[776]: ts=2026-09-21T09:35:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_out=1284333worker2 # [ 57.521138] nix-grpc-daemon[777]: ts=2026-09-21T09:35:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:439604334worker1 # [ 57.480499] nix-grpc-daemon[776]: ts=2026-09-21T09:35:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164335worker1 # [ 57.493940] nix-grpc-daemon[852]: time=2026-09-21T09:35:28.120Z level=INFO msg="All 1 paths already cached"4336worker2 # [ 57.564170] nix-grpc-daemon[777]: ts=2026-09-21T09:35:28Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04337worker2 # [ 57.570507] nix-grpc-daemon[777]: ts=2026-09-21T09:35:28Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14338worker1 # [ 57.529181] nix-grpc-daemon[776]: ts=2026-09-21T09:35:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164339worker1 # [ 57.551044] nix-grpc-daemon[776]: ts=2026-09-21T09:35:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164340worker1 # [ 57.570343] nix-grpc-daemon[852]: time=2026-09-21T09:35:28.197Z level=INFO msg="All 1 paths already cached"4341worker1 # [ 57.583149] nix-grpc-daemon[776]: ts=2026-09-21T09:35:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520164342worker1 # [ 57.598027] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4343worker1 # [ 57.833703] nix-daemon[839]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4344worker1 # [ 57.857114] nix-daemon[839]: reaped child process 1081, status = succeeded4345worker1 # [ 58.037248] nix-daemon[839]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)4346worker1 # [ 58.055138] nix-daemon[839]: reaped child process 1086, status = succeeded4347lb # [ 58.071500] niks3-server[974]: 2026/09/21 09:35:28 INFO Received uploads request method=POST path=/api/pending_closures4348worker1 # [ 58.079153] nix-grpc-daemon[852]: time=2026-09-21T09:35:28.705Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4349worker1 # [ 58.080799] nix-grpc-daemon[852]: time=2026-09-21T09:35:28.705Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4350lb # [ 58.103471] niks3-server[974]: 2026/09/21 09:35:28 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4351lb # [ 58.119239] niks3-server[974]: 2026/09/21 09:35:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4352worker1 # [ 58.122732] nix-grpc-daemon[852]: time=2026-09-21T09:35:28.749Z level=INFO msg="Uploading 1 narinfos"4353lb # [ 58.132783] niks3-server[974]: 2026/09/21 09:35:28 INFO Signed narinfos id=13 count=14354lb # [ 58.139674] niks3-server[974]: 2026/09/21 09:35:28 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4355lb # [ 58.146749] niks3-server[974]: 2026/09/21 09:35:28 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4356worker1 # [ 58.149626] nix-grpc-daemon[852]: time=2026-09-21T09:35:28.776Z level=INFO msg="Upload complete. (532ms)"4357lb # [ 58.154722] niks3-server[974]: 2026/09/21 09:35:28 INFO Completed upload id=134358lb # [ 58.158313] niks3-server[974]: 2026/09/21 09:35:28 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4359worker1 # [ 58.161813] nix-grpc-daemon[776]: ts=2026-09-21T09:35:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_in=6244360worker1 # [ 58.174729] nix-grpc-daemon[776]: ts=2026-09-21T09:35:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164361worker1 # [ 58.180793] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4362worker1 # [ 58.185700] nix-daemon[839]: reaped child process 1076, status = succeeded4363worker1 # [ 58.199754] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4364worker1 # [ 59.221784] nix-daemon[839]: reaped child process 1090, status = succeeded4365worker1 # [ 59.373194] nix-daemon[839]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)4366worker1 # [ 59.392618] nix-daemon[839]: reaped child process 1109, status = succeeded4367lb # [ 59.409346] niks3-server[974]: 2026/09/21 09:35:30 INFO Received uploads request method=POST path=/api/pending_closures4368worker1 # [ 59.415520] nix-grpc-daemon[852]: time=2026-09-21T09:35:30.040Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4369worker1 # [ 59.417168] nix-grpc-daemon[852]: time=2026-09-21T09:35:30.040Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4370lb # [ 59.444495] niks3-server[974]: 2026/09/21 09:35:30 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4371lb # [ 59.456452] niks3-server[974]: 2026/09/21 09:35:30 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4372lb # [ 59.466962] niks3-server[974]: 2026/09/21 09:35:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4373worker1 # [ 59.469662] nix-grpc-daemon[852]: time=2026-09-21T09:35:30.096Z level=INFO msg="Uploading 1 narinfos"4374lb # [ 59.476715] niks3-server[974]: 2026/09/21 09:35:30 INFO Signed narinfos id=14 count=14375lb # [ 59.479394] niks3-server[974]: 2026/09/21 09:35:30 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4376lb # [ 59.489966] niks3-server[974]: 2026/09/21 09:35:30 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4377lb # [ 59.495987] niks3-server[974]: 2026/09/21 09:35:30 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4378lb # [ 59.502420] niks3-server[974]: 2026/09/21 09:35:30 INFO Completed upload id=144379worker1 # [ 59.499915] nix-grpc-daemon[852]: time=2026-09-21T09:35:30.126Z level=INFO msg="Upload complete. (290ms)"4380worker2 # [ 59.560873] nix-grpc-daemon[777]: ts=2026-09-21T09:35:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:439604381worker1 # [ 59.513024] nix-grpc-daemon[776]: ts=2026-09-21T09:35:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14382worker1 # [ 59.526650] nix-daemon[839]: reaped child process 1088, status = succeeded4383worker2 # [ 59.573913] nix-grpc-daemon[777]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4384worker1 # [ 59.571965] nix-grpc-daemon[776]: ts=2026-09-21T09:35:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:520164385worker1 # [ 59.577336] nix-grpc-daemon[776]: ts=2026-09-21T09:35:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_out=1764386worker1 # [ 59.847037] nix-grpc-daemon[776]: ts=2026-09-21T09:35:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:520164387worker1 # [ 59.854399] nix-grpc-daemon[776]: ts=2026-09-21T09:35:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164388worker1 # [ 59.860893] nix-grpc-daemon[852]: time=2026-09-21T09:35:30.486Z level=INFO msg="All 2 paths already cached"4389lb # [ 59.870242] envoy[917]: conn peer=[2001:db8:1::1]:56714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1453 ms=48984390lb # [ 59.875621] envoy[917]: conn peer=[2001:db8:1::1]:56728 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1085 ms=37254391lb # [ 59.882164] envoy[917]: conn peer=[2001:db8:1::1]:53742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3420 tx=1850 ms=28414392lb # [ 59.888773] envoy[917]: conn peer=[2001:db8:1::1]:53748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=754393worker2 # [ 59.932935] nix-grpc-daemon[777]: ts=2026-09-21T09:35:30Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04394lb # [ 59.894247] envoy[917]: conn peer=[2001:db8:1::1]:53764 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=254395worker2 # [ 59.938970] nix-grpc-daemon[777]: ts=2026-09-21T09:35:30Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14396lb # [ 59.899694] envoy[917]: conn peer=[2001:db8:1::1]:53758 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3886 tx=2044 ms=21224397worker1 # [ 59.937647] nix-grpc-daemon[776]: ts=2026-09-21T09:35:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164398worker1 # [ 59.953392] nix-grpc-daemon[776]: ts=2026-09-21T09:35:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164399worker1 # [ 59.967532] nix-grpc-daemon[852]: time=2026-09-21T09:35:30.594Z level=INFO msg="All 2 paths already cached"4400worker1 # [ 59.977302] nix-grpc-daemon[776]: ts=2026-09-21T09:35:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520164401worker1 # [ 59.993520] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4402worker1 # [ 60.197145] nix-daemon[839]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4403worker1 # [ 60.214790] nix-daemon[839]: reaped child process 1116, status = succeeded4404worker1 # [ 60.386328] nix-daemon[839]: accepted connection from pid 1118, user nix-grpc-daemon (trusted)4405worker1 # [ 60.402085] nix-daemon[839]: reaped child process 1121, status = succeeded4406lb # [ 60.417110] niks3-server[974]: 2026/09/21 09:35:31 INFO Received uploads request method=POST path=/api/pending_closures4407worker1 # [ 60.423856] nix-grpc-daemon[852]: time=2026-09-21T09:35:31.050Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4408worker1 # [ 60.425455] nix-grpc-daemon[852]: time=2026-09-21T09:35:31.050Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4409lb # [ 60.444469] niks3-server[974]: 2026/09/21 09:35:31 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4410worker1 # [ 60.454882] nix-grpc-daemon[852]: time=2026-09-21T09:35:31.081Z level=INFO msg="Uploading 1 narinfos"4411lb # [ 60.458275] niks3-server[974]: 2026/09/21 09:35:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4412lb # [ 60.468251] niks3-server[974]: 2026/09/21 09:35:31 INFO Signed narinfos id=15 count=14413lb # [ 60.474178] niks3-server[974]: 2026/09/21 09:35:31 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4414lb # [ 60.482127] niks3-server[974]: 2026/09/21 09:35:31 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4415worker1 # [ 60.483994] nix-grpc-daemon[852]: time=2026-09-21T09:35:31.110Z level=INFO msg="Upload complete. (472ms)"4416lb # [ 60.486716] niks3-server[974]: 2026/09/21 09:35:31 INFO Completed upload id=154417lb # [ 60.489617] niks3-server[974]: 2026/09/21 09:35:31 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4418worker1 # [ 60.490672] nix-grpc-daemon[776]: ts=2026-09-21T09:35:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_in=7524419worker1 # [ 60.503921] nix-grpc-daemon[776]: ts=2026-09-21T09:35:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164420worker1 # [ 60.513367] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4421worker1 # [ 60.516387] nix-daemon[839]: reaped child process 1111, status = succeeded4422worker1 # [ 60.529830] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4423worker1 # [ 61.349882] nix-daemon[839]: reaped child process 1125, status = succeeded4424worker1 # [ 61.504898] nix-daemon[839]: accepted connection from pid 1141, user nix-grpc-daemon (trusted)4425worker1 # [ 61.523554] nix-daemon[839]: reaped child process 1144, status = succeeded4426lb # [ 61.538970] niks3-server[974]: 2026/09/21 09:35:32 INFO Received uploads request method=POST path=/api/pending_closures4427worker1 # [ 61.548800] nix-grpc-daemon[852]: time=2026-09-21T09:35:32.172Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4428worker1 # [ 61.550251] nix-grpc-daemon[852]: time=2026-09-21T09:35:32.172Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4429lb # [ 61.582589] niks3-server[974]: 2026/09/21 09:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4430worker1 # [ 61.589273] nix-grpc-daemon[852]: time=2026-09-21T09:35:32.216Z level=INFO msg="Uploading 1 narinfos"4431lb # [ 61.598867] niks3-server[974]: 2026/09/21 09:35:32 INFO Signed narinfos id=16 count=14432lb # [ 61.605665] niks3-server[974]: 2026/09/21 09:35:32 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4433lb # [ 61.616772] niks3-server[974]: 2026/09/21 09:35:32 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4434worker1 # [ 61.618090] nix-grpc-daemon[852]: time=2026-09-21T09:35:32.244Z level=INFO msg="Upload complete. (279ms)"4435lb # [ 61.621803] niks3-server[974]: 2026/09/21 09:35:32 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4436lb # [ 61.626150] niks3-server[974]: 2026/09/21 09:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4437lb # [ 61.629972] niks3-server[974]: 2026/09/21 09:35:32 INFO Completed upload id=164438lb # [ 61.632443] niks3-server[974]: 2026/09/21 09:35:32 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4439worker1 # [ 61.628857] nix-grpc-daemon[776]: ts=2026-09-21T09:35:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14440worker2 # [ 61.681783] nix-grpc-daemon[777]: ts=2026-09-21T09:35:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:439604441worker1 # [ 61.640846] nix-daemon[839]: reaped child process 1123, status = succeeded4442worker2 # [ 61.694828] nix-grpc-daemon[777]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4443worker1 # [ 61.690641] nix-grpc-daemon[776]: ts=2026-09-21T09:35:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:520164444worker1 # [ 61.696116] nix-grpc-daemon[776]: ts=2026-09-21T09:35:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_out=2324445client # [ 61.792966] nix-daemon[938]: reaped child process 940, status = succeeded4446client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.07 seconds)4447client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4448client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4449client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4450client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4451client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4452client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4453(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.19 seconds)4454subtest: an input only one worker has still reaches the builder4455client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4456client # this derivation will be built:4457client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4458client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4459client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.43 seconds)4460client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4461client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4462worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4463worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4464worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4465worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4466client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24467client # [ 63.074639] nix-daemon[938]: accepted connection from pid 1020, user root (trusted)4468client # this derivation will be built:4469client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4470client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4471worker2 # [ 63.326451] nix-grpc-daemon[777]: ts=2026-09-21T09:35:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:439604472worker2 # [ 63.334507] nix-grpc-daemon[777]: ts=2026-09-21T09:35:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:439604473client # copying 1 paths...4474client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4475worker2 # [ 63.345488] nix-grpc-daemon[777]: ts=2026-09-21T09:35:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:439604476worker2 # [ 63.361087] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4477worker2 # [ 63.572700] nix-daemon[860]: accepted connection from pid 1000, user nix-grpc-daemon (trusted)4478worker2 # [ 63.593069] nix-daemon[860]: reaped child process 1003, status = succeeded4479lb # [ 63.561675] niks3-server[974]: 2026/09/21 09:35:34 INFO Received uploads request method=POST path=/api/pending_closures4480worker2 # [ 63.616114] nix-grpc-daemon[915]: time=2026-09-21T09:35:34.225Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4481worker2 # [ 63.617572] nix-grpc-daemon[915]: time=2026-09-21T09:35:34.225Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4482lb # [ 63.596917] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4483worker2 # [ 63.650408] nix-grpc-daemon[915]: time=2026-09-21T09:35:34.260Z level=INFO msg="Uploading 1 narinfos"4484lb # [ 63.608741] niks3-server[974]: 2026/09/21 09:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4485lb # [ 63.619541] niks3-server[974]: 2026/09/21 09:35:34 INFO Signed narinfos id=17 count=14486lb # [ 63.625264] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4487lb # [ 63.635901] niks3-server[974]: 2026/09/21 09:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4488lb # [ 63.641835] niks3-server[974]: 2026/09/21 09:35:34 INFO Completed upload id=174489worker2 # [ 63.685205] nix-grpc-daemon[915]: time=2026-09-21T09:35:34.295Z level=INFO msg="Upload complete. (307ms)"4490lb # [ 63.644778] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4491worker2 # [ 63.692145] nix-grpc-daemon[777]: ts=2026-09-21T09:35:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=0 paths=1 nar_bytes_in=1204492worker2 # [ 63.706967] nix-daemon[860]: reaped child process 998, status = succeeded4493worker2 # [ 63.713505] nix-grpc-daemon[777]: ts=2026-09-21T09:35:34Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04494worker2 # [ 63.718683] nix-grpc-daemon[777]: ts=2026-09-21T09:35:34Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14495worker2 # [ 63.760108] nix-grpc-daemon[777]: ts=2026-09-21T09:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439604496worker2 # [ 63.775935] nix-grpc-daemon[777]: ts=2026-09-21T09:35:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:439604497client # copying 2 paths to 'lb:50051'...4498client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4499worker2 # [ 63.793400] nix-grpc-daemon[777]: ts=2026-09-21T09:35:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:439604500client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4501worker2 # [ 63.809175] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4502worker2 # [ 64.009012] nix-daemon[860]: accepted connection from pid 1009, user nix-grpc-daemon (trusted)4503worker2 # [ 64.026548] nix-daemon[860]: reaped child process 1012, status = succeeded4504worker2 # [ 64.200111] nix-daemon[860]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4505worker2 # [ 64.217558] nix-daemon[860]: reaped child process 1017, status = succeeded4506lb # [ 64.185352] niks3-server[974]: 2026/09/21 09:35:34 INFO Received uploads request method=POST path=/api/pending_closures4507lb # [ 64.196133] niks3-server[974]: 2026/09/21 09:35:34 INFO Received uploads request method=POST path=/api/pending_closures4508worker2 # [ 64.248559] nix-grpc-daemon[915]: time=2026-09-21T09:35:34.858Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4509worker2 # [ 64.250064] nix-grpc-daemon[915]: time=2026-09-21T09:35:34.858Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4510worker2 # [ 64.251728] nix-grpc-daemon[915]: time=2026-09-21T09:35:34.858Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4511lb # [ 64.239266] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4512lb # [ 64.251774] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4513lb # [ 64.257479] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4514lb # [ 64.264854] niks3-server[974]: 2026/09/21 09:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4515worker2 # [ 64.309050] nix-grpc-daemon[915]: time=2026-09-21T09:35:34.918Z level=INFO msg="Uploading 3 narinfos"4516lb # [ 64.271697] niks3-server[974]: 2026/09/21 09:35:34 INFO Signed narinfos id=18 count=14517lb # [ 64.277714] niks3-server[974]: 2026/09/21 09:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4518lb # [ 64.284520] niks3-server[974]: 2026/09/21 09:35:34 INFO Signed narinfos id=19 count=24519lb # [ 64.286546] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4520lb # [ 64.300554] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4521lb # [ 64.307599] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4522lb # [ 64.311528] niks3-server[974]: 2026/09/21 09:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4523lb # [ 64.315395] niks3-server[974]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4524lb # [ 64.318416] niks3-server[974]: 2026/09/21 09:35:34 INFO Completed upload id=184525lb # [ 64.321431] niks3-server[974]: 2026/09/21 09:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4526lb # [ 64.324433] niks3-server[974]: 2026/09/21 09:35:34 INFO Completed upload id=194527worker2 # [ 64.368206] nix-grpc-daemon[915]: time=2026-09-21T09:35:34.978Z level=INFO msg="Upload complete. (540ms)"4528worker2 # [ 64.381240] nix-grpc-daemon[777]: ts=2026-09-21T09:35:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=0 paths=2 nar_bytes_in=9844529worker2 # [ 64.399869] nix-grpc-daemon[777]: ts=2026-09-21T09:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439604530worker2 # [ 64.410292] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4531worker2 # [ 64.420433] nix-daemon[860]: reaped child process 1007, status = succeeded4532client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4533client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4534worker2 # [ 64.432178] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4535client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4536worker2 # [ 65.458266] nix-daemon[860]: reaped child process 1021, status = succeeded4537worker2 # [ 65.616768] nix-daemon[860]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)4538worker2 # [ 65.634714] nix-daemon[860]: reaped child process 1040, status = succeeded4539lb # [ 65.603266] niks3-server[974]: 2026/09/21 09:35:36 INFO Received uploads request method=POST path=/api/pending_closures4540worker2 # [ 65.656353] nix-grpc-daemon[915]: time=2026-09-21T09:35:36.266Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4541lb # [ 65.634067] niks3-server[974]: 2026/09/21 09:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4542worker2 # [ 65.684052] nix-grpc-daemon[915]: time=2026-09-21T09:35:36.293Z level=INFO msg="Uploading 1 narinfos"4543lb # [ 65.648854] niks3-server[974]: 2026/09/21 09:35:36 INFO Signed narinfos id=20 count=14544lb # [ 65.651975] niks3-server[974]: 2026/09/21 09:35:36 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4545worker2 # [ 65.711469] nix-grpc-daemon[915]: time=2026-09-21T09:35:36.321Z level=INFO msg="Upload complete. (266ms)"4546lb # [ 65.662859] niks3-server[974]: 2026/09/21 09:35:36 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4547lb # [ 65.673387] niks3-server[974]: 2026/09/21 09:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4548lb # [ 65.677368] niks3-server[974]: 2026/09/21 09:35:36 INFO Completed upload id=204549client # copying 1 paths...4550lb # [ 65.680004] niks3-server[974]: 2026/09/21 09:35:36 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4551worker2 # [ 65.721436] nix-grpc-daemon[777]: ts=2026-09-21T09:35:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14552client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4553worker2 # [ 65.733759] nix-grpc-daemon[777]: ts=2026-09-21T09:35:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:439604554worker2 # [ 65.737033] nix-daemon[860]: reaped child process 1019, status = succeeded4555worker2 # [ 65.757380] nix-grpc-daemon[777]: ts=2026-09-21T09:35:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:439604556worker2 # [ 65.764105] nix-grpc-daemon[777]: ts=2026-09-21T09:35:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=0 paths=1 nar_bytes_out=1204557client # [ 65.856867] nix-daemon[938]: reaped child process 1023, status = succeeded4558client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.02 seconds)4559client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24560client # [ 66.069060] nix-daemon[938]: accepted connection from pid 1041, user root (trusted)4561client # this derivation will be built:4562client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4563client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4564worker2 # [ 66.313539] nix-grpc-daemon[777]: ts=2026-09-21T09:35:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:439604565worker2 # [ 66.321212] nix-grpc-daemon[777]: ts=2026-09-21T09:35:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:439604566worker2 # [ 66.327771] nix-grpc-daemon[915]: time=2026-09-21T09:35:36.936Z level=INFO msg="All 1 paths already cached"4567worker2 # [ 66.339676] nix-grpc-daemon[777]: ts=2026-09-21T09:35:36Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04568worker1 # [ 66.298547] nix-grpc-daemon[776]: ts=2026-09-21T09:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164569worker2 # [ 66.346070] nix-grpc-daemon[777]: ts=2026-09-21T09:35:36Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14570worker1 # [ 66.314051] nix-grpc-daemon[776]: ts=2026-09-21T09:35:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164571client # copying 1 paths to 'lb:50051'...4572client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4573worker1 # [ 66.335448] nix-grpc-daemon[776]: ts=2026-09-21T09:35:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520164574worker1 # [ 66.350374] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4575worker1 # [ 66.373572] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)4576worker1 # [ 66.389197] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)4577worker1 # [ 66.539812] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 148 ms (attempt 3/5)4578worker1 # [ 66.688282] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 747 ms (attempt 4/5)4579worker1 # [ 67.437215] nix-grpc-daemon[776]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4580worker1 # [ 67.442650] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4581worker1 # [ 67.451810] nix-grpc-daemon[776]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4582worker1 # [ 67.671785] nix-daemon[839]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4583worker1 # [ 67.690016] nix-daemon[839]: reaped child process 1161, status = succeeded4584worker1 # [ 67.861140] nix-daemon[839]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)4585worker1 # [ 67.877811] nix-daemon[839]: reaped child process 1166, status = succeeded4586lb # [ 67.894093] niks3-server[974]: 2026/09/21 09:35:38 INFO Received uploads request method=POST path=/api/pending_closures4587worker1 # [ 67.902570] nix-grpc-daemon[852]: time=2026-09-21T09:35:38.528Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4588worker1 # [ 67.904088] nix-grpc-daemon[852]: time=2026-09-21T09:35:38.528Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4589lb # [ 67.921818] niks3-server[974]: 2026/09/21 09:35:38 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4590lb # [ 67.937836] niks3-server[974]: 2026/09/21 09:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4591worker1 # [ 67.941493] nix-grpc-daemon[852]: time=2026-09-21T09:35:38.568Z level=INFO msg="Uploading 1 narinfos"4592lb # [ 67.947929] niks3-server[974]: 2026/09/21 09:35:38 INFO Signed narinfos id=21 count=14593lb # [ 67.950950] niks3-server[974]: 2026/09/21 09:35:38 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4594lb # [ 67.961825] niks3-server[974]: 2026/09/21 09:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4595worker1 # [ 67.964726] nix-grpc-daemon[852]: time=2026-09-21T09:35:38.591Z level=INFO msg="Upload complete. (487ms)"4596lb # [ 67.969329] niks3-server[974]: 2026/09/21 09:35:38 INFO Completed upload id=214597lb # [ 67.972601] niks3-server[974]: 2026/09/21 09:35:38 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4598worker1 # [ 67.972153] nix-grpc-daemon[776]: ts=2026-09-21T09:35:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=1 paths=1 nar_bytes_in=5684599worker1 # [ 67.983268] nix-grpc-daemon[776]: ts=2026-09-21T09:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520164600worker1 # [ 67.993999] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4601worker1 # [ 67.999388] nix-daemon[839]: reaped child process 1151, status = succeeded4602client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4603worker1 # [ 68.011303] nix-daemon[839]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4604client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4605client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4606worker1 # [ 69.055605] nix-daemon[839]: reaped child process 1170, status = succeeded4607worker1 # [ 69.229587] nix-daemon[839]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)4608worker1 # [ 69.246962] nix-daemon[839]: reaped child process 1189, status = succeeded4609lb # [ 69.265869] niks3-server[974]: 2026/09/21 09:35:39 INFO Received uploads request method=POST path=/api/pending_closures4610worker1 # [ 69.272765] nix-grpc-daemon[852]: time=2026-09-21T09:35:39.899Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4611lb # [ 69.298333] niks3-server[974]: 2026/09/21 09:35:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4612worker1 # [ 69.304106] nix-grpc-daemon[852]: time=2026-09-21T09:35:39.930Z level=INFO msg="Uploading 1 narinfos"4613lb # [ 69.311202] niks3-server[974]: 2026/09/21 09:35:39 INFO Signed narinfos id=22 count=14614lb # [ 69.319815] niks3-server[974]: 2026/09/21 09:35:39 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4615lb # [ 69.327170] niks3-server[974]: 2026/09/21 09:35:39 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4616lb # [ 69.334785] niks3-server[974]: 2026/09/21 09:35:39 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4617lb # [ 69.339583] niks3-server[974]: 2026/09/21 09:35:39 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4618lb # [ 69.343845] niks3-server[974]: 2026/09/21 09:35:39 INFO Completed upload id=224619worker1 # [ 69.341014] nix-grpc-daemon[852]: time=2026-09-21T09:35:39.967Z level=INFO msg="Upload complete. (320ms)"4620client # copying 1 paths...4621worker1 # [ 69.353063] nix-grpc-daemon[776]: ts=2026-09-21T09:35:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14622worker2 # [ 69.403900] nix-grpc-daemon[777]: ts=2026-09-21T09:35:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:439604623worker1 # [ 69.367889] nix-daemon[839]: reaped child process 1168, status = succeeded4624worker2 # [ 69.416730] nix-grpc-daemon[777]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4625client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4626worker1 # [ 69.417532] nix-grpc-daemon[776]: ts=2026-09-21T09:35:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:520164627worker1 # [ 69.423075] nix-grpc-daemon[776]: ts=2026-09-21T09:35:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52016 duration_s=0 paths=1 nar_bytes_out=1204628client # [ 69.513943] nix-daemon[938]: reaped child process 1044, status = succeeded4629client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.66 seconds)4630client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24631client # [ 69.728568] nix-daemon[938]: accepted connection from pid 1063, user root (trusted)4632client # this derivation will be built:4633client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4634lb # [ 69.868185] envoy[917]: conn peer=[2001:db8:1::1]:53772 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4313 tx=2302 ms=18604635lb # [ 69.873652] envoy[917]: conn peer=[2001:db8:1::1]:40742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=224636lb # [ 69.878762] envoy[917]: conn peer=[2001:db8:1::1]:40754 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=704637lb # [ 69.883785] envoy[917]: conn peer=[2001:db8:1::1]:40750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4262 tx=1846 ms=24984638lb # [ 69.889375] envoy[917]: conn peer=[2001:db8:1::1]:40764 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3743 tx=2013 ms=31674639lb # [ 69.894469] envoy[917]: conn peer=[2001:db8:1::1]:40776 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=234640worker1 # [ 69.935309] nix-grpc-daemon[776]: ts=2026-09-21T09:35:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:520164641client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4642worker1 # [ 69.943738] nix-grpc-daemon[776]: ts=2026-09-21T09:35:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520164643worker1 # [ 69.954617] nix-grpc-daemon[852]: time=2026-09-21T09:35:40.581Z level=INFO msg="All 1 paths already cached"4644worker2 # [ 70.019034] nix-grpc-daemon[777]: ts=2026-09-21T09:35:40Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04645worker2 # [ 70.029371] nix-grpc-daemon[777]: ts=2026-09-21T09:35:40Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14646worker2 # [ 70.038948] nix-grpc-daemon[777]: ts=2026-09-21T09:35:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439604647worker2 # [ 70.046992] nix-grpc-daemon[777]: ts=2026-09-21T09:35:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:439604648worker2 # [ 70.059541] nix-grpc-daemon[915]: time=2026-09-21T09:35:40.668Z level=INFO msg="All 1 paths already cached"4649client # copying 1 paths to 'lb:50051'...4650client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4651worker2 # [ 70.070366] nix-grpc-daemon[777]: ts=2026-09-21T09:35:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:439604652worker2 # [ 70.085519] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4653worker2 # [ 70.286215] nix-daemon[860]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)4654worker2 # [ 70.308684] nix-daemon[860]: reaped child process 1052, status = succeeded4655worker2 # [ 70.480297] nix-daemon[860]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)4656worker2 # [ 70.501184] nix-daemon[860]: reaped child process 1057, status = succeeded4657lb # [ 70.469325] niks3-server[974]: 2026/09/21 09:35:41 INFO Received uploads request method=POST path=/api/pending_closures4658worker2 # [ 70.653658] nix-grpc-daemon[915]: time=2026-09-21T09:35:41.262Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4659worker2 # [ 70.658561] nix-grpc-daemon[915]: time=2026-09-21T09:35:41.263Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4660lb # [ 70.633701] niks3-server[974]: 2026/09/21 09:35:41 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4661worker2 # [ 70.688127] nix-grpc-daemon[915]: time=2026-09-21T09:35:41.298Z level=INFO msg="Uploading 1 narinfos"4662lb # [ 70.647831] niks3-server[974]: 2026/09/21 09:35:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4663lb # [ 70.655715] niks3-server[974]: 2026/09/21 09:35:41 INFO Signed narinfos id=23 count=14664lb # [ 70.662385] niks3-server[974]: 2026/09/21 09:35:41 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4665lb # [ 70.673276] niks3-server[974]: 2026/09/21 09:35:41 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4666worker2 # [ 70.724447] nix-grpc-daemon[915]: time=2026-09-21T09:35:41.334Z level=INFO msg="Upload complete. (621ms)"4667lb # [ 70.681843] niks3-server[974]: 2026/09/21 09:35:41 INFO Completed upload id=234668lb # [ 70.685368] niks3-server[974]: 2026/09/21 09:35:41 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4669worker2 # [ 70.738192] nix-grpc-daemon[777]: ts=2026-09-21T09:35:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=0 paths=1 nar_bytes_in=5684670worker2 # [ 70.752990] nix-grpc-daemon[777]: ts=2026-09-21T09:35:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439604671worker2 # [ 70.767602] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4672worker2 # [ 70.771119] nix-daemon[860]: reaped child process 1047, status = succeeded4673client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4674worker2 # [ 70.787639] nix-daemon[860]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4675client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4676client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4677worker2 # [ 71.415181] nix-daemon[860]: reaped child process 1061, status = succeeded4678worker2 # [ 71.575865] nix-daemon[860]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)4679worker2 # [ 71.596111] nix-daemon[860]: reaped child process 1080, status = succeeded4680lb # [ 71.566416] niks3-server[974]: 2026/09/21 09:35:42 INFO Received uploads request method=POST path=/api/pending_closures4681worker2 # [ 71.623209] nix-grpc-daemon[915]: time=2026-09-21T09:35:42.231Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4682lb # [ 71.601528] niks3-server[974]: 2026/09/21 09:35:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4683worker2 # [ 71.651629] nix-grpc-daemon[915]: time=2026-09-21T09:35:42.261Z level=INFO msg="Uploading 1 narinfos"4684lb # [ 71.615837] niks3-server[974]: 2026/09/21 09:35:42 INFO Signed narinfos id=24 count=14685lb # [ 71.624235] niks3-server[974]: 2026/09/21 09:35:42 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4686lb # [ 71.630680] niks3-server[974]: 2026/09/21 09:35:42 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4687worker2 # [ 71.684588] nix-grpc-daemon[915]: time=2026-09-21T09:35:42.294Z level=INFO msg="Upload complete. (281ms)"4688lb # [ 71.640921] niks3-server[974]: 2026/09/21 09:35:42 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4689lb # [ 71.646512] niks3-server[974]: 2026/09/21 09:35:42 INFO Completed upload id=244690lb # [ 71.649778] niks3-server[974]: 2026/09/21 09:35:42 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4691worker2 # [ 71.692148] nix-grpc-daemon[777]: ts=2026-09-21T09:35:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=0 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14692client # copying 1 paths...4693worker2 # [ 71.708098] nix-daemon[860]: reaped child process 1059, status = succeeded4694worker1 # [ 71.661634] nix-grpc-daemon[776]: ts=2026-09-21T09:35:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:520164695worker1 # [ 71.677107] nix-grpc-daemon[776]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4696client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4697worker2 # [ 71.767832] nix-grpc-daemon[777]: ts=2026-09-21T09:35:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:439604698worker2 # [ 71.774747] nix-grpc-daemon[777]: ts=2026-09-21T09:35:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43960 duration_s=0 paths=1 nar_bytes_out=1204699client # [ 71.822647] nix-daemon[938]: reaped child process 1066, status = succeeded4700client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.31 seconds)4701(finished: subtest: an input only one worker has still reaches the builder, in 9.91 seconds)4702subtest: upload whose reference the worker lacks is completed from the cache4703client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4704client # this derivation will be built:4705client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4706client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4707client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.46 seconds)4708client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4709client # this derivation will be built:4710client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4711client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4712client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.44 seconds)4713client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4714client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4715worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4716worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds)4717worker2 # [ 73.282234] systemd[1]: Stopping Nix worker-protocol over gRPC...4718worker2 # [ 73.382569] nix-grpc-daemon[777]: ts=2026-09-21T09:35:43Z level=info event=signal_exit4719worker2 # [ 73.611564] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=scheduler_yield4720worker2 # [ 73.616855] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=scheduler_restarting addr=lb:500514721lb # [ 73.573964] niks3-server[974]: 2026/09/21 09:35:44 INFO lead: released remote=192.168.1.4:556624722worker2 # [ 73.622337] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:450704723worker1 # [ 73.580400] nix-grpc-daemon[776]: ts=2026-09-21T09:35:44Z level=info event=scheduler_restarting addr=lb:500514724worker2 # [ 73.627063] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=worker_gone addr=192.168.1.4:500514725worker2 # [ 73.632561] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:450704726worker2 # [ 73.637029] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=worker_gone addr=192.168.1.3:500514727worker2 # [ 73.818261] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=scheduler_connected addr=lb:500514728worker2 # [ 73.823292] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4729worker1 # [ 73.780358] nix-grpc-daemon[776]: ts=2026-09-21T09:35:44Z level=info event=scheduler_connected addr=lb:500514730worker1 # [ 73.785470] nix-grpc-daemon[776]: ts=2026-09-21T09:35:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4731worker2 # [ 74.024393] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=scheduler_connected addr=lb:500514732worker2 # [ 74.029275] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4733worker1 # [ 73.986129] nix-grpc-daemon[776]: ts=2026-09-21T09:35:44Z level=info event=scheduler_connected addr=lb:500514734worker1 # [ 73.991231] nix-grpc-daemon[776]: ts=2026-09-21T09:35:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4735worker2 # [ 74.230384] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=scheduler_connected addr=lb:500514736worker2 # [ 74.235445] nix-grpc-daemon[777]: ts=2026-09-21T09:35:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4737worker1 # [ 74.192691] nix-grpc-daemon[776]: ts=2026-09-21T09:35:44Z level=info event=scheduler_connected addr=lb:500514738worker1 # [ 74.197553] nix-grpc-daemon[776]: ts=2026-09-21T09:35:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4739worker2 # [ 74.438043] nix-grpc-daemon[777]: ts=2026-09-21T09:35:45Z level=info event=scheduler_connected addr=lb:500514740worker2 # [ 74.442289] nix-grpc-daemon[777]: ts=2026-09-21T09:35:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4741worker1 # [ 74.398375] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_connected addr=lb:500514742worker1 # [ 74.403466] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4743worker2 # [ 74.617712] nix-grpc-daemon[777]: ts=2026-09-21T09:35:45Z level=info event=scheduler_restarting4744worker1 # [ 74.605999] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_connected addr=lb:500514745worker1 # [ 74.610122] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4746worker2 # [ 74.661905] nix-grpc-daemon[777]: ts=2026-09-21T09:35:45Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4747worker2 # [ 74.680620] nix-daemon[860]: reaped child process 862, status = succeeded4748worker1 # [ 74.806576] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_connected addr=lb:500514749worker1 # [ 74.811734] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4750worker1 # [ 75.012590] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_connected addr=lb:500514751worker1 # [ 75.017633] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4752worker1 # [ 75.218420] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_connected addr=lb:500514753worker1 # [ 75.223497] nix-grpc-daemon[776]: ts=2026-09-21T09:35:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4754worker1 # [ 75.424209] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_connected addr=lb:500514755worker1 # [ 75.429218] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4756worker2 # [ 75.488138] nix-grpc-daemon[777]: E0000 00:00:1789983346.096848 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4757worker2 # [ 75.508176] nix-grpc-daemon[777]: E0000 00:00:1789983346.118037 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4758worker1 # [ 75.631821] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_connected addr=lb:500514759worker1 # [ 75.636106] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4760worker2 # [ 75.836415] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4761worker2 # [ 75.845283] systemd[1]: Stopped Nix worker-protocol over gRPC.4762worker2 # [ 75.847919] systemd[1]: nix-grpc-daemon.service: Consumed 1.931s CPU time over 55.805s wall clock time, 54.8M memory peak, 56.2K incoming IP traffic, 147.9K outgoing IP traffic.4763worker2 # [ 75.864887] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4764worker2 # [ 75.867065] systemd[1]: Closed Nix worker-protocol over gRPC.4765worker2 # [ 75.869504] systemd[1]: nix-grpc-daemon.socket: Consumed 555us CPU time over 1min 4.526s wall clock time, 256K memory peak, 74.7K incoming IP traffic, 63.7K outgoing IP traffic.4766lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4767lb: must succeed: curl -sf localhost:9901/clusters4768worker1 # [ 75.832418] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_connected addr=lb:500514769worker1 # [ 75.837329] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4770lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4771lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4772client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24773worker1 # [ 76.038174] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_connected addr=lb:500514774worker1 # [ 76.043320] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4775worker1 # [ 76.173268] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520164776worker1 # [ 76.182619] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:520164777client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4778client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.30 seconds)4779worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4780worker1 # [ 76.243968] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_connected addr=lb:500514781worker1 # [ 76.249000] nix-grpc-daemon[776]: ts=2026-09-21T09:35:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4782worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4783worker2 # [ 76.369522] systemd[1]: Listening on Nix worker-protocol over gRPC.4784worker1 # [ 76.398260] systemd[1]: Stopping Nix worker-protocol over gRPC...4785worker1 # [ 76.450187] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_connected addr=lb:500514786worker1 # [ 76.455346] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4787worker1 # [ 76.657903] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_connected addr=lb:500514788worker1 # [ 76.661964] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4789worker2 # [ 76.825702] systemd[1]: Starting scheduler-token.service...4790worker1 # [ 76.858454] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_connected addr=lb:500514791worker1 # [ 76.863410] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4792worker2 # [ 76.937211] systemd[1]: scheduler-token.service: Deactivated successfully.4793worker2 # [ 76.944973] systemd[1]: Finished scheduler-token.service.4794worker2 # [ 76.946870] systemd[1]: scheduler-token.service: Consumed 51ms CPU time over 115ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4795worker2 # [ 76.955447] systemd[1]: Starting Nix worker-protocol over gRPC...4796worker1 # [ 77.063980] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_connected addr=lb:500514797worker1 # [ 77.068996] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4798worker2 # [ 77.146633] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4799worker2 # [ 77.149051] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4800worker2 # [ 77.153013] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14801worker2 # [ 77.160055] nix-grpc-daemon[1104]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4802worker2 # [ 77.161481] nix-grpc-daemon[1104]: E0000 00:00:1789983347.769541 1104 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4803worker2 # [ 77.173525] nix-grpc-daemon[1104]: E0000 00:00:1789983347.775462 1104 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4804worker2 # [ 77.178752] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4805worker2 # [ 77.183068] systemd[1]: Started Nix worker-protocol over gRPC.4806worker2 # [ 77.186253] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:47Z level=info event=scheduler_take_over4807lb # [ 77.142676] niks3-server[974]: 2026/09/21 09:35:47 INFO lead: acquired remote=192.168.1.4:416564808worker2 # [ 77.192065] nix-grpc-daemon[1104]: E0000 00:00:1789983347.799205 1144 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4809worker2 # [ 77.216230] nix-grpc-daemon[1104]: E0000 00:00:1789983347.816752 1144 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4810worker2 # [ 77.218520] nix-grpc-daemon[1104]: E0000 00:00:1789983347.816915 1144 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4811worker2 # [ 77.226951] nix-grpc-daemon[1104]: E0000 00:00:1789983347.836317 1144 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4812worker2 # [ 77.239179] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:47Z level=info event=scheduler_connected addr=lb:500514813worker2 # [ 77.240801] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4814worker1 # [ 77.269677] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_connected addr=lb:500514815worker1 # [ 77.274677] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4816worker1 # [ 77.288310] nix-grpc-daemon[776]: ts=2026-09-21T09:35:47Z level=info event=signal_exit4817worker2 # [ 77.442246] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514818worker2 # [ 77.447165] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4819worker1 # [ 77.475363] nix-grpc-daemon[776]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514820worker1 # [ 77.480681] nix-grpc-daemon[776]: ts=2026-09-21T09:35:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4821worker2 # [ 77.648374] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514822worker2 # [ 77.653475] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4823worker1 # [ 77.683201] nix-grpc-daemon[776]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514824worker1 # [ 77.687331] nix-grpc-daemon[776]: ts=2026-09-21T09:35:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4825worker2 # [ 77.854571] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514826worker2 # [ 77.859786] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4827worker1 # [ 77.883939] nix-grpc-daemon[776]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514828worker1 # [ 77.889007] nix-grpc-daemon[776]: ts=2026-09-21T09:35:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4829worker2 # [ 78.060872] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514830worker2 # [ 78.065974] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4831worker1 # [ 78.089728] nix-grpc-daemon[776]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514832worker1 # [ 78.094766] nix-grpc-daemon[776]: ts=2026-09-21T09:35:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4833worker2 # [ 78.268419] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514834worker2 # [ 78.284138] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=oidc_jwks provider=mock keys=14835worker2 # [ 78.288525] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:373904836worker2 # [ 78.293820] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04837worker1 # [ 78.295548] nix-grpc-daemon[776]: ts=2026-09-21T09:35:48Z level=info event=scheduler_connected addr=lb:500514838worker2 # [ 78.343749] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:373904839worker2 # [ 78.348783] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04840worker1 # [ 78.546470] nix-grpc-daemon[776]: ts=2026-09-21T09:35:49Z level=info event=scheduler_restarting4841worker2 # [ 78.603347] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:373904842worker2 # [ 78.608341] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:49Z level=info event=worker_gone addr=192.168.1.3:500514843worker1 # [ 78.587992] nix-grpc-daemon[776]: ts=2026-09-21T09:35:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4844worker1 # [ 78.606123] nix-daemon[839]: reaped child process 841, status = succeeded4845worker1 # [ 79.411432] nix-grpc-daemon[776]: E0000 00:00:1789983350.037022 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4846worker1 # [ 79.418226] nix-grpc-daemon[776]: E0000 00:00:1789983350.037260 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4847worker1 # [ 79.719286] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4848worker1 # [ 79.728304] systemd[1]: Stopped Nix worker-protocol over gRPC.4849worker1 # [ 79.730847] systemd[1]: nix-grpc-daemon.service: Consumed 3.520s CPU time over 59.839s wall clock time, 63.2M memory peak, 87.6K incoming IP traffic, 159.9K outgoing IP traffic.4850worker1 # [ 79.748857] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4851worker1 # [ 79.751884] systemd[1]: Closed Nix worker-protocol over gRPC.4852worker1 # [ 79.754924] systemd[1]: nix-grpc-daemon.socket: Consumed 563us CPU time over 1min 8.952s wall clock time, 256K memory peak, 77.7K incoming IP traffic, 63.6K outgoing IP traffic.4853lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4854lb: must succeed: curl -sf localhost:9901/clusters4855lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4856lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4857client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24858lb # [ 79.865458] envoy[917]: conn peer=[2001:db8:1::1]:40792 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3739 tx=1994 ms=18104859lb # [ 79.870026] envoy[917]: conn peer=[2001:db8:1::1]:53230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=264860lb # [ 79.874329] envoy[917]: conn peer=[2001:db8:1::4]:36934 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=34465 tx=1899 ms=542094861lb # [ 79.878973] envoy[917]: conn peer=[2001:db8:1::1]:53236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=284862lb # [ 79.883528] envoy[917]: conn peer=[2001:db8:1::3]:44964 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10435 tx=2895 ms=582034863worker2 # [ 80.157311] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373984864worker2 # [ 80.173437] nix-daemon[860]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4865worker2 # [ 80.202770] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5)4866worker2 # [ 80.223681] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 2/5)4867worker2 # [ 80.301350] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 325 ms (attempt 3/5)4868worker2 # [ 80.634131] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 371 ms (attempt 4/5)4869worker2 # [ 81.006036] nix-grpc-daemon[1104]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4870worker2 # [ 81.012221] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4871client # copying 2 paths...4872client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4873worker2 # [ 81.036468] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373984874client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4875worker2 # [ 81.054604] nix-daemon[860]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4876worker2 # [ 81.084696] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:51Z level=info event=niks3_push_started pid=11644877worker2 # [ 81.300335] nix-grpc-daemon[1164]: time=2026-09-21T09:35:51.909Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4878worker2 # [ 81.473069] nix-daemon[860]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)4879worker2 # [ 81.490427] nix-daemon[860]: reaped child process 1180, status = succeeded4880lb # [ 81.461330] niks3-server[974]: 2026/09/21 09:35:52 INFO Received uploads request method=POST path=/api/pending_closures4881lb # [ 81.470988] niks3-server[974]: 2026/09/21 09:35:52 INFO Received uploads request method=POST path=/api/pending_closures4882worker2 # [ 81.524073] nix-grpc-daemon[1164]: time=2026-09-21T09:35:52.132Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4883worker2 # [ 81.525531] nix-grpc-daemon[1164]: time=2026-09-21T09:35:52.132Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4884worker2 # [ 81.527240] nix-grpc-daemon[1164]: time=2026-09-21T09:35:52.132Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4885lb # [ 81.515155] niks3-server[974]: 2026/09/21 09:35:52 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4886lb # [ 81.531073] niks3-server[974]: 2026/09/21 09:35:52 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4887lb # [ 81.542166] niks3-server[974]: 2026/09/21 09:35:52 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4888lb # [ 81.551647] niks3-server[974]: 2026/09/21 09:35:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4889lb # [ 81.560652] niks3-server[974]: 2026/09/21 09:35:52 INFO Signed narinfos id=25 count=14890worker2 # [ 81.604214] nix-grpc-daemon[1164]: time=2026-09-21T09:35:52.213Z level=INFO msg="Uploading 3 narinfos"4891lb # [ 81.564354] niks3-server[974]: 2026/09/21 09:35:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4892lb # [ 81.569693] niks3-server[974]: 2026/09/21 09:35:52 INFO Signed narinfos id=26 count=24893lb # [ 81.574211] niks3-server[974]: 2026/09/21 09:35:52 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4894lb # [ 81.594016] niks3-server[974]: 2026/09/21 09:35:52 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4895lb # [ 81.600287] niks3-server[974]: 2026/09/21 09:35:52 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4896lb # [ 81.604565] niks3-server[974]: 2026/09/21 09:35:52 INFO Completed upload id=254897lb # [ 81.610337] niks3-server[974]: 2026/09/21 09:35:52 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4898lb # [ 81.617356] niks3-server[974]: 2026/09/21 09:35:52 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4899worker2 # [ 81.659164] nix-grpc-daemon[1164]: time=2026-09-21T09:35:52.269Z level=INFO msg="Upload complete. (354ms)"4900lb # [ 81.619804] niks3-server[974]: 2026/09/21 09:35:52 INFO Completed upload id=264901worker2 # [ 81.669218] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37398 duration_s=0 paths=2 nar_bytes_in=3124902worker2 # [ 81.688117] nix-grpc-daemon[1164]: time=2026-09-21T09:35:52.287Z level=WARN msg="Failed to register uploaded object" key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo error="server returned 404: object is not pending upload\n"4903worker2 # [ 81.694925] nix-daemon[860]: reaped child process 1162, status = succeeded4904client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.83 seconds)4905worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4906worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)4907worker1 # [ 81.791753] systemd[1]: Listening on Nix worker-protocol over gRPC.4908worker1 # [ 81.801343] systemd[1]: Starting Nix worker-protocol over gRPC...4909worker1 # [ 82.002366] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4910worker1 # [ 82.004975] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4911worker1 # [ 82.009177] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14912worker1 # [ 82.016059] nix-grpc-daemon[1213]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4913worker1 # [ 82.017518] nix-grpc-daemon[1213]: E0000 00:00:1789983352.642461 1213 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4914worker1 # [ 82.028079] nix-grpc-daemon[1213]: E0000 00:00:1789983352.648939 1213 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4915worker1 # [ 82.031287] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4916worker1 # [ 82.035636] systemd[1]: Started Nix worker-protocol over gRPC.4917lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4918lb: must succeed: curl -sf localhost:9901/clusters4919worker1 # [ 82.040100] nix-grpc-daemon[1213]: E0000 00:00:1789983352.665776 1252 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4920worker1 # [ 82.068168] nix-grpc-daemon[1213]: E0000 00:00:1789983352.683391 1252 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4921worker1 # [ 82.070471] nix-grpc-daemon[1213]: E0000 00:00:1789983352.683554 1252 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4922worker1 # [ 82.077047] nix-grpc-daemon[1213]: E0000 00:00:1789983352.702751 1252 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4923worker1 # [ 82.088236] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:52Z level=info event=scheduler_connected addr=lb:500514924worker2 # [ 82.137397] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:373904925worker2 # [ 82.142997] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04926lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4927lb: must succeed: curl -sf localhost:9901/clusters4928lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4929lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)4930(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.38 seconds)4931subtest: scheduler down: the other node takes the lock and keeps it4932lb: must succeed: curl -sf localhost:9901/clusters4933lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4934lb: must succeed: curl -sf localhost:9901/clusters4935lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4936lb: must succeed: curl -sf localhost:9901/clusters4937lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4938lb: must succeed: curl -sf localhost:9901/clusters4939lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4940worker2 # [ 83.513149] systemd[1]: Stopping Nix worker-protocol over gRPC...4941worker2 # [ 84.177381] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:54Z level=info event=signal_exit4942worker2 # [ 84.388355] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:54Z level=info event=scheduler_yield4943lb # [ 84.350344] niks3-server[974]: 2026/09/21 09:35:54 INFO lead: released remote=192.168.1.4:416564944worker1 # [ 84.348715] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:54Z level=info event=scheduler_restarting addr=lb:500514945worker2 # [ 84.395370] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:373904946worker2 # [ 84.400218] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=worker_gone addr=192.168.1.3:500514947worker2 # [ 84.407195] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=scheduler_restarting addr=lb:500514948worker2 # [ 84.411396] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:373904949worker2 # [ 84.415919] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=worker_gone addr=192.168.1.4:500514950worker1 # [ 84.548727] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:55Z level=info event=scheduler_connected addr=lb:500514951worker1 # [ 84.553885] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4952worker2 # [ 84.607057] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=scheduler_connected addr=lb:500514953worker2 # [ 84.612225] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4954worker1 # [ 84.754499] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:55Z level=info event=scheduler_connected addr=lb:500514955worker1 # [ 84.759756] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4956worker2 # [ 84.813264] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=scheduler_connected addr=lb:500514957worker2 # [ 84.818363] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4958worker1 # [ 84.960610] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:55Z level=info event=scheduler_connected addr=lb:500514959worker1 # [ 84.965629] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4960worker2 # [ 85.019526] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=scheduler_connected addr=lb:500514961worker2 # [ 85.024416] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4962worker1 # [ 85.166566] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:55Z level=info event=scheduler_connected addr=lb:500514963worker1 # [ 85.171767] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4964worker2 # [ 85.227098] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=scheduler_connected addr=lb:500514965worker2 # [ 85.231189] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4966worker2 # [ 85.393582] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:56Z level=info event=scheduler_restarting4967worker1 # [ 85.374253] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:55Z level=info event=scheduler_connected addr=lb:500514968worker2 # [ 85.423587] nix-daemon[860]: reaped child process 1156, status = succeeded4969worker1 # [ 85.378494] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4970worker2 # [ 85.446733] nix-grpc-daemon[1104]: ts=2026-09-21T09:35:56Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4971worker1 # [ 85.574845] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:56Z level=info event=scheduler_connected addr=lb:500514972worker1 # [ 85.580130] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4973worker1 # [ 85.780890] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:56Z level=info event=scheduler_connected addr=lb:500514974worker1 # [ 85.785871] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4975worker1 # [ 85.986642] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:56Z level=info event=scheduler_connected addr=lb:500514976worker1 # [ 85.991735] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4977worker2 # [ 86.206851] nix-grpc-daemon[1104]: E0000 00:00:1789983356.816626 1149 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4978worker2 # [ 86.207213] nix-grpc-daemon[1104]: E0000 00:00:1789983356.816860 1149 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4979worker1 # [ 86.192639] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:56Z level=info event=scheduler_connected addr=lb:500514980worker1 # [ 86.197770] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4981worker1 # [ 86.400603] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_connected addr=lb:500514982worker1 # [ 86.404903] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4983worker1 # [ 86.601153] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_connected addr=lb:500514984worker1 # [ 86.606212] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4985worker1 # [ 86.806863] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_connected addr=lb:500514986worker1 # [ 86.811965] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4987worker1 # [ 87.012824] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_connected addr=lb:500514988worker1 # [ 87.018094] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4989worker1 # [ 87.037581] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_take_over4990lb # [ 87.041235] niks3-server[974]: 2026/09/21 09:35:57 INFO lead: acquired remote=192.168.1.3:396764991worker2 # [ 87.198581] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4992worker2 # [ 87.203121] systemd[1]: Stopped Nix worker-protocol over gRPC.4993worker2 # [ 87.211619] systemd[1]: nix-grpc-daemon.service: Consumed 492ms CPU time over 10.246s wall clock time, 33.5M memory peak, 20.5K incoming IP traffic, 30.7K outgoing IP traffic.4994worker2 # [ 87.223190] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4995worker2 # [ 87.228301] systemd[1]: Closed Nix worker-protocol over gRPC.4996worker2 # [ 87.230131] systemd[1]: nix-grpc-daemon.socket: Consumed 755us CPU time over 10.860s wall clock time, 256K memory peak, 21.2K incoming IP traffic, 16K outgoing IP traffic.4997worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over4998worker1 # [ 87.218911] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_connected addr=lb:500514999worker1 # [ 87.230912] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5000worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5001worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5002worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5003worker1 # [ 87.430167] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:58Z level=info event=scheduler_connected addr=lb:500515004worker1 # [ 87.435253] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5005worker1 # [ 87.636140] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:58Z level=info event=scheduler_connected addr=lb:500515006worker1 # [ 87.641286] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5007worker1 # [ 87.841828] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:58Z level=info event=scheduler_connected addr=lb:500515008worker1 # [ 87.850873] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:58Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:508985009worker1 # [ 87.856489] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05010worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5011worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5012client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25013worker1 # [ 88.678006] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:509045014worker1 # [ 88.691015] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5015client # don't know how to build these paths:5016client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5017worker1 # [ 88.711417] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:509045018worker1 # [ 88.729379] nix-grpc-daemon[1213]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5)5019worker1 # [ 88.750308] nix-grpc-daemon[1213]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 2/5)5020worker1 # [ 88.805024] nix-grpc-daemon[1213]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 279 ms (attempt 3/5)5021worker1 # [ 89.091665] nix-grpc-daemon[1213]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 243 ms (attempt 4/5)5022worker1 # [ 89.335574] nix-grpc-daemon[1213]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5023worker1 # [ 89.341585] nix-grpc-daemon[1213]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5024worker1 # [ 89.373272] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:59Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05025worker1 # [ 89.380258] nix-grpc-daemon[1213]: ts=2026-09-21T09:35:59Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15026worker1 # [ 89.421850] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:509045027worker1 # [ 89.437349] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:509045028client # copying 1 paths to 'lb:50051'...5029client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5030worker1 # [ 89.451607] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:509045031worker1 # [ 89.466310] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5032worker1 # [ 89.495872] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:00Z level=info event=niks3_push_started pid=12875033worker1 # [ 89.708670] nix-grpc-daemon[1287]: time=2026-09-21T09:36:00.335Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5034lb # [ 89.865705] envoy[917]: conn peer=[2001:db8:1::1]:53252 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1010 tx=304 ms=15295035lb # [ 89.873165] envoy[917]: conn peer=[2001:db8:1::4]:56798 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7963 tx=535 ms=81985036worker1 # [ 89.882270] nix-daemon[839]: accepted connection from pid 1300, user nix-grpc-daemon (trusted)5037worker1 # [ 89.899310] nix-daemon[839]: reaped child process 1303, status = succeeded5038worker1 # [ 90.074866] nix-daemon[839]: accepted connection from pid 1305, user nix-grpc-daemon (trusted)5039worker1 # [ 90.091860] nix-daemon[839]: reaped child process 1308, status = succeeded5040lb # [ 90.107027] niks3-server[974]: 2026/09/21 09:36:00 INFO Received uploads request method=POST path=/api/pending_closures5041worker1 # [ 90.114699] nix-grpc-daemon[1287]: time=2026-09-21T09:36:00.740Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5042worker1 # [ 90.116480] nix-grpc-daemon[1287]: time=2026-09-21T09:36:00.740Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5043lb # [ 90.145829] niks3-server[974]: 2026/09/21 09:36:00 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5044worker1 # [ 90.156951] nix-grpc-daemon[1287]: time=2026-09-21T09:36:00.783Z level=INFO msg="Uploading 1 narinfos"5045lb # [ 90.159411] niks3-server[974]: 2026/09/21 09:36:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5046lb # [ 90.166250] niks3-server[974]: 2026/09/21 09:36:00 INFO Signed narinfos id=27 count=15047lb # [ 90.174728] niks3-server[974]: 2026/09/21 09:36:00 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5048lb # [ 90.188628] niks3-server[974]: 2026/09/21 09:36:00 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5049worker1 # [ 90.191323] nix-grpc-daemon[1287]: time=2026-09-21T09:36:00.818Z level=INFO msg="Upload complete. (478ms)"5050lb # [ 90.195283] niks3-server[974]: 2026/09/21 09:36:00 INFO Completed upload id=275051lb # [ 90.199575] niks3-server[974]: 2026/09/21 09:36:00 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5052worker1 # [ 90.200884] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50904 duration_s=0 paths=1 nar_bytes_in=5205053worker1 # [ 90.214489] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:509045054worker1 # [ 90.224308] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5055worker1 # [ 90.227485] nix-daemon[839]: reaped child process 1285, status = succeeded5056client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5057client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5058worker1 # [ 90.240866] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5059client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5060client # farm-a> LOG-farm-a-on-standby5061worker1 # [ 91.286884] nix-daemon[839]: reaped child process 1312, status = succeeded5062worker1 # [ 91.470412] nix-daemon[839]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5063worker1 # [ 91.486903] nix-daemon[839]: reaped child process 1331, status = succeeded5064lb # [ 91.501805] niks3-server[974]: 2026/09/21 09:36:02 INFO Received uploads request method=POST path=/api/pending_closures5065worker1 # [ 91.511773] nix-grpc-daemon[1287]: time=2026-09-21T09:36:02.135Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5066worker1 # [ 91.514068] nix-grpc-daemon[1287]: time=2026-09-21T09:36:02.135Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5067lb # [ 91.535436] niks3-server[974]: 2026/09/21 09:36:02 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5068lb # [ 91.556994] niks3-server[974]: 2026/09/21 09:36:02 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5069lb # [ 91.567951] niks3-server[974]: 2026/09/21 09:36:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5070worker1 # [ 91.571718] nix-grpc-daemon[1287]: time=2026-09-21T09:36:02.197Z level=INFO msg="Uploading 1 narinfos"5071lb # [ 91.578730] niks3-server[974]: 2026/09/21 09:36:02 INFO Signed narinfos id=28 count=15072lb # [ 91.584824] niks3-server[974]: 2026/09/21 09:36:02 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5073lb # [ 91.592632] niks3-server[974]: 2026/09/21 09:36:02 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5074lb # [ 91.598447] niks3-server[974]: 2026/09/21 09:36:02 INFO Completed upload id=285075worker1 # [ 91.595188] nix-grpc-daemon[1287]: time=2026-09-21T09:36:02.222Z level=INFO msg="Upload complete. (344ms)"5076lb # [ 91.600862] niks3-server[974]: 2026/09/21 09:36:02 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5077worker1 # [ 91.618530] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50904 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15078worker1 # [ 91.635557] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:02Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05079worker1 # [ 91.639006] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:02Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15080worker1 # [ 91.642923] nix-daemon[839]: reaped child process 1310, status = succeeded5081worker1 # [ 91.667358] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:509045082worker1 # [ 91.683451] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:509045083worker1 # [ 91.699035] nix-grpc-daemon[1287]: time=2026-09-21T09:36:02.325Z level=INFO msg="All 1 paths already cached"5084client # copying 1 paths to 'lb:50051'...5085client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5086worker1 # [ 91.709611] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:509045087worker1 # [ 91.723695] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5088worker1 # [ 91.928607] nix-daemon[839]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5089worker1 # [ 91.948580] nix-daemon[839]: reaped child process 1340, status = succeeded5090worker1 # [ 92.131116] nix-daemon[839]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5091worker1 # [ 92.149222] nix-daemon[839]: reaped child process 1345, status = succeeded5092lb # [ 92.163197] niks3-server[974]: 2026/09/21 09:36:02 INFO Received uploads request method=POST path=/api/pending_closures5093worker1 # [ 92.170757] nix-grpc-daemon[1287]: time=2026-09-21T09:36:02.797Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5094worker1 # [ 92.172394] nix-grpc-daemon[1287]: time=2026-09-21T09:36:02.797Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5095lb # [ 92.191892] niks3-server[974]: 2026/09/21 09:36:02 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5096lb # [ 92.207429] niks3-server[974]: 2026/09/21 09:36:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5097worker1 # [ 92.210940] nix-grpc-daemon[1287]: time=2026-09-21T09:36:02.837Z level=INFO msg="Uploading 1 narinfos"5098lb # [ 92.220774] niks3-server[974]: 2026/09/21 09:36:02 INFO Signed narinfos id=29 count=15099lb # [ 92.223870] niks3-server[974]: 2026/09/21 09:36:02 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5100lb # [ 92.237968] niks3-server[974]: 2026/09/21 09:36:02 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5101worker1 # [ 92.240858] nix-grpc-daemon[1287]: time=2026-09-21T09:36:02.867Z level=INFO msg="Upload complete. (499ms)"5102lb # [ 92.244914] niks3-server[974]: 2026/09/21 09:36:02 INFO Completed upload id=295103lb # [ 92.250397] niks3-server[974]: 2026/09/21 09:36:02 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5104worker1 # [ 92.252135] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50904 duration_s=0 paths=1 nar_bytes_in=6405105worker1 # [ 92.263597] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:509045106worker1 # [ 92.271571] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5107worker1 # [ 92.277466] nix-daemon[839]: reaped child process 1335, status = succeeded5108client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5109client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5110worker1 # [ 92.292769] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5111client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5112client # farm-b> LOG-farm-b-on-standby5113worker1 # [ 93.776004] nix-daemon[839]: reaped child process 1349, status = succeeded5114worker1 # [ 93.963221] nix-daemon[839]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5115worker1 # [ 93.982548] nix-daemon[839]: reaped child process 1368, status = succeeded5116lb # [ 93.998520] niks3-server[974]: 2026/09/21 09:36:04 INFO Received uploads request method=POST path=/api/pending_closures5117worker1 # [ 94.007956] nix-grpc-daemon[1287]: time=2026-09-21T09:36:04.633Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5118worker1 # [ 94.009938] nix-grpc-daemon[1287]: time=2026-09-21T09:36:04.633Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5119lb # [ 94.040374] niks3-server[974]: 2026/09/21 09:36:04 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5120lb # [ 94.051828] niks3-server[974]: 2026/09/21 09:36:04 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5121worker1 # [ 94.056188] nix-grpc-daemon[1287]: time=2026-09-21T09:36:04.682Z level=INFO msg="Uploading 1 narinfos"5122lb # [ 94.060328] niks3-server[974]: 2026/09/21 09:36:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5123lb # [ 94.067353] niks3-server[974]: 2026/09/21 09:36:04 INFO Signed narinfos id=30 count=15124lb # [ 94.072963] niks3-server[974]: 2026/09/21 09:36:04 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5125lb # [ 94.081805] niks3-server[974]: 2026/09/21 09:36:04 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5126lb # [ 94.087938] niks3-server[974]: 2026/09/21 09:36:04 INFO Completed upload id=305127worker1 # [ 94.085284] nix-grpc-daemon[1287]: time=2026-09-21T09:36:04.712Z level=INFO msg="Upload complete. (378ms)"5128lb # [ 94.090990] niks3-server[974]: 2026/09/21 09:36:04 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5129worker1 # [ 94.104573] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50904 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15130worker1 # [ 94.120399] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:04Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05131worker1 # [ 94.124315] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:04Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15132worker1 # [ 94.128462] nix-daemon[839]: reaped child process 1347, status = succeeded5133worker1 # [ 94.149992] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:509045134worker1 # [ 94.166402] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:509045135client # copying 1 paths to 'lb:50051'...5136worker1 # [ 94.181164] nix-grpc-daemon[1287]: time=2026-09-21T09:36:04.808Z level=INFO msg="All 2 paths already cached"5137client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5138worker1 # [ 94.188119] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:509045139worker1 # [ 94.201446] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5140worker1 # [ 94.397413] nix-daemon[839]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)5141worker1 # [ 94.414349] nix-daemon[839]: reaped child process 1377, status = succeeded5142worker1 # [ 94.583271] nix-daemon[839]: accepted connection from pid 1379, user nix-grpc-daemon (trusted)5143worker1 # [ 94.599259] nix-daemon[839]: reaped child process 1382, status = succeeded5144lb # [ 94.615587] niks3-server[974]: 2026/09/21 09:36:05 INFO Received uploads request method=POST path=/api/pending_closures5145worker1 # [ 94.621445] nix-grpc-daemon[1287]: time=2026-09-21T09:36:05.248Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5146worker1 # [ 94.622938] nix-grpc-daemon[1287]: time=2026-09-21T09:36:05.248Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5147lb # [ 94.641754] niks3-server[974]: 2026/09/21 09:36:05 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5148worker1 # [ 94.658092] nix-grpc-daemon[1287]: time=2026-09-21T09:36:05.284Z level=INFO msg="Uploading 1 narinfos"5149lb # [ 94.655104] niks3-server[974]: 2026/09/21 09:36:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5150lb # [ 94.667785] niks3-server[974]: 2026/09/21 09:36:05 INFO Signed narinfos id=31 count=15151lb # [ 94.674288] niks3-server[974]: 2026/09/21 09:36:05 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5152lb # [ 94.683530] niks3-server[974]: 2026/09/21 09:36:05 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5153worker1 # [ 94.685321] nix-grpc-daemon[1287]: time=2026-09-21T09:36:05.312Z level=INFO msg="Upload complete. (468ms)"5154lb # [ 94.688645] niks3-server[974]: 2026/09/21 09:36:05 INFO Completed upload id=315155lb # [ 94.692720] niks3-server[974]: 2026/09/21 09:36:05 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5156worker1 # [ 94.692139] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50904 duration_s=0 paths=1 nar_bytes_in=7685157worker1 # [ 94.705662] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:509045158worker1 # [ 94.712624] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5159worker1 # [ 94.717204] nix-daemon[839]: reaped child process 1372, status = succeeded5160client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5161client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5162worker1 # [ 94.729990] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5163client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5164client # farm-top> LOG-farm-top-on-standby5165worker1 # [ 95.354851] nix-daemon[839]: reaped child process 1386, status = succeeded5166worker1 # [ 95.528444] nix-daemon[839]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)5167worker1 # [ 95.546686] nix-daemon[839]: reaped child process 1405, status = succeeded5168lb # [ 95.561864] niks3-server[974]: 2026/09/21 09:36:06 INFO Received uploads request method=POST path=/api/pending_closures5169worker1 # [ 95.568359] nix-grpc-daemon[1287]: time=2026-09-21T09:36:06.195Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5170worker1 # [ 95.571576] nix-grpc-daemon[1287]: time=2026-09-21T09:36:06.196Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5171lb # [ 95.591220] niks3-server[974]: 2026/09/21 09:36:06 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5172lb # [ 95.609157] niks3-server[974]: 2026/09/21 09:36:06 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5173lb # [ 95.623451] niks3-server[974]: 2026/09/21 09:36:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5174worker1 # [ 95.626595] nix-grpc-daemon[1287]: time=2026-09-21T09:36:06.253Z level=INFO msg="Uploading 1 narinfos"5175lb # [ 95.634731] niks3-server[974]: 2026/09/21 09:36:06 INFO Signed narinfos id=32 count=15176lb # [ 95.640571] niks3-server[974]: 2026/09/21 09:36:06 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5177lb # [ 95.646792] niks3-server[974]: 2026/09/21 09:36:06 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5178lb # [ 95.653669] niks3-server[974]: 2026/09/21 09:36:06 INFO Completed upload id=325179lb # [ 95.656490] niks3-server[974]: 2026/09/21 09:36:06 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5180worker1 # [ 95.650393] nix-grpc-daemon[1287]: time=2026-09-21T09:36:06.277Z level=INFO msg="Upload complete. (310ms)"5181worker1 # [ 95.665204] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50904 duration_s=0 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15182worker1 # [ 95.679517] nix-daemon[839]: reaped child process 1384, status = succeeded5183client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 7.31 seconds)5184client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5185client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5186worker2 # [ 96.098924] systemd[1]: Listening on Nix worker-protocol over gRPC.5187worker2 # [ 96.107861] systemd[1]: Starting scheduler-token.service...5188worker2 # [ 96.208777] systemd[1]: scheduler-token.service: Deactivated successfully.5189worker2 # [ 96.214738] systemd[1]: Finished scheduler-token.service.5190worker2 # [ 96.216393] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 101ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5191worker2 # [ 96.221958] systemd[1]: Starting Nix worker-protocol over gRPC...5192worker2 # [ 96.401987] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5193worker2 # [ 96.404799] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5194worker2 # [ 96.409172] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15195worker2 # [ 96.416070] nix-grpc-daemon[1204]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5196worker2 # [ 96.417523] nix-grpc-daemon[1204]: E0000 00:00:1789983367.025675 1204 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5197worker2 # [ 96.433288] nix-grpc-daemon[1204]: E0000 00:00:1789983367.032095 1204 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5198worker2 # [ 96.438542] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5199lb: must succeed: curl -sf localhost:9901/clusters5200worker2 # [ 96.452234] systemd[1]: Started Nix worker-protocol over gRPC.5201worker2 # [ 96.453123] nix-grpc-daemon[1204]: E0000 00:00:1789983367.054517 1243 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5202worker2 # [ 96.479272] nix-grpc-daemon[1204]: E0000 00:00:1789983367.073888 1243 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5203worker2 # [ 96.482169] nix-grpc-daemon[1204]: E0000 00:00:1789983367.074038 1243 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5204worker2 # [ 96.484659] nix-grpc-daemon[1204]: E0000 00:00:1789983367.088466 1243 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5205worker2 # [ 96.496283] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:07Z level=info event=scheduler_connected addr=lb:500515206worker1 # [ 96.460150] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:07Z level=info event=oidc_jwks provider=mock keys=15207worker1 # [ 96.464967] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:07Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:508985208worker1 # [ 96.470339] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05209lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5210worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5211worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5212lb: must succeed: curl -sf localhost:9901/clusters5213lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5214lb: must succeed: curl -sf localhost:9901/clusters5215lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5216worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5217worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5218(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.56 seconds)5219subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5220client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5221client # [ 96.849917] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5222client # Running as unit: rs.service; invocation ID: d79c9da2c29a418d9aeb9779761a725f5223client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5224worker1 # [ 97.095273] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:509045225client # [ 97.113017] nix[1165]: don't know how to build these paths:5226client # [ 97.114043] nix[1165]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5227worker1 # [ 97.114733] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:07Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05228worker1 # [ 97.121739] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:07Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15229worker1 # [ 97.161671] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:509045230worker1 # [ 97.177510] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:509045231client # [ 97.199043] nix[1165]: copying 1 paths to 'lb:50051'...5232client # [ 97.205070] nix[1165]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5233worker1 # [ 97.194104] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:509045234worker1 # [ 97.211112] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5235worker1 # [ 97.414354] nix-daemon[839]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5236worker1 # [ 97.433110] nix-daemon[839]: reaped child process 1421, status = succeeded5237worker1 # [ 97.605990] nix-daemon[839]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5238worker1 # [ 97.622275] nix-daemon[839]: reaped child process 1426, status = succeeded5239lb # [ 97.642816] niks3-server[974]: 2026/09/21 09:36:08 INFO Received uploads request method=POST path=/api/pending_closures5240worker1 # [ 97.653259] nix-grpc-daemon[1287]: time=2026-09-21T09:36:08.279Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5241worker1 # [ 97.654698] nix-grpc-daemon[1287]: time=2026-09-21T09:36:08.279Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5242lb # [ 97.677112] niks3-server[974]: 2026/09/21 09:36:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5243worker1 # [ 97.686241] nix-grpc-daemon[1287]: time=2026-09-21T09:36:08.313Z level=INFO msg="Uploading 1 narinfos"5244lb # [ 97.693716] niks3-server[974]: 2026/09/21 09:36:08 INFO Signed narinfos id=33 count=15245lb # [ 97.702263] niks3-server[974]: 2026/09/21 09:36:08 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5246lb # [ 97.714860] niks3-server[974]: 2026/09/21 09:36:08 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5247lb # [ 97.722968] niks3-server[974]: 2026/09/21 09:36:08 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5248lb # [ 97.727617] niks3-server[974]: 2026/09/21 09:36:08 INFO Completed upload id=335249worker1 # [ 97.724893] nix-grpc-daemon[1287]: time=2026-09-21T09:36:08.351Z level=INFO msg="Upload complete. (496ms)"5250lb # [ 97.730347] niks3-server[974]: 2026/09/21 09:36:08 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5251worker1 # [ 97.732153] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50904 duration_s=0 paths=1 nar_bytes_in=4165252worker1 # [ 97.746097] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:509045253worker1 # [ 97.756073] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5254worker1 # [ 97.758570] nix-daemon[839]: reaped child process 1416, status = succeeded5255client # [ 97.779346] nix[1165]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5256worker1 # [ 97.773207] nix-daemon[839]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5257lb: must succeed: curl -sf localhost:9901/clusters5258lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5259lb: must succeed: curl -sf localhost:9901/clusters5260lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5261lb: must succeed: curl -sf localhost:9901/clusters5262lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5263lb: must succeed: curl -sf localhost:9901/clusters5264lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5265worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5266worker2 # [ 99.824780] systemd-journald[274]: Received client request to rotate journal, rotating.5267worker2 # [ 99.847874] systemd-journald[274]: Vacuuming done, freed 0B of archived journals from /var/log/journal/ab255008e5414c8e875cf15eb74662d0.5268worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5269worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5270worker1 # [ 99.862339] systemd[1]: Stopping Nix worker-protocol over gRPC...5271lb # [ 99.867286] envoy[917]: conn peer=[2001:db8:1::1]:57028 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9610 tx=4103 ms=70045272worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5273client: must succeed: sleep 5.05274worker1 # [ 100.034578] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:10Z level=info event=draining5275client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5276worker1 # [ 105.198766] nix-daemon[839]: reaped child process 1430, status = succeeded5277worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5278worker1 # [ 105.356056] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5279worker1 # [ 105.374338] nix-daemon[839]: reaped child process 1474, status = succeeded5280lb # [ 105.391045] niks3-server[974]: 2026/09/21 09:36:16 INFO Received uploads request method=POST path=/api/pending_closures5281worker1 # [ 105.400382] nix-grpc-daemon[1287]: time=2026-09-21T09:36:16.024Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5282worker1 # [ 105.401878] nix-grpc-daemon[1287]: time=2026-09-21T09:36:16.024Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5283lb # [ 105.435660] niks3-server[974]: 2026/09/21 09:36:16 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5284worker1 # [ 105.445140] nix-grpc-daemon[1287]: time=2026-09-21T09:36:16.071Z level=INFO msg="Uploading 1 narinfos"5285lb # [ 105.448209] niks3-server[974]: 2026/09/21 09:36:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5286lb # [ 105.458276] niks3-server[974]: 2026/09/21 09:36:16 INFO Signed narinfos id=34 count=15287lb # [ 105.463811] niks3-server[974]: 2026/09/21 09:36:16 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5288worker1 # [ 105.478476] nix-grpc-daemon[1287]: time=2026-09-21T09:36:16.105Z level=INFO msg="Upload complete. (290ms)"5289lb # [ 105.477021] niks3-server[974]: 2026/09/21 09:36:16 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5290lb # [ 105.486876] niks3-server[974]: 2026/09/21 09:36:16 INFO Completed upload id=345291lb # [ 105.492700] niks3-server[974]: 2026/09/21 09:36:16 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5292worker1 # [ 105.492961] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50904 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15293worker1 # [ 105.504317] nix-daemon[839]: reaped child process 1428, status = succeeded5294lb # [ 105.509367] niks3-server[974]: 2026/09/21 09:36:16 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5295client # [ 105.548325] systemd[1]: rs.service: Deactivated successfully.5296client # [ 105.553556] systemd[1]: rs.service: Consumed 183ms CPU time over 8.699s wall clock time, 41.2M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5297worker1 # [ 106.035621] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:16Z level=info event=signal_exit5298worker1 # [ 106.260115] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:16Z level=info event=scheduler_yield5299lb # [ 106.267774] niks3-server[974]: 2026/09/21 09:36:16 INFO lead: released remote=192.168.1.3:396765300worker1 # [ 106.267264] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:16Z level=info event=scheduler_restarting addr=lb:500515301worker2 # [ 106.314997] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:16Z level=info event=scheduler_restarting addr=lb:500515302worker1 # [ 106.273021] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:508985303worker1 # [ 106.277542] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:16Z level=info event=worker_gone addr=192.168.1.3:500515304worker1 # [ 106.281471] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:508985305worker1 # [ 106.285831] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:16Z level=info event=worker_gone addr=192.168.1.4:500515306worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.13 seconds)5307client: waiting for success: ! systemctl is-active rs5308lb # [ 106.393112] niks3-server[974]: 2026/09/21 09:36:17 INFO lead: acquired remote=192.168.1.4:408385309worker2 # [ 106.440625] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=scheduler_take_over5310client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5311client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5312worker1 # [ 106.468416] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:17Z level=info event=scheduler_connected addr=lb:500515313worker2 # [ 106.515140] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=scheduler_connected addr=lb:500515314worker2 # [ 106.524353] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5315client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds)5316client: must fail: journalctl -u rs | grep -q 'reconnecting'5317worker2 # [ 106.529996] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:420585318worker2 # [ 106.534718] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=0 running=05319client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5320lb: must succeed: curl -sf localhost:9901/clusters5321lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5322worker2 # [ 106.723530] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=scheduler_connected addr=lb:500515323worker2 # [ 106.736112] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=oidc_jwks provider=mock keys=15324worker2 # [ 106.739743] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:420585325worker2 # [ 106.745097] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05326worker2 # [ 107.319111] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:420585327worker1 # [ 107.277101] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:17Z level=info event=scheduler_restarting5328worker2 # [ 107.323882] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:17Z level=info event=worker_gone addr=192.168.1.3:500515329worker1 # [ 107.302289] nix-grpc-daemon[1213]: ts=2026-09-21T09:36:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5330worker1 # [ 107.329200] nix-daemon[839]: reaped child process 1277, status = succeeded5331lb: must succeed: curl -sf localhost:9901/clusters5332lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5333worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5334worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5335worker1 # [ 107.756640] nix-grpc-daemon[1213]: E0000 00:00:1789983378.383394 1258 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5336worker1 # [ 107.777610] nix-grpc-daemon[1213]: E0000 00:00:1789983378.404320 1258 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5337worker1 # [ 108.634701] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5338worker1 # [ 108.637877] systemd[1]: Stopped Nix worker-protocol over gRPC.5339worker1 # [ 108.643788] systemd[1]: nix-grpc-daemon.service: Consumed 1.850s CPU time over 26.831s wall clock time, 36.8M memory peak, 48.5K incoming IP traffic, 90.5K outgoing IP traffic.5340worker1 # [ 108.663686] systemd[1]: Starting Nix worker-protocol over gRPC...5341lb: must succeed: curl -sf localhost:9901/clusters5342lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5343worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5344worker1 # [ 108.856482] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5345worker1 # [ 108.859042] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5346worker1 # [ 108.863194] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15347worker1 # [ 108.870755] nix-grpc-daemon[1481]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5348worker1 # [ 108.877525] nix-grpc-daemon[1481]: E0000 00:00:1789983379.497077 1481 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5349worker1 # [ 108.881372] nix-grpc-daemon[1481]: E0000 00:00:1789983379.500793 1481 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5350worker1 # [ 108.884325] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5351worker1 # [ 108.886241] systemd[1]: Started Nix worker-protocol over gRPC.5352worker1 # [ 108.892070] nix-grpc-daemon[1481]: E0000 00:00:1789983379.516756 1523 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5353worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5354worker1 # [ 108.920047] nix-grpc-daemon[1481]: E0000 00:00:1789983379.534337 1523 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5355worker1 # [ 108.922349] nix-grpc-daemon[1481]: E0000 00:00:1789983379.534510 1523 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5356worker1 # [ 108.927823] nix-grpc-daemon[1481]: E0000 00:00:1789983379.553828 1523 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5357worker1 # [ 108.940320] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:19Z level=info event=scheduler_connected addr=lb:500515358worker2 # [ 108.988553] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:420585359worker2 # [ 108.994569] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05360lb # [ 109.868869] envoy[917]: conn peer=[2001:db8:1::1]:35996 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1428 ms=84125361lb # [ 109.873827] envoy[917]: conn peer=[2001:db8:1::3]:43698 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3774 tx=1157 ms=251945362lb: must succeed: curl -sf localhost:9901/clusters5363lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5364worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5365worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5366(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.30 seconds)5367subtest: scheduler restart mid-build: build finishes, no double build5368client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5369client # [ 110.148708] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5370client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5371worker2 # [ 110.446827] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:420665372worker2 # [ 110.461313] nix-daemon[860]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5373client # Running as unit: mid.service; invocation ID: a20370c1f3294376802b1ae933e57ab55374client # [ 110.442926] nix[1201]: don't know how to build these paths:5375client # [ 110.446775] nix[1201]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5376worker2 # [ 110.495798] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:21Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05377worker2 # [ 110.501969] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:21Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15378worker2 # [ 110.543483] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420665379worker2 # [ 110.563511] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 1/5)5380worker2 # [ 110.622154] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 43 ms (attempt 2/5)5381worker2 # [ 110.672670] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 289 ms (attempt 3/5)5382worker2 # [ 110.969028] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 296 ms (attempt 4/5)5383worker2 # [ 111.273756] nix-grpc-daemon[1204]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5384worker2 # [ 111.280350] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5385worker2 # [ 111.299104] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420665386client # [ 111.275858] nix[1201]: copying 1 paths to 'lb:50051'...5387client # [ 111.280190] nix[1201]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5388worker2 # [ 111.317584] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420665389worker2 # [ 111.332637] nix-daemon[860]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5390worker2 # [ 111.361482] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:21Z level=info event=niks3_push_started pid=13235391worker2 # [ 111.588640] nix-grpc-daemon[1323]: time=2026-09-21T09:36:22.197Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5392worker2 # [ 111.760608] nix-daemon[860]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5393worker2 # [ 111.777560] nix-daemon[860]: reaped child process 1343, status = succeeded5394worker2 # [ 111.949950] nix-daemon[860]: accepted connection from pid 1345, user nix-grpc-daemon (trusted)5395worker2 # [ 111.966079] nix-daemon[860]: reaped child process 1348, status = succeeded5396lb # [ 111.934910] niks3-server[974]: 2026/09/21 09:36:22 INFO Received uploads request method=POST path=/api/pending_closures5397worker2 # [ 111.989941] nix-grpc-daemon[1323]: time=2026-09-21T09:36:22.598Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5398worker2 # [ 111.991390] nix-grpc-daemon[1323]: time=2026-09-21T09:36:22.598Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5399lb # [ 111.971189] niks3-server[974]: 2026/09/21 09:36:22 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5400worker2 # [ 112.034606] nix-grpc-daemon[1323]: time=2026-09-21T09:36:22.644Z level=INFO msg="Uploading 1 narinfos"5401lb # [ 111.984858] niks3-server[974]: 2026/09/21 09:36:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5402lb # [ 111.997557] niks3-server[974]: 2026/09/21 09:36:22 INFO Signed narinfos id=35 count=15403lb # [ 112.003699] niks3-server[974]: 2026/09/21 09:36:22 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5404lb # [ 112.017493] niks3-server[974]: 2026/09/21 09:36:22 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5405worker2 # [ 112.065986] nix-grpc-daemon[1323]: time=2026-09-21T09:36:22.675Z level=INFO msg="Upload complete. (473ms)"5406lb # [ 112.022782] niks3-server[974]: 2026/09/21 09:36:22 INFO Completed upload id=355407lb # [ 112.026624] niks3-server[974]: 2026/09/21 09:36:22 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5408worker2 # [ 112.074160] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42066 duration_s=0 paths=1 nar_bytes_in=4245409worker2 # [ 112.090136] nix-grpc-daemon[1204]: ts=2026-09-21T09:36:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420665410worker2 # [ 112.100177] nix-daemon[860]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5411worker2 # [ 112.102361] nix-daemon[860]: reaped child process 1321, status = succeeded5412worker2 # [ 112.113296] nix-daemon[860]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5413client # [ 112.076571] nix[1201]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5414lb: must succeed: curl -sf localhost:9901/clusters5415lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5416lb: must succeed: curl -sf localhost:9901/clusters5417lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5418worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5419worker2 # [ 114.192357] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1204 (nix-grpc-daemon) on client request.5420worker2 # [ 114.193968] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5421lb # [ 114.164582] niks3-server[974]: 2026/09/21 09:36:24 INFO lead: released remote=192.168.1.4:408385422worker2 # [ 114.220353] systemd[1]: Starting scheduler-token.service...5423client # [ 114.180681] nix[1201]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5424worker1 # [ 114.170232] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:24Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5425client # [ 114.181120] nix[1201]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5426worker2 # [ 114.238616] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5427worker2 # [ 114.243425] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5428worker2 # [ 114.246376] systemd[1]: nix-grpc-daemon.service: Consumed 568ms CPU time over 18.012s wall clock time, 34.4M memory peak, 15.8K incoming IP traffic, 27.8K outgoing IP traffic.5429worker2 # [ 114.259259] nix-daemon[860]: reaped child process 1313, status = succeeded5430worker2 # [ 114.261541] nix-daemon[860]: reaped child process 1350, status = succeeded5431worker2 # [ 114.268577] nix-daemon[1352]: unexpected Nix daemon error: error: interrupted by the user5432worker2 # [ 114.275585] nix-daemon[860]: reaped child process 1352, status = failed with exit code 15433worker2 # [ 114.321805] systemd[1]: scheduler-token.service: Deactivated successfully.5434worker2 # [ 114.324042] systemd[1]: Finished scheduler-token.service.5435worker2 # [ 114.324818] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 103ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5436worker2 # [ 114.328158] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5437worker2 # [ 114.333199] systemd[1]: Starting scheduler-token.service...5438worker2 # [ 114.401801] systemd[1]: scheduler-token.service: Deactivated successfully.5439worker2 # [ 114.404302] systemd[1]: Finished scheduler-token.service.5440worker2 # [ 114.405084] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 70ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5441worker2 # [ 114.410346] systemd[1]: Starting Nix worker-protocol over gRPC...5442worker1 # [ 114.370550] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:24Z level=info event=scheduler_connected addr=lb:500515443worker2 # [ 114.575277] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5444worker2 # [ 114.577813] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5445worker2 # [ 114.581754] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15446worker2 # [ 114.589016] nix-grpc-daemon[1395]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5447worker2 # [ 114.590485] nix-grpc-daemon[1395]: E0000 00:00:1789983385.198236 1395 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5448lb # [ 114.564144] niks3-server[974]: 2026/09/21 09:36:25 INFO lead: acquired remote=192.168.1.4:329205449worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.48 seconds)5450worker2 # [ 114.607962] nix-grpc-daemon[1395]: E0000 00:00:1789983385.204036 1395 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5451client: must succeed: sleep 5.05452worker2 # [ 114.632261] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5453worker2 # [ 114.633861] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=scheduler_take_over5454worker2 # [ 114.634959] systemd[1]: Started Nix worker-protocol over gRPC.5455worker2 # [ 114.635729] nix-grpc-daemon[1395]: E0000 00:00:1789983385.221819 1434 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5456worker2 # [ 114.652067] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:481545457worker2 # [ 114.654016] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05458worker2 # [ 114.655883] nix-grpc-daemon[1395]: E0000 00:00:1789983385.240616 1434 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5459worker2 # [ 114.660859] nix-grpc-daemon[1395]: E0000 00:00:1789983385.240745 1434 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5460worker2 # [ 114.665565] nix-grpc-daemon[1395]: E0000 00:00:1789983385.261449 1434 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5461worker2 # [ 114.668676] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=scheduler_connected addr=lb:500515462worker2 # [ 114.672057] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=oidc_jwks provider=mock keys=15463worker2 # [ 114.673787] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:481545464worker2 # [ 114.675495] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05465worker2 # [ 114.722311] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05466worker2 # [ 114.725587] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:25Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15467worker1 # [ 114.735361] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428125468worker1 # [ 114.749810] nix-daemon[839]: accepted connection from pid 1481, user nix-grpc-daemon (trusted)5469worker1 # [ 114.775820] nix-grpc-daemon[1481]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)5470worker1 # [ 114.863842] nix-grpc-daemon[1481]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 179 ms (attempt 2/5)5471worker1 # [ 115.050564] nix-grpc-daemon[1481]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 181 ms (attempt 3/5)5472worker1 # [ 115.232902] nix-grpc-daemon[1481]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 378 ms (attempt 4/5)5473worker1 # [ 115.610795] nix-grpc-daemon[1481]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5474worker1 # [ 115.616314] nix-grpc-daemon[1481]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5475worker1 # [ 115.632975] nix-grpc-daemon[1481]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5476worker1 # [ 115.663219] nix-daemon[839]: accepted connection from pid 1481, user nix-grpc-daemon (trusted)5477worker1 # [ 115.681237] nix-daemon[839]: accepted connection from pid 1481, user nix-grpc-daemon (trusted)5478client # [ 115.690684] nix[1201]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5479client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5480lb # [ 119.866562] envoy[917]: conn peer=[2001:db8:1::4]:39626 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2679 tx=399 ms=177315481worker1 # [ 119.904671] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:30Z level=info event=niks3_push_started pid=15875482worker1 # [ 119.920861] nix-daemon[839]: reaped child process 1562, status = succeeded5483client: waiting for success: ! systemctl is-active mid5484worker1 # [ 120.102751] nix-grpc-daemon[1587]: time=2026-09-21T09:36:30.728Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5485worker1 # [ 120.274828] nix-daemon[839]: accepted connection from pid 1600, user nix-grpc-daemon (trusted)5486worker1 # [ 120.291198] nix-daemon[839]: reaped child process 1603, status = succeeded5487lb # [ 120.307433] niks3-server[974]: 2026/09/21 09:36:30 INFO Received uploads request method=POST path=/api/pending_closures5488worker1 # [ 120.314913] nix-grpc-daemon[1587]: time=2026-09-21T09:36:30.940Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5489worker1 # [ 120.317658] nix-grpc-daemon[1587]: time=2026-09-21T09:36:30.940Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5490lb # [ 120.348481] niks3-server[974]: 2026/09/21 09:36:30 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5491lb # [ 120.371152] niks3-server[974]: 2026/09/21 09:36:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5492lb # [ 120.378950] niks3-server[974]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5493lb # [ 120.404205] niks3-server[974]: 2026/09/21 09:36:31 INFO Signed narinfos id=36 count=15494worker1 # [ 120.402989] nix-grpc-daemon[1587]: time=2026-09-21T09:36:31.028Z level=INFO msg="Uploading 1 narinfos"5495lb # [ 120.421806] niks3-server[974]: 2026/09/21 09:36:31 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5496lb # [ 120.430358] niks3-server[974]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5497lb # [ 120.440211] niks3-server[974]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5498lb # [ 120.446926] niks3-server[974]: 2026/09/21 09:36:31 INFO Completed upload id=365499worker1 # [ 120.444463] nix-grpc-daemon[1587]: time=2026-09-21T09:36:31.071Z level=INFO msg="Upload complete. (337ms)"5500worker1 # [ 120.456324] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42812 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15501worker1 # [ 120.475411] nix-daemon[839]: reaped child process 1560, status = succeeded5502client # [ 120.509788] systemd[1]: mid.service: Deactivated successfully.5503client # [ 120.514047] systemd[1]: mid.service: Consumed 188ms CPU time over 10.364s wall clock time, 41.2M memory peak, 8.6K incoming IP traffic, 10.4K outgoing IP traffic.5504client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5505client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5506client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5507(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.07 seconds)5508subtest: interrupting the client stops the build on the worker5509client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5510client # [ 121.189112] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5511client # Running as unit: intr.service; invocation ID: b2f1f517321140a29024b2a6b63385515512client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5513worker1 # [ 121.418864] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428125514client # [ 121.436530] nix[1237]: don't know how to build these paths:5515client # [ 121.438134] nix[1237]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5516worker2 # [ 121.489092] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:32Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05517worker2 # [ 121.489423] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:32Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15518worker2 # [ 121.546090] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533825519worker2 # [ 121.560401] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5520worker2 # [ 121.584198] nix-grpc-daemon[1395]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5)5521worker2 # [ 121.621828] nix-grpc-daemon[1395]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 2/5)5522worker2 # [ 121.820371] nix-grpc-daemon[1395]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 3/5)5523worker2 # [ 121.859441] nix-grpc-daemon[1395]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 714 ms (attempt 4/5)5524worker2 # [ 122.582094] nix-grpc-daemon[1395]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5525worker2 # [ 122.587430] nix-grpc-daemon[1395]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5526worker2 # [ 122.607563] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533825527client # [ 122.585237] nix[1237]: copying 1 paths to 'lb:50051'...5528worker2 # [ 122.624420] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533825529client # [ 122.590390] nix[1237]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5530worker2 # [ 122.641393] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5531worker2 # [ 122.668635] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:33Z level=info event=niks3_push_started pid=14755532worker2 # [ 122.883710] nix-grpc-daemon[1475]: time=2026-09-21T09:36:33.493Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5533worker2 # [ 123.056716] nix-daemon[860]: accepted connection from pid 1488, user nix-grpc-daemon (trusted)5534worker2 # [ 123.072696] nix-daemon[860]: reaped child process 1491, status = succeeded5535worker2 # [ 123.248761] nix-daemon[860]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)5536worker2 # [ 123.264717] nix-daemon[860]: reaped child process 1496, status = succeeded5537lb # [ 123.234163] niks3-server[974]: 2026/09/21 09:36:33 INFO Received uploads request method=POST path=/api/pending_closures5538worker2 # [ 123.290763] nix-grpc-daemon[1475]: time=2026-09-21T09:36:33.899Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5539worker2 # [ 123.292503] nix-grpc-daemon[1475]: time=2026-09-21T09:36:33.899Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5540lb # [ 123.275815] niks3-server[974]: 2026/09/21 09:36:33 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5541worker2 # [ 123.331493] nix-grpc-daemon[1475]: time=2026-09-21T09:36:33.941Z level=INFO msg="Uploading 1 narinfos"5542lb # [ 123.287448] niks3-server[974]: 2026/09/21 09:36:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5543lb # [ 123.298409] niks3-server[974]: 2026/09/21 09:36:33 INFO Signed narinfos id=37 count=15544lb # [ 123.308957] niks3-server[974]: 2026/09/21 09:36:33 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5545lb # [ 123.317620] niks3-server[974]: 2026/09/21 09:36:33 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5546lb # [ 123.323533] niks3-server[974]: 2026/09/21 09:36:33 INFO Completed upload id=375547worker2 # [ 123.366143] nix-grpc-daemon[1475]: time=2026-09-21T09:36:33.976Z level=INFO msg="Upload complete. (477ms)"5548worker2 # [ 123.375512] nix-grpc-daemon[1475]: time=2026-09-21T09:36:33.979Z level=WARN msg="Failed to register uploaded object" key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo error="server returned 404: object is not pending upload\n"5549worker2 # [ 123.390340] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=0 paths=1 nar_bytes_in=4245550worker2 # [ 123.395827] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533825551worker2 # [ 123.403075] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5552worker2 # [ 123.405201] nix-daemon[860]: reaped child process 1473, status = succeeded5553worker2 # [ 123.414424] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5554client # [ 123.376525] nix[1237]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5555client: must succeed: systemctl kill -s INT intr5556worker2 # [ 124.938709] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:35Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5557client # [ 124.900900] systemd[1]: intr.service: Sent signal SIGINT to main process 1237 (nix) on client request.5558client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5559client # [ 124.909121] nix[1237]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5560client # [ 124.913686] nix[1237]: error: interrupted by the user5561client # [ 124.962505] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5562client # [ 124.968399] systemd[1]: intr.service: Failed with result 'exit-code'.5563client # [ 124.971045] systemd[1]: intr.service: Consumed 163ms CPU time over 3.777s wall clock time, 41.2M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.5564worker2 # [ 125.045332] nix-daemon[1500]: unexpected Nix daemon error: error: interrupted by the user5565worker2 # [ 125.057264] nix-daemon[860]: reaped child process 1498, status = succeeded5566worker2 # [ 125.061468] nix-daemon[860]: reaped child process 1500, status = failed with exit code 15567(finished: subtest: interrupting the client stops the build on the worker, in 3.91 seconds)5568subtest: a deploy mid-build drains: the build finishes, the new generation takes over5569client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5570client # [ 125.119080] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5571client # Running as unit: sw.service; invocation ID: 879e809df1104ff09d718d94fc40df795572client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.08 seconds)5573worker1 # [ 125.356993] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428125574client # [ 125.375095] nix[1257]: don't know how to build these paths:5575client # [ 125.376641] nix[1257]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5576worker2 # [ 125.421168] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:36Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05577worker2 # [ 125.426958] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:36Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15578worker1 # [ 125.421310] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428125579worker1 # [ 125.436448] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428125580client # [ 125.460146] nix[1257]: copying 1 paths to 'lb:50051'...5581worker1 # [ 125.450947] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428125582client # [ 125.462389] nix[1257]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5583worker1 # [ 125.467542] nix-daemon[839]: accepted connection from pid 1481, user nix-grpc-daemon (trusted)5584worker1 # [ 125.673010] nix-daemon[839]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)5585worker1 # [ 125.692229] nix-daemon[839]: reaped child process 1634, status = succeeded5586worker1 # [ 125.861386] nix-daemon[839]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)5587worker1 # [ 125.878717] nix-daemon[839]: reaped child process 1639, status = succeeded5588lb # [ 125.893991] niks3-server[974]: 2026/09/21 09:36:36 INFO Received uploads request method=POST path=/api/pending_closures5589worker1 # [ 125.902214] nix-grpc-daemon[1587]: time=2026-09-21T09:36:36.528Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5590worker1 # [ 125.903688] nix-grpc-daemon[1587]: time=2026-09-21T09:36:36.528Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5591lb # [ 125.922436] niks3-server[974]: 2026/09/21 09:36:36 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5592lb # [ 125.938336] niks3-server[974]: 2026/09/21 09:36:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5593worker1 # [ 125.941462] nix-grpc-daemon[1587]: time=2026-09-21T09:36:36.568Z level=INFO msg="Uploading 1 narinfos"5594lb # [ 125.949806] niks3-server[974]: 2026/09/21 09:36:36 INFO Signed narinfos id=38 count=15595lb # [ 125.955140] niks3-server[974]: 2026/09/21 09:36:36 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5596lb # [ 125.963380] niks3-server[974]: 2026/09/21 09:36:36 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5597lb # [ 125.970249] niks3-server[974]: 2026/09/21 09:36:36 INFO Completed upload id=385598worker1 # [ 125.966976] nix-grpc-daemon[1587]: time=2026-09-21T09:36:36.593Z level=INFO msg="Upload complete. (482ms)"5599lb # [ 125.972606] niks3-server[974]: 2026/09/21 09:36:36 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5600worker1 # [ 125.977106] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42812 duration_s=0 paths=1 nar_bytes_in=4165601worker1 # [ 125.986678] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428125602worker1 # [ 125.993572] nix-daemon[839]: accepted connection from pid 1481, user nix-grpc-daemon (trusted)5603worker1 # [ 125.998321] nix-daemon[839]: reaped child process 1629, status = succeeded5604client # [ 126.017317] nix[1257]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5605worker1 # [ 126.010656] nix-daemon[839]: accepted connection from pid 1481, user nix-grpc-daemon (trusted)5606worker1: must succeed: systemctl reload nix-grpc-daemon.service5607worker1 # [ 127.807541] systemd[1]: Reloading Nix worker-protocol over gRPC...5608worker1 # [ 127.853128] systemd[1]: Reloaded Nix worker-protocol over gRPC.5609worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.10 seconds)5610worker1: must succeed: pgrep -f 'read -t [9]0 x'5611worker1 # [ 127.883336] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:38Z level=info event=draining5612worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5613client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25614worker1 # [ 128.185833] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428125615client # don't know how to build these paths:5616client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5617worker1 # [ 128.192912] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428125618worker2 # [ 128.264553] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:38Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05619worker2 # [ 128.269930] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:38Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15620worker2 # [ 128.308091] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533825621worker2 # [ 128.323932] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533825622client # copying 1 paths to 'lb:50051'...5623worker2 # [ 128.340108] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533825624client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5625worker2 # [ 128.359588] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5626worker2 # [ 128.571733] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)5627worker2 # [ 128.589298] nix-daemon[860]: reaped child process 1554, status = succeeded5628worker2 # [ 128.774602] nix-daemon[860]: accepted connection from pid 1556, user nix-grpc-daemon (trusted)5629worker2 # [ 128.792637] nix-daemon[860]: reaped child process 1559, status = succeeded5630lb # [ 128.763119] niks3-server[974]: 2026/09/21 09:36:39 INFO Received uploads request method=POST path=/api/pending_closures5631worker2 # [ 128.816324] nix-grpc-daemon[1475]: time=2026-09-21T09:36:39.425Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5632worker2 # [ 128.817887] nix-grpc-daemon[1475]: time=2026-09-21T09:36:39.425Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5633lb # [ 128.789346] niks3-server[974]: 2026/09/21 09:36:39 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5634lb # [ 128.804385] niks3-server[974]: 2026/09/21 09:36:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5635worker2 # [ 128.855338] nix-grpc-daemon[1475]: time=2026-09-21T09:36:39.465Z level=INFO msg="Uploading 1 narinfos"5636lb # [ 128.817535] niks3-server[974]: 2026/09/21 09:36:39 INFO Signed narinfos id=39 count=15637lb # [ 128.823865] niks3-server[974]: 2026/09/21 09:36:39 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5638worker2 # [ 128.874918] nix-grpc-daemon[1475]: time=2026-09-21T09:36:39.484Z level=INFO msg="Upload complete. (498ms)"5639lb # [ 128.832957] niks3-server[974]: 2026/09/21 09:36:39 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5640lb # [ 128.838311] niks3-server[974]: 2026/09/21 09:36:39 INFO Completed upload id=395641worker2 # [ 128.881821] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=0 paths=1 nar_bytes_in=5285642lb # [ 128.841749] niks3-server[974]: 2026/09/21 09:36:39 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5643worker2 # [ 128.895182] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533825644worker2 # [ 128.901527] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5645worker2 # [ 128.906640] nix-daemon[860]: reaped child process 1549, status = succeeded5646client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5647client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5648worker2 # [ 128.919677] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5649client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5650lb # [ 129.866915] envoy[917]: conn peer=[2001:db8:1::1]:43432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3927 tx=1877 ms=100855651lb # [ 129.872232] envoy[917]: conn peer=[2001:db8:1::1]:43470 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=853 ms=35065652client # farm-a> LOG-farm-a-during-drain5653worker2 # [ 130.161271] nix-daemon[860]: reaped child process 1563, status = succeeded5654worker2 # [ 130.343571] nix-daemon[860]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5655worker2 # [ 130.358934] nix-daemon[860]: reaped child process 1582, status = succeeded5656lb # [ 130.331878] niks3-server[974]: 2026/09/21 09:36:40 INFO Received uploads request method=POST path=/api/pending_closures5657worker2 # [ 130.388417] nix-grpc-daemon[1475]: time=2026-09-21T09:36:40.997Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5658worker2 # [ 130.390165] nix-grpc-daemon[1475]: time=2026-09-21T09:36:40.997Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5659lb # [ 130.377349] niks3-server[974]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5660lb # [ 130.389679] niks3-server[974]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5661lb # [ 130.400405] niks3-server[974]: 2026/09/21 09:36:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5662worker2 # [ 130.451203] nix-grpc-daemon[1475]: time=2026-09-21T09:36:41.059Z level=INFO msg="Uploading 1 narinfos"5663lb # [ 130.409865] niks3-server[974]: 2026/09/21 09:36:41 INFO Signed narinfos id=40 count=15664lb # [ 130.416294] niks3-server[974]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5665lb # [ 130.425492] niks3-server[974]: 2026/09/21 09:36:41 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5666worker2 # [ 130.474004] nix-grpc-daemon[1475]: time=2026-09-21T09:36:41.084Z level=INFO msg="Upload complete. (343ms)"5667lb # [ 130.432740] niks3-server[974]: 2026/09/21 09:36:41 INFO Completed upload id=405668lb # [ 130.436134] niks3-server[974]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5669worker2 # [ 130.490316] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15670worker2 # [ 130.510812] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:41Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05671worker2 # [ 130.517810] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:41Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15672worker2 # [ 130.523364] nix-daemon[860]: reaped child process 1561, status = succeeded5673worker2 # [ 130.561607] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533825674worker2 # [ 130.576135] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533825675worker2 # [ 130.599165] nix-grpc-daemon[1475]: time=2026-09-21T09:36:41.209Z level=INFO msg="All 1 paths already cached"5676client # copying 1 paths to 'lb:50051'...5677worker2 # [ 130.608138] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533825678client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5679worker2 # [ 130.632281] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5680worker2 # [ 130.835289] nix-daemon[860]: accepted connection from pid 1588, user nix-grpc-daemon (trusted)5681worker2 # [ 130.854334] nix-daemon[860]: reaped child process 1591, status = succeeded5682worker2 # [ 131.024077] nix-daemon[860]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)5683worker2 # [ 131.041959] nix-daemon[860]: reaped child process 1596, status = succeeded5684lb # [ 131.011244] niks3-server[974]: 2026/09/21 09:36:41 INFO Received uploads request method=POST path=/api/pending_closures5685worker2 # [ 131.063471] nix-grpc-daemon[1475]: time=2026-09-21T09:36:41.673Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5686worker2 # [ 131.065090] nix-grpc-daemon[1475]: time=2026-09-21T09:36:41.673Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5687lb # [ 131.042842] niks3-server[974]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5688worker2 # [ 131.093990] nix-grpc-daemon[1475]: time=2026-09-21T09:36:41.703Z level=INFO msg="Uploading 1 narinfos"5689lb # [ 131.056652] niks3-server[974]: 2026/09/21 09:36:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5690lb # [ 131.066638] niks3-server[974]: 2026/09/21 09:36:41 INFO Signed narinfos id=41 count=15691lb # [ 131.071302] niks3-server[974]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5692worker2 # [ 131.125583] nix-grpc-daemon[1475]: time=2026-09-21T09:36:41.735Z level=INFO msg="Upload complete. (476ms)"5693lb # [ 131.082454] niks3-server[974]: 2026/09/21 09:36:41 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5694lb # [ 131.088776] niks3-server[974]: 2026/09/21 09:36:41 INFO Completed upload id=415695worker2 # [ 131.132051] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=0 paths=1 nar_bytes_in=6405696lb # [ 131.092646] niks3-server[974]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5697worker2 # [ 131.143417] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533825698worker2 # [ 131.149465] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5699worker2 # [ 131.154065] nix-daemon[860]: reaped child process 1586, status = succeeded5700client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5701worker2 # [ 131.166038] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5702client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5703client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5704client # farm-b> LOG-farm-b-during-drain5705worker2 # [ 132.212829] nix-daemon[860]: reaped child process 1600, status = succeeded5706worker2 # [ 132.388110] nix-daemon[860]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)5707worker2 # [ 132.405031] nix-daemon[860]: reaped child process 1619, status = succeeded5708lb # [ 132.373832] niks3-server[974]: 2026/09/21 09:36:43 INFO Received uploads request method=POST path=/api/pending_closures5709worker2 # [ 132.427108] nix-grpc-daemon[1475]: time=2026-09-21T09:36:43.035Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5710worker2 # [ 132.429754] nix-grpc-daemon[1475]: time=2026-09-21T09:36:43.035Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5711lb # [ 132.407879] niks3-server[974]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5712lb # [ 132.419919] niks3-server[974]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5713lb # [ 132.431267] niks3-server[974]: 2026/09/21 09:36:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5714worker2 # [ 132.479756] nix-grpc-daemon[1475]: time=2026-09-21T09:36:43.089Z level=INFO msg="Uploading 1 narinfos"5715lb # [ 132.440071] niks3-server[974]: 2026/09/21 09:36:43 INFO Signed narinfos id=42 count=15716lb # [ 132.445858] niks3-server[974]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5717lb # [ 132.452109] niks3-server[974]: 2026/09/21 09:36:43 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5718lb # [ 132.461193] niks3-server[974]: 2026/09/21 09:36:43 INFO Completed upload id=425719worker2 # [ 132.504747] nix-grpc-daemon[1475]: time=2026-09-21T09:36:43.114Z level=INFO msg="Upload complete. (329ms)"5720lb # [ 132.464296] niks3-server[974]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5721worker2 # [ 132.522265] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15722worker2 # [ 132.539262] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:43Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05723worker2 # [ 132.543696] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:43Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15724worker2 # [ 132.549022] nix-daemon[860]: reaped child process 1598, status = succeeded5725worker2 # [ 132.567872] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533825726worker2 # [ 132.583044] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533825727worker2 # [ 132.596637] nix-grpc-daemon[1475]: time=2026-09-21T09:36:43.206Z level=INFO msg="All 2 paths already cached"5728client # copying 1 paths to 'lb:50051'...5729worker2 # [ 132.606603] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533825730client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5731worker2 # [ 132.629439] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5732worker2 # [ 132.838042] nix-daemon[860]: accepted connection from pid 1625, user nix-grpc-daemon (trusted)5733worker2 # [ 132.854225] nix-daemon[860]: reaped child process 1628, status = succeeded5734worker2 # [ 133.027232] nix-daemon[860]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)5735worker2 # [ 133.043345] nix-daemon[860]: reaped child process 1633, status = succeeded5736lb # [ 133.011944] niks3-server[974]: 2026/09/21 09:36:43 INFO Received uploads request method=POST path=/api/pending_closures5737worker2 # [ 133.064055] nix-grpc-daemon[1475]: time=2026-09-21T09:36:43.674Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5738worker2 # [ 133.065973] nix-grpc-daemon[1475]: time=2026-09-21T09:36:43.675Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5739lb # [ 133.038247] niks3-server[974]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5740lb # [ 133.053066] niks3-server[974]: 2026/09/21 09:36:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5741worker2 # [ 133.102443] nix-grpc-daemon[1475]: time=2026-09-21T09:36:43.712Z level=INFO msg="Uploading 1 narinfos"5742lb # [ 133.068410] niks3-server[974]: 2026/09/21 09:36:43 INFO Signed narinfos id=43 count=15743lb # [ 133.075129] niks3-server[974]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5744lb # [ 133.085004] niks3-server[974]: 2026/09/21 09:36:43 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5745lb # [ 133.089951] niks3-server[974]: 2026/09/21 09:36:43 INFO Completed upload id=435746worker2 # [ 133.133556] nix-grpc-daemon[1475]: time=2026-09-21T09:36:43.743Z level=INFO msg="Upload complete. (487ms)"5747lb # [ 133.094533] niks3-server[974]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5748worker2 # [ 133.140394] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=0 paths=1 nar_bytes_in=7685749worker2 # [ 133.152657] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533825750worker2 # [ 133.161827] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5751worker2 # [ 133.164556] nix-daemon[860]: reaped child process 1623, status = succeeded5752client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5753client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5754worker2 # [ 133.177995] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5755client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5756client # farm-top> LOG-farm-top-during-drain5757worker2 # [ 134.226558] nix-daemon[860]: reaped child process 1637, status = succeeded5758worker2 # [ 134.404477] nix-daemon[860]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)5759worker2 # [ 134.420197] nix-daemon[860]: reaped child process 1656, status = succeeded5760lb # [ 134.394206] niks3-server[974]: 2026/09/21 09:36:45 INFO Received uploads request method=POST path=/api/pending_closures5761worker2 # [ 134.446409] nix-grpc-daemon[1475]: time=2026-09-21T09:36:45.054Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5762worker2 # [ 134.447863] nix-grpc-daemon[1475]: time=2026-09-21T09:36:45.054Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5763lb # [ 134.435837] niks3-server[974]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5764worker2 # [ 134.490942] nix-grpc-daemon[1475]: time=2026-09-21T09:36:45.100Z level=INFO msg="Uploading 1 narinfos"5765lb # [ 134.452403] niks3-server[974]: 2026/09/21 09:36:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5766lb # [ 134.455917] niks3-server[974]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5767lb # [ 134.463118] niks3-server[974]: 2026/09/21 09:36:45 INFO Signed narinfos id=44 count=15768lb # [ 134.469934] niks3-server[974]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5769worker2 # [ 134.513569] nix-grpc-daemon[1475]: time=2026-09-21T09:36:45.123Z level=INFO msg="Upload complete. (322ms)"5770lb # [ 134.473238] niks3-server[974]: 2026/09/21 09:36:45 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5771lb # [ 134.477968] niks3-server[974]: 2026/09/21 09:36:45 INFO Completed upload id=445772lb # [ 134.480606] niks3-server[974]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5773worker2 # [ 134.525366] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15774worker2 # [ 134.539823] nix-daemon[860]: reaped child process 1635, status = succeeded5775client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.62 seconds)5776client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5777client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5778worker1: must succeed: pkill -f 'read -t [9]0 x'5779worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5780client: waiting for success: ! systemctl is-active sw5781worker1 # [ 135.057541] nix-daemon[839]: reaped child process 1643, status = succeeded5782worker1 # [ 135.242222] nix-daemon[839]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)5783worker1 # [ 135.261233] nix-daemon[839]: reaped child process 1688, status = succeeded5784lb # [ 135.275623] niks3-server[974]: 2026/09/21 09:36:45 INFO Received uploads request method=POST path=/api/pending_closures5785worker1 # [ 135.286607] nix-grpc-daemon[1587]: time=2026-09-21T09:36:45.912Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5786worker1 # [ 135.288207] nix-grpc-daemon[1587]: time=2026-09-21T09:36:45.912Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5787lb # [ 135.310148] niks3-server[974]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5788lb # [ 135.324191] niks3-server[974]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5789lb # [ 135.340734] niks3-server[974]: 2026/09/21 09:36:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5790worker1 # [ 135.344166] nix-grpc-daemon[1587]: time=2026-09-21T09:36:45.969Z level=INFO msg="Uploading 1 narinfos"5791lb # [ 135.347759] niks3-server[974]: 2026/09/21 09:36:45 INFO Signed narinfos id=45 count=15792lb # [ 135.353807] niks3-server[974]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5793lb # [ 135.366887] niks3-server[974]: 2026/09/21 09:36:46 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5794lb # [ 135.378826] niks3-server[974]: 2026/09/21 09:36:46 INFO Completed upload id=455795worker1 # [ 135.376210] nix-grpc-daemon[1587]: time=2026-09-21T09:36:46.003Z level=INFO msg="Upload complete. (348ms)"5796lb # [ 135.382026] niks3-server[974]: 2026/09/21 09:36:46 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5797worker1 # [ 135.389092] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42812 duration_s=9 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15798worker1 # [ 135.408238] nix-daemon[839]: reaped child process 1641, status = succeeded5799client # [ 135.441425] systemd[1]: sw.service: Deactivated successfully.5800client # [ 135.445829] systemd[1]: sw.service: Consumed 174ms CPU time over 10.325s wall clock time, 41.2M memory peak, 7K incoming IP traffic, 8.4K outgoing IP traffic.5801worker1 # [ 135.885176] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:46Z level=info event=signal_exit5802client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds)5803client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5804client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)5805worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5806worker2 # [ 137.166043] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:481545807worker1 # [ 137.120914] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:47Z level=info event=scheduler_restarting5808worker2 # [ 137.170604] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:47Z level=info event=worker_gone addr=192.168.1.3:500515809worker1 # [ 137.144792] nix-grpc-daemon[1481]: ts=2026-09-21T09:36:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5810worker1 # [ 137.168939] nix-daemon[839]: reaped child process 1553, status = succeeded5811worker1 # [ 137.675616] nix-grpc-daemon[1481]: E0000 00:00:1789983408.302275 1526 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5812worker1 # [ 137.696896] nix-grpc-daemon[1481]: E0000 00:00:1789983408.323699 1526 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5813worker1 # [ 138.937090] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5814worker1 # [ 138.942882] systemd[1]: nix-grpc-daemon.service: Consumed 869ms CPU time over 30.271s wall clock time, 33.6M memory peak, 20.8K incoming IP traffic, 27.3K outgoing IP traffic.5815worker1 # [ 138.962382] systemd[1]: Starting Nix worker-protocol over gRPC...5816worker1 # [ 139.162054] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5817worker1 # [ 139.164621] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5818worker1 # [ 139.168676] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15819worker1 # [ 139.176067] nix-grpc-daemon[1702]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5820worker1 # [ 139.177527] nix-grpc-daemon[1702]: E0000 00:00:1789983409.801875 1702 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5821worker1 # [ 139.187483] nix-grpc-daemon[1702]: E0000 00:00:1789983409.808107 1702 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5822worker1 # [ 139.190548] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5823worker1 # [ 139.193741] systemd[1]: Started Nix worker-protocol over gRPC.5824worker1 # [ 139.200065] nix-grpc-daemon[1702]: E0000 00:00:1789983409.823997 1741 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5825worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.23 seconds)5826lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5827lb: must succeed: curl -sf localhost:9901/clusters5828worker1 # [ 139.228094] nix-grpc-daemon[1702]: E0000 00:00:1789983409.841939 1741 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5829worker1 # [ 139.233738] nix-grpc-daemon[1702]: E0000 00:00:1789983409.842088 1741 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5830worker1 # [ 139.236683] nix-grpc-daemon[1702]: E0000 00:00:1789983409.859796 1741 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5831worker1 # [ 139.247799] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:49Z level=info event=scheduler_connected addr=lb:500515832worker2 # [ 139.297077] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:49Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:481545833worker2 # [ 139.297495] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:49Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05834lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5835lb # [ 139.866925] envoy[917]: conn peer=[2001:db8:1::1]:43484 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9638 tx=4081 ms=63115836lb # [ 139.872233] envoy[917]: conn peer=[2001:db8:1::1]:43474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1429 ms=100455837lb # [ 139.877283] envoy[917]: conn peer=[2001:db8:1::3]:46106 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=986 tx=448 ms=281875838lb: must succeed: curl -sf localhost:9901/clusters5839lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5840lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5841lb: must succeed: curl -sf localhost:9901/clusters5842lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5843worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5844worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5845(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.48 seconds)5846subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5847client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15848worker1 # [ 140.789167] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:519545849worker1 # [ 140.803724] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5850worker2 # [ 140.871888] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533825851worker2 # [ 140.917564] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:51Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05852worker2 # [ 140.923530] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:51Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15853worker2 # [ 140.929635] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:51Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05854worker2 # [ 140.934817] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:51Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05855worker1 # [ 140.917950] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519545856worker1 # [ 140.935147] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 1/5)5857worker1 # [ 141.036820] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 2/5)5858worker1 # [ 141.057384] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 3/5)5859worker1 # [ 141.230903] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 428 ms (attempt 4/5)5860worker1 # [ 141.659630] nix-grpc-daemon[1702]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5861worker1 # [ 141.666279] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5862worker1 # [ 141.684134] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519545863worker1 # [ 141.699242] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:519545864worker1 # [ 141.713894] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5865worker1 # [ 141.742697] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:52Z level=info event=niks3_push_started pid=17625866worker1 # [ 141.959219] nix-grpc-daemon[1762]: time=2026-09-21T09:36:52.585Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5867worker1 # [ 142.132461] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)5868worker1 # [ 142.151216] nix-daemon[839]: reaped child process 1778, status = succeeded5869worker1 # [ 142.324224] nix-daemon[839]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)5870worker1 # [ 142.342093] nix-daemon[839]: reaped child process 1783, status = succeeded5871lb # [ 142.357263] niks3-server[974]: 2026/09/21 09:36:52 INFO Received uploads request method=POST path=/api/pending_closures5872worker1 # [ 142.366443] nix-grpc-daemon[1762]: time=2026-09-21T09:36:52.992Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5873worker1 # [ 142.367942] nix-grpc-daemon[1762]: time=2026-09-21T09:36:52.992Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5874lb # [ 142.399402] niks3-server[974]: 2026/09/21 09:36:53 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5875worker1 # [ 142.406540] nix-grpc-daemon[1762]: time=2026-09-21T09:36:53.032Z level=INFO msg="Uploading 1 narinfos"5876lb # [ 142.409295] niks3-server[974]: 2026/09/21 09:36:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5877lb # [ 142.418062] niks3-server[974]: 2026/09/21 09:36:53 INFO Signed narinfos id=46 count=15878lb # [ 142.426537] niks3-server[974]: 2026/09/21 09:36:53 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5879lb # [ 142.438956] niks3-server[974]: 2026/09/21 09:36:53 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5880worker1 # [ 142.444344] nix-grpc-daemon[1762]: time=2026-09-21T09:36:53.071Z level=INFO msg="Upload complete. (481ms)"5881lb # [ 142.447218] niks3-server[974]: 2026/09/21 09:36:53 INFO Completed upload id=465882lb # [ 142.451175] niks3-server[974]: 2026/09/21 09:36:53 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5883worker1 # [ 142.452768] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=0 paths=1 nar_bytes_in=5125884worker1 # [ 142.467477] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519545885worker1 # [ 142.476278] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5886worker1 # [ 142.479453] nix-daemon[839]: reaped child process 1760, status = succeeded5887worker1 # [ 142.490339] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5888worker1 # [ 143.732003] nix-daemon[839]: reaped child process 1787, status = succeeded5889worker1 # [ 143.913194] nix-daemon[839]: accepted connection from pid 1803, user nix-grpc-daemon (trusted)5890worker1 # [ 143.931118] nix-daemon[839]: reaped child process 1806, status = succeeded5891lb # [ 143.947804] niks3-server[974]: 2026/09/21 09:36:54 INFO Received uploads request method=POST path=/api/pending_closures5892worker1 # [ 143.957172] nix-grpc-daemon[1762]: time=2026-09-21T09:36:54.583Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5893worker1 # [ 143.962349] nix-grpc-daemon[1762]: time=2026-09-21T09:36:54.585Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5894lb # [ 143.988992] niks3-server[974]: 2026/09/21 09:36:54 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5895lb # [ 144.001845] niks3-server[974]: 2026/09/21 09:36:54 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5896worker1 # [ 144.009247] nix-grpc-daemon[1762]: time=2026-09-21T09:36:54.634Z level=INFO msg="Uploading 1 narinfos"5897lb # [ 144.015722] niks3-server[974]: 2026/09/21 09:36:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5898lb # [ 144.022740] niks3-server[974]: 2026/09/21 09:36:54 INFO Signed narinfos id=47 count=15899worker1 # [ 144.024782] nix-grpc-daemon[1762]: time=2026-09-21T09:36:54.651Z level=INFO msg="Upload complete. (321ms)"5900lb # [ 144.027729] niks3-server[974]: 2026/09/21 09:36:54 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5901lb # [ 144.032913] niks3-server[974]: 2026/09/21 09:36:54 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5902lb # [ 144.037160] niks3-server[974]: 2026/09/21 09:36:54 INFO Completed upload id=475903worker2 # [ 144.078490] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:54Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15904worker1 # [ 144.032796] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=15905lb # [ 144.039066] niks3-server[974]: 2026/09/21 09:36:54 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5906worker2 # [ 144.091337] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:54Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05907worker1 # [ 144.050846] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519545908worker1 # [ 144.055143] nix-daemon[839]: reaped child process 1785, status = succeeded5909worker1 # [ 144.061500] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519545910worker1 # [ 144.072884] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:519545911worker1 # [ 144.083307] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5912worker1 # [ 144.267935] nix-daemon[839]: accepted connection from pid 1810, user nix-grpc-daemon (trusted)5913worker1 # [ 144.285717] nix-daemon[839]: reaped child process 1813, status = succeeded5914worker1 # [ 144.455847] nix-daemon[839]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)5915worker1 # [ 144.470849] nix-daemon[839]: reaped child process 1818, status = succeeded5916lb # [ 144.489067] niks3-server[974]: 2026/09/21 09:36:55 INFO Received uploads request method=POST path=/api/pending_closures5917worker1 # [ 144.494965] nix-grpc-daemon[1762]: time=2026-09-21T09:36:55.121Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5918worker1 # [ 144.496551] nix-grpc-daemon[1762]: time=2026-09-21T09:36:55.121Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5919lb # [ 144.517264] niks3-server[974]: 2026/09/21 09:36:55 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5920worker1 # [ 144.532708] nix-grpc-daemon[1762]: time=2026-09-21T09:36:55.159Z level=INFO msg="Uploading 1 narinfos"5921lb # [ 144.528117] niks3-server[974]: 2026/09/21 09:36:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5922lb # [ 144.540722] niks3-server[974]: 2026/09/21 09:36:55 INFO Signed narinfos id=48 count=15923lb # [ 144.543472] niks3-server[974]: 2026/09/21 09:36:55 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5924lb # [ 144.554741] niks3-server[974]: 2026/09/21 09:36:55 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5925lb # [ 144.561951] niks3-server[974]: 2026/09/21 09:36:55 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5926lb # [ 144.566983] niks3-server[974]: 2026/09/21 09:36:55 INFO Completed upload id=485927worker1 # [ 144.564460] nix-grpc-daemon[1762]: time=2026-09-21T09:36:55.191Z level=INFO msg="Upload complete. (469ms)"5928worker1 # [ 144.574028] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=0 paths=1 nar_bytes_in=5125929worker1 # [ 144.589050] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519545930worker1 # [ 144.595105] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5931worker1 # [ 144.604303] nix-daemon[839]: reaped child process 1808, status = succeeded5932worker1 # [ 144.613746] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5933worker1 # [ 146.058286] nix-daemon[839]: reaped child process 1822, status = succeeded5934worker1 # [ 146.241495] nix-daemon[839]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)5935worker1 # [ 146.258765] nix-daemon[839]: reaped child process 1841, status = succeeded5936lb # [ 146.274292] niks3-server[974]: 2026/09/21 09:36:56 INFO Received uploads request method=POST path=/api/pending_closures5937worker1 # [ 146.281475] nix-grpc-daemon[1762]: time=2026-09-21T09:36:56.907Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5938worker1 # [ 146.282915] nix-grpc-daemon[1762]: time=2026-09-21T09:36:56.907Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5939lb # [ 146.304290] niks3-server[974]: 2026/09/21 09:36:56 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5940lb # [ 146.317892] niks3-server[974]: 2026/09/21 09:36:56 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5941lb # [ 146.331093] niks3-server[974]: 2026/09/21 09:36:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5942worker1 # [ 146.333081] nix-grpc-daemon[1762]: time=2026-09-21T09:36:56.959Z level=INFO msg="Uploading 1 narinfos"5943lb # [ 146.340551] niks3-server[974]: 2026/09/21 09:36:56 INFO Signed narinfos id=49 count=15944lb # [ 146.342777] niks3-server[974]: 2026/09/21 09:36:56 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5945lb # [ 146.350232] niks3-server[974]: 2026/09/21 09:36:56 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5946lb # [ 146.356171] niks3-server[974]: 2026/09/21 09:36:56 INFO Completed upload id=495947worker1 # [ 146.352974] nix-grpc-daemon[1762]: time=2026-09-21T09:36:56.979Z level=INFO msg="Upload complete. (328ms)"5948lb # [ 146.358331] niks3-server[974]: 2026/09/21 09:36:56 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5949worker2 # [ 146.407348] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:57Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15950worker1 # [ 146.364986] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=15951worker2 # [ 146.420884] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:57Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05952worker1 # [ 146.381226] nix-daemon[839]: reaped child process 1820, status = succeeded5953worker1 # [ 146.413670] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519545954worker1 # [ 146.428695] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519545955worker1 # [ 146.442786] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:519545956worker1 # [ 146.456154] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5957worker1 # [ 146.661706] nix-daemon[839]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)5958worker1 # [ 146.678162] nix-daemon[839]: reaped child process 1848, status = succeeded5959worker1 # [ 146.849446] nix-daemon[839]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)5960worker1 # [ 146.866010] nix-daemon[839]: reaped child process 1853, status = succeeded5961lb # [ 146.881303] niks3-server[974]: 2026/09/21 09:36:57 INFO Received uploads request method=POST path=/api/pending_closures5962worker1 # [ 146.886990] nix-grpc-daemon[1762]: time=2026-09-21T09:36:57.513Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5963worker1 # [ 146.888524] nix-grpc-daemon[1762]: time=2026-09-21T09:36:57.513Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"5964lb # [ 146.906915] niks3-server[974]: 2026/09/21 09:36:57 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst5965worker1 # [ 146.917306] nix-grpc-daemon[1762]: time=2026-09-21T09:36:57.544Z level=INFO msg="Uploading 1 narinfos"5966lb # [ 146.923208] niks3-server[974]: 2026/09/21 09:36:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5967lb # [ 146.933774] niks3-server[974]: 2026/09/21 09:36:57 INFO Signed narinfos id=50 count=15968lb # [ 146.939508] niks3-server[974]: 2026/09/21 09:36:57 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls5969worker1 # [ 146.945591] nix-grpc-daemon[1762]: time=2026-09-21T09:36:57.572Z level=INFO msg="Upload complete. (473ms)"5970lb # [ 146.948653] niks3-server[974]: 2026/09/21 09:36:57 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5971lb # [ 146.953933] niks3-server[974]: 2026/09/21 09:36:57 INFO Completed upload id=505972worker1 # [ 146.952104] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=0 paths=1 nar_bytes_in=5125973lb # [ 146.957641] niks3-server[974]: 2026/09/21 09:36:57 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo5974worker1 # [ 146.959721] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519545975worker1 # [ 146.970180] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5976worker1 # [ 146.974350] nix-daemon[839]: reaped child process 1843, status = succeeded5977worker1 # [ 146.986999] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)5978worker1 # [ 148.227661] nix-daemon[839]: reaped child process 1857, status = succeeded5979worker1 # [ 148.405724] nix-daemon[839]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)5980worker1 # [ 148.421906] nix-daemon[839]: reaped child process 1876, status = succeeded5981lb # [ 148.438476] niks3-server[974]: 2026/09/21 09:36:59 INFO Received uploads request method=POST path=/api/pending_closures5982worker1 # [ 148.447545] nix-grpc-daemon[1762]: time=2026-09-21T09:36:59.072Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5983worker1 # [ 148.449180] nix-grpc-daemon[1762]: time=2026-09-21T09:36:59.073Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"5984lb # [ 148.477824] niks3-server[974]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv5985lb # [ 148.488977] niks3-server[974]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst5986worker1 # [ 148.502442] nix-grpc-daemon[1762]: time=2026-09-21T09:36:59.129Z level=INFO msg="Uploading 1 narinfos"5987lb # [ 148.501125] niks3-server[974]: 2026/09/21 09:36:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5988lb # [ 148.510497] niks3-server[974]: 2026/09/21 09:36:59 INFO Signed narinfos id=51 count=15989lb # [ 148.514779] niks3-server[974]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls5990lb # [ 148.522455] niks3-server[974]: 2026/09/21 09:36:59 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5991lb # [ 148.526658] niks3-server[974]: 2026/09/21 09:36:59 INFO Completed upload id=515992worker1 # [ 148.523503] nix-grpc-daemon[1762]: time=2026-09-21T09:36:59.150Z level=INFO msg="Upload complete. (326ms)"5993lb # [ 148.529011] niks3-server[974]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo5994worker2 # [ 148.578571] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:59Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=15995worker1 # [ 148.537017] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=15996worker2 # [ 148.592694] nix-grpc-daemon[1395]: ts=2026-09-21T09:36:59Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=05997worker1 # [ 148.553929] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519545998worker1 # [ 148.564154] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519545999worker1 # [ 148.569820] nix-daemon[839]: reaped child process 1855, status = succeeded6000worker1 # [ 148.573465] nix-grpc-daemon[1762]: time=2026-09-21T09:36:59.200Z level=INFO msg="All 1 paths already cached"6001worker1 # [ 148.581657] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:519546002worker1 # [ 148.592507] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6003worker1 # [ 148.787886] nix-daemon[839]: accepted connection from pid 1880, user nix-grpc-daemon (trusted)6004worker1 # [ 148.804908] nix-daemon[839]: reaped child process 1883, status = succeeded6005worker1 # [ 148.977934] nix-daemon[839]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6006worker1 # [ 148.993963] nix-daemon[839]: reaped child process 1888, status = succeeded6007lb # [ 149.008931] niks3-server[974]: 2026/09/21 09:36:59 INFO Received uploads request method=POST path=/api/pending_closures6008worker1 # [ 149.016212] nix-grpc-daemon[1762]: time=2026-09-21T09:36:59.643Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6009worker1 # [ 149.018115] nix-grpc-daemon[1762]: time=2026-09-21T09:36:59.644Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6010lb # [ 149.041809] niks3-server[974]: 2026/09/21 09:36:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6011worker1 # [ 149.046532] nix-grpc-daemon[1762]: time=2026-09-21T09:36:59.673Z level=INFO msg="Uploading 1 narinfos"6012lb # [ 149.056763] niks3-server[974]: 2026/09/21 09:36:59 INFO Signed narinfos id=52 count=16013lb # [ 149.059947] niks3-server[974]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6014worker1 # [ 149.074106] nix-grpc-daemon[1762]: time=2026-09-21T09:36:59.701Z level=INFO msg="Upload complete. (468ms)"6015lb # [ 149.077682] niks3-server[974]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6016lb # [ 149.081804] niks3-server[974]: 2026/09/21 09:36:59 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6017worker1 # [ 149.080319] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=0 paths=1 nar_bytes_in=6246018lb # [ 149.086448] niks3-server[974]: 2026/09/21 09:36:59 INFO Completed upload id=526019lb # [ 149.088593] niks3-server[974]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6020worker1 # [ 149.093403] nix-grpc-daemon[1702]: ts=2026-09-21T09:36:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546021worker1 # [ 149.098971] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6022worker1 # [ 149.103226] nix-daemon[839]: reaped child process 1878, status = succeeded6023worker1 # [ 149.115230] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6024worker1 # [ 150.758465] nix-daemon[839]: reaped child process 1892, status = succeeded6025worker1 # [ 150.942854] nix-daemon[839]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6026worker1 # [ 150.959676] nix-daemon[839]: reaped child process 1911, status = succeeded6027lb # [ 150.977962] niks3-server[974]: 2026/09/21 09:37:01 INFO Received uploads request method=POST path=/api/pending_closures6028worker1 # [ 150.984398] nix-grpc-daemon[1762]: time=2026-09-21T09:37:01.610Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6029worker1 # [ 150.985895] nix-grpc-daemon[1762]: time=2026-09-21T09:37:01.610Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6030lb # [ 151.030560] niks3-server[974]: 2026/09/21 09:37:01 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6031worker1 # [ 151.036453] nix-grpc-daemon[1762]: time=2026-09-21T09:37:01.663Z level=INFO msg="Uploading 1 narinfos"6032lb # [ 151.037617] niks3-server[974]: 2026/09/21 09:37:01 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6033lb # [ 151.051409] niks3-server[974]: 2026/09/21 09:37:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6034lb # [ 151.057654] niks3-server[974]: 2026/09/21 09:37:01 INFO Signed narinfos id=53 count=16035lb # [ 151.060805] niks3-server[974]: 2026/09/21 09:37:01 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6036lb # [ 151.063715] niks3-server[974]: 2026/09/21 09:37:01 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6037lb # [ 151.066954] niks3-server[974]: 2026/09/21 09:37:01 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6038lb # [ 151.070570] niks3-server[974]: 2026/09/21 09:37:01 INFO Completed upload id=536039worker1 # [ 151.067761] nix-grpc-daemon[1762]: time=2026-09-21T09:37:01.694Z level=INFO msg="Upload complete. (342ms)"6040worker2 # [ 151.122189] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:01Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16041worker1 # [ 151.081212] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16042worker2 # [ 151.136246] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:01Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06043worker1 # [ 151.099452] nix-daemon[839]: reaped child process 1890, status = succeeded6044worker1 # [ 151.129713] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546045worker1 # [ 151.144882] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519546046worker1 # [ 151.158265] nix-grpc-daemon[1762]: time=2026-09-21T09:37:01.785Z level=INFO msg="All 1 paths already cached"6047worker1 # [ 151.167016] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:519546048worker1 # [ 151.181305] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6049worker1 # [ 151.380071] nix-daemon[839]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6050worker1 # [ 151.398404] nix-daemon[839]: reaped child process 1918, status = succeeded6051worker1 # [ 151.570114] nix-daemon[839]: accepted connection from pid 1920, user nix-grpc-daemon (trusted)6052worker1 # [ 151.587342] nix-daemon[839]: reaped child process 1923, status = succeeded6053lb # [ 151.602440] niks3-server[974]: 2026/09/21 09:37:02 INFO Received uploads request method=POST path=/api/pending_closures6054worker1 # [ 151.608879] nix-grpc-daemon[1762]: time=2026-09-21T09:37:02.235Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6055worker1 # [ 151.610357] nix-grpc-daemon[1762]: time=2026-09-21T09:37:02.235Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6056lb # [ 151.627973] niks3-server[974]: 2026/09/21 09:37:02 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6057worker1 # [ 151.638615] nix-grpc-daemon[1762]: time=2026-09-21T09:37:02.265Z level=INFO msg="Uploading 1 narinfos"6058lb # [ 151.642791] niks3-server[974]: 2026/09/21 09:37:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6059lb # [ 151.653374] niks3-server[974]: 2026/09/21 09:37:02 INFO Signed narinfos id=54 count=16060lb # [ 151.660274] niks3-server[974]: 2026/09/21 09:37:02 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6061lb # [ 151.668630] niks3-server[974]: 2026/09/21 09:37:02 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6062lb # [ 151.674711] niks3-server[974]: 2026/09/21 09:37:02 INFO Completed upload id=546063worker1 # [ 151.672037] nix-grpc-daemon[1762]: time=2026-09-21T09:37:02.298Z level=INFO msg="Upload complete. (474ms)"6064lb # [ 151.677546] niks3-server[974]: 2026/09/21 09:37:02 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6065worker1 # [ 151.680207] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=0 paths=1 nar_bytes_in=6246066worker1 # [ 151.691907] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546067worker1 # [ 151.700146] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6068worker1 # [ 151.704820] nix-daemon[839]: reaped child process 1913, status = succeeded6069worker1 # [ 151.716945] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6070worker1 # [ 153.360625] nix-daemon[839]: reaped child process 1927, status = succeeded6071worker1 # [ 153.543785] nix-daemon[839]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)6072worker1 # [ 153.562253] nix-daemon[839]: reaped child process 1946, status = succeeded6073lb # [ 153.577472] niks3-server[974]: 2026/09/21 09:37:04 INFO Received uploads request method=POST path=/api/pending_closures6074worker1 # [ 153.585986] nix-grpc-daemon[1762]: time=2026-09-21T09:37:04.210Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6075worker1 # [ 153.587792] nix-grpc-daemon[1762]: time=2026-09-21T09:37:04.211Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6076lb # [ 153.612706] niks3-server[974]: 2026/09/21 09:37:04 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6077worker1 # [ 153.623724] nix-grpc-daemon[1762]: time=2026-09-21T09:37:04.250Z level=INFO msg="Uploading 1 narinfos"6078lb # [ 153.626198] niks3-server[974]: 2026/09/21 09:37:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6079lb # [ 153.635086] niks3-server[974]: 2026/09/21 09:37:04 INFO Signed narinfos id=55 count=16080lb # [ 153.643187] niks3-server[974]: 2026/09/21 09:37:04 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6081lb # [ 153.651155] niks3-server[974]: 2026/09/21 09:37:04 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6082lb # [ 153.659966] niks3-server[974]: 2026/09/21 09:37:04 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6083lb # [ 153.665194] niks3-server[974]: 2026/09/21 09:37:04 INFO Completed upload id=556084worker1 # [ 153.662008] nix-grpc-daemon[1762]: time=2026-09-21T09:37:04.288Z level=INFO msg="Upload complete. (332ms)"6085lb # [ 153.668389] niks3-server[974]: 2026/09/21 09:37:04 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6086worker2 # [ 153.719838] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:04Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16087worker1 # [ 153.676241] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16088worker2 # [ 153.733903] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:04Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06089worker1 # [ 153.692418] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546090worker1 # [ 153.701476] nix-daemon[839]: reaped child process 1925, status = succeeded6091worker1 # [ 153.704922] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519546092worker1 # [ 153.716335] nix-grpc-daemon[1762]: time=2026-09-21T09:37:04.343Z level=INFO msg="All 1 paths already cached"6093worker1 # [ 153.724104] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:519546094worker1 # [ 153.736321] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6095worker1 # [ 153.926623] nix-daemon[839]: accepted connection from pid 1950, user nix-grpc-daemon (trusted)6096worker1 # [ 153.942890] nix-daemon[839]: reaped child process 1953, status = succeeded6097worker1 # [ 154.115906] nix-daemon[839]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)6098worker1 # [ 154.130438] nix-daemon[839]: reaped child process 1958, status = succeeded6099lb # [ 154.149887] niks3-server[974]: 2026/09/21 09:37:04 INFO Received uploads request method=POST path=/api/pending_closures6100worker1 # [ 154.156833] nix-grpc-daemon[1762]: time=2026-09-21T09:37:04.783Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6101worker1 # [ 154.158274] nix-grpc-daemon[1762]: time=2026-09-21T09:37:04.783Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6102lb # [ 154.179656] niks3-server[974]: 2026/09/21 09:37:04 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6103worker1 # [ 154.185863] nix-grpc-daemon[1762]: time=2026-09-21T09:37:04.812Z level=INFO msg="Uploading 1 narinfos"6104lb # [ 154.194709] niks3-server[974]: 2026/09/21 09:37:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6105lb # [ 154.204733] niks3-server[974]: 2026/09/21 09:37:04 INFO Signed narinfos id=56 count=16106lb # [ 154.207541] niks3-server[974]: 2026/09/21 09:37:04 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6107lb # [ 154.215147] niks3-server[974]: 2026/09/21 09:37:04 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6108lb # [ 154.221884] niks3-server[974]: 2026/09/21 09:37:04 INFO Completed upload id=566109worker1 # [ 154.219267] nix-grpc-daemon[1762]: time=2026-09-21T09:37:04.846Z level=INFO msg="Upload complete. (469ms)"6110lb # [ 154.224863] niks3-server[974]: 2026/09/21 09:37:04 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6111worker1 # [ 154.228265] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=0 paths=1 nar_bytes_in=6246112worker1 # [ 154.237818] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546113worker1 # [ 154.246456] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6114worker1 # [ 154.248786] nix-daemon[839]: reaped child process 1948, status = succeeded6115worker1 # [ 154.261793] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6116worker1 # [ 155.286277] nix-daemon[839]: reaped child process 1962, status = succeeded6117worker1 # [ 155.447839] nix-daemon[839]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)6118worker1 # [ 155.465364] nix-daemon[839]: reaped child process 1981, status = succeeded6119lb # [ 155.480997] niks3-server[974]: 2026/09/21 09:37:06 INFO Received uploads request method=POST path=/api/pending_closures6120worker1 # [ 155.488384] nix-grpc-daemon[1762]: time=2026-09-21T09:37:06.114Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6121worker1 # [ 155.489814] nix-grpc-daemon[1762]: time=2026-09-21T09:37:06.114Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6122lb # [ 155.521486] niks3-server[974]: 2026/09/21 09:37:06 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6123worker1 # [ 155.527944] nix-grpc-daemon[1762]: time=2026-09-21T09:37:06.154Z level=INFO msg="Uploading 1 narinfos"6124lb # [ 155.533440] niks3-server[974]: 2026/09/21 09:37:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6125lb # [ 155.544343] niks3-server[974]: 2026/09/21 09:37:06 INFO Signed narinfos id=57 count=16126lb # [ 155.549597] niks3-server[974]: 2026/09/21 09:37:06 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6127worker1 # [ 155.554750] nix-grpc-daemon[1762]: time=2026-09-21T09:37:06.181Z level=INFO msg="Upload complete. (284ms)"6128lb # [ 155.557952] niks3-server[974]: 2026/09/21 09:37:06 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6129lb # [ 155.563066] niks3-server[974]: 2026/09/21 09:37:06 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6130lb # [ 155.566960] niks3-server[974]: 2026/09/21 09:37:06 INFO Completed upload id=576131worker2 # [ 155.607549] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:06Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16132lb # [ 155.569029] niks3-server[974]: 2026/09/21 09:37:06 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6133worker1 # [ 155.564785] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16134worker1 # [ 155.577200] nix-daemon[839]: reaped child process 1960, status = succeeded6135worker2 # [ 155.621534] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:06Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06136worker1 # [ 155.617749] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546137worker1 # [ 155.633095] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519546138worker1 # [ 155.646607] nix-grpc-daemon[1762]: time=2026-09-21T09:37:06.273Z level=INFO msg="All 2 paths already cached"6139worker1 # [ 155.655375] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:519546140worker1 # [ 155.669171] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6141worker1 # [ 155.868400] nix-daemon[839]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6142worker1 # [ 155.886714] nix-daemon[839]: reaped child process 1988, status = succeeded6143worker1 # [ 156.059313] nix-daemon[839]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)6144worker1 # [ 156.076183] nix-daemon[839]: reaped child process 1993, status = succeeded6145lb # [ 156.092577] niks3-server[974]: 2026/09/21 09:37:06 INFO Received uploads request method=POST path=/api/pending_closures6146worker1 # [ 156.099037] nix-grpc-daemon[1762]: time=2026-09-21T09:37:06.725Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6147worker1 # [ 156.100572] nix-grpc-daemon[1762]: time=2026-09-21T09:37:06.725Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6148lb # [ 156.118703] niks3-server[974]: 2026/09/21 09:37:06 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6149worker1 # [ 156.128140] nix-grpc-daemon[1762]: time=2026-09-21T09:37:06.754Z level=INFO msg="Uploading 1 narinfos"6150lb # [ 156.134956] niks3-server[974]: 2026/09/21 09:37:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6151lb # [ 156.145263] niks3-server[974]: 2026/09/21 09:37:06 INFO Signed narinfos id=58 count=16152lb # [ 156.150013] niks3-server[974]: 2026/09/21 09:37:06 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6153lb # [ 156.157576] niks3-server[974]: 2026/09/21 09:37:06 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6154lb # [ 156.162464] niks3-server[974]: 2026/09/21 09:37:06 INFO Completed upload id=586155worker1 # [ 156.159540] nix-grpc-daemon[1762]: time=2026-09-21T09:37:06.786Z level=INFO msg="Upload complete. (474ms)"6156lb # [ 156.165275] niks3-server[974]: 2026/09/21 09:37:06 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6157worker1 # [ 156.168200] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=0 paths=1 nar_bytes_in=7526158worker1 # [ 156.176468] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546159worker1 # [ 156.184258] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6160worker1 # [ 156.187738] nix-daemon[839]: reaped child process 1983, status = succeeded6161worker1 # [ 156.199856] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6162worker1 # [ 157.233391] nix-daemon[839]: reaped child process 1997, status = succeeded6163worker1 # [ 157.393889] nix-daemon[839]: accepted connection from pid 2013, user nix-grpc-daemon (trusted)6164worker1 # [ 157.411157] nix-daemon[839]: reaped child process 2016, status = succeeded6165lb # [ 157.434056] niks3-server[974]: 2026/09/21 09:37:08 INFO Received uploads request method=POST path=/api/pending_closures6166worker1 # [ 157.444191] nix-grpc-daemon[1762]: time=2026-09-21T09:37:08.067Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6167worker1 # [ 157.445642] nix-grpc-daemon[1762]: time=2026-09-21T09:37:08.067Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6168lb # [ 157.472709] niks3-server[974]: 2026/09/21 09:37:08 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6169worker1 # [ 157.479785] nix-grpc-daemon[1762]: time=2026-09-21T09:37:08.106Z level=INFO msg="Uploading 1 narinfos"6170lb # [ 157.484725] niks3-server[974]: 2026/09/21 09:37:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6171lb # [ 157.487844] niks3-server[974]: 2026/09/21 09:37:08 INFO Signed narinfos id=59 count=16172lb # [ 157.501097] niks3-server[974]: 2026/09/21 09:37:08 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6173lb # [ 157.503801] niks3-server[974]: 2026/09/21 09:37:08 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6174lb # [ 157.508692] niks3-server[974]: 2026/09/21 09:37:08 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6175worker1 # [ 157.505916] nix-grpc-daemon[1762]: time=2026-09-21T09:37:08.132Z level=INFO msg="Upload complete. (296ms)"6176lb # [ 157.511489] niks3-server[974]: 2026/09/21 09:37:08 INFO Completed upload id=596177lb # [ 157.514274] niks3-server[974]: 2026/09/21 09:37:08 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6178worker2 # [ 157.559279] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:08Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16179worker1 # [ 157.515509] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16180worker1 # [ 157.527061] nix-daemon[839]: reaped child process 1995, status = succeeded6181worker1 # [ 157.560920] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546182worker1 # [ 157.575531] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519546183worker1 # [ 157.589364] nix-grpc-daemon[1762]: time=2026-09-21T09:37:08.216Z level=INFO msg="All 2 paths already cached"6184worker1 # [ 157.597877] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:519546185worker1 # [ 157.611723] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6186worker1 # [ 157.806549] nix-daemon[839]: accepted connection from pid 2020, user nix-grpc-daemon (trusted)6187worker1 # [ 157.825536] nix-daemon[839]: reaped child process 2023, status = succeeded6188worker1 # [ 157.997973] nix-daemon[839]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)6189worker1 # [ 158.014684] nix-daemon[839]: reaped child process 2028, status = succeeded6190lb # [ 158.034188] niks3-server[974]: 2026/09/21 09:37:08 INFO Received uploads request method=POST path=/api/pending_closures6191worker1 # [ 158.040027] nix-grpc-daemon[1762]: time=2026-09-21T09:37:08.666Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6192worker1 # [ 158.041499] nix-grpc-daemon[1762]: time=2026-09-21T09:37:08.666Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6193lb # [ 158.060388] niks3-server[974]: 2026/09/21 09:37:08 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6194lb # [ 158.073374] niks3-server[974]: 2026/09/21 09:37:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6195worker1 # [ 158.078780] nix-grpc-daemon[1762]: time=2026-09-21T09:37:08.705Z level=INFO msg="Uploading 1 narinfos"6196lb # [ 158.081696] niks3-server[974]: 2026/09/21 09:37:08 INFO Signed narinfos id=60 count=16197lb # [ 158.090242] niks3-server[974]: 2026/09/21 09:37:08 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6198lb # [ 158.101245] niks3-server[974]: 2026/09/21 09:37:08 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6199worker1 # [ 158.103842] nix-grpc-daemon[1762]: time=2026-09-21T09:37:08.730Z level=INFO msg="Upload complete. (476ms)"6200lb # [ 158.107637] niks3-server[974]: 2026/09/21 09:37:08 INFO Completed upload id=606201lb # [ 158.111313] niks3-server[974]: 2026/09/21 09:37:08 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6202worker1 # [ 158.109934] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=0 paths=1 nar_bytes_in=7526203worker1 # [ 158.121758] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546204worker1 # [ 158.126859] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6205worker1 # [ 158.131395] nix-daemon[839]: reaped child process 2018, status = succeeded6206worker1 # [ 158.143945] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6207worker1 # [ 159.569435] nix-daemon[839]: reaped child process 2032, status = succeeded6208worker1 # [ 159.730544] nix-daemon[839]: accepted connection from pid 2048, user nix-grpc-daemon (trusted)6209worker1 # [ 159.748437] nix-daemon[839]: reaped child process 2051, status = succeeded6210lb # [ 159.766801] niks3-server[974]: 2026/09/21 09:37:10 INFO Received uploads request method=POST path=/api/pending_closures6211worker1 # [ 159.776401] nix-grpc-daemon[1762]: time=2026-09-21T09:37:10.402Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6212worker1 # [ 159.777877] nix-grpc-daemon[1762]: time=2026-09-21T09:37:10.402Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6213lb # [ 159.804922] niks3-server[974]: 2026/09/21 09:37:10 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6214lb # [ 159.818712] niks3-server[974]: 2026/09/21 09:37:10 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6215worker1 # [ 159.823948] nix-grpc-daemon[1762]: time=2026-09-21T09:37:10.450Z level=INFO msg="Uploading 1 narinfos"6216lb # [ 159.828876] niks3-server[974]: 2026/09/21 09:37:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6217lb # [ 159.837016] niks3-server[974]: 2026/09/21 09:37:10 INFO Signed narinfos id=61 count=16218lb # [ 159.839023] niks3-server[974]: 2026/09/21 09:37:10 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6219lb # [ 159.844526] niks3-server[974]: 2026/09/21 09:37:10 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6220lb # [ 159.851721] niks3-server[974]: 2026/09/21 09:37:10 INFO Completed upload id=616221worker1 # [ 159.849087] nix-grpc-daemon[1762]: time=2026-09-21T09:37:10.476Z level=INFO msg="Upload complete. (292ms)"6222lb # [ 159.854258] niks3-server[974]: 2026/09/21 09:37:10 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6223worker2 # [ 159.903354] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:10Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16224worker1 # [ 159.858024] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16225worker1 # [ 159.873659] nix-daemon[839]: reaped child process 2030, status = succeeded6226worker1 # [ 159.901389] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546227worker1 # [ 159.915807] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519546228worker1 # [ 159.930280] nix-grpc-daemon[1762]: time=2026-09-21T09:37:10.557Z level=INFO msg="All 2 paths already cached"6229worker1 # [ 159.939257] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:519546230worker1 # [ 159.953029] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6231worker1 # [ 160.148619] nix-daemon[839]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)6232worker1 # [ 160.166631] nix-daemon[839]: reaped child process 2058, status = succeeded6233worker1 # [ 160.340876] nix-daemon[839]: accepted connection from pid 2060, user nix-grpc-daemon (trusted)6234worker1 # [ 160.358549] nix-daemon[839]: reaped child process 2063, status = succeeded6235lb # [ 160.374968] niks3-server[974]: 2026/09/21 09:37:11 INFO Received uploads request method=POST path=/api/pending_closures6236worker1 # [ 160.385577] nix-grpc-daemon[1762]: time=2026-09-21T09:37:11.012Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6237worker1 # [ 160.387041] nix-grpc-daemon[1762]: time=2026-09-21T09:37:11.012Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6238lb # [ 160.410814] niks3-server[974]: 2026/09/21 09:37:11 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6239worker1 # [ 160.419497] nix-grpc-daemon[1762]: time=2026-09-21T09:37:11.046Z level=INFO msg="Uploading 1 narinfos"6240lb # [ 160.424240] niks3-server[974]: 2026/09/21 09:37:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6241lb # [ 160.434001] niks3-server[974]: 2026/09/21 09:37:11 INFO Signed narinfos id=62 count=16242lb # [ 160.440555] niks3-server[974]: 2026/09/21 09:37:11 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6243lb # [ 160.453983] niks3-server[974]: 2026/09/21 09:37:11 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6244lb # [ 160.459188] niks3-server[974]: 2026/09/21 09:37:11 INFO Completed upload id=626245worker1 # [ 160.455710] nix-grpc-daemon[1762]: time=2026-09-21T09:37:11.082Z level=INFO msg="Upload complete. (487ms)"6246lb # [ 160.464459] niks3-server[974]: 2026/09/21 09:37:11 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6247worker1 # [ 160.465590] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=0 paths=1 nar_bytes_in=7526248worker1 # [ 160.480370] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:519546249worker1 # [ 160.489237] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6250worker1 # [ 160.492443] nix-daemon[839]: reaped child process 2053, status = succeeded6251worker1 # [ 160.507279] nix-daemon[839]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6252worker1 # [ 161.341361] nix-daemon[839]: reaped child process 2067, status = succeeded6253worker1 # [ 161.516054] nix-daemon[839]: accepted connection from pid 2083, user nix-grpc-daemon (trusted)6254worker1 # [ 161.536291] nix-daemon[839]: reaped child process 2086, status = succeeded6255lb # [ 161.553566] niks3-server[974]: 2026/09/21 09:37:12 INFO Received uploads request method=POST path=/api/pending_closures6256worker1 # [ 161.563738] nix-grpc-daemon[1762]: time=2026-09-21T09:37:12.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6257worker1 # [ 161.565437] nix-grpc-daemon[1762]: time=2026-09-21T09:37:12.186Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6258lb # [ 161.592577] niks3-server[974]: 2026/09/21 09:37:12 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6259lb # [ 161.605469] niks3-server[974]: 2026/09/21 09:37:12 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6260lb # [ 161.616449] niks3-server[974]: 2026/09/21 09:37:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6261worker1 # [ 161.619158] nix-grpc-daemon[1762]: time=2026-09-21T09:37:12.245Z level=INFO msg="Uploading 1 narinfos"6262lb # [ 161.624190] niks3-server[974]: 2026/09/21 09:37:12 INFO Signed narinfos id=63 count=16263lb # [ 161.629965] niks3-server[974]: 2026/09/21 09:37:12 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6264worker1 # [ 161.639572] nix-grpc-daemon[1762]: time=2026-09-21T09:37:12.266Z level=INFO msg="Upload complete. (322ms)"6265lb # [ 161.642305] niks3-server[974]: 2026/09/21 09:37:12 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6266lb # [ 161.647219] niks3-server[974]: 2026/09/21 09:37:12 INFO Completed upload id=636267lb # [ 161.651012] niks3-server[974]: 2026/09/21 09:37:12 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6268worker1 # [ 161.653176] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51954 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16269worker1 # [ 161.667454] nix-daemon[839]: reaped child process 2065, status = succeeded6270client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.21 seconds)6271worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6272worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6273client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6274client # [ 161.888737] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6275client # Running as unit: gpu.service; invocation ID: 07e2f986984f4534b2b3302852480d636276client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6277lb: must succeed: curl -sf localhost:9901/clusters6278lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6279lb: must succeed: curl -sf localhost:9901/clusters6280lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6281worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6282worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6283client # [ 162.150888] nix[1336]: don't know how to build these paths:6284client # [ 162.151868] nix[1336]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6285worker2 # [ 162.186639] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:533826286worker1 # [ 162.145384] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:519546287worker2 # [ 162.216576] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:12Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06288client # [ 162.187115] nix[1336]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6289lb: must succeed: curl -sf localhost:9901/clusters6290lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6291lb: must succeed: curl -sf localhost:9901/clusters6292lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6293worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6294worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6295client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6296lb # [ 169.869060] envoy[917]: conn peer=[2001:db8:1::1]:33590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27187 tx=11430 ms=208896297client # [ 192.188827] nix[1336]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6298client # [ 192.201765] nix[1336]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6299client # [ 192.240899] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6300client # [ 192.248956] systemd[1]: gpu.service: Failed with result 'exit-code'.6301client # [ 192.252462] systemd[1]: gpu.service: Consumed 183ms CPU time over 30.357s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6302client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.24 seconds)6303client: must succeed: journalctl -u gpu -o cat6304client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6305lb: must succeed: curl -sf localhost:9901/clusters6306lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6307lb: must succeed: curl -sf localhost:9901/clusters6308lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6309worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6310worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6311(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 52.31 seconds)6312subtest: low disk drains a worker and builds go to the other6313worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6314worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.12 seconds)6315worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6316worker1 # [ 194.206660] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:44Z level=info event=unhealthy reason=min_free6317worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6318client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26319client # don't know how to build these paths:6320client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6321worker1 # [ 195.343088] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:519546322worker2 # [ 195.393918] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533826323worker2 # [ 195.419639] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:46Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06324worker2 # [ 195.425895] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:46Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16325worker2 # [ 195.467418] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533826326worker2 # [ 195.482696] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533826327client # copying 1 paths to 'lb:50051'...6328client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6329worker2 # [ 195.498208] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533826330worker2 # [ 195.514206] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6331worker2 # [ 195.722539] nix-daemon[860]: accepted connection from pid 1690, user nix-grpc-daemon (trusted)6332worker2 # [ 195.742093] nix-daemon[860]: reaped child process 1693, status = succeeded6333worker2 # [ 195.918211] nix-daemon[860]: accepted connection from pid 1695, user nix-grpc-daemon (trusted)6334worker2 # [ 195.934482] nix-daemon[860]: reaped child process 1698, status = succeeded6335lb # [ 195.903962] niks3-server[974]: 2026/09/21 09:37:46 INFO Received uploads request method=POST path=/api/pending_closures6336worker2 # [ 195.958169] nix-grpc-daemon[1475]: time=2026-09-21T09:37:46.567Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6337worker2 # [ 195.959625] nix-grpc-daemon[1475]: time=2026-09-21T09:37:46.567Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6338lb # [ 195.936953] niks3-server[974]: 2026/09/21 09:37:46 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6339worker2 # [ 195.993108] nix-grpc-daemon[1475]: time=2026-09-21T09:37:46.602Z level=INFO msg="Uploading 1 narinfos"6340lb # [ 195.951297] niks3-server[974]: 2026/09/21 09:37:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6341lb # [ 195.961970] niks3-server[974]: 2026/09/21 09:37:46 INFO Signed narinfos id=64 count=16342lb # [ 195.967439] niks3-server[974]: 2026/09/21 09:37:46 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6343lb # [ 195.975380] niks3-server[974]: 2026/09/21 09:37:46 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6344lb # [ 195.982020] niks3-server[974]: 2026/09/21 09:37:46 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6345lb # [ 195.986029] niks3-server[974]: 2026/09/21 09:37:46 INFO Completed upload id=646346worker2 # [ 196.029593] nix-grpc-daemon[1475]: time=2026-09-21T09:37:46.639Z level=INFO msg="Upload complete. (499ms)"6347worker2 # [ 196.037919] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=0 paths=1 nar_bytes_in=5126348worker2 # [ 196.052542] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533826349worker2 # [ 196.059058] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6350worker2 # [ 196.064078] nix-daemon[860]: reaped child process 1688, status = succeeded6351client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6352client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6353worker2 # [ 196.077539] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6354client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6355client # farm-a> LOG-farm-a-drain6356worker2 # [ 197.526370] nix-daemon[860]: reaped child process 1702, status = succeeded6357worker2 # [ 197.705174] nix-daemon[860]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)6358worker2 # [ 197.725317] nix-daemon[860]: reaped child process 1721, status = succeeded6359lb # [ 197.694231] niks3-server[974]: 2026/09/21 09:37:48 INFO Received uploads request method=POST path=/api/pending_closures6360worker2 # [ 197.748525] nix-grpc-daemon[1475]: time=2026-09-21T09:37:48.356Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6361worker2 # [ 197.750390] nix-grpc-daemon[1475]: time=2026-09-21T09:37:48.357Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6362lb # [ 197.737334] niks3-server[974]: 2026/09/21 09:37:48 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6363lb # [ 197.748721] niks3-server[974]: 2026/09/21 09:37:48 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6364lb # [ 197.758427] niks3-server[974]: 2026/09/21 09:37:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6365worker2 # [ 197.807808] nix-grpc-daemon[1475]: time=2026-09-21T09:37:48.417Z level=INFO msg="Uploading 1 narinfos"6366lb # [ 197.767015] niks3-server[974]: 2026/09/21 09:37:48 INFO Signed narinfos id=65 count=16367lb # [ 197.775453] niks3-server[974]: 2026/09/21 09:37:48 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6368lb # [ 197.784105] niks3-server[974]: 2026/09/21 09:37:48 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6369worker2 # [ 197.833307] nix-grpc-daemon[1475]: time=2026-09-21T09:37:48.443Z level=INFO msg="Upload complete. (344ms)"6370lb # [ 197.792107] niks3-server[974]: 2026/09/21 09:37:48 INFO Completed upload id=656371lb # [ 197.796266] niks3-server[974]: 2026/09/21 09:37:48 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6372worker2 # [ 197.849286] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16373worker2 # [ 197.871546] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:48Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06374worker2 # [ 197.876628] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:48Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16375worker2 # [ 197.881812] nix-daemon[860]: reaped child process 1700, status = succeeded6376worker2 # [ 197.908142] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533826377worker2 # [ 197.925343] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533826378worker2 # [ 197.938760] nix-grpc-daemon[1475]: time=2026-09-21T09:37:48.548Z level=INFO msg="All 1 paths already cached"6379client # copying 1 paths to 'lb:50051'...6380client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6381worker2 # [ 197.948951] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533826382worker2 # [ 197.964295] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6383worker2 # [ 198.168972] nix-daemon[860]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6384worker2 # [ 198.186446] nix-daemon[860]: reaped child process 1730, status = succeeded6385worker2 # [ 198.356272] nix-daemon[860]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)6386worker2 # [ 198.371740] nix-daemon[860]: reaped child process 1735, status = succeeded6387lb # [ 198.341581] niks3-server[974]: 2026/09/21 09:37:48 INFO Received uploads request method=POST path=/api/pending_closures6388worker2 # [ 198.394354] nix-grpc-daemon[1475]: time=2026-09-21T09:37:49.004Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6389worker2 # [ 198.395813] nix-grpc-daemon[1475]: time=2026-09-21T09:37:49.004Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6390lb # [ 198.373186] niks3-server[974]: 2026/09/21 09:37:49 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6391worker2 # [ 198.426926] nix-grpc-daemon[1475]: time=2026-09-21T09:37:49.036Z level=INFO msg="Uploading 1 narinfos"6392lb # [ 198.383408] niks3-server[974]: 2026/09/21 09:37:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6393lb # [ 198.395838] niks3-server[974]: 2026/09/21 09:37:49 INFO Signed narinfos id=66 count=16394lb # [ 198.403054] niks3-server[974]: 2026/09/21 09:37:49 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6395worker2 # [ 198.457549] nix-grpc-daemon[1475]: time=2026-09-21T09:37:49.067Z level=INFO msg="Upload complete. (476ms)"6396lb # [ 198.414359] niks3-server[974]: 2026/09/21 09:37:49 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6397lb # [ 198.419524] niks3-server[974]: 2026/09/21 09:37:49 INFO Completed upload id=666398lb # [ 198.422656] niks3-server[974]: 2026/09/21 09:37:49 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6399worker2 # [ 198.466733] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=0 paths=1 nar_bytes_in=6326400worker2 # [ 198.478712] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533826401worker2 # [ 198.484909] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6402worker2 # [ 198.489720] nix-daemon[860]: reaped child process 1725, status = succeeded6403client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6404client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6405worker2 # [ 198.502761] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6406client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6407client # farm-b> LOG-farm-b-drain6408lb # [ 199.868620] envoy[917]: conn peer=[2001:db8:1::1]:37916 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300626409worker2 # [ 199.927547] nix-daemon[860]: reaped child process 1739, status = succeeded6410worker2 # [ 200.094539] nix-daemon[860]: accepted connection from pid 1755, user nix-grpc-daemon (trusted)6411worker2 # [ 200.111856] nix-daemon[860]: reaped child process 1758, status = succeeded6412lb # [ 200.083361] niks3-server[974]: 2026/09/21 09:37:50 INFO Received uploads request method=POST path=/api/pending_closures6413worker2 # [ 200.139209] nix-grpc-daemon[1475]: time=2026-09-21T09:37:50.745Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6414worker2 # [ 200.140821] nix-grpc-daemon[1475]: time=2026-09-21T09:37:50.745Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6415lb # [ 200.124726] niks3-server[974]: 2026/09/21 09:37:50 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6416worker2 # [ 200.178478] nix-grpc-daemon[1475]: time=2026-09-21T09:37:50.788Z level=INFO msg="Uploading 1 narinfos"6417lb # [ 200.140393] niks3-server[974]: 2026/09/21 09:37:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6418lb # [ 200.149039] niks3-server[974]: 2026/09/21 09:37:50 INFO Signed narinfos id=67 count=16419lb # [ 200.151201] niks3-server[974]: 2026/09/21 09:37:50 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6420lb # [ 200.155125] niks3-server[974]: 2026/09/21 09:37:50 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6421lb # [ 200.161535] niks3-server[974]: 2026/09/21 09:37:50 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6422lb # [ 200.165743] niks3-server[974]: 2026/09/21 09:37:50 INFO Completed upload id=676423worker2 # [ 200.209340] nix-grpc-daemon[1475]: time=2026-09-21T09:37:50.819Z level=INFO msg="Upload complete. (296ms)"6424lb # [ 200.168086] niks3-server[974]: 2026/09/21 09:37:50 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6425worker2 # [ 200.228412] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16426worker2 # [ 200.238184] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:50Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06427worker2 # [ 200.241763] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:50Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16428worker2 # [ 200.245372] nix-daemon[860]: reaped child process 1737, status = succeeded6429worker2 # [ 200.267203] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533826430worker2 # [ 200.280542] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533826431worker2 # [ 200.292921] nix-grpc-daemon[1475]: time=2026-09-21T09:37:50.902Z level=INFO msg="All 2 paths already cached"6432client # copying 1 paths to 'lb:50051'...6433client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6434worker2 # [ 200.302058] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533826435worker2 # [ 200.315109] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6436worker2 # [ 200.519342] nix-daemon[860]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6437worker2 # [ 200.538447] nix-daemon[860]: reaped child process 1767, status = succeeded6438worker2 # [ 200.713099] nix-daemon[860]: accepted connection from pid 1769, user nix-grpc-daemon (trusted)6439worker2 # [ 200.729474] nix-daemon[860]: reaped child process 1772, status = succeeded6440lb # [ 200.697433] niks3-server[974]: 2026/09/21 09:37:51 INFO Received uploads request method=POST path=/api/pending_closures6441worker2 # [ 200.749548] nix-grpc-daemon[1475]: time=2026-09-21T09:37:51.359Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6442worker2 # [ 200.751070] nix-grpc-daemon[1475]: time=2026-09-21T09:37:51.359Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6443lb # [ 200.722824] niks3-server[974]: 2026/09/21 09:37:51 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6444worker2 # [ 200.780201] nix-grpc-daemon[1475]: time=2026-09-21T09:37:51.390Z level=INFO msg="Uploading 1 narinfos"6445lb # [ 200.739496] niks3-server[974]: 2026/09/21 09:37:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6446lb # [ 200.749657] niks3-server[974]: 2026/09/21 09:37:51 INFO Signed narinfos id=68 count=16447lb # [ 200.759218] niks3-server[974]: 2026/09/21 09:37:51 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6448lb # [ 200.763123] niks3-server[974]: 2026/09/21 09:37:51 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6449worker2 # [ 200.810702] nix-grpc-daemon[1475]: time=2026-09-21T09:37:51.420Z level=INFO msg="Upload complete. (480ms)"6450lb # [ 200.767745] niks3-server[974]: 2026/09/21 09:37:51 INFO Completed upload id=686451lb # [ 200.771291] niks3-server[974]: 2026/09/21 09:37:51 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6452worker2 # [ 200.817471] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=0 paths=1 nar_bytes_in=7606453worker2 # [ 200.830331] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533826454worker2 # [ 200.835687] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6455worker2 # [ 200.840491] nix-daemon[860]: reaped child process 1762, status = succeeded6456worker2 # [ 200.850844] nix-daemon[860]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6457client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6458client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6459client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6460client # farm-top> LOG-farm-top-drain6461worker2 # [ 201.472370] nix-daemon[860]: reaped child process 1776, status = succeeded6462worker2 # [ 201.628433] nix-daemon[860]: accepted connection from pid 1792, user nix-grpc-daemon (trusted)6463worker2 # [ 201.644853] nix-daemon[860]: reaped child process 1795, status = succeeded6464lb # [ 201.617856] niks3-server[974]: 2026/09/21 09:37:52 INFO Received uploads request method=POST path=/api/pending_closures6465worker2 # [ 201.669904] nix-grpc-daemon[1475]: time=2026-09-21T09:37:52.278Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6466worker2 # [ 201.671351] nix-grpc-daemon[1475]: time=2026-09-21T09:37:52.278Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6467lb # [ 201.653603] niks3-server[974]: 2026/09/21 09:37:52 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6468worker2 # [ 201.710022] nix-grpc-daemon[1475]: time=2026-09-21T09:37:52.319Z level=INFO msg="Uploading 1 narinfos"6469lb # [ 201.668493] niks3-server[974]: 2026/09/21 09:37:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6470lb # [ 201.678929] niks3-server[974]: 2026/09/21 09:37:52 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6471lb # [ 201.688771] niks3-server[974]: 2026/09/21 09:37:52 INFO Signed narinfos id=69 count=16472lb # [ 201.691393] niks3-server[974]: 2026/09/21 09:37:52 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6473worker2 # [ 201.738812] nix-grpc-daemon[1475]: time=2026-09-21T09:37:52.348Z level=INFO msg="Upload complete. (277ms)"6474lb # [ 201.695829] niks3-server[974]: 2026/09/21 09:37:52 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6475lb # [ 201.702380] niks3-server[974]: 2026/09/21 09:37:52 INFO Completed upload id=696476lb # [ 201.705204] niks3-server[974]: 2026/09/21 09:37:52 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6477worker2 # [ 201.749440] nix-grpc-daemon[1395]: ts=2026-09-21T09:37:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53382 duration_s=0 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16478worker2 # [ 201.761271] nix-daemon[860]: reaped child process 1774, status = succeeded6479client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.69 seconds)6480client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6481client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.24 seconds)6482worker1: must succeed: rm /nix/.rw-store/fill6483worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6484worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6485worker1 # [ 203.214022] nix-grpc-daemon[1702]: ts=2026-09-21T09:37:53Z level=info event=healthy reason=min_free6486worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6487(finished: subtest: low disk drains a worker and builds go to the other, in 10.55 seconds)6488(finished: run the VM test script, in 204.29 seconds)6489test script finished in 204.58s6490cleanup6491kill QemuMachine (pid 49)6492client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6493kill QemuMachine (pid 50)6494lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6495kill QemuMachine (pid 52)6496worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6497kill QemuMachine (pid 51)6498worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6499(finished: cleanup, in 1.08 seconds)6500additionally exposed symbols:6501 client, lb, worker1, worker2,6502 vlan1,6503 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh