vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #347
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.kJlQ46DaVT', fmt=raw size=107374182414worker2 # mke2fs 1.47.4 (6-Mar-2025)15lb: QEMU running (pid 50)16client # Disk image does not exist, creating the virtualisation disk image...17client: QEMU running (pid 49)18worker2: QEMU running (pid 52)19worker1 # Disk image does not exist, creating the virtualisation disk image...20client # Formatting '/build/vm-state-client/tmp.b05vjOMe96', fmt=raw size=107374182421lb # Disk image does not exist, creating the virtualisation disk image...22worker1 # Formatting '/build/vm-state-worker1/tmp.PepdL3BtmA', fmt=raw size=107374182423client # mke2fs 1.47.4 (6-Mar-2025)24lb # Formatting '/build/vm-state-lb/tmp.UWdvQNKpR7', fmt=raw size=107374182425worker1 # mke2fs 1.47.4 (6-Mar-2025)26client # Discarding device blocks: 0/262144 done27lb # mke2fs 1.47.4 (6-Mar-2025)28worker1 # Discarding device blocks: 0/262144 done29client # Creating filesystem with 262144 4k blocks and 65536 inodes30lb # Discarding device blocks: 0/262144 done31worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes32client # Filesystem UUID: 291c9adc-67bd-468f-8271-a9cfc0e6273833lb # Creating filesystem with 262144 4k blocks and 65536 inodes34worker1 # Filesystem UUID: 7cf2ff25-fb6d-43f1-9fa1-b0f62ac739a235client # Superblock backups stored on blocks:36client # 32768, 98304, 163840, 22937637worker1 # Superblock backups stored on blocks:38lb # Filesystem UUID: fc7c4b41-2e3c-4992-88b1-b76f0ca097a139worker2 # Discarding device blocks: 0/262144 done40worker1: QEMU running (pid 51)41client # 42worker1 # 32768, 98304, 163840, 22937643worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes44lb # Superblock backups stored on blocks:45client # Allocating group tables: 0/8 done46worker2 # Filesystem UUID: b333ff5c-a552-45bd-8136-190a86afb19c47lb # 32768, 98304, 163840, 22937648client # Writing inode tables: 0/8 done49worker2 # Superblock backups stored on blocks:50lb # 51client # Creating journal (8192 blocks): done52worker2 # 32768, 98304, 163840, 22937653lb # Allocating group tables: 0/8 done54client # Writing superblocks and filesystem accounting information: 0/8 done55worker2 # 56lb # Writing inode tables: 0/8 done57client # 58(finished: start all VMs, in 0.58 seconds)59worker1 # 60worker2 # Allocating group tables: 0/8 done61lb # Creating journal (8192 blocks): done62client # Virtualisation disk image created.63worker2 # Writing inode tables: 0/8 done64lb # Writing superblocks and filesystem accounting information: 0/8 done65worker1 # Allocating group tables: 0/8 done66worker2 # Creating journal (8192 blocks): done67client: waiting for unit network-addresses-eth1.service68lb # 69worker1 # Writing inode tables: 0/8 done70lb # Virtualisation disk image created.71worker2 # Writing superblocks and filesystem accounting information: 0/8 done72client: waiting for the VM to finish booting73worker2 # 74worker2 # Virtualisation disk image created.75worker1 # Creating journal (8192 blocks): done76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77worker1 # 78worker1 # Virtualisation disk image created.79worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker2 # [ 0.000000] KASLR enabled82worker2 # [ 0.000000] random: crng init done83worker2 # [ 0.000000] Machine model: linux,dummy-virt84worker2 # [ 0.000000] efi: UEFI not found.85worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88worker2 # [ 0.000000] Zone ranges:89worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90worker2 # [ 0.000000] DMA32 empty91worker2 # [ 0.000000] Normal empty92worker2 # [ 0.000000] Device empty93worker2 # [ 0.000000] Movable zone start for each node94worker2 # [ 0.000000] Early memory node ranges95worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098worker2 # [ 0.000000] psci: probing for conduit method from DT.99worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.100worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101worker2 # [ 0.000000] psci: Trusted OS migration not required102worker2 # [ 0.000000] psci: SMC Calling Convention v1.1103worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105worker2 # [ 0.000000] Detected PIPT I-cache on CPU0106worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface108worker2 # [ 0.000000] CPU features: detected: Spectre-v4109worker2 # [ 0.000000] CPU features: detected: Spectre-BHB110worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112worker2 # [ 0.000000] alternatives: applying boot alternatives113worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/sz04kyfsy91zmkawdbs4qc3nwqq4ifxa-nixos-system-worker2-test/init regInfo=/nix/store/v1zilvhj14c2kyy75xkp2aamz014w5a1-closure-info/registration console=ttyAMA0,115200n8 console=tty0114worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/v1zilvhj14c2kyy75xkp2aamz014w5a1-closure-info/registration", will be passed to user space.115worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119worker2 # [ 0.000000] software IO TLB: area num 1.120worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)122worker2 # [ 0.000000] Fallback order for Node 0: 0123worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216124worker2 # [ 0.000000] Policy zone: DMA125worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off126worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1127worker2 # [ 0.000000] allocated 3145728 bytes of page_ext128worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages129worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups130worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.131worker2 # [ 0.000000] rcu: RCU event tracing is enabled.132worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.133worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.134worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.135worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.136worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.137worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1138worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.139worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.140worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.141worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0142worker2 # [ 0.000000] GICv3: 256 SPIs implemented143client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]144worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented145worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq146worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI147client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026148worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0149client # [ 0.000000] KASLR enabled150client # [ 0.000000] random: crng init done151worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000152client # [ 0.000000] Machine model: linux,dummy-virt153worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]154client # [ 0.000000] efi: UEFI not found.155lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]156client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT157worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)158client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]159worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)160lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026161client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]162lb # [ 0.000000] KASLR enabled163client # [ 0.000000] Zone ranges:164worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000165lb # [ 0.000000] random: crng init done166client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]167lb # [ 0.000000] Machine model: linux,dummy-virt168worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000169client # [ 0.000000] DMA32 empty170lb # [ 0.000000] efi: UEFI not found.171client # [ 0.000000] Normal empty172worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.173client # [ 0.000000] Device empty174lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT175client # [ 0.000000] Movable zone start for each node176client # [ 0.000000] Early memory node ranges177worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns178lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]179client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]180worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).181lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]182lb # [ 0.000000] Zone ranges:183client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]184lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]185worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns186client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000187lb # [ 0.000000] DMA32 empty188lb # [ 0.000000] Normal empty189client # [ 0.000000] psci: probing for conduit method from DT.190lb # [ 0.000000] Device empty191worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns192client # [ 0.000000] psci: PSCIv1.3 detected in firmware.193lb # [ 0.000000] Movable zone start for each node194worker2 # [ 0.000046] arm-pv: using stolen time PV195client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs196lb # [ 0.000000] Early memory node ranges197client # [ 0.000000] psci: Trusted OS migration not required198worker2 # [ 0.000928] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)199lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]200client # [ 0.000000] psci: SMC Calling Convention v1.1201worker2 # [ 0.001140] Console: colour dummy device 80x25202lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]203worker2 # [ 0.001148] printk: legacy console [tty0] enabled204client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)205lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000206lb # [ 0.000000] psci: probing for conduit method from DT.207client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296208worker2 # [ 0.001364] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)209lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.210client # [ 0.000000] Detected PIPT I-cache on CPU0211worker2 # [ 0.001371] pid_max: default: 32768 minimum: 301212lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs213worker2 # [ 0.001472] LSM: initializing lsm=capability,landlock,yama,bpf,ima214client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)215lb # [ 0.000000] psci: Trusted OS migration not required216worker2 # [ 0.001707] landlock: Up and running.217client # [ 0.000000] CPU features: detected: GICv3 CPU interface218lb # [ 0.000000] psci: SMC Calling Convention v1.1219worker2 # [ 0.001710] Yama: becoming mindful.220client # [ 0.000000] CPU features: detected: Spectre-v4221worker2 # [ 0.002497] LSM support for eBPF active222client # [ 0.000000] CPU features: detected: Spectre-BHB223lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)224worker2 # [ 0.002671] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)225lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296226client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38227worker2 # [ 0.002714] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)228client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23229lb # [ 0.000000] Detected PIPT I-cache on CPU0230client # [ 0.000000] alternatives: applying boot alternatives231worker2 # [ 0.003918] cacheinfo: Unable to detect cache hierarchy for CPU 0232lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)233worker2 # [ 0.004784] rcu: Hierarchical SRCU implementation.234lb # [ 0.000000] CPU features: detected: GICv3 CPU interface235worker2 # [ 0.004790] rcu: Max phase no-delay instances is 1000.236lb # [ 0.000000] CPU features: detected: Spectre-v4237worker2 # [ 0.006030] fsl-mc MSI: its@8080000 domain created238lb # [ 0.000000] CPU features: detected: Spectre-BHB239worker2 # [ 0.006119] EFI services will not be available.240lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38241worker2 # [ 0.006192] smp: Bringing up secondary CPUs ...242worker2 # [ 0.006200] smp: Brought up 1 node, 1 CPU243lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23244client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7mgb5khx78dzs8ml2mm99qhfy7adra5b-nixos-system-client-test/init regInfo=/nix/store/84zhlzxk1bvixcgdyc6kiif8qp4br93z-closure-info/registration console=ttyAMA0,115200n8 console=tty0245worker2 # [ 0.006204] SMP: Total of 1 processors activated.246lb # [ 0.000000] alternatives: applying boot alternatives247worker2 # [ 0.006210] CPU: All CPU(s) started at EL1248worker2 # [ 0.006221] CPU features: detected: Branch Target Identification249client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/84zhlzxk1bvixcgdyc6kiif8qp4br93z-closure-info/registration", will be passed to user space.250worker2 # [ 0.006226] CPU features: detected: ARMv8.4 Translation Table Level251client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes252worker2 # [ 0.006229] CPU features: detected: Instruction cache invalidation not required for I/D coherence253client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)254lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lai4n7chb0cnvnygfis27sipihspygia-nixos-system-lb-test/init regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration console=ttyAMA0,115200n8 console=tty0255worker2 # [ 0.006233] CPU features: detected: Data cache clean to the PoU not required for I/D coherence256client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)257worker2 # [ 0.006236] CPU features: detected: Common not Private translations258client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB259client # [ 0.000000] software IO TLB: area num 1.260worker2 # [ 0.006239] CPU features: detected: CRC32 instructions261lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration", will be passed to user space.262client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB263worker2 # [ 0.006242] CPU features: detected: Data cache clean to Point of Deep Persistence264lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes265client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)266worker2 # [ 0.006246] CPU features: detected: Data cache clean to Point of Persistence267client # [ 0.000000] Fallback order for Node 0: 0268lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)269worker2 # [ 0.006249] CPU features: detected: Data independent timing control (DIT)270worker2 # [ 0.006252] CPU features: detected: E0PD271client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216272lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)273client # [ 0.000000] Policy zone: DMA274worker2 # [ 0.006255] CPU features: detected: Enhanced Counter Virtualization275lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB276client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off277lb # [ 0.000000] software IO TLB: area num 1.278worker2 # [ 0.006258] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)279client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1280lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB281worker2 # [ 0.006261] CPU features: detected: Enhanced Virtualization Traps282client # [ 0.000000] allocated 3145728 bytes of page_ext283worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]284worker2 # [ 0.006264] CPU features: detected: Fine Grained Traps285lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)286client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages287lb # [ 0.000000] Fallback order for Node 0: 0288client # [ 0.000000] ftrace: allocated 294 pages with 4 groups289worker2 # [ 0.006267] CPU features: detected: Generic authentication (architected QARMA5 algorithm)290worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026291client # [ 0.000000] rcu: Hierarchical RCU implementation.292worker2 # [ 0.006273] CPU features: detected: RCpc load-acquire (LDAPR)293worker1 # [ 0.000000] KASLR enabled294lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216295lb # [ 0.000000] Policy zone: DMA296client # [ 0.000000] rcu: RCU event tracing is enabled.297worker1 # [ 0.000000] random: crng init done298worker2 # [ 0.006276] CPU features: detected: LSE atomic instructions299worker1 # [ 0.000000] Machine model: linux,dummy-virt300client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.301lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off302worker1 # [ 0.000000] efi: UEFI not found.303worker2 # [ 0.006279] CPU features: detected: Privileged Access Never304client # [ 0.000000] Trampoline variant of Tasks RCU enabled.305worker2 # [ 0.006281] CPU features: detected: PMUv3306lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1307worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT308client # [ 0.000000] Rude variant of Tasks RCU enabled.309worker2 # [ 0.006284] CPU features: detected: RAS Extension Support310lb # [ 0.000000] allocated 3145728 bytes of page_ext311client # [ 0.000000] Tracing variant of Tasks RCU enabled.312worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]313lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages314worker2 # [ 0.006287] CPU features: detected: RASv1p1 Extension Support315lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups316worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]317worker2 # [ 0.006289] CPU features: detected: Random Number Generator318client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.319worker1 # [ 0.000000] Zone ranges:320lb # [ 0.000000] rcu: Hierarchical RCU implementation.321worker2 # [ 0.006292] CPU features: detected: Speculation barrier (SB)322client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1323worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]324lb # [ 0.000000] rcu: RCU event tracing is enabled.325worker1 # [ 0.000000] DMA32 empty326worker2 # [ 0.006295] CPU features: detected: Stage-2 Force Write-Back327worker1 # [ 0.000000] Normal empty328lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.329client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.330worker1 # [ 0.000000] Device empty331worker2 # [ 0.006298] CPU features: detected: TLB range maintenance instructions332lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.333worker1 # [ 0.000000] Movable zone start for each node334worker2 # [ 0.006303] CPU features: detected: Speculative Store Bypassing Safe (SSBS)335client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.336lb # [ 0.000000] Rude variant of Tasks RCU enabled.337worker1 # [ 0.000000] Early memory node ranges338lb # [ 0.000000] Tracing variant of Tasks RCU enabled.339worker2 # [ 0.006348] alternatives: applying system-wide alternatives340worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]341client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.342worker2 # [ 0.010338] CPU features: detected: BBM Level 2 without TLB conflict abort343lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.344worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]345client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0346client # [ 0.000000] GICv3: 256 SPIs implemented347lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1348client # [ 0.000000] GICv3: 0 Extended SPIs implemented349worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000350worker2 # [ 0.010547] Memory: 1406544K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)351client # [ 0.000000] Root IRQ handler: gic_handle_irq352lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.353worker1 # [ 0.000000] psci: probing for conduit method from DT.354worker2 # [ 0.011033] devtmpfs: initialized355client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI356worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.357client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0358lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.359worker2 # [ 0.012944] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)360worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs361client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000362worker1 # [ 0.000000] psci: Trusted OS migration not required363lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.364worker2 # [ 0.012970] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).365client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]366worker1 # [ 0.000000] psci: SMC Calling Convention v1.1367lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0368worker2 # [ 0.013197] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL369lb # [ 0.000000] GICv3: 256 SPIs implemented370client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)371worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)372worker2 # [ 0.013202] 0 pages in range for non-PLT usage373lb # [ 0.000000] GICv3: 0 Extended SPIs implemented374worker2 # [ 0.013203] 508288 pages in range for PLT usage375lb # [ 0.000000] Root IRQ handler: gic_handle_irq376worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296377client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)378worker2 # [ 0.013357] pinctrl core: initialized pinctrl subsystem379worker1 # [ 0.000000] Detected PIPT I-cache on CPU0380lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI381worker2 # [ 0.014137] DMI not present or invalid.382client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000383lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0384worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)385worker2 # [ 0.018412] NET: Registered PF_NETLINK/PF_ROUTE protocol family386client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000387worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface388lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000389worker2 # [ 0.020801] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations390worker1 # [ 0.000000] CPU features: detected: Spectre-v4391lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]392client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.393worker1 # [ 0.000000] CPU features: detected: Spectre-BHB394worker2 # [ 0.021013] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations395lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)396client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns397worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38398worker2 # [ 0.021227] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations399client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).400worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23401lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)402worker1 # [ 0.000000] alternatives: applying boot alternatives403lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000404client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns405lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000406client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns407client # [ 0.000037] arm-pv: using stolen time PV408lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.409client # [ 0.000529] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)410lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns411worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vm1y97vvahz8imhr98ssg2a2k7azswsy-nixos-system-worker1-test/init regInfo=/nix/store/alkhn6bpcgc98abc8xkqwq30rmpcigfs-closure-info/registration console=ttyAMA0,115200n8 console=tty0412lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).413client # [ 0.000752] Console: colour dummy device 80x25414client # [ 0.000761] printk: legacy console [tty0] enabled415worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/alkhn6bpcgc98abc8xkqwq30rmpcigfs-closure-info/registration", will be passed to user space.416lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns417worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes418lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns419client # [ 0.000954] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)420lb # [ 0.000044] arm-pv: using stolen time PV421client # [ 0.000961] pid_max: default: 32768 minimum: 301422worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)423worker2 # [ 0.021272] audit: initializing netlink subsys (disabled)424client # [ 0.001046] LSM: initializing lsm=capability,landlock,yama,bpf,ima425worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)426lb # [ 0.000534] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)427client # [ 0.001206] landlock: Up and running.428worker2 # [ 0.021966] thermal_sys: Registered thermal governor 'fair_share'429lb # [ 0.000733] Console: colour dummy device 80x25430client # [ 0.001209] Yama: becoming mindful.431worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB432worker2 # [ 0.021970] thermal_sys: Registered thermal governor 'bang_bang'433client # [ 0.001816] LSM support for eBPF active434lb # [ 0.000743] printk: legacy console [tty0] enabled435worker1 # [ 0.000000] software IO TLB: area num 1.436worker2 # [ 0.021974] thermal_sys: Registered thermal governor 'step_wise'437client # [ 0.002030] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)438worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB439worker2 # [ 0.021979] thermal_sys: Registered thermal governor 'user_space'440lb # [ 0.000956] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)441client # [ 0.002064] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)442worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)443lb # [ 0.000963] pid_max: default: 32768 minimum: 301444worker2 # [ 0.021982] thermal_sys: Registered thermal governor 'power_allocator'445worker1 # [ 0.000000] Fallback order for Node 0: 0446client # [ 0.003318] cacheinfo: Unable to detect cache hierarchy for CPU 0447lb # [ 0.001039] LSM: initializing lsm=capability,landlock,yama,bpf,ima448client # [ 0.004054] rcu: Hierarchical SRCU implementation.449worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216450lb # [ 0.001239] landlock: Up and running.451worker2 # [ 0.022016] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1452worker2 # [ 0.022027] cpuidle: using governor ladder453lb # [ 0.001242] Yama: becoming mindful.454worker1 # [ 0.000000] Policy zone: DMA455client # [ 0.004058] rcu: Max phase no-delay instances is 1000.456lb # [ 0.001842] LSM support for eBPF active457worker2 # [ 0.022032] cpuidle: using governor menu458client # [ 0.005273] fsl-mc MSI: its@8080000 domain created459worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off460client # [ 0.005365] EFI services will not be available.461lb # [ 0.002026] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)462worker2 # [ 0.022248] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.463worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1464client # [ 0.005441] smp: Bringing up secondary CPUs ...465worker2 # [ 0.022266] ASID allocator initialised with 65536 entries466worker1 # [ 0.000000] allocated 3145728 bytes of page_ext467client # [ 0.005450] smp: Brought up 1 node, 1 CPU468lb # [ 0.002064] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)469worker2 # [ 0.023627] Serial: AMBA PL011 UART driver470worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages471client # [ 0.005453] SMP: Total of 1 processors activated.472lb # [ 0.003278] cacheinfo: Unable to detect cache hierarchy for CPU 0473worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups474client # [ 0.005458] CPU: All CPU(s) started at EL1475worker2 # [ 0.029311] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1476lb # [ 0.004077] rcu: Hierarchical SRCU implementation.477worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.478worker2 # [ 0.029444] printk: console [ttyAMA0] enabled479client # [ 0.005472] CPU features: detected: Branch Target Identification480lb # [ 0.004081] rcu: Max phase no-delay instances is 1000.481worker1 # [ 0.000000] rcu: RCU event tracing is enabled.482lb # [ 0.005320] fsl-mc MSI: its@8080000 domain created483client # [ 0.005477] CPU features: detected: ARMv8.4 Translation Table Level484worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.485lb # [ 0.005428] EFI services will not be available.486worker2 # [ 0.153582] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages487worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.488lb # [ 0.005532] smp: Bringing up secondary CPUs ...489client # [ 0.005480] CPU features: detected: Instruction cache invalidation not required for I/D coherence490worker2 # [ 0.153607] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page491worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.492lb # [ 0.005559] smp: Brought up 1 node, 1 CPU493worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.494client # [ 0.005484] CPU features: detected: Data cache clean to the PoU not required for I/D coherence495worker2 # [ 0.153612] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages496lb # [ 0.005563] SMP: Total of 1 processors activated.497lb # [ 0.005569] CPU: All CPU(s) started at EL1498client # [ 0.005488] CPU features: detected: Common not Private translations499worker2 # [ 0.153616] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page500worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.501lb # [ 0.005580] CPU features: detected: Branch Target Identification502client # [ 0.005491] CPU features: detected: CRC32 instructions503worker2 # [ 0.153620] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages504worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1505lb # [ 0.005585] CPU features: detected: ARMv8.4 Translation Table Level506client # [ 0.005493] CPU features: detected: Data cache clean to Point of Deep Persistence507worker2 # [ 0.153624] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page508worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.509client # [ 0.005497] CPU features: detected: Data cache clean to Point of Persistence510worker2 # [ 0.153628] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages511lb # [ 0.005588] CPU features: detected: Instruction cache invalidation not required for I/D coherence512worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.513client # [ 0.005500] CPU features: detected: Data independent timing control (DIT)514worker2 # [ 0.153632] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page515lb # [ 0.005592] CPU features: detected: Data cache clean to the PoU not required for I/D coherence516client # [ 0.005503] CPU features: detected: E0PD517worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.518lb # [ 0.005595] CPU features: detected: Common not Private translations519client # [ 0.005506] CPU features: detected: Enhanced Counter Virtualization520worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0521lb # [ 0.005599] CPU features: detected: CRC32 instructions522worker1 # [ 0.000000] GICv3: 256 SPIs implemented523client # [ 0.005509] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)524lb # [ 0.005602] CPU features: detected: Data cache clean to Point of Deep Persistence525worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented526worker2 # [ 0.161120] fbcon: Taking over console527client # [ 0.005512] CPU features: detected: Enhanced Virtualization Traps528worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq529worker2 # [ 0.161137] ACPI: Interpreter disabled.530lb # [ 0.005605] CPU features: detected: Data cache clean to Point of Persistence531client # [ 0.005515] CPU features: detected: Fine Grained Traps532worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI533worker2 # [ 0.163081] iommu: Default domain type: Translated534lb # [ 0.005608] CPU features: detected: Data independent timing control (DIT)535worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0536client # [ 0.005518] CPU features: detected: Generic authentication (architected QARMA5 algorithm)537worker2 # [ 0.163090] iommu: DMA domain TLB invalidation policy: strict mode538lb # [ 0.005612] CPU features: detected: E0PD539worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000540client # [ 0.005523] CPU features: detected: RCpc load-acquire (LDAPR)541worker2 # [ 0.164764] SCSI subsystem initialized542lb # [ 0.005614] CPU features: detected: Enhanced Counter Virtualization543worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]544client # [ 0.005526] CPU features: detected: LSE atomic instructions545lb # [ 0.005617] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)546client # [ 0.005529] CPU features: detected: Privileged Access Never547worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)548client # [ 0.005531] CPU features: detected: PMUv3549lb # [ 0.005621] CPU features: detected: Enhanced Virtualization Traps550client # [ 0.005534] CPU features: detected: RAS Extension Support551lb # [ 0.005623] CPU features: detected: Fine Grained Traps552worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)553client # [ 0.005537] CPU features: detected: RASv1p1 Extension Support554worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000555lb # [ 0.005628] CPU features: detected: Generic authentication (architected QARMA5 algorithm)556client # [ 0.005539] CPU features: detected: Random Number Generator557lb # [ 0.005633] CPU features: detected: RCpc load-acquire (LDAPR)558worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000559client # [ 0.005542] CPU features: detected: Speculation barrier (SB)560lb # [ 0.005636] CPU features: detected: LSE atomic instructions561worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.562client # [ 0.005545] CPU features: detected: Stage-2 Force Write-Back563lb # [ 0.005639] CPU features: detected: Privileged Access Never564client # [ 0.005548] CPU features: detected: TLB range maintenance instructions565lb # [ 0.005641] CPU features: detected: PMUv3566worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns567lb # [ 0.005644] CPU features: detected: RAS Extension Support568client # [ 0.005553] CPU features: detected: Speculative Store Bypassing Safe (SSBS)569worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).570lb # [ 0.005646] CPU features: detected: RASv1p1 Extension Support571client # [ 0.005595] alternatives: applying system-wide alternatives572lb # [ 0.005649] CPU features: detected: Random Number Generator573worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns574client # [ 0.008727] CPU features: detected: BBM Level 2 without TLB conflict abort575lb # [ 0.005652] CPU features: detected: Speculation barrier (SB)576worker2 # [ 0.172738] usbcore: registered new interface driver usbfs577worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns578lb # [ 0.005655] CPU features: detected: Stage-2 Force Write-Back579worker2 # [ 0.172771] usbcore: registered new interface driver hub580worker1 # [ 0.000033] arm-pv: using stolen time PV581client # [ 0.008898] Memory: 1406524K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)582lb # [ 0.005658] CPU features: detected: TLB range maintenance instructions583worker2 # [ 0.172791] usbcore: registered new device driver usb584client # [ 0.009368] devtmpfs: initialized585worker2 # [ 0.173101] pps_core: LinuxPPS API ver. 1 registered586worker1 # [ 0.000469] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)587lb # [ 0.005662] CPU features: detected: Speculative Store Bypassing Safe (SSBS)588client # [ 0.011378] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)589worker1 # [ 0.000648] Console: colour dummy device 80x25590lb # [ 0.005703] alternatives: applying system-wide alternatives591worker2 # [ 0.173108] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>592worker1 # [ 0.000656] printk: legacy console [tty0] enabled593client # [ 0.011403] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).594worker2 # [ 0.173119] PTP clock support registered595lb # [ 0.008921] CPU features: detected: BBM Level 2 without TLB conflict abort596worker2 # [ 0.173188] EDAC MC: Ver: 3.0.0597client # [ 0.011609] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL598worker1 # [ 0.000852] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)599client # [ 0.011616] 0 pages in range for non-PLT usage600worker2 # [ 0.177750] scmi_core: SCMI protocol bus registered601worker1 # [ 0.000859] pid_max: default: 32768 minimum: 301602client # [ 0.011617] 508288 pages in range for PLT usage603lb # [ 0.009107] Memory: 1406532K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)604worker1 # [ 0.000928] LSM: initializing lsm=capability,landlock,yama,bpf,ima605lb # [ 0.009644] devtmpfs: initialized606client # [ 0.012541] pinctrl core: initialized pinctrl subsystem607worker1 # [ 0.001103] landlock: Up and running.608client # [ 0.013383] DMI not present or invalid.609worker1 # [ 0.001106] Yama: becoming mindful.610worker2 # [ 0.179634] FPGA manager framework611worker1 # [ 0.001626] LSM support for eBPF active612client # [ 0.016956] NET: Registered PF_NETLINK/PF_ROUTE protocol family613lb # [ 0.011687] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)614client # [ 0.019195] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations615worker1 # [ 0.001784] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)616worker2 # [ 0.180658] vgaarb: loaded617lb # [ 0.011711] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).618client # [ 0.019394] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations619worker1 # [ 0.001818] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)620lb # [ 0.011905] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL621worker1 # [ 0.002981] cacheinfo: Unable to detect cache hierarchy for CPU 0622worker1 # [ 0.003780] rcu: Hierarchical SRCU implementation.623worker2 # [ 0.182495] clocksource: Switched to clocksource arch_sys_counter624lb # [ 0.011910] 0 pages in range for non-PLT usage625client # [ 0.019598] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations626lb # [ 0.011912] 508288 pages in range for PLT usage627worker2 # [ 0.183873] VFS: Disk quotas dquot_6.6.0628worker1 # [ 0.003785] rcu: Max phase no-delay instances is 1000.629client # [ 0.019624] audit: initializing netlink subsys (disabled)630client # [ 0.020255] thermal_sys: Registered thermal governor 'fair_share'631lb # [ 0.012039] pinctrl core: initialized pinctrl subsystem632worker1 # [ 0.005080] fsl-mc MSI: its@8080000 domain created633worker2 # [ 0.183912] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)634lb # [ 0.012877] DMI not present or invalid.635client # [ 0.020258] thermal_sys: Registered thermal governor 'bang_bang'636worker1 # [ 0.005176] EFI services will not be available.637lb # [ 0.016285] NET: Registered PF_NETLINK/PF_ROUTE protocol family638worker2 # [ 0.185495] netfs: FS-Cache loaded639client # [ 0.020262] thermal_sys: Registered thermal governor 'step_wise'640worker1 # [ 0.005257] smp: Bringing up secondary CPUs ...641worker2 # [ 0.185630] pnp: PnP ACPI: disabled642worker1 # [ 0.005267] smp: Brought up 1 node, 1 CPU643lb # [ 0.018673] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations644client # [ 0.020267] thermal_sys: Registered thermal governor 'user_space'645worker1 # [ 0.005270] SMP: Total of 1 processors activated.646worker1 # [ 0.005275] CPU: All CPU(s) started at EL1647client # [ 0.020270] thermal_sys: Registered thermal governor 'power_allocator'648lb # [ 0.018882] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations649worker1 # [ 0.005286] CPU features: detected: Branch Target Identification650client # [ 0.020298] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1651lb # [ 0.019094] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations652worker1 # [ 0.005291] CPU features: detected: ARMv8.4 Translation Table Level653client # [ 0.020309] cpuidle: using governor ladder654lb # [ 0.019124] audit: initializing netlink subsys (disabled)655client # [ 0.020314] cpuidle: using governor menu656lb # [ 0.019767] thermal_sys: Registered thermal governor 'fair_share'657worker1 # [ 0.005294] CPU features: detected: Instruction cache invalidation not required for I/D coherence658client # [ 0.020508] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.659lb # [ 0.019769] thermal_sys: Registered thermal governor 'bang_bang'660client # [ 0.020524] ASID allocator initialised with 65536 entries661worker1 # [ 0.005298] CPU features: detected: Data cache clean to the PoU not required for I/D coherence662lb # [ 0.019773] thermal_sys: Registered thermal governor 'step_wise'663client # [ 0.021830] Serial: AMBA PL011 UART driver664worker1 # [ 0.005302] CPU features: detected: Common not Private translations665lb # [ 0.019778] thermal_sys: Registered thermal governor 'user_space'666worker1 # [ 0.005305] CPU features: detected: CRC32 instructions667client # [ 0.027381] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1668lb # [ 0.019781] thermal_sys: Registered thermal governor 'power_allocator'669client # [ 0.027526] printk: console [ttyAMA0] enabled670worker1 # [ 0.005308] CPU features: detected: Data cache clean to Point of Deep Persistence671lb # [ 0.019811] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1672worker1 # [ 0.005311] CPU features: detected: Data cache clean to Point of Persistence673lb # [ 0.019820] cpuidle: using governor ladder674worker1 # [ 0.005314] CPU features: detected: Data independent timing control (DIT)675lb # [ 0.019826] cpuidle: using governor menu676worker2 # [ 0.191852] NET: Registered PF_INET protocol family677worker1 # [ 0.005318] CPU features: detected: E0PD678lb # [ 0.020022] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.679worker2 # [ 0.192159] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)680worker1 # [ 0.005320] CPU features: detected: Enhanced Counter Virtualization681lb # [ 0.020038] ASID allocator initialised with 65536 entries682lb # [ 0.021386] Serial: AMBA PL011 UART driver683worker1 # [ 0.005323] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)684worker1 # [ 0.005326] CPU features: detected: Enhanced Virtualization Traps685lb # [ 0.026958] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1686worker1 # [ 0.005330] CPU features: detected: Fine Grained Traps687lb # [ 0.027134] printk: console [ttyAMA0] enabled688worker1 # [ 0.005333] CPU features: detected: Generic authentication (architected QARMA5 algorithm)689worker1 # [ 0.005338] CPU features: detected: RCpc load-acquire (LDAPR)690worker1 # [ 0.005342] CPU features: detected: LSE atomic instructions691worker1 # [ 0.005345] CPU features: detected: Privileged Access Never692worker1 # [ 0.005348] CPU features: detected: PMUv3693client # [ 0.150235] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages694worker1 # [ 0.005350] CPU features: detected: RAS Extension Support695client # [ 0.150256] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page696worker1 # [ 0.005353] CPU features: detected: RASv1p1 Extension Support697worker1 # [ 0.005356] CPU features: detected: Random Number Generator698client # [ 0.150261] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages699worker1 # [ 0.005358] CPU features: detected: Speculation barrier (SB)700client # [ 0.150266] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page701worker1 # [ 0.005361] CPU features: detected: Stage-2 Force Write-Back702client # [ 0.150270] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages703worker1 # [ 0.005365] CPU features: detected: TLB range maintenance instructions704client # [ 0.150275] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page705worker1 # [ 0.005369] CPU features: detected: Speculative Store Bypassing Safe (SSBS)706client # [ 0.150279] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages707lb # [ 0.150938] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages708worker1 # [ 0.005407] alternatives: applying system-wide alternatives709client # [ 0.150283] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page710lb # [ 0.150962] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page711worker1 # [ 0.008484] CPU features: detected: BBM Level 2 without TLB conflict abort712lb # [ 0.150968] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages713lb # [ 0.150972] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page714worker1 # [ 0.008749] Memory: 1406532K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)715client # [ 0.158025] fbcon: Taking over console716worker1 # [ 0.009231] devtmpfs: initialized717client # [ 0.158041] ACPI: Interpreter disabled.718lb # [ 0.150977] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages719worker1 # [ 0.011096] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)720lb # [ 0.150981] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page721client # [ 0.159963] iommu: Default domain type: Translated722lb # [ 0.150985] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages723client # [ 0.159974] iommu: DMA domain TLB invalidation policy: strict mode724worker1 # [ 0.011120] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).725lb # [ 0.150990] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page726worker1 # [ 0.011313] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL727client # [ 0.161688] SCSI subsystem initialized728worker1 # [ 0.011320] 0 pages in range for non-PLT usage729worker1 # [ 0.011321] 508288 pages in range for PLT usage730worker1 # [ 0.011436] pinctrl core: initialized pinctrl subsystem731lb # [ 0.159026] fbcon: Taking over console732worker1 # [ 0.012219] DMI not present or invalid.733lb # [ 0.159045] ACPI: Interpreter disabled.734worker1 # [ 0.015731] NET: Registered PF_NETLINK/PF_ROUTE protocol family735worker1 # [ 0.018041] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations736worker1 # [ 0.018229] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations737worker1 # [ 0.018457] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations738client # [ 0.166906] usbcore: registered new interface driver usbfs739worker1 # [ 0.018487] audit: initializing netlink subsys (disabled)740client # [ 0.166939] usbcore: registered new interface driver hub741worker1 # [ 0.019136] thermal_sys: Registered thermal governor 'fair_share'742client # [ 0.166966] usbcore: registered new device driver usb743worker1 # [ 0.019139] thermal_sys: Registered thermal governor 'bang_bang'744client # [ 0.167281] pps_core: LinuxPPS API ver. 1 registered745worker1 # [ 0.019143] thermal_sys: Registered thermal governor 'step_wise'746client # [ 0.167290] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>747worker1 # [ 0.019148] thermal_sys: Registered thermal governor 'user_space'748client # [ 0.167301] PTP clock support registered749client # [ 0.167353] EDAC MC: Ver: 3.0.0750worker1 # [ 0.019151] thermal_sys: Registered thermal governor 'power_allocator'751lb # [ 0.167351] iommu: Default domain type: Translated752client # [ 0.172288] scmi_core: SCMI protocol bus registered753worker1 # [ 0.019182] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1754lb # [ 0.167365] iommu: DMA domain TLB invalidation policy: strict mode755worker1 # [ 0.019190] cpuidle: using governor ladder756lb # [ 0.167821] SCSI subsystem initialized757client # [ 0.173285] FPGA manager framework758worker1 # [ 0.019196] cpuidle: using governor menu759client # [ 0.174263] vgaarb: loaded760worker1 # [ 0.019386] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.761lb # [ 0.169901] usbcore: registered new interface driver usbfs762worker1 # [ 0.019404] ASID allocator initialised with 65536 entries763client # [ 0.174919] clocksource: Switched to clocksource arch_sys_counter764worker1 # [ 0.020707] Serial: AMBA PL011 UART driver765lb # [ 0.169936] usbcore: registered new interface driver hub766lb # [ 0.169954] usbcore: registered new device driver usb767worker1 # [ 0.026143] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1768lb # [ 0.170274] pps_core: LinuxPPS API ver. 1 registered769worker1 # [ 0.026310] printk: console [ttyAMA0] enabled770lb # [ 0.170280] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>771lb # [ 0.170290] PTP clock support registered772lb # [ 0.170355] EDAC MC: Ver: 3.0.0773worker2 # [ 0.223827] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)774lb # [ 0.175338] scmi_core: SCMI protocol bus registered775worker2 # [ 0.223892] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)776lb # [ 0.176470] FPGA manager framework777client # [ 0.181554] VFS: Disk quotas dquot_6.6.0778worker2 # [ 0.223927] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)779lb # [ 0.177709] vgaarb: loaded780client # [ 0.181606] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)781worker2 # [ 0.224016] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)782lb # [ 0.178393] clocksource: Switched to clocksource arch_sys_counter783client # [ 0.183330] netfs: FS-Cache loaded784worker2 # [ 0.224161] TCP: Hash tables configured (established 16384 bind 16384)785client # [ 0.183439] pnp: PnP ACPI: disabled786worker2 # [ 0.224261] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)787worker2 # [ 0.224301] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)788worker2 # [ 0.224342] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)789lb # [ 0.181776] VFS: Disk quotas dquot_6.6.0790worker1 # [ 0.148854] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages791worker2 # [ 0.224451] NET: Registered PF_UNIX/PF_LOCAL protocol family792lb # [ 0.181827] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)793worker1 # [ 0.148880] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page794worker2 # [ 0.224480] NET: Registered PF_XDP protocol family795worker2 # [ 0.224498] PCI: CLS 0 bytes, default 64796worker1 # [ 0.148886] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages797worker2 # [ 0.225862] Trying to unpack rootfs image as initramfs...798worker1 # [ 0.148891] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page799worker1 # [ 0.148896] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages800lb # [ 0.185739] netfs: FS-Cache loaded801worker1 # [ 0.148900] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page802lb # [ 0.185891] pnp: PnP ACPI: disabled803client # [ 0.189472] NET: Registered PF_INET protocol family804worker1 # [ 0.148905] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages805worker2 # [ 0.240535] kvm [1]: HYP mode not available806worker1 # [ 0.148910] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page807client # [ 0.189775] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)808worker1 # [ 0.156770] fbcon: Taking over console809lb # [ 0.190367] NET: Registered PF_INET protocol family810worker1 # [ 0.156792] ACPI: Interpreter disabled.811lb # [ 0.191306] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)812worker1 # [ 0.164457] iommu: Default domain type: Translated813worker1 # [ 0.164470] iommu: DMA domain TLB invalidation policy: strict mode814worker1 # [ 0.164972] SCSI subsystem initialized815worker1 # [ 0.167065] usbcore: registered new interface driver usbfs816worker1 # [ 0.167100] usbcore: registered new interface driver hub817worker1 # [ 0.167126] usbcore: registered new device driver usb818worker1 # [ 0.167444] pps_core: LinuxPPS API ver. 1 registered819worker1 # [ 0.167454] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>820worker1 # [ 0.167465] PTP clock support registered821worker1 # [ 0.167517] EDAC MC: Ver: 3.0.0822worker1 # [ 0.172302] scmi_core: SCMI protocol bus registered823worker1 # [ 0.173353] FPGA manager framework824worker1 # [ 0.174332] vgaarb: loaded825worker1 # [ 0.175050] clocksource: Switched to clocksource arch_sys_counter826worker1 # [ 0.180419] VFS: Disk quotas dquot_6.6.0827worker1 # [ 0.180464] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)828worker1 # [ 0.184324] netfs: FS-Cache loaded829worker1 # [ 0.184456] pnp: PnP ACPI: disabled830worker1 # [ 0.188544] NET: Registered PF_INET protocol family831worker1 # [ 0.188873] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)832client # [ 0.221998] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833client # [ 0.222074] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834client # [ 0.222110] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)835client # [ 0.222200] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)836client # [ 0.222346] TCP: Hash tables configured (established 16384 bind 16384)837lb # [ 0.222385] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)838client # [ 0.222462] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)839lb # [ 0.222617] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)840client # [ 0.222507] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)841lb # [ 0.222652] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)842client # [ 0.222546] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)843lb # [ 0.222743] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)844client # [ 0.222647] NET: Registered PF_UNIX/PF_LOCAL protocol family845lb # [ 0.222888] TCP: Hash tables configured (established 16384 bind 16384)846client # [ 0.222682] NET: Registered PF_XDP protocol family847client # [ 0.222703] PCI: CLS 0 bytes, default 64848lb # [ 0.222979] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)849client # [ 0.223023] Trying to unpack rootfs image as initramfs...850lb # [ 0.223016] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)851lb # [ 0.223054] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)852lb # [ 0.223161] NET: Registered PF_UNIX/PF_LOCAL protocol family853lb # [ 0.223192] NET: Registered PF_XDP protocol family854lb # [ 0.223213] PCI: CLS 0 bytes, default 64855lb # [ 0.223473] Trying to unpack rootfs image as initramfs...856client # [ 0.241138] kvm [1]: HYP mode not available857lb # [ 0.246437] kvm [1]: HYP mode not available858worker1 # [ 0.219277] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)859worker1 # [ 0.219348] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)860worker1 # [ 0.219384] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)861worker1 # [ 0.219477] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)862worker1 # [ 0.219623] TCP: Hash tables configured (established 16384 bind 16384)863worker1 # [ 0.219721] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)864worker1 # [ 0.219761] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865worker1 # [ 0.219797] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866worker1 # [ 0.219914] NET: Registered PF_UNIX/PF_LOCAL protocol family867worker1 # [ 0.219982] NET: Registered PF_XDP protocol family868worker1 # [ 0.220002] PCI: CLS 0 bytes, default 64869worker1 # [ 0.220245] Trying to unpack rootfs image as initramfs...870worker1 # [ 0.237089] kvm [1]: HYP mode not available871worker2 # [ 0.363855] Initialise system trusted keyrings872worker2 # [ 0.364669] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker2 # [ 0.366082] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker2 # [ 0.366967] 9p: Installing v9fs 9p2000 file system support875worker2 # [ 0.387063] Key type asymmetric registered876worker2 # [ 0.387098] Asymmetric key parser 'x509' registered877worker2 # [ 0.387190] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)878worker2 # [ 0.394566] io scheduler mq-deadline registered879worker2 # [ 0.394593] io scheduler kyber registered880worker2 # [ 0.406726] pl061_gpio 9030000.pl061: PL061 GPIO chip registered881worker2 # [ 0.409672] ledtrig-cpu: registered to indicate activity on CPUs882worker2 # [ 0.410196] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:883worker2 # [ 0.410217] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000884worker2 # [ 0.410229] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000885worker2 # [ 0.410238] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000886client # [ 0.363582] Initialise system trusted keyrings887worker2 # [ 0.410289] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits888client # [ 0.364434] workingset: timestamp_bits=42 max_order=19 bucket_order=0889worker2 # [ 0.410318] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]890client # [ 0.365830] squashfs: version 4.0 (2009/01/31) Phillip Lougher891lb # [ 0.361922] Initialise system trusted keyrings892worker2 # [ 0.410433] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00893client # [ 0.366679] 9p: Installing v9fs 9p2000 file system support894worker2 # [ 0.410450] pci_bus 0000:00: root bus resource [bus 00-ff]895lb # [ 0.362783] workingset: timestamp_bits=42 max_order=19 bucket_order=0896worker2 # [ 0.410457] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]897lb # [ 0.364110] squashfs: version 4.0 (2009/01/31) Phillip Lougher898worker2 # [ 0.410462] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]899worker2 # [ 0.410467] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]900lb # [ 0.364933] 9p: Installing v9fs 9p2000 file system support901worker2 # [ 0.421909] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint902worker2 # [ 0.422387] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint903worker2 # [ 0.422591] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]904worker2 # [ 0.422608] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]905worker2 # [ 0.422638] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]906worker2 # [ 0.422654] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]907worker2 # [ 0.423108] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint908worker2 # [ 0.423290] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]909worker2 # [ 0.423306] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]910worker2 # [ 0.423338] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]911worker2 # [ 0.423797] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint912worker2 # [ 0.423982] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]913worker2 # [ 0.423999] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]914worker2 # [ 0.424028] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]915worker2 # [ 0.424492] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint916worker2 # [ 0.424673] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]917worker2 # [ 0.424689] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]918worker2 # [ 0.424720] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]919lb # [ 0.389031] Key type asymmetric registered920lb # [ 0.389066] Asymmetric key parser 'x509' registered921worker2 # [ 0.425177] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint922worker2 # [ 0.425376] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]923lb # [ 0.389163] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)924worker2 # [ 0.425393] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]925client # [ 0.395543] Key type asymmetric registered926client # [ 0.395577] Asymmetric key parser 'x509' registered927worker2 # [ 0.425424] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]928worker1 # [ 0.359709] Initialise system trusted keyrings929client # [ 0.395673] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)930worker2 # [ 0.425879] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint931worker1 # [ 0.360506] workingset: timestamp_bits=42 max_order=19 bucket_order=0932client # [ 0.397881] io scheduler mq-deadline registered933worker2 # [ 0.426066] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]934client # [ 0.397894] io scheduler kyber registered935worker2 # [ 0.426084] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]936lb # [ 0.394472] io scheduler mq-deadline registered937worker1 # [ 0.361945] squashfs: version 4.0 (2009/01/31) Phillip Lougher938lb # [ 0.394495] io scheduler kyber registered939worker2 # [ 0.426112] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]940worker1 # [ 0.362726] 9p: Installing v9fs 9p2000 file system support941worker2 # [ 0.426585] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint942worker2 # [ 0.426766] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]943worker2 # [ 0.426782] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]944worker2 # [ 0.426810] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]945worker2 # [ 0.426826] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]946worker2 # [ 0.427316] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint947worker2 # [ 0.427505] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]948worker2 # [ 0.427535] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]949worker2 # [ 0.427988] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint950worker2 # [ 0.428172] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]951worker2 # [ 0.428202] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]952worker2 # [ 0.428601] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint953worker2 # [ 0.428778] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]954worker2 # [ 0.429022] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint955lb # [ 0.406583] pl061_gpio 9030000.pl061: PL061 GPIO chip registered956worker2 # [ 0.429336] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]957worker2 # [ 0.429358] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]958worker2 # [ 0.429387] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]959worker2 # [ 0.429840] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint960worker2 # [ 0.430024] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]961worker2 # [ 0.430041] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]962worker2 # [ 0.430071] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]963lb # [ 0.408214] ledtrig-cpu: registered to indicate activity on CPUs964client # [ 0.411524] pl061_gpio 9030000.pl061: PL061 GPIO chip registered965client # [ 0.412244] ledtrig-cpu: registered to indicate activity on CPUs966lb # [ 0.408653] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:967client # [ 0.412651] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:968lb # [ 0.408673] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000969client # [ 0.412671] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000970lb # [ 0.408686] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000971client # [ 0.412686] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000972lb # [ 0.408695] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000973client # [ 0.412695] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000974lb # [ 0.408718] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits975client # [ 0.412719] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits976lb # [ 0.408744] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]977lb # [ 0.408828] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00978client # [ 0.412744] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]979lb # [ 0.408848] pci_bus 0000:00: root bus resource [bus 00-ff]980client # [ 0.412839] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00981lb # [ 0.408855] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]982client # [ 0.412850] pci_bus 0000:00: root bus resource [bus 00-ff]983lb # [ 0.408861] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]984client # [ 0.412857] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]985worker1 # [ 0.391554] Key type asymmetric registered986worker1 # [ 0.391592] Asymmetric key parser 'x509' registered987lb # [ 0.408866] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]988client # [ 0.412862] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]989worker1 # [ 0.391695] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)990client # [ 0.412867] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]991lb # [ 0.408932] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint992worker1 # [ 0.393894] io scheduler mq-deadline registered993client # [ 0.412943] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint994lb # [ 0.409407] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint995worker1 # [ 0.393909] io scheduler kyber registered996lb # [ 0.409621] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]997client # [ 0.414267] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint998lb # [ 0.409640] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]999client # [ 0.414482] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1000lb # [ 0.409671] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1001client # [ 0.414499] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1002lb # [ 0.409688] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1003client # [ 0.414535] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1004client # [ 0.414552] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1005lb # [ 0.410171] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1006lb # [ 0.410365] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1007client # [ 0.415104] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1008worker2 # [ 0.482790] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1009lb # [ 0.410382] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1010client # [ 0.415298] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1011worker2 # [ 0.482828] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1012lb # [ 0.410447] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1013client # [ 0.415315] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1014client # [ 0.415347] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1015lb # [ 0.410930] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1016worker2 # [ 0.482834] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1017lb # [ 0.411119] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1018client # [ 0.416916] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1019worker2 # [ 0.482891] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1020lb # [ 0.411135] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1021client # [ 0.417116] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1022worker2 # [ 0.482942] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1023lb # [ 0.411166] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1024client # [ 0.417133] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1025worker2 # [ 0.482991] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1026client # [ 0.417164] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1027lb # [ 0.411642] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1028lb # [ 0.411837] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1029worker2 # [ 0.483039] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1030client # [ 0.417648] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1031lb # [ 0.411853] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1032client # [ 0.417845] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1033worker2 # [ 0.483086] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1034lb # [ 0.411888] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035client # [ 0.417861] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1036worker2 # [ 0.483135] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1037lb # [ 0.412357] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1038client # [ 0.417895] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1039lb # [ 0.412543] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1040worker2 # [ 0.483183] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1041client # [ 0.418386] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1042lb # [ 0.412559] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1043client # [ 0.418579] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1044worker2 # [ 0.483231] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1045lb # [ 0.412589] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046client # [ 0.418595] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1047worker2 # [ 0.483280] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1048lb # [ 0.413047] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1049client # [ 0.418626] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1050worker2 # [ 0.483406] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1051lb # [ 0.413232] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1052worker2 # [ 0.483460] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1053lb # [ 0.413248] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1054worker2 # [ 0.483483] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1055lb # [ 0.413277] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1056worker2 # [ 0.483504] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1057lb # [ 0.413767] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1058worker2 # [ 0.483526] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1059lb # [ 0.413949] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1060worker1 # [ 0.407501] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1061worker2 # [ 0.483555] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1062lb # [ 0.413965] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1063worker1 # [ 0.408245] ledtrig-cpu: registered to indicate activity on CPUs1064lb # [ 0.413994] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065worker2 # [ 0.483578] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1066worker1 # [ 0.408666] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1067lb # [ 0.414010] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1068worker2 # [ 0.483600] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1069worker1 # [ 0.408692] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001070worker2 # [ 0.483622] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1071worker2 # [ 0.483643] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1072worker1 # [ 0.408705] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001073worker2 # [ 0.483666] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1074worker1 # [ 0.408713] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001075worker2 # [ 0.483692] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1076worker1 # [ 0.408735] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1077worker2 # [ 0.483716] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1078worker2 # [ 0.483737] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1079client # [ 0.456141] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1080worker1 # [ 0.408759] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1081client # [ 0.456351] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1082worker2 # [ 0.483758] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1083worker1 # [ 0.408839] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001084client # [ 0.456369] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1085worker2 # [ 0.483780] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1086worker1 # [ 0.408850] pci_bus 0000:00: root bus resource [bus 00-ff]1087worker2 # [ 0.483802] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1088client # [ 0.456399] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089worker1 # [ 0.408856] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1090worker2 # [ 0.483823] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1091client # [ 0.456887] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1092worker1 # [ 0.408862] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1093worker2 # [ 0.483847] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1094lb # [ 0.454527] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1095client # [ 0.457079] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1096worker1 # [ 0.408867] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1097worker2 # [ 0.483869] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1098lb # [ 0.454747] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1099client # [ 0.457095] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1100worker1 # [ 0.408981] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1101worker2 # [ 0.483891] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1102client # [ 0.457126] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1103lb # [ 0.454778] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1104worker2 # [ 0.483912] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1105worker1 # [ 0.409471] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1106client # [ 0.457143] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1107lb # [ 0.455315] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1108worker2 # [ 0.483942] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1109worker1 # [ 0.409664] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1110lb # [ 0.455505] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1111client # [ 0.457674] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1112worker2 # [ 0.483952] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1113worker1 # [ 0.409683] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1114client # [ 0.457879] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1115lb # [ 0.455536] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1116worker2 # [ 0.483957] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1117worker1 # [ 0.409720] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118client # [ 0.457914] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1119worker2 # [ 0.485171] pci 0000:00:0a.0: enabling device (0000 -> 0002)1120worker1 # [ 0.409737] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1121lb # [ 0.455933] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1122client # [ 0.458419] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1123lb # [ 0.456118] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1124worker1 # [ 0.410732] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1125client # [ 0.458622] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1126worker1 # [ 0.410925] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1127lb # [ 0.456376] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1128client # [ 0.458654] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129worker1 # [ 0.410943] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1130lb # [ 0.456657] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1131lb # [ 0.456674] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1132worker1 # [ 0.410975] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133lb # [ 0.456704] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1134worker1 # [ 0.411490] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1135worker1 # [ 0.411678] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1136lb # [ 0.457186] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1137worker1 # [ 0.411695] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1138lb # [ 0.457372] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1139lb # [ 0.457387] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1140worker1 # [ 0.411727] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1141client # [ 0.471218] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1142lb # [ 0.457418] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1143worker1 # [ 0.412207] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1144client # [ 0.471415] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1145lb # [ 0.458027] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1146worker1 # [ 0.412394] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1147worker1 # [ 0.412412] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1148client # [ 0.471686] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1149lb # [ 0.458039] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1150client # [ 0.471981] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1151worker1 # [ 0.412449] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1152lb # [ 0.458045] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1153client # [ 0.472001] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1154worker1 # [ 0.412944] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1155lb # [ 0.458090] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1156client # [ 0.472032] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1157worker1 # [ 0.413151] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1158lb # [ 0.458137] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1159worker1 # [ 0.413170] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1160client # [ 0.472524] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1161client # [ 0.472714] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1162worker1 # [ 0.413201] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1163lb # [ 0.458183] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1164client # [ 0.472731] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1165worker1 # [ 0.413663] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1166lb # [ 0.458229] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1167client # [ 0.472763] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1168worker1 # [ 0.413849] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1169lb # [ 0.458275] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1170client # [ 0.473368] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1171worker1 # [ 0.413866] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1172lb # [ 0.458323] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1173client # [ 0.473381] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1174worker1 # [ 0.413898] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1175lb # [ 0.458371] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1176client # [ 0.473387] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1177worker1 # [ 0.414370] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1178worker1 # [ 0.414558] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1179client # [ 0.473435] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1180worker1 # [ 0.414574] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1181client # [ 0.473485] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1182worker1 # [ 0.414604] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1183client # [ 0.473536] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1184worker1 # [ 0.414621] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1185client # [ 0.473583] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1186client # [ 0.473633] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1187client # [ 0.473682] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1188worker2 # [ 0.540172] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1189client # [ 0.473731] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1190client # [ 0.473779] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1191client # [ 0.473834] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1192lb # [ 0.486474] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1193client # [ 0.473909] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1194client # [ 0.473959] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1195lb # [ 0.486540] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1196worker1 # [ 0.455165] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1197client # [ 0.474008] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1198worker1 # [ 0.455371] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1199lb # [ 0.486663] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1200client # [ 0.474031] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1201worker1 # [ 0.455405] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1202lb # [ 0.486714] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1203client # [ 0.474057] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1204lb # [ 0.486738] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1205worker1 # [ 0.455897] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1206client # [ 0.474084] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1207worker1 # [ 0.456090] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1208lb # [ 0.486761] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1209client # [ 0.474109] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1210lb # [ 0.486785] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1211worker1 # [ 0.456121] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1212client # [ 0.474134] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1213lb # [ 0.486813] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1214worker1 # [ 0.456531] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1215client # [ 0.474157] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1216lb # [ 0.486838] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1217worker1 # [ 0.456721] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1218client # [ 0.474180] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1219lb # [ 0.486862] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1220worker1 # [ 0.456981] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1221client # [ 0.474204] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1222worker2 # [ 0.550618] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1223worker1 # [ 0.457291] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1224lb # [ 0.486886] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1225client # [ 0.474230] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1226worker1 # [ 0.457314] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1227lb # [ 0.486909] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1228client # [ 0.474256] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1229worker1 # [ 0.457346] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1230lb # [ 0.486932] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1231client # [ 0.474279] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1232worker1 # [ 0.457833] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1233lb # [ 0.486957] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1234client # [ 0.474301] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1235worker1 # [ 0.458021] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1236client # [ 0.474324] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1237lb # [ 0.486982] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1238worker1 # [ 0.458038] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1239client # [ 0.474348] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1240lb # [ 0.487004] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1241worker1 # [ 0.458069] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1242client # [ 0.474371] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1243lb # [ 0.487027] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1244worker1 # [ 0.458662] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1245client # [ 0.474395] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1246lb # [ 0.487049] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1247worker1 # [ 0.458676] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1248client # [ 0.474418] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1249lb # [ 0.487071] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1250client # [ 0.474444] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1251lb # [ 0.487093] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1252worker1 # [ 0.458682] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1253client # [ 0.474467] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1254lb # [ 0.487116] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1255worker2 # [ 0.558777] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1256worker1 # [ 0.458731] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1257client # [ 0.474494] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1258lb # [ 0.487138] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1259worker2 # [ 0.561008] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1260client # [ 0.474504] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1261worker1 # [ 0.458781] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1262lb # [ 0.487161] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1263client # [ 0.474509] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1264lb # [ 0.487184] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1265worker1 # [ 0.458831] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1266lb # [ 0.487216] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1267worker1 # [ 0.458880] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1268lb # [ 0.487226] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1269worker1 # [ 0.458930] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1270lb # [ 0.487231] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1271lb # [ 0.488465] pci 0000:00:0a.0: enabling device (0000 -> 0002)1272worker1 # [ 0.458979] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1273worker1 # [ 0.459027] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1274client # [ 0.519840] pci 0000:00:0a.0: enabling device (0000 -> 0002)1275worker2 # [ 0.571235] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1276worker2 # [ 0.573724] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1277worker2 # [ 0.577627] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1278worker1 # [ 0.487174] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1279worker1 # [ 0.487247] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1280worker1 # [ 0.487969] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1281worker1 # [ 0.488023] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1282worker1 # [ 0.488047] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1283worker1 # [ 0.488071] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1284worker1 # [ 0.488094] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1285worker1 # [ 0.488122] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1286worker1 # [ 0.488146] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1287worker2 # [ 0.586360] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1288worker1 # [ 0.488170] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1289worker1 # [ 0.488194] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1290worker1 # [ 0.488217] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1291worker1 # [ 0.488241] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1292worker1 # [ 0.488265] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1293worker1 # [ 0.488289] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1294worker1 # [ 0.488312] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1295worker1 # [ 0.488335] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1296worker1 # [ 0.488359] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1297worker1 # [ 0.488385] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1298worker1 # [ 0.488413] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1299worker1 # [ 0.488436] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1300worker1 # [ 0.488465] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1301worker1 # [ 0.488488] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1302lb # [ 0.538868] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1303worker1 # [ 0.488511] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1304worker2 # [ 0.596326] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1305client # [ 0.546069] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1306lb # [ 0.541223] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1307worker1 # [ 0.488543] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1308worker1 # [ 0.488554] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1309worker1 # [ 0.488559] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1310worker1 # [ 0.489743] pci 0000:00:0a.0: enabling device (0000 -> 0002)1311worker2 # [ 0.603223] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1312worker2 # [ 0.605915] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1313client # [ 0.556822] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1314lb # [ 0.551988] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1315lb # [ 0.554149] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1316client # [ 0.560017] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1317client # [ 0.562280] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1318lb # [ 0.564538] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1319worker2 # [ 0.623935] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1320client # [ 0.572489] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1321client # [ 0.574916] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1322worker1 # [ 0.540270] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1323worker1 # [ 0.542870] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1324worker2 # [ 0.631958] msm_serial: driver initialized1325worker2 # [ 0.632178] SuperH (H)SCI(F) driver initialized1326worker2 # [ 0.632240] STM32 USART driver initialized1327lb # [ 0.574669] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1328lb # [ 0.577335] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1329lb # [ 0.579789] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1330client # [ 0.587347] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1331lb # [ 0.582243] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1332client # [ 0.589591] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1333lb # [ 0.591989] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1334worker1 # [ 0.560592] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1335worker1 # [ 0.562693] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1336client # [ 0.600199] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1337client # [ 0.602375] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1338worker1 # [ 0.566518] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1339worker1 # [ 0.569809] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1340client # [ 0.614272] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1341lb # [ 0.606794] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1342worker2 # [ 0.666811] loop: module loaded1343worker2 # [ 0.667031] virtio_blk virtio5: 1/0/0 default/read/poll queues1344worker2 # [ 0.667866] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1345lb # [ 0.614334] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1346worker1 # [ 0.580459] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1347worker1 # [ 0.582782] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1348lb # [ 0.619888] msm_serial: driver initialized1349lb # [ 0.620148] SuperH (H)SCI(F) driver initialized1350lb # [ 0.620219] STM32 USART driver initialized1351worker2 # [ 0.675302] megasas: 07.734.00.00-rc11352worker1 # [ 0.594384] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1353worker2 # [ 0.676346] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1354worker2 # [ 0.678846] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001355worker2 # [ 0.678875] Intel/Sharp Extended Query Table at 0x00311356worker2 # [ 0.680610] Using buffer write method1357worker1 # [ 0.597333] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1358worker2 # [ 0.680706] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1359client # [ 0.632467] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1360worker2 # [ 0.688344] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001361client # [ 0.634394] msm_serial: driver initialized1362worker2 # [ 0.688378] Intel/Sharp Extended Query Table at 0x00311363client # [ 0.634632] SuperH (H)SCI(F) driver initialized1364worker2 # [ 0.689992] Using buffer write method1365client # [ 0.634691] STM32 USART driver initialized1366worker2 # [ 0.690032] Concatenating MTD devices:1367worker2 # [ 0.690036] (0): "0.flash"1368worker2 # [ 0.690040] (1): "0.flash"1369worker2 # [ 0.690044] into device "0.flash"1370worker1 # [ 0.607772] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1371worker1 # [ 0.619629] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1372worker1 # [ 0.622479] msm_serial: driver initialized1373worker1 # [ 0.622641] SuperH (H)SCI(F) driver initialized1374worker1 # [ 0.622696] STM32 USART driver initialized1375lb # [ 0.663741] loop: module loaded1376client # [ 0.670317] loop: module loaded1377lb # [ 0.664000] virtio_blk virtio5: 1/0/0 default/read/poll queues1378lb # [ 0.664944] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1379client # [ 0.671431] virtio_blk virtio5: 1/0/0 default/read/poll queues1380client # [ 0.672360] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1381lb # [ 0.671405] megasas: 07.734.00.00-rc11382lb # [ 0.672472] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1383lb # [ 0.683299] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001384lb # [ 0.683365] Intel/Sharp Extended Query Table at 0x00311385worker1 # [ 0.654197] loop: module loaded1386lb # [ 0.685155] Using buffer write method1387worker1 # [ 0.654419] virtio_blk virtio5: 1/0/0 default/read/poll queues1388lb # [ 0.685303] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1389worker1 # [ 0.655307] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1390lb # [ 0.691263] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001391lb # [ 0.691317] Intel/Sharp Extended Query Table at 0x00311392lb # [ 0.693174] Using buffer write method1393lb # [ 0.693225] Concatenating MTD devices:1394lb # [ 0.693229] (0): "0.flash"1395lb # [ 0.693234] (1): "0.flash"1396lb # [ 0.693237] into device "0.flash"1397client # [ 0.684306] megasas: 07.734.00.00-rc11398client # [ 0.685213] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399worker1 # [ 0.667813] megasas: 07.734.00.00-rc11400worker1 # [ 0.668604] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1401worker1 # [ 0.671295] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402worker1 # [ 0.671322] Intel/Sharp Extended Query Table at 0x00311403worker1 # [ 0.672982] Using buffer write method1404worker1 # [ 0.673053] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1405worker1 # [ 0.674926] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406worker1 # [ 0.675001] Intel/Sharp Extended Query Table at 0x00311407client # [ 0.720214] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408client # [ 0.720273] Intel/Sharp Extended Query Table at 0x00311409client # [ 0.721997] Using buffer write method1410client # [ 0.722102] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1411worker1 # [ 0.693365] Using buffer write method1412worker1 # [ 0.693409] Concatenating MTD devices:1413worker1 # [ 0.693413] (0): "0.flash"1414worker1 # [ 0.693418] (1): "0.flash"1415worker1 # [ 0.693421] into device "0.flash"1416client # [ 0.735788] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417client # [ 0.735848] Intel/Sharp Extended Query Table at 0x00311418client # [ 0.737722] Using buffer write method1419client # [ 0.737766] Concatenating MTD devices:1420client # [ 0.737770] (0): "0.flash"1421client # [ 0.737775] (1): "0.flash"1422client # [ 0.737778] into device "0.flash"1423worker2 # [ 0.995649] Freeing initrd memory: 26148K1424worker2 # [ 1.002355] tun: Universal TUN/TAP device driver, 1.61425worker2 # [ 1.006450] thunder_xcv, ver 1.01426worker2 # [ 1.007424] thunder_bgx, ver 1.01427worker2 # [ 1.007473] nicpf, ver 1.01428worker2 # [ 1.008115] e1000: Intel(R) PRO/1000 Network Driver1429worker2 # [ 1.008126] e1000: Copyright (c) 1999-2006 Intel Corporation.1430worker2 # [ 1.008150] e1000e: Intel(R) PRO/1000 Network Driver1431worker2 # [ 1.008171] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432worker2 # [ 1.008220] igb: Intel(R) Gigabit Ethernet Network Driver1433worker2 # [ 1.008229] igb: Copyright (c) 2007-2014 Intel Corporation.1434worker2 # [ 1.008255] igbvf: Intel(R) Gigabit Virtual Function Network Driver1435worker2 # [ 1.008261] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1436worker2 # [ 1.008408] sky2: driver version 1.301437worker2 # [ 1.010192] usbcore: registered new interface driver usb-storage1438worker2 # [ 1.010337] usbcore: registered new interface driver usbserial_generic1439worker2 # [ 1.010354] usbserial: USB Serial support registered for generic1440worker2 # [ 1.011325] ehci-pci 0000:00:0a.0: EHCI Host Controller1441worker2 # [ 1.011358] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11442worker2 # [ 1.011575] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001443worker2 # [ 1.023288] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001444worker2 # [ 1.023672] hub 1-0:1.0: USB hub found1445worker2 # [ 1.023744] hub 1-0:1.0: 6 ports detected1446worker2 # [ 1.026494] hv_vmbus: registering driver hyperv_keyboard1447worker2 # [ 1.028100] rtc-pl031 9010000.pl031: registered as rtc01448worker2 # [ 1.028132] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:47:15 UTC (1789908435)1449worker2 # [ 1.028499] i2c_dev: i2c /dev entries driver1450worker2 # [ 1.033695] sdhci: Secure Digital Host Controller Interface driver1451worker2 # [ 1.033710] sdhci: Copyright(c) Pierre Ossman1452worker2 # [ 1.034003] Synopsys Designware Multimedia Card Interface Driver1453worker2 # [ 1.034428] sdhci-pltfm: SDHCI platform and OF driver helper1454worker2 # [ 1.038972] hid: raw HID events driver (C) Jiri Kosina1455worker2 # [ 1.039277] usbcore: registered new interface driver usbhid1456worker2 # [ 1.039284] usbhid: USB HID core driver1457worker2 # [ 1.042229] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1458worker2 # [ 1.044996] drop_monitor: Initializing network drop monitor service1459worker2 # [ 1.045180] NET: Registered PF_INET6 protocol family1460worker2 # [ 1.047380] Segment Routing with IPv61461worker2 # [ 1.047404] In-situ OAM (IOAM) with IPv61462worker2 # [ 1.047465] NET: Registered PF_PACKET protocol family1463worker2 # [ 1.049162] 9pnet: Installing 9P2000 support1464worker2 # [ 1.052124] Key type dns_resolver registered1465worker2 # [ 1.058736] registered taskstats version 11466worker2 # [ 1.058903] Loading compiled-in X.509 certificates1467worker2 # [ 1.067959] Demotion targets for Node 0: null1468worker2 # [ 1.068093] Key type .fscrypt registered1469worker2 # [ 1.068099] Key type fscrypt-provisioning registered1470worker2 # [ 1.068204] ima: No TPM chip found, activating TPM-bypass!1471worker2 # [ 1.068228] ima: Allocated hash algorithm: sha11472worker2 # [ 1.068255] ima: No architecture policies found1473worker2 # [ 1.072554] input: gpio-keys as /devices/platform/gpio-keys/input/input01474worker1 # [ 0.989439] Freeing initrd memory: 26148K1475worker1 # [ 0.997853] tun: Universal TUN/TAP device driver, 1.61476lb # [ 1.033187] Freeing initrd memory: 26140K1477worker1 # [ 1.001994] thunder_xcv, ver 1.01478worker1 # [ 1.002046] thunder_bgx, ver 1.01479worker1 # [ 1.002070] nicpf, ver 1.01480worker1 # [ 1.002647] e1000: Intel(R) PRO/1000 Network Driver1481client # [ 1.040242] Freeing initrd memory: 26160K1482worker1 # [ 1.002654] e1000: Copyright (c) 1999-2006 Intel Corporation.1483worker1 # [ 1.002683] e1000e: Intel(R) PRO/1000 Network Driver1484worker1 # [ 1.002692] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1485worker1 # [ 1.002724] igb: Intel(R) Gigabit Ethernet Network Driver1486worker1 # [ 1.002730] igb: Copyright (c) 2007-2014 Intel Corporation.1487worker1 # [ 1.002755] igbvf: Intel(R) Gigabit Virtual Function Network Driver1488lb # [ 1.041463] tun: Universal TUN/TAP device driver, 1.61489worker1 # [ 1.002761] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1490worker1 # [ 1.002901] sky2: driver version 1.301491client # [ 1.046912] tun: Universal TUN/TAP device driver, 1.61492worker2 # [ 1.096167] clk: Disabling unused clocks1493worker2 # [ 1.096206] PM: genpd: Disabling unused power domains1494worker2 # [ 1.100793] Freeing unused kernel memory: 4736K1495worker2 # [ 1.101028] Run /init as init process1496worker1 # [ 1.012222] usbcore: registered new interface driver usb-storage1497worker1 # [ 1.012315] usbcore: registered new interface driver usbserial_generic1498worker1 # [ 1.012331] usbserial: USB Serial support registered for generic1499worker1 # [ 1.012928] hv_vmbus: registering driver hyperv_keyboard1500worker1 # [ 1.017364] ehci-pci 0000:00:0a.0: EHCI Host Controller1501lb # [ 1.047769] thunder_xcv, ver 1.01502lb # [ 1.047865] thunder_bgx, ver 1.01503worker1 # [ 1.017397] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11504lb # [ 1.047909] nicpf, ver 1.01505client # [ 1.051289] thunder_xcv, ver 1.01506client # [ 1.051351] thunder_bgx, ver 1.01507lb # [ 1.048722] e1000: Intel(R) PRO/1000 Network Driver1508worker1 # [ 1.017641] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001509client # [ 1.051374] nicpf, ver 1.01510lb # [ 1.048737] e1000: Copyright (c) 1999-2006 Intel Corporation.1511client # [ 1.051986] e1000: Intel(R) PRO/1000 Network Driver1512lb # [ 1.048776] e1000e: Intel(R) PRO/1000 Network Driver1513client # [ 1.051998] e1000: Copyright (c) 1999-2006 Intel Corporation.1514lb # [ 1.048785] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1515client # [ 1.052022] e1000e: Intel(R) PRO/1000 Network Driver1516lb # [ 1.048824] igb: Intel(R) Gigabit Ethernet Network Driver1517client # [ 1.052031] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1518worker1 # [ 1.020281] rtc-pl031 9010000.pl031: registered as rtc01519client # [ 1.052064] igb: Intel(R) Gigabit Ethernet Network Driver1520lb # [ 1.048830] igb: Copyright (c) 2007-2014 Intel Corporation.1521client # [ 1.052070] igb: Copyright (c) 2007-2014 Intel Corporation.1522worker1 # [ 1.020316] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:47:15 UTC (1789908435)1523lb # [ 1.048861] igbvf: Intel(R) Gigabit Virtual Function Network Driver1524worker1 # [ 1.020685] i2c_dev: i2c /dev entries driver1525client # [ 1.052092] igbvf: Intel(R) Gigabit Virtual Function Network Driver1526lb # [ 1.048868] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1527client # [ 1.052098] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1528lb # [ 1.049057] sky2: driver version 1.301529client # [ 1.052268] sky2: driver version 1.301530client # [ 1.053922] usbcore: registered new interface driver usb-storage1531client # [ 1.054039] usbcore: registered new interface driver usbserial_generic1532client # [ 1.054056] usbserial: USB Serial support registered for generic1533client # [ 1.054694] hv_vmbus: registering driver hyperv_keyboard1534worker2 # [ 1.116866] systemd[1]: Successfully made /usr/ read-only.1535client # [ 1.055680] ehci-pci 0000:00:0a.0: EHCI Host Controller1536client # [ 1.055714] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11537client # [ 1.055961] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001538lb # [ 1.060161] usbcore: registered new interface driver usb-storage1539worker1 # [ 1.027170] sdhci: Secure Digital Host Controller Interface driver1540worker1 # [ 1.027188] sdhci: Copyright(c) Pierre Ossman1541lb # [ 1.060360] usbcore: registered new interface driver usbserial_generic1542client # [ 1.068609] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001543worker1 # [ 1.027477] Synopsys Designware Multimedia Card Interface Driver1544lb # [ 1.060386] usbserial: USB Serial support registered for generic1545client # [ 1.069045] hub 1-0:1.0: USB hub found1546worker1 # [ 1.027851] sdhci-pltfm: SDHCI platform and OF driver helper1547client # [ 1.069066] hub 1-0:1.0: 6 ports detected1548lb # [ 1.061374] hv_vmbus: registering driver hyperv_keyboard1549worker1 # [ 1.029801] hid: raw HID events driver (C) Jiri Kosina1550lb # [ 1.063200] ehci-pci 0000:00:0a.0: EHCI Host Controller1551worker1 # [ 1.030080] usbcore: registered new interface driver usbhid1552worker1 # [ 1.030087] usbhid: USB HID core driver1553lb # [ 1.063250] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11554lb # [ 1.063469] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001555client # [ 1.072015] rtc-pl031 9010000.pl031: registered as rtc01556worker1 # [ 1.035952] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001557client # [ 1.072049] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:47:15 UTC (1789908435)1558worker1 # [ 1.036331] hub 1-0:1.0: USB hub found1559worker1 # [ 1.036367] hub 1-0:1.0: 6 ports detected1560client # [ 1.072353] i2c_dev: i2c /dev entries driver1561lb # [ 1.069763] rtc-pl031 9010000.pl031: registered as rtc01562worker1 # [ 1.038813] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1563lb # [ 1.069810] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:47:15 UTC (1789908435)1564lb # [ 1.070373] i2c_dev: i2c /dev entries driver1565client # [ 1.077608] sdhci: Secure Digital Host Controller Interface driver1566worker1 # [ 1.041718] drop_monitor: Initializing network drop monitor service1567client # [ 1.077628] sdhci: Copyright(c) Pierre Ossman1568worker1 # [ 1.041917] NET: Registered PF_INET6 protocol family1569client # [ 1.077892] Synopsys Designware Multimedia Card Interface Driver1570worker1 # [ 1.044201] Segment Routing with IPv61571client # [ 1.078275] sdhci-pltfm: SDHCI platform and OF driver helper1572worker1 # [ 1.044223] In-situ OAM (IOAM) with IPv61573worker1 # [ 1.044257] NET: Registered PF_PACKET protocol family1574lb # [ 1.077144] sdhci: Secure Digital Host Controller Interface driver1575lb # [ 1.077168] sdhci: Copyright(c) Pierre Ossman1576worker1 # [ 1.046002] 9pnet: Installing 9P2000 support1577client # [ 1.083035] hid: raw HID events driver (C) Jiri Kosina1578lb # [ 1.077602] Synopsys Designware Multimedia Card Interface Driver1579client # [ 1.083279] usbcore: registered new interface driver usbhid1580lb # [ 1.078061] sdhci-pltfm: SDHCI platform and OF driver helper1581client # [ 1.083288] usbhid: USB HID core driver1582lb # [ 1.081444] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001583lb # [ 1.081994] hub 1-0:1.0: USB hub found1584client # [ 1.086379] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1585worker1 # [ 1.048999] Key type dns_resolver registered1586lb # [ 1.083364] hub 1-0:1.0: 6 ports detected1587client # [ 1.089129] drop_monitor: Initializing network drop monitor service1588client # [ 1.089298] NET: Registered PF_INET6 protocol family1589client # [ 1.091927] Segment Routing with IPv61590client # [ 1.091951] In-situ OAM (IOAM) with IPv61591client # [ 1.091984] NET: Registered PF_PACKET protocol family1592lb # [ 1.087718] hid: raw HID events driver (C) Jiri Kosina1593lb # [ 1.088185] usbcore: registered new interface driver usbhid1594client # [ 1.093628] 9pnet: Installing 9P2000 support1595lb # [ 1.088199] usbhid: USB HID core driver1596worker1 # [ 1.057157] registered taskstats version 11597worker1 # [ 1.057338] Loading compiled-in X.509 certificates1598client # [ 1.096682] Key type dns_resolver registered1599lb # [ 1.091541] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1600lb # [ 1.093896] drop_monitor: Initializing network drop monitor service1601lb # [ 1.094110] NET: Registered PF_INET6 protocol family1602lb # [ 1.097683] Segment Routing with IPv61603lb # [ 1.097713] In-situ OAM (IOAM) with IPv61604lb # [ 1.097772] NET: Registered PF_PACKET protocol family1605worker1 # [ 1.066742] Demotion targets for Node 0: null1606worker1 # [ 1.067875] Key type .fscrypt registered1607worker1 # [ 1.067886] Key type fscrypt-provisioning registered1608worker1 # [ 1.068009] ima: No TPM chip found, activating TPM-bypass!1609lb # [ 1.099717] 9pnet: Installing 9P2000 support1610worker1 # [ 1.068034] ima: Allocated hash algorithm: sha11611client # [ 1.103231] registered taskstats version 11612worker1 # [ 1.068060] ima: No architecture policies found1613client # [ 1.103435] Loading compiled-in X.509 certificates1614worker1 # [ 1.071869] input: gpio-keys as /devices/platform/gpio-keys/input/input01615lb # [ 1.103300] Key type dns_resolver registered1616client # [ 1.112375] Demotion targets for Node 0: null1617client # [ 1.112517] Key type .fscrypt registered1618client # [ 1.112524] Key type fscrypt-provisioning registered1619client # [ 1.112628] ima: No TPM chip found, activating TPM-bypass!1620client # [ 1.112648] ima: Allocated hash algorithm: sha11621client # [ 1.112671] ima: No architecture policies found1622client # [ 1.116995] input: gpio-keys as /devices/platform/gpio-keys/input/input01623lb # [ 1.112363] registered taskstats version 11624lb # [ 1.112575] Loading compiled-in X.509 certificates1625lb # [ 1.122157] Demotion targets for Node 0: null1626lb # [ 1.122317] Key type .fscrypt registered1627lb # [ 1.122326] Key type fscrypt-provisioning registered1628lb # [ 1.124248] ima: No TPM chip found, activating TPM-bypass!1629lb # [ 1.124281] ima: Allocated hash algorithm: sha11630lb # [ 1.124312] ima: No architecture policies found1631lb # [ 1.127287] input: gpio-keys as /devices/platform/gpio-keys/input/input01632client # [ 1.137268] clk: Disabling unused clocks1633client # [ 1.137315] PM: genpd: Disabling unused power domains1634client # [ 1.141886] Freeing unused kernel memory: 4736K1635client # [ 1.142122] Run /init as init process1636worker1 # [ 1.107909] clk: Disabling unused clocks1637worker1 # [ 1.107959] PM: genpd: Disabling unused power domains1638worker1 # [ 1.112596] Freeing unused kernel memory: 4736K1639worker1 # [ 1.112831] Run /init as init process1640lb # [ 1.150729] clk: Disabling unused clocks1641lb # [ 1.150780] PM: genpd: Disabling unused power domains1642client # [ 1.158540] systemd[1]: Successfully made /usr/ read-only.1643lb # [ 1.155527] Freeing unused kernel memory: 4736K1644lb # [ 1.155775] Run /init as init process1645worker1 # [ 1.129757] systemd[1]: Successfully made /usr/ read-only.1646lb # [ 1.173036] systemd[1]: Successfully made /usr/ read-only.1647worker2 # [ 1.270611] usb 1-1: new high-speed USB device number 2 using ehci-pci1648client # [ 1.315022] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker1 # [ 1.283153] usb 1-1: new high-speed USB device number 2 using ehci-pci1650lb # [ 1.330543] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker2 # [ 1.425251] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652worker2 # [ 1.451925] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1653worker2 # [ 1.464687] systemd[1]: Detected virtualization qemu.1654worker2 # [ 1.466872] systemd[1]: Detected architecture arm64.1655worker2 # [ 1.468854] systemd[1]: Running in initrd.1656worker2 # [ 1.471722] systemd[1]: Initializing machine ID from random generator.1657worker2 # [ 1.474796] systemd[1]: Hostname set to <worker2>.1658worker2 # [ 1.519082] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01659client # [ 1.469308] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660worker1 # [ 1.437662] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11661lb # [ 1.487180] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11662client # [ 1.493611] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1663client # [ 1.500079] systemd[1]: Detected virtualization qemu.1664client # [ 1.501598] systemd[1]: Detected architecture arm64.1665client # [ 1.503498] systemd[1]: Running in initrd.1666client # [ 1.506668] systemd[1]: Initializing machine ID from random generator.1667client # [ 1.510208] systemd[1]: Hostname set to <client>.1668worker1 # [ 1.465181] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1669worker1 # [ 1.478029] systemd[1]: Detected virtualization qemu.1670worker1 # [ 1.480346] systemd[1]: Detected architecture arm64.1671worker1 # [ 1.482393] systemd[1]: Running in initrd.1672worker1 # [ 1.485511] systemd[1]: Initializing machine ID from random generator.1673lb # [ 1.508382] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1674worker1 # [ 1.488725] systemd[1]: Hostname set to <worker1>.1675lb # [ 1.521483] systemd[1]: Detected virtualization qemu.1676lb # [ 1.524066] systemd[1]: Detected architecture arm64.1677lb # [ 1.526142] systemd[1]: Running in initrd.1678lb # [ 1.529564] systemd[1]: Initializing machine ID from random generator.1679lb # [ 1.532898] systemd[1]: Hostname set to <lb>.1680client # [ 1.567213] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681worker1 # [ 1.535544] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682lb # [ 1.586823] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker2 # [ 1.646862] usb 1-2: new high-speed USB device number 3 using ehci-pci1684client # [ 1.686976] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker1 # [ 1.659173] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.710463] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker2 # [ 1.801627] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21688worker2 # [ 1.804918] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01689worker2 # [ 1.813107] systemd[1]: bpf-restrict-fs: LSM BPF program attached1690client # [ 1.801850] systemd[1]: bpf-restrict-fs: LSM BPF program attached1691worker2 # [ 1.892993] systemd[1]: Queued start job for default target Initrd Default Target.1692worker1 # [ 1.806517] systemd[1]: bpf-restrict-fs: LSM BPF program attached1693client # [ 1.845711] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21694client # [ 1.851235] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01695worker2 # [ 1.905157] systemd[1]: Created slice Slice /system/modprobe.1696worker2 # [ 1.906536] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1697worker2 # [ 1.906579] systemd[1]: Expecting device /dev/disk/by-label/nixos...1698worker2 # [ 1.906611] systemd[1]: Reached target Path Units.1699worker1 # [ 1.820613] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21700worker2 # [ 1.906630] systemd[1]: Reached target Slice Units.1701worker2 # [ 1.906649] systemd[1]: Reached target Swaps.1702worker2 # [ 1.906668] systemd[1]: Reached target Timer Units.1703worker2 # [ 1.906858] systemd[1]: Listening on D-Bus System Message Bus Socket.1704worker2 # [ 1.907010] systemd[1]: Listening on Journal Socket (/dev/log).1705worker2 # [ 1.907156] systemd[1]: Listening on Journal Sockets.1706worker2 # [ 1.907294] systemd[1]: Listening on udev Control Socket.1707worker2 # [ 1.907405] systemd[1]: Listening on udev Kernel Socket.1708worker1 # [ 1.827347] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01709worker2 # [ 1.907427] systemd[1]: Reached target Socket Units.1710worker2 # [ 1.919822] systemd[1]: Starting Create List of Static Device Nodes...1711lb # [ 1.868902] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712worker2 # [ 1.930749] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1713worker2 # [ 1.931932] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1714lb # [ 1.873588] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21715lb # [ 1.880426] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01716worker2 # [ 1.940554] systemd[1]: Mounting Kernel Configuration File System...1717client # [ 1.906360] systemd[1]: Queued start job for default target Initrd Default Target.1718worker2 # [ 1.966349] systemd[1]: Starting Journal Service...1719client # [ 1.917679] systemd[1]: Created slice Slice /system/modprobe.1720client # [ 1.918897] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1721client # [ 1.920116] systemd[1]: Expecting device /dev/disk/by-label/nixos...1722client # [ 1.921047] systemd[1]: Reached target Path Units.1723client # [ 1.921756] systemd[1]: Reached target Slice Units.1724client # [ 1.922505] systemd[1]: Reached target Swaps.1725client # [ 1.923278] systemd[1]: Reached target Timer Units.1726worker2 # [ 1.974897] systemd[1]: Starting Load Kernel Modules...1727client # [ 1.924283] systemd[1]: Listening on D-Bus System Message Bus Socket.1728worker2 # [ 1.975771] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1729client # [ 1.925393] systemd[1]: Listening on Journal Socket (/dev/log).1730client # [ 1.926491] systemd[1]: Listening on Journal Sockets.1731client # [ 1.927535] systemd[1]: Listening on udev Control Socket.1732client # [ 1.927669] systemd[1]: Listening on udev Kernel Socket.1733client # [ 1.927694] systemd[1]: Reached target Socket Units.1734client # [ 1.931817] systemd[1]: Starting Create List of Static Device Nodes...1735worker2 # [ 1.988575] systemd[1]: Starting Coldplug All udev Devices...1736client # [ 1.943223] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1737worker1 # [ 1.906729] systemd[1]: Queued start job for default target Initrd Default Target.1738client # [ 1.944359] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1739client # [ 1.951339] systemd[1]: Mounting Kernel Configuration File System...1740worker1 # [ 1.917955] systemd[1]: Created slice Slice /system/modprobe.1741worker1 # [ 1.919224] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1742worker1 # [ 1.919276] systemd[1]: Expecting device /dev/disk/by-label/nixos...1743worker1 # [ 1.919309] systemd[1]: Reached target Path Units.1744worker1 # [ 1.919332] systemd[1]: Reached target Slice Units.1745worker1 # [ 1.919353] systemd[1]: Reached target Swaps.1746worker1 # [ 1.919372] systemd[1]: Reached target Timer Units.1747worker1 # [ 1.919582] systemd[1]: Listening on D-Bus System Message Bus Socket.1748worker1 # [ 1.919744] systemd[1]: Listening on Journal Socket (/dev/log).1749worker1 # [ 1.919885] systemd[1]: Listening on Journal Sockets.1750worker1 # [ 1.920019] systemd[1]: Listening on udev Control Socket.1751worker1 # [ 1.920130] systemd[1]: Listening on udev Kernel Socket.1752worker1 # [ 1.920154] systemd[1]: Reached target Socket Units.1753worker1 # [ 1.932128] systemd[1]: Starting Create List of Static Device Nodes...1754worker2 # [ 2.018879] systemd[1]: Finished Create List of Static Device Nodes.1755worker2 # [ 2.020121] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1756worker2 # [ 2.020527] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1757worker2 # [ 2.020814] systemd[1]: Mounted Kernel Configuration File System.1758client # [ 1.975573] systemd[1]: Starting Journal Service...1759lb # [ 1.974538] systemd[1]: Queued start job for default target Initrd Default Target.1760worker1 # [ 1.943309] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1761worker1 # [ 1.944464] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1762worker1 # [ 1.951394] systemd[1]: Mounting Kernel Configuration File System...1763worker2 # [ 2.036151] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1764lb # [ 1.984708] systemd[1]: Created slice Slice /system/modprobe.1765worker2 # [ 2.039127] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1766lb # [ 1.985714] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1767lb # [ 1.986796] systemd[1]: Expecting device /dev/disk/by-label/nixos...1768lb # [ 1.987666] systemd[1]: Reached target Path Units.1769lb # [ 1.988305] systemd[1]: Reached target Slice Units.1770lb # [ 1.988968] systemd[1]: Reached target Swaps.1771lb # [ 1.989568] systemd[1]: Reached target Timer Units.1772lb # [ 1.990501] systemd[1]: Listening on D-Bus System Message Bus Socket.1773lb # [ 1.991534] systemd[1]: Listening on Journal Socket (/dev/log).1774client # [ 1.995258] systemd[1]: Starting Load Kernel Modules...1775lb # [ 1.992448] systemd[1]: Listening on Journal Sockets.1776lb # [ 1.993226] systemd[1]: Listening on udev Control Socket.1777client # [ 1.997431] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1778lb # [ 1.994101] systemd[1]: Listening on udev Kernel Socket.1779lb # [ 1.994864] systemd[1]: Reached target Socket Units.1780client # [ 2.000499] systemd[1]: Starting Coldplug All udev Devices...1781lb # [ 1.997033] systemd[1]: Starting Create List of Static Device Nodes...1782worker2 # [ 2.052641] systemd-journald[73]: Collecting audit messages is disabled.1783worker2 # [ 2.053714] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1784lb # [ 2.003675] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1785lb # [ 2.004662] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1786worker1 # [ 1.971552] systemd[1]: Starting Journal Service...1787lb # [ 2.014658] systemd[1]: Mounting Kernel Configuration File System...1788client # [ 2.023277] systemd[1]: Finished Create List of Static Device Nodes.1789client # [ 2.024728] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1790lb # [ 2.023645] systemd[1]: Starting Journal Service...1791client # [ 2.025074] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1792worker1 # [ 1.991354] systemd[1]: Starting Load Kernel Modules...1793client # [ 2.025423] systemd[1]: Mounted Kernel Configuration File System.1794worker1 # [ 1.992752] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1795worker2 # [ 2.088891] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01796worker2 # [ 2.089268] [drm] features: -virgl +edid -resource_blob -host_visible1797worker2 # [ 2.089283] [drm] features: -context_init1798worker2 # [ 2.090076] [drm] number of scanouts: 11799worker2 # [ 2.090103] [drm] number of cap sets: 01800client # [ 2.043658] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1801worker1 # [ 2.007299] systemd[1]: Starting Coldplug All udev Devices...1802worker2 # [ 2.095322] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1803worker1 # [ 2.011670] systemd[1]: Finished Create List of Static Device Nodes.1804worker1 # [ 2.013604] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1805lb # [ 2.050686] systemd[1]: Starting Load Kernel Modules...1806lb # [ 2.050851] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1807worker2 # [ 2.103035] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1808worker2 # [ 2.103065] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01809worker1 # [ 2.027409] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1810worker1 # [ 2.028110] systemd[1]: Mounted Kernel Configuration File System.1811worker2 # [ 2.113016] systemd[1]: Starting Create Static Device Nodes in /dev...1812client # [ 2.074828] systemd-journald[73]: Collecting audit messages is disabled.1813lb # [ 2.073795] systemd[1]: Starting Coldplug All udev Devices...1814worker1 # [ 2.042188] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1815lb # [ 2.077308] systemd-journald[73]: Collecting audit messages is disabled.1816lb # [ 2.079464] systemd[1]: Finished Create List of Static Device Nodes.1817client # [ 2.083198] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1818lb # [ 2.081338] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1819client # [ 2.092009] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1820lb # [ 2.091545] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1821lb # [ 2.092775] systemd[1]: Mounted Kernel Configuration File System.1822worker2 # [ 2.150592] Console: switching to colour frame buffer device 160x501823worker2 # [ 2.151346] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1824client # [ 2.103309] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01825client # [ 2.103578] [drm] features: -virgl +edid -resource_blob -host_visible1826client # [ 2.103588] [drm] features: -context_init1827client # [ 2.104395] [drm] number of scanouts: 11828client # [ 2.104413] [drm] number of cap sets: 01829worker2 # [ 2.159369] systemd[1]: Finished Load Kernel Modules.1830worker1 # [ 2.074160] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1831lb # [ 2.103060] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1832worker2 # [ 2.167415] systemd[1]: Starting Apply Kernel Variables...1833worker1 # [ 2.081243] systemd-journald[73]: Collecting audit messages is disabled.1834client # [ 2.123779] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1835client # [ 2.126767] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1836client # [ 2.126799] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01837worker1 # [ 2.093056] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1838client # [ 2.135618] systemd[1]: Starting Create Static Device Nodes in /dev...1839worker1 # [ 2.103483] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01840worker2 # [ 2.191690] systemd[1]: Finished Create Static Device Nodes in /dev.1841worker1 # [ 2.103745] [drm] features: -virgl +edid -resource_blob -host_visible1842worker1 # [ 2.103755] [drm] features: -context_init1843worker2 # [ 2.192006] systemd[1]: Reached target Preparation for Local File Systems.1844worker1 # [ 2.104536] [drm] number of scanouts: 11845worker1 # [ 2.104563] [drm] number of cap sets: 01846worker2 # [ 2.192045] systemd[1]: Reached target Local File Systems.1847lb # [ 2.136707] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1848worker2 # [ 2.206987] systemd[1]: Starting Rule-based Manager for Device Events and Files...1849worker1 # [ 2.123875] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1850worker1 # [ 2.126852] systemd[1]: Starting Create Static Device Nodes in /dev...1851lb # [ 2.162635] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1852worker1 # [ 2.128330] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1853client # [ 2.160021] Console: switching to colour frame buffer device 160x501854worker1 # [ 2.128347] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01855client # [ 2.167811] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1856lb # [ 2.165685] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1857lb # [ 2.179007] systemd[1]: Starting Create Static Device Nodes in /dev...1858lb # [ 2.185614] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01859lb # [ 2.185932] [drm] features: -virgl +edid -resource_blob -host_visible1860lb # [ 2.185949] [drm] features: -context_init1861lb # [ 2.196747] [drm] number of scanouts: 11862worker2 # [ 2.251006] systemd[1]: Finished Apply Kernel Variables.1863client # [ 2.200183] systemd[1]: Finished Load Kernel Modules.1864lb # [ 2.196796] [drm] number of cap sets: 01865client # [ 2.202790] systemd[1]: Starting Apply Kernel Variables...1866worker1 # [ 2.163436] Console: switching to colour frame buffer device 160x501867worker1 # [ 2.170248] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1868lb # [ 2.198717] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1869lb # [ 2.198736] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01870worker2 # [ 2.283372] systemd[1]: Started Journal Service.1871client # [ 2.231364] systemd[1]: Finished Create Static Device Nodes in /dev.1872client # [ 2.232910] systemd[1]: Reached target Preparation for Local File Systems.1873client # [ 2.233199] systemd[1]: Reached target Local File Systems.1874worker1 # [ 2.199887] systemd[1]: Finished Create Static Device Nodes in /dev.1875worker1 # [ 2.200145] systemd[1]: Reached target Preparation for Local File Systems.1876lb # [ 2.232409] Console: switching to colour frame buffer device 160x501877worker1 # [ 2.200185] systemd[1]: Reached target Local File Systems.1878worker2 # [ 2.272432] systemd-modules-load[74]: Inserted module 'dm_mod'1879worker1 # [ 2.206019] systemd[1]: Starting Rule-based Manager for Device Events and Files...1880worker2 # [ 2.276360] systemd-modules-load[74]: Module 'virtio_balloon' is built in1881worker2 # [ 2.277625] systemd-modules-load[74]: Module 'virtio_console' is built in1882worker2 # [ 2.278720] systemd-modules-load[74]: Inserted module 'virtio_gpu'1883worker2 # [ 2.279744] systemd-modules-load[74]: Module 'virtio_rng' is built in1884client # [ 2.251432] systemd[1]: Starting Rule-based Manager for Device Events and Files...1885lb # [ 2.251325] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1886worker1 # [ 2.223284] systemd[1]: Finished Load Kernel Modules.1887worker2 # [ 2.290602] systemd[1]: Starting Create System Files and Directories...1888lb # [ 2.259478] systemd[1]: Finished Load Kernel Modules.1889worker1 # [ 2.231458] systemd[1]: Starting Apply Kernel Variables...1890client # [ 2.275290] systemd[1]: Finished Apply Kernel Variables.1891lb # [ 2.271237] systemd[1]: Starting Apply Kernel Variables...1892worker2 # [ 2.310549] systemd-udevd[81]: Using default interface naming scheme 'v261'.1893client # [ 2.289082] systemd[1]: Started Journal Service.1894client # [ 2.273996] systemd-modules-load[74]: Inserted module 'dm_mod'1895client # [ 2.275326] systemd-modules-load[74]: Module 'virtio_balloon' is built in1896lb # [ 2.290664] systemd[1]: Finished Create Static Device Nodes in /dev.1897lb # [ 2.293389] systemd[1]: Reached target Preparation for Local File Systems.1898client # [ 2.285564] systemd-modules-load[74]: Module 'virtio_console' is built in1899client # [ 2.286704] systemd-modules-load[74]: Inserted module 'virtio_gpu'1900worker2 # [ 2.334676] systemd[1]: Finished Create System Files and Directories.1901client # [ 2.287764] systemd-modules-load[74]: Module 'virtio_rng' is built in1902lb # [ 2.302637] systemd[1]: Reached target Local File Systems.1903lb # [ 2.311077] systemd[1]: Starting Rule-based Manager for Device Events and Files...1904client # [ 2.300511] systemd[1]: Starting Create System Files and Directories...1905worker2 # [ 2.357557] systemd[1]: Started Rule-based Manager for Device Events and Files.1906worker1 # [ 2.293695] systemd[1]: Started Journal Service.1907worker1 # [ 2.288453] systemd-modules-load[74]: Inserted module 'dm_mod'1908worker1 # [ 2.292489] systemd-modules-load[74]: Module 'virtio_balloon' is built in1909worker1 # [ 2.293650] systemd-modules-load[74]: Module 'virtio_console' is built in1910worker1 # [ 2.294809] systemd-modules-load[74]: Inserted module 'virtio_gpu'1911worker1 # [ 2.295837] systemd-modules-load[74]: Module 'virtio_rng' is built in1912client # [ 2.334094] systemd-udevd[80]: Using default interface naming scheme 'v261'.1913lb # [ 2.354995] systemd[1]: Finished Apply Kernel Variables.1914worker1 # [ 2.309415] systemd[1]: Starting Create System Files and Directories...1915lb # [ 2.339681] systemd-modules-load[75]: Inserted module 'dm_mod'1916worker1 # [ 2.310584] systemd[1]: Finished Apply Kernel Variables.1917lb # [ 2.359466] systemd[1]: Started Journal Service.1918lb # [ 2.345298] systemd-modules-load[75]: Module 'virtio_balloon' is built in1919lb # [ 2.346726] systemd-modules-load[75]: Module 'virtio_console' is built in1920client # [ 2.350859] systemd[1]: Finished Create System Files and Directories.1921lb # [ 2.347934] systemd-modules-load[75]: Inserted module 'virtio_gpu'1922lb # [ 2.364133] systemd-modules-load[75]: Module 'virtio_rng' is built in1923worker1 # [ 2.335229] systemd-udevd[79]: Using default interface naming scheme 'v261'.1924lb # [ 2.368595] systemd[1]: Starting Create System Files and Directories...1925client # [ 2.376123] systemd[1]: Started Rule-based Manager for Device Events and Files.1926worker2 # [ 2.436199] systemd[1]: Starting Virtual Console Setup...1927worker1 # [ 2.359536] systemd[1]: Finished Create System Files and Directories.1928worker1 # [ 2.382187] systemd[1]: Started Rule-based Manager for Device Events and Files.1929lb # [ 2.432445] systemd-udevd[81]: Using default interface naming scheme 'v261'.1930worker2 # [ 2.492967] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1931worker2 # [ 2.495976] systemd[1]: Finished Virtual Console Setup.1932lb # [ 2.444505] systemd[1]: Finished Create System Files and Directories.1933client # [ 2.468825] systemd[1]: Starting Virtual Console Setup...1934worker1 # [ 2.452311] systemd[1]: Starting Virtual Console Setup...1935lb # [ 2.483332] systemd[1]: Started Rule-based Manager for Device Events and Files.1936client # [ 2.520615] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1937client # [ 2.524978] systemd[1]: Finished Virtual Console Setup.1938worker1 # [ 2.512627] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1939worker1 # [ 2.516882] systemd[1]: Finished Virtual Console Setup.1940lb # [ 2.584434] systemd[1]: Starting Virtual Console Setup...1941lb # [ 2.651703] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942lb # [ 2.660454] systemd[1]: Finished Virtual Console Setup.1943worker2 # [ 3.173419] systemd[1]: Finished Coldplug All udev Devices.1944worker2 # [ 3.174456] systemd[1]: Reached target System Initialization.1945worker2 # [ 3.175375] systemd[1]: Reached target Basic System.1946worker1 # [ 3.169542] systemd[1]: Finished Coldplug All udev Devices.1947worker1 # [ 3.170612] systemd[1]: Reached target System Initialization.1948worker1 # [ 3.171468] systemd[1]: Reached target Basic System.1949client # [ 3.236373] systemd[1]: Finished Coldplug All udev Devices.1950client # [ 3.237411] systemd[1]: Reached target System Initialization.1951client # [ 3.238280] systemd[1]: Reached target Basic System.1952worker2 # [ 3.377501] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1953worker2 # [ 3.401397] (udev-worker)[106]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1954worker2 # [ 3.406467] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1955worker1 # [ 3.351259] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1956client # [ 3.401068] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1957worker1 # [ 3.387245] (udev-worker)[106]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1958worker2 # [ 3.473835] systemd[1]: Found device /dev/disk/by-label/nixos.1959worker1 # [ 3.392771] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1960client # [ 3.427071] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1961worker2 # [ 3.475929] systemd[1]: Reached target Initrd Root Device.1962worker2 # [ 3.480313] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1963client # [ 3.435316] (udev-worker)[105]: Network interface NamePolicy= disabled on kernel command line.1964lb # [ 3.457779] systemd[1]: Finished Coldplug All udev Devices.1965lb # [ 3.458799] systemd[1]: Reached target System Initialization.1966lb # [ 3.459726] systemd[1]: Reached target Basic System.1967worker1 # [ 3.462321] systemd[1]: Found device /dev/disk/by-label/nixos.1968worker1 # [ 3.466131] systemd[1]: Reached target Initrd Root Device.1969worker2 # [ 3.548969] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1970worker1 # [ 3.469556] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1971worker2 # [ 3.559533] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1972client # [ 3.515291] systemd[1]: Found device /dev/disk/by-label/nixos.1973client # [ 3.517357] systemd[1]: Reached target Initrd Root Device.1974client # [ 3.523477] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1975worker2 # [ 3.576443] systemd[1]: Mounting /sysroot...1976worker1 # [ 3.541577] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1977worker2 # [ 3.643235] EXT4-fs (vda): mounted filesystem b333ff5c-a552-45bd-8136-190a86afb19c r/w with ordered data mode. Quota mode: none.1978worker2 # [ 3.629711] systemd[1]: Mounted /sysroot.1979worker1 # [ 3.549015] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1980client # [ 3.582500] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1981worker2 # [ 3.634510] systemd[1]: Reached target Initrd Root File System.1982worker1 # [ 3.555889] systemd[1]: Mounting /sysroot...1983client # [ 3.591961] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1984worker2 # [ 3.640126] systemd[1]: Starting Mountpoints Configured in the Real Root...1985client # [ 3.595130] systemd[1]: Mounting /sysroot...1986worker2 # [ 3.674769] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1987worker1 # [ 3.614339] EXT4-fs (vda): mounted filesystem 7cf2ff25-fb6d-43f1-9fa1-b0f62ac739a2 r/w with ordered data mode. Quota mode: none.1988worker2 # [ 3.683354] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1989worker2 # [ 3.687398] systemd[1]: Reloading...1990worker1 # [ 3.607342] systemd[1]: Mounted /sysroot.1991worker1 # [ 3.609474] systemd[1]: Reached target Initrd Root File System.1992worker1 # [ 3.614138] systemd[1]: Starting Mountpoints Configured in the Real Root...1993client # [ 3.664495] EXT4-fs (vda): mounted filesystem 291c9adc-67bd-468f-8271-a9cfc0e62738 r/w with ordered data mode. Quota mode: none.1994client # [ 3.653697] systemd[1]: Mounted /sysroot.1995client # [ 3.656178] systemd[1]: Reached target Initrd Root File System.1996client # [ 3.662962] systemd[1]: Starting Mountpoints Configured in the Real Root...1997worker1 # [ 3.644957] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1998worker1 # [ 3.652119] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1999worker1 # [ 3.656606] systemd[1]: Reloading...2000client # [ 3.696456] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload.
loading 1,433 hidden lines…
3434lb # [ 19.437416] systemd[1]: Started PostgreSQL Server.3435lb # [ 19.447293] systemd[1]: Starting PostgreSQL Setup Scripts...3436lb # [ 19.700367] postgresql-setup-start[847]: CREATE DATABASE3437lb # [ 19.760277] postgresql-setup-start[852]: CREATE ROLE3438lb # [ 19.787511] postgresql-setup-start[854]: ALTER DATABASE3439lb # [ 19.795641] systemd[1]: Finished PostgreSQL Setup Scripts.3440lb # [ 19.797804] systemd[1]: Reached target PostgreSQL.3441lb # [ 20.528746] dhcpcd[690]: eth0: leased 10.0.2.15 for 86400 seconds3442lb # [ 20.533886] dhcpcd[690]: eth0: adding route to 10.0.2.0/243443lb # [ 20.534163] dhcpcd[690]: eth0: adding default route via 10.0.2.23444lb # [ 20.721992] systemd[1]: Started DHCP Client.3445lb # [ 20.730925] systemd[1]: Reached target Network is Online.3446lb # [ 20.734286] systemd[1]: Started Envoy reverse proxy.3447lb # [ 20.741069] systemd[1]: Started mock-oidc.service.3448lb # [ 20.843956] mock-oidc-server[922]: Mock OIDC Server running3449lb # [ 20.845613] mock-oidc-server[922]: OIDC Address: 192.168.1.2:80803450lb # [ 20.847365] mock-oidc-server[922]: Issue Address: 0.0.0.0:80813451lb # [ 20.849147] mock-oidc-server[922]: Issuer: http://192.168.1.2:8080/oidc3452lb # [ 20.850596] mock-oidc-server[922]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3453lb # [ 20.852404] mock-oidc-server[922]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3454lb # [ 20.854166] mock-oidc-server[922]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3455worker1 # [ 20.824910] dhcpcd[724]: eth0: leased 10.0.2.15 for 86400 seconds3456worker1 # [ 20.826720] dhcpcd[724]: eth0: adding route to 10.0.2.0/243457worker1 # [ 20.828915] dhcpcd[724]: eth0: adding default route via 10.0.2.23458worker1 # [ 20.957644] systemd[1]: Started DHCP Client.3459worker1 # [ 20.959499] systemd[1]: Reached target Multi-User System.3460worker1 # [ 20.961085] systemd[1]: Startup finished in 1.102s (kernel) + 5.568s (initrd) + 14.288s (userspace) = 20.959s.3461lb # [ 21.494311] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120)3462lb # [ 21.494462] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:469] statically linked extensions:3463lb # [ 21.494502] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response3464lb # [ 21.494575] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls3465lb # [ 21.519599] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls3466lb # [ 21.525571] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query3467lb # [ 21.529521] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3468lb # [ 21.533417] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default3469lb # [ 21.535851] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83470lb # [ 21.538070] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory3471lb # [ 21.541011] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router3472lb # [ 21.572499] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3473lb # [ 21.576411] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san3474lb # [ 21.583945] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle3475lb # [ 21.588143] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter3476lb # [ 21.590231] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3477lb # [ 21.595106] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3478lb # [ 21.598638] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3479lb # [ 21.601171] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats3480lb # [ 21.605512] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3481lb # [ 21.613139] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3482lb # [ 21.615753] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router3483lb # [ 21.617943] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd3484lb # [ 21.624656] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input3485lb # [ 21.626673] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple3486lb # [ 21.632518] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3487lb # [ 21.635037] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override3488lb # [ 21.639564] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default3489lb # [ 21.641595] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert3490lb # [ 21.643546] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3491lb # [ 21.646610] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3492lb # [ 21.651945] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3493lb # [ 21.654123] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log3494lb # [ 21.663892] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector3495lb # [ 21.671154] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3496lb # [ 21.673651] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection3497lb # [ 21.679271] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3498lb # [ 21.682307] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3499lb # [ 21.685890] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re23500lb # [ 21.687736] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3501lb # [ 21.695720] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3502lb # [ 21.699271] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3503lb # [ 21.702634] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3504lb # [ 21.705190] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit3505lb # [ 21.707924] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3506lb # [ 21.711051] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo3507lb # [ 21.713007] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3508lb # [ 21.717643] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port3509lb # [ 21.726509] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3510lb # [ 21.729379] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3511lb # [ 21.731529] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy3512lb # [ 21.734470] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3513lb # [ 21.737101] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3514lb # [ 21.741684] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3515lb # [ 21.743651] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3516lb # [ 21.745581] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config3517lb # [ 21.749570] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter3518lb # [ 21.752049] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator3519lb # [ 21.754611] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection3520lb # [ 21.756710] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3521lb # [ 21.759636] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3522lb # [ 21.764585] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3523lb # [ 21.766596] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3524lb # [ 21.769041] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3525lb # [ 21.774207] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3526lb # [ 21.777128] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3527lb # [ 21.779416] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3528lb # [ 21.782393] envoy[921]: [2026-09-20 12:47:35.941][921][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3529lb # [ 21.785234] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3530lb # [ 21.787169] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default3531lb # [ 21.789346] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme3532lb # [ 21.793018] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr3533lb # [ 21.795027] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case3534lb # [ 21.798021] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3535lb # [ 21.799904] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder3536lb # [ 21.802614] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3537lb # [ 21.810098] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3538lb # [ 21.812485] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3539lb # [ 21.814321] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin3540lb # [ 21.818218] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3541lb # [ 21.823103] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable3542lb # [ 21.825991] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default3543lb # [ 21.828061] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy3544lb # [ 21.841710] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23545lb # [ 21.843544] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3546lb # [ 21.846981] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3547lb # [ 21.849304] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3548lb # [ 21.851151] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23549lb # [ 21.854128] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13550lb # [ 21.856692] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic3551lb # [ 21.858656] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3552lb # [ 21.862813] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3553lb # [ 21.865655] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3554lb # [ 21.867922] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3555lb # [ 21.870637] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3556lb # [ 21.874185] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3557lb # [ 21.875957] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift3558lb # [ 21.879681] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action3559lb # [ 21.882100] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3560lb # [ 21.884243] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3561lb # [ 21.886962] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3562lb # [ 21.898649] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3563lb # [ 21.900564] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3564lb # [ 21.903250] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal3565lb # [ 21.905422] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3566lb # [ 21.907607] envoy[921]: [2026-09-20 12:47:35.942][921][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3567lb # [ 21.909990] envoy[921]: [2026-09-20 12:47:35.964][921][info][main] [source/server/server.cc:531] HTTP header map info:3568lb # [ 21.911720] envoy[921]: [2026-09-20 12:47:35.966][921][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3569lb # [ 21.927959] envoy[921]: [2026-09-20 12:47:35.966][921][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3570lb # [ 21.929781] envoy[921]: [2026-09-20 12:47:35.966][921][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3571lb # [ 21.939081] envoy[921]: [2026-09-20 12:47:35.966][921][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3572lb # [ 21.941307] envoy[921]: [2026-09-20 12:47:36.207][921][info][main] [source/server/server.cc:968] runtime: {}3573lb # [ 21.942734] envoy[921]: [2026-09-20 12:47:36.230][921][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3574lb # [ 21.944817] envoy[921]: [2026-09-20 12:47:36.230][921][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013575lb # [ 21.946643] envoy[921]: [2026-09-20 12:47:36.230][921][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013576lb # [ 21.948610] envoy[921]: [2026-09-20 12:47:36.237][921][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3577lb # [ 21.950369] envoy[921]: [2026-09-20 12:47:36.237][921][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3578lb # [ 21.952401] envoy[921]: [2026-09-20 12:47:36.237][921][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3579lb # [ 21.977799] envoy[921]: [2026-09-20 12:47:36.425][921][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3580worker2 # [ 22.033271] systemd[1]: Reached target Network is Online.3581lb # [ 21.988635] envoy[921]: [2026-09-20 12:47:36.437][921][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3582worker1 # [ 21.960433] systemd[1]: Starting Nix worker-protocol over gRPC...3583lb # [ 21.991543] envoy[921]: [2026-09-20 12:47:36.439][921][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3584worker2 # [ 22.045717] systemd[1]: Starting scheduler-token.service...3585lb # [ 21.995213] envoy[921]: [2026-09-20 12:47:36.439][921][info][main] [source/server/server.cc:1077] starting main dispatch loop3586worker2 # [ 22.182011] systemd[1]: scheduler-token.service: Deactivated successfully.3587worker2 # [ 22.188347] systemd[1]: Finished scheduler-token.service.3588worker2 # [ 22.189911] systemd[1]: scheduler-token.service: Consumed 58ms CPU time over 139ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3589worker2 # [ 22.198481] systemd[1]: Starting Nix worker-protocol over gRPC...3590worker1 # [ 22.212236] nix-grpc-daemon[832]: ts=2026-09-20T12:47:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3591worker1 # [ 22.224820] nix-grpc-daemon[832]: ts=2026-09-20T12:47:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3592worker1 # [ 22.227132] nix-grpc-daemon[832]: ts=2026-09-20T12:47:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13593worker1 # [ 22.276508] nix-grpc-daemon[832]: ts=2026-09-20T12:47:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3594worker1 # [ 22.278348] systemd[1]: Started Nix worker-protocol over gRPC.3595worker1 # [ 22.309963] nix-grpc-daemon[832]: ts=2026-09-20T12:47:36Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3596lb # [ 22.343653] envoy[921]: [2026-09-20 12:47:36.791][921][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3597lb # [ 22.351201] envoy[921]: [2026-09-20 12:47:36.791][921][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3598lb # [ 22.356903] envoy[921]: [2026-09-20 12:47:36.791][921][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3599lb # [ 22.362125] envoy[921]: [2026-09-20 12:47:36.791][921][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3600worker2 # [ 22.423807] nix-grpc-daemon[780]: ts=2026-09-20T12:47:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3601worker2 # [ 22.436381] nix-grpc-daemon[780]: ts=2026-09-20T12:47:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3602worker2 # [ 22.438712] nix-grpc-daemon[780]: ts=2026-09-20T12:47:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13603worker2 # [ 22.485329] nix-grpc-daemon[780]: ts=2026-09-20T12:47:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3604worker2 # [ 22.491564] systemd[1]: Started Nix worker-protocol over gRPC.3605worker2 # [ 22.529164] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_connected addr=lb:500513606worker2 # [ 22.534705] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3607worker1 # [ 22.511044] nix-grpc-daemon[832]: ts=2026-09-20T12:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3608worker2 # [ 22.736837] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_connected addr=lb:500513609worker2 # [ 22.742380] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3610worker2 # [ 22.943829] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_connected addr=lb:500513611worker2 # [ 22.949361] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3612worker1 # [ 22.911725] nix-grpc-daemon[832]: ts=2026-09-20T12:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3613worker2 # [ 23.150587] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_connected addr=lb:500513614worker2 # [ 23.156397] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3615worker2 # [ 23.357964] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_connected addr=lb:500513616worker2 # [ 23.363622] nix-grpc-daemon[780]: ts=2026-09-20T12:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3617worker2 # [ 23.567120] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500513618worker2 # [ 23.571869] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3619worker2 # [ 23.768471] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500513620worker2 # [ 23.774009] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3621worker1 # [ 23.715403] nix-grpc-daemon[832]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500513622worker1 # [ 23.715778] nix-grpc-daemon[832]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3623worker2 # [ 23.975516] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500513624worker2 # [ 23.981410] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3625worker1 # [ 23.916764] nix-grpc-daemon[832]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500513626worker1 # [ 23.922238] nix-grpc-daemon[832]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3627worker2 # [ 24.182649] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500513628worker2 # [ 24.188284] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3629worker1 # [ 24.122990] nix-grpc-daemon[832]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500513630worker1 # [ 24.126716] nix-grpc-daemon[832]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3631worker2 # [ 24.389780] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500513632worker2 # [ 24.395496] nix-grpc-daemon[780]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3633worker1 # [ 24.327779] nix-grpc-daemon[832]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500513634worker1 # [ 24.333854] nix-grpc-daemon[832]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3635worker2 # [ 24.598967] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513636worker2 # [ 24.603271] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3637worker1 # [ 24.535424] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513638worker1 # [ 24.540958] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3639worker2 # [ 24.800503] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513640worker2 # [ 24.806022] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3641worker1 # [ 24.744160] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513642worker1 # [ 24.748342] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3643worker2 # [ 25.007481] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513644worker2 # [ 25.013159] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3645worker1 # [ 24.944866] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513646worker1 # [ 24.950295] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3647worker2 # [ 25.214322] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513648worker2 # [ 25.219836] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3649worker1 # [ 25.151352] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513650worker1 # [ 25.157000] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3651worker2 # [ 25.421104] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513652worker2 # [ 25.426394] nix-grpc-daemon[780]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3653worker1 # [ 25.358011] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500513654worker1 # [ 25.363550] nix-grpc-daemon[832]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3655worker2 # [ 25.629519] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513656worker2 # [ 25.633738] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3657worker1 # [ 25.564578] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513658worker1 # [ 25.570250] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3659worker2 # [ 25.830884] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513660worker2 # [ 25.836406] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3661worker1 # [ 25.773542] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513662worker1 # [ 25.777874] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3663worker2 # [ 26.037822] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513664worker2 # [ 26.043731] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3665worker1 # [ 25.974159] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513666worker1 # [ 25.979703] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3667worker2 # [ 26.245182] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513668worker2 # [ 26.250767] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3669worker1 # [ 26.181143] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513670worker1 # [ 26.186741] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3671worker2 # [ 26.452228] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513672worker2 # [ 26.457783] nix-grpc-daemon[780]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3673worker1 # [ 26.387669] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500513674worker1 # [ 26.393266] nix-grpc-daemon[832]: ts=2026-09-20T12:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3675worker2 # [ 26.660507] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513676worker2 # [ 26.665571] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3677worker1 # [ 26.594551] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513678worker1 # [ 26.600073] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3679worker2 # [ 26.867068] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513680worker2 # [ 26.872728] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3681worker1 # [ 26.803059] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513682worker1 # [ 26.807395] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3683worker2 # [ 27.074172] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513684worker2 # [ 27.080206] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3685worker1 # [ 27.003849] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513686worker1 # [ 27.009389] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3687worker2 # [ 27.281644] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513688worker2 # [ 27.287192] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3689worker1 # [ 27.210177] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513690worker1 # [ 27.215632] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3691worker2 # [ 27.489313] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513692worker2 # [ 27.494752] nix-grpc-daemon[780]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3693worker1 # [ 27.417843] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500513694worker1 # [ 27.423261] nix-grpc-daemon[832]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3695worker2 # [ 27.698132] nix-grpc-daemon[780]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500513696worker1 # [ 27.619005] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500513697worker2 # [ 27.702662] nix-grpc-daemon[780]: ts=2026-09-20T12:47:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3698worker1 # [ 27.624582] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3699worker2 # [ 27.899091] nix-grpc-daemon[780]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500513700worker2 # [ 27.905213] nix-grpc-daemon[780]: ts=2026-09-20T12:47:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3701worker1 # [ 27.827638] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500513702worker1 # [ 27.831937] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3703worker2 # [ 28.106607] nix-grpc-daemon[780]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500513704worker1 # [ 28.028727] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500513705worker2 # [ 28.112312] nix-grpc-daemon[780]: ts=2026-09-20T12:47:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3706worker1 # [ 28.033111] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3707worker1 # [ 28.230128] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500513708worker2 # [ 28.313508] nix-grpc-daemon[780]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500513709worker1 # [ 28.235590] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3710worker2 # [ 28.319146] nix-grpc-daemon[780]: ts=2026-09-20T12:47:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3711worker1 # [ 28.436530] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500513712worker2 # [ 28.520488] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513713worker1 # [ 28.441961] nix-grpc-daemon[832]: ts=2026-09-20T12:47:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3714worker2 # [ 28.526010] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3715worker1 # [ 28.642942] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513716worker2 # [ 28.729429] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513717worker1 # [ 28.648827] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3718worker2 # [ 28.733802] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3719worker2 # [ 28.930371] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513720worker1 # [ 28.851177] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513721worker2 # [ 28.936171] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3722worker1 # [ 28.856377] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3723worker1 # [ 29.053448] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513724worker2 # [ 29.137342] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513725worker1 # [ 29.058937] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3726worker2 # [ 29.142868] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker1 # [ 29.259691] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513728worker1 # [ 29.263236] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3729worker2 # [ 29.344880] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513730worker2 # [ 29.349768] nix-grpc-daemon[780]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker1 # [ 29.464532] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_connected addr=lb:500513732worker2 # [ 29.550670] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 29.470050] nix-grpc-daemon[832]: ts=2026-09-20T12:47:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3734worker2 # [ 29.555984] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker1 # [ 29.671039] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513736worker1 # [ 29.676586] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3737worker2 # [ 29.759305] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513738worker2 # [ 29.763678] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3739worker1 # [ 29.879368] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513740worker2 # [ 29.960080] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 29.883678] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3742worker2 # [ 29.965283] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3743worker1 # [ 30.080243] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513744worker2 # [ 30.166543] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513745worker1 # [ 30.085503] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3746worker2 # [ 30.171903] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3747worker1 # [ 30.286527] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513748worker2 # [ 30.373150] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513749worker1 # [ 30.291961] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3750worker2 # [ 30.378433] nix-grpc-daemon[780]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751worker1 # [ 30.492945] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_connected addr=lb:500513752worker1 # [ 30.498218] nix-grpc-daemon[832]: ts=2026-09-20T12:47:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3753worker2 # [ 30.579577] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500513754worker2 # [ 30.585348] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3755worker1 # [ 30.700859] nix-grpc-daemon[832]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500513756worker2 # [ 30.788742] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500513757worker1 # [ 30.709431] nix-grpc-daemon[832]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3758worker2 # [ 30.792945] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3759lb # [ 30.912837] rustfs-bucket-start[1022]: mb s3://farm3760lb # [ 30.924607] systemd[1]: Finished rustfs-bucket.service.3761worker1 # [ 30.903103] nix-grpc-daemon[832]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500513762lb # [ 30.940110] systemd[1]: Starting niks3 server...3763worker2 # [ 30.989535] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500513764worker1 # [ 30.908556] nix-grpc-daemon[832]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3765worker2 # [ 30.994710] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3766worker1 # [ 31.109498] nix-grpc-daemon[832]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500513767worker1 # [ 31.114852] nix-grpc-daemon[832]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3768worker2 # [ 31.195890] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500513769worker2 # [ 31.201337] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3770lb # [ 31.162730] postgres[1038]: [1038] ERROR: relation "goose_db_version" does not exist at character 363771lb # [ 31.164642] postgres[1038]: [1038] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3772lb # [ 31.206226] niks3-server[1032]: 2026/09/20 12:47:45 OK 20241026095416_initial_model.sql (25.97ms)3773lb # [ 31.211515] niks3-server[1032]: 2026/09/20 12:47:45 OK 20251210153512_drop_unused_gin_index.sql (5.87ms)3774lb # [ 31.221669] niks3-server[1032]: 2026/09/20 12:47:45 OK 20251218171726_add_pins.sql (10.14ms)3775lb # [ 31.232735] niks3-server[1032]: 2026/09/20 12:47:45 OK 20260628120000_add_object_size_and_stats.sql (11.07ms)3776lb # [ 31.240275] niks3-server[1032]: 2026/09/20 12:47:45 OK 20260905000000_add_claims.sql (5.62ms)3777lb # [ 31.248133] niks3-server[1032]: 2026/09/20 12:47:45 OK 20260920000000_drop_claims.sql (7.63ms)3778lb # [ 31.249543] niks3-server[1032]: 2026/09/20 12:47:45 goose: successfully migrated database to version: 202609200000003779lb # [ 31.257440] niks3-server[1032]: 2026/09/20 12:47:45 OK 1_commit_pending_closure.sql (8.91ms)3780lb # [ 31.262546] niks3-server[1032]: 2026/09/20 12:47:45 OK 2_object_stats_trigger.sql (5.46ms)3781lb # [ 31.266701] niks3-server[1032]: 2026/09/20 12:47:45 goose: up to current file version: 23782lb # [ 31.271098] niks3-server[1032]: 2026/09/20 12:47:45 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3783lb # [ 31.333917] niks3-server[1032]: 2026/09/20 12:47:45 INFO Read proxy enabled — serving cache objects from S33784lb # [ 31.335512] niks3-server[1032]: 2026/09/20 12:47:45 INFO Using socket-activated listener address=0.0.0.0:57513785lb # [ 31.340552] niks3-server[1032]: 2026/09/20 12:47:45 INFO systemd watchdog enabled interval=15s3786lb # [ 31.343907] niks3-server[1032]: 2026/09/20 12:47:45 INFO Starting HTTP server address=0.0.0.0:57513787lb # [ 31.347449] systemd[1]: Started niks3 server.3788worker1 # [ 31.315946] nix-grpc-daemon[832]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500513789lb # [ 31.348920] systemd[1]: Reached target Multi-User System.3790worker1 # [ 31.321396] nix-grpc-daemon[832]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3791worker2 # [ 31.402520] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500513792worker2 # [ 31.407697] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3793lb # [ 31.351119] systemd[1]: Startup finished in 1.142s (kernel) + 5.955s (initrd) + 24.244s (userspace) = 31.342s.3794lb # [ 31.362165] niks3-server[1032]: 2026/09/20 12:47:45 INFO lead: acquired remote=192.168.1.4:553303795worker2 # [ 31.414981] nix-grpc-daemon[780]: ts=2026-09-20T12:47:45Z level=info event=scheduler_take_over3796lb: (finished: waiting for unit niks3.service, in 15.99 seconds)3797lb: waiting for TCP port 5751 on localhost3798lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3799lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)3800lb: waiting for unit envoy.service3801worker1 # [ 31.522605] nix-grpc-daemon[832]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500513802worker2 # [ 31.606817] nix-grpc-daemon[780]: ts=2026-09-20T12:47:46Z level=info event=scheduler_connected addr=lb:500513803worker2 # [ 31.616553] nix-grpc-daemon[780]: ts=2026-09-20T12:47:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:517243804worker2 # [ 31.623343] nix-grpc-daemon[780]: ts=2026-09-20T12:47:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03805worker2 # [ 31.635951] nix-grpc-daemon[780]: ts=2026-09-20T12:47:46Z level=info event=oidc_jwks provider=mock keys=13806worker2 # [ 31.640425] nix-grpc-daemon[780]: ts=2026-09-20T12:47:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:517243807worker2 # [ 31.645712] nix-grpc-daemon[780]: ts=2026-09-20T12:47:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03808lb: (finished: waiting for unit envoy.service, in 0.06 seconds)3809lb: waiting for TCP port 50051 on localhost3810lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3811lb # [ 31.645386] envoy[921]: conn peer=[::1]:55872 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33812lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3813worker2 # [ 31.878383] systemd[1]: Starting scheduler-token.service...3814worker2 # [ 31.969730] systemd[1]: scheduler-token.service: Deactivated successfully.3815worker2 # [ 31.973618] systemd[1]: Finished scheduler-token.service.3816worker2 # [ 31.974796] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 96ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3817lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3818lb: must succeed: curl -sf localhost:9901/clusters3819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3820lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3821subtest: one node schedules and both builders hold a WorkerSession on it3822lb: must succeed: curl -sf localhost:9901/clusters3823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3824worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3825worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3826lb: must succeed: curl -sf localhost:9901/clusters3827lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3828lb: must succeed: curl -sf localhost:9901/clusters3829lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3830lb: must succeed: curl -sf localhost:9901/clusters3831lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3832lb: must succeed: curl -sf localhost:9901/clusters3833lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3834worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3835worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.07 seconds)3836worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3837worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3838lb: must succeed: curl -sf localhost:9901/clusters3839lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3840lb: must succeed: curl -sf localhost:9901/clusters3841lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3842worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3843worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3844worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3845worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3846worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3847worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3848worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3849worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3850(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.91 seconds)3851subtest: balancer auth: client cert, bearer token, nothing, unknown CN3852worker1 # [ 33.255373] nix-grpc-daemon[832]: ts=2026-09-20T12:47:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:329963853worker1 # [ 33.273350] systemd[1]: Started Nix Daemon.3854worker1 # [ 33.549370] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)3855worker1 # [ 33.590919] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)3856worker1 # [ 33.628042] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 2/5)3857worker1 # [ 33.724805] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 3/5)3858worker1 # [ 33.810321] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 282 ms (attempt 4/5)3859worker1 # [ 34.102424] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3860worker2 # [ 34.219077] nix-grpc-daemon[780]: ts=2026-09-20T12:47:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:517323861worker2 # [ 34.236603] systemd[1]: Started Nix Daemon.3862worker2 # [ 34.530832] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)3863worker2 # [ 34.561108] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)3864worker2 # [ 34.638114] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 159 ms (attempt 2/5)3865worker2 # [ 34.797625] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 130 ms (attempt 3/5)3866worker2 # [ 34.927944] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 654 ms (attempt 4/5)3867worker2 # [ 35.586062] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3868worker2 # [ 35.602875] nix-grpc-daemon[780]: ts=2026-09-20T12:47:50Z level=debug event=path_info_miss path=7mgb5khx78dzs8ml2mm99qhfy7adra5b-nixos-system-client-test error="error: path '/nix/store/7mgb5khx78dzs8ml2mm99qhfy7adra5b-nixos-system-client-test' is required, but there is no substituter that can build it"3869worker1 # [ 35.536866] nix-grpc-daemon[832]: ts=2026-09-20T12:47:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329963870worker1 # [ 35.553678] nix-grpc-daemon[832]: ts=2026-09-20T12:47:50Z level=debug event=path_info_miss path=7mgb5khx78dzs8ml2mm99qhfy7adra5b-nixos-system-client-test error="error: path '/nix/store/7mgb5khx78dzs8ml2mm99qhfy7adra5b-nixos-system-client-test' is required, but there is no substituter that can build it"3871client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3872client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.11 seconds)3873client: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3874client: (finished: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)3875worker1 # [ 36.008204] nix-grpc-daemon[832]: ts=2026-09-20T12:47:50Z level=info event=oidc_jwks provider=mock keys=13876worker1 # [ 36.014672] nix-grpc-daemon[832]: ts=2026-09-20T12:47:50Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:329963877worker2 # [ 36.109873] nix-grpc-daemon[780]: ts=2026-09-20T12:47:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:517323878worker2 # [ 36.123226] nix-grpc-daemon[780]: ts=2026-09-20T12:47:50Z level=debug event=path_info_miss path=7mgb5khx78dzs8ml2mm99qhfy7adra5b-nixos-system-client-test error="error: path '/nix/store/7mgb5khx78dzs8ml2mm99qhfy7adra5b-nixos-system-client-test' is required, but there is no substituter that can build it"3879worker1 # [ 36.050727] nix-grpc-daemon[832]: ts=2026-09-20T12:47:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:329963880worker1 # [ 36.063583] nix-grpc-daemon[832]: ts=2026-09-20T12:47:50Z level=debug event=path_info_miss path=7mgb5khx78dzs8ml2mm99qhfy7adra5b-nixos-system-client-test error="error: path '/nix/store/7mgb5khx78dzs8ml2mm99qhfy7adra5b-nixos-system-client-test' is required, but there is no substituter that can build it"3881client: must succeed: rm -r /var/lib/nix-grpc-store3882client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)3883worker2 # [ 36.441440] nix-grpc-daemon[780]: ts=2026-09-20T12:47:50Z level=info event=denied method=QueryMissing cn=- role=none3884worker2 # [ 36.693951] nix-grpc-daemon[780]: ts=2026-09-20T12:47:51Z level=info event=denied method=QueryMissing cn=stranger role=none3885(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.73 seconds)3886subtest: DAG build is scheduled across workers and lands in the cache3887client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23888client # don't know how to build these paths:3889client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3890worker1 # [ 36.901778] nix-grpc-daemon[832]: ts=2026-09-20T12:47:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:329963891worker1 # [ 36.909773] nix-grpc-daemon[832]: ts=2026-09-20T12:47:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329963892worker2 # [ 37.032898] nix-grpc-daemon[780]: ts=2026-09-20T12:47:51Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03893worker2 # [ 37.038962] nix-grpc-daemon[780]: ts=2026-09-20T12:47:51Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=13894worker1 # [ 37.004655] nix-grpc-daemon[832]: ts=2026-09-20T12:47:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329963895worker1 # [ 37.025436] nix-grpc-daemon[832]: ts=2026-09-20T12:47:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329963896client # copying 1 paths to 'lb:50051'...3897worker1 # [ 37.043132] nix-grpc-daemon[832]: ts=2026-09-20T12:47:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329963898client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3899worker1 # [ 37.073188] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)3900worker1 # [ 37.105356] nix-grpc-daemon[832]: ts=2026-09-20T12:47:51Z level=info event=niks3_push_started pid=9023901worker1 # [ 37.404498] nix-grpc-daemon[902]: time=2026-09-20T12:47:51.897Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""3902worker1 # [ 37.612666] nix-daemon[892]: accepted connection from pid 915, user nix-grpc-daemon (trusted)3903worker1 # [ 37.634056] nix-daemon[892]: reaped child process 919, status = succeeded3904worker1 # [ 37.817246] nix-daemon[892]: accepted connection from pid 921, user nix-grpc-daemon (trusted)3905worker1 # [ 37.837117] nix-daemon[892]: reaped child process 924, status = succeeded3906lb # [ 37.878726] niks3-server[1032]: 2026/09/20 12:47:52 INFO Received uploads request method=POST path=/api/pending_closures3907worker1 # [ 37.889585] nix-grpc-daemon[902]: time=2026-09-20T12:47:52.384Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3908worker1 # [ 37.893884] nix-grpc-daemon[902]: time=2026-09-20T12:47:52.384Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3909lb # [ 37.983285] niks3-server[1032]: 2026/09/20 12:47:52 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3910lb # [ 37.991604] niks3-server[1032]: 2026/09/20 12:47:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3911lb # [ 38.005045] niks3-server[1032]: 2026/09/20 12:47:52 INFO Signed narinfos id=1 count=13912worker1 # [ 37.977176] nix-grpc-daemon[902]: time=2026-09-20T12:47:52.470Z level=INFO msg="Uploading 1 narinfos"3913lb # [ 38.037195] niks3-server[1032]: 2026/09/20 12:47:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3914lb # [ 38.059064] niks3-server[1032]: 2026/09/20 12:47:52 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3915lb # [ 38.067119] niks3-server[1032]: 2026/09/20 12:47:52 INFO Completed upload id=13916worker1 # [ 38.037189] nix-grpc-daemon[902]: time=2026-09-20T12:47:52.531Z level=INFO msg="Upload complete. (622ms)"3917worker1 # [ 38.053126] nix-grpc-daemon[902]: time=2026-09-20T12:47:52.536Z level=WARN msg="Failed to register uploaded object" key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo error="server returned 404: object is not pending upload\n"3918worker1 # [ 38.078759] nix-grpc-daemon[832]: ts=2026-09-20T12:47:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_in=5043919worker1 # [ 38.084507] nix-grpc-daemon[832]: ts=2026-09-20T12:47:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329963920worker1 # [ 38.097109] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)3921worker1 # [ 38.099825] nix-daemon[892]: reaped child process 900, status = succeeded3922client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3923client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3924worker1 # [ 38.111007] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)3925client # farm-a> LOG-farm-a-t13926worker1 # [ 39.349379] nix-daemon[892]: reaped child process 928, status = succeeded3927worker1 # [ 39.547514] nix-daemon[892]: accepted connection from pid 943, user nix-grpc-daemon (trusted)3928worker1 # [ 39.569660] nix-daemon[892]: reaped child process 946, status = succeeded3929lb # [ 39.611628] niks3-server[1032]: 2026/09/20 12:47:54 INFO Received uploads request method=POST path=/api/pending_closures3930worker1 # [ 39.604366] nix-grpc-daemon[902]: time=2026-09-20T12:47:54.097Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3931worker1 # [ 39.606680] nix-grpc-daemon[902]: time=2026-09-20T12:47:54.097Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3932lb # [ 39.697997] niks3-server[1032]: 2026/09/20 12:47:54 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3933lb # [ 39.706500] niks3-server[1032]: 2026/09/20 12:47:54 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3934worker1 # [ 39.687069] nix-grpc-daemon[902]: time=2026-09-20T12:47:54.179Z level=INFO msg="Uploading 1 narinfos"3935lb # [ 39.716840] niks3-server[1032]: 2026/09/20 12:47:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3936lb # [ 39.722425] niks3-server[1032]: 2026/09/20 12:47:54 INFO Signed narinfos id=2 count=13937lb # [ 39.728078] niks3-server[1032]: 2026/09/20 12:47:54 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3938lb # [ 39.736196] niks3-server[1032]: 2026/09/20 12:47:54 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3939lb # [ 39.749418] niks3-server[1032]: 2026/09/20 12:47:54 INFO Completed upload id=23940worker1 # [ 39.720692] nix-grpc-daemon[902]: time=2026-09-20T12:47:54.215Z level=INFO msg="Upload complete. (395ms)"3941lb # [ 39.752077] niks3-server[1032]: 2026/09/20 12:47:54 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo3942worker1 # [ 39.733332] nix-grpc-daemon[832]: ts=2026-09-20T12:47:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=13943worker2 # [ 39.831123] nix-grpc-daemon[780]: ts=2026-09-20T12:47:54Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=03944worker1 # [ 39.758431] nix-daemon[892]: reaped child process 926, status = succeeded3945worker2 # [ 39.837441] nix-grpc-daemon[780]: ts=2026-09-20T12:47:54Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=13946worker1 # [ 39.803294] nix-grpc-daemon[832]: ts=2026-09-20T12:47:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329963947worker1 # [ 39.823152] nix-grpc-daemon[832]: ts=2026-09-20T12:47:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329963948worker1 # [ 39.841987] nix-grpc-daemon[902]: time=2026-09-20T12:47:54.336Z level=INFO msg="All 1 paths already cached"3949client # copying 1 paths to 'lb:50051'...3950worker1 # [ 39.854395] nix-grpc-daemon[832]: ts=2026-09-20T12:47:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329963951client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...3952worker1 # [ 39.880323] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)3953worker1 # [ 40.117704] nix-daemon[892]: accepted connection from pid 950, user nix-grpc-daemon (trusted)3954worker1 # [ 40.138668] nix-daemon[892]: reaped child process 953, status = succeeded3955worker1 # [ 40.324961] nix-daemon[892]: accepted connection from pid 955, user nix-grpc-daemon (trusted)3956worker1 # [ 40.343869] nix-daemon[892]: reaped child process 958, status = succeeded3957lb # [ 40.385042] niks3-server[1032]: 2026/09/20 12:47:54 INFO Received uploads request method=POST path=/api/pending_closures3958worker1 # [ 40.370567] nix-grpc-daemon[902]: time=2026-09-20T12:47:54.864Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3959worker1 # [ 40.372213] nix-grpc-daemon[902]: time=2026-09-20T12:47:54.864Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"3960lb # [ 40.425481] niks3-server[1032]: 2026/09/20 12:47:54 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst3961worker1 # [ 40.416087] nix-grpc-daemon[902]: time=2026-09-20T12:47:54.910Z level=INFO msg="Uploading 1 narinfos"3962lb # [ 40.443895] niks3-server[1032]: 2026/09/20 12:47:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign3963lb # [ 40.453454] niks3-server[1032]: 2026/09/20 12:47:54 INFO Signed narinfos id=3 count=13964lb # [ 40.464165] niks3-server[1032]: 2026/09/20 12:47:54 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls3965lb # [ 40.472354] niks3-server[1032]: 2026/09/20 12:47:54 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete3966lb # [ 40.479102] niks3-server[1032]: 2026/09/20 12:47:54 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo3967lb # [ 40.483228] niks3-server[1032]: 2026/09/20 12:47:54 INFO Completed upload id=33968worker1 # [ 40.454504] nix-grpc-daemon[902]: time=2026-09-20T12:47:54.949Z level=INFO msg="Upload complete. (552ms)"3969worker1 # [ 40.464439] nix-grpc-daemon[832]: ts=2026-09-20T12:47:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_in=6243970worker1 # [ 40.484996] nix-grpc-daemon[832]: ts=2026-09-20T12:47:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329963971worker1 # [ 40.497816] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)3972worker1 # [ 40.501392] nix-daemon[892]: reaped child process 948, status = succeeded3973client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...3974worker1 # [ 40.516766] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)3975client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3976client # farm-b> LOG-farm-b-t13977worker1 # [ 41.165528] nix-daemon[892]: reaped child process 962, status = succeeded3978worker1 # [ 41.348536] nix-daemon[892]: accepted connection from pid 980, user nix-grpc-daemon (trusted)3979worker1 # [ 41.371258] nix-daemon[892]: reaped child process 983, status = succeeded3980lb # [ 41.413328] niks3-server[1032]: 2026/09/20 12:47:55 INFO Received uploads request method=POST path=/api/pending_closures3981worker1 # [ 41.398120] nix-grpc-daemon[902]: time=2026-09-20T12:47:55.892Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3982worker1 # [ 41.401240] nix-grpc-daemon[902]: time=2026-09-20T12:47:55.894Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"3983lb # [ 41.467522] niks3-server[1032]: 2026/09/20 12:47:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign3984worker1 # [ 41.446402] nix-grpc-daemon[902]: time=2026-09-20T12:47:55.940Z level=INFO msg="Uploading 1 narinfos"3985lb # [ 41.479058] niks3-server[1032]: 2026/09/20 12:47:55 INFO Signed narinfos id=4 count=13986lb # [ 41.484282] niks3-server[1032]: 2026/09/20 12:47:55 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3987lb # [ 41.487862] niks3-server[1032]: 2026/09/20 12:47:55 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst3988lb # [ 41.502878] niks3-server[1032]: 2026/09/20 12:47:55 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls3989lb # [ 41.510269] niks3-server[1032]: 2026/09/20 12:47:55 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete3990lb # [ 41.513640] niks3-server[1032]: 2026/09/20 12:47:55 INFO Completed upload id=43991lb # [ 41.516482] niks3-server[1032]: 2026/09/20 12:47:55 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo3992worker1 # [ 41.479303] nix-grpc-daemon[902]: time=2026-09-20T12:47:55.973Z level=INFO msg="Upload complete. (346ms)"3993worker2 # [ 41.581533] nix-grpc-daemon[780]: ts=2026-09-20T12:47:56Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=03994worker1 # [ 41.493632] nix-grpc-daemon[832]: ts=2026-09-20T12:47:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=13995worker1 # [ 41.517017] nix-daemon[892]: reaped child process 960, status = succeeded3996worker1 # [ 41.556442] nix-grpc-daemon[832]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329963997worker1 # [ 41.574101] nix-grpc-daemon[832]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329963998worker1 # [ 41.590771] nix-grpc-daemon[902]: time=2026-09-20T12:47:56.085Z level=INFO msg="All 2 paths already cached"3999client # copying 1 paths to 'lb:50051'...4000worker1 # [ 41.602344] nix-grpc-daemon[832]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329964001client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4002worker1 # [ 41.627355] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4003worker1 # [ 41.855852] nix-daemon[892]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4004worker1 # [ 41.875581] nix-daemon[892]: reaped child process 990, status = succeeded4005lb # [ 41.980572] envoy[921]: conn peer=[2001:db8:1::1]:50802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=23494006lb # [ 41.986126] envoy[921]: conn peer=[2001:db8:1::1]:50818 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=814007lb # [ 41.991025] envoy[921]: conn peer=[2001:db8:1::1]:50826 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134008lb # [ 41.996692] envoy[921]: conn peer=[2001:db8:1::1]:50838 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134009worker1 # [ 42.064151] nix-daemon[892]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4010worker1 # [ 42.083194] nix-daemon[892]: reaped child process 995, status = succeeded4011lb # [ 42.124491] niks3-server[1032]: 2026/09/20 12:47:56 INFO Received uploads request method=POST path=/api/pending_closures4012worker1 # [ 42.109994] nix-grpc-daemon[902]: time=2026-09-20T12:47:56.604Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4013worker1 # [ 42.111520] nix-grpc-daemon[902]: time=2026-09-20T12:47:56.604Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4014worker2 # [ 41.588554] nix-grpc-daemon[780]: ts=2026-09-20T12:47:56Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14015lb # [ 42.163921] niks3-server[1032]: 2026/09/20 12:47:56 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4016lb # [ 42.182580] niks3-server[1032]: 2026/09/20 12:47:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4017worker1 # [ 42.160197] nix-grpc-daemon[902]: time=2026-09-20T12:47:56.654Z level=INFO msg="Uploading 1 narinfos"4018lb # [ 42.196307] niks3-server[1032]: 2026/09/20 12:47:56 INFO Signed narinfos id=5 count=14019lb # [ 42.203344] niks3-server[1032]: 2026/09/20 12:47:56 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4020lb # [ 42.213270] niks3-server[1032]: 2026/09/20 12:47:56 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4021lb # [ 42.225909] niks3-server[1032]: 2026/09/20 12:47:56 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4022lb # [ 42.233616] niks3-server[1032]: 2026/09/20 12:47:56 INFO Completed upload id=54023worker1 # [ 42.204730] nix-grpc-daemon[902]: time=2026-09-20T12:47:56.699Z level=INFO msg="Upload complete. (557ms)"4024worker1 # [ 42.216414] nix-grpc-daemon[832]: ts=2026-09-20T12:47:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_in=7524025worker1 # [ 42.230234] nix-grpc-daemon[832]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964026worker1 # [ 42.245893] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4027worker1 # [ 42.252715] nix-daemon[892]: reaped child process 985, status = succeeded4028client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4029worker1 # [ 42.267712] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4030client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4031worker1 # [ 43.296650] nix-daemon[892]: reaped child process 999, status = succeeded4032client # farm-top> LOG-farm-top-t14033worker1 # [ 43.465479] nix-daemon[892]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4034worker1 # [ 43.486053] nix-daemon[892]: reaped child process 1017, status = succeeded4035lb # [ 43.530261] niks3-server[1032]: 2026/09/20 12:47:57 INFO Received uploads request method=POST path=/api/pending_closures4036worker1 # [ 43.515642] nix-grpc-daemon[902]: time=2026-09-20T12:47:58.009Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4037worker1 # [ 43.519100] nix-grpc-daemon[902]: time=2026-09-20T12:47:58.009Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4038lb # [ 43.579657] niks3-server[1032]: 2026/09/20 12:47:58 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4039lb # [ 43.593703] niks3-server[1032]: 2026/09/20 12:47:58 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4040lb # [ 43.608811] niks3-server[1032]: 2026/09/20 12:47:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4041worker1 # [ 43.584606] nix-grpc-daemon[902]: time=2026-09-20T12:47:58.078Z level=INFO msg="Uploading 1 narinfos"4042lb # [ 43.618005] niks3-server[1032]: 2026/09/20 12:47:58 INFO Signed narinfos id=6 count=14043lb # [ 43.619981] niks3-server[1032]: 2026/09/20 12:47:58 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4044lb # [ 43.628758] niks3-server[1032]: 2026/09/20 12:47:58 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4045lb # [ 43.636676] niks3-server[1032]: 2026/09/20 12:47:58 INFO Completed upload id=64046worker1 # [ 43.607651] nix-grpc-daemon[902]: time=2026-09-20T12:47:58.102Z level=INFO msg="Upload complete. (324ms)"4047lb # [ 43.639969] niks3-server[1032]: 2026/09/20 12:47:58 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4048worker1 # [ 43.621450] nix-grpc-daemon[832]: ts=2026-09-20T12:47:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14049worker1 # [ 43.639764] nix-daemon[892]: reaped child process 997, status = succeeded4050client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.03 seconds)4051client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4052client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.24 seconds)4053client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4054client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4055client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4056client # copying 3 paths...4057client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4058client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4059client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4060client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.35 seconds)4061(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.75 seconds)4062subtest: repeat is answered Cached by the scheduler without building4063worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4064worker1 # finding garbage collector roots...4065worker1 # deleting specified paths...4066worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4067worker1 # deleting unused links...4068worker1 # note: hard linking is currently saving 0.0 KiB4069worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds)4070worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4071worker2 # finding garbage collector roots...4072worker2 # deleting specified paths...4073worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4074worker2 # deleting unused links...4075worker2 # note: hard linking is currently saving 0.0 KiB4076worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.32 seconds)4077client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24078worker2 # [ 45.363812] nix-grpc-daemon[780]: ts=2026-09-20T12:47:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517324079client # don't know how to build these paths:4080client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4081worker2 # [ 45.375741] nix-grpc-daemon[780]: ts=2026-09-20T12:47:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517324082worker2 # [ 45.413819] nix-grpc-daemon[780]: ts=2026-09-20T12:47:59Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14083client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.36 seconds)4084client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4085client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.23 seconds)4086lb: must succeed: curl -sf localhost:9901/clusters4087lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4088lb: must succeed: curl -sf localhost:9901/clusters4089lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4090worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4091worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4092(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.56 seconds)4093subtest: inputs already in the cache are not sent to the scheduler4094lb: must succeed: curl -sf localhost:9901/clusters4095lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4096lb: must succeed: curl -sf localhost:9901/clusters4097lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4098worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4099worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4100client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24101client # don't know how to build these paths:4102client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4103worker1 # [ 46.443622] nix-grpc-daemon[832]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:329964104worker1 # [ 46.451842] nix-grpc-daemon[832]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964105worker1 # [ 46.460357] nix-grpc-daemon[902]: time=2026-09-20T12:48:00.952Z level=INFO msg="All 2 paths already cached"4106worker2 # [ 46.554561] nix-grpc-daemon[780]: ts=2026-09-20T12:48:01Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04107worker1 # [ 46.520197] nix-grpc-daemon[832]: ts=2026-09-20T12:48:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964108worker1 # [ 46.539816] nix-grpc-daemon[832]: ts=2026-09-20T12:48:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964109worker1 # [ 46.557786] nix-grpc-daemon[902]: time=2026-09-20T12:48:01.052Z level=INFO msg="All 2 paths already cached"4110client # copying 1 paths to 'lb:50051'...4111client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4112worker1 # [ 46.569949] nix-grpc-daemon[832]: ts=2026-09-20T12:48:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329964113worker1 # [ 46.586911] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4114worker1 # [ 46.815409] nix-daemon[892]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)4115worker1 # [ 46.837095] nix-daemon[892]: reaped child process 1038, status = succeeded4116worker1 # [ 47.030474] nix-daemon[892]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)4117worker1 # [ 47.049994] nix-daemon[892]: reaped child process 1043, status = succeeded4118lb # [ 47.096157] niks3-server[1032]: 2026/09/20 12:48:01 INFO Received uploads request method=POST path=/api/pending_closures4119worker1 # [ 47.077938] nix-grpc-daemon[902]: time=2026-09-20T12:48:01.572Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4120worker1 # [ 47.079459] nix-grpc-daemon[902]: time=2026-09-20T12:48:01.572Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4121worker2 # [ 46.561036] nix-grpc-daemon[780]: ts=2026-09-20T12:48:01Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14122lb # [ 47.136181] niks3-server[1032]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4123worker1 # [ 47.121125] nix-grpc-daemon[902]: time=2026-09-20T12:48:01.615Z level=INFO msg="Uploading 1 narinfos"4124lb # [ 47.155398] niks3-server[1032]: 2026/09/20 12:48:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4125lb # [ 47.163334] niks3-server[1032]: 2026/09/20 12:48:01 INFO Signed narinfos id=7 count=14126lb # [ 47.168814] niks3-server[1032]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4127lb # [ 47.176517] niks3-server[1032]: 2026/09/20 12:48:01 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4128lb # [ 47.184185] niks3-server[1032]: 2026/09/20 12:48:01 INFO Completed upload id=74129worker1 # [ 47.155274] nix-grpc-daemon[902]: time=2026-09-20T12:48:01.649Z level=INFO msg="Upload complete. (548ms)"4130lb # [ 47.187699] niks3-server[1032]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4131worker1 # [ 47.164350] nix-grpc-daemon[832]: ts=2026-09-20T12:48:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_in=7524132worker1 # [ 47.177089] nix-grpc-daemon[832]: ts=2026-09-20T12:48:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964133worker1 # [ 47.190392] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4134worker1 # [ 47.196368] nix-daemon[892]: reaped child process 1033, status = succeeded4135client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4136worker1 # [ 47.210593] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4137client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4138client # farm-top> LOG-farm-top-t1b4139worker1 # [ 48.654894] nix-daemon[892]: reaped child process 1047, status = succeeded4140worker1 # [ 48.839742] nix-daemon[892]: accepted connection from pid 1063, user nix-grpc-daemon (trusted)4141worker1 # [ 48.859343] nix-daemon[892]: reaped child process 1066, status = succeeded4142lb # [ 48.902523] niks3-server[1032]: 2026/09/20 12:48:03 INFO Received uploads request method=POST path=/api/pending_closures4143worker1 # [ 48.888108] nix-grpc-daemon[902]: time=2026-09-20T12:48:03.379Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4144worker1 # [ 48.889838] nix-grpc-daemon[902]: time=2026-09-20T12:48:03.379Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4145lb # [ 48.957391] niks3-server[1032]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4146lb # [ 48.971192] niks3-server[1032]: 2026/09/20 12:48:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4147worker1 # [ 48.943085] nix-grpc-daemon[902]: time=2026-09-20T12:48:03.437Z level=INFO msg="Uploading 1 narinfos"4148lb # [ 48.978369] niks3-server[1032]: 2026/09/20 12:48:03 INFO Signed narinfos id=8 count=14149lb # [ 48.985031] niks3-server[1032]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4150lb # [ 48.993136] niks3-server[1032]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4151lb # [ 49.003133] niks3-server[1032]: 2026/09/20 12:48:03 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4152worker1 # [ 48.978837] nix-grpc-daemon[902]: time=2026-09-20T12:48:03.473Z level=INFO msg="Upload complete. (353ms)"4153lb # [ 49.010584] niks3-server[1032]: 2026/09/20 12:48:03 INFO Completed upload id=84154lb # [ 49.013199] niks3-server[1032]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4155worker1 # [ 48.993927] nix-grpc-daemon[832]: ts=2026-09-20T12:48:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14156worker1 # [ 49.012816] nix-daemon[892]: reaped child process 1045, status = succeeded4157client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.87 seconds)4158client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4159client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.26 seconds)4160lb: must succeed: curl -sf localhost:9901/clusters4161lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4162lb: must succeed: curl -sf localhost:9901/clusters4163lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4164worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4165worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4166(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.57 seconds)4167subtest: two clients wanting the same drv build it once4168client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4169client # [ 49.638797] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4170client # Running as unit: dedup1.service; invocation ID: 5c22513bfc41467cbf2c580bfb31a24c4171client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4172client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4173worker2 # [ 49.954645] nix-grpc-daemon[780]: ts=2026-09-20T12:48:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517324174client # [ 49.915458] nix[880]: don't know how to build these paths:4175client # [ 49.916776] nix[880]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4176worker2 # [ 49.977821] nix-grpc-daemon[780]: ts=2026-09-20T12:48:04Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04177worker2 # [ 49.985894] nix-grpc-daemon[780]: ts=2026-09-20T12:48:04Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14178worker2 # [ 50.024281] nix-grpc-daemon[780]: ts=2026-09-20T12:48:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517324179worker2 # [ 50.049669] nix-grpc-daemon[780]: ts=2026-09-20T12:48:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517324180client # [ 50.018468] nix[880]: copying 1 paths to 'lb:50051'...4181worker2 # [ 50.067290] nix-grpc-daemon[780]: ts=2026-09-20T12:48:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517324182client # [ 50.023585] nix[880]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4183worker2 # [ 50.086745] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4184worker2 # [ 50.119151] nix-grpc-daemon[780]: ts=2026-09-20T12:48:04Z level=info event=niks3_push_started pid=9124185worker2 # [ 50.405622] nix-grpc-daemon[912]: time=2026-09-20T12:48:04.897Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4186worker2 # [ 50.608525] nix-daemon[861]: accepted connection from pid 925, user nix-grpc-daemon (trusted)4187worker2 # [ 50.631694] nix-daemon[861]: reaped child process 928, status = succeeded4188worker2 # [ 50.819885] nix-daemon[861]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4189client # [ 50.765712] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4190worker2 # [ 50.843479] nix-daemon[861]: reaped child process 933, status = succeeded4191client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4192lb # [ 50.805587] niks3-server[1032]: 2026/09/20 12:48:05 INFO Received uploads request method=POST path=/api/pending_closures4193worker2 # [ 50.874193] nix-grpc-daemon[912]: time=2026-09-20T12:48:05.368Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4194worker2 # [ 50.878002] nix-grpc-daemon[912]: time=2026-09-20T12:48:05.368Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4195lb # [ 50.862900] niks3-server[1032]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4196lb # [ 50.874243] niks3-server[1032]: 2026/09/20 12:48:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4197worker2 # [ 50.932537] nix-grpc-daemon[912]: time=2026-09-20T12:48:05.426Z level=INFO msg="Uploading 1 narinfos"4198lb # [ 50.887150] niks3-server[1032]: 2026/09/20 12:48:05 INFO Signed narinfos id=9 count=14199lb # [ 50.893695] niks3-server[1032]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4200lb # [ 50.905210] niks3-server[1032]: 2026/09/20 12:48:05 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4201worker2 # [ 50.962851] nix-grpc-daemon[912]: time=2026-09-20T12:48:05.456Z level=INFO msg="Upload complete. (549ms)"4202lb # [ 50.912913] niks3-server[1032]: 2026/09/20 12:48:05 INFO Completed upload id=94203lb # [ 50.916557] niks3-server[1032]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4204worker2 # [ 50.973189] nix-grpc-daemon[780]: ts=2026-09-20T12:48:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=0 paths=1 nar_bytes_in=4324205worker2 # [ 50.984306] nix-grpc-daemon[780]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517324206worker2 # [ 50.994086] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4207worker2 # [ 50.999426] nix-daemon[861]: reaped child process 910, status = succeeded4208client # Running as unit: dedup2.service; invocation ID: acde04d10cd24a799106a0c4b8d0993a4209worker2 # [ 51.010911] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4210client # [ 50.962394] nix[880]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4211worker2 # [ 51.142289] nix-grpc-daemon[780]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517324212client # [ 51.111160] nix[896]: this derivation will be built:4213worker1 # [ 51.081915] nix-grpc-daemon[832]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329964214worker1 # [ 51.103260] nix-grpc-daemon[832]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' from 'http://lb:5751'...4215client # [ 51.165249] nix[896]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4216worker2 # [ 51.228183] nix-grpc-daemon[780]: ts=2026-09-20T12:48:05Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04217worker2 # [ 51.245001] nix-grpc-daemon[780]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517324218client # [ 51.205285] nix[896]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4219lb # [ 51.979825] envoy[921]: conn peer=[2001:db8:1::1]:50850 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9536 tx=3885 ms=67634220lb # [ 51.985296] envoy[921]: conn peer=[2001:db8:1::1]:35586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=481 ms=774221lb # [ 51.990321] envoy[921]: conn peer=[2001:db8:1::1]:35592 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1728 ms=25664222client: must succeed: sleep 24223client: (finished: must succeed: sleep 2, in 2.07 seconds)4224client: waiting for success: ! systemctl is-active dedup1 dedup24225worker2 # [ 54.653973] nix-daemon[861]: reaped child process 941, status = succeeded4226worker2 # [ 54.845446] nix-daemon[861]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4227worker2 # [ 54.864381] nix-daemon[861]: reaped child process 971, status = succeeded4228lb # [ 54.825548] niks3-server[1032]: 2026/09/20 12:48:09 INFO Received uploads request method=POST path=/api/pending_closures4229worker2 # [ 54.889640] nix-grpc-daemon[912]: time=2026-09-20T12:48:09.382Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4230worker2 # [ 54.891383] nix-grpc-daemon[912]: time=2026-09-20T12:48:09.382Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4231lb # [ 54.878185] niks3-server[1032]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4232lb # [ 54.889495] niks3-server[1032]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4233lb # [ 54.901181] niks3-server[1032]: 2026/09/20 12:48:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4234lb # [ 54.907037] niks3-server[1032]: 2026/09/20 12:48:09 INFO Signed narinfos id=10 count=14235worker2 # [ 54.959392] nix-grpc-daemon[912]: time=2026-09-20T12:48:09.451Z level=INFO msg="Uploading 1 narinfos"4236lb # [ 54.909758] niks3-server[1032]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4237lb # [ 54.920112] niks3-server[1032]: 2026/09/20 12:48:09 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4238lb # [ 54.928887] niks3-server[1032]: 2026/09/20 12:48:09 INFO Completed upload id=104239worker2 # [ 54.980978] nix-grpc-daemon[912]: time=2026-09-20T12:48:09.475Z level=INFO msg="Upload complete. (349ms)"4240lb # [ 54.932034] niks3-server[1032]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4241worker2 # [ 54.997383] nix-grpc-daemon[780]: ts=2026-09-20T12:48:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14242worker2 # [ 55.012297] nix-grpc-daemon[780]: ts=2026-09-20T12:48:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14243worker2 # [ 55.017688] nix-daemon[861]: reaped child process 939, status = succeeded4244client # [ 54.994257] systemd[1]: dedup1.service: Deactivated successfully.4245client # [ 55.000291] systemd[1]: dedup1.service: Consumed 176ms CPU time over 5.359s wall clock time, 39.2M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.4246client # [ 55.039758] systemd[1]: dedup2.service: Deactivated successfully.4247client # [ 55.042564] systemd[1]: dedup2.service: Consumed 191ms CPU time over 4.277s wall clock time, 29M memory peak, 8K incoming IP traffic, 6.7K outgoing IP traffic.4248client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.15 seconds)4249client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4250client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4251worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4252worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4253worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4254worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4255worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4256worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4257worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4258worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4259(finished: subtest: two clients wanting the same drv build it once, in 6.36 seconds)4260subtest: build hook: nix-daemon with builders = grpc://lb4261client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4262client # [ 56.156693] systemd[1]: Started Nix Daemon.4263client # [ 56.354863] nix-daemon[938]: accepted connection from pid 935, user root (trusted)4264worker2 # [ 56.614445] nix-grpc-daemon[780]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:517324265worker1 # [ 56.547404] nix-grpc-daemon[832]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964266worker2 # [ 56.648286] nix-grpc-daemon[780]: ts=2026-09-20T12:48:11Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04267worker2 # [ 56.654857] nix-grpc-daemon[780]: ts=2026-09-20T12:48:11Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14268worker1 # [ 56.620145] nix-grpc-daemon[832]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964269worker1 # [ 56.637202] nix-grpc-daemon[832]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964270worker1 # [ 56.653865] nix-grpc-daemon[832]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329964271worker1 # [ 56.669480] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4272worker1 # [ 56.880864] nix-daemon[892]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)4273worker1 # [ 56.902293] nix-daemon[892]: reaped child process 1095, status = succeeded4274worker1 # [ 57.078717] nix-daemon[892]: accepted connection from pid 1097, user nix-grpc-daemon (trusted)4275worker1 # [ 57.099058] nix-daemon[892]: reaped child process 1100, status = succeeded4276lb # [ 57.141263] niks3-server[1032]: 2026/09/20 12:48:11 INFO Received uploads request method=POST path=/api/pending_closures4277worker1 # [ 57.122240] nix-grpc-daemon[902]: time=2026-09-20T12:48:11.616Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4278worker1 # [ 57.123758] nix-grpc-daemon[902]: time=2026-09-20T12:48:11.616Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4279lb # [ 57.169688] niks3-server[1032]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4280lb # [ 57.186961] niks3-server[1032]: 2026/09/20 12:48:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4281worker1 # [ 57.164449] nix-grpc-daemon[902]: time=2026-09-20T12:48:11.658Z level=INFO msg="Uploading 1 narinfos"4282lb # [ 57.200769] niks3-server[1032]: 2026/09/20 12:48:11 INFO Signed narinfos id=11 count=14283lb # [ 57.208300] niks3-server[1032]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4284lb # [ 57.218819] niks3-server[1032]: 2026/09/20 12:48:11 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4285lb # [ 57.224529] niks3-server[1032]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4286lb # [ 57.229180] niks3-server[1032]: 2026/09/20 12:48:11 INFO Completed upload id=114287worker1 # [ 57.200268] nix-grpc-daemon[902]: time=2026-09-20T12:48:11.694Z level=INFO msg="Upload complete. (513ms)"4288worker1 # [ 57.209663] nix-grpc-daemon[832]: ts=2026-09-20T12:48:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_in=5124289worker1 # [ 57.224288] nix-grpc-daemon[832]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964290worker1 # [ 57.231696] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4291worker1 # [ 57.237220] nix-daemon[892]: reaped child process 1090, status = succeeded4292worker1 # [ 57.251917] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4293worker1 # [ 58.081464] nix-daemon[892]: reaped child process 1104, status = succeeded4294worker1 # [ 58.241780] nix-daemon[892]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4295worker1 # [ 58.262406] nix-daemon[892]: reaped child process 1122, status = succeeded4296lb # [ 58.306423] niks3-server[1032]: 2026/09/20 12:48:12 INFO Received uploads request method=POST path=/api/pending_closures4297worker1 # [ 58.288432] nix-grpc-daemon[902]: time=2026-09-20T12:48:12.782Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4298worker1 # [ 58.289950] nix-grpc-daemon[902]: time=2026-09-20T12:48:12.782Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4299lb # [ 58.355369] niks3-server[1032]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4300worker1 # [ 58.339738] nix-grpc-daemon[902]: time=2026-09-20T12:48:12.834Z level=INFO msg="Uploading 1 narinfos"4301lb # [ 58.369202] niks3-server[1032]: 2026/09/20 12:48:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4302lb # [ 58.379781] niks3-server[1032]: 2026/09/20 12:48:12 INFO Signed narinfos id=12 count=14303lb # [ 58.386556] niks3-server[1032]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4304lb # [ 58.391704] niks3-server[1032]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4305lb # [ 58.399674] niks3-server[1032]: 2026/09/20 12:48:12 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4306lb # [ 58.404182] niks3-server[1032]: 2026/09/20 12:48:12 INFO Completed upload id=124307worker1 # [ 58.375294] nix-grpc-daemon[902]: time=2026-09-20T12:48:12.869Z level=INFO msg="Upload complete. (310ms)"4308lb # [ 58.406671] niks3-server[1032]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4309worker1 # [ 58.389200] nix-grpc-daemon[832]: ts=2026-09-20T12:48:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14310worker2 # [ 58.473938] nix-grpc-daemon[780]: ts=2026-09-20T12:48:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:517324311worker1 # [ 58.407157] nix-daemon[892]: reaped child process 1102, status = succeeded4312worker2 # [ 58.485214] nix-grpc-daemon[780]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4313worker2 # [ 58.533456] nix-grpc-daemon[780]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:517324314worker2 # [ 58.540530] nix-grpc-daemon[780]: ts=2026-09-20T12:48:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=0 paths=1 nar_bytes_out=1284315worker1 # [ 58.763352] nix-grpc-daemon[832]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329964316worker1 # [ 58.772798] nix-grpc-daemon[832]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964317worker1 # [ 58.784590] nix-grpc-daemon[902]: time=2026-09-20T12:48:13.279Z level=INFO msg="All 1 paths already cached"4318worker2 # [ 58.887087] nix-grpc-daemon[780]: ts=2026-09-20T12:48:13Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04319worker2 # [ 58.893655] nix-grpc-daemon[780]: ts=2026-09-20T12:48:13Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14320worker1 # [ 58.816996] nix-grpc-daemon[832]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964321worker1 # [ 58.835131] nix-grpc-daemon[832]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964322worker1 # [ 58.860061] nix-grpc-daemon[902]: time=2026-09-20T12:48:13.354Z level=INFO msg="All 1 paths already cached"4323worker1 # [ 58.870595] nix-grpc-daemon[832]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329964324worker1 # [ 58.885832] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4325worker1 # [ 59.097122] nix-daemon[892]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)4326worker1 # [ 59.118414] nix-daemon[892]: reaped child process 1129, status = succeeded4327worker1 # [ 59.295329] nix-daemon[892]: accepted connection from pid 1131, user nix-grpc-daemon (trusted)4328worker1 # [ 59.313480] nix-daemon[892]: reaped child process 1134, status = succeeded4329lb # [ 59.357274] niks3-server[1032]: 2026/09/20 12:48:13 INFO Received uploads request method=POST path=/api/pending_closures4330worker1 # [ 59.339071] nix-grpc-daemon[902]: time=2026-09-20T12:48:13.833Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4331worker1 # [ 59.340682] nix-grpc-daemon[902]: time=2026-09-20T12:48:13.833Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4332lb # [ 59.392507] niks3-server[1032]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4333worker1 # [ 59.377765] nix-grpc-daemon[902]: time=2026-09-20T12:48:13.872Z level=INFO msg="Uploading 1 narinfos"4334lb # [ 59.406780] niks3-server[1032]: 2026/09/20 12:48:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4335lb # [ 59.416348] niks3-server[1032]: 2026/09/20 12:48:13 INFO Signed narinfos id=13 count=14336lb # [ 59.419158] niks3-server[1032]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4337lb # [ 59.431597] niks3-server[1032]: 2026/09/20 12:48:13 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4338worker1 # [ 59.407574] nix-grpc-daemon[902]: time=2026-09-20T12:48:13.902Z level=INFO msg="Upload complete. (503ms)"4339lb # [ 59.437128] niks3-server[1032]: 2026/09/20 12:48:13 INFO Completed upload id=134340lb # [ 59.441020] niks3-server[1032]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4341worker1 # [ 59.417774] nix-grpc-daemon[832]: ts=2026-09-20T12:48:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_in=6244342worker1 # [ 59.431034] nix-grpc-daemon[832]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964343worker1 # [ 59.437708] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4344worker1 # [ 59.442645] nix-daemon[892]: reaped child process 1124, status = succeeded4345worker1 # [ 59.457025] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4346worker1 # [ 60.903759] nix-daemon[892]: reaped child process 1138, status = succeeded4347worker1 # [ 61.088397] nix-daemon[892]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)4348worker1 # [ 61.107449] nix-daemon[892]: reaped child process 1156, status = succeeded4349lb # [ 61.149705] niks3-server[1032]: 2026/09/20 12:48:15 INFO Received uploads request method=POST path=/api/pending_closures4350worker1 # [ 61.132471] nix-grpc-daemon[902]: time=2026-09-20T12:48:15.625Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4351worker1 # [ 61.134050] nix-grpc-daemon[902]: time=2026-09-20T12:48:15.625Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4352lb # [ 61.188243] niks3-server[1032]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4353lb # [ 61.208465] niks3-server[1032]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4354worker1 # [ 61.196891] nix-grpc-daemon[902]: time=2026-09-20T12:48:15.691Z level=INFO msg="Uploading 1 narinfos"4355lb # [ 61.221447] niks3-server[1032]: 2026/09/20 12:48:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4356lb # [ 61.230200] niks3-server[1032]: 2026/09/20 12:48:15 INFO Signed narinfos id=14 count=14357lb # [ 61.235522] niks3-server[1032]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4358lb # [ 61.244622] niks3-server[1032]: 2026/09/20 12:48:15 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4359lb # [ 61.250095] niks3-server[1032]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4360lb # [ 61.254643] niks3-server[1032]: 2026/09/20 12:48:15 INFO Completed upload id=144361worker1 # [ 61.225463] nix-grpc-daemon[902]: time=2026-09-20T12:48:15.720Z level=INFO msg="Upload complete. (357ms)"4362worker1 # [ 61.237285] nix-grpc-daemon[832]: ts=2026-09-20T12:48:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14363worker1 # [ 61.256431] nix-grpc-daemon[832]: ts=2026-09-20T12:48:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329964364worker1 # [ 61.263105] nix-daemon[892]: reaped child process 1136, status = succeeded4365worker1 # [ 61.275885] nix-grpc-daemon[832]: ts=2026-09-20T12:48:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329964366worker1 # [ 61.282652] nix-grpc-daemon[832]: ts=2026-09-20T12:48:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_out=1764367worker2 # [ 61.672820] nix-grpc-daemon[780]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:517324368worker1 # [ 61.594298] nix-grpc-daemon[832]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964369worker1 # [ 61.604942] nix-grpc-daemon[902]: time=2026-09-20T12:48:16.099Z level=INFO msg="All 2 paths already cached"4370worker2 # [ 61.704292] nix-grpc-daemon[780]: ts=2026-09-20T12:48:16Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04371worker2 # [ 61.710741] nix-grpc-daemon[780]: ts=2026-09-20T12:48:16Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14372worker1 # [ 61.633304] nix-grpc-daemon[832]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964373worker1 # [ 61.649028] nix-grpc-daemon[832]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964374worker1 # [ 61.663589] nix-grpc-daemon[902]: time=2026-09-20T12:48:16.158Z level=INFO msg="All 2 paths already cached"4375worker1 # [ 61.673272] nix-grpc-daemon[832]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329964376worker1 # [ 61.687048] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4377worker1 # [ 61.899985] nix-daemon[892]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)4378worker1 # [ 61.922241] nix-daemon[892]: reaped child process 1163, status = succeeded4379lb # [ 61.978271] envoy[921]: conn peer=[2001:db8:1::1]:35596 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1375 ms=50544380lb # [ 61.983416] envoy[921]: conn peer=[2001:db8:1::1]:49374 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1487 ms=39114381lb # [ 61.988474] envoy[921]: conn peer=[2001:db8:1::1]:49390 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3408 tx=1785 ms=19884382lb # [ 61.993436] envoy[921]: conn peer=[2001:db8:1::1]:49392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=333 ms=724383lb # [ 61.999181] envoy[921]: conn peer=[2001:db8:1::1]:49394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=1877 ms=25764384lb # [ 62.005329] envoy[921]: conn peer=[2001:db8:1::1]:53012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=704385worker1 # [ 62.099910] nix-daemon[892]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)4386worker1 # [ 62.118133] nix-daemon[892]: reaped child process 1168, status = succeeded4387lb # [ 62.160974] niks3-server[1032]: 2026/09/20 12:48:16 INFO Received uploads request method=POST path=/api/pending_closures4388worker1 # [ 62.141565] nix-grpc-daemon[902]: time=2026-09-20T12:48:16.635Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4389worker1 # [ 62.143030] nix-grpc-daemon[902]: time=2026-09-20T12:48:16.635Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4390lb # [ 62.192707] niks3-server[1032]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4391worker1 # [ 62.173133] nix-grpc-daemon[902]: time=2026-09-20T12:48:16.667Z level=INFO msg="Uploading 1 narinfos"4392lb # [ 62.207535] niks3-server[1032]: 2026/09/20 12:48:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4393lb # [ 62.218298] niks3-server[1032]: 2026/09/20 12:48:16 INFO Signed narinfos id=15 count=14394lb # [ 62.226939] niks3-server[1032]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4395worker1 # [ 62.202843] nix-grpc-daemon[902]: time=2026-09-20T12:48:16.697Z level=INFO msg="Upload complete. (498ms)"4396lb # [ 62.232768] niks3-server[1032]: 2026/09/20 12:48:16 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4397lb # [ 62.238650] niks3-server[1032]: 2026/09/20 12:48:16 INFO Completed upload id=154398lb # [ 62.242514] niks3-server[1032]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4399worker1 # [ 62.209223] nix-grpc-daemon[832]: ts=2026-09-20T12:48:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_in=7524400worker1 # [ 62.224467] nix-grpc-daemon[832]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964401worker1 # [ 62.230141] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4402worker1 # [ 62.234937] nix-daemon[892]: reaped child process 1158, status = succeeded4403worker1 # [ 62.246582] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4404worker1 # [ 63.484575] nix-daemon[892]: reaped child process 1172, status = succeeded4405worker1 # [ 63.658741] nix-daemon[892]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)4406worker1 # [ 63.679625] nix-daemon[892]: reaped child process 1190, status = succeeded4407lb # [ 63.721520] niks3-server[1032]: 2026/09/20 12:48:18 INFO Received uploads request method=POST path=/api/pending_closures4408worker1 # [ 63.702549] nix-grpc-daemon[902]: time=2026-09-20T12:48:18.196Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4409worker1 # [ 63.706501] nix-grpc-daemon[902]: time=2026-09-20T12:48:18.198Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4410lb # [ 63.758573] niks3-server[1032]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4411worker1 # [ 63.746576] nix-grpc-daemon[902]: time=2026-09-20T12:48:18.240Z level=INFO msg="Uploading 1 narinfos"4412lb # [ 63.775241] niks3-server[1032]: 2026/09/20 12:48:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4413lb # [ 63.782644] niks3-server[1032]: 2026/09/20 12:48:18 INFO Signed narinfos id=16 count=14414lb # [ 63.795763] niks3-server[1032]: 2026/09/20 12:48:18 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4415lb # [ 63.832891] niks3-server[1032]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4416lb # [ 63.843381] niks3-server[1032]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4417lb # [ 63.850352] niks3-server[1032]: 2026/09/20 12:48:18 INFO Completed upload id=164418worker1 # [ 63.821499] nix-grpc-daemon[902]: time=2026-09-20T12:48:18.316Z level=INFO msg="Upload complete. (362ms)"4419lb # [ 63.856570] niks3-server[1032]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4420worker1 # [ 63.833284] nix-grpc-daemon[832]: ts=2026-09-20T12:48:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14421worker1 # [ 63.854840] nix-grpc-daemon[832]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329964422worker1 # [ 63.860802] nix-daemon[892]: reaped child process 1170, status = succeeded4423worker1 # [ 63.871958] nix-grpc-daemon[832]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329964424worker1 # [ 63.878676] nix-grpc-daemon[832]: ts=2026-09-20T12:48:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_out=2324425client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.13 seconds)4426client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4427client # [ 64.091404] nix-daemon[938]: reaped child process 940, status = succeeded4428client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.06 seconds)4429client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4430client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4431client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4432client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4433(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.29 seconds)4434subtest: an input only one worker has still reaches the builder4435client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4436client # this derivation will be built:4437client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4438client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4439client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.46 seconds)4440client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4441client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.21 seconds)4442worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4443worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4444worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4445worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4446client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24447client # [ 65.395338] nix-daemon[938]: accepted connection from pid 1021, user root (trusted)4448client # this derivation will be built:4449client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4450worker2 # [ 65.649935] nix-grpc-daemon[780]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:517324451client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4452worker2 # [ 65.657723] nix-grpc-daemon[780]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517324453client # copying 1 paths...4454client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4455worker1 # [ 65.595473] nix-grpc-daemon[832]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329964456worker1 # [ 65.611753] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4457worker1 # [ 65.823128] nix-daemon[892]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)4458worker1 # [ 65.841066] nix-daemon[892]: reaped child process 1202, status = succeeded4459lb # [ 65.885045] niks3-server[1032]: 2026/09/20 12:48:20 INFO Received uploads request method=POST path=/api/pending_closures4460worker1 # [ 65.866528] nix-grpc-daemon[902]: time=2026-09-20T12:48:20.360Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4461worker1 # [ 65.868425] nix-grpc-daemon[902]: time=2026-09-20T12:48:20.360Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4462lb # [ 65.915441] niks3-server[1032]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4463worker1 # [ 65.902786] nix-grpc-daemon[902]: time=2026-09-20T12:48:20.397Z level=INFO msg="Uploading 1 narinfos"4464lb # [ 65.931682] niks3-server[1032]: 2026/09/20 12:48:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4465lb # [ 65.945144] niks3-server[1032]: 2026/09/20 12:48:20 INFO Signed narinfos id=17 count=14466lb # [ 65.953891] niks3-server[1032]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4467lb # [ 65.965518] niks3-server[1032]: 2026/09/20 12:48:20 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4468lb # [ 65.971043] niks3-server[1032]: 2026/09/20 12:48:20 INFO Completed upload id=174469worker1 # [ 65.941705] nix-grpc-daemon[902]: time=2026-09-20T12:48:20.436Z level=INFO msg="Upload complete. (314ms)"4470lb # [ 65.974526] niks3-server[1032]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4471worker1 # [ 65.952359] nix-grpc-daemon[832]: ts=2026-09-20T12:48:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_in=1204472worker1 # [ 65.965855] nix-daemon[892]: reaped child process 1197, status = succeeded4473worker2 # [ 66.050735] nix-grpc-daemon[780]: ts=2026-09-20T12:48:20Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04474worker2 # [ 66.058950] nix-grpc-daemon[780]: ts=2026-09-20T12:48:20Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14475worker2 # [ 66.096986] nix-grpc-daemon[780]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517324476worker2 # [ 66.114187] nix-grpc-daemon[780]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517324477client # copying 2 paths to 'lb:50051'...4478client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4479worker2 # [ 66.135792] nix-grpc-daemon[780]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517324480client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4481worker2 # [ 66.152536] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4482worker2 # [ 66.372866] nix-daemon[861]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4483worker2 # [ 66.392777] nix-daemon[861]: reaped child process 999, status = succeeded4484worker2 # [ 66.567213] nix-daemon[861]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4485worker2 # [ 66.585439] nix-daemon[861]: reaped child process 1004, status = succeeded4486lb # [ 66.547589] niks3-server[1032]: 2026/09/20 12:48:20 INFO Received uploads request method=POST path=/api/pending_closures4487lb # [ 66.558287] niks3-server[1032]: 2026/09/20 12:48:21 INFO Received uploads request method=POST path=/api/pending_closures4488worker2 # [ 66.620801] nix-grpc-daemon[912]: time=2026-09-20T12:48:21.113Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4489worker2 # [ 66.622522] nix-grpc-daemon[912]: time=2026-09-20T12:48:21.114Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4490worker2 # [ 66.624378] nix-grpc-daemon[912]: time=2026-09-20T12:48:21.114Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4491lb # [ 66.594264] niks3-server[1032]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4492lb # [ 66.615230] niks3-server[1032]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4493lb # [ 66.631046] niks3-server[1032]: 2026/09/20 12:48:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4494worker2 # [ 66.689990] nix-grpc-daemon[912]: time=2026-09-20T12:48:21.183Z level=INFO msg="Uploading 3 narinfos"4495lb # [ 66.640340] niks3-server[1032]: 2026/09/20 12:48:21 INFO Signed narinfos id=18 count=14496lb # [ 66.646834] niks3-server[1032]: 2026/09/20 12:48:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4497lb # [ 66.652873] niks3-server[1032]: 2026/09/20 12:48:21 INFO Signed narinfos id=19 count=24498lb # [ 66.654781] niks3-server[1032]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4499lb # [ 66.662128] niks3-server[1032]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4500lb # [ 66.680908] niks3-server[1032]: 2026/09/20 12:48:21 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4501lb # [ 66.689279] niks3-server[1032]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4502lb # [ 66.692999] niks3-server[1032]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4503lb # [ 66.695251] niks3-server[1032]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4504lb # [ 66.698736] niks3-server[1032]: 2026/09/20 12:48:21 INFO Completed upload id=184505lb # [ 66.700956] niks3-server[1032]: 2026/09/20 12:48:21 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4506lb # [ 66.705313] niks3-server[1032]: 2026/09/20 12:48:21 INFO Completed upload id=194507worker2 # [ 66.757351] nix-grpc-daemon[912]: time=2026-09-20T12:48:21.251Z level=INFO msg="Upload complete. (585ms)"4508worker2 # [ 66.769105] nix-grpc-daemon[780]: ts=2026-09-20T12:48:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=0 paths=2 nar_bytes_in=9844509worker2 # [ 66.788770] nix-grpc-daemon[780]: ts=2026-09-20T12:48:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517324510worker2 # [ 66.801398] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4511worker2 # [ 66.809614] nix-daemon[861]: reaped child process 994, status = succeeded4512worker2 # [ 66.829840] nix-grpc-daemon[780]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4513client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4514client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4515worker2 # [ 66.852299] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4516worker2 # [ 67.705439] nix-daemon[861]: reaped child process 1009, status = succeeded4517worker2 # [ 67.882628] nix-daemon[861]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4518worker2 # [ 67.904281] nix-daemon[861]: reaped child process 1027, status = succeeded4519lb # [ 67.865058] niks3-server[1032]: 2026/09/20 12:48:22 INFO Received uploads request method=POST path=/api/pending_closures4520worker2 # [ 67.927382] nix-grpc-daemon[912]: time=2026-09-20T12:48:22.421Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4521lb # [ 67.900462] niks3-server[1032]: 2026/09/20 12:48:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4522worker2 # [ 67.958815] nix-grpc-daemon[912]: time=2026-09-20T12:48:22.452Z level=INFO msg="Uploading 1 narinfos"4523lb # [ 67.917396] niks3-server[1032]: 2026/09/20 12:48:22 INFO Signed narinfos id=20 count=14524lb # [ 67.925091] niks3-server[1032]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4525worker2 # [ 67.991518] nix-grpc-daemon[912]: time=2026-09-20T12:48:22.485Z level=INFO msg="Upload complete. (325ms)"4526lb # [ 67.939872] niks3-server[1032]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4527lb # [ 67.947620] niks3-server[1032]: 2026/09/20 12:48:22 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4528lb # [ 67.954123] niks3-server[1032]: 2026/09/20 12:48:22 INFO Completed upload id=204529client # copying 1 paths...4530lb # [ 67.956824] niks3-server[1032]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4531worker2 # [ 68.006274] nix-grpc-daemon[780]: ts=2026-09-20T12:48:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14532client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4533worker2 # [ 68.020579] nix-grpc-daemon[780]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:517324534worker2 # [ 68.025021] nix-daemon[861]: reaped child process 1006, status = succeeded4535worker2 # [ 68.044161] nix-grpc-daemon[780]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:517324536worker2 # [ 68.050230] nix-grpc-daemon[780]: ts=2026-09-20T12:48:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=0 paths=1 nar_bytes_out=1204537client # [ 68.139769] nix-daemon[938]: reaped child process 1024, status = succeeded4538client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.98 seconds)4539client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24540client # [ 68.357266] nix-daemon[938]: accepted connection from pid 1043, user root (trusted)4541client # this derivation will be built:4542client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4543worker1 # [ 68.532160] nix-grpc-daemon[832]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329964544client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4545worker1 # [ 68.538733] nix-grpc-daemon[832]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964546worker1 # [ 68.550323] nix-grpc-daemon[902]: time=2026-09-20T12:48:23.044Z level=INFO msg="All 1 paths already cached"4547worker2 # [ 68.651944] nix-grpc-daemon[780]: ts=2026-09-20T12:48:23Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04548worker1 # [ 68.575056] nix-grpc-daemon[832]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964549worker2 # [ 68.658360] nix-grpc-daemon[780]: ts=2026-09-20T12:48:23Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14550worker1 # [ 68.592427] nix-grpc-daemon[832]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329964551client # copying 1 paths to 'lb:50051'...4552client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4553worker1 # [ 68.613051] nix-grpc-daemon[832]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329964554worker1 # [ 68.629170] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4555worker1 # [ 68.651689] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)4556worker1 # [ 68.747200] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 2/5)4557worker1 # [ 68.774997] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 3/5)4558worker1 # [ 68.917771] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 760 ms (attempt 4/5)4559worker1 # [ 69.687272] nix-grpc-daemon[832]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4560worker1 # [ 69.695581] nix-grpc-daemon[832]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4561worker1 # [ 69.924286] nix-daemon[892]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)4562worker1 # [ 69.942856] nix-daemon[892]: reaped child process 1213, status = succeeded4563worker1 # [ 70.126657] nix-daemon[892]: accepted connection from pid 1215, user nix-grpc-daemon (trusted)4564worker1 # [ 70.145905] nix-daemon[892]: reaped child process 1218, status = succeeded4565lb # [ 70.187106] niks3-server[1032]: 2026/09/20 12:48:24 INFO Received uploads request method=POST path=/api/pending_closures4566worker1 # [ 70.169778] nix-grpc-daemon[902]: time=2026-09-20T12:48:24.663Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4567worker1 # [ 70.171256] nix-grpc-daemon[902]: time=2026-09-20T12:48:24.663Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4568lb # [ 70.222580] niks3-server[1032]: 2026/09/20 12:48:24 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4569worker1 # [ 70.204231] nix-grpc-daemon[902]: time=2026-09-20T12:48:24.698Z level=INFO msg="Uploading 1 narinfos"4570lb # [ 70.235351] niks3-server[1032]: 2026/09/20 12:48:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4571lb # [ 70.247602] niks3-server[1032]: 2026/09/20 12:48:24 INFO Signed narinfos id=21 count=14572lb # [ 70.254018] niks3-server[1032]: 2026/09/20 12:48:24 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4573lb # [ 70.263728] niks3-server[1032]: 2026/09/20 12:48:24 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4574lb # [ 70.268202] niks3-server[1032]: 2026/09/20 12:48:24 INFO Completed upload id=214575worker1 # [ 70.238904] nix-grpc-daemon[902]: time=2026-09-20T12:48:24.733Z level=INFO msg="Upload complete. (514ms)"4576lb # [ 70.270951] niks3-server[1032]: 2026/09/20 12:48:24 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4577worker1 # [ 70.248406] nix-grpc-daemon[832]: ts=2026-09-20T12:48:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=1 paths=1 nar_bytes_in=5684578worker1 # [ 70.256328] nix-grpc-daemon[832]: ts=2026-09-20T12:48:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329964579worker1 # [ 70.271730] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4580worker1 # [ 70.274253] nix-daemon[892]: reaped child process 1204, status = succeeded4581client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4582worker1 # [ 70.289392] nix-daemon[892]: accepted connection from pid 832, user nix-grpc-daemon (trusted)4583client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4584worker1 # [ 70.910243] nix-daemon[892]: reaped child process 1222, status = succeeded4585worker1 # [ 71.075462] nix-daemon[892]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)4586worker1 # [ 71.096728] nix-daemon[892]: reaped child process 1240, status = succeeded4587lb # [ 71.136589] niks3-server[1032]: 2026/09/20 12:48:25 INFO Received uploads request method=POST path=/api/pending_closures4588worker1 # [ 71.123011] nix-grpc-daemon[902]: time=2026-09-20T12:48:25.616Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4589lb # [ 71.180963] niks3-server[1032]: 2026/09/20 12:48:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4590worker1 # [ 71.158839] nix-grpc-daemon[902]: time=2026-09-20T12:48:25.653Z level=INFO msg="Uploading 1 narinfos"4591lb # [ 71.195996] niks3-server[1032]: 2026/09/20 12:48:25 INFO Signed narinfos id=22 count=14592lb # [ 71.203239] niks3-server[1032]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4593lb # [ 71.211370] niks3-server[1032]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4594worker1 # [ 71.189125] nix-grpc-daemon[902]: time=2026-09-20T12:48:25.682Z level=INFO msg="Upload complete. (289ms)"4595lb # [ 71.217959] niks3-server[1032]: 2026/09/20 12:48:25 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4596lb # [ 71.223701] niks3-server[1032]: 2026/09/20 12:48:25 INFO Completed upload id=224597lb # [ 71.227107] niks3-server[1032]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4598client # copying 1 paths...4599worker1 # [ 71.200784] nix-grpc-daemon[832]: ts=2026-09-20T12:48:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14600client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4601worker1 # [ 71.213217] nix-grpc-daemon[832]: ts=2026-09-20T12:48:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329964602worker1 # [ 71.217106] nix-daemon[892]: reaped child process 1220, status = succeeded4603worker2 # [ 71.318789] nix-grpc-daemon[780]: ts=2026-09-20T12:48:25Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:517324604worker2 # [ 71.333612] nix-grpc-daemon[780]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4605worker2 # [ 71.352246] nix-grpc-daemon[780]: ts=2026-09-20T12:48:25Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=0 paths=1 nar_bytes_out=1204606client # [ 71.437608] nix-daemon[938]: reaped child process 1046, status = succeeded4607client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.30 seconds)4608client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24609client # [ 71.659700] nix-daemon[938]: accepted connection from pid 1065, user root (trusted)4610client # this derivation will be built:4611client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4612worker1 # [ 71.831627] nix-grpc-daemon[832]: ts=2026-09-20T12:48:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329964613client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4614worker2 # [ 71.916655] nix-grpc-daemon[780]: ts=2026-09-20T12:48:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517324615worker2 # [ 71.928954] nix-grpc-daemon[912]: time=2026-09-20T12:48:26.422Z level=INFO msg="All 1 paths already cached"4616worker2 # [ 71.949068] nix-grpc-daemon[780]: ts=2026-09-20T12:48:26Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04617worker2 # [ 71.957477] nix-grpc-daemon[780]: ts=2026-09-20T12:48:26Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14618worker2 # [ 71.967073] nix-grpc-daemon[780]: ts=2026-09-20T12:48:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517324619worker2 # [ 71.979864] nix-grpc-daemon[780]: ts=2026-09-20T12:48:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517324620worker2 # [ 71.995977] nix-grpc-daemon[912]: time=2026-09-20T12:48:26.490Z level=INFO msg="All 1 paths already cached"4621client # copying 1 paths to 'lb:50051'...4622client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4623worker2 # [ 72.007874] nix-grpc-daemon[780]: ts=2026-09-20T12:48:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517324624worker2 # [ 72.023929] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4625lb # [ 71.981571] envoy[921]: conn peer=[2001:db8:1::1]:53028 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4313 tx=2110 ms=23454626lb # [ 71.986872] envoy[921]: conn peer=[2001:db8:1::1]:53036 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=704627lb # [ 71.993789] envoy[921]: conn peer=[2001:db8:1::1]:53052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4209 tx=1768 ms=24624628lb # [ 72.000050] envoy[921]: conn peer=[2001:db8:1::1]:53060 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=704629lb # [ 72.005527] envoy[921]: conn peer=[2001:db8:1::1]:53064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3699 tx=1804 ms=28024630lb # [ 72.010927] envoy[921]: conn peer=[2001:db8:1::1]:55302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1004631worker2 # [ 72.231855] nix-daemon[861]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)4632worker2 # [ 72.254704] nix-daemon[861]: reaped child process 1039, status = succeeded4633worker2 # [ 72.429836] nix-daemon[861]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4634worker2 # [ 72.450328] nix-daemon[861]: reaped child process 1044, status = succeeded4635lb # [ 72.414558] niks3-server[1032]: 2026/09/20 12:48:26 INFO Received uploads request method=POST path=/api/pending_closures4636worker2 # [ 72.475907] nix-grpc-daemon[912]: time=2026-09-20T12:48:26.969Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4637worker2 # [ 72.477674] nix-grpc-daemon[912]: time=2026-09-20T12:48:26.969Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4638lb # [ 72.447926] niks3-server[1032]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4639worker2 # [ 72.511801] nix-grpc-daemon[912]: time=2026-09-20T12:48:27.005Z level=INFO msg="Uploading 1 narinfos"4640lb # [ 72.459952] niks3-server[1032]: 2026/09/20 12:48:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4641lb # [ 72.473176] niks3-server[1032]: 2026/09/20 12:48:26 INFO Signed narinfos id=23 count=14642lb # [ 72.482004] niks3-server[1032]: 2026/09/20 12:48:26 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4643lb # [ 72.489079] niks3-server[1032]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4644lb # [ 72.493555] niks3-server[1032]: 2026/09/20 12:48:26 INFO Completed upload id=234645worker2 # [ 72.545323] nix-grpc-daemon[912]: time=2026-09-20T12:48:27.039Z level=INFO msg="Upload complete. (503ms)"4646lb # [ 72.496078] niks3-server[1032]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4647worker2 # [ 72.552994] nix-grpc-daemon[780]: ts=2026-09-20T12:48:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=0 paths=1 nar_bytes_in=5684648worker2 # [ 72.567351] nix-grpc-daemon[780]: ts=2026-09-20T12:48:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517324649worker2 # [ 72.574869] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4650worker2 # [ 72.580338] nix-daemon[861]: reaped child process 1034, status = succeeded4651client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4652worker2 # [ 72.595557] nix-daemon[861]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4653client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4654worker2 # [ 74.043542] nix-daemon[861]: reaped child process 1048, status = succeeded4655worker2 # [ 74.223020] nix-daemon[861]: accepted connection from pid 1063, user nix-grpc-daemon (trusted)4656worker2 # [ 74.242834] nix-daemon[861]: reaped child process 1066, status = succeeded4657lb # [ 74.211622] niks3-server[1032]: 2026/09/20 12:48:28 INFO Received uploads request method=POST path=/api/pending_closures4658worker2 # [ 74.274894] nix-grpc-daemon[912]: time=2026-09-20T12:48:28.767Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4659lb # [ 74.246343] niks3-server[1032]: 2026/09/20 12:48:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4660worker2 # [ 74.308294] nix-grpc-daemon[912]: time=2026-09-20T12:48:28.802Z level=INFO msg="Uploading 1 narinfos"4661lb # [ 74.261238] niks3-server[1032]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4662lb # [ 74.273396] niks3-server[1032]: 2026/09/20 12:48:28 INFO Signed narinfos id=24 count=14663lb # [ 74.277652] niks3-server[1032]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4664worker2 # [ 74.339597] nix-grpc-daemon[912]: time=2026-09-20T12:48:28.833Z level=INFO msg="Upload complete. (332ms)"4665lb # [ 74.287854] niks3-server[1032]: 2026/09/20 12:48:28 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4666lb # [ 74.292590] niks3-server[1032]: 2026/09/20 12:48:28 INFO Completed upload id=244667lb # [ 74.296158] niks3-server[1032]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4668worker2 # [ 74.353243] nix-grpc-daemon[780]: ts=2026-09-20T12:48:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51732 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14669worker2 # [ 74.368584] nix-daemon[861]: reaped child process 1046, status = succeeded4670client # copying 1 paths...4671worker2 # [ 74.398530] nix-grpc-daemon[780]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:517324672client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4673worker1 # [ 74.357631] nix-grpc-daemon[832]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329964674worker1 # [ 74.374816] nix-grpc-daemon[832]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4675worker1 # [ 74.395350] nix-grpc-daemon[832]: ts=2026-09-20T12:48:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32996 duration_s=0 paths=1 nar_bytes_out=1204676client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 3.13 seconds)4677client # [ 74.576874] nix-daemon[938]: reaped child process 1068, status = succeeded4678(finished: subtest: an input only one worker has still reaches the builder, in 10.37 seconds)4679subtest: upload whose reference the worker lacks is completed from the cache4680client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4681client # this derivation will be built:4682client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4683client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4684client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.46 seconds)4685client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4686client # this derivation will be built:4687client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4688client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4689client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.49 seconds)4690client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4691client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.21 seconds)4692worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4693worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4694worker2 # [ 76.102404] systemd[1]: Stopping Nix worker-protocol over gRPC...4695worker2 # [ 76.514324] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=signal_exit4696worker2 # [ 76.761705] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=scheduler_yield4697worker2 # [ 76.767657] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=scheduler_restarting addr=lb:500514698lb # [ 76.717256] niks3-server[1032]: 2026/09/20 12:48:31 INFO lead: released remote=192.168.1.4:553304699worker2 # [ 76.773771] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:517244700worker1 # [ 76.698428] nix-grpc-daemon[832]: ts=2026-09-20T12:48:31Z level=info event=scheduler_restarting addr=lb:500514701worker2 # [ 76.780361] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=worker_gone addr=192.168.1.4:500514702worker2 # [ 76.784766] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:517244703worker2 # [ 76.790150] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=worker_gone addr=192.168.1.3:500514704worker2 # [ 76.969424] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=scheduler_connected addr=lb:500514705worker2 # [ 76.975008] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4706worker1 # [ 76.904486] nix-grpc-daemon[832]: ts=2026-09-20T12:48:31Z level=info event=scheduler_connected addr=lb:500514707worker1 # [ 76.909782] nix-grpc-daemon[832]: ts=2026-09-20T12:48:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4708worker2 # [ 77.176342] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=scheduler_connected addr=lb:500514709worker2 # [ 77.181821] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4710worker1 # [ 77.110503] nix-grpc-daemon[832]: ts=2026-09-20T12:48:31Z level=info event=scheduler_connected addr=lb:500514711worker1 # [ 77.115806] nix-grpc-daemon[832]: ts=2026-09-20T12:48:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4712worker2 # [ 77.383140] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=scheduler_connected addr=lb:500514713worker2 # [ 77.388779] nix-grpc-daemon[780]: ts=2026-09-20T12:48:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4714worker1 # [ 77.316792] nix-grpc-daemon[832]: ts=2026-09-20T12:48:31Z level=info event=scheduler_connected addr=lb:500514715worker1 # [ 77.322185] nix-grpc-daemon[832]: ts=2026-09-20T12:48:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4716worker2 # [ 77.592156] nix-grpc-daemon[780]: ts=2026-09-20T12:48:32Z level=info event=scheduler_connected addr=lb:500514717worker2 # [ 77.596421] nix-grpc-daemon[780]: ts=2026-09-20T12:48:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4718worker1 # [ 77.523091] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_connected addr=lb:500514719worker1 # [ 77.528591] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4720worker2 # [ 77.768192] nix-grpc-daemon[780]: ts=2026-09-20T12:48:32Z level=info event=scheduler_restarting4721worker1 # [ 77.730777] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_connected addr=lb:500514722worker2 # [ 77.814113] nix-grpc-daemon[780]: ts=2026-09-20T12:48:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4723worker1 # [ 77.736275] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4724worker2 # [ 77.935027] nix-daemon[861]: reaped child process 863, status = succeeded4725worker1 # [ 77.937109] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_connected addr=lb:500514726worker2 # [ 78.024564] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4727worker1 # [ 77.942413] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4728worker2 # [ 78.032480] systemd[1]: Stopped Nix worker-protocol over gRPC.4729worker2 # [ 78.035145] systemd[1]: nix-grpc-daemon.service: Consumed 1.949s CPU time over 55.830s wall clock time, 52M memory peak, 54.9K incoming IP traffic, 147.8K outgoing IP traffic.4730worker2 # [ 78.055346] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4731worker2 # [ 78.059309] systemd[1]: Closed Nix worker-protocol over gRPC.4732worker2 # [ 78.061787] systemd[1]: nix-grpc-daemon.socket: Consumed 772us CPU time over 1min 6.318s wall clock time, 256K memory peak, 71.6K incoming IP traffic, 61.6K outgoing IP traffic.4733lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4734lb: must succeed: curl -sf localhost:9901/clusters4735lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4736lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4737client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24738worker1 # [ 78.143398] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_connected addr=lb:500514739worker1 # [ 78.148799] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4740worker1 # [ 78.314588] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:329964741worker1 # [ 78.323175] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329964742client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4743worker1 # [ 78.349589] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_connected addr=lb:500514744worker1 # [ 78.354643] nix-grpc-daemon[832]: ts=2026-09-20T12:48:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4745client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.29 seconds)4746worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4747worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4748worker2 # [ 78.546764] systemd[1]: Listening on Nix worker-protocol over gRPC.4749worker1 # [ 78.547053] systemd[1]: Stopping Nix worker-protocol over gRPC...4750worker1 # [ 78.555776] nix-grpc-daemon[832]: ts=2026-09-20T12:48:33Z level=info event=scheduler_connected addr=lb:500514751worker1 # [ 78.561527] nix-grpc-daemon[832]: ts=2026-09-20T12:48:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4752worker2 # [ 78.690887] systemd[1]: Starting scheduler-token.service...4753worker2 # [ 78.797319] systemd[1]: scheduler-token.service: Deactivated successfully.4754worker2 # [ 78.804483] systemd[1]: Finished scheduler-token.service.4755worker2 # [ 78.806192] systemd[1]: scheduler-token.service: Consumed 52ms CPU time over 109ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4756worker2 # [ 78.814429] systemd[1]: Starting Nix worker-protocol over gRPC...4757worker1 # [ 78.762129] nix-grpc-daemon[832]: ts=2026-09-20T12:48:33Z level=info event=scheduler_connected addr=lb:500514758worker1 # [ 78.766321] nix-grpc-daemon[832]: ts=2026-09-20T12:48:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4759worker2 # [ 79.005363] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4760worker2 # [ 79.008115] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4761worker2 # [ 79.012339] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14762worker2 # [ 79.021255] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4763worker2 # [ 79.023071] systemd[1]: Started Nix worker-protocol over gRPC.4764worker2 # [ 79.034305] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=scheduler_take_over4765lb # [ 78.983925] niks3-server[1032]: 2026/09/20 12:48:33 INFO lead: acquired remote=192.168.1.4:468624766worker1 # [ 78.962941] nix-grpc-daemon[832]: ts=2026-09-20T12:48:33Z level=info event=scheduler_connected addr=lb:500514767worker2 # [ 79.057653] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=scheduler_connected addr=lb:500514768worker1 # [ 78.973448] nix-grpc-daemon[832]: ts=2026-09-20T12:48:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4769worker2 # [ 79.066140] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=oidc_jwks provider=mock keys=14770worker2 # [ 79.067535] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:423584771worker2 # [ 79.069564] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04772worker1 # [ 79.174400] nix-grpc-daemon[832]: ts=2026-09-20T12:48:33Z level=info event=scheduler_connected addr=lb:500514773worker2 # [ 79.258150] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:423584774worker2 # [ 79.263240] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:33Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04775worker1 # [ 79.308851] nix-grpc-daemon[832]: ts=2026-09-20T12:48:33Z level=info event=signal_exit4776worker1 # [ 80.569332] nix-grpc-daemon[832]: ts=2026-09-20T12:48:35Z level=info event=scheduler_restarting4777worker2 # [ 80.663861] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:423584778worker2 # [ 80.668775] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:35Z level=info event=worker_gone addr=192.168.1.3:500514779worker1 # [ 80.602595] nix-grpc-daemon[832]: ts=2026-09-20T12:48:35Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4780worker1 # [ 80.726982] nix-daemon[892]: reaped child process 894, status = succeeded4781worker1 # [ 81.451031] nix-grpc-daemon[832]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4782worker1 # [ 81.451465] nix-grpc-daemon[832]: E0000 00:00:1789908515.945252 875 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4783worker1 # [ 81.451625] nix-grpc-daemon[832]: E0000 00:00:1789908515.945576 875 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4784worker1 # [ 81.872582] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4785worker1 # [ 81.883547] systemd[1]: Stopped Nix worker-protocol over gRPC.4786worker1 # [ 81.887852] systemd[1]: nix-grpc-daemon.service: Consumed 3.795s CPU time over 59.914s wall clock time, 59.7M memory peak, 86.4K incoming IP traffic, 156.8K outgoing IP traffic.4787worker1 # [ 81.905780] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4788worker1 # [ 81.907974] systemd[1]: Closed Nix worker-protocol over gRPC.4789lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4790lb: must succeed: curl -sf localhost:9901/clusters4791worker1 # [ 81.911388] systemd[1]: nix-grpc-daemon.socket: Consumed 704us CPU time over 1min 8.953s wall clock time, 256K memory peak, 78.6K incoming IP traffic, 63.9K outgoing IP traffic.4792lb # [ 81.981868] envoy[921]: conn peer=[2001:db8:1::1]:55312 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1054793lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4794lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.10 seconds)4795client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24796worker2 # [ 82.365272] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480364797worker2 # [ 82.381643] nix-daemon[861]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4798worker2 # [ 82.409729] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 1/5)4799worker2 # [ 82.442065] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 127 ms (attempt 2/5)4800worker2 # [ 82.569502] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5)4801lb # [ 81.987280] envoy[921]: conn peer=[2001:db8:1::1]:55308 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3712 tx=1805 ms=26254802lb # [ 81.987485] envoy[921]: conn peer=[2001:db8:1::4]:37672 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=38270 tx=2108 ms=552824803lb # [ 81.987614] envoy[921]: conn peer=[2001:db8:1::1]:55328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=294804lb # [ 81.987742] envoy[921]: conn peer=192.168.1.3:49816 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=8815 tx=2536 ms=573564805worker2 # [ 82.694412] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 624 ms (attempt 4/5)4806worker2 # [ 83.327541] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4807client # copying 2 paths...4808client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4809worker2 # [ 83.356151] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480364810client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4811worker2 # [ 83.374710] nix-daemon[861]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4812worker2 # [ 83.486629] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:37Z level=info event=niks3_push_started pid=11464813worker2 # [ 83.720720] nix-grpc-daemon[1146]: time=2026-09-20T12:48:38.213Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4814worker2 # [ 83.899351] nix-daemon[861]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)4815worker2 # [ 83.919608] nix-daemon[861]: reaped child process 1162, status = succeeded4816lb # [ 83.886041] niks3-server[1032]: 2026/09/20 12:48:38 INFO Received uploads request method=POST path=/api/pending_closures4817lb # [ 83.903387] niks3-server[1032]: 2026/09/20 12:48:38 INFO Received uploads request method=POST path=/api/pending_closures4818worker2 # [ 84.056260] nix-grpc-daemon[1146]: time=2026-09-20T12:48:38.546Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4819worker2 # [ 84.070536] nix-grpc-daemon[1146]: time=2026-09-20T12:48:38.546Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4820worker2 # [ 84.078802] nix-grpc-daemon[1146]: time=2026-09-20T12:48:38.546Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4821lb # [ 84.085396] niks3-server[1032]: 2026/09/20 12:48:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4822lb # [ 84.103490] niks3-server[1032]: 2026/09/20 12:48:38 INFO Signed narinfos id=25 count=14823worker2 # [ 84.163422] nix-grpc-daemon[1146]: time=2026-09-20T12:48:38.655Z level=INFO msg="Uploading 3 narinfos"4824lb # [ 84.114200] niks3-server[1032]: 2026/09/20 12:48:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4825lb # [ 84.124939] niks3-server[1032]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4826lb # [ 84.135549] niks3-server[1032]: 2026/09/20 12:48:38 INFO Signed narinfos id=26 count=24827lb # [ 84.140885] niks3-server[1032]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4828lb # [ 84.148877] niks3-server[1032]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4829lb # [ 84.151655] niks3-server[1032]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4830lb # [ 84.164766] niks3-server[1032]: 2026/09/20 12:48:38 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4831lb # [ 84.171923] niks3-server[1032]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4832lb # [ 84.175318] niks3-server[1032]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4833lb # [ 84.178527] niks3-server[1032]: 2026/09/20 12:48:38 INFO Completed upload id=254834lb # [ 84.184647] niks3-server[1032]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4835lb # [ 84.187279] niks3-server[1032]: 2026/09/20 12:48:38 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4836lb # [ 84.190817] niks3-server[1032]: 2026/09/20 12:48:38 INFO Completed upload id=264837worker2 # [ 84.242421] nix-grpc-daemon[1146]: time=2026-09-20T12:48:38.736Z level=INFO msg="Upload complete. (516ms)"4838worker2 # [ 84.252690] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48036 duration_s=0 paths=2 nar_bytes_in=3124839worker2 # [ 84.271701] nix-daemon[861]: reaped child process 1144, status = succeeded4840client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.20 seconds)4841worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4842worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)4843worker1 # [ 84.336744] systemd[1]: Listening on Nix worker-protocol over gRPC.4844worker1 # [ 84.343986] systemd[1]: Starting Nix worker-protocol over gRPC...4845worker1 # [ 84.536450] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4846worker1 # [ 84.539125] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4847worker1 # [ 84.543251] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14848worker1 # [ 84.551746] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4849worker1 # [ 84.557950] systemd[1]: Started Nix worker-protocol over gRPC.4850lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4851lb: must succeed: curl -sf localhost:9901/clusters4852worker1 # [ 84.575587] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:39Z level=info event=scheduler_connected addr=lb:500514853worker2 # [ 84.661424] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:423584854worker2 # [ 84.666582] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04855lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.12 seconds)4856lb: must succeed: curl -sf localhost:9901/clusters4857lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4858lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.20 seconds)4859(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.22 seconds)4860subtest: scheduler down: the other node takes the lock and keeps it4861lb: must succeed: curl -sf localhost:9901/clusters4862lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4863lb: must succeed: curl -sf localhost:9901/clusters4864lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4865lb: must succeed: curl -sf localhost:9901/clusters4866lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4867lb: must succeed: curl -sf localhost:9901/clusters4868lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4869worker2 # [ 86.133033] systemd[1]: Stopping Nix worker-protocol over gRPC...4870worker2 # [ 87.027965] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:41Z level=info event=signal_exit4871worker2 # [ 87.242008] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:41Z level=info event=scheduler_yield4872lb # [ 87.193508] niks3-server[1032]: 2026/09/20 12:48:41 INFO lead: released remote=192.168.1.4:468624873worker1 # [ 87.167783] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:41Z level=info event=scheduler_restarting addr=lb:500514874worker2 # [ 87.251608] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:423584875worker2 # [ 87.258281] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:41Z level=info event=worker_gone addr=192.168.1.3:500514876worker2 # [ 87.264532] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:41Z level=info event=scheduler_restarting addr=lb:500514877worker2 # [ 87.268671] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:423584878worker2 # [ 87.273075] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:41Z level=info event=worker_gone addr=192.168.1.4:500514879worker1 # [ 87.368090] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:41Z level=info event=scheduler_connected addr=lb:500514880worker1 # [ 87.373799] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4881worker2 # [ 87.460116] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:41Z level=info event=scheduler_connected addr=lb:500514882worker2 # [ 87.466136] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4883worker1 # [ 87.574853] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_connected addr=lb:500514884worker1 # [ 87.580757] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4885worker2 # [ 87.667527] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:42Z level=info event=scheduler_connected addr=lb:500514886worker2 # [ 87.673490] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4887worker1 # [ 87.781609] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_connected addr=lb:500514888worker1 # [ 87.787168] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4889worker2 # [ 87.875257] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:42Z level=info event=scheduler_connected addr=lb:500514890worker2 # [ 87.880639] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4891worker1 # [ 87.988491] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_connected addr=lb:500514892worker1 # [ 87.994096] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4893worker2 # [ 88.081855] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:42Z level=info event=scheduler_connected addr=lb:500514894worker2 # [ 88.086342] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4895worker2 # [ 88.243684] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:42Z level=info event=scheduler_restarting4896worker2 # [ 88.277787] nix-daemon[861]: reaped child process 1139, status = succeeded4897worker1 # [ 88.196548] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_connected addr=lb:500514898worker1 # [ 88.201734] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4899worker2 # [ 88.299060] nix-grpc-daemon[1090]: ts=2026-09-20T12:48:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4900worker1 # [ 88.402742] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_connected addr=lb:500514901worker1 # [ 88.408483] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4902worker1 # [ 88.609910] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_connected addr=lb:500514903worker1 # [ 88.615694] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4904worker1 # [ 88.816785] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_connected addr=lb:500514905worker1 # [ 88.822534] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4906worker2 # [ 89.067696] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4907worker2 # [ 89.071278] systemd[1]: Stopped Nix worker-protocol over gRPC.4908worker2 # [ 89.073930] systemd[1]: nix-grpc-daemon.service: Consumed 499ms CPU time over 10.245s wall clock time, 31.1M memory peak, 19.7K incoming IP traffic, 26.4K outgoing IP traffic.4909worker2 # [ 89.089538] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4910worker2 # [ 89.091648] systemd[1]: Closed Nix worker-protocol over gRPC.4911worker2 # [ 89.094573] systemd[1]: nix-grpc-daemon.socket: Consumed 964us CPU time over 10.552s wall clock time, 256K memory peak, 21.6K incoming IP traffic, 16K outgoing IP traffic.4912worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over4913worker1 # [ 89.023814] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_connected addr=lb:500514914worker1 # [ 89.031313] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4915worker1 # [ 89.233064] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_connected addr=lb:500514916worker1 # [ 89.237495] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4917worker1 # [ 89.434005] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_connected addr=lb:500514918worker1 # [ 89.439705] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4919lb # [ 89.594306] niks3-server[1032]: 2026/09/20 12:48:44 INFO lead: acquired remote=192.168.1.3:334304920worker1 # [ 89.566476] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:44Z level=info event=scheduler_take_over4921worker1 # [ 89.640846] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:44Z level=info event=scheduler_connected addr=lb:500514922worker1 # [ 89.646615] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4923worker1 # [ 89.847781] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:44Z level=info event=scheduler_connected addr=lb:500514924worker1 # [ 89.853629] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4925worker1 # [ 90.054501] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:44Z level=info event=scheduler_connected addr=lb:500514926worker1 # [ 90.065226] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:571744927worker1 # [ 90.072410] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04928worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.18 seconds)4929worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4930worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4931client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&24932worker1 # [ 90.514432] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:571804933worker1 # [ 90.535087] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)4934client # don't know how to build these paths:4935client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv4936worker1 # [ 90.554645] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:571804937worker1 # [ 90.572592] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)4938worker1 # [ 90.625729] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 2/5)4939worker1 # [ 90.685826] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 235 ms (attempt 3/5)4940worker1 # [ 90.928816] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 508 ms (attempt 4/5)4941worker1 # [ 91.447255] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4942worker1 # [ 91.484404] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:45Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=04943worker1 # [ 91.491733] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:45Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=14944worker1 # [ 91.533680] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:571804945worker1 # [ 91.552700] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:571804946client # copying 1 paths to 'lb:50051'...4947client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...4948worker1 # [ 91.571546] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:571804949worker1 # [ 91.589745] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)4950worker1 # [ 91.620376] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:46Z level=info event=niks3_push_started pid=13374951worker1 # [ 91.840278] nix-grpc-daemon[1337]: time=2026-09-20T12:48:46.333Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""4952lb # [ 91.981130] envoy[921]: conn peer=[2001:db8:1::1]:55230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1007 tx=305 ms=19044953lb # [ 91.981533] envoy[921]: conn peer=[2001:db8:1::4]:58490 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4158 tx=349 ms=92384954worker1 # [ 92.021605] nix-daemon[892]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)4955worker1 # [ 92.040940] nix-daemon[892]: reaped child process 1353, status = succeeded4956worker1 # [ 92.222045] nix-daemon[892]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)4957worker1 # [ 92.241075] nix-daemon[892]: reaped child process 1358, status = succeeded4958lb # [ 92.282018] niks3-server[1032]: 2026/09/20 12:48:46 INFO Received uploads request method=POST path=/api/pending_closures4959worker1 # [ 92.264082] nix-grpc-daemon[1337]: time=2026-09-20T12:48:46.757Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4960worker1 # [ 92.265625] nix-grpc-daemon[1337]: time=2026-09-20T12:48:46.757Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"4961lb # [ 92.323531] niks3-server[1032]: 2026/09/20 12:48:46 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst4962worker1 # [ 92.309934] nix-grpc-daemon[1337]: time=2026-09-20T12:48:46.803Z level=INFO msg="Uploading 1 narinfos"4963lb # [ 92.338812] niks3-server[1032]: 2026/09/20 12:48:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4964lb # [ 92.352739] niks3-server[1032]: 2026/09/20 12:48:46 INFO Signed narinfos id=27 count=14965lb # [ 92.360131] niks3-server[1032]: 2026/09/20 12:48:46 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls4966lb # [ 92.375158] niks3-server[1032]: 2026/09/20 12:48:46 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4967lb # [ 92.382777] niks3-server[1032]: 2026/09/20 12:48:46 INFO Completed upload id=274968lb # [ 92.386784] niks3-server[1032]: 2026/09/20 12:48:46 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo4969worker1 # [ 92.352958] nix-grpc-daemon[1337]: time=2026-09-20T12:48:46.847Z level=INFO msg="Upload complete. (509ms)"4970worker1 # [ 92.364623] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57180 duration_s=0 paths=1 nar_bytes_in=5204971worker1 # [ 92.376616] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:571804972worker1 # [ 92.390803] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)4973worker1 # [ 92.397208] nix-daemon[892]: reaped child process 1335, status = succeeded4974client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...4975worker1 # [ 92.410626] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)4976client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv4977client # farm-a> LOG-farm-a-on-standby4978worker1 # [ 93.470669] nix-daemon[892]: reaped child process 1362, status = succeeded4979worker1 # [ 93.662725] nix-daemon[892]: accepted connection from pid 1377, user nix-grpc-daemon (trusted)4980worker1 # [ 93.682104] nix-daemon[892]: reaped child process 1380, status = succeeded4981lb # [ 93.726713] niks3-server[1032]: 2026/09/20 12:48:48 INFO Received uploads request method=POST path=/api/pending_closures4982worker1 # [ 93.708629] nix-grpc-daemon[1337]: time=2026-09-20T12:48:48.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4983worker1 # [ 93.710320] nix-grpc-daemon[1337]: time=2026-09-20T12:48:48.200Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"4984lb # [ 93.776897] niks3-server[1032]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst4985lb # [ 93.797424] niks3-server[1032]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv4986lb # [ 93.808763] niks3-server[1032]: 2026/09/20 12:48:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4987worker1 # [ 93.787142] nix-grpc-daemon[1337]: time=2026-09-20T12:48:48.279Z level=INFO msg="Uploading 1 narinfos"4988lb # [ 93.815515] niks3-server[1032]: 2026/09/20 12:48:48 INFO Signed narinfos id=28 count=14989lb # [ 93.825155] niks3-server[1032]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls4990lb # [ 93.837479] niks3-server[1032]: 2026/09/20 12:48:48 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4991lb # [ 93.844884] niks3-server[1032]: 2026/09/20 12:48:48 INFO Completed upload id=284992worker1 # [ 93.814785] nix-grpc-daemon[1337]: time=2026-09-20T12:48:48.309Z level=INFO msg="Upload complete. (388ms)"4993lb # [ 93.847803] niks3-server[1032]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo4994worker1 # [ 93.844798] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57180 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=14995worker1 # [ 93.856393] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:48Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=04996worker1 # [ 93.859992] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:48Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=14997worker1 # [ 93.866162] nix-daemon[892]: reaped child process 1360, status = succeeded4998worker1 # [ 93.886639] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:571804999worker1 # [ 93.905825] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:571805000worker1 # [ 93.929533] nix-grpc-daemon[1337]: time=2026-09-20T12:48:48.424Z level=INFO msg="All 1 paths already cached"5001client # copying 1 paths to 'lb:50051'...5002client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5003worker1 # [ 93.940771] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:571805004worker1 # [ 93.955571] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5005worker1 # [ 94.171442] nix-daemon[892]: accepted connection from pid 1386, user nix-grpc-daemon (trusted)5006worker1 # [ 94.193018] nix-daemon[892]: reaped child process 1389, status = succeeded5007worker1 # [ 94.366329] nix-daemon[892]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5008worker1 # [ 94.386225] nix-daemon[892]: reaped child process 1394, status = succeeded5009lb # [ 94.428174] niks3-server[1032]: 2026/09/20 12:48:48 INFO Received uploads request method=POST path=/api/pending_closures5010worker1 # [ 94.409533] nix-grpc-daemon[1337]: time=2026-09-20T12:48:48.903Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5011worker1 # [ 94.411050] nix-grpc-daemon[1337]: time=2026-09-20T12:48:48.903Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5012lb # [ 94.459372] niks3-server[1032]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5013worker1 # [ 94.446664] nix-grpc-daemon[1337]: time=2026-09-20T12:48:48.940Z level=INFO msg="Uploading 1 narinfos"5014lb # [ 94.476200] niks3-server[1032]: 2026/09/20 12:48:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5015lb # [ 94.487084] niks3-server[1032]: 2026/09/20 12:48:48 INFO Signed narinfos id=29 count=15016lb # [ 94.492886] niks3-server[1032]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5017lb # [ 94.501474] niks3-server[1032]: 2026/09/20 12:48:48 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5018lb # [ 94.505879] niks3-server[1032]: 2026/09/20 12:48:48 INFO Completed upload id=295019worker1 # [ 94.476305] nix-grpc-daemon[1337]: time=2026-09-20T12:48:48.970Z level=INFO msg="Upload complete. (502ms)"5020lb # [ 94.508560] niks3-server[1032]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5021worker1 # [ 94.483722] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57180 duration_s=0 paths=1 nar_bytes_in=6405022worker1 # [ 94.497884] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:571805023worker1 # [ 94.504839] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5024worker1 # [ 94.509524] nix-daemon[892]: reaped child process 1384, status = succeeded5025client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5026client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5027worker1 # [ 94.523675] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5028worker1 # [ 95.343597] nix-daemon[892]: reaped child process 1398, status = succeeded5029client # farm-b> LOG-farm-b-on-standby5030worker1 # [ 95.501502] nix-daemon[892]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)5031worker1 # [ 95.521556] nix-daemon[892]: reaped child process 1416, status = succeeded5032lb # [ 95.565068] niks3-server[1032]: 2026/09/20 12:48:50 INFO Received uploads request method=POST path=/api/pending_closures5033worker1 # [ 95.547755] nix-grpc-daemon[1337]: time=2026-09-20T12:48:50.041Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5034worker1 # [ 95.549451] nix-grpc-daemon[1337]: time=2026-09-20T12:48:50.041Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5035lb # [ 95.602423] niks3-server[1032]: 2026/09/20 12:48:50 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5036lb # [ 95.619183] niks3-server[1032]: 2026/09/20 12:48:50 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5037worker1 # [ 95.597139] nix-grpc-daemon[1337]: time=2026-09-20T12:48:50.091Z level=INFO msg="Uploading 1 narinfos"5038lb # [ 95.628143] niks3-server[1032]: 2026/09/20 12:48:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5039lb # [ 95.638640] niks3-server[1032]: 2026/09/20 12:48:50 INFO Signed narinfos id=30 count=15040lb # [ 95.643585] niks3-server[1032]: 2026/09/20 12:48:50 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5041lb # [ 95.652372] niks3-server[1032]: 2026/09/20 12:48:50 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5042lb # [ 95.658505] niks3-server[1032]: 2026/09/20 12:48:50 INFO Completed upload id=305043worker1 # [ 95.629634] nix-grpc-daemon[1337]: time=2026-09-20T12:48:50.124Z level=INFO msg="Upload complete. (296ms)"5044lb # [ 95.661449] niks3-server[1032]: 2026/09/20 12:48:50 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5045worker1 # [ 95.643528] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57180 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15046worker1 # [ 95.664171] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:50Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05047worker1 # [ 95.668555] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:50Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15048worker1 # [ 95.673077] nix-daemon[892]: reaped child process 1396, status = succeeded5049worker1 # [ 95.703509] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:571805050worker1 # [ 95.719221] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:571805051worker1 # [ 95.734131] nix-grpc-daemon[1337]: time=2026-09-20T12:48:50.228Z level=INFO msg="All 2 paths already cached"5052client # copying 1 paths to 'lb:50051'...5053client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5054worker1 # [ 95.745279] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:571805055worker1 # [ 95.759218] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5056worker1 # [ 95.967378] nix-daemon[892]: accepted connection from pid 1422, user nix-grpc-daemon (trusted)5057worker1 # [ 95.988168] nix-daemon[892]: reaped child process 1425, status = succeeded5058worker1 # [ 96.168062] nix-daemon[892]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)5059worker1 # [ 96.188307] nix-daemon[892]: reaped child process 1430, status = succeeded5060lb # [ 96.228176] niks3-server[1032]: 2026/09/20 12:48:50 INFO Received uploads request method=POST path=/api/pending_closures5061worker1 # [ 96.208991] nix-grpc-daemon[1337]: time=2026-09-20T12:48:50.703Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5062worker1 # [ 96.210676] nix-grpc-daemon[1337]: time=2026-09-20T12:48:50.703Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5063lb # [ 96.258983] niks3-server[1032]: 2026/09/20 12:48:50 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5064worker1 # [ 96.241500] nix-grpc-daemon[1337]: time=2026-09-20T12:48:50.735Z level=INFO msg="Uploading 1 narinfos"5065lb # [ 96.274407] niks3-server[1032]: 2026/09/20 12:48:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5066lb # [ 96.285929] niks3-server[1032]: 2026/09/20 12:48:50 INFO Signed narinfos id=31 count=15067lb # [ 96.294498] niks3-server[1032]: 2026/09/20 12:48:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5068lb # [ 96.298514] niks3-server[1032]: 2026/09/20 12:48:50 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5069lb # [ 96.302819] niks3-server[1032]: 2026/09/20 12:48:50 INFO Completed upload id=315070worker1 # [ 96.273387] nix-grpc-daemon[1337]: time=2026-09-20T12:48:50.768Z level=INFO msg="Upload complete. (497ms)"5071lb # [ 96.305538] niks3-server[1032]: 2026/09/20 12:48:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5072worker1 # [ 96.280191] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57180 duration_s=0 paths=1 nar_bytes_in=7685073worker1 # [ 96.295533] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:571805074worker1 # [ 96.301537] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5075worker1 # [ 96.306040] nix-daemon[892]: reaped child process 1420, status = succeeded5076client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5077worker1 # [ 96.318197] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5078client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5079client # farm-top> LOG-farm-top-on-standby5080worker1 # [ 97.368617] nix-daemon[892]: reaped child process 1434, status = succeeded5081worker1 # [ 97.543479] nix-daemon[892]: accepted connection from pid 1449, user nix-grpc-daemon (trusted)5082worker1 # [ 97.560861] nix-daemon[892]: reaped child process 1452, status = succeeded5083lb # [ 97.604966] niks3-server[1032]: 2026/09/20 12:48:52 INFO Received uploads request method=POST path=/api/pending_closures5084worker1 # [ 97.588115] nix-grpc-daemon[1337]: time=2026-09-20T12:48:52.080Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5085worker1 # [ 97.590423] nix-grpc-daemon[1337]: time=2026-09-20T12:48:52.080Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5086lb # [ 97.639461] niks3-server[1032]: 2026/09/20 12:48:52 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5087lb # [ 97.650897] niks3-server[1032]: 2026/09/20 12:48:52 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5088lb # [ 97.662766] niks3-server[1032]: 2026/09/20 12:48:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5089worker1 # [ 97.639795] nix-grpc-daemon[1337]: time=2026-09-20T12:48:52.133Z level=INFO msg="Uploading 1 narinfos"5090lb # [ 97.672386] niks3-server[1032]: 2026/09/20 12:48:52 INFO Signed narinfos id=32 count=15091lb # [ 97.675109] niks3-server[1032]: 2026/09/20 12:48:52 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5092lb # [ 97.684112] niks3-server[1032]: 2026/09/20 12:48:52 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5093lb # [ 97.693459] niks3-server[1032]: 2026/09/20 12:48:52 INFO Completed upload id=325094worker1 # [ 97.664059] nix-grpc-daemon[1337]: time=2026-09-20T12:48:52.158Z level=INFO msg="Upload complete. (332ms)"5095lb # [ 97.696948] niks3-server[1032]: 2026/09/20 12:48:52 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5096worker1 # [ 97.680621] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57180 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15097worker1 # [ 97.695605] nix-daemon[892]: reaped child process 1432, status = succeeded5098client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 7.48 seconds)5099client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5100client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.25 seconds)5101worker2 # [ 98.153020] systemd[1]: Listening on Nix worker-protocol over gRPC.5102worker2 # [ 98.159689] systemd[1]: Starting scheduler-token.service...5103worker2 # [ 98.259756] systemd[1]: scheduler-token.service: Deactivated successfully.5104worker2 # [ 98.264703] systemd[1]: Finished scheduler-token.service.5105worker2 # [ 98.266026] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 103ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5106worker2 # [ 98.274231] systemd[1]: Starting Nix worker-protocol over gRPC...5107worker2 # [ 98.452582] nix-grpc-daemon[1186]: ts=2026-09-20T12:48:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5108worker2 # [ 98.455426] nix-grpc-daemon[1186]: ts=2026-09-20T12:48:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5109worker2 # [ 98.459883] nix-grpc-daemon[1186]: ts=2026-09-20T12:48:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15110worker2 # [ 98.468273] nix-grpc-daemon[1186]: ts=2026-09-20T12:48:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5111worker2 # [ 98.470044] systemd[1]: Started Nix worker-protocol over gRPC.5112lb: must succeed: curl -sf localhost:9901/clusters5113worker2 # [ 98.498482] nix-grpc-daemon[1186]: ts=2026-09-20T12:48:52Z level=info event=scheduler_connected addr=lb:500515114worker1 # [ 98.431079] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:52Z level=info event=oidc_jwks provider=mock keys=15115worker1 # [ 98.431455] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:571745116worker1 # [ 98.440915] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05117lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5118worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5119worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5120lb: must succeed: curl -sf localhost:9901/clusters5121lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5122lb: must succeed: curl -sf localhost:9901/clusters5123lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5124worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5125worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5126(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.02 seconds)5127subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5128client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5129client # [ 98.891557] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5130client # Running as unit: rs.service; invocation ID: 5e4b2be525f34b39a45781ab1273f9895131client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds)5132client # [ 99.133029] nix[1169]: don't know how to build these paths:5133client # [ 99.134156] nix[1169]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5134worker1 # [ 99.098638] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:571805135worker1 # [ 99.113091] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:53Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05136worker1 # [ 99.120137] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:53Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15137worker1 # [ 99.159356] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:571805138worker1 # [ 99.176378] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:571805139client # [ 99.224277] nix[1169]: copying 1 paths to 'lb:50051'...5140worker1 # [ 99.190957] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:571805141client # [ 99.229401] nix[1169]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5142worker1 # [ 99.212453] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5143worker1 # [ 99.429851] nix-daemon[892]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5144worker1 # [ 99.448129] nix-daemon[892]: reaped child process 1468, status = succeeded5145worker1 # [ 99.627265] nix-daemon[892]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)5146worker1 # [ 99.644723] nix-daemon[892]: reaped child process 1473, status = succeeded5147lb # [ 99.690155] niks3-server[1032]: 2026/09/20 12:48:54 INFO Received uploads request method=POST path=/api/pending_closures5148worker1 # [ 99.671921] nix-grpc-daemon[1337]: time=2026-09-20T12:48:54.165Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5149worker1 # [ 99.673595] nix-grpc-daemon[1337]: time=2026-09-20T12:48:54.166Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5150lb # [ 99.728296] niks3-server[1032]: 2026/09/20 12:48:54 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5151worker1 # [ 99.710273] nix-grpc-daemon[1337]: time=2026-09-20T12:48:54.204Z level=INFO msg="Uploading 1 narinfos"5152lb # [ 99.743447] niks3-server[1032]: 2026/09/20 12:48:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5153lb # [ 99.751956] niks3-server[1032]: 2026/09/20 12:48:54 INFO Signed narinfos id=33 count=15154lb # [ 99.754366] niks3-server[1032]: 2026/09/20 12:48:54 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5155lb # [ 99.763070] niks3-server[1032]: 2026/09/20 12:48:54 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5156lb # [ 99.768968] niks3-server[1032]: 2026/09/20 12:48:54 INFO Completed upload id=335157worker1 # [ 99.740062] nix-grpc-daemon[1337]: time=2026-09-20T12:48:54.234Z level=INFO msg="Upload complete. (510ms)"5158lb # [ 99.771445] niks3-server[1032]: 2026/09/20 12:48:54 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5159worker1 # [ 99.746718] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57180 duration_s=0 paths=1 nar_bytes_in=4165160worker1 # [ 99.762108] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:571805161worker1 # [ 99.769654] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5162worker1 # [ 99.775093] nix-daemon[892]: reaped child process 1463, status = succeeded5163worker1 # [ 99.786274] nix-daemon[892]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5164client # [ 99.818200] nix[1169]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5165lb: must succeed: curl -sf localhost:9901/clusters5166lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5167lb: must succeed: curl -sf localhost:9901/clusters5168lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5169lb: must succeed: curl -sf localhost:9901/clusters5170lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5171lb: must succeed: curl -sf localhost:9901/clusters5172lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5173worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5174worker2 # [ 101.922148] systemd-journald[267]: Received client request to rotate journal, rotating.5175worker2 # [ 101.948143] systemd-journald[267]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e8ac0ab43e904f1cbf9078204d48ceb0.5176worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5177worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5178worker1 # [ 101.910861] systemd[1]: Stopping Nix worker-protocol over gRPC...5179worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5180client: must succeed: sleep 5.05181lb # [ 101.980185] envoy[921]: conn peer=[2001:db8:1::1]:55234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9554 tx=3820 ms=71855182worker1 # [ 102.568548] nix-grpc-daemon[1264]: ts=2026-09-20T12:48:57Z level=info event=draining5183client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5184worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5185worker1 # [ 107.446394] nix-daemon[892]: reaped child process 1477, status = succeeded5186worker1 # [ 107.629951] nix-daemon[892]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5187worker1 # [ 107.649889] nix-daemon[892]: reaped child process 1520, status = succeeded5188lb # [ 107.694700] niks3-server[1032]: 2026/09/20 12:49:02 INFO Received uploads request method=POST path=/api/pending_closures5189worker1 # [ 107.679470] nix-grpc-daemon[1337]: time=2026-09-20T12:49:02.171Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5190worker1 # [ 107.681163] nix-grpc-daemon[1337]: time=2026-09-20T12:49:02.171Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5191lb # [ 107.740428] niks3-server[1032]: 2026/09/20 12:49:02 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5192lb # [ 107.753204] niks3-server[1032]: 2026/09/20 12:49:02 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5193lb # [ 107.764183] niks3-server[1032]: 2026/09/20 12:49:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5194lb # [ 107.769987] niks3-server[1032]: 2026/09/20 12:49:02 INFO Signed narinfos id=34 count=15195worker1 # [ 107.741034] nix-grpc-daemon[1337]: time=2026-09-20T12:49:02.235Z level=INFO msg="Uploading 1 narinfos"5196lb # [ 107.774544] niks3-server[1032]: 2026/09/20 12:49:02 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5197lb # [ 107.783017] niks3-server[1032]: 2026/09/20 12:49:02 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5198lb # [ 107.788463] niks3-server[1032]: 2026/09/20 12:49:02 INFO Completed upload id=345199worker1 # [ 107.758405] nix-grpc-daemon[1337]: time=2026-09-20T12:49:02.253Z level=INFO msg="Upload complete. (348ms)"5200lb # [ 107.791062] niks3-server[1032]: 2026/09/20 12:49:02 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5201worker1 # [ 107.773126] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57180 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15202worker1 # [ 107.788136] nix-daemon[892]: reaped child process 1475, status = succeeded5203client # [ 107.846328] systemd[1]: rs.service: Deactivated successfully.5204client # [ 107.850658] systemd[1]: rs.service: Consumed 163ms CPU time over 8.958s wall clock time, 37.2M memory peak, 8.4K incoming IP traffic, 8.8K outgoing IP traffic.5205worker1 # [ 108.570484] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=signal_exit5206worker1 # [ 108.787829] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=scheduler_yield5207lb # [ 108.824824] niks3-server[1032]: 2026/09/20 12:49:03 INFO lead: released remote=192.168.1.3:334305208worker1 # [ 108.797935] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=scheduler_restarting addr=lb:500515209worker2 # [ 108.880308] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:03Z level=info event=scheduler_restarting addr=lb:500515210worker1 # [ 108.803514] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:571745211worker1 # [ 108.808245] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=worker_gone addr=192.168.1.4:500515212worker1 # [ 108.812154] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:571745213worker1 # [ 108.816509] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=worker_gone addr=192.168.1.3:500515214worker1 # [ 108.999081] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515215worker2 # [ 109.080598] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515216worker1 # [ 109.004702] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5217worker2 # [ 109.086161] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5218worker1 # [ 109.205624] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515219worker2 # [ 109.287439] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515220worker1 # [ 109.211021] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5221worker2 # [ 109.292795] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5222worker1 # [ 109.411818] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515223worker2 # [ 109.493919] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515224worker1 # [ 109.417281] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5225worker2 # [ 109.499130] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5226worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.21 seconds)5227client: waiting for success: ! systemctl is-active rs5228client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5229client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5230client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5231client: must fail: journalctl -u rs | grep -q 'reconnecting'5232worker1 # [ 109.617921] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515233worker2 # [ 109.700502] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515234worker1 # [ 109.623366] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5235worker2 # [ 109.706026] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5236client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5237lb: must succeed: curl -sf localhost:9901/clusters5238lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5239worker1 # [ 109.797628] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:04Z level=info event=scheduler_restarting5240worker2 # [ 109.908952] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515241worker2 # [ 109.912858] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5242worker1 # [ 109.847565] nix-grpc-daemon[1264]: ts=2026-09-20T12:49:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5243worker1 # [ 109.853270] nix-daemon[892]: reaped child process 1328, status = succeeded5244worker1 # [ 109.936467] nix-grpc-daemon[1264]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5245worker1 # [ 109.941181] nix-grpc-daemon[1264]: E0000 00:00:1789908544.430652 1307 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5246worker1 # [ 109.947644] nix-grpc-daemon[1264]: E0000 00:00:1789908544.431020 1307 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5247worker2 # [ 110.109945] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515248worker2 # [ 110.115080] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5249worker2 # [ 110.316253] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515250worker2 # [ 110.321425] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5251worker2 # [ 110.522623] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515252worker2 # [ 110.527940] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5253worker1 # [ 110.623267] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5254worker1 # [ 110.626645] systemd[1]: Stopped Nix worker-protocol over gRPC.5255worker1 # [ 110.632905] systemd[1]: nix-grpc-daemon.service: Consumed 1.907s CPU time over 26.277s wall clock time, 34.5M memory peak, 48.4K incoming IP traffic, 88.8K outgoing IP traffic.5256worker2 # [ 110.729406] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515257worker1 # [ 110.651566] systemd[1]: Starting Nix worker-protocol over gRPC...5258worker2 # [ 110.734683] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5259lb: must succeed: curl -sf localhost:9901/clusters5260lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5261worker1 # [ 110.851912] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5262worker1 # [ 110.854646] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5263worker2 # [ 110.938168] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515264worker1 # [ 110.859143] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15265worker2 # [ 110.942424] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5266worker1 # [ 110.867185] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5267worker1 # [ 110.871462] systemd[1]: Started Nix worker-protocol over gRPC.5268worker1 # [ 110.877541] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=scheduler_take_over5269lb # [ 110.906035] niks3-server[1032]: 2026/09/20 12:49:05 INFO lead: acquired remote=192.168.1.3:424165270worker1 # [ 110.890659] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515271worker1 # [ 110.892496] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5272worker2 # [ 111.139156] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515273worker2 # [ 111.144757] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5274worker1 # [ 111.093652] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515275worker1 # [ 111.098817] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5276worker2 # [ 111.346006] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515277worker2 # [ 111.351506] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5278worker1 # [ 111.299792] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515279worker1 # [ 111.305162] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5280worker2 # [ 111.552711] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:06Z level=info event=scheduler_connected addr=lb:500515281worker2 # [ 111.558412] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5282worker1 # [ 111.505937] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=info event=scheduler_connected addr=lb:500515283worker1 # [ 111.511336] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5284worker2 # [ 111.759838] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:06Z level=info event=scheduler_connected addr=lb:500515285worker1 # [ 111.691178] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=info event=oidc_jwks provider=mock keys=15286worker1 # [ 111.697514] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:355525287worker1 # [ 111.702907] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05288worker1 # [ 111.712071] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=info event=scheduler_connected addr=lb:500515289worker1 # [ 111.717615] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:355525290worker1 # [ 111.722445] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05291lb: must succeed: curl -sf localhost:9901/clusters5292lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5293worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5294lb # [ 111.980923] envoy[921]: conn peer=[2001:db8:1::1]:51980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2940 tx=1349 ms=86875295lb # [ 111.986019] envoy[921]: conn peer=[2001:db8:1::3]:58202 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3755 tx=1208 ms=252605296worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5297(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.21 seconds)5298subtest: scheduler restart mid-build: build finishes, no double build5299client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5300client # [ 112.099778] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5301client # Running as unit: mid.service; invocation ID: 78e466f024444faf90da7d0661a6f29f5302client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.10 seconds)5303worker1 # [ 112.307913] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355665304worker1 # [ 112.325995] nix-daemon[892]: accepted connection from pid 1527, user nix-grpc-daemon (trusted)5305client # [ 112.376264] nix[1205]: don't know how to build these paths:5306client # [ 112.378890] nix[1205]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5307worker1 # [ 112.356813] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05308worker1 # [ 112.363688] nix-grpc-daemon[1527]: ts=2026-09-20T12:49:06Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15309worker2 # [ 112.508696] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476285310worker2 # [ 112.523335] nix-daemon[861]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5311worker2 # [ 112.549324] nix-grpc-daemon[1186]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 1/5)5312worker2 # [ 112.581578] nix-grpc-daemon[1186]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 131 ms (attempt 2/5)5313worker2 # [ 112.720342] nix-grpc-daemon[1186]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 3/5)5314worker2 # [ 112.863713] nix-grpc-daemon[1186]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 454 ms (attempt 4/5)5315worker2 # [ 113.318760] nix-grpc-daemon[1186]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5316worker2 # [ 113.340858] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476285317client # [ 113.308181] nix[1205]: copying 1 paths to 'lb:50051'...5318worker2 # [ 113.357979] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:476285319client # [ 113.311958] nix[1205]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5320worker2 # [ 113.375007] nix-daemon[861]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5321worker2 # [ 113.400779] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:07Z level=info event=niks3_push_started pid=12965322worker2 # [ 113.642248] nix-grpc-daemon[1296]: time=2026-09-20T12:49:08.135Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5323worker2 # [ 113.819000] nix-daemon[861]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5324worker2 # [ 113.837016] nix-daemon[861]: reaped child process 1316, status = succeeded5325worker2 # [ 114.013381] nix-daemon[861]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5326worker2 # [ 114.031047] nix-daemon[861]: reaped child process 1321, status = succeeded5327lb # [ 113.991820] niks3-server[1032]: 2026/09/20 12:49:08 INFO Received uploads request method=POST path=/api/pending_closures5328worker2 # [ 114.057820] nix-grpc-daemon[1296]: time=2026-09-20T12:49:08.550Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5329worker2 # [ 114.059302] nix-grpc-daemon[1296]: time=2026-09-20T12:49:08.550Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5330lb # [ 114.032655] niks3-server[1032]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5331worker2 # [ 114.100948] nix-grpc-daemon[1296]: time=2026-09-20T12:49:08.594Z level=INFO msg="Uploading 1 narinfos"5332lb # [ 114.048344] niks3-server[1032]: 2026/09/20 12:49:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5333lb # [ 114.058528] niks3-server[1032]: 2026/09/20 12:49:08 INFO Signed narinfos id=35 count=15334lb # [ 114.063978] niks3-server[1032]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5335lb # [ 114.072328] niks3-server[1032]: 2026/09/20 12:49:08 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5336worker2 # [ 114.129670] nix-grpc-daemon[1296]: time=2026-09-20T12:49:08.623Z level=INFO msg="Upload complete. (482ms)"5337lb # [ 114.080091] niks3-server[1032]: 2026/09/20 12:49:08 INFO Completed upload id=355338worker2 # [ 114.136910] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47628 duration_s=0 paths=1 nar_bytes_in=4245339worker2 # [ 114.150751] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476285340worker2 # [ 114.159956] nix-grpc-daemon[1296]: time=2026-09-20T12:49:08.640Z level=WARN msg="Failed to register uploaded object" key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo error="server returned 404: object is not pending upload\n"5341worker2 # [ 114.172261] nix-daemon[861]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5342worker2 # [ 114.174215] nix-daemon[861]: reaped child process 1294, status = succeeded5343client # [ 114.133609] nix[1205]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5344worker2 # [ 114.183821] nix-daemon[861]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5345lb: must succeed: curl -sf localhost:9901/clusters5346lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5347lb: must succeed: curl -sf localhost:9901/clusters5348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5349worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5350worker1 # [ 116.121529] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1527 (nix-grpc-daemon) on client request.5351worker1 # [ 116.140551] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5352lb # [ 116.173935] niks3-server[1032]: 2026/09/20 12:49:10 INFO lead: released remote=192.168.1.3:424165353worker2 # [ 116.236957] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:10Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"5354client # [ 116.190807] nix[1205]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5355worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.11 seconds)5356client: must succeed: sleep 5.05357worker1 # [ 116.187091] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5358worker1 # [ 116.197350] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5359worker1 # [ 116.199165] systemd[1]: nix-grpc-daemon.service: Consumed 195ms CPU time over 5.533s wall clock time, 11.6M memory peak, 9.9K incoming IP traffic, 11.8K outgoing IP traffic.5360worker1 # [ 116.202817] nix-daemon[892]: reaped child process 1583, status = succeeded5361worker2 # [ 116.437336] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:10Z level=info event=scheduler_connected addr=lb:500515362worker1 # [ 116.357078] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5363worker1 # [ 116.371073] systemd[1]: Starting Nix worker-protocol over gRPC...5364worker1 # [ 116.580496] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5365worker1 # [ 116.583304] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5366worker1 # [ 116.587503] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15367worker1 # [ 116.595160] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5368worker1 # [ 116.604129] systemd[1]: Started Nix worker-protocol over gRPC.5369worker1 # [ 116.608139] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=scheduler_take_over5370lb # [ 116.637367] niks3-server[1032]: 2026/09/20 12:49:11 INFO lead: acquired remote=192.168.1.3:424365371worker1 # [ 116.627633] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=oidc_jwks provider=mock keys=15372worker1 # [ 116.630319] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:355705373worker1 # [ 116.636175] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:355705374worker1 # [ 116.638152] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=scheduler_connected addr=lb:500515375worker1 # [ 116.639711] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=15376worker1 # [ 116.643738] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05377client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5378client: waiting for success: ! systemctl is-active mid5379worker2 # [ 121.838770] nix-daemon[861]: reaped child process 1325, status = succeeded5380worker2 # [ 122.030475] nix-daemon[861]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5381lb # [ 121.981751] envoy[921]: conn peer=[2001:db8:1::3]:56376 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=972 tx=362 ms=52735382worker2 # [ 122.049853] nix-daemon[861]: reaped child process 1364, status = succeeded5383lb # [ 122.011756] niks3-server[1032]: 2026/09/20 12:49:16 INFO Received uploads request method=POST path=/api/pending_closures5384worker2 # [ 122.076437] nix-grpc-daemon[1296]: time=2026-09-20T12:49:16.568Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5385worker2 # [ 122.079692] nix-grpc-daemon[1296]: time=2026-09-20T12:49:16.568Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5386lb # [ 122.049974] niks3-server[1032]: 2026/09/20 12:49:16 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5387lb # [ 122.067225] niks3-server[1032]: 2026/09/20 12:49:16 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5388lb # [ 122.081980] niks3-server[1032]: 2026/09/20 12:49:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5389worker2 # [ 122.140217] nix-grpc-daemon[1296]: time=2026-09-20T12:49:16.633Z level=INFO msg="Uploading 1 narinfos"5390lb # [ 122.089967] niks3-server[1032]: 2026/09/20 12:49:16 INFO Signed narinfos id=36 count=15391lb # [ 122.097370] niks3-server[1032]: 2026/09/20 12:49:16 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5392lb # [ 122.112153] niks3-server[1032]: 2026/09/20 12:49:16 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5393worker2 # [ 122.169804] nix-grpc-daemon[1296]: time=2026-09-20T12:49:16.664Z level=INFO msg="Upload complete. (368ms)"5394lb # [ 122.119951] niks3-server[1032]: 2026/09/20 12:49:16 INFO Completed upload id=365395worker2 # [ 122.181082] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47628 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15396worker2 # [ 122.197179] nix-daemon[861]: reaped child process 1323, status = succeeded5397lb # [ 122.144637] niks3-server[1032]: 2026/09/20 12:49:16 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5398client # [ 122.175223] systemd[1]: mid.service: Deactivated successfully.5399client # [ 122.179453] systemd[1]: mid.service: Consumed 164ms CPU time over 10.079s wall clock time, 39.2M memory peak, 8.8K incoming IP traffic, 9.3K outgoing IP traffic.5400client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5401client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5402client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5403(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.74 seconds)5404subtest: interrupting the client stops the build on the worker5405client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5406client # [ 122.809558] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5407client # Running as unit: intr.service; invocation ID: 27c4c648f16e48d3b76890ea2a08135e5408client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5409worker1 # [ 123.008605] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590645410worker1 # [ 123.022891] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5411client # [ 123.073678] nix[1241]: don't know how to build these paths:5412client # [ 123.075131] nix[1241]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5413worker1 # [ 123.056792] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:17Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05414worker1 # [ 123.063953] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:17Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15415worker1 # [ 123.103350] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645416worker1 # [ 123.120557] nix-grpc-daemon[1611]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)5417worker1 # [ 123.209166] nix-grpc-daemon[1611]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 2/5)5418worker1 # [ 123.217916] nix-grpc-daemon[1611]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 159 ms (attempt 3/5)5419worker1 # [ 123.376969] nix-grpc-daemon[1611]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 109 ms (attempt 4/5)5420worker1 # [ 123.493983] nix-grpc-daemon[1611]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5421worker1 # [ 123.515715] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645422client # [ 123.562120] nix[1241]: copying 1 paths to 'lb:50051'...5423client # [ 123.568449] nix[1241]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5424worker1 # [ 123.534734] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645425worker1 # [ 123.549140] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5426worker1 # [ 123.578360] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:18Z level=info event=niks3_push_started pid=16815427worker1 # [ 123.802716] nix-grpc-daemon[1681]: time=2026-09-20T12:49:18.296Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5428worker1 # [ 123.977376] nix-daemon[892]: accepted connection from pid 1694, user nix-grpc-daemon (trusted)5429worker1 # [ 124.001779] nix-daemon[892]: reaped child process 1700, status = succeeded5430worker1 # [ 124.183988] nix-daemon[892]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)5431worker1 # [ 124.201164] nix-daemon[892]: reaped child process 1706, status = succeeded5432lb # [ 124.242409] niks3-server[1032]: 2026/09/20 12:49:18 INFO Received uploads request method=POST path=/api/pending_closures5433worker1 # [ 124.230116] nix-grpc-daemon[1681]: time=2026-09-20T12:49:18.723Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5434worker1 # [ 124.231648] nix-grpc-daemon[1681]: time=2026-09-20T12:49:18.723Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5435lb # [ 124.289716] niks3-server[1032]: 2026/09/20 12:49:18 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5436worker1 # [ 124.270149] nix-grpc-daemon[1681]: time=2026-09-20T12:49:18.764Z level=INFO msg="Uploading 1 narinfos"5437lb # [ 124.298664] niks3-server[1032]: 2026/09/20 12:49:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5438lb # [ 124.309128] niks3-server[1032]: 2026/09/20 12:49:18 INFO Signed narinfos id=37 count=15439lb # [ 124.320632] niks3-server[1032]: 2026/09/20 12:49:18 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5440lb # [ 124.334933] niks3-server[1032]: 2026/09/20 12:49:18 INFO Completed upload id=375441worker1 # [ 124.305786] nix-grpc-daemon[1681]: time=2026-09-20T12:49:18.800Z level=INFO msg="Upload complete. (499ms)"5442lb # [ 124.340083] niks3-server[1032]: 2026/09/20 12:49:18 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5443worker1 # [ 124.316966] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=4245444lb # [ 124.354867] niks3-server[1032]: 2026/09/20 12:49:18 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5445worker1 # [ 124.331393] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645446worker1 # [ 124.337591] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5447worker1 # [ 124.342717] nix-daemon[892]: reaped child process 1679, status = succeeded5448worker1 # [ 124.353693] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5449client # [ 124.386761] nix[1241]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5450client: must succeed: systemctl kill -s INT intr5451worker1 # [ 126.494737] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:20Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5452client # [ 126.525096] systemd[1]: intr.service: Sent signal SIGINT to main process 1241 (nix) on client request.5453client # [ 126.532932] nix[1241]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5454client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5455client # [ 126.544996] nix[1241]: error: interrupted by the user5456client # [ 126.576214] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5457client # [ 126.580433] systemd[1]: intr.service: Failed with result 'exit-code'.5458client # [ 126.582260] systemd[1]: intr.service: Consumed 144ms CPU time over 3.769s wall clock time, 39.2M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.5459worker1 # [ 126.591513] nix-daemon[892]: reaped child process 1708, status = succeeded5460worker1 # [ 126.596703] nix-daemon[1710]: unexpected Nix daemon error: error: interrupted by the user5461worker1 # [ 126.602467] nix-daemon[892]: reaped child process 1710, status = failed with exit code 15462(finished: subtest: interrupting the client stops the build on the worker, in 3.93 seconds)5463subtest: a deploy mid-build drains: the build finishes, the new generation takes over5464client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5465client # [ 126.770703] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5466client # Running as unit: sw.service; invocation ID: 94d129bdab0845a4a2655bff8cea7f5b5467client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5468client # [ 127.015040] nix[1262]: don't know how to build these paths:5469client # [ 127.016257] nix[1262]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5470worker2 # [ 127.061401] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:476285471worker1 # [ 126.994496] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:21Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05472worker1 # [ 127.000697] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:21Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15473worker2 # [ 127.119901] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476285474worker2 # [ 127.136807] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476285475client # [ 127.104279] nix[1262]: copying 1 paths to 'lb:50051'...5476worker2 # [ 127.151979] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:476285477client # [ 127.106661] nix[1262]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5478worker2 # [ 127.168636] nix-daemon[861]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5479worker2 # [ 127.377759] nix-daemon[861]: accepted connection from pid 1392, user nix-grpc-daemon (trusted)5480worker2 # [ 127.397935] nix-daemon[861]: reaped child process 1395, status = succeeded5481worker2 # [ 127.581968] nix-daemon[861]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)5482worker2 # [ 127.600553] nix-daemon[861]: reaped child process 1400, status = succeeded5483lb # [ 127.561862] niks3-server[1032]: 2026/09/20 12:49:22 INFO Received uploads request method=POST path=/api/pending_closures5484worker2 # [ 127.633899] nix-grpc-daemon[1296]: time=2026-09-20T12:49:22.127Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5485worker2 # [ 127.635404] nix-grpc-daemon[1296]: time=2026-09-20T12:49:22.127Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5486lb # [ 127.610167] niks3-server[1032]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5487worker2 # [ 127.673520] nix-grpc-daemon[1296]: time=2026-09-20T12:49:22.167Z level=INFO msg="Uploading 1 narinfos"5488lb # [ 127.624330] niks3-server[1032]: 2026/09/20 12:49:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5489lb # [ 127.632555] niks3-server[1032]: 2026/09/20 12:49:22 INFO Signed narinfos id=38 count=15490lb # [ 127.641830] niks3-server[1032]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5491lb # [ 127.654340] niks3-server[1032]: 2026/09/20 12:49:22 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5492lb # [ 127.661063] niks3-server[1032]: 2026/09/20 12:49:22 INFO Completed upload id=385493worker2 # [ 127.713150] nix-grpc-daemon[1296]: time=2026-09-20T12:49:22.207Z level=INFO msg="Upload complete. (526ms)"5494lb # [ 127.663922] niks3-server[1032]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5495worker2 # [ 127.723890] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47628 duration_s=0 paths=1 nar_bytes_in=4165496worker2 # [ 127.736834] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476285497worker2 # [ 127.748466] nix-daemon[861]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5498worker2 # [ 127.754751] nix-daemon[861]: reaped child process 1390, status = succeeded5499worker2 # [ 127.766010] nix-daemon[861]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5500client # [ 127.716839] nix[1262]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5501worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25502worker2 # Checking switch inhibitors... done5503worker2 # activating the configuration...5504worker2 # [ 130.401337] nixos[1435]: switching to system configuration /nix/store/1mq10vv1kmplwf7sq7b8gilrff9bgn7b-nixos-system-worker2-test5505worker2 # [ 130.403255] systemd[1]: Stopped target Local File Systems.5506worker2 # [ 130.408139] systemd[1]: Stopped target Remote File Systems.5507worker2 # setting up /etc...5508worker2 # [ 131.257408] systemd[1]: Reload requested from client PID 1435 ('.switch-to-conf') (unit backdoor.service)...5509worker2 # [ 131.259355] systemd[1]: Reloading...5510lb # [ 131.983829] envoy[921]: conn peer=[2001:db8:1::1]:60104 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3035 tx=1424 ms=98115511worker2 # [ 132.243488] systemd[1]: Reloading finished in 980 ms.5512worker2 # restarting sysinit-reactivation.target5513worker2 # [ 132.327066] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5514worker2 # [ 132.496465] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5515worker2 # [ 132.497674] systemd[1]: Reached target Reactivate sysinit units.5516worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5517worker2 # [ 132.503427] systemd[1]: Reloading D-Bus System Message Bus...5518worker2 # [ 132.517353] dbus-broker-launch[493]: Caught SIGHUP, trigger reload.5519worker2 # [ 132.520645] systemd[1]: Reloading Nix worker-protocol over gRPC...5520worker2 # [ 132.541303] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'...5521worker2 # [ 132.543733] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync'5522worker2 # [ 132.545183] dbus-broker-launch[493]: Invalid user-name in /nix/store/c8a4kacp5lnfwrcxrjmwczbl8ba32m4a-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5523worker2 # [ 132.580095] systemd[1]: Reloaded Nix worker-protocol over gRPC.5524worker2 # [ 132.589053] systemd[1]: Reloaded D-Bus System Message Bus.5525worker2 # [ 132.598650] systemd[1]: Reached target Remote File Systems.5526worker2 # [ 132.602190] systemd[1]: Starting linger-users.service...5527worker2 # [ 132.606286] systemd[1]: Reached target Local File Systems.5528worker2 # [ 132.627214] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5529worker2 # [ 132.630484] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5530worker2 # [ 132.631766] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5531worker2 # [ 132.644633] systemd[1]: Starting Create SUID/SGID Wrappers...5532worker2 # [ 132.645636] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5533worker2 # [ 132.647281] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675534worker2 # [ 132.658098] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5535worker2 # [ 132.659647] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5536worker2 # [ 132.662741] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5537lb # [ 131.989479] envoy[921]: conn peer=[2001:db8:1::1]:46216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2944 tx=731 ms=35195538worker2 # [ 132.700393] systemd[1]: linger-users.service: Deactivated successfully.5539worker2 # [ 132.704255] systemd[1]: Finished linger-users.service.5540worker2 # [ 133.189266] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5541worker2 # [ 133.192844] systemd[1]: Finished Create SUID/SGID Wrappers.5542worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5543worker2 # [ 133.447532] nixos[1435]: finished switching to system configuration /nix/store/1mq10vv1kmplwf7sq7b8gilrff9bgn7b-nixos-system-worker2-test5544worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.96 seconds)5545worker2: must succeed: pgrep -f 'read -t [9]0 x'5546worker2 # [ 133.489988] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:27Z level=info event=draining5547worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.07 seconds)5548client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25549client # don't know how to build these paths:5550client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5551worker1 # [ 133.732223] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590645552worker1 # [ 133.739689] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645553worker1 # [ 133.774055] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:28Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05554worker1 # [ 133.782025] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:28Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15555worker1 # [ 133.820324] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645556worker1 # [ 133.838164] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645557client # copying 1 paths to 'lb:50051'...5558worker1 # [ 133.858794] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645559client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5560worker1 # [ 133.884595] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5561worker1 # [ 134.123767] nix-daemon[892]: accepted connection from pid 1760, user nix-grpc-daemon (trusted)5562worker1 # [ 134.146304] nix-daemon[892]: reaped child process 1763, status = succeeded5563worker1 # [ 134.339056] nix-daemon[892]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)5564worker1 # [ 134.360975] nix-daemon[892]: reaped child process 1768, status = succeeded5565lb # [ 134.407211] niks3-server[1032]: 2026/09/20 12:49:28 INFO Received uploads request method=POST path=/api/pending_closures5566worker1 # [ 134.388060] nix-grpc-daemon[1681]: time=2026-09-20T12:49:28.881Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5567worker1 # [ 134.389654] nix-grpc-daemon[1681]: time=2026-09-20T12:49:28.881Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5568lb # [ 134.444272] niks3-server[1032]: 2026/09/20 12:49:28 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5569lb # [ 134.460601] niks3-server[1032]: 2026/09/20 12:49:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5570worker1 # [ 134.438594] nix-grpc-daemon[1681]: time=2026-09-20T12:49:28.932Z level=INFO msg="Uploading 1 narinfos"5571lb # [ 134.474380] niks3-server[1032]: 2026/09/20 12:49:28 INFO Signed narinfos id=39 count=15572lb # [ 134.480434] niks3-server[1032]: 2026/09/20 12:49:28 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5573lb # [ 134.487477] niks3-server[1032]: 2026/09/20 12:49:28 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5574worker1 # [ 134.464282] nix-grpc-daemon[1681]: time=2026-09-20T12:49:28.958Z level=INFO msg="Upload complete. (558ms)"5575lb # [ 134.495412] niks3-server[1032]: 2026/09/20 12:49:28 INFO Completed upload id=395576lb # [ 134.499209] niks3-server[1032]: 2026/09/20 12:49:28 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5577worker1 # [ 134.476094] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=5285578worker1 # [ 134.492087] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645579worker1 # [ 134.499388] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5580worker1 # [ 134.505632] nix-daemon[892]: reaped child process 1758, status = succeeded5581client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5582client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5583worker1 # [ 134.522223] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5584client # farm-a> LOG-farm-a-during-drain5585worker1 # [ 135.772402] nix-daemon[892]: reaped child process 1772, status = succeeded5586worker1 # [ 135.958968] nix-daemon[892]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)5587worker1 # [ 135.977035] nix-daemon[892]: reaped child process 1790, status = succeeded5588lb # [ 136.021111] niks3-server[1032]: 2026/09/20 12:49:30 INFO Received uploads request method=POST path=/api/pending_closures5589worker1 # [ 136.004530] nix-grpc-daemon[1681]: time=2026-09-20T12:49:30.496Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5590worker1 # [ 136.006613] nix-grpc-daemon[1681]: time=2026-09-20T12:49:30.496Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5591lb # [ 136.063644] niks3-server[1032]: 2026/09/20 12:49:30 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5592lb # [ 136.078259] niks3-server[1032]: 2026/09/20 12:49:30 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5593worker1 # [ 136.070507] nix-grpc-daemon[1681]: time=2026-09-20T12:49:30.563Z level=INFO msg="Uploading 1 narinfos"5594lb # [ 136.092815] niks3-server[1032]: 2026/09/20 12:49:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5595lb # [ 136.104579] niks3-server[1032]: 2026/09/20 12:49:30 INFO Signed narinfos id=40 count=15596lb # [ 136.111314] niks3-server[1032]: 2026/09/20 12:49:30 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5597lb # [ 136.120453] niks3-server[1032]: 2026/09/20 12:49:30 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5598lb # [ 136.126229] niks3-server[1032]: 2026/09/20 12:49:30 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5599worker1 # [ 136.101667] nix-grpc-daemon[1681]: time=2026-09-20T12:49:30.596Z level=INFO msg="Upload complete. (367ms)"5600lb # [ 136.132512] niks3-server[1032]: 2026/09/20 12:49:30 INFO Completed upload id=405601worker1 # [ 136.121120] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=15602worker1 # [ 136.136436] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:30Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05603worker1 # [ 136.142277] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:30Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15604worker1 # [ 136.146651] nix-daemon[892]: reaped child process 1770, status = succeeded5605worker1 # [ 136.169907] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645606worker1 # [ 136.186516] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645607worker1 # [ 136.203313] nix-grpc-daemon[1681]: time=2026-09-20T12:49:30.697Z level=INFO msg="All 1 paths already cached"5608client # copying 1 paths to 'lb:50051'...5609worker1 # [ 136.213255] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645610client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5611worker1 # [ 136.237691] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5612worker1 # [ 136.443151] nix-daemon[892]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)5613worker1 # [ 136.464128] nix-daemon[892]: reaped child process 1799, status = succeeded5614worker1 # [ 136.642092] nix-daemon[892]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)5615worker1 # [ 136.662176] nix-daemon[892]: reaped child process 1804, status = succeeded5616lb # [ 136.701944] niks3-server[1032]: 2026/09/20 12:49:31 INFO Received uploads request method=POST path=/api/pending_closures5617worker1 # [ 136.683538] nix-grpc-daemon[1681]: time=2026-09-20T12:49:31.177Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5618worker1 # [ 136.685331] nix-grpc-daemon[1681]: time=2026-09-20T12:49:31.177Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5619lb # [ 136.740404] niks3-server[1032]: 2026/09/20 12:49:31 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5620worker1 # [ 136.724774] nix-grpc-daemon[1681]: time=2026-09-20T12:49:31.219Z level=INFO msg="Uploading 1 narinfos"5621lb # [ 136.756821] niks3-server[1032]: 2026/09/20 12:49:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5622lb # [ 136.766798] niks3-server[1032]: 2026/09/20 12:49:31 INFO Signed narinfos id=41 count=15623lb # [ 136.771096] niks3-server[1032]: 2026/09/20 12:49:31 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5624lb # [ 136.781946] niks3-server[1032]: 2026/09/20 12:49:31 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5625lb # [ 136.787711] niks3-server[1032]: 2026/09/20 12:49:31 INFO Completed upload id=415626worker1 # [ 136.758066] nix-grpc-daemon[1681]: time=2026-09-20T12:49:31.252Z level=INFO msg="Upload complete. (503ms)"5627lb # [ 136.791202] niks3-server[1032]: 2026/09/20 12:49:31 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5628worker1 # [ 136.768365] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=6405629worker1 # [ 136.783564] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645630worker1 # [ 136.795762] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5631worker1 # [ 136.799046] nix-daemon[892]: reaped child process 1794, status = succeeded5632client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5633client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5634worker1 # [ 136.816728] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5635client # farm-b> LOG-farm-b-during-drain5636worker1 # [ 137.865032] nix-daemon[892]: reaped child process 1808, status = succeeded5637worker1 # [ 138.039183] nix-daemon[892]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)5638worker1 # [ 138.060246] nix-daemon[892]: reaped child process 1826, status = succeeded5639lb # [ 138.101433] niks3-server[1032]: 2026/09/20 12:49:32 INFO Received uploads request method=POST path=/api/pending_closures5640worker1 # [ 138.085314] nix-grpc-daemon[1681]: time=2026-09-20T12:49:32.579Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5641worker1 # [ 138.088465] nix-grpc-daemon[1681]: time=2026-09-20T12:49:32.582Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5642lb # [ 138.155572] niks3-server[1032]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5643worker1 # [ 138.139191] nix-grpc-daemon[1681]: time=2026-09-20T12:49:32.633Z level=INFO msg="Uploading 1 narinfos"5644lb # [ 138.170230] niks3-server[1032]: 2026/09/20 12:49:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5645lb # [ 138.177017] niks3-server[1032]: 2026/09/20 12:49:32 INFO Signed narinfos id=42 count=15646lb # [ 138.178969] niks3-server[1032]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5647lb # [ 138.184852] niks3-server[1032]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5648lb # [ 138.191156] niks3-server[1032]: 2026/09/20 12:49:32 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5649lb # [ 138.194938] niks3-server[1032]: 2026/09/20 12:49:32 INFO Completed upload id=425650worker1 # [ 138.166094] nix-grpc-daemon[1681]: time=2026-09-20T12:49:32.660Z level=INFO msg="Upload complete. (336ms)"5651lb # [ 138.197439] niks3-server[1032]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5652worker1 # [ 138.185276] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=15653worker1 # [ 138.203216] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:32Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05654worker1 # [ 138.207497] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:32Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=15655worker1 # [ 138.211947] nix-daemon[892]: reaped child process 1806, status = succeeded5656worker1 # [ 138.233443] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645657worker1 # [ 138.250070] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645658client # copying 1 paths to 'lb:50051'...5659worker1 # [ 138.265637] nix-grpc-daemon[1681]: time=2026-09-20T12:49:32.759Z level=INFO msg="All 2 paths already cached"5660worker1 # [ 138.271263] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645661client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5662worker1 # [ 138.294520] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5663worker1 # [ 138.498498] nix-daemon[892]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)5664worker1 # [ 138.519023] nix-daemon[892]: reaped child process 1835, status = succeeded5665worker1 # [ 138.692698] nix-daemon[892]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)5666worker1 # [ 138.712519] nix-daemon[892]: reaped child process 1840, status = succeeded5667lb # [ 138.752452] niks3-server[1032]: 2026/09/20 12:49:33 INFO Received uploads request method=POST path=/api/pending_closures5668worker1 # [ 138.735351] nix-grpc-daemon[1681]: time=2026-09-20T12:49:33.229Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5669worker1 # [ 138.737048] nix-grpc-daemon[1681]: time=2026-09-20T12:49:33.229Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5670lb # [ 138.784576] niks3-server[1032]: 2026/09/20 12:49:33 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5671worker1 # [ 138.770370] nix-grpc-daemon[1681]: time=2026-09-20T12:49:33.264Z level=INFO msg="Uploading 1 narinfos"5672lb # [ 138.802804] niks3-server[1032]: 2026/09/20 12:49:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5673lb # [ 138.814807] niks3-server[1032]: 2026/09/20 12:49:33 INFO Signed narinfos id=43 count=15674worker1 # [ 138.798929] nix-grpc-daemon[1681]: time=2026-09-20T12:49:33.293Z level=INFO msg="Upload complete. (486ms)"5675lb # [ 138.820998] niks3-server[1032]: 2026/09/20 12:49:33 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5676lb # [ 138.834247] niks3-server[1032]: 2026/09/20 12:49:33 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5677worker1 # [ 138.808321] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=7685678lb # [ 138.839569] niks3-server[1032]: 2026/09/20 12:49:33 INFO Completed upload id=435679lb # [ 138.842285] niks3-server[1032]: 2026/09/20 12:49:33 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5680worker1 # [ 138.816372] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645681worker1 # [ 138.829935] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5682worker1 # [ 138.833285] nix-daemon[892]: reaped child process 1830, status = succeeded5683client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5684client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5685worker1 # [ 138.847300] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5686client # farm-top> LOG-farm-top-during-drain5687worker1 # [ 140.490257] nix-daemon[892]: reaped child process 1844, status = succeeded5688worker1 # [ 140.662618] nix-daemon[892]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)5689worker1 # [ 140.683025] nix-daemon[892]: reaped child process 1862, status = succeeded5690lb # [ 140.731036] niks3-server[1032]: 2026/09/20 12:49:35 INFO Received uploads request method=POST path=/api/pending_closures5691worker1 # [ 140.713960] nix-grpc-daemon[1681]: time=2026-09-20T12:49:35.206Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5692worker1 # [ 140.715630] nix-grpc-daemon[1681]: time=2026-09-20T12:49:35.206Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5693lb # [ 140.778137] niks3-server[1032]: 2026/09/20 12:49:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5694worker1 # [ 140.760114] nix-grpc-daemon[1681]: time=2026-09-20T12:49:35.254Z level=INFO msg="Uploading 1 narinfos"5695lb # [ 140.791577] niks3-server[1032]: 2026/09/20 12:49:35 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5696lb # [ 140.803129] niks3-server[1032]: 2026/09/20 12:49:35 INFO Signed narinfos id=44 count=15697worker1 # [ 140.782266] nix-grpc-daemon[1681]: time=2026-09-20T12:49:35.276Z level=INFO msg="Upload complete. (323ms)"5698lb # [ 140.812573] niks3-server[1032]: 2026/09/20 12:49:35 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5699lb # [ 140.818444] niks3-server[1032]: 2026/09/20 12:49:35 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5700lb # [ 140.822366] niks3-server[1032]: 2026/09/20 12:49:35 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5701lb # [ 140.826562] niks3-server[1032]: 2026/09/20 12:49:35 INFO Completed upload id=445702worker1 # [ 140.796217] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=15703lb # [ 140.828840] niks3-server[1032]: 2026/09/20 12:49:35 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5704worker1 # [ 140.809764] nix-daemon[892]: reaped child process 1842, status = succeeded5705client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 7.40 seconds)5706client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5707client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5708worker2: must succeed: pkill -f 'read -t [9]0 x'5709worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5710client: waiting for success: ! systemctl is-active sw5711worker2 # [ 141.448005] nix-daemon[861]: reaped child process 1404, status = succeeded5712worker2 # [ 141.635031] nix-daemon[861]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)5713worker2 # [ 141.653395] nix-daemon[861]: reaped child process 1578, status = succeeded5714lb # [ 141.615106] niks3-server[1032]: 2026/09/20 12:49:36 INFO Received uploads request method=POST path=/api/pending_closures5715worker2 # [ 141.676455] nix-grpc-daemon[1296]: time=2026-09-20T12:49:36.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5716worker2 # [ 141.678005] nix-grpc-daemon[1296]: time=2026-09-20T12:49:36.170Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5717lb # [ 141.648622] niks3-server[1032]: 2026/09/20 12:49:36 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5718lb # [ 141.666530] niks3-server[1032]: 2026/09/20 12:49:36 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5719lb # [ 141.677062] niks3-server[1032]: 2026/09/20 12:49:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5720worker2 # [ 141.735897] nix-grpc-daemon[1296]: time=2026-09-20T12:49:36.228Z level=INFO msg="Uploading 1 narinfos"5721lb # [ 141.682991] niks3-server[1032]: 2026/09/20 12:49:36 INFO Signed narinfos id=45 count=15722lb # [ 141.688438] niks3-server[1032]: 2026/09/20 12:49:36 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5723lb # [ 141.698489] niks3-server[1032]: 2026/09/20 12:49:36 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5724lb # [ 141.707149] niks3-server[1032]: 2026/09/20 12:49:36 INFO Completed upload id=455725worker2 # [ 141.759079] nix-grpc-daemon[1296]: time=2026-09-20T12:49:36.253Z level=INFO msg="Upload complete. (356ms)"5726lb # [ 141.710096] niks3-server[1032]: 2026/09/20 12:49:36 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5727worker2 # [ 141.773215] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47628 duration_s=14 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=15728worker2 # [ 141.791709] nix-daemon[861]: reaped child process 1402, status = succeeded5729client # [ 141.767001] systemd[1]: sw.service: Deactivated successfully.5730client # [ 141.772268] systemd[1]: sw.service: Consumed 165ms CPU time over 14.999s wall clock time, 37.2M memory peak, 8.3K incoming IP traffic, 8.9K outgoing IP traffic.5731lb # [ 141.982299] envoy[921]: conn peer=[2001:db8:1::1]:37420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9625 tx=3865 ms=70925732lb # [ 141.987585] envoy[921]: conn peer=[2001:db8:1::1]:46222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1352 ms=147265733client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds)5734client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5735client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5736worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5737worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.07 seconds)5738lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5739lb: must succeed: curl -sf localhost:9901/clusters5740worker2 # [ 142.497712] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:36Z level=info event=signal_exit5741lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5742worker1 # [ 143.450910] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:355705743worker2 # [ 143.533200] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:38Z level=info event=scheduler_restarting5744worker1 # [ 143.455901] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:37Z level=info event=worker_gone addr=192.168.1.4:500515745worker2 # [ 143.554214] nix-grpc-daemon[1186]: ts=2026-09-20T12:49:38Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5746lb: must succeed: curl -sf localhost:9901/clusters5747worker2 # [ 143.583173] nix-daemon[861]: reaped child process 1289, status = succeeded5748lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5749worker2 # [ 144.212290] nix-grpc-daemon[1186]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5750worker2 # [ 144.212645] nix-grpc-daemon[1186]: E0000 00:00:1789908578.706196 1229 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5751worker2 # [ 144.229224] nix-grpc-daemon[1186]: E0000 00:00:1789908578.723318 1229 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5752lb: must succeed: curl -sf localhost:9901/clusters5753lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5754worker2 # [ 144.945936] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5755worker2 # [ 144.949462] systemd[1]: nix-grpc-daemon.service: Consumed 1.230s CPU time over 46.667s wall clock time, 31.4M memory peak, 27.1K incoming IP traffic, 49.8K outgoing IP traffic.5756lb: must succeed: curl -sf localhost:9901/clusters5757lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5758worker2 # [ 145.931332] systemd[1]: Starting scheduler-token.service...5759worker2 # [ 146.036182] systemd[1]: scheduler-token.service: Deactivated successfully.5760worker2 # [ 146.043901] systemd[1]: Finished scheduler-token.service.5761worker2 # [ 146.048630] systemd[1]: scheduler-token.service: Consumed 53ms CPU time over 108ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5762worker2 # [ 146.053316] systemd[1]: Starting Nix worker-protocol over gRPC...5763worker2 # [ 146.237601] nix-grpc-daemon[1593]: ts=2026-09-20T12:49:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5764worker2 # [ 146.240189] nix-grpc-daemon[1593]: ts=2026-09-20T12:49:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5765worker2 # [ 146.243482] nix-grpc-daemon[1593]: ts=2026-09-20T12:49:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15766worker2 # [ 146.251431] nix-grpc-daemon[1593]: ts=2026-09-20T12:49:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5767worker2 # [ 146.260497] systemd[1]: Started Nix worker-protocol over gRPC.5768worker2 # [ 146.279868] nix-grpc-daemon[1593]: ts=2026-09-20T12:49:40Z level=info event=scheduler_connected addr=lb:500515769worker1 # [ 146.203538] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:355705770worker1 # [ 146.209767] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05771lb: must succeed: curl -sf localhost:9901/clusters5772lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5773lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 4.43 seconds)5774lb: must succeed: curl -sf localhost:9901/clusters5775lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5776worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5777worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5778(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 20.31 seconds)5779subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5780client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15781worker2 # [ 147.314417] nix-grpc-daemon[1593]: ts=2026-09-20T12:49:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590525782worker2 # [ 147.330132] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)5783worker2 # [ 147.351640] nix-grpc-daemon[1593]: ts=2026-09-20T12:49:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590525784worker2 # [ 147.369097] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 1/5)5785worker2 # [ 147.447378] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 2/5)5786worker2 # [ 147.499010] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 3/5)5787worker2 # [ 147.517579] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 569 ms (attempt 4/5)5788worker2 # [ 148.094983] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5789worker1 # [ 148.067823] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:42Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05790worker1 # [ 148.073207] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:42Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15791worker1 # [ 148.079070] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:42Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05792worker1 # [ 148.084462] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:42Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05793worker1 # [ 148.115471] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645794worker1 # [ 148.131192] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645795worker1 # [ 148.147292] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645796worker1 # [ 148.162509] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5797worker1 # [ 148.360199] nix-daemon[892]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)5798worker1 # [ 148.380978] nix-daemon[892]: reaped child process 1874, status = succeeded5799worker1 # [ 148.552114] nix-daemon[892]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)5800worker1 # [ 148.571698] nix-daemon[892]: reaped child process 1879, status = succeeded5801lb # [ 148.613271] niks3-server[1032]: 2026/09/20 12:49:43 INFO Received uploads request method=POST path=/api/pending_closures5802worker1 # [ 148.594385] nix-grpc-daemon[1681]: time=2026-09-20T12:49:43.088Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5803worker1 # [ 148.595883] nix-grpc-daemon[1681]: time=2026-09-20T12:49:43.088Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5804lb # [ 148.647089] niks3-server[1032]: 2026/09/20 12:49:43 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5805lb # [ 148.663783] niks3-server[1032]: 2026/09/20 12:49:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5806worker1 # [ 148.640537] nix-grpc-daemon[1681]: time=2026-09-20T12:49:43.134Z level=INFO msg="Uploading 1 narinfos"5807lb # [ 148.676318] niks3-server[1032]: 2026/09/20 12:49:43 INFO Signed narinfos id=46 count=15808lb # [ 148.682368] niks3-server[1032]: 2026/09/20 12:49:43 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5809lb # [ 148.690447] niks3-server[1032]: 2026/09/20 12:49:43 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5810lb # [ 148.698828] niks3-server[1032]: 2026/09/20 12:49:43 INFO Completed upload id=465811worker1 # [ 148.669395] nix-grpc-daemon[1681]: time=2026-09-20T12:49:43.164Z level=INFO msg="Upload complete. (490ms)"5812lb # [ 148.702155] niks3-server[1032]: 2026/09/20 12:49:43 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5813worker1 # [ 148.678181] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=5125814worker1 # [ 148.693893] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645815worker1 # [ 148.700793] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5816worker1 # [ 148.706224] nix-daemon[892]: reaped child process 1869, status = succeeded5817worker1 # [ 148.719353] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5818worker1 # [ 150.165066] nix-daemon[892]: reaped child process 1883, status = succeeded5819worker1 # [ 150.356855] nix-daemon[892]: accepted connection from pid 1898, user nix-grpc-daemon (trusted)5820worker1 # [ 150.376613] nix-daemon[892]: reaped child process 1901, status = succeeded5821lb # [ 150.417736] niks3-server[1032]: 2026/09/20 12:49:44 INFO Received uploads request method=POST path=/api/pending_closures5822worker1 # [ 150.398303] nix-grpc-daemon[1681]: time=2026-09-20T12:49:44.892Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5823worker1 # [ 150.402992] nix-grpc-daemon[1681]: time=2026-09-20T12:49:44.894Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5824lb # [ 150.461292] niks3-server[1032]: 2026/09/20 12:49:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5825worker1 # [ 150.440803] nix-grpc-daemon[1681]: time=2026-09-20T12:49:44.935Z level=INFO msg="Uploading 1 narinfos"5826lb # [ 150.474330] niks3-server[1032]: 2026/09/20 12:49:44 INFO Signed narinfos id=47 count=15827lb # [ 150.480154] niks3-server[1032]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5828lb # [ 150.492685] niks3-server[1032]: 2026/09/20 12:49:44 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5829lb # [ 150.495903] niks3-server[1032]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5830lb # [ 150.500290] niks3-server[1032]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5831lb # [ 150.504364] niks3-server[1032]: 2026/09/20 12:49:44 INFO Completed upload id=475832worker1 # [ 150.474931] nix-grpc-daemon[1681]: time=2026-09-20T12:49:44.969Z level=INFO msg="Upload complete. (340ms)"5833lb # [ 150.506734] niks3-server[1032]: 2026/09/20 12:49:44 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5834worker1 # [ 150.493201] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=15835worker1 # [ 150.507147] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:44Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15836worker1 # [ 150.518910] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645837worker1 # [ 150.523097] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:44Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05838worker1 # [ 150.527904] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645839worker1 # [ 150.534862] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645840worker1 # [ 150.543956] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5841worker1 # [ 150.546044] nix-daemon[892]: reaped child process 1881, status = succeeded5842worker1 # [ 150.726403] nix-daemon[892]: accepted connection from pid 1907, user nix-grpc-daemon (trusted)5843worker1 # [ 150.744937] nix-daemon[892]: reaped child process 1910, status = succeeded5844worker1 # [ 150.917625] nix-daemon[892]: accepted connection from pid 1912, user nix-grpc-daemon (trusted)5845worker1 # [ 150.935050] nix-daemon[892]: reaped child process 1915, status = succeeded5846lb # [ 150.975185] niks3-server[1032]: 2026/09/20 12:49:45 INFO Received uploads request method=POST path=/api/pending_closures5847worker1 # [ 150.955319] nix-grpc-daemon[1681]: time=2026-09-20T12:49:45.449Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5848worker1 # [ 150.956949] nix-grpc-daemon[1681]: time=2026-09-20T12:49:45.449Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5849lb # [ 151.002486] niks3-server[1032]: 2026/09/20 12:49:45 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5850worker1 # [ 150.986422] nix-grpc-daemon[1681]: time=2026-09-20T12:49:45.480Z level=INFO msg="Uploading 1 narinfos"5851lb # [ 151.019237] niks3-server[1032]: 2026/09/20 12:49:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5852lb # [ 151.029645] niks3-server[1032]: 2026/09/20 12:49:45 INFO Signed narinfos id=48 count=15853lb # [ 151.034770] niks3-server[1032]: 2026/09/20 12:49:45 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5854lb # [ 151.043782] niks3-server[1032]: 2026/09/20 12:49:45 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5855lb # [ 151.049212] niks3-server[1032]: 2026/09/20 12:49:45 INFO Completed upload id=485856worker1 # [ 151.020285] nix-grpc-daemon[1681]: time=2026-09-20T12:49:45.514Z level=INFO msg="Upload complete. (469ms)"5857lb # [ 151.052324] niks3-server[1032]: 2026/09/20 12:49:45 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5858worker1 # [ 151.028261] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=5125859worker1 # [ 151.036682] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645860worker1 # [ 151.046736] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5861worker1 # [ 151.051394] nix-daemon[892]: reaped child process 1905, status = succeeded5862worker1 # [ 151.064518] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5863lb # [ 151.981800] envoy[921]: conn peer=[2001:db8:1::4]:34368 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=12988 tx=878 ms=450485864worker1 # [ 152.511085] nix-daemon[892]: reaped child process 1919, status = succeeded5865worker1 # [ 152.697015] nix-daemon[892]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)5866worker1 # [ 152.715272] nix-daemon[892]: reaped child process 1937, status = succeeded5867lb # [ 152.757154] niks3-server[1032]: 2026/09/20 12:49:47 INFO Received uploads request method=POST path=/api/pending_closures5868worker1 # [ 152.744527] nix-grpc-daemon[1681]: time=2026-09-20T12:49:47.235Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5869worker1 # [ 152.746855] nix-grpc-daemon[1681]: time=2026-09-20T12:49:47.235Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5870lb # [ 152.807808] niks3-server[1032]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5871worker1 # [ 152.790974] nix-grpc-daemon[1681]: time=2026-09-20T12:49:47.285Z level=INFO msg="Uploading 1 narinfos"5872lb # [ 152.821559] niks3-server[1032]: 2026/09/20 12:49:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5873lb # [ 152.829942] niks3-server[1032]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5874lb # [ 152.834681] niks3-server[1032]: 2026/09/20 12:49:47 INFO Signed narinfos id=49 count=15875lb # [ 152.840955] niks3-server[1032]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5876lb # [ 152.843673] niks3-server[1032]: 2026/09/20 12:49:47 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5877lb # [ 152.847623] niks3-server[1032]: 2026/09/20 12:49:47 INFO Completed upload id=495878worker1 # [ 152.818299] nix-grpc-daemon[1681]: time=2026-09-20T12:49:47.312Z level=INFO msg="Upload complete. (342ms)"5879lb # [ 152.850156] niks3-server[1032]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5880worker1 # [ 152.837188] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=15881worker1 # [ 152.849898] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:47Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15882worker1 # [ 152.859843] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645883worker1 # [ 152.865135] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:47Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05884worker1 # [ 152.870976] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645885worker1 # [ 152.877464] nix-daemon[892]: reaped child process 1917, status = succeeded5886worker1 # [ 152.890724] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645887worker1 # [ 152.896781] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5888worker1 # [ 153.084238] nix-daemon[892]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)5889worker1 # [ 153.101772] nix-daemon[892]: reaped child process 1946, status = succeeded5890worker1 # [ 153.277615] nix-daemon[892]: accepted connection from pid 1948, user nix-grpc-daemon (trusted)5891worker1 # [ 153.296162] nix-daemon[892]: reaped child process 1951, status = succeeded5892lb # [ 153.339129] niks3-server[1032]: 2026/09/20 12:49:47 INFO Received uploads request method=POST path=/api/pending_closures5893worker1 # [ 153.318976] nix-grpc-daemon[1681]: time=2026-09-20T12:49:47.813Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5894worker1 # [ 153.320655] nix-grpc-daemon[1681]: time=2026-09-20T12:49:47.813Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"5895lb # [ 153.364659] niks3-server[1032]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst5896lb # [ 153.379580] niks3-server[1032]: 2026/09/20 12:49:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5897worker1 # [ 153.356648] nix-grpc-daemon[1681]: time=2026-09-20T12:49:47.851Z level=INFO msg="Uploading 1 narinfos"5898lb # [ 153.392702] niks3-server[1032]: 2026/09/20 12:49:47 INFO Signed narinfos id=50 count=15899lb # [ 153.400406] niks3-server[1032]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls5900worker1 # [ 153.383095] nix-grpc-daemon[1681]: time=2026-09-20T12:49:47.877Z level=INFO msg="Upload complete. (484ms)"5901lb # [ 153.413118] niks3-server[1032]: 2026/09/20 12:49:47 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5902lb # [ 153.419151] niks3-server[1032]: 2026/09/20 12:49:47 INFO Completed upload id=505903worker1 # [ 153.392230] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=5125904lb # [ 153.423119] niks3-server[1032]: 2026/09/20 12:49:47 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo5905worker1 # [ 153.400680] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645906worker1 # [ 153.410688] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5907worker1 # [ 153.415412] nix-daemon[892]: reaped child process 1941, status = succeeded5908worker1 # [ 153.428838] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5909worker1 # [ 154.673015] nix-daemon[892]: reaped child process 1955, status = succeeded5910worker1 # [ 154.859459] nix-daemon[892]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)5911worker1 # [ 154.876944] nix-daemon[892]: reaped child process 1973, status = succeeded5912lb # [ 154.921162] niks3-server[1032]: 2026/09/20 12:49:49 INFO Received uploads request method=POST path=/api/pending_closures5913worker1 # [ 154.903486] nix-grpc-daemon[1681]: time=2026-09-20T12:49:49.397Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5914worker1 # [ 154.905743] nix-grpc-daemon[1681]: time=2026-09-20T12:49:49.397Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"5915lb # [ 154.956956] niks3-server[1032]: 2026/09/20 12:49:49 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv5916lb # [ 154.974720] niks3-server[1032]: 2026/09/20 12:49:49 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst5917lb # [ 154.983974] niks3-server[1032]: 2026/09/20 12:49:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5918worker1 # [ 154.959525] nix-grpc-daemon[1681]: time=2026-09-20T12:49:49.453Z level=INFO msg="Uploading 1 narinfos"5919lb # [ 154.992486] niks3-server[1032]: 2026/09/20 12:49:49 INFO Signed narinfos id=51 count=15920lb # [ 154.997694] niks3-server[1032]: 2026/09/20 12:49:49 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls5921lb # [ 155.006904] niks3-server[1032]: 2026/09/20 12:49:49 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5922lb # [ 155.011999] niks3-server[1032]: 2026/09/20 12:49:49 INFO Completed upload id=515923worker1 # [ 154.982836] nix-grpc-daemon[1681]: time=2026-09-20T12:49:49.477Z level=INFO msg="Upload complete. (343ms)"5924lb # [ 155.014479] niks3-server[1032]: 2026/09/20 12:49:49 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo5925worker1 # [ 155.001489] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=15926worker1 # [ 155.017508] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:49Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=15927worker1 # [ 155.021721] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:49Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=05928worker1 # [ 155.025212] nix-daemon[892]: reaped child process 1953, status = succeeded5929worker1 # [ 155.043638] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645930worker1 # [ 155.059570] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645931worker1 # [ 155.074696] nix-grpc-daemon[1681]: time=2026-09-20T12:49:49.569Z level=INFO msg="All 1 paths already cached"5932worker1 # [ 155.084106] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645933worker1 # [ 155.097988] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5934worker1 # [ 155.295131] nix-daemon[892]: accepted connection from pid 1979, user nix-grpc-daemon (trusted)5935worker1 # [ 155.314820] nix-daemon[892]: reaped child process 1982, status = succeeded5936worker1 # [ 155.492894] nix-daemon[892]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)5937worker1 # [ 155.510984] nix-daemon[892]: reaped child process 1987, status = succeeded5938lb # [ 155.551389] niks3-server[1032]: 2026/09/20 12:49:49 INFO Received uploads request method=POST path=/api/pending_closures5939worker1 # [ 155.532111] nix-grpc-daemon[1681]: time=2026-09-20T12:49:50.026Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5940worker1 # [ 155.533638] nix-grpc-daemon[1681]: time=2026-09-20T12:49:50.026Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"5941lb # [ 155.583277] niks3-server[1032]: 2026/09/20 12:49:50 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst5942lb # [ 155.597993] niks3-server[1032]: 2026/09/20 12:49:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5943worker1 # [ 155.574588] nix-grpc-daemon[1681]: time=2026-09-20T12:49:50.068Z level=INFO msg="Uploading 1 narinfos"5944lb # [ 155.608882] niks3-server[1032]: 2026/09/20 12:49:50 INFO Signed narinfos id=52 count=15945lb # [ 155.611320] niks3-server[1032]: 2026/09/20 12:49:50 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls5946lb # [ 155.620722] niks3-server[1032]: 2026/09/20 12:49:50 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5947lb # [ 155.626957] niks3-server[1032]: 2026/09/20 12:49:50 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo5948lb # [ 155.630878] niks3-server[1032]: 2026/09/20 12:49:50 INFO Completed upload id=525949worker1 # [ 155.602111] nix-grpc-daemon[1681]: time=2026-09-20T12:49:50.096Z level=INFO msg="Upload complete. (488ms)"5950worker1 # [ 155.612331] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=6245951worker1 # [ 155.628806] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645952worker1 # [ 155.632144] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5953worker1 # [ 155.640250] nix-daemon[892]: reaped child process 1977, status = succeeded5954worker1 # [ 155.648159] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5955worker1 # [ 156.895736] nix-daemon[892]: reaped child process 1991, status = succeeded5956worker1 # [ 157.083063] nix-daemon[892]: accepted connection from pid 2006, user nix-grpc-daemon (trusted)5957worker1 # [ 157.102661] nix-daemon[892]: reaped child process 2009, status = succeeded5958lb # [ 157.145072] niks3-server[1032]: 2026/09/20 12:49:51 INFO Received uploads request method=POST path=/api/pending_closures5959worker1 # [ 157.128443] nix-grpc-daemon[1681]: time=2026-09-20T12:49:51.621Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5960worker1 # [ 157.129927] nix-grpc-daemon[1681]: time=2026-09-20T12:49:51.621Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"5961lb # [ 157.186805] niks3-server[1032]: 2026/09/20 12:49:51 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv5962lb # [ 157.200461] niks3-server[1032]: 2026/09/20 12:49:51 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst5963lb # [ 157.210219] niks3-server[1032]: 2026/09/20 12:49:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign5964worker1 # [ 157.188555] nix-grpc-daemon[1681]: time=2026-09-20T12:49:51.682Z level=INFO msg="Uploading 1 narinfos"5965lb # [ 157.217321] niks3-server[1032]: 2026/09/20 12:49:51 INFO Signed narinfos id=53 count=15966lb # [ 157.225724] niks3-server[1032]: 2026/09/20 12:49:51 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls5967lb # [ 157.236149] niks3-server[1032]: 2026/09/20 12:49:51 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete5968lb # [ 157.243187] niks3-server[1032]: 2026/09/20 12:49:51 INFO Completed upload id=535969worker1 # [ 157.214388] nix-grpc-daemon[1681]: time=2026-09-20T12:49:51.708Z level=INFO msg="Upload complete. (355ms)"5970lb # [ 157.246793] niks3-server[1032]: 2026/09/20 12:49:51 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo5971worker1 # [ 157.229208] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=15972worker1 # [ 157.245970] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:51Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=15973worker1 # [ 157.250765] nix-daemon[892]: reaped child process 1989, status = succeeded5974worker1 # [ 157.270782] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645975worker1 # [ 157.279055] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:51Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=05976worker1 # [ 157.290948] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645977worker1 # [ 157.305286] nix-grpc-daemon[1681]: time=2026-09-20T12:49:51.799Z level=INFO msg="All 1 paths already cached"5978worker1 # [ 157.314758] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645979worker1 # [ 157.328154] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5980worker1 # [ 157.534658] nix-daemon[892]: accepted connection from pid 2015, user nix-grpc-daemon (trusted)5981worker1 # [ 157.552470] nix-daemon[892]: reaped child process 2018, status = succeeded5982worker1 # [ 157.726985] nix-daemon[892]: accepted connection from pid 2020, user nix-grpc-daemon (trusted)5983worker1 # [ 157.745967] nix-daemon[892]: reaped child process 2023, status = succeeded5984lb # [ 157.785806] niks3-server[1032]: 2026/09/20 12:49:52 INFO Received uploads request method=POST path=/api/pending_closures5985worker1 # [ 157.765837] nix-grpc-daemon[1681]: time=2026-09-20T12:49:52.260Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5986worker1 # [ 157.767355] nix-grpc-daemon[1681]: time=2026-09-20T12:49:52.260Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"5987lb # [ 157.815228] niks3-server[1032]: 2026/09/20 12:49:52 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst5988lb # [ 157.826471] niks3-server[1032]: 2026/09/20 12:49:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign5989worker1 # [ 157.804287] nix-grpc-daemon[1681]: time=2026-09-20T12:49:52.298Z level=INFO msg="Uploading 1 narinfos"5990lb # [ 157.837465] niks3-server[1032]: 2026/09/20 12:49:52 INFO Signed narinfos id=54 count=15991lb # [ 157.843578] niks3-server[1032]: 2026/09/20 12:49:52 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls5992lb # [ 157.852475] niks3-server[1032]: 2026/09/20 12:49:52 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete5993lb # [ 157.857540] niks3-server[1032]: 2026/09/20 12:49:52 INFO Completed upload id=545994worker1 # [ 157.829572] nix-grpc-daemon[1681]: time=2026-09-20T12:49:52.324Z level=INFO msg="Upload complete. (484ms)"5995lb # [ 157.860325] niks3-server[1032]: 2026/09/20 12:49:52 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo5996worker1 # [ 157.836360] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=6245997worker1 # [ 157.849929] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645998worker1 # [ 157.855724] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5999worker1 # [ 157.860821] nix-daemon[892]: reaped child process 2013, status = succeeded6000worker1 # [ 157.873272] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6001worker1 # [ 159.322173] nix-daemon[892]: reaped child process 2027, status = succeeded6002worker1 # [ 159.495509] nix-daemon[892]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)6003worker1 # [ 159.515142] nix-daemon[892]: reaped child process 2045, status = succeeded6004lb # [ 159.561080] niks3-server[1032]: 2026/09/20 12:49:54 INFO Received uploads request method=POST path=/api/pending_closures6005worker1 # [ 159.543937] nix-grpc-daemon[1681]: time=2026-09-20T12:49:54.037Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6006worker1 # [ 159.546229] nix-grpc-daemon[1681]: time=2026-09-20T12:49:54.037Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6007lb # [ 159.614966] niks3-server[1032]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6008lb # [ 159.627097] niks3-server[1032]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6009lb # [ 159.634264] niks3-server[1032]: 2026/09/20 12:49:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6010lb # [ 159.638482] niks3-server[1032]: 2026/09/20 12:49:54 INFO Signed narinfos id=55 count=16011worker1 # [ 159.609414] nix-grpc-daemon[1681]: time=2026-09-20T12:49:54.103Z level=INFO msg="Uploading 1 narinfos"6012lb # [ 159.641057] niks3-server[1032]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6013lb # [ 159.649000] niks3-server[1032]: 2026/09/20 12:49:54 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6014lb # [ 159.657147] niks3-server[1032]: 2026/09/20 12:49:54 INFO Completed upload id=556015worker1 # [ 159.628052] nix-grpc-daemon[1681]: time=2026-09-20T12:49:54.122Z level=INFO msg="Upload complete. (344ms)"6016lb # [ 159.659533] niks3-server[1032]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6017worker1 # [ 159.645923] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=16018worker1 # [ 159.665086] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:54Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16019worker1 # [ 159.675925] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590646020worker1 # [ 159.682091] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:54Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06021worker1 # [ 159.687351] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590646022worker1 # [ 159.693901] nix-daemon[892]: reaped child process 2025, status = succeeded6023worker1 # [ 159.707170] nix-grpc-daemon[1681]: time=2026-09-20T12:49:54.181Z level=INFO msg="All 1 paths already cached"6024worker1 # [ 159.711550] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6025worker1 # [ 159.716461] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590646026worker1 # [ 159.898731] nix-daemon[892]: accepted connection from pid 2051, user nix-grpc-daemon (trusted)6027worker1 # [ 159.916449] nix-daemon[892]: reaped child process 2054, status = succeeded6028worker1 # [ 160.087615] nix-daemon[892]: accepted connection from pid 2056, user nix-grpc-daemon (trusted)6029worker1 # [ 160.103327] nix-daemon[892]: reaped child process 2059, status = succeeded6030lb # [ 160.147530] niks3-server[1032]: 2026/09/20 12:49:54 INFO Received uploads request method=POST path=/api/pending_closures6031worker1 # [ 160.129039] nix-grpc-daemon[1681]: time=2026-09-20T12:49:54.623Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6032worker1 # [ 160.130510] nix-grpc-daemon[1681]: time=2026-09-20T12:49:54.623Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6033lb # [ 160.181607] niks3-server[1032]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6034worker1 # [ 160.167198] nix-grpc-daemon[1681]: time=2026-09-20T12:49:54.661Z level=INFO msg="Uploading 1 narinfos"6035lb # [ 160.197850] niks3-server[1032]: 2026/09/20 12:49:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6036lb # [ 160.208417] niks3-server[1032]: 2026/09/20 12:49:54 INFO Signed narinfos id=56 count=16037lb # [ 160.213783] niks3-server[1032]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6038lb # [ 160.221227] niks3-server[1032]: 2026/09/20 12:49:54 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6039lb # [ 160.226022] niks3-server[1032]: 2026/09/20 12:49:54 INFO Completed upload id=566040worker1 # [ 160.196357] nix-grpc-daemon[1681]: time=2026-09-20T12:49:54.691Z level=INFO msg="Upload complete. (480ms)"6041lb # [ 160.228699] niks3-server[1032]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6042worker1 # [ 160.204166] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=6246043worker1 # [ 160.218930] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590646044worker1 # [ 160.225714] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6045worker1 # [ 160.230580] nix-daemon[892]: reaped child process 2049, status = succeeded6046worker1 # [ 160.244204] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6047worker1 # [ 161.491056] nix-daemon[892]: reaped child process 2063, status = succeeded6048worker1 # [ 161.673447] nix-daemon[892]: accepted connection from pid 2078, user nix-grpc-daemon (trusted)6049worker1 # [ 161.693638] nix-daemon[892]: reaped child process 2081, status = succeeded6050lb # [ 161.735721] niks3-server[1032]: 2026/09/20 12:49:56 INFO Received uploads request method=POST path=/api/pending_closures6051worker1 # [ 161.720434] nix-grpc-daemon[1681]: time=2026-09-20T12:49:56.212Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6052worker1 # [ 161.721970] nix-grpc-daemon[1681]: time=2026-09-20T12:49:56.212Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6053lb # [ 161.781403] niks3-server[1032]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6054lb # [ 161.794539] niks3-server[1032]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6055lb # [ 161.804174] niks3-server[1032]: 2026/09/20 12:49:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6056worker1 # [ 161.780252] nix-grpc-daemon[1681]: time=2026-09-20T12:49:56.274Z level=INFO msg="Uploading 1 narinfos"6057lb # [ 161.812733] niks3-server[1032]: 2026/09/20 12:49:56 INFO Signed narinfos id=57 count=16058lb # [ 161.814963] niks3-server[1032]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6059lb # [ 161.823499] niks3-server[1032]: 2026/09/20 12:49:56 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6060lb # [ 161.828912] niks3-server[1032]: 2026/09/20 12:49:56 INFO Completed upload id=576061worker1 # [ 161.800177] nix-grpc-daemon[1681]: time=2026-09-20T12:49:56.294Z level=INFO msg="Upload complete. (344ms)"6062lb # [ 161.831425] niks3-server[1032]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6063worker1 # [ 161.813253] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=16064worker1 # [ 161.829599] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:56Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16065worker1 # [ 161.834235] nix-daemon[892]: reaped child process 2061, status = succeeded6066worker1 # [ 161.854082] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590646067worker1 # [ 161.862769] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:56Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06068worker1 # [ 161.874001] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590646069worker1 # [ 161.887534] nix-grpc-daemon[1681]: time=2026-09-20T12:49:56.382Z level=INFO msg="All 2 paths already cached"6070worker1 # [ 161.896498] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590646071worker1 # [ 161.909406] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6072worker1 # [ 162.104166] nix-daemon[892]: accepted connection from pid 2087, user nix-grpc-daemon (trusted)6073worker1 # [ 162.121550] nix-daemon[892]: reaped child process 2090, status = succeeded6074worker1 # [ 162.296222] nix-daemon[892]: accepted connection from pid 2092, user nix-grpc-daemon (trusted)6075worker1 # [ 162.313813] nix-daemon[892]: reaped child process 2095, status = succeeded6076lb # [ 162.357055] niks3-server[1032]: 2026/09/20 12:49:56 INFO Received uploads request method=POST path=/api/pending_closures6077worker1 # [ 162.337819] nix-grpc-daemon[1681]: time=2026-09-20T12:49:56.832Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6078worker1 # [ 162.339419] nix-grpc-daemon[1681]: time=2026-09-20T12:49:56.832Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6079lb # [ 162.384594] niks3-server[1032]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6080lb # [ 162.399593] niks3-server[1032]: 2026/09/20 12:49:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6081worker1 # [ 162.376544] nix-grpc-daemon[1681]: time=2026-09-20T12:49:56.871Z level=INFO msg="Uploading 1 narinfos"6082lb # [ 162.412365] niks3-server[1032]: 2026/09/20 12:49:56 INFO Signed narinfos id=58 count=16083lb # [ 162.415619] niks3-server[1032]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6084lb # [ 162.428064] niks3-server[1032]: 2026/09/20 12:49:56 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6085lb # [ 162.433299] niks3-server[1032]: 2026/09/20 12:49:56 INFO Completed upload id=586086worker1 # [ 162.404368] nix-grpc-daemon[1681]: time=2026-09-20T12:49:56.898Z level=INFO msg="Upload complete. (479ms)"6087lb # [ 162.436286] niks3-server[1032]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6088worker1 # [ 162.412149] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=7526089worker1 # [ 162.420679] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590646090worker1 # [ 162.431231] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6091worker1 # [ 162.435952] nix-daemon[892]: reaped child process 2085, status = succeeded6092worker1 # [ 162.447400] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6093worker1 # [ 163.698478] nix-daemon[892]: reaped child process 2099, status = succeeded6094worker1 # [ 163.882479] nix-daemon[892]: accepted connection from pid 2114, user nix-grpc-daemon (trusted)6095worker1 # [ 163.900480] nix-daemon[892]: reaped child process 2117, status = succeeded6096lb # [ 163.942387] niks3-server[1032]: 2026/09/20 12:49:58 INFO Received uploads request method=POST path=/api/pending_closures6097worker1 # [ 163.924419] nix-grpc-daemon[1681]: time=2026-09-20T12:49:58.417Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6098worker1 # [ 163.925941] nix-grpc-daemon[1681]: time=2026-09-20T12:49:58.417Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6099lb # [ 163.975357] niks3-server[1032]: 2026/09/20 12:49:58 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6100lb # [ 163.987777] niks3-server[1032]: 2026/09/20 12:49:58 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6101worker1 # [ 163.977277] nix-grpc-daemon[1681]: time=2026-09-20T12:49:58.471Z level=INFO msg="Uploading 1 narinfos"6102lb # [ 164.001266] niks3-server[1032]: 2026/09/20 12:49:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6103lb # [ 164.010867] niks3-server[1032]: 2026/09/20 12:49:58 INFO Signed narinfos id=59 count=16104lb # [ 164.018216] niks3-server[1032]: 2026/09/20 12:49:58 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6105lb # [ 164.025725] niks3-server[1032]: 2026/09/20 12:49:58 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6106lb # [ 164.033199] niks3-server[1032]: 2026/09/20 12:49:58 INFO Completed upload id=596107lb # [ 164.035862] niks3-server[1032]: 2026/09/20 12:49:58 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6108worker1 # [ 164.003662] nix-grpc-daemon[1681]: time=2026-09-20T12:49:58.498Z level=INFO msg="Upload complete. (341ms)"6109worker1 # [ 164.017397] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=16110worker1 # [ 164.034719] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:58Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16111worker1 # [ 164.039829] nix-daemon[892]: reaped child process 2097, status = succeeded6112worker1 # [ 164.059413] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590646113worker1 # [ 164.074605] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590646114worker1 # [ 164.088686] nix-grpc-daemon[1681]: time=2026-09-20T12:49:58.583Z level=INFO msg="All 2 paths already cached"6115worker1 # [ 164.097758] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590646116worker1 # [ 164.111549] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6117worker1 # [ 164.313645] nix-daemon[892]: accepted connection from pid 2121, user nix-grpc-daemon (trusted)6118worker1 # [ 164.332778] nix-daemon[892]: reaped child process 2124, status = succeeded6119worker1 # [ 164.509855] nix-daemon[892]: accepted connection from pid 2126, user nix-grpc-daemon (trusted)6120worker1 # [ 164.527325] nix-daemon[892]: reaped child process 2129, status = succeeded6121lb # [ 164.568261] niks3-server[1032]: 2026/09/20 12:49:59 INFO Received uploads request method=POST path=/api/pending_closures6122worker1 # [ 164.548069] nix-grpc-daemon[1681]: time=2026-09-20T12:49:59.042Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6123worker1 # [ 164.549611] nix-grpc-daemon[1681]: time=2026-09-20T12:49:59.042Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6124lb # [ 164.600469] niks3-server[1032]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6125lb # [ 164.612594] niks3-server[1032]: 2026/09/20 12:49:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6126worker1 # [ 164.591429] nix-grpc-daemon[1681]: time=2026-09-20T12:49:59.085Z level=INFO msg="Uploading 1 narinfos"6127lb # [ 164.623831] niks3-server[1032]: 2026/09/20 12:49:59 INFO Signed narinfos id=60 count=16128lb # [ 164.630643] niks3-server[1032]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6129lb # [ 164.642890] niks3-server[1032]: 2026/09/20 12:49:59 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6130worker1 # [ 164.619762] nix-grpc-daemon[1681]: time=2026-09-20T12:49:59.114Z level=INFO msg="Upload complete. (493ms)"6131lb # [ 164.649312] niks3-server[1032]: 2026/09/20 12:49:59 INFO Completed upload id=606132lb # [ 164.653728] niks3-server[1032]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6133worker1 # [ 164.628325] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=7526134worker1 # [ 164.636957] nix-grpc-daemon[1611]: ts=2026-09-20T12:49:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590646135worker1 # [ 164.646872] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6136worker1 # [ 164.652738] nix-daemon[892]: reaped child process 2119, status = succeeded6137worker1 # [ 164.662443] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6138worker1 # [ 165.703852] nix-daemon[892]: reaped child process 2133, status = succeeded6139worker1 # [ 165.879694] nix-daemon[892]: accepted connection from pid 2148, user nix-grpc-daemon (trusted)6140worker1 # [ 165.899689] nix-daemon[892]: reaped child process 2151, status = succeeded6141lb # [ 165.942566] niks3-server[1032]: 2026/09/20 12:50:00 INFO Received uploads request method=POST path=/api/pending_closures6142worker1 # [ 165.924418] nix-grpc-daemon[1681]: time=2026-09-20T12:50:00.417Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6143worker1 # [ 165.925888] nix-grpc-daemon[1681]: time=2026-09-20T12:50:00.417Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6144lb # [ 166.000801] niks3-server[1032]: 2026/09/20 12:50:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6145worker1 # [ 165.977660] nix-grpc-daemon[1681]: time=2026-09-20T12:50:00.471Z level=INFO msg="Uploading 1 narinfos"6146lb # [ 166.010937] niks3-server[1032]: 2026/09/20 12:50:00 INFO Signed narinfos id=61 count=16147lb # [ 166.014120] niks3-server[1032]: 2026/09/20 12:50:00 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6148lb # [ 166.027180] niks3-server[1032]: 2026/09/20 12:50:00 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6149worker1 # [ 165.998197] nix-grpc-daemon[1681]: time=2026-09-20T12:50:00.492Z level=INFO msg="Upload complete. (323ms)"6150lb # [ 166.030994] niks3-server[1032]: 2026/09/20 12:50:00 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6151lb # [ 166.035019] niks3-server[1032]: 2026/09/20 12:50:00 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6152lb # [ 166.038242] niks3-server[1032]: 2026/09/20 12:50:00 INFO Completed upload id=616153lb # [ 166.040162] niks3-server[1032]: 2026/09/20 12:50:00 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6154worker1 # [ 166.010091] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=16155worker1 # [ 166.025079] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:00Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16156worker1 # [ 166.029070] nix-daemon[892]: reaped child process 2131, status = succeeded6157worker1 # [ 166.058139] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590646158worker1 # [ 166.074164] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590646159worker1 # [ 166.090039] nix-grpc-daemon[1681]: time=2026-09-20T12:50:00.584Z level=INFO msg="All 2 paths already cached"6160worker1 # [ 166.099185] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590646161worker1 # [ 166.113465] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6162worker1 # [ 166.311543] nix-daemon[892]: accepted connection from pid 2155, user nix-grpc-daemon (trusted)6163worker1 # [ 166.331552] nix-daemon[892]: reaped child process 2158, status = succeeded6164worker1 # [ 166.509237] nix-daemon[892]: accepted connection from pid 2160, user nix-grpc-daemon (trusted)6165worker1 # [ 166.527159] nix-daemon[892]: reaped child process 2163, status = succeeded6166lb # [ 166.568353] niks3-server[1032]: 2026/09/20 12:50:01 INFO Received uploads request method=POST path=/api/pending_closures6167worker1 # [ 166.552788] nix-grpc-daemon[1681]: time=2026-09-20T12:50:01.047Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6168worker1 # [ 166.554376] nix-grpc-daemon[1681]: time=2026-09-20T12:50:01.047Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6169lb # [ 166.603262] niks3-server[1032]: 2026/09/20 12:50:01 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6170worker1 # [ 166.594759] nix-grpc-daemon[1681]: time=2026-09-20T12:50:01.089Z level=INFO msg="Uploading 1 narinfos"6171lb # [ 166.616229] niks3-server[1032]: 2026/09/20 12:50:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6172lb # [ 166.628790] niks3-server[1032]: 2026/09/20 12:50:01 INFO Signed narinfos id=62 count=16173lb # [ 166.631537] niks3-server[1032]: 2026/09/20 12:50:01 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6174worker1 # [ 166.617981] nix-grpc-daemon[1681]: time=2026-09-20T12:50:01.112Z level=INFO msg="Upload complete. (488ms)"6175lb # [ 166.645820] niks3-server[1032]: 2026/09/20 12:50:01 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6176lb # [ 166.651635] niks3-server[1032]: 2026/09/20 12:50:01 INFO Completed upload id=626177lb # [ 166.655071] niks3-server[1032]: 2026/09/20 12:50:01 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6178worker1 # [ 166.626549] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=7526179worker1 # [ 166.634946] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590646180worker1 # [ 166.645893] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6181worker1 # [ 166.655091] nix-daemon[892]: reaped child process 2153, status = succeeded6182worker1 # [ 166.663085] nix-daemon[892]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6183worker1 # [ 167.686576] nix-daemon[892]: reaped child process 2167, status = succeeded6184worker1 # [ 167.850154] nix-daemon[892]: accepted connection from pid 2182, user nix-grpc-daemon (trusted)6185worker1 # [ 167.869110] nix-daemon[892]: reaped child process 2185, status = succeeded6186lb # [ 167.910142] niks3-server[1032]: 2026/09/20 12:50:02 INFO Received uploads request method=POST path=/api/pending_closures6187worker1 # [ 167.891790] nix-grpc-daemon[1681]: time=2026-09-20T12:50:02.385Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6188worker1 # [ 167.893476] nix-grpc-daemon[1681]: time=2026-09-20T12:50:02.385Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6189lb # [ 167.951385] niks3-server[1032]: 2026/09/20 12:50:02 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6190lb # [ 167.963886] niks3-server[1032]: 2026/09/20 12:50:02 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6191lb # [ 167.971870] niks3-server[1032]: 2026/09/20 12:50:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6192worker1 # [ 167.947908] nix-grpc-daemon[1681]: time=2026-09-20T12:50:02.442Z level=INFO msg="Uploading 1 narinfos"6193lb # [ 167.980513] niks3-server[1032]: 2026/09/20 12:50:02 INFO Signed narinfos id=63 count=16194lb # [ 167.982676] niks3-server[1032]: 2026/09/20 12:50:02 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6195lb # [ 167.990390] niks3-server[1032]: 2026/09/20 12:50:02 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6196lb # [ 167.995726] niks3-server[1032]: 2026/09/20 12:50:02 INFO Completed upload id=636197worker1 # [ 167.966103] nix-grpc-daemon[1681]: time=2026-09-20T12:50:02.460Z level=INFO msg="Upload complete. (293ms)"6198lb # [ 167.998341] niks3-server[1032]: 2026/09/20 12:50:02 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6199worker1 # [ 167.976670] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=16200worker1 # [ 167.992581] nix-daemon[892]: reaped child process 2165, status = succeeded6201client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.08 seconds)6202worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6203worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6204client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6205client # [ 168.231297] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6206client # Running as unit: gpu.service; invocation ID: f0fceabd483e4a1987f321459fcf34da6207client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.10 seconds)6208lb: must succeed: curl -sf localhost:9901/clusters6209lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6210lb: must succeed: curl -sf localhost:9901/clusters6211lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6212worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6213client # [ 168.488969] nix[1341]: don't know how to build these paths:6214client # [ 168.490129] nix[1341]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6215worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6216worker2 # [ 168.536279] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590526217worker2 # [ 168.543169] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590526218worker1 # [ 168.486580] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:02Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06219client # [ 168.527064] nix[1341]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6220lb: must succeed: curl -sf localhost:9901/clusters6221lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6222lb: must succeed: curl -sf localhost:9901/clusters6223lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6224worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6225worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6226client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6227lb # [ 171.981965] envoy[921]: conn peer=[2001:db8:1::1]:49252 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27187 tx=10746 ms=207706228client # [ 198.534666] nix[1341]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6229client # [ 198.546003] nix[1341]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6230client # [ 198.596593] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6231client # [ 198.603765] systemd[1]: gpu.service: Failed with result 'exit-code'.6232client # [ 198.607703] systemd[1]: gpu.service: Consumed 175ms CPU time over 30.370s wall clock time, 33.2M memory peak, 6.7K incoming IP traffic, 5.8K outgoing IP traffic.6233client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.41 seconds)6234client: must succeed: journalctl -u gpu -o cat6235client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6236lb: must succeed: curl -sf localhost:9901/clusters6237lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6238lb: must succeed: curl -sf localhost:9901/clusters6239lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6240worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6241worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6242(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 52.38 seconds)6243subtest: low disk drains a worker and builds go to the other6244worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6245worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.14 seconds)6246worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6247worker1 # [ 200.664289] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:35Z level=info event=unhealthy reason=min_free6248worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.13 seconds)6249client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26250worker2 # [ 201.949485] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590526251client # don't know how to build these paths:6252client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6253worker1 # [ 201.880831] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590646254worker1 # [ 201.912136] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:36Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06255worker1 # [ 201.918237] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:36Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16256lb # [ 201.982477] envoy[921]: conn peer=[2001:db8:1::1]:33834 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300766257worker2 # [ 202.044970] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590526258worker2 # [ 202.062989] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590526259client # copying 1 paths to 'lb:50051'...6260client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6261worker2 # [ 202.078924] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590526262worker2 # [ 202.097555] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6263worker2 # [ 202.130586] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:36Z level=info event=niks3_push_started pid=16496264worker2 # [ 202.349818] nix-grpc-daemon[1649]: time=2026-09-20T12:50:36.843Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""6265worker2 # [ 202.525906] nix-daemon[861]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)6266worker2 # [ 202.544213] nix-daemon[861]: reaped child process 1665, status = succeeded6267worker2 # [ 202.714189] nix-daemon[861]: accepted connection from pid 1667, user nix-grpc-daemon (trusted)6268worker2 # [ 202.730996] nix-daemon[861]: reaped child process 1670, status = succeeded6269lb # [ 202.691904] niks3-server[1032]: 2026/09/20 12:50:37 INFO Received uploads request method=POST path=/api/pending_closures6270worker2 # [ 202.757329] nix-grpc-daemon[1649]: time=2026-09-20T12:50:37.250Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6271worker2 # [ 202.758829] nix-grpc-daemon[1649]: time=2026-09-20T12:50:37.250Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6272lb # [ 202.734540] niks3-server[1032]: 2026/09/20 12:50:37 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6273worker2 # [ 202.799118] nix-grpc-daemon[1649]: time=2026-09-20T12:50:37.292Z level=INFO msg="Uploading 1 narinfos"6274lb # [ 202.750409] niks3-server[1032]: 2026/09/20 12:50:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6275lb # [ 202.760329] niks3-server[1032]: 2026/09/20 12:50:37 INFO Signed narinfos id=64 count=16276lb # [ 202.768195] niks3-server[1032]: 2026/09/20 12:50:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6277lb # [ 202.771074] niks3-server[1032]: 2026/09/20 12:50:37 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6278lb # [ 202.776165] niks3-server[1032]: 2026/09/20 12:50:37 INFO Completed upload id=646279worker2 # [ 202.827271] nix-grpc-daemon[1649]: time=2026-09-20T12:50:37.321Z level=INFO msg="Upload complete. (473ms)"6280lb # [ 202.778907] niks3-server[1032]: 2026/09/20 12:50:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6281worker2 # [ 202.836282] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59052 duration_s=0 paths=1 nar_bytes_in=5126282worker2 # [ 202.848904] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590526283worker2 # [ 202.855277] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6284worker2 # [ 202.860261] nix-daemon[861]: reaped child process 1647, status = succeeded6285client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6286client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6287worker2 # [ 202.873300] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6288client # farm-a> LOG-farm-a-drain6289worker2 # [ 204.521538] nix-daemon[861]: reaped child process 1674, status = succeeded6290worker2 # [ 204.690939] nix-daemon[861]: accepted connection from pid 1689, user nix-grpc-daemon (trusted)6291worker2 # [ 204.707737] nix-daemon[861]: reaped child process 1692, status = succeeded6292lb # [ 204.669027] niks3-server[1032]: 2026/09/20 12:50:39 INFO Received uploads request method=POST path=/api/pending_closures6293worker2 # [ 204.732587] nix-grpc-daemon[1649]: time=2026-09-20T12:50:39.224Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6294worker2 # [ 204.734633] nix-grpc-daemon[1649]: time=2026-09-20T12:50:39.224Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6295lb # [ 204.718997] niks3-server[1032]: 2026/09/20 12:50:39 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6296worker2 # [ 204.783294] nix-grpc-daemon[1649]: time=2026-09-20T12:50:39.275Z level=INFO msg="Uploading 1 narinfos"6297lb # [ 204.727883] niks3-server[1032]: 2026/09/20 12:50:39 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6298lb # [ 204.742019] niks3-server[1032]: 2026/09/20 12:50:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6299lb # [ 204.746035] niks3-server[1032]: 2026/09/20 12:50:39 INFO Signed narinfos id=65 count=16300lb # [ 204.752338] niks3-server[1032]: 2026/09/20 12:50:39 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6301worker2 # [ 204.803465] nix-grpc-daemon[1649]: time=2026-09-20T12:50:39.297Z level=INFO msg="Upload complete. (319ms)"6302lb # [ 204.755860] niks3-server[1032]: 2026/09/20 12:50:39 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6303lb # [ 204.760228] niks3-server[1032]: 2026/09/20 12:50:39 INFO Completed upload id=656304lb # [ 204.762651] niks3-server[1032]: 2026/09/20 12:50:39 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6305worker2 # [ 204.817163] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59052 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=16306worker1 # [ 204.745001] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:39Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06307worker2 # [ 204.834331] nix-daemon[861]: reaped child process 1672, status = succeeded6308worker1 # [ 204.750496] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:39Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16309worker2 # [ 204.872961] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590526310worker2 # [ 204.888917] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590526311worker2 # [ 204.903255] nix-grpc-daemon[1649]: time=2026-09-20T12:50:39.397Z level=INFO msg="All 1 paths already cached"6312client # copying 1 paths to 'lb:50051'...6313worker2 # [ 204.913547] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590526314client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6315worker2 # [ 204.930846] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6316worker2 # [ 205.128433] nix-daemon[861]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6317worker2 # [ 205.147061] nix-daemon[861]: reaped child process 1699, status = succeeded6318worker2 # [ 205.314639] nix-daemon[861]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6319worker2 # [ 205.332431] nix-daemon[861]: reaped child process 1704, status = succeeded6320lb # [ 205.295010] niks3-server[1032]: 2026/09/20 12:50:39 INFO Received uploads request method=POST path=/api/pending_closures6321worker2 # [ 205.355561] nix-grpc-daemon[1649]: time=2026-09-20T12:50:39.849Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6322worker2 # [ 205.357266] nix-grpc-daemon[1649]: time=2026-09-20T12:50:39.849Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6323lb # [ 205.319832] niks3-server[1032]: 2026/09/20 12:50:39 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6324worker2 # [ 205.392685] nix-grpc-daemon[1649]: time=2026-09-20T12:50:39.886Z level=INFO msg="Uploading 1 narinfos"6325lb # [ 205.335471] niks3-server[1032]: 2026/09/20 12:50:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6326lb # [ 205.347355] niks3-server[1032]: 2026/09/20 12:50:39 INFO Signed narinfos id=66 count=16327lb # [ 205.355762] niks3-server[1032]: 2026/09/20 12:50:39 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6328lb # [ 205.363454] niks3-server[1032]: 2026/09/20 12:50:39 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6329lb # [ 205.370647] niks3-server[1032]: 2026/09/20 12:50:39 INFO Completed upload id=666330worker2 # [ 205.422514] nix-grpc-daemon[1649]: time=2026-09-20T12:50:39.916Z level=INFO msg="Upload complete. (474ms)"6331lb # [ 205.374030] niks3-server[1032]: 2026/09/20 12:50:39 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6332worker2 # [ 205.432260] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59052 duration_s=0 paths=1 nar_bytes_in=6326333worker2 # [ 205.443035] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590526334worker2 # [ 205.452328] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6335worker2 # [ 205.455339] nix-daemon[861]: reaped child process 1694, status = succeeded6336client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6337worker2 # [ 205.467132] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6338client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6339client # farm-b> LOG-farm-b-drain6340worker2 # [ 207.117823] nix-daemon[861]: reaped child process 1708, status = succeeded6341worker2 # [ 207.290690] nix-daemon[861]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6342worker2 # [ 207.306337] nix-daemon[861]: reaped child process 1726, status = succeeded6343lb # [ 207.270957] niks3-server[1032]: 2026/09/20 12:50:41 INFO Received uploads request method=POST path=/api/pending_closures6344worker2 # [ 207.332853] nix-grpc-daemon[1649]: time=2026-09-20T12:50:41.826Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6345worker2 # [ 207.334452] nix-grpc-daemon[1649]: time=2026-09-20T12:50:41.826Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6346lb # [ 207.303993] niks3-server[1032]: 2026/09/20 12:50:41 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6347lb # [ 207.319883] niks3-server[1032]: 2026/09/20 12:50:41 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6348lb # [ 207.327155] niks3-server[1032]: 2026/09/20 12:50:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6349worker2 # [ 207.384525] nix-grpc-daemon[1649]: time=2026-09-20T12:50:41.878Z level=INFO msg="Uploading 1 narinfos"6350lb # [ 207.337964] niks3-server[1032]: 2026/09/20 12:50:41 INFO Signed narinfos id=67 count=16351lb # [ 207.343602] niks3-server[1032]: 2026/09/20 12:50:41 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6352lb # [ 207.352328] niks3-server[1032]: 2026/09/20 12:50:41 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6353lb # [ 207.361782] niks3-server[1032]: 2026/09/20 12:50:41 INFO Completed upload id=676354worker2 # [ 207.412662] nix-grpc-daemon[1649]: time=2026-09-20T12:50:41.906Z level=INFO msg="Upload complete. (331ms)"6355lb # [ 207.364930] niks3-server[1032]: 2026/09/20 12:50:41 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6356worker2 # [ 207.425257] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59052 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=16357worker1 # [ 207.351217] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:41Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06358worker1 # [ 207.357384] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:41Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16359worker2 # [ 207.444233] nix-daemon[861]: reaped child process 1706, status = succeeded6360worker2 # [ 207.476175] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590526361worker2 # [ 207.493083] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590526362worker2 # [ 207.506676] nix-grpc-daemon[1649]: time=2026-09-20T12:50:42.000Z level=INFO msg="All 2 paths already cached"6363client # copying 1 paths to 'lb:50051'...6364client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6365worker2 # [ 207.517059] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590526366worker2 # [ 207.532240] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6367worker2 # [ 207.734678] nix-daemon[861]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)6368worker2 # [ 207.753325] nix-daemon[861]: reaped child process 1733, status = succeeded6369worker2 # [ 207.923127] nix-daemon[861]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6370worker2 # [ 207.941183] nix-daemon[861]: reaped child process 1738, status = succeeded6371lb # [ 207.901860] niks3-server[1032]: 2026/09/20 12:50:42 INFO Received uploads request method=POST path=/api/pending_closures6372worker2 # [ 207.962870] nix-grpc-daemon[1649]: time=2026-09-20T12:50:42.456Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6373worker2 # [ 207.964435] nix-grpc-daemon[1649]: time=2026-09-20T12:50:42.456Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6374lb # [ 207.927115] niks3-server[1032]: 2026/09/20 12:50:42 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6375lb # [ 207.941382] niks3-server[1032]: 2026/09/20 12:50:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6376worker2 # [ 208.001130] nix-grpc-daemon[1649]: time=2026-09-20T12:50:42.495Z level=INFO msg="Uploading 1 narinfos"6377lb # [ 207.949463] niks3-server[1032]: 2026/09/20 12:50:42 INFO Signed narinfos id=68 count=16378lb # [ 207.958259] niks3-server[1032]: 2026/09/20 12:50:42 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6379lb # [ 207.966431] niks3-server[1032]: 2026/09/20 12:50:42 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6380worker2 # [ 208.022824] nix-grpc-daemon[1649]: time=2026-09-20T12:50:42.517Z level=INFO msg="Upload complete. (473ms)"6381lb # [ 207.972991] niks3-server[1032]: 2026/09/20 12:50:42 INFO Completed upload id=686382lb # [ 207.976265] niks3-server[1032]: 2026/09/20 12:50:42 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6383worker2 # [ 208.032678] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59052 duration_s=0 paths=1 nar_bytes_in=7606384worker2 # [ 208.041395] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590526385worker2 # [ 208.047937] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6386worker2 # [ 208.052777] nix-daemon[861]: reaped child process 1728, status = succeeded6387client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6388worker2 # [ 208.065029] nix-daemon[861]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6389client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6390client # farm-top> LOG-farm-top-drain6391worker2 # [ 208.896253] nix-daemon[861]: reaped child process 1742, status = succeeded6392worker2 # [ 209.058786] nix-daemon[861]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6393worker2 # [ 209.076621] nix-daemon[861]: reaped child process 1760, status = succeeded6394lb # [ 209.042024] niks3-server[1032]: 2026/09/20 12:50:43 INFO Received uploads request method=POST path=/api/pending_closures6395worker2 # [ 209.103316] nix-grpc-daemon[1649]: time=2026-09-20T12:50:43.595Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6396worker2 # [ 209.105038] nix-grpc-daemon[1649]: time=2026-09-20T12:50:43.595Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6397lb # [ 209.075990] niks3-server[1032]: 2026/09/20 12:50:43 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6398lb # [ 209.088467] niks3-server[1032]: 2026/09/20 12:50:43 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6399worker2 # [ 209.157536] nix-grpc-daemon[1649]: time=2026-09-20T12:50:43.651Z level=INFO msg="Uploading 1 narinfos"6400lb # [ 209.100270] niks3-server[1032]: 2026/09/20 12:50:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6401lb # [ 209.109899] niks3-server[1032]: 2026/09/20 12:50:43 INFO Signed narinfos id=69 count=16402lb # [ 209.115227] niks3-server[1032]: 2026/09/20 12:50:43 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6403lb # [ 209.125660] niks3-server[1032]: 2026/09/20 12:50:43 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6404lb # [ 209.130976] niks3-server[1032]: 2026/09/20 12:50:43 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6405lb # [ 209.135036] niks3-server[1032]: 2026/09/20 12:50:43 INFO Completed upload id=696406worker2 # [ 209.186649] nix-grpc-daemon[1649]: time=2026-09-20T12:50:43.680Z level=INFO msg="Upload complete. (312ms)"6407worker2 # [ 209.197225] nix-grpc-daemon[1593]: ts=2026-09-20T12:50:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59052 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=16408worker2 # [ 209.216255] nix-daemon[861]: reaped child process 1740, status = succeeded6409client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 7.55 seconds)6410client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6411client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6412worker1: must succeed: rm /nix/.rw-store/fill6413worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6414worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6415worker1 # [ 209.667751] nix-grpc-daemon[1611]: ts=2026-09-20T12:50:44Z level=info event=healthy reason=min_free6416worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 0.05 seconds)6417(finished: subtest: low disk drains a worker and builds go to the other, in 10.35 seconds)6418(finished: run the VM test script, in 210.78 seconds)6419test script finished in 211.07s6420cleanup6421kill QemuMachine (pid 49)6422client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6423kill QemuMachine (pid 50)6424lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6425kill QemuMachine (pid 51)6426worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6427kill QemuMachine (pid 52)6428worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6429(finished: cleanup, in 1.21 seconds)6430additionally exposed symbols:6431 client, lb, worker1, worker2,6432 vlan1,6433 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh