vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #517
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13worker2 # Disk image does not exist, creating the virtualisation disk image...14client: QEMU running (pid 51)15client # Formatting '/build/vm-state-client/tmp.Zy9QhGmZxx', fmt=raw size=107374182416worker2: QEMU running (pid 52)17lb: QEMU running (pid 49)18client # mke2fs 1.47.4 (6-Mar-2025)19worker2 # Formatting '/build/vm-state-worker2/tmp.pXNuHuyN3Q', fmt=raw size=107374182420client # Discarding device blocks: 0/262144 done21worker2 # mke2fs 1.47.4 (6-Mar-2025)22client # Creating filesystem with 262144 4k blocks and 65536 inodes23worker2 # Discarding device blocks: 0/262144 done24client # Filesystem UUID: a62f6ea9-c1fc-4c8f-81b0-c7a81bf4e5dd25worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes26client # Superblock backups stored on blocks:27worker2 # Filesystem UUID: 66c3ca7b-51d7-4602-8e39-346aa8580ad628client # 32768, 98304, 163840, 22937629worker2 # Superblock backups stored on blocks:30client # 31worker2 # 32768, 98304, 163840, 22937632client # Allocating group tables: 0/8 done33worker2 # 34client # Writing inode tables: 0/8 done35worker2 # Allocating group tables: 0/8 done36lb # Disk image does not exist, creating the virtualisation disk image...37worker1: QEMU running (pid 50)38worker1 # Disk image does not exist, creating the virtualisation disk image...39worker1 # Formatting '/build/vm-state-worker1/tmp.PkMhTaEi8c', fmt=raw size=107374182440worker2 # Writing inode tables: 0/8 done41client # Creating journal (8192 blocks): done42worker2 # Creating journal (8192 blocks): done43(finished: start all VMs, in 0.55 seconds)44lb # Formatting '/build/vm-state-lb/tmp.1HpU8q3NUG', fmt=raw size=107374182445worker2 # Writing superblocks and filesystem accounting information: 0/8 done46client: waiting for unit network-addresses-eth1.service47lb # mke2fs 1.47.4 (6-Mar-2025)48worker2 # 49client: waiting for the VM to finish booting50lb # Discarding device blocks: 0/262144 done51worker2 # Virtualisation disk image created.52worker1 # mke2fs 1.47.4 (6-Mar-2025)53client # Writing superblocks and filesystem accounting information: 0/8 done54lb # Creating filesystem with 262144 4k blocks and 65536 inodes55worker2 # Starting virtiofs daemons...56worker1 # Discarding device blocks: 0/262144 done57client # 58lb # Filesystem UUID: 1cd19ffa-09ce-40ed-92b0-dba9e25172a259worker2 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)60worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes61client # Virtualisation disk image created.62lb # Superblock backups stored on blocks:63worker2 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether64worker1 # Filesystem UUID: 6e4ea723-c219-4b7b-8934-6eb51079b2c265client # Starting virtiofs daemons...66lb # 32768, 98304, 163840, 22937667worker2 # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...68worker1 # Superblock backups stored on blocks:69client # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)70lb # 71worker2 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)72worker1 # 32768, 98304, 163840, 22937673client # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether74lb # Allocating group tables: 0/8 done75lb # Writing inode tables: 0/8 done76worker1 # 77worker2 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78client # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...79lb # Creating journal (8192 blocks): done80worker1 # Allocating group tables: 0/8 done81worker2 # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...82client # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83lb # Writing superblocks and filesystem accounting information: 0/8 done84worker1 # Writing inode tables: 0/8 done85worker2 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86client # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87lb # 88worker1 # Creating journal (8192 blocks): done89worker2 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90client # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91lb # Virtualisation disk image created.92worker1 # Writing superblocks and filesystem accounting information: 0/8 done93worker2 # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...94client # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95lb # Starting virtiofs daemons...96worker1 # 97worker2 # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests98client # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...99lb # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100worker1 # Virtualisation disk image created.101worker2 # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests102client # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...103lb # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104worker1 # Starting virtiofs daemons...105worker2 # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests106client # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests107lb # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...108client # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests109lb # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)110client # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests111lb # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether112worker1 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)113lb # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...114worker1 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether115lb # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)116worker1 # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...117lb # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether118worker1 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)119lb # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...120worker1 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests122worker1 # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...123lb # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests124worker1 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)125lb # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests126worker1 # [2026-09-23T05:11:53Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker1 # [2026-09-23T05:11:53Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-23T05:11:53Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133client # [ 0.000000] KASLR enabled134client # [ 0.000000] random: crng init done135client # [ 0.000000] Machine model: linux,dummy-virt136client # [ 0.000000] efi: UEFI not found.137client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140client # [ 0.000000] Zone ranges:141client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] DMA32 empty143client # [ 0.000000] Normal empty144client # [ 0.000000] Device empty145client # [ 0.000000] Movable zone start for each node146client # [ 0.000000] Early memory node ranges147client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150client # [ 0.000000] psci: probing for conduit method from DT.151client # [ 0.000000] psci: PSCIv1.3 detected in firmware.152client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153client # [ 0.000000] psci: Trusted OS migration not required154client # [ 0.000000] psci: SMC Calling Convention v1.1155client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157client # [ 0.000000] Detected PIPT I-cache on CPU0158client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159client # [ 0.000000] CPU features: detected: GICv3 CPU interface160client # [ 0.000000] CPU features: detected: Spectre-v4161client # [ 0.000000] CPU features: detected: Spectre-BHB162client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164client # [ 0.000000] alternatives: applying boot alternatives165client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5fx3y1hbxhkdng69ksrxcn3js8649kk7-nixos-system-client-test/init regInfo=/nix/store/nsqdzrzkl3zyhgxccqczms6gsa6s08c3-closure-info/registration console=ttyAMA0,115200n8 console=tty0166client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/nsqdzrzkl3zyhgxccqczms6gsa6s08c3-closure-info/registration", will be passed to user space.167client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes168client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)169client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)170client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB171client # [ 0.000000] software IO TLB: area num 1.172client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB173client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)174client # [ 0.000000] Fallback order for Node 0: 0175client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216176client # [ 0.000000] Policy zone: DMA177client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off178client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1179client # [ 0.000000] allocated 3145728 bytes of page_ext180client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages181client # [ 0.000000] ftrace: allocated 294 pages with 4 groups182client # [ 0.000000] rcu: Hierarchical RCU implementation.183client # [ 0.000000] rcu: RCU event tracing is enabled.184lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]185client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.186client # [ 0.000000] Trampoline variant of Tasks RCU enabled.187lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026188client # [ 0.000000] Rude variant of Tasks RCU enabled.189lb # [ 0.000000] KASLR enabled190lb # [ 0.000000] random: crng init done191client # [ 0.000000] Tracing variant of Tasks RCU enabled.192lb # [ 0.000000] Machine model: linux,dummy-virt193lb # [ 0.000000] efi: UEFI not found.194client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.195lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT196client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1197lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]198client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.199lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]200lb # [ 0.000000] Zone ranges:201client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.202lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]203lb # [ 0.000000] DMA32 empty204client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.205lb # [ 0.000000] Normal empty206lb # [ 0.000000] Device empty207client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0208lb # [ 0.000000] Movable zone start for each node209client # [ 0.000000] GICv3: 256 SPIs implemented210lb # [ 0.000000] Early memory node ranges211client # [ 0.000000] GICv3: 0 Extended SPIs implemented212lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]213client # [ 0.000000] Root IRQ handler: gic_handle_irq214client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI215lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]216client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0217worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]218lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000219client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000220lb # [ 0.000000] psci: probing for conduit method from DT.221client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]222lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.223worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026224worker2 # [ 0.000000] KASLR enabled225lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs226client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)227worker2 # [ 0.000000] random: crng init done228lb # [ 0.000000] psci: Trusted OS migration not required229worker2 # [ 0.000000] Machine model: linux,dummy-virt230lb # [ 0.000000] psci: SMC Calling Convention v1.1231worker2 # [ 0.000000] efi: UEFI not found.232client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)233client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000234worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT235lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)236client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000237worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]238lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296239lb # [ 0.000000] Detected PIPT I-cache on CPU0240worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]241client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.242worker2 # [ 0.000000] Zone ranges:243lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)244worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]245client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns246worker2 # [ 0.000000] DMA32 empty247lb # [ 0.000000] CPU features: detected: GICv3 CPU interface248worker2 # [ 0.000000] Normal empty249client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).250lb # [ 0.000000] CPU features: detected: Spectre-v4251worker2 # [ 0.000000] Device empty252lb # [ 0.000000] CPU features: detected: Spectre-BHB253worker2 # [ 0.000000] Movable zone start for each node254worker2 # [ 0.000000] Early memory node ranges255client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns256lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38257worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]258lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23259client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns260worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]261lb # [ 0.000000] alternatives: applying boot alternatives262client # [ 0.000037] arm-pv: using stolen time PV263worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]264worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000265worker2 # [ 0.000000] psci: probing for conduit method from DT.266client # [ 0.000847] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)267worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.268client # [ 0.001015] Console: colour dummy device 80x25269worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026270client # [ 0.001023] printk: legacy console [tty0] enabled271worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs272worker1 # [ 0.000000] KASLR enabled273worker1 # [ 0.000000] random: crng init done274worker2 # [ 0.000000] psci: Trusted OS migration not required275worker1 # [ 0.000000] Machine model: linux,dummy-virt276worker2 # [ 0.000000] psci: SMC Calling Convention v1.1277client # [ 0.001216] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)278worker1 # [ 0.000000] efi: UEFI not found.279lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test/init regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration console=ttyAMA0,115200n8 console=tty0280client # [ 0.001222] pid_max: default: 32768 minimum: 301281worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)282worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT283client # [ 0.001300] LSM: initializing lsm=capability,landlock,yama,bpf,ima284worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296285worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]286client # [ 0.001485] landlock: Up and running.287lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration", will be passed to user space.288worker2 # [ 0.000000] Detected PIPT I-cache on CPU0289client # [ 0.001488] Yama: becoming mindful.290worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]291client # [ 0.002204] LSM support for eBPF active292lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes293worker1 # [ 0.000000] Zone ranges:294worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)295worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]296client # [ 0.002388] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)297worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface298lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)299worker1 # [ 0.000000] DMA32 empty300worker2 # [ 0.000000] CPU features: detected: Spectre-v4301worker1 # [ 0.000000] Normal empty302client # [ 0.002445] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)303lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)304worker1 # [ 0.000000] Device empty305worker2 # [ 0.000000] CPU features: detected: Spectre-BHB306client # [ 0.004501] rcu: Hierarchical SRCU implementation.307worker1 # [ 0.000000] Movable zone start for each node308lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB309worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38310worker1 # [ 0.000000] Early memory node ranges311client # [ 0.004508] rcu: Max phase no-delay instances is 1000.312lb # [ 0.000000] software IO TLB: area num 1.313client # [ 0.005770] fsl-mc MSI: its@8080000 domain created314worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23315worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]316lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB317client # [ 0.005868] EFI services will not be available.318worker2 # [ 0.000000] alternatives: applying boot alternatives319worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]320client # [ 0.005937] smp: Bringing up secondary CPUs ...321lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)322client # [ 0.005946] smp: Brought up 1 node, 1 CPU323worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000324lb # [ 0.000000] Fallback order for Node 0: 0325worker1 # [ 0.000000] psci: probing for conduit method from DT.326client # [ 0.005949] SMP: Total of 1 processors activated.327client # [ 0.005955] CPU: All CPU(s) started at EL1328lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216329worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.330lb # [ 0.000000] Policy zone: DMA331worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs332client # [ 0.005965] CPU features: detected: Branch Target Identification333lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off334worker1 # [ 0.000000] psci: Trusted OS migration not required335client # [ 0.005970] CPU features: detected: ARMv8.4 Translation Table Level336worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8rxzyh0zqd97xa57p8wqswwb0drnznwf-nixos-system-worker2-test/init regInfo=/nix/store/lrx87j4jd62l04f6j8b9gzxii6h2jx5g-closure-info/registration console=ttyAMA0,115200n8 console=tty0337worker1 # [ 0.000000] psci: SMC Calling Convention v1.1338lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1339client # [ 0.005973] CPU features: detected: Instruction cache invalidation not required for I/D coherence340lb # [ 0.000000] allocated 3145728 bytes of page_ext341worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)342lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages343worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lrx87j4jd62l04f6j8b9gzxii6h2jx5g-closure-info/registration", will be passed to user space.344client # [ 0.005977] CPU features: detected: Data cache clean to the PoU not required for I/D coherence345worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296346lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups347worker1 # [ 0.000000] Detected PIPT I-cache on CPU0348client # [ 0.005981] CPU features: detected: Common not Private translations349worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes350lb # [ 0.000000] rcu: Hierarchical RCU implementation.351client # [ 0.005984] CPU features: detected: CRC32 instructions352lb # [ 0.000000] rcu: RCU event tracing is enabled.353worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)354worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)355client # [ 0.005987] CPU features: detected: Data cache clean to Point of Deep Persistence356worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface357lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.358worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)359worker1 # [ 0.000000] CPU features: detected: Spectre-v4360lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.361client # [ 0.005990] CPU features: detected: Data cache clean to Point of Persistence362worker1 # [ 0.000000] CPU features: detected: Spectre-BHB363worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB364lb # [ 0.000000] Rude variant of Tasks RCU enabled.365client # [ 0.005993] CPU features: detected: Data independent timing control (DIT)366worker2 # [ 0.000000] software IO TLB: area num 1.367lb # [ 0.000000] Tracing variant of Tasks RCU enabled.368worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38369client # [ 0.005997] CPU features: detected: E0PD370worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB371worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23372lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.373client # [ 0.005999] CPU features: detected: Enhanced Counter Virtualization374worker1 # [ 0.000000] alternatives: applying boot alternatives375worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)376lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1377client # [ 0.006002] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)378worker2 # [ 0.000000] Fallback order for Node 0: 0379client # [ 0.006006] CPU features: detected: Enhanced Virtualization Traps380worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216381lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.382worker2 # [ 0.000000] Policy zone: DMA383client # [ 0.006009] CPU features: detected: Fine Grained Traps384worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off385lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.386client # [ 0.006012] CPU features: detected: Generic authentication (architected QARMA5 algorithm)387worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1388client # [ 0.006017] CPU features: detected: RCpc load-acquire (LDAPR)389worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3j2yqs8i8ynzphhqwmkrm3aah3k3l7j5-nixos-system-worker1-test/init regInfo=/nix/store/rjgpcx9an49sbd2sna23nqm8paywvxlm-closure-info/registration console=ttyAMA0,115200n8 console=tty0390client # [ 0.006020] CPU features: detected: LSE atomic instructions391worker2 # [ 0.000000] allocated 3145728 bytes of page_ext392lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.393worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages394lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0395client # [ 0.006023] CPU features: detected: Privileged Access Never396lb # [ 0.000000] GICv3: 256 SPIs implemented397worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups398client # [ 0.006026] CPU features: detected: PMUv3399worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/rjgpcx9an49sbd2sna23nqm8paywvxlm-closure-info/registration", will be passed to user space.400lb # [ 0.000000] GICv3: 0 Extended SPIs implemented401worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.402client # [ 0.006029] CPU features: detected: RAS Extension Support403lb # [ 0.000000] Root IRQ handler: gic_handle_irq404worker2 # [ 0.000000] rcu: RCU event tracing is enabled.405worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes406client # [ 0.006032] CPU features: detected: RASv1p1 Extension Support407lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI408worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.409lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0410client # [ 0.006034] CPU features: detected: Random Number Generator411worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)412worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.413client # [ 0.006037] CPU features: detected: Speculation barrier (SB)414lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000415worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)416worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.417lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]418client # [ 0.006039] CPU features: detected: Stage-2 Force Write-Back419worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.420worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB421client # [ 0.006042] CPU features: detected: TLB range maintenance instructions422worker1 # [ 0.000000] software IO TLB: area num 1.423worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.424lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)425worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB426client # [ 0.006047] CPU features: detected: Speculative Store Bypassing Safe (SSBS)427worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1428client # [ 0.006089] alternatives: applying system-wide alternatives429lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)430worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)431worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.432worker1 # [ 0.000000] Fallback order for Node 0: 0433lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000434client # [ 0.009142] CPU features: detected: BBM Level 2 without TLB conflict abort435worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216436lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000437worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.438worker1 # [ 0.000000] Policy zone: DMA439lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.440client # [ 0.009323] Memory: 1405648K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)441worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off442worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.443client # [ 0.009794] devtmpfs: initialized444worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0445worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1446lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns447worker2 # [ 0.000000] GICv3: 256 SPIs implemented448worker1 # [ 0.000000] allocated 3145728 bytes of page_ext449client # [ 0.011693] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)450lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).451worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented452worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages453client # [ 0.011717] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).454worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq455worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups456worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI457client # [ 0.011930] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL458lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns459worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.460worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0461client # [ 0.011934] 0 pages in range for non-PLT usage462worker1 # [ 0.000000] rcu: RCU event tracing is enabled.463lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns464client # [ 0.011935] 508272 pages in range for PLT usage465worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000466worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.467lb # [ 0.000041] arm-pv: using stolen time PV468client # [ 0.012067] pinctrl core: initialized pinctrl subsystem469worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]470worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.471client # [ 0.012846] DMI not present or invalid.472worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.473lb # [ 0.000675] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)474worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)475client # [ 0.017005] NET: Registered PF_NETLINK/PF_ROUTE protocol family476worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.477lb # [ 0.000851] Console: colour dummy device 80x25478client # [ 0.019334] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations479lb # [ 0.000860] printk: legacy console [tty0] enabled480worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)481worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.482client # [ 0.019538] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations483worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000484worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1485lb # [ 0.001068] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)486client # [ 0.019747] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations487worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000488lb # [ 0.001075] pid_max: default: 32768 minimum: 301489worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.490client # [ 0.019777] audit: initializing netlink subsys (disabled)491lb # [ 0.001176] LSM: initializing lsm=capability,landlock,yama,bpf,ima492worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.493lb # [ 0.001344] landlock: Up and running.494client # [ 0.020409] thermal_sys: Registered thermal governor 'fair_share'495worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.496lb # [ 0.001347] Yama: becoming mindful.497worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns498client # [ 0.020412] thermal_sys: Registered thermal governor 'bang_bang'499lb # [ 0.001965] LSM support for eBPF active500worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.501worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).502client # [ 0.020415] thermal_sys: Registered thermal governor 'step_wise'503lb # [ 0.002145] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)504worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0505client # [ 0.020418] thermal_sys: Registered thermal governor 'user_space'506worker1 # [ 0.000000] GICv3: 256 SPIs implemented507lb # [ 0.002178] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)508worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns509worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented510client # [ 0.020424] thermal_sys: Registered thermal governor 'power_allocator'511lb # [ 0.004192] rcu: Hierarchical SRCU implementation.512worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq513worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns514client # [ 0.020450] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1515worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI516lb # [ 0.004199] rcu: Max phase no-delay instances is 1000.517worker2 # [ 0.000032] arm-pv: using stolen time PV518client # [ 0.020461] cpuidle: using governor ladder519lb # [ 0.005505] fsl-mc MSI: its@8080000 domain created520worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0521client # [ 0.020466] cpuidle: using governor menu522lb # [ 0.005606] EFI services will not be available.523worker2 # [ 0.000503] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)524worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000525client # [ 0.020685] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.526lb # [ 0.005720] smp: Bringing up secondary CPUs ...527worker2 # [ 0.000690] Console: colour dummy device 80x25528worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]529lb # [ 0.005729] smp: Brought up 1 node, 1 CPU530client # [ 0.020703] ASID allocator initialised with 65536 entries531worker2 # [ 0.000698] printk: legacy console [tty0] enabled532lb # [ 0.005732] SMP: Total of 1 processors activated.533client # [ 0.021948] Serial: AMBA PL011 UART driver534worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)535lb # [ 0.005739] CPU: All CPU(s) started at EL1536worker2 # [ 0.000906] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)537client # [ 0.027465] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1538lb # [ 0.005749] CPU features: detected: Branch Target Identification539worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)540worker2 # [ 0.000915] pid_max: default: 32768 minimum: 301541client # [ 0.027603] printk: console [ttyAMA0] enabled542lb # [ 0.005754] CPU features: detected: ARMv8.4 Translation Table Level543worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000544worker2 # [ 0.001003] LSM: initializing lsm=capability,landlock,yama,bpf,ima545worker2 # [ 0.001196] landlock: Up and running.546worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000547worker2 # [ 0.001200] Yama: becoming mindful.548lb # [ 0.005757] CPU features: detected: Instruction cache invalidation not required for I/D coherence549worker2 # [ 0.001690] LSM support for eBPF active550worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.551lb # [ 0.005761] CPU features: detected: Data cache clean to the PoU not required for I/D coherence552worker2 # [ 0.001852] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)553lb # [ 0.005765] CPU features: detected: Common not Private translations554worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns555worker2 # [ 0.001888] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)556lb # [ 0.005768] CPU features: detected: CRC32 instructions557worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).558worker2 # [ 0.003919] rcu: Hierarchical SRCU implementation.559lb # [ 0.005771] CPU features: detected: Data cache clean to Point of Deep Persistence560worker2 # [ 0.003925] rcu: Max phase no-delay instances is 1000.561worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns562worker2 # [ 0.005255] fsl-mc MSI: its@8080000 domain created563lb # [ 0.005774] CPU features: detected: Data cache clean to Point of Persistence564worker2 # [ 0.005350] EFI services will not be available.565lb # [ 0.005777] CPU features: detected: Data independent timing control (DIT)566worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns567worker2 # [ 0.005422] smp: Bringing up secondary CPUs ...568lb # [ 0.005780] CPU features: detected: E0PD569worker1 # [ 0.000036] arm-pv: using stolen time PV570worker2 # [ 0.005431] smp: Brought up 1 node, 1 CPU571lb # [ 0.005783] CPU features: detected: Enhanced Counter Virtualization572worker2 # [ 0.005434] SMP: Total of 1 processors activated.573worker1 # [ 0.000423] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)574worker2 # [ 0.005439] CPU: All CPU(s) started at EL1575lb # [ 0.005786] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)576worker1 # [ 0.000625] Console: colour dummy device 80x25577worker2 # [ 0.005450] CPU features: detected: Branch Target Identification578lb # [ 0.005789] CPU features: detected: Enhanced Virtualization Traps579worker1 # [ 0.000633] printk: legacy console [tty0] enabled580client # [ 0.149809] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages581worker2 # [ 0.005455] CPU features: detected: ARMv8.4 Translation Table Level582lb # [ 0.005792] CPU features: detected: Fine Grained Traps583client # [ 0.149830] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page584worker1 # [ 0.000825] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)585worker2 # [ 0.005458] CPU features: detected: Instruction cache invalidation not required for I/D coherence586lb # [ 0.005795] CPU features: detected: Generic authentication (architected QARMA5 algorithm)587client # [ 0.149836] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages588worker1 # [ 0.000832] pid_max: default: 32768 minimum: 301589lb # [ 0.005800] CPU features: detected: RCpc load-acquire (LDAPR)590worker2 # [ 0.005462] CPU features: detected: Data cache clean to the PoU not required for I/D coherence591client # [ 0.149840] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page592worker1 # [ 0.000915] LSM: initializing lsm=capability,landlock,yama,bpf,ima593lb # [ 0.005803] CPU features: detected: LSE atomic instructions594worker1 # [ 0.001053] landlock: Up and running.595worker2 # [ 0.005466] CPU features: detected: Common not Private translations596client # [ 0.149844] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages597worker1 # [ 0.001056] Yama: becoming mindful.598lb # [ 0.005806] CPU features: detected: Privileged Access Never599worker2 # [ 0.005469] CPU features: detected: CRC32 instructions600worker1 # [ 0.001529] LSM support for eBPF active601lb # [ 0.005830] CPU features: detected: PMUv3602client # [ 0.149848] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page603worker2 # [ 0.005472] CPU features: detected: Data cache clean to Point of Deep Persistence604lb # [ 0.005833] CPU features: detected: RAS Extension Support605worker1 # [ 0.001675] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)606client # [ 0.149853] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages607lb # [ 0.005836] CPU features: detected: RASv1p1 Extension Support608worker2 # [ 0.005475] CPU features: detected: Data cache clean to Point of Persistence609worker1 # [ 0.001709] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)610client # [ 0.149857] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page611lb # [ 0.005838] CPU features: detected: Random Number Generator612worker1 # [ 0.003640] rcu: Hierarchical SRCU implementation.613worker2 # [ 0.005479] CPU features: detected: Data independent timing control (DIT)614worker2 # [ 0.005482] CPU features: detected: E0PD615lb # [ 0.005841] CPU features: detected: Speculation barrier (SB)616worker1 # [ 0.003646] rcu: Max phase no-delay instances is 1000.617worker1 # [ 0.005006] fsl-mc MSI: its@8080000 domain created618lb # [ 0.005844] CPU features: detected: Stage-2 Force Write-Back619worker2 # [ 0.005484] CPU features: detected: Enhanced Counter Virtualization620client # [ 0.157255] fbcon: Taking over console621worker1 # [ 0.005097] EFI services will not be available.622lb # [ 0.005847] CPU features: detected: TLB range maintenance instructions623worker2 # [ 0.005487] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)624client # [ 0.157272] ACPI: Interpreter disabled.625worker1 # [ 0.005182] smp: Bringing up secondary CPUs ...626worker1 # [ 0.005191] smp: Brought up 1 node, 1 CPU627worker2 # [ 0.005491] CPU features: detected: Enhanced Virtualization Traps628lb # [ 0.005852] CPU features: detected: Speculative Store Bypassing Safe (SSBS)629client # [ 0.159175] iommu: Default domain type: Translated630worker1 # [ 0.005194] SMP: Total of 1 processors activated.631worker2 # [ 0.005494] CPU features: detected: Fine Grained Traps632lb # [ 0.005890] alternatives: applying system-wide alternatives633worker1 # [ 0.005200] CPU: All CPU(s) started at EL1634client # [ 0.159186] iommu: DMA domain TLB invalidation policy: strict mode635lb # [ 0.008913] CPU features: detected: BBM Level 2 without TLB conflict abort636worker2 # [ 0.005497] CPU features: detected: Generic authentication (architected QARMA5 algorithm)637worker1 # [ 0.005210] CPU features: detected: Branch Target Identification638client # [ 0.160873] SCSI subsystem initialized639worker2 # [ 0.005502] CPU features: detected: RCpc load-acquire (LDAPR)640worker1 # [ 0.005214] CPU features: detected: ARMv8.4 Translation Table Level641worker2 # [ 0.005505] CPU features: detected: LSE atomic instructions642lb # [ 0.009114] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)643worker2 # [ 0.005508] CPU features: detected: Privileged Access Never644worker1 # [ 0.005217] CPU features: detected: Instruction cache invalidation not required for I/D coherence645lb # [ 0.009589] devtmpfs: initialized646worker2 # [ 0.005510] CPU features: detected: PMUv3647lb # [ 0.011538] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)648worker1 # [ 0.005221] CPU features: detected: Data cache clean to the PoU not required for I/D coherence649worker2 # [ 0.005513] CPU features: detected: RAS Extension Support650worker2 # [ 0.005516] CPU features: detected: RASv1p1 Extension Support651worker1 # [ 0.005225] CPU features: detected: Common not Private translations652worker1 # [ 0.005229] CPU features: detected: CRC32 instructions653worker2 # [ 0.005519] CPU features: detected: Random Number Generator654lb # [ 0.011563] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).655worker1 # [ 0.005232] CPU features: detected: Data cache clean to Point of Deep Persistence656lb # [ 0.011767] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL657worker2 # [ 0.005521] CPU features: detected: Speculation barrier (SB)658worker1 # [ 0.005235] CPU features: detected: Data cache clean to Point of Persistence659lb # [ 0.011771] 0 pages in range for non-PLT usage660client # [ 0.166163] usbcore: registered new interface driver usbfs661worker2 # [ 0.005524] CPU features: detected: Stage-2 Force Write-Back662lb # [ 0.011772] 508272 pages in range for PLT usage663client # [ 0.166200] usbcore: registered new interface driver hub664worker1 # [ 0.005238] CPU features: detected: Data independent timing control (DIT)665worker2 # [ 0.005527] CPU features: detected: TLB range maintenance instructions666worker1 # [ 0.005241] CPU features: detected: E0PD667client # [ 0.166218] usbcore: registered new device driver usb668lb # [ 0.011899] pinctrl core: initialized pinctrl subsystem669lb # [ 0.012764] DMI not present or invalid.670client # [ 0.166528] pps_core: LinuxPPS API ver. 1 registered671worker2 # [ 0.005531] CPU features: detected: Speculative Store Bypassing Safe (SSBS)672worker1 # [ 0.005244] CPU features: detected: Enhanced Counter Virtualization673lb # [ 0.016177] NET: Registered PF_NETLINK/PF_ROUTE protocol family674worker2 # [ 0.005568] alternatives: applying system-wide alternatives675worker1 # [ 0.005247] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)676client # [ 0.166534] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>677lb # [ 0.018622] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations678worker1 # [ 0.005250] CPU features: detected: Enhanced Virtualization Traps679client # [ 0.166545] PTP clock support registered680client # [ 0.166594] EDAC MC: Ver: 3.0.0681worker2 # [ 0.008750] CPU features: detected: BBM Level 2 without TLB conflict abort682worker1 # [ 0.005254] CPU features: detected: Fine Grained Traps683lb # [ 0.018828] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations684client # [ 0.171127] scmi_core: SCMI protocol bus registered685worker1 # [ 0.005257] CPU features: detected: Generic authentication (architected QARMA5 algorithm)686lb # [ 0.019031] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations687worker2 # [ 0.008936] Memory: 1405648K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)688worker2 # [ 0.009447] devtmpfs: initialized689worker1 # [ 0.005261] CPU features: detected: RCpc load-acquire (LDAPR)690lb # [ 0.019056] audit: initializing netlink subsys (disabled)691client # [ 0.172126] FPGA manager framework692client # [ 0.173157] vgaarb: loaded693worker2 # [ 0.011431] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)694worker1 # [ 0.005264] CPU features: detected: LSE atomic instructions695lb # [ 0.019716] thermal_sys: Registered thermal governor 'fair_share'696worker1 # [ 0.005267] CPU features: detected: Privileged Access Never697lb # [ 0.019719] thermal_sys: Registered thermal governor 'bang_bang'698worker2 # [ 0.011453] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).699client # [ 0.173881] clocksource: Switched to clocksource arch_sys_counter700worker1 # [ 0.005269] CPU features: detected: PMUv3701lb # [ 0.019722] thermal_sys: Registered thermal governor 'step_wise'702worker2 # [ 0.011650] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL703worker1 # [ 0.005272] CPU features: detected: RAS Extension Support704lb # [ 0.019725] thermal_sys: Registered thermal governor 'user_space'705worker2 # [ 0.011654] 0 pages in range for non-PLT usage706worker1 # [ 0.005274] CPU features: detected: RASv1p1 Extension Support707worker2 # [ 0.011656] 508272 pages in range for PLT usage708lb # [ 0.019731] thermal_sys: Registered thermal governor 'power_allocator'709worker1 # [ 0.005277] CPU features: detected: Random Number Generator710worker2 # [ 0.011778] pinctrl core: initialized pinctrl subsystem711worker1 # [ 0.005279] CPU features: detected: Speculation barrier (SB)712worker2 # [ 0.012557] DMI not present or invalid.713lb # [ 0.019764] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1714worker1 # [ 0.005282] CPU features: detected: Stage-2 Force Write-Back715lb # [ 0.019773] cpuidle: using governor ladder716worker2 # [ 0.015795] NET: Registered PF_NETLINK/PF_ROUTE protocol family717lb # [ 0.019778] cpuidle: using governor menu718worker1 # [ 0.005284] CPU features: detected: TLB range maintenance instructions719worker2 # [ 0.018415] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations720lb # [ 0.019971] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.721worker1 # [ 0.005289] CPU features: detected: Speculative Store Bypassing Safe (SSBS)722worker2 # [ 0.018621] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations723lb # [ 0.019991] ASID allocator initialised with 65536 entries724worker1 # [ 0.005326] alternatives: applying system-wide alternatives725lb # [ 0.021275] Serial: AMBA PL011 UART driver726client # [ 0.180284] VFS: Disk quotas dquot_6.6.0727worker2 # [ 0.018888] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations728worker1 # [ 0.008403] CPU features: detected: BBM Level 2 without TLB conflict abort729worker2 # [ 0.018916] audit: initializing netlink subsys (disabled)730lb # [ 0.026743] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1731client # [ 0.180338] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)732lb # [ 0.026884] printk: console [ttyAMA0] enabled733worker2 # [ 0.019553] thermal_sys: Registered thermal governor 'fair_share'734client # [ 0.181977] netfs: FS-Cache loaded735worker1 # [ 0.008659] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)736worker2 # [ 0.019556] thermal_sys: Registered thermal governor 'bang_bang'737client # [ 0.182112] pnp: PnP ACPI: disabled738worker1 # [ 0.009135] devtmpfs: initialized739worker2 # [ 0.019560] thermal_sys: Registered thermal governor 'step_wise'740worker1 # [ 0.011138] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)741worker2 # [ 0.019563] thermal_sys: Registered thermal governor 'user_space'742worker1 # [ 0.011163] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).743worker2 # [ 0.019568] thermal_sys: Registered thermal governor 'power_allocator'744worker1 # [ 0.011346] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL745worker2 # [ 0.019597] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1746worker1 # [ 0.011351] 0 pages in range for non-PLT usage747worker2 # [ 0.019606] cpuidle: using governor ladder748worker1 # [ 0.011352] 508272 pages in range for PLT usage749worker2 # [ 0.019612] cpuidle: using governor menu750worker1 # [ 0.011462] pinctrl core: initialized pinctrl subsystem751worker1 # [ 0.012352] DMI not present or invalid.752worker2 # [ 0.019809] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.753worker2 # [ 0.019826] ASID allocator initialised with 65536 entries754worker1 # [ 0.015667] NET: Registered PF_NETLINK/PF_ROUTE protocol family755worker2 # [ 0.021068] Serial: AMBA PL011 UART driver756worker1 # [ 0.018220] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations757worker2 # [ 0.026416] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1758worker1 # [ 0.018430] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations759client # [ 0.186015] NET: Registered PF_INET protocol family760worker2 # [ 0.026558] printk: console [ttyAMA0] enabled761client # [ 0.186297] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)762worker1 # [ 0.018643] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations763lb # [ 0.149999] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages764worker1 # [ 0.018674] audit: initializing netlink subsys (disabled)765lb # [ 0.150020] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page766worker1 # [ 0.019305] thermal_sys: Registered thermal governor 'fair_share'767lb # [ 0.150026] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages768worker1 # [ 0.019307] thermal_sys: Registered thermal governor 'bang_bang'769lb # [ 0.150031] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page770worker1 # [ 0.019310] thermal_sys: Registered thermal governor 'step_wise'771lb # [ 0.150035] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages772worker1 # [ 0.019313] thermal_sys: Registered thermal governor 'user_space'773lb # [ 0.150039] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page774worker1 # [ 0.019318] thermal_sys: Registered thermal governor 'power_allocator'775lb # [ 0.150044] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages776worker1 # [ 0.019346] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1777lb # [ 0.150048] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page778worker1 # [ 0.019355] cpuidle: using governor ladder779worker2 # [ 0.144373] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages780worker1 # [ 0.019361] cpuidle: using governor menu781worker2 # [ 0.144394] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page782worker1 # [ 0.019565] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.783worker2 # [ 0.144400] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages784worker1 # [ 0.019585] ASID allocator initialised with 65536 entries785lb # [ 0.157735] fbcon: Taking over console786worker1 # [ 0.020982] Serial: AMBA PL011 UART driver787worker2 # [ 0.144404] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page788lb # [ 0.157751] ACPI: Interpreter disabled.789worker1 # [ 0.026357] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1790worker2 # [ 0.144409] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages791worker1 # [ 0.026507] printk: console [ttyAMA0] enabled792worker2 # [ 0.144413] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page793worker2 # [ 0.144417] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages794worker2 # [ 0.144421] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page795worker2 # [ 0.152118] fbcon: Taking over console796worker2 # [ 0.152134] ACPI: Interpreter disabled.797lb # [ 0.166387] iommu: Default domain type: Translated798lb # [ 0.166399] iommu: DMA domain TLB invalidation policy: strict mode799lb # [ 0.166763] SCSI subsystem initialized800lb # [ 0.168787] usbcore: registered new interface driver usbfs801worker1 # [ 0.146973] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages802lb # [ 0.168819] usbcore: registered new interface driver hub803worker1 # [ 0.146993] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page804lb # [ 0.168835] usbcore: registered new device driver usb805worker1 # [ 0.146999] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages806lb # [ 0.169127] pps_core: LinuxPPS API ver. 1 registered807worker1 # [ 0.147004] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page808lb # [ 0.169133] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>809worker1 # [ 0.147009] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages810lb # [ 0.169143] PTP clock support registered811worker2 # [ 0.161199] iommu: Default domain type: Translated812lb # [ 0.169193] EDAC MC: Ver: 3.0.0813worker1 # [ 0.147013] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page814worker2 # [ 0.161212] iommu: DMA domain TLB invalidation policy: strict mode815lb # [ 0.174011] scmi_core: SCMI protocol bus registered816worker1 # [ 0.147018] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages817worker2 # [ 0.161648] SCSI subsystem initialized818worker1 # [ 0.147022] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page819lb # [ 0.175002] FPGA manager framework820worker2 # [ 0.163660] usbcore: registered new interface driver usbfs821lb # [ 0.175970] vgaarb: loaded822worker2 # [ 0.163693] usbcore: registered new interface driver hub823lb # [ 0.176682] clocksource: Switched to clocksource arch_sys_counter824worker2 # [ 0.163710] usbcore: registered new device driver usb825worker1 # [ 0.154708] fbcon: Taking over console826worker2 # [ 0.164007] pps_core: LinuxPPS API ver. 1 registered827worker1 # [ 0.154729] ACPI: Interpreter disabled.828client # [ 0.217261] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)829client # [ 0.217331] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)830worker2 # [ 0.164013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>831worker2 # [ 0.164024] PTP clock support registered832client # [ 0.217367] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)833worker2 # [ 0.164074] EDAC MC: Ver: 3.0.0834client # [ 0.217454] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)835worker2 # [ 0.168816] scmi_core: SCMI protocol bus registered836client # [ 0.217596] TCP: Hash tables configured (established 16384 bind 16384)837lb # [ 0.181301] VFS: Disk quotas dquot_6.6.0838worker2 # [ 0.169835] FPGA manager framework839worker2 # [ 0.170821] vgaarb: loaded840client # [ 0.217690] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)841lb # [ 0.181354] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)842client # [ 0.217736] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)843lb # [ 0.183077] netfs: FS-Cache loaded844worker2 # [ 0.171479] clocksource: Switched to clocksource arch_sys_counter845lb # [ 0.183194] pnp: PnP ACPI: disabled846client # [ 0.217773] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)847client # [ 0.217884] NET: Registered PF_UNIX/PF_LOCAL protocol family848worker1 # [ 0.163329] iommu: Default domain type: Translated849worker1 # [ 0.163341] iommu: DMA domain TLB invalidation policy: strict mode850worker1 # [ 0.163805] SCSI subsystem initialized851worker2 # [ 0.176240] VFS: Disk quotas dquot_6.6.0852worker1 # [ 0.165842] usbcore: registered new interface driver usbfs853worker2 # [ 0.176284] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)854lb # [ 0.187100] NET: Registered PF_INET protocol family855worker1 # [ 0.165876] usbcore: registered new interface driver hub856lb # [ 0.187363] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)857worker1 # [ 0.165893] usbcore: registered new device driver usb858worker1 # [ 0.166192] pps_core: LinuxPPS API ver. 1 registered859worker1 # [ 0.166197] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>860worker2 # [ 0.180279] netfs: FS-Cache loaded861worker1 # [ 0.166207] PTP clock support registered862worker2 # [ 0.180425] pnp: PnP ACPI: disabled863worker1 # [ 0.166258] EDAC MC: Ver: 3.0.0864client # [ 0.225806] NET: Registered PF_XDP protocol family865worker1 # [ 0.170999] scmi_core: SCMI protocol bus registered866client # [ 0.225828] PCI: CLS 0 bytes, default 64867client # [ 0.225982] Trying to unpack rootfs image as initramfs...868worker1 # [ 0.172033] FPGA manager framework869worker1 # [ 0.173043] vgaarb: loaded870worker1 # [ 0.173694] clocksource: Switched to clocksource arch_sys_counter871client # [ 0.235866] kvm [1]: HYP mode not available872worker2 # [ 0.184494] NET: Registered PF_INET protocol family873worker1 # [ 0.176990] VFS: Disk quotas dquot_6.6.0874worker2 # [ 0.184780] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)875worker1 # [ 0.177028] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)876worker1 # [ 0.181122] netfs: FS-Cache loaded877worker1 # [ 0.181268] pnp: PnP ACPI: disabled878worker1 # [ 0.185428] NET: Registered PF_INET protocol family879worker1 # [ 0.186361] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)880lb # [ 0.219612] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881lb # [ 0.219671] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882lb # [ 0.219704] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)883lb # [ 0.219792] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)884lb # [ 0.219938] TCP: Hash tables configured (established 16384 bind 16384)885lb # [ 0.220033] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)886lb # [ 0.220088] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)887worker2 # [ 0.216215] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)888lb # [ 0.220130] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)889lb # [ 0.220228] NET: Registered PF_UNIX/PF_LOCAL protocol family890worker2 # [ 0.216275] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)891lb # [ 0.220249] NET: Registered PF_XDP protocol family892worker2 # [ 0.216309] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)893lb # [ 0.220271] PCI: CLS 0 bytes, default 64894lb # [ 0.220524] Trying to unpack rootfs image as initramfs...895worker2 # [ 0.216389] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)896worker2 # [ 0.216534] TCP: Hash tables configured (established 16384 bind 16384)897worker2 # [ 0.216647] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)898worker2 # [ 0.216684] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)899worker2 # [ 0.216721] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker2 # [ 0.216860] NET: Registered PF_UNIX/PF_LOCAL protocol family901worker2 # [ 0.216886] NET: Registered PF_XDP protocol family902worker2 # [ 0.216908] PCI: CLS 0 bytes, default 64903worker2 # [ 0.217158] Trying to unpack rootfs image as initramfs...904lb # [ 0.238625] kvm [1]: HYP mode not available905worker1 # [ 0.216809] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)906worker1 # [ 0.216865] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)907worker1 # [ 0.216898] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)908worker2 # [ 0.233325] kvm [1]: HYP mode not available909worker1 # [ 0.216985] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910worker1 # [ 0.217129] TCP: Hash tables configured (established 16384 bind 16384)911worker1 # [ 0.217222] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912worker1 # [ 0.217261] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913worker1 # [ 0.217297] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914worker1 # [ 0.217400] NET: Registered PF_UNIX/PF_LOCAL protocol family915worker1 # [ 0.217446] NET: Registered PF_XDP protocol family916worker1 # [ 0.217464] PCI: CLS 0 bytes, default 64917worker1 # [ 0.217754] Trying to unpack rootfs image as initramfs...918worker1 # [ 0.235639] kvm [1]: HYP mode not available919client # [ 0.358566] Initialise system trusted keyrings920client # [ 0.359327] workingset: timestamp_bits=42 max_order=19 bucket_order=0921client # [ 0.360663] squashfs: version 4.0 (2009/01/31) Phillip Lougher922client # [ 0.361428] 9p: Installing v9fs 9p2000 file system support923client # [ 0.390369] Key type asymmetric registered924client # [ 0.390405] Asymmetric key parser 'x509' registered925client # [ 0.390506] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)926client # [ 0.392636] io scheduler mq-deadline registered927client # [ 0.392649] io scheduler kyber registered928lb # [ 0.361311] Initialise system trusted keyrings929client # [ 0.402057] pl061_gpio 9030000.pl061: PL061 GPIO chip registered930lb # [ 0.362148] workingset: timestamp_bits=42 max_order=19 bucket_order=0931lb # [ 0.363437] squashfs: version 4.0 (2009/01/31) Phillip Lougher932lb # [ 0.364268] 9p: Installing v9fs 9p2000 file system support933worker2 # [ 0.356149] Initialise system trusted keyrings934worker2 # [ 0.356917] workingset: timestamp_bits=42 max_order=19 bucket_order=0935client # [ 0.403599] ledtrig-cpu: registered to indicate activity on CPUs936worker2 # [ 0.358229] squashfs: version 4.0 (2009/01/31) Phillip Lougher937client # [ 0.404004] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:938worker2 # [ 0.359031] 9p: Installing v9fs 9p2000 file system support939client # [ 0.404022] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000940client # [ 0.404034] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000941client # [ 0.404043] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000942client # [ 0.404063] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits943worker1 # [ 0.352959] Initialise system trusted keyrings944client # [ 0.404090] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]945client # [ 0.404168] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00946client # [ 0.404178] pci_bus 0000:00: root bus resource [bus 00-ff]947client # [ 0.404184] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]948client # [ 0.404189] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]949client # [ 0.404194] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]950client # [ 0.404291] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint951client # [ 0.404737] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint952client # [ 0.404925] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]953client # [ 0.404945] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]954client # [ 0.404974] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]955worker1 # [ 0.361789] workingset: timestamp_bits=42 max_order=19 bucket_order=0956client # [ 0.404990] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]957worker1 # [ 0.363233] squashfs: version 4.0 (2009/01/31) Phillip Lougher958client # [ 0.405443] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint959client # [ 0.405628] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]960client # [ 0.405644] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]961client # [ 0.405673] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]962client # [ 0.406158] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint963client # [ 0.406339] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]964client # [ 0.406355] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]965client # [ 0.406386] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]966worker1 # [ 0.364043] 9p: Installing v9fs 9p2000 file system support967client # [ 0.406863] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint968client # [ 0.407048] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]969client # [ 0.407064] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]970lb # [ 0.393159] Key type asymmetric registered971lb # [ 0.393190] Asymmetric key parser 'x509' registered972client # [ 0.407093] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]973lb # [ 0.393268] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)974client # [ 0.407112] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]975lb # [ 0.395463] io scheduler mq-deadline registered976client # [ 0.407564] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint977lb # [ 0.395475] io scheduler kyber registered978client # [ 0.407751] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]979client # [ 0.407780] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]980client # [ 0.408230] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint981worker1 # [ 0.376816] Key type asymmetric registered982client # [ 0.408416] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]983worker2 # [ 0.387843] Key type asymmetric registered984worker1 # [ 0.376828] Asymmetric key parser 'x509' registered985client # [ 0.408445] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]986worker2 # [ 0.387873] Asymmetric key parser 'x509' registered987worker1 # [ 0.376886] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)988client # [ 0.408845] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint989worker2 # [ 0.387949] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)990client # [ 0.409025] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]991worker2 # [ 0.390104] io scheduler mq-deadline registered992worker2 # [ 0.390116] io scheduler kyber registered993client # [ 0.409301] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint994client # [ 0.409491] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]995client # [ 0.409521] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996lb # [ 0.404851] pl061_gpio 9030000.pl061: PL061 GPIO chip registered997worker1 # [ 0.385780] io scheduler mq-deadline registered998worker1 # [ 0.385803] io scheduler kyber registered999worker2 # [ 0.399679] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1000lb # [ 0.406423] ledtrig-cpu: registered to indicate activity on CPUs1001lb # [ 0.406851] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1002lb # [ 0.406869] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001003worker2 # [ 0.401209] ledtrig-cpu: registered to indicate activity on CPUs1004lb # [ 0.406883] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001005worker2 # [ 0.401667] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1006lb # [ 0.406892] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001007worker2 # [ 0.401689] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001008lb # [ 0.406913] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1009worker2 # [ 0.401703] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001010lb # [ 0.406941] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1011worker2 # [ 0.401712] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001012lb # [ 0.407038] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001013worker2 # [ 0.401733] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1014worker1 # [ 0.397895] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1015lb # [ 0.407048] pci_bus 0000:00: root bus resource [bus 00-ff]1016worker2 # [ 0.401761] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1017worker2 # [ 0.401841] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001018lb # [ 0.407054] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1019lb # [ 0.407060] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1020worker2 # [ 0.401850] pci_bus 0000:00: root bus resource [bus 00-ff]1021lb # [ 0.407065] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1022worker1 # [ 0.399370] ledtrig-cpu: registered to indicate activity on CPUs1023worker2 # [ 0.401857] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1024lb # [ 0.407123] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1025worker2 # [ 0.401863] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1026worker1 # [ 0.399833] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1027client # [ 0.458089] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1028lb # [ 0.407569] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1029client # [ 0.458312] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1030worker2 # [ 0.401868] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1031worker1 # [ 0.399850] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001032lb # [ 0.407752] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1033client # [ 0.458344] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1034worker2 # [ 0.401927] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1035lb # [ 0.407768] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1036worker1 # [ 0.399863] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001037lb # [ 0.407798] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1038client # [ 0.458909] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1039worker2 # [ 0.402391] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1040worker1 # [ 0.399871] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001041client # [ 0.459112] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1042lb # [ 0.407814] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1043worker2 # [ 0.402579] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1044worker2 # [ 0.402596] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1045client # [ 0.459145] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046worker1 # [ 0.399896] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1047lb # [ 0.408271] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1048worker2 # [ 0.402635] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049lb # [ 0.408455] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1050client # [ 0.459618] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1051worker1 # [ 0.399930] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1052worker2 # [ 0.402651] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1053lb # [ 0.408471] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1054client # [ 0.459922] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1055worker1 # [ 0.400005] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001056client # [ 0.459940] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1057lb # [ 0.408500] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1058worker2 # [ 0.403115] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1059worker1 # [ 0.400014] pci_bus 0000:00: root bus resource [bus 00-ff]1060client # [ 0.459970] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1061worker2 # [ 0.403296] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1062worker1 # [ 0.400020] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1063lb # [ 0.408992] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1064worker2 # [ 0.403312] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1065lb # [ 0.409175] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1066client # [ 0.460458] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1067worker1 # [ 0.400025] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1068worker2 # [ 0.403341] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069lb # [ 0.409191] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1070client # [ 0.460649] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1071worker1 # [ 0.400031] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1072client # [ 0.460665] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1073lb # [ 0.409220] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074worker1 # [ 0.400094] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1075client # [ 0.460695] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1076lb # [ 0.409665] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1077worker1 # [ 0.400543] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1078client # [ 0.461282] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1079lb # [ 0.409866] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1080worker1 # [ 0.400734] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1081lb # [ 0.409882] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1082client # [ 0.461294] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1083worker1 # [ 0.400750] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1084lb # [ 0.409911] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085client # [ 0.461300] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1086worker1 # [ 0.400780] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1087lb # [ 0.409929] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1088worker1 # [ 0.400797] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1089client # [ 0.461347] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1090lb # [ 0.410409] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1091client # [ 0.461395] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1092worker1 # [ 0.401257] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1093lb # [ 0.410597] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1094worker1 # [ 0.401440] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1095client # [ 0.461443] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1096lb # [ 0.410626] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1097worker1 # [ 0.401455] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1098client # [ 0.461493] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1099worker1 # [ 0.401485] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100lb # [ 0.411071] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1101client # [ 0.461543] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1102lb # [ 0.411255] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1103lb # [ 0.411285] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1104client # [ 0.461593] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1105worker1 # [ 0.420490] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1106lb # [ 0.411676] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1107client # [ 0.461640] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1108worker1 # [ 0.420676] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1109lb # [ 0.411856] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1110client # [ 0.461687] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1111worker1 # [ 0.420693] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1112lb # [ 0.412144] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1113client # [ 0.461733] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1114worker1 # [ 0.420723] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115lb # [ 0.412332] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1116worker1 # [ 0.421183] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1117client # [ 0.461810] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1118lb # [ 0.412364] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1119worker2 # [ 0.427940] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1120worker1 # [ 0.421365] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1121worker2 # [ 0.428154] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1122client # [ 0.461857] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1123worker1 # [ 0.421380] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1124worker2 # [ 0.428172] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1125client # [ 0.461879] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1126worker1 # [ 0.421410] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127worker2 # [ 0.428202] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1128client # [ 0.461946] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1129worker1 # [ 0.421429] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1130client # [ 0.461968] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1131worker2 # [ 0.428666] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1132worker2 # [ 0.428856] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1133client # [ 0.461991] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1134worker2 # [ 0.428873] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1135client # [ 0.462013] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1136worker2 # [ 0.428902] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137client # [ 0.462034] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1138worker2 # [ 0.428923] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1139client # [ 0.462063] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1140worker2 # [ 0.429379] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1141client # [ 0.462085] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1142worker2 # [ 0.429568] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1143client # [ 0.462106] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1144worker2 # [ 0.429598] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1145client # [ 0.462129] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1146worker2 # [ 0.430058] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1147lb # [ 0.452955] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1148client # [ 0.462151] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1149worker2 # [ 0.430251] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1150lb # [ 0.453176] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1151client # [ 0.462172] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1152worker2 # [ 0.430280] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1153client # [ 0.462193] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1154lb # [ 0.453206] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1155client # [ 0.462214] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1156worker2 # [ 0.430725] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1157lb # [ 0.453696] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1158client # [ 0.462234] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1159worker2 # [ 0.430910] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1160lb # [ 0.453936] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1161client # [ 0.462255] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1162worker2 # [ 0.431160] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1163lb # [ 0.453970] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164client # [ 0.462276] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1165worker2 # [ 0.431348] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1166client # [ 0.462303] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1167lb # [ 0.454459] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1168worker2 # [ 0.431378] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1169lb # [ 0.454797] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1170client # [ 0.462312] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1171lb # [ 0.454815] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1172worker2 # [ 0.431847] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1173client # [ 0.462317] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1174worker2 # [ 0.432037] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1175lb # [ 0.454848] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1176client # [ 0.463168] pci 0000:00:07.0: enabling device (0000 -> 0002)1177worker2 # [ 0.432068] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1178worker1 # [ 0.433991] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1179lb # [ 0.455318] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1180worker1 # [ 0.434198] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1181worker1 # [ 0.434229] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1182lb # [ 0.455502] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1183worker2 # [ 0.432526] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1184lb # [ 0.455518] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1185worker2 # [ 0.432713] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1186worker1 # [ 0.434698] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1187lb # [ 0.455548] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1188worker1 # [ 0.434888] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1189worker2 # [ 0.432742] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1190lb # [ 0.456182] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1191worker1 # [ 0.434917] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1192worker2 # [ 0.433199] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1193lb # [ 0.456193] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1194worker2 # [ 0.433516] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1195worker1 # [ 0.435313] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1196worker2 # [ 0.433534] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1197lb # [ 0.456198] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1198worker1 # [ 0.435493] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1199worker2 # [ 0.433564] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1200lb # [ 0.456245] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1201worker1 # [ 0.435789] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1202worker2 # [ 0.434028] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1203worker1 # [ 0.435976] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1204lb # [ 0.456293] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1205worker2 # [ 0.434213] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1206worker1 # [ 0.436006] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1207lb # [ 0.456342] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1208worker2 # [ 0.434229] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1209worker1 # [ 0.436461] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1210worker2 # [ 0.434259] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1211lb # [ 0.456391] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1212worker1 # [ 0.436647] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1213worker2 # [ 0.434860] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1214lb # [ 0.456439] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1215worker1 # [ 0.436676] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1216worker2 # [ 0.434872] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1217lb # [ 0.456487] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1218worker1 # [ 0.437134] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1219worker2 # [ 0.434877] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1220worker1 # [ 0.437321] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1221lb # [ 0.456536] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1222worker2 # [ 0.434922] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1223worker1 # [ 0.437351] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1224lb # [ 0.456582] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1225worker2 # [ 0.434970] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1226worker1 # [ 0.437836] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1227lb # [ 0.456629] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1228worker1 # [ 0.438114] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1229worker2 # [ 0.435017] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1230worker1 # [ 0.438131] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1231worker2 # [ 0.435064] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1232worker1 # [ 0.438160] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1233worker2 # [ 0.435110] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1234worker1 # [ 0.438630] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1235client # [ 0.520198] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1236worker2 # [ 0.435157] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1237worker1 # [ 0.438826] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1238worker1 # [ 0.438842] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1239worker2 # [ 0.435205] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1240worker1 # [ 0.438873] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1241worker2 # [ 0.435251] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1242worker1 # [ 0.439462] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1243worker2 # [ 0.435298] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1244worker1 # [ 0.439474] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1245worker2 # [ 0.435380] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1246worker1 # [ 0.439480] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1247worker2 # [ 0.435433] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1248worker1 # [ 0.439530] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1249worker2 # [ 0.435455] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1250lb # [ 0.484815] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1251worker2 # [ 0.435476] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1252worker1 # [ 0.439576] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1253lb # [ 0.484883] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1254worker1 # [ 0.439623] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1255lb # [ 0.484906] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1256worker1 # [ 0.439671] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1257lb # [ 0.484929] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1258worker1 # [ 0.439732] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1259lb # [ 0.484951] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1260worker1 # [ 0.439779] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1261lb # [ 0.484974] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1262worker1 # [ 0.439824] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1263lb # [ 0.484997] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1264lb # [ 0.485020] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1265worker1 # [ 0.439869] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1266lb # [ 0.485048] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1267worker1 # [ 0.439919] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1268lb # [ 0.485071] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1269worker1 # [ 0.440010] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1270lb # [ 0.485094] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1271worker1 # [ 0.440056] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1272lb # [ 0.485118] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1273worker1 # [ 0.440077] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1274worker2 # [ 0.483604] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1275lb # [ 0.485141] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1276worker1 # [ 0.440098] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1277worker2 # [ 0.483650] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1278lb # [ 0.485162] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1279worker1 # [ 0.440122] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1280lb # [ 0.485184] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1281worker2 # [ 0.483673] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1282worker1 # [ 0.440143] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1283lb # [ 0.485205] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1284worker2 # [ 0.483698] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1285worker1 # [ 0.440165] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1286lb # [ 0.485227] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1287worker2 # [ 0.483725] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1288lb # [ 0.485248] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1289worker1 # [ 0.440186] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1290worker2 # [ 0.483748] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1291lb # [ 0.485270] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1292worker1 # [ 0.440211] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1293worker2 # [ 0.483772] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1294lb # [ 0.485301] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1295worker1 # [ 0.440233] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1296worker2 # [ 0.483796] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1297lb # [ 0.485310] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1298worker1 # [ 0.440254] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1299worker2 # [ 0.483821] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1300lb # [ 0.485316] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1301client # [ 0.538184] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1302worker1 # [ 0.440276] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1303worker2 # [ 0.483845] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1304lb # [ 0.486176] pci 0000:00:07.0: enabling device (0000 -> 0002)1305client # [ 0.540303] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1306worker1 # [ 0.440298] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1307worker2 # [ 0.483866] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1308client # [ 0.542521] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1309worker1 # [ 0.440319] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1310worker2 # [ 0.483887] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1311worker1 # [ 0.440340] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1312worker2 # [ 0.483909] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1313client # [ 0.545251] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1314worker1 # [ 0.440360] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1315worker2 # [ 0.483932] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1316worker1 # [ 0.440381] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1317worker2 # [ 0.483954] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1318worker1 # [ 0.440401] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1319worker2 # [ 0.483985] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1320worker1 # [ 0.440422] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1321worker2 # [ 0.483995] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1322worker1 # [ 0.440449] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1323worker2 # [ 0.484000] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1324worker1 # [ 0.440458] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1325worker2 # [ 0.484838] pci 0000:00:07.0: enabling device (0000 -> 0002)1326worker1 # [ 0.440463] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1327worker1 # [ 0.441292] pci 0000:00:07.0: enabling device (0000 -> 0002)1328client # [ 0.556130] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1329worker1 # [ 0.497886] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55270 usecs1330client # [ 0.562020] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1331client # [ 0.564244] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1332lb # [ 0.531761] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1333lb # [ 0.534941] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1334client # [ 0.575085] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1335client # [ 0.577226] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1336worker2 # [ 0.526423] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1337worker2 # [ 0.529562] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1338worker1 # [ 0.524400] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1339lb # [ 0.544963] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1340client # [ 0.589867] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1341lb # [ 0.547218] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1342worker1 # [ 0.527804] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1343lb # [ 0.551563] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1344worker2 # [ 0.540242] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1345worker2 # [ 0.542787] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1346worker2 # [ 0.546699] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1347lb # [ 0.559147] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1348worker1 # [ 0.538041] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1349worker1 # [ 0.540319] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1350client # [ 0.604611] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1351worker1 # [ 0.544493] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1352client # [ 0.607668] msm_serial: driver initialized1353client # [ 0.607825] SuperH (H)SCI(F) driver initialized1354client # [ 0.607882] STM32 USART driver initialized1355worker2 # [ 0.555748] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1356worker2 # [ 0.557749] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1357worker1 # [ 0.552566] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1358lb # [ 0.568975] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1359lb # [ 0.571503] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1360lb # [ 0.573613] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1361lb # [ 0.575633] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1362worker1 # [ 0.562413] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1363worker1 # [ 0.566702] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1364worker1 # [ 0.568729] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1365worker2 # [ 0.575855] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1366worker2 # [ 0.578050] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1367lb # [ 0.594372] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1368worker2 # [ 0.580121] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1369worker2 # [ 0.582663] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1370lb # [ 0.600474] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1371worker1 # [ 0.580320] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1372client # [ 0.645348] loop: module loaded1373client # [ 0.645575] virtio_blk virtio2: 1/0/0 default/read/poll queues1374client # [ 0.647750] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1375lb # [ 0.606064] msm_serial: driver initialized1376lb # [ 0.606224] SuperH (H)SCI(F) driver initialized1377lb # [ 0.606281] STM32 USART driver initialized1378worker1 # [ 0.587155] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1379worker2 # [ 0.600825] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1380worker2 # [ 0.603699] msm_serial: driver initialized1381worker2 # [ 0.603917] SuperH (H)SCI(F) driver initialized1382worker2 # [ 0.603976] STM32 USART driver initialized1383client # [ 0.658791] megasas: 07.734.00.00-rc11384client # [ 0.659547] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1385worker1 # [ 0.604627] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1386client # [ 0.661991] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001387client # [ 0.662017] Intel/Sharp Extended Query Table at 0x00311388client # [ 0.663860] Using buffer write method1389client # [ 0.663948] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1390worker1 # [ 0.607278] msm_serial: driver initialized1391worker1 # [ 0.607423] SuperH (H)SCI(F) driver initialized1392client # [ 0.665771] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001393worker1 # [ 0.607479] STM32 USART driver initialized1394client # [ 0.665796] Intel/Sharp Extended Query Table at 0x00311395lb # [ 0.643803] loop: module loaded1396lb # [ 0.644003] virtio_blk virtio2: 1/0/0 default/read/poll queues1397client # [ 0.686195] Using buffer write method1398client # [ 0.686237] Concatenating MTD devices:1399client # [ 0.686241] (0): "0.flash"1400client # [ 0.686246] (1): "0.flash"1401lb # [ 0.646060] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1402client # [ 0.686256] into device "0.flash"1403worker2 # [ 0.638625] loop: module loaded1404worker2 # [ 0.638830] virtio_blk virtio2: 1/0/0 default/read/poll queues1405worker2 # [ 0.640856] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1406lb # [ 0.652822] megasas: 07.734.00.00-rc11407lb # [ 0.653776] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1408worker1 # [ 0.638841] loop: module loaded1409worker1 # [ 0.639041] virtio_blk virtio2: 1/0/0 default/read/poll queues1410worker1 # [ 0.640041] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1411lb # [ 0.665928] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001412lb # [ 0.666004] Intel/Sharp Extended Query Table at 0x00311413lb # [ 0.667782] Using buffer write method1414worker2 # [ 0.652269] megasas: 07.734.00.00-rc11415lb # [ 0.667867] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1416worker2 # [ 0.653061] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1417worker2 # [ 0.655270] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001418worker2 # [ 0.655304] Intel/Sharp Extended Query Table at 0x00311419worker2 # [ 0.657150] Using buffer write method1420worker2 # [ 0.657242] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1421worker2 # [ 0.659116] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001422worker2 # [ 0.659146] Intel/Sharp Extended Query Table at 0x00311423worker1 # [ 0.654520] megasas: 07.734.00.00-rc11424worker1 # [ 0.655296] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1425lb # [ 0.683704] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001426worker1 # [ 0.657892] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001427lb # [ 0.683748] Intel/Sharp Extended Query Table at 0x00311428worker1 # [ 0.657922] Intel/Sharp Extended Query Table at 0x00311429worker1 # [ 0.659739] Using buffer write method1430lb # [ 0.687792] Using buffer write method1431worker1 # [ 0.659833] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1432lb # [ 0.687835] Concatenating MTD devices:1433lb # [ 0.687840] (0): "0.flash"1434lb # [ 0.687844] (1): "0.flash"1435lb # [ 0.687856] into device "0.flash"1436worker2 # [ 0.679974] Using buffer write method1437worker2 # [ 0.680019] Concatenating MTD devices:1438worker2 # [ 0.680025] (0): "0.flash"1439worker2 # [ 0.680030] (1): "0.flash"1440worker2 # [ 0.680043] into device "0.flash"1441worker1 # [ 0.669747] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001442worker1 # [ 0.669785] Intel/Sharp Extended Query Table at 0x00311443worker1 # [ 0.681755] Using buffer write method1444worker1 # [ 0.681794] Concatenating MTD devices:1445worker1 # [ 0.681798] (0): "0.flash"1446worker1 # [ 0.681807] (1): "0.flash"1447worker1 # [ 0.681810] into device "0.flash"1448client # [ 0.978660] Freeing initrd memory: 26972K1449client # [ 0.985005] tun: Universal TUN/TAP device driver, 1.61450client # [ 0.988870] thunder_xcv, ver 1.01451client # [ 0.988918] thunder_bgx, ver 1.01452client # [ 0.988940] nicpf, ver 1.01453client # [ 0.989499] e1000: Intel(R) PRO/1000 Network Driver1454client # [ 0.989507] e1000: Copyright (c) 1999-2006 Intel Corporation.1455client # [ 0.989530] e1000e: Intel(R) PRO/1000 Network Driver1456client # [ 0.989540] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457client # [ 0.989569] igb: Intel(R) Gigabit Ethernet Network Driver1458client # [ 0.989575] igb: Copyright (c) 2007-2014 Intel Corporation.1459client # [ 0.989618] igbvf: Intel(R) Gigabit Virtual Function Network Driver1460client # [ 0.989625] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1461client # [ 0.989757] sky2: driver version 1.301462client # [ 0.998643] usbcore: registered new interface driver usb-storage1463client # [ 0.998774] usbcore: registered new interface driver usbserial_generic1464client # [ 0.998792] usbserial: USB Serial support registered for generic1465client # [ 0.999407] hv_vmbus: registering driver hyperv_keyboard1466client # [ 1.000802] ehci-pci 0000:00:07.0: EHCI Host Controller1467client # [ 1.000838] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11468client # [ 1.001079] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001469client # [ 1.006574] rtc-pl031 9010000.pl031: registered as rtc01470client # [ 1.006608] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:11:55 UTC (1790140315)1471client # [ 1.007011] i2c_dev: i2c /dev entries driver1472client # [ 1.010093] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001473client # [ 1.011105] hub 1-0:1.0: USB hub found1474client # [ 1.011589] hub 1-0:1.0: 6 ports detected1475client # [ 1.014940] sdhci: Secure Digital Host Controller Interface driver1476client # [ 1.014951] sdhci: Copyright(c) Pierre Ossman1477client # [ 1.015247] Synopsys Designware Multimedia Card Interface Driver1478lb # [ 0.979210] Freeing initrd memory: 26948K1479client # [ 1.015645] sdhci-pltfm: SDHCI platform and OF driver helper1480client # [ 1.017212] hid: raw HID events driver (C) Jiri Kosina1481client # [ 1.017499] usbcore: registered new interface driver usbhid1482client # [ 1.017509] usbhid: USB HID core driver1483client # [ 1.023232] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1484client # [ 1.024810] drop_monitor: Initializing network drop monitor service1485lb # [ 0.985750] tun: Universal TUN/TAP device driver, 1.61486client # [ 1.025013] NET: Registered PF_INET6 protocol family1487client # [ 1.028222] Segment Routing with IPv61488client # [ 1.028241] In-situ OAM (IOAM) with IPv61489client # [ 1.028273] NET: Registered PF_PACKET protocol family1490lb # [ 0.989857] thunder_xcv, ver 1.01491lb # [ 0.989904] thunder_bgx, ver 1.01492lb # [ 0.989927] nicpf, ver 1.01493client # [ 1.029964] 9pnet: Installing 9P2000 support1494client # [ 1.030013] Key type dns_resolver registered1495lb # [ 0.990519] e1000: Intel(R) PRO/1000 Network Driver1496lb # [ 0.990529] e1000: Copyright (c) 1999-2006 Intel Corporation.1497lb # [ 0.990552] e1000e: Intel(R) PRO/1000 Network Driver1498lb # [ 0.990559] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1499lb # [ 0.990588] igb: Intel(R) Gigabit Ethernet Network Driver1500worker2 # [ 0.985243] Freeing initrd memory: 26944K1501client # [ 1.037051] registered taskstats version 11502lb # [ 0.990593] igb: Copyright (c) 2007-2014 Intel Corporation.1503client # [ 1.037215] Loading compiled-in X.509 certificates1504lb # [ 0.990615] igbvf: Intel(R) Gigabit Virtual Function Network Driver1505lb # [ 0.990620] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1506lb # [ 0.990752] sky2: driver version 1.301507lb # [ 0.992324] usbcore: registered new interface driver usb-storage1508lb # [ 0.992450] usbcore: registered new interface driver usbserial_generic1509lb # [ 0.992463] usbserial: USB Serial support registered for generic1510worker2 # [ 0.991546] tun: Universal TUN/TAP device driver, 1.61511lb # [ 1.003132] ehci-pci 0000:00:07.0: EHCI Host Controller1512lb # [ 1.003166] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11513lb # [ 1.003409] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001514worker1 # [ 0.981175] Freeing initrd memory: 26976K1515client # [ 1.046165] Demotion targets for Node 0: null1516client # [ 1.046298] Key type .fscrypt registered1517lb # [ 1.006388] hv_vmbus: registering driver hyperv_keyboard1518client # [ 1.046305] Key type fscrypt-provisioning registered1519client # [ 1.046425] ima: No TPM chip found, activating TPM-bypass!1520client # [ 1.046445] ima: Allocated hash algorithm: sha11521lb # [ 1.008040] rtc-pl031 9010000.pl031: registered as rtc01522client # [ 1.046469] ima: No architecture policies found1523lb # [ 1.008072] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:11:55 UTC (1790140315)1524lb # [ 1.008375] i2c_dev: i2c /dev entries driver1525client # [ 1.050828] input: gpio-keys as /devices/platform/gpio-keys/input/input01526worker1 # [ 0.988357] tun: Universal TUN/TAP device driver, 1.61527worker2 # [ 0.995713] thunder_xcv, ver 1.01528worker2 # [ 0.995756] thunder_bgx, ver 1.01529worker2 # [ 0.995782] nicpf, ver 1.01530worker2 # [ 0.996353] e1000: Intel(R) PRO/1000 Network Driver1531worker2 # [ 0.996364] e1000: Copyright (c) 1999-2006 Intel Corporation.1532worker2 # [ 0.996387] e1000e: Intel(R) PRO/1000 Network Driver1533worker2 # [ 0.996396] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1534worker1 # [ 0.992308] thunder_xcv, ver 1.01535worker2 # [ 0.996428] igb: Intel(R) Gigabit Ethernet Network Driver1536worker1 # [ 0.992357] thunder_bgx, ver 1.01537worker1 # [ 0.992379] nicpf, ver 1.01538worker2 # [ 0.996434] igb: Copyright (c) 2007-2014 Intel Corporation.1539lb # [ 1.013879] sdhci: Secure Digital Host Controller Interface driver1540worker1 # [ 0.992961] e1000: Intel(R) PRO/1000 Network Driver1541lb # [ 1.013895] sdhci: Copyright(c) Pierre Ossman1542worker2 # [ 0.996457] igbvf: Intel(R) Gigabit Virtual Function Network Driver1543worker1 # [ 0.992971] e1000: Copyright (c) 1999-2006 Intel Corporation.1544worker2 # [ 0.996465] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1545lb # [ 1.014169] Synopsys Designware Multimedia Card Interface Driver1546worker1 # [ 0.992994] e1000e: Intel(R) PRO/1000 Network Driver1547worker2 # [ 0.996618] sky2: driver version 1.301548lb # [ 1.014543] sdhci-pltfm: SDHCI platform and OF driver helper1549worker1 # [ 0.993001] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1550worker2 # [ 0.998258] usbcore: registered new interface driver usb-storage1551lb # [ 1.016134] hid: raw HID events driver (C) Jiri Kosina1552worker1 # [ 0.993030] igb: Intel(R) Gigabit Ethernet Network Driver1553lb # [ 1.016400] usbcore: registered new interface driver usbhid1554worker2 # [ 0.998438] usbcore: registered new interface driver usbserial_generic1555worker1 # [ 0.993036] igb: Copyright (c) 2007-2014 Intel Corporation.1556lb # [ 1.016406] usbhid: USB HID core driver1557worker2 # [ 0.998457] usbserial: USB Serial support registered for generic1558worker1 # [ 0.993080] igbvf: Intel(R) Gigabit Virtual Function Network Driver1559worker2 # [ 0.999063] hv_vmbus: registering driver hyperv_keyboard1560worker1 # [ 0.993086] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1561lb # [ 1.021188] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001562worker1 # [ 0.993226] sky2: driver version 1.301563lb # [ 1.021525] hub 1-0:1.0: USB hub found1564worker2 # [ 1.000199] ehci-pci 0000:00:07.0: EHCI Host Controller1565lb # [ 1.021593] hub 1-0:1.0: 6 ports detected1566worker2 # [ 1.000225] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11567worker2 # [ 1.000446] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001568worker1 # [ 1.002411] ehci-pci 0000:00:07.0: EHCI Host Controller1569lb # [ 1.024464] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1570worker2 # [ 1.012822] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001571worker2 # [ 1.013202] hub 1-0:1.0: USB hub found1572worker1 # [ 1.002448] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11573worker2 # [ 1.013223] hub 1-0:1.0: 6 ports detected1574worker1 # [ 1.002695] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001575lb # [ 1.027214] drop_monitor: Initializing network drop monitor service1576worker1 # [ 1.005528] usbcore: registered new interface driver usb-storage1577lb # [ 1.027417] NET: Registered PF_INET6 protocol family1578worker2 # [ 1.015916] rtc-pl031 9010000.pl031: registered as rtc01579worker1 # [ 1.005665] usbcore: registered new interface driver usbserial_generic1580client # [ 1.069672] clk: Disabling unused clocks1581worker1 # [ 1.005680] usbserial: USB Serial support registered for generic1582lb # [ 1.029410] Segment Routing with IPv61583worker2 # [ 1.015943] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:11:55 UTC (1790140315)1584client # [ 1.069710] PM: genpd: Disabling unused power domains1585lb # [ 1.029429] In-situ OAM (IOAM) with IPv61586worker2 # [ 1.016334] i2c_dev: i2c /dev entries driver1587lb # [ 1.029460] NET: Registered PF_PACKET protocol family1588worker1 # [ 1.008595] hv_vmbus: registering driver hyperv_keyboard1589lb # [ 1.031086] 9pnet: Installing 9P2000 support1590lb # [ 1.031143] Key type dns_resolver registered1591worker1 # [ 1.010202] rtc-pl031 9010000.pl031: registered as rtc01592client # [ 1.073923] Freeing unused kernel memory: 4736K1593client # [ 1.074120] Run /init as init process1594worker1 # [ 1.010238] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:11:55 UTC (1790140315)1595worker1 # [ 1.010551] i2c_dev: i2c /dev entries driver1596worker2 # [ 1.021783] sdhci: Secure Digital Host Controller Interface driver1597worker2 # [ 1.021800] sdhci: Copyright(c) Pierre Ossman1598worker2 # [ 1.022064] Synopsys Designware Multimedia Card Interface Driver1599worker2 # [ 1.022466] sdhci-pltfm: SDHCI platform and OF driver helper1600worker2 # [ 1.026922] hid: raw HID events driver (C) Jiri Kosina1601worker2 # [ 1.027191] usbcore: registered new interface driver usbhid1602worker1 # [ 1.016074] sdhci: Secure Digital Host Controller Interface driver1603worker2 # [ 1.027199] usbhid: USB HID core driver1604worker1 # [ 1.016086] sdhci: Copyright(c) Pierre Ossman1605lb # [ 1.038116] registered taskstats version 11606worker1 # [ 1.016349] Synopsys Designware Multimedia Card Interface Driver1607lb # [ 1.038259] Loading compiled-in X.509 certificates1608worker1 # [ 1.016728] sdhci-pltfm: SDHCI platform and OF driver helper1609worker1 # [ 1.020465] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001610worker1 # [ 1.021528] hub 1-0:1.0: USB hub found1611worker2 # [ 1.030217] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1612worker1 # [ 1.022069] hub 1-0:1.0: 6 ports detected1613worker2 # [ 1.032917] drop_monitor: Initializing network drop monitor service1614worker2 # [ 1.033126] NET: Registered PF_INET6 protocol family1615worker2 # [ 1.035081] Segment Routing with IPv61616worker2 # [ 1.035113] In-situ OAM (IOAM) with IPv61617worker1 # [ 1.023684] hid: raw HID events driver (C) Jiri Kosina1618worker2 # [ 1.035145] NET: Registered PF_PACKET protocol family1619worker1 # [ 1.023963] usbcore: registered new interface driver usbhid1620lb # [ 1.047321] Demotion targets for Node 0: null1621worker1 # [ 1.023970] usbhid: USB HID core driver1622lb # [ 1.047457] Key type .fscrypt registered1623lb # [ 1.047463] Key type fscrypt-provisioning registered1624lb # [ 1.047568] ima: No TPM chip found, activating TPM-bypass!1625lb # [ 1.047590] ima: Allocated hash algorithm: sha11626lb # [ 1.047614] ima: No architecture policies found1627worker2 # [ 1.037016] 9pnet: Installing 9P2000 support1628worker1 # [ 1.026933] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1629worker2 # [ 1.037069] Key type dns_resolver registered1630worker1 # [ 1.028569] drop_monitor: Initializing network drop monitor service1631lb # [ 1.052027] input: gpio-keys as /devices/platform/gpio-keys/input/input01632client # [ 1.092859] systemd[1]: Successfully made /usr/ read-only.1633worker1 # [ 1.028723] NET: Registered PF_INET6 protocol family1634worker1 # [ 1.031806] Segment Routing with IPv61635worker1 # [ 1.031825] In-situ OAM (IOAM) with IPv61636worker1 # [ 1.031851] NET: Registered PF_PACKET protocol family1637worker1 # [ 1.033469] 9pnet: Installing 9P2000 support1638worker1 # [ 1.033522] Key type dns_resolver registered1639worker2 # [ 1.043932] registered taskstats version 11640worker2 # [ 1.044101] Loading compiled-in X.509 certificates1641worker1 # [ 1.040525] registered taskstats version 11642worker1 # [ 1.040693] Loading compiled-in X.509 certificates1643worker2 # [ 1.053007] Demotion targets for Node 0: null1644worker2 # [ 1.053135] Key type .fscrypt registered1645worker2 # [ 1.053141] Key type fscrypt-provisioning registered1646worker2 # [ 1.053249] ima: No TPM chip found, activating TPM-bypass!1647worker2 # [ 1.053268] ima: Allocated hash algorithm: sha11648worker2 # [ 1.053292] ima: No architecture policies found1649worker2 # [ 1.057464] input: gpio-keys as /devices/platform/gpio-keys/input/input01650lb # [ 1.071006] clk: Disabling unused clocks1651worker1 # [ 1.050074] Demotion targets for Node 0: null1652worker1 # [ 1.050220] Key type .fscrypt registered1653lb # [ 1.071043] PM: genpd: Disabling unused power domains1654worker1 # [ 1.050226] Key type fscrypt-provisioning registered1655worker1 # [ 1.050339] ima: No TPM chip found, activating TPM-bypass!1656worker1 # [ 1.050361] ima: Allocated hash algorithm: sha11657worker1 # [ 1.050386] ima: No architecture policies found1658lb # [ 1.075365] Freeing unused kernel memory: 4736K1659lb # [ 1.075728] Run /init as init process1660worker1 # [ 1.054627] input: gpio-keys as /devices/platform/gpio-keys/input/input01661worker2 # [ 1.077375] clk: Disabling unused clocks1662lb # [ 1.091391] systemd[1]: Successfully made /usr/ read-only.1663worker2 # [ 1.077420] PM: genpd: Disabling unused power domains1664worker2 # [ 1.081798] Freeing unused kernel memory: 4736K1665worker2 # [ 1.082026] Run /init as init process1666worker1 # [ 1.074112] clk: Disabling unused clocks1667worker1 # [ 1.074145] PM: genpd: Disabling unused power domains1668worker1 # [ 1.078465] Freeing unused kernel memory: 4736K1669worker1 # [ 1.078674] Run /init as init process1670worker2 # [ 1.098093] systemd[1]: Successfully made /usr/ read-only.1671worker1 # [ 1.096181] systemd[1]: Successfully made /usr/ read-only.1672client # [ 1.258023] usb 1-1: new high-speed USB device number 2 using ehci-pci1673lb # [ 1.268778] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker2 # [ 1.259581] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker1 # [ 1.273789] usb 1-1: new high-speed USB device number 2 using ehci-pci1676client # [ 1.414875] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677client # [ 1.428328] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678client # [ 1.440820] systemd[1]: Detected virtualization qemu.1679client # [ 1.443115] systemd[1]: Detected architecture arm64.1680client # [ 1.445081] systemd[1]: Running in initrd.1681client # [ 1.448032] systemd[1]: Initializing machine ID from random generator.1682client # [ 1.451202] systemd[1]: Hostname set to <client>.1683lb # [ 1.421571] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684worker2 # [ 1.412295] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11685lb # [ 1.428307] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1686lb # [ 1.441303] systemd[1]: Detected virtualization qemu.1687lb # [ 1.441425] systemd[1]: Detected architecture arm64.1688lb # [ 1.441449] systemd[1]: Running in initrd.1689lb # [ 1.442662] systemd[1]: Initializing machine ID from random generator.1690lb # [ 1.442975] systemd[1]: Hostname set to <lb>.1691worker1 # [ 1.428456] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11692worker2 # [ 1.433203] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1693worker2 # [ 1.445768] systemd[1]: Detected virtualization qemu.1694worker2 # [ 1.448048] systemd[1]: Detected architecture arm64.1695worker2 # [ 1.450009] systemd[1]: Running in initrd.1696worker2 # [ 1.452858] systemd[1]: Initializing machine ID from random generator.1697worker2 # [ 1.456123] systemd[1]: Hostname set to <worker2>.1698worker1 # [ 1.435019] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1699client # [ 1.510292] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01700worker1 # [ 1.435080] systemd[1]: Detected virtualization qemu.1701worker1 # [ 1.435178] systemd[1]: Detected architecture arm64.1702worker1 # [ 1.435204] systemd[1]: Running in initrd.1703worker1 # [ 1.436415] systemd[1]: Initializing machine ID from random generator.1704worker1 # [ 1.436750] systemd[1]: Hostname set to <worker1>.1705lb # [ 1.513076] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker2 # [ 1.507838] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker1 # [ 1.518067] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708client # [ 1.629947] usb 1-2: new high-speed USB device number 3 using ehci-pci1709lb # [ 1.636749] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker2 # [ 1.627581] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker1 # [ 1.641739] usb 1-2: new high-speed USB device number 3 using ehci-pci1712client # [ 1.774521] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713client # [ 1.786099] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21714client # [ 1.794057] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01715lb # [ 1.783938] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716lb # [ 1.799439] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21717worker2 # [ 1.791635] systemd[1]: bpf-restrict-fs: LSM BPF program attached1718lb # [ 1.805341] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01719worker2 # [ 1.794892] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21720worker2 # [ 1.795314] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01721worker1 # [ 1.798238] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21722worker1 # [ 1.804038] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723worker1 # [ 1.817453] systemd[1]: bpf-restrict-fs: LSM BPF program attached1724client # [ 1.884776] systemd[1]: Queued start job for default target Initrd Default Target.1725client # [ 1.897460] systemd[1]: Created slice Slice /system/modprobe.1726client # [ 1.898908] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1727client # [ 1.900321] systemd[1]: Expecting device /dev/disk/by-label/nixos...1728client # [ 1.901454] systemd[1]: Reached target Path Units.1729client # [ 1.902379] systemd[1]: Reached target Slice Units.1730client # [ 1.903284] systemd[1]: Reached target Swaps.1731client # [ 1.904064] systemd[1]: Reached target Timer Units.1732client # [ 1.905169] systemd[1]: Listening on D-Bus System Message Bus Socket.1733client # [ 1.906562] systemd[1]: Listening on Journal Socket (/dev/log).1734client # [ 1.906726] systemd[1]: Listening on Journal Sockets.1735client # [ 1.906909] systemd[1]: Listening on udev Control Socket.1736client # [ 1.907065] systemd[1]: Listening on udev Kernel Socket.1737client # [ 1.907093] systemd[1]: Reached target Socket Units.1738client # [ 1.913795] systemd[1]: Starting Create List of Static Device Nodes...1739client # [ 1.915089] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1740client # [ 1.923645] systemd[1]: Mounting Kernel Configuration File System...1741client # [ 1.935202] systemd[1]: Starting Journal Service...1742lb # [ 1.907609] systemd[1]: Queued start job for default target Initrd Default Target.1743worker2 # [ 1.905979] systemd[1]: Queued start job for default target Initrd Default Target.1744lb # [ 1.919448] systemd[1]: Created slice Slice /system/modprobe.1745lb # [ 1.920805] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1746lb # [ 1.920851] systemd[1]: Expecting device /dev/disk/by-label/nixos...1747lb # [ 1.920881] systemd[1]: Reached target Path Units.1748lb # [ 1.920902] systemd[1]: Reached target Slice Units.1749lb # [ 1.920923] systemd[1]: Reached target Swaps.1750lb # [ 1.920943] systemd[1]: Reached target Timer Units.1751lb # [ 1.921141] systemd[1]: Listening on D-Bus System Message Bus Socket.1752lb # [ 1.921305] systemd[1]: Listening on Journal Socket (/dev/log).1753lb # [ 1.921485] systemd[1]: Listening on Journal Sockets.1754worker2 # [ 1.918881] systemd[1]: Created slice Slice /system/modprobe.1755client # [ 1.970528] systemd[1]: Starting Load Kernel Modules...1756lb # [ 1.921680] systemd[1]: Listening on udev Control Socket.1757client # [ 1.970681] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1758worker1 # [ 1.910034] systemd[1]: Queued start job for default target Initrd Default Target.1759lb # [ 1.921829] systemd[1]: Listening on udev Kernel Socket.1760worker2 # [ 1.920395] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1761lb # [ 1.921855] systemd[1]: Reached target Socket Units.1762worker2 # [ 1.921929] systemd[1]: Expecting device /dev/disk/by-label/nixos...1763worker2 # [ 1.923189] systemd[1]: Reached target Path Units.1764lb # [ 1.934808] systemd[1]: Starting Create List of Static Device Nodes...1765worker2 # [ 1.924196] systemd[1]: Reached target Slice Units.1766worker2 # [ 1.925144] systemd[1]: Reached target Swaps.1767lb # [ 1.936012] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1768worker2 # [ 1.926036] systemd[1]: Reached target Timer Units.1769worker2 # [ 1.927244] systemd[1]: Listening on D-Bus System Message Bus Socket.1770worker2 # [ 1.928754] systemd[1]: Listening on Journal Socket (/dev/log).1771worker2 # [ 1.930054] systemd[1]: Listening on Journal Sockets.1772worker1 # [ 1.920242] systemd[1]: Created slice Slice /system/modprobe.1773worker2 # [ 1.931201] systemd[1]: Listening on udev Control Socket.1774worker1 # [ 1.921380] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1775worker1 # [ 1.922672] systemd[1]: Expecting device /dev/disk/by-label/nixos...1776lb # [ 1.944944] systemd[1]: Mounting Kernel Configuration File System...1777worker1 # [ 1.923654] systemd[1]: Reached target Path Units.1778worker2 # [ 1.932524] systemd[1]: Listening on udev Kernel Socket.1779worker1 # [ 1.924423] systemd[1]: Reached target Slice Units.1780worker2 # [ 1.932575] systemd[1]: Reached target Socket Units.1781worker1 # [ 1.925188] systemd[1]: Reached target Swaps.1782worker1 # [ 1.925892] systemd[1]: Reached target Timer Units.1783worker1 # [ 1.926827] systemd[1]: Listening on D-Bus System Message Bus Socket.1784worker2 # [ 1.936474] systemd[1]: Starting Create List of Static Device Nodes...1785worker1 # [ 1.927964] systemd[1]: Listening on Journal Socket (/dev/log).1786worker1 # [ 1.929058] systemd[1]: Listening on Journal Sockets.1787worker2 # [ 1.936579] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1788worker1 # [ 1.930025] systemd[1]: Listening on udev Control Socket.1789worker1 # [ 1.930172] systemd[1]: Listening on udev Kernel Socket.1790worker1 # [ 1.930228] systemd[1]: Reached target Socket Units.1791worker1 # [ 1.934354] systemd[1]: Starting Create List of Static Device Nodes...1792lb # [ 1.957033] systemd[1]: Starting Journal Service...1793worker1 # [ 1.935399] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1794client # [ 2.000912] systemd-journald[72]: Collecting audit messages is disabled.1795worker2 # [ 1.947758] systemd[1]: Mounting Kernel Configuration File System...1796client # [ 2.002074] systemd[1]: Starting Coldplug All udev Devices...1797worker1 # [ 1.942921] systemd[1]: Mounting Kernel Configuration File System...1798client # [ 2.007412] systemd[1]: Finished Create List of Static Device Nodes.1799worker2 # [ 1.955858] systemd[1]: Starting Journal Service...1800worker1 # [ 1.954170] systemd[1]: Starting Journal Service...1801lb # [ 1.978388] systemd[1]: Starting Load Kernel Modules...1802lb # [ 1.980086] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1803client # [ 2.019609] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1804client # [ 2.022453] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1805client # [ 2.025252] systemd[1]: Mounted Kernel Configuration File System.1806lb # [ 1.985169] systemd[1]: Starting Coldplug All udev Devices...1807worker2 # [ 1.991838] systemd[1]: Starting Load Kernel Modules...1808client # [ 2.037994] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1809worker2 # [ 1.991989] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1810client # [ 2.044679] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01811lb # [ 2.007132] systemd[1]: Finished Create List of Static Device Nodes.1812client # [ 2.044984] [drm] features: -virgl +edid -resource_blob -host_visible1813client # [ 2.045000] [drm] features: -context_init1814worker1 # [ 1.986049] systemd[1]: Starting Load Kernel Modules...1815client # [ 2.045769] [drm] number of scanouts: 11816client # [ 2.045791] [drm] number of cap sets: 01817worker1 # [ 1.986967] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1818lb # [ 2.011994] systemd-journald[72]: Collecting audit messages is disabled.1819lb # [ 2.013822] systemd[1]: Mounted Kernel Configuration File System.1820lb # [ 2.025213] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1821worker2 # [ 2.015733] systemd[1]: Starting Coldplug All udev Devices...1822worker2 # [ 2.024688] systemd-journald[72]: Collecting audit messages is disabled.1823client # [ 2.070937] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1824client # [ 2.070965] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01825worker2 # [ 2.028705] systemd[1]: Finished Create List of Static Device Nodes.1826worker2 # [ 2.029529] systemd[1]: Mounted Kernel Configuration File System.1827worker1 # [ 2.017882] systemd[1]: Starting Coldplug All udev Devices...1828worker1 # [ 2.019876] systemd-journald[72]: Collecting audit messages is disabled.1829worker1 # [ 2.024361] systemd[1]: Finished Create List of Static Device Nodes.1830worker1 # [ 2.026224] systemd[1]: Mounted Kernel Configuration File System.1831worker2 # [ 2.045002] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1832worker2 # [ 2.047697] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1833worker1 # [ 2.040755] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1834client # [ 2.099413] Console: switching to colour frame buffer device 160x501835client # [ 2.110735] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1836client # [ 2.113502] systemd[1]: Starting Create Static Device Nodes in /dev...1837worker2 # [ 2.055601] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1838worker2 # [ 2.062071] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01839worker2 # [ 2.062344] [drm] features: -virgl +edid -resource_blob -host_visible1840worker2 # [ 2.062354] [drm] features: -context_init1841worker2 # [ 2.063067] [drm] number of scanouts: 11842worker2 # [ 2.063086] [drm] number of cap sets: 01843worker1 # [ 2.054197] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1844worker1 # [ 2.058773] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1845client # [ 2.127270] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1846lb # [ 2.085565] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1847worker1 # [ 2.075158] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01848worker1 # [ 2.075402] [drm] features: -virgl +edid -resource_blob -host_visible1849worker1 # [ 2.075412] [drm] features: -context_init1850worker1 # [ 2.076153] [drm] number of scanouts: 11851worker1 # [ 2.076171] [drm] number of cap sets: 01852lb # [ 2.101481] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1853lb # [ 2.103952] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1854client # [ 2.150637] systemd[1]: Finished Load Kernel Modules.1855lb # [ 2.111729] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01856lb # [ 2.111973] [drm] features: -virgl +edid -resource_blob -host_visible1857worker2 # [ 2.095913] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1858lb # [ 2.111984] [drm] features: -context_init1859worker2 # [ 2.095939] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01860lb # [ 2.115274] systemd[1]: Starting Create Static Device Nodes in /dev...1861worker1 # [ 2.095324] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1862worker1 # [ 2.095351] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01863client # [ 2.158477] systemd[1]: Starting Apply Kernel Variables...1864worker2 # [ 2.112268] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1865worker2 # [ 2.115934] systemd[1]: Starting Create Static Device Nodes in /dev...1866lb # [ 2.136805] [drm] number of scanouts: 11867lb # [ 2.136844] [drm] number of cap sets: 01868client # [ 2.176519] systemd[1]: Finished Create Static Device Nodes in /dev.1869worker2 # [ 2.119914] Console: switching to colour frame buffer device 160x501870client # [ 2.176740] systemd[1]: Reached target Preparation for Local File Systems.1871client # [ 2.176804] systemd[1]: Reached target Local File Systems.1872worker1 # [ 2.122481] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1873lb # [ 2.139336] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1874lb # [ 2.139361] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01875client # [ 2.186273] systemd[1]: Starting Rule-based Manager for Device Events and Files...1876worker1 # [ 2.126116] Console: switching to colour frame buffer device 160x501877worker1 # [ 2.138148] systemd[1]: Starting Create Static Device Nodes in /dev...1878client # [ 2.188433] systemd-modules-load[73]: Inserted module 'dm_mod'1879worker2 # [ 2.152273] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1880client # [ 2.189637] systemd-modules-load[73]: Module 'virtio_balloon' is built in1881client # [ 2.190775] systemd-modules-load[73]: Module 'virtio_console' is built in1882client # [ 2.191874] systemd-modules-load[73]: Inserted module 'virtio_gpu'1883lb # [ 2.169076] Console: switching to colour frame buffer device 160x501884client # [ 2.214206] systemd[1]: Started Journal Service.1885worker2 # [ 2.165080] systemd[1]: Finished Load Kernel Modules.1886worker1 # [ 2.154487] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1887lb # [ 2.180124] systemd[1]: Finished Create Static Device Nodes in /dev.1888lb # [ 2.180348] systemd[1]: Reached target Preparation for Local File Systems.1889lb # [ 2.180373] systemd[1]: Reached target Local File Systems.1890worker2 # [ 2.172046] systemd[1]: Starting Apply Kernel Variables...1891client # [ 2.209410] systemd-modules-load[73]: Module 'virtio_rng' is built in1892worker1 # [ 2.167364] systemd[1]: Finished Load Kernel Modules.1893lb # [ 2.189365] systemd[1]: Starting Rule-based Manager for Device Events and Files...1894worker1 # [ 2.174288] systemd[1]: Starting Apply Kernel Variables...1895lb # [ 2.197440] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1896client # [ 2.226066] systemd[1]: Finished Apply Kernel Variables.1897worker2 # [ 2.192302] systemd[1]: Finished Create Static Device Nodes in /dev.1898worker2 # [ 2.195082] systemd[1]: Reached target Preparation for Local File Systems.1899lb # [ 2.211858] systemd[1]: Finished Load Kernel Modules.1900client # [ 2.240132] systemd[1]: Starting Create System Files and Directories...1901worker2 # [ 2.203869] systemd[1]: Reached target Local File Systems.1902lb # [ 2.217176] systemd[1]: Starting Apply Kernel Variables...1903worker2 # [ 2.212022] systemd[1]: Starting Rule-based Manager for Device Events and Files...1904lb # [ 2.216352] systemd-modules-load[73]: Inserted module 'dm_mod'1905worker1 # [ 2.210152] systemd[1]: Finished Create Static Device Nodes in /dev.1906lb # [ 2.219033] systemd-modules-load[73]: Module 'virtio_balloon' is built in1907worker1 # [ 2.210646] systemd[1]: Reached target Preparation for Local File Systems.1908worker2 # [ 2.209524] systemd-modules-load[73]: Inserted module 'dm_mod'1909worker1 # [ 2.210709] systemd[1]: Reached target Local File Systems.1910worker2 # [ 2.210673] systemd-modules-load[73]: Module 'virtio_balloon' is built in1911worker1 # [ 2.202512] systemd-modules-load[73]: Inserted module 'dm_mod'1912worker1 # [ 2.203847] systemd-modules-load[73]: Module 'virtio_balloon' is built in1913lb # [ 2.224403] systemd-modules-load[73]: Module 'virtio_console' is built in1914lb # [ 2.225559] systemd-modules-load[73]: Inserted module 'virtio_gpu'1915worker1 # [ 2.219178] systemd[1]: Starting Rule-based Manager for Device Events and Files...1916lb # [ 2.226543] systemd-modules-load[73]: Module 'virtio_rng' is built in1917worker1 # [ 2.221647] systemd[1]: Started Journal Service.1918worker2 # [ 2.219675] systemd-modules-load[73]: Module 'virtio_console' is built in1919lb # [ 2.244826] systemd[1]: Started Journal Service.1920worker2 # [ 2.236559] systemd[1]: Started Journal Service.1921client # [ 2.272962] systemd[1]: Finished Create System Files and Directories.1922worker2 # [ 2.224482] systemd-modules-load[73]: Inserted module 'virtio_gpu'1923worker1 # [ 2.216583] systemd-modules-load[73]: Module 'virtio_console' is built in1924worker2 # [ 2.226093] systemd-modules-load[73]: Module 'virtio_rng' is built in1925client # [ 2.275658] systemd-udevd[80]: Using default interface naming scheme 'v261'.1926worker1 # [ 2.217747] systemd-modules-load[73]: Inserted module 'virtio_gpu'1927worker1 # [ 2.218826] systemd-modules-load[73]: Module 'virtio_rng' is built in1928worker2 # [ 2.236443] systemd[1]: Starting Create System Files and Directories...1929lb # [ 2.254268] systemd[1]: Starting Create System Files and Directories...1930worker1 # [ 2.244204] systemd[1]: Starting Create System Files and Directories...1931worker2 # [ 2.253904] systemd[1]: Finished Apply Kernel Variables.1932client # [ 2.305699] systemd[1]: Started Rule-based Manager for Device Events and Files.1933worker1 # [ 2.248383] systemd[1]: Finished Apply Kernel Variables.1934lb # [ 2.290043] systemd-udevd[78]: Using default interface naming scheme 'v261'.1935lb # [ 2.292177] systemd[1]: Finished Apply Kernel Variables.1936worker2 # [ 2.285438] systemd[1]: Finished Create System Files and Directories.1937worker1 # [ 2.285692] systemd[1]: Finished Create System Files and Directories.1938worker2 # [ 2.298169] systemd-udevd[80]: Using default interface naming scheme 'v261'.1939lb # [ 2.313130] systemd[1]: Finished Create System Files and Directories.1940client # [ 2.370818] systemd[1]: Starting Virtual Console Setup...1941worker1 # [ 2.313845] systemd-udevd[80]: Using default interface naming scheme 'v261'.1942lb # [ 2.333411] systemd[1]: Started Rule-based Manager for Device Events and Files.1943worker2 # [ 2.330109] systemd[1]: Started Rule-based Manager for Device Events and Files.1944worker1 # [ 2.345049] systemd[1]: Started Rule-based Manager for Device Events and Files.1945client # [ 2.424687] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1946client # [ 2.428227] systemd[1]: Finished Virtual Console Setup.1947lb # [ 2.400188] systemd[1]: Starting Virtual Console Setup...1948worker2 # [ 2.396785] systemd[1]: Starting Virtual Console Setup...1949worker1 # [ 2.414684] systemd[1]: Starting Virtual Console Setup...1950lb # [ 2.460660] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1951lb # [ 2.464563] systemd[1]: Finished Virtual Console Setup.1952worker2 # [ 2.460580] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1953worker2 # [ 2.463461] systemd[1]: Finished Virtual Console Setup.1954worker1 # [ 2.476581] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1955worker1 # [ 2.480298] systemd[1]: Finished Virtual Console Setup.1956client # [ 3.080458] systemd[1]: Finished Coldplug All udev Devices.1957client # [ 3.081530] systemd[1]: Reached target System Initialization.1958client # [ 3.084129] systemd[1]: Reached target Basic System.1959lb # [ 3.098496] systemd[1]: Finished Coldplug All udev Devices.1960lb # [ 3.099512] systemd[1]: Reached target System Initialization.1961lb # [ 3.100413] systemd[1]: Reached target Basic System.1962worker2 # [ 3.101240] systemd[1]: Finished Coldplug All udev Devices.1963worker2 # [ 3.103002] systemd[1]: Reached target System Initialization.1964worker2 # [ 3.103856] systemd[1]: Reached target Basic System.1965worker1 # [ 3.097211] systemd[1]: Finished Coldplug All udev Devices.1966worker1 # [ 3.098976] systemd[1]: Reached target System Initialization.1967worker1 # [ 3.099869] systemd[1]: Reached target Basic System.1968client # [ 3.253498] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1969client # [ 3.279679] (udev-worker)[104]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1970client # [ 3.284212] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line.1971lb # [ 3.256728] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972lb # [ 3.294698] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1973lb # [ 3.300489] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1974worker2 # [ 3.290446] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975worker1 # [ 3.285668] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1976client # [ 3.345132] systemd[1]: Found device /dev/disk/by-label/nixos.1977client # [ 3.346487] systemd[1]: Reached target Initrd Root Device.1978client # [ 3.350744] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1979worker2 # [ 3.307106] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1980worker1 # [ 3.302698] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1981worker2 # [ 3.316695] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1982worker1 # [ 3.309469] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1983client # [ 3.411506] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1984client # [ 3.420342] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1985client # [ 3.424746] systemd[1]: Mounting /sysroot...1986lb # [ 3.388908] systemd[1]: Found device /dev/disk/by-label/nixos.1987lb # [ 3.390791] systemd[1]: Reached target Initrd Root Device.1988lb # [ 3.395504] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989worker1 # [ 3.382047] systemd[1]: Found device /dev/disk/by-label/nixos.1990worker1 # [ 3.385417] systemd[1]: Reached target Initrd Root Device.1991worker1 # [ 3.392477] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992client # [ 3.485485] EXT4-fs (vda): mounted filesystem a62f6ea9-c1fc-4c8f-81b0-c7a81bf4e5dd r/w with ordered data mode. Quota mode: none.1993worker2 # [ 3.421875] systemd[1]: Found device /dev/disk/by-label/nixos.1994client # [ 3.474060] systemd[1]: Mounted /sysroot.1995worker2 # [ 3.424676] systemd[1]: Reached target Initrd Root Device.1996client # [ 3.476557] systemd[1]: Reached target Initrd Root File System.1997worker2 # [ 3.428884] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1998client # [ 3.480494] systemd[1]: Starting Mountpoints Configured in the Real Root...1999lb # [ 3.460638] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks2000worker1 # [ 3.447628] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks
loading 2,278 hidden lines…
4279worker1 # [ 48.523223] nix-grpc-daemon[865]: time=2026-09-23T05:12:43.026Z level=INFO msg="Upload complete. (316ms)"4280lb # [ 48.543686] niks3-server[970]: 2026/09/23 05:12:43 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4281lb # [ 48.549230] niks3-server[970]: 2026/09/23 05:12:43 INFO Completed upload id=84282worker1 # [ 48.530733] nix-grpc-daemon[767]: ts=2026-09-23T05:12:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14283lb # [ 48.551985] niks3-server[970]: 2026/09/23 05:12:43 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4284worker1 # [ 48.543602] nix-daemon[850]: reaped child process 1004, status = succeeded4285client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.06 seconds)4286client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4287client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.15 seconds)4288lb: must succeed: curl -sf localhost:9901/clusters4289lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4290lb: must succeed: curl -sf localhost:9901/clusters4291lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4292worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4293worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4294(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.58 seconds)4295subtest: outputs read back through any worker carry the cache signature4296client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24297worker1 # [ 49.121552] nix-grpc-daemon[767]: ts=2026-09-23T05:12:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467464298client # don't know how to build these paths:4299client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4300worker1 # [ 49.133359] nix-grpc-daemon[767]: ts=2026-09-23T05:12:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464301worker1 # [ 49.161104] nix-grpc-daemon[767]: ts=2026-09-23T05:12:43Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04302worker1 # [ 49.167733] nix-grpc-daemon[767]: ts=2026-09-23T05:12:43Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14303worker2 # [ 49.182005] nix-grpc-daemon[773]: ts=2026-09-23T05:12:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644304worker2 # [ 49.199817] nix-grpc-daemon[773]: ts=2026-09-23T05:12:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590644305client # copying 1 paths to 'lb:50051'...4306client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4307worker2 # [ 49.217968] nix-grpc-daemon[773]: ts=2026-09-23T05:12:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590644308worker2 # [ 49.234918] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4309worker2 # [ 49.268169] nix-grpc-daemon[773]: ts=2026-09-23T05:12:43Z level=info event=niks3_push_started pid=8704310worker2 # [ 49.449253] nix-grpc-daemon[870]: time=2026-09-23T05:12:43.946Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4311worker2 # [ 49.563531] nix-daemon[842]: accepted connection from pid 883, user nix-grpc-daemon (trusted)4312worker2 # [ 49.583057] nix-daemon[842]: reaped child process 886, status = succeeded4313lb # [ 49.603560] niks3-server[970]: 2026/09/23 05:12:44 INFO Received uploads request method=POST path=/api/pending_closures4314worker2 # [ 49.605978] nix-grpc-daemon[870]: time=2026-09-23T05:12:44.104Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4315worker2 # [ 49.607645] nix-grpc-daemon[870]: time=2026-09-23T05:12:44.104Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4316lb # [ 49.647500] niks3-server[970]: 2026/09/23 05:12:44 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4317worker2 # [ 49.648272] nix-grpc-daemon[870]: time=2026-09-23T05:12:44.147Z level=INFO msg="Uploading 1 narinfos"4318lb # [ 49.661364] niks3-server[970]: 2026/09/23 05:12:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4319lb # [ 49.672863] niks3-server[970]: 2026/09/23 05:12:44 INFO Signed narinfos id=9 count=14320lb # [ 49.675642] niks3-server[970]: 2026/09/23 05:12:44 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4321lb # [ 49.687016] niks3-server[970]: 2026/09/23 05:12:44 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4322lb # [ 49.698671] niks3-server[970]: 2026/09/23 05:12:44 INFO Completed upload id=94323worker2 # [ 49.687690] nix-grpc-daemon[870]: time=2026-09-23T05:12:44.186Z level=INFO msg="Upload complete. (239ms)"4324lb # [ 49.704982] niks3-server[970]: 2026/09/23 05:12:44 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4325worker2 # [ 49.700515] nix-grpc-daemon[773]: ts=2026-09-23T05:12:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=5124326worker2 # [ 49.716865] nix-grpc-daemon[773]: ts=2026-09-23T05:12:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644327worker2 # [ 49.723285] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4328worker2 # [ 49.729133] nix-daemon[842]: reaped child process 868, status = succeeded4329client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4330client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4331worker2 # [ 49.743370] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4332lb # [ 50.274963] envoy[908]: conn peer=[2001:db8:1::1]:46506 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9523 tx=4411 ms=63934333lb # [ 50.280271] envoy[908]: conn peer=[2001:db8:1::1]:36144 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=503 ms=904334lb # [ 50.285089] envoy[908]: conn peer=[2001:db8:1::1]:36148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1916 ms=18154335client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4336client # farm-a> LOG-farm-a-sig4337worker2 # [ 50.789094] nix-daemon[842]: reaped child process 890, status = succeeded4338worker2 # [ 50.900129] nix-daemon[842]: accepted connection from pid 906, user nix-grpc-daemon (trusted)4339worker2 # [ 50.925826] nix-daemon[842]: reaped child process 909, status = succeeded4340lb # [ 50.933828] niks3-server[970]: 2026/09/23 05:12:45 INFO Received uploads request method=POST path=/api/pending_closures4341worker2 # [ 50.936628] nix-grpc-daemon[870]: time=2026-09-23T05:12:45.433Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4342worker2 # [ 50.938172] nix-grpc-daemon[870]: time=2026-09-23T05:12:45.433Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4343lb # [ 50.969622] niks3-server[970]: 2026/09/23 05:12:45 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4344worker2 # [ 50.988363] nix-grpc-daemon[870]: time=2026-09-23T05:12:45.487Z level=INFO msg="Uploading 1 narinfos"4345lb # [ 50.991304] niks3-server[970]: 2026/09/23 05:12:45 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4346lb # [ 51.006222] niks3-server[970]: 2026/09/23 05:12:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4347lb # [ 51.014718] niks3-server[970]: 2026/09/23 05:12:45 INFO Signed narinfos id=10 count=14348lb # [ 51.024258] niks3-server[970]: 2026/09/23 05:12:45 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4349lb # [ 51.027765] niks3-server[970]: 2026/09/23 05:12:45 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4350lb # [ 51.036946] niks3-server[970]: 2026/09/23 05:12:45 INFO Completed upload id=104351worker2 # [ 51.025080] nix-grpc-daemon[870]: time=2026-09-23T05:12:45.524Z level=INFO msg="Upload complete. (269ms)"4352lb # [ 51.039074] niks3-server[970]: 2026/09/23 05:12:45 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4353worker2 # [ 51.037453] nix-grpc-daemon[773]: ts=2026-09-23T05:12:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14354worker1 # [ 51.033418] nix-grpc-daemon[767]: ts=2026-09-23T05:12:45Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04355worker1 # [ 51.039866] nix-grpc-daemon[767]: ts=2026-09-23T05:12:45Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14356worker2 # [ 51.059695] nix-daemon[842]: reaped child process 888, status = succeeded4357worker2 # [ 51.095465] nix-grpc-daemon[773]: ts=2026-09-23T05:12:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644358worker2 # [ 51.111626] nix-grpc-daemon[773]: ts=2026-09-23T05:12:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590644359worker2 # [ 51.124720] nix-grpc-daemon[870]: time=2026-09-23T05:12:45.624Z level=INFO msg="All 1 paths already cached"4360client # copying 1 paths to 'lb:50051'...4361client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4362worker2 # [ 51.137905] nix-grpc-daemon[773]: ts=2026-09-23T05:12:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590644363worker2 # [ 51.154325] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4364worker2 # [ 51.317168] nix-daemon[842]: accepted connection from pid 913, user nix-grpc-daemon (trusted)4365worker2 # [ 51.343575] nix-daemon[842]: reaped child process 916, status = succeeded4366lb # [ 51.350951] niks3-server[970]: 2026/09/23 05:12:45 INFO Received uploads request method=POST path=/api/pending_closures4367worker2 # [ 51.351554] nix-grpc-daemon[870]: time=2026-09-23T05:12:45.850Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4368worker2 # [ 51.353668] nix-grpc-daemon[870]: time=2026-09-23T05:12:45.850Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4369lb # [ 51.387162] niks3-server[970]: 2026/09/23 05:12:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4370worker2 # [ 51.388766] nix-grpc-daemon[870]: time=2026-09-23T05:12:45.887Z level=INFO msg="Uploading 1 narinfos"4371lb # [ 51.405227] niks3-server[970]: 2026/09/23 05:12:45 INFO Signed narinfos id=11 count=14372lb # [ 51.407808] niks3-server[970]: 2026/09/23 05:12:45 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4373lb # [ 51.424597] niks3-server[970]: 2026/09/23 05:12:45 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4374lb # [ 51.433742] niks3-server[970]: 2026/09/23 05:12:45 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4375worker2 # [ 51.421346] nix-grpc-daemon[870]: time=2026-09-23T05:12:45.920Z level=INFO msg="Upload complete. (248ms)"4376lb # [ 51.440973] niks3-server[970]: 2026/09/23 05:12:45 INFO Completed upload id=114377lb # [ 51.443211] niks3-server[970]: 2026/09/23 05:12:45 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4378worker2 # [ 51.432610] nix-grpc-daemon[773]: ts=2026-09-23T05:12:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=6244379worker2 # [ 51.448921] nix-grpc-daemon[773]: ts=2026-09-23T05:12:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644380worker2 # [ 51.455780] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4381worker2 # [ 51.462448] nix-daemon[842]: reaped child process 911, status = succeeded4382client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4383client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4384worker2 # [ 51.478513] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4385client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4386client # farm-b> LOG-farm-b-sig4387worker2 # [ 52.114823] nix-daemon[842]: reaped child process 920, status = succeeded4388worker2 # [ 52.228248] nix-daemon[842]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4389worker2 # [ 52.248471] nix-daemon[842]: reaped child process 939, status = succeeded4390lb # [ 52.267090] niks3-server[970]: 2026/09/23 05:12:46 INFO Received uploads request method=POST path=/api/pending_closures4391worker2 # [ 52.268530] nix-grpc-daemon[870]: time=2026-09-23T05:12:46.766Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4392worker2 # [ 52.270205] nix-grpc-daemon[870]: time=2026-09-23T05:12:46.766Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4393lb # [ 52.306642] niks3-server[970]: 2026/09/23 05:12:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4394worker2 # [ 52.307637] nix-grpc-daemon[870]: time=2026-09-23T05:12:46.806Z level=INFO msg="Uploading 1 narinfos"4395lb # [ 52.322586] niks3-server[970]: 2026/09/23 05:12:46 INFO Signed narinfos id=12 count=14396lb # [ 52.336349] niks3-server[970]: 2026/09/23 05:12:46 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4397lb # [ 52.346697] niks3-server[970]: 2026/09/23 05:12:46 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4398lb # [ 52.354661] niks3-server[970]: 2026/09/23 05:12:46 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4399worker2 # [ 52.356888] nix-grpc-daemon[870]: time=2026-09-23T05:12:46.856Z level=INFO msg="Upload complete. (266ms)"4400lb # [ 52.368141] niks3-server[970]: 2026/09/23 05:12:46 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4401lb # [ 52.378084] niks3-server[970]: 2026/09/23 05:12:46 INFO Completed upload id=124402lb # [ 52.381141] niks3-server[970]: 2026/09/23 05:12:46 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4403worker2 # [ 52.369441] nix-grpc-daemon[773]: ts=2026-09-23T05:12:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14404worker1 # [ 52.372612] nix-grpc-daemon[767]: ts=2026-09-23T05:12:46Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04405worker1 # [ 52.372947] nix-grpc-daemon[767]: ts=2026-09-23T05:12:46Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14406worker2 # [ 52.392254] nix-daemon[842]: reaped child process 918, status = succeeded4407worker2 # [ 52.426684] nix-grpc-daemon[773]: ts=2026-09-23T05:12:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644408worker2 # [ 52.444474] nix-grpc-daemon[773]: ts=2026-09-23T05:12:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590644409worker2 # [ 52.460775] nix-grpc-daemon[870]: time=2026-09-23T05:12:46.960Z level=INFO msg="All 2 paths already cached"4410client # copying 1 paths to 'lb:50051'...4411client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4412worker2 # [ 52.473493] nix-grpc-daemon[773]: ts=2026-09-23T05:12:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590644413worker2 # [ 52.488923] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4414worker2 # [ 52.656415] nix-daemon[842]: accepted connection from pid 943, user nix-grpc-daemon (trusted)4415lb # [ 52.694075] niks3-server[970]: 2026/09/23 05:12:47 INFO Received uploads request method=POST path=/api/pending_closures4416worker2 # [ 52.687161] nix-daemon[842]: reaped child process 946, status = succeeded4417worker2 # [ 52.693345] nix-grpc-daemon[870]: time=2026-09-23T05:12:47.192Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4418worker2 # [ 52.695664] nix-grpc-daemon[870]: time=2026-09-23T05:12:47.192Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4419lb # [ 52.726060] niks3-server[970]: 2026/09/23 05:12:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4420worker2 # [ 52.726444] nix-grpc-daemon[870]: time=2026-09-23T05:12:47.225Z level=INFO msg="Uploading 1 narinfos"4421lb # [ 52.743019] niks3-server[970]: 2026/09/23 05:12:47 INFO Signed narinfos id=13 count=14422lb # [ 52.749856] niks3-server[970]: 2026/09/23 05:12:47 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4423lb # [ 52.758710] niks3-server[970]: 2026/09/23 05:12:47 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4424lb # [ 52.769020] niks3-server[970]: 2026/09/23 05:12:47 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4425worker2 # [ 52.764389] nix-grpc-daemon[870]: time=2026-09-23T05:12:47.263Z level=INFO msg="Upload complete. (256ms)"4426lb # [ 52.775937] niks3-server[970]: 2026/09/23 05:12:47 INFO Completed upload id=134427lb # [ 52.780729] niks3-server[970]: 2026/09/23 05:12:47 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4428worker2 # [ 52.772441] nix-grpc-daemon[773]: ts=2026-09-23T05:12:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=7524429worker2 # [ 52.791251] nix-grpc-daemon[773]: ts=2026-09-23T05:12:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644430worker2 # [ 52.798332] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4431worker2 # [ 52.803708] nix-daemon[842]: reaped child process 941, status = succeeded4432client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4433worker2 # [ 52.817708] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4434client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4435client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4436client # farm-top> LOG-farm-top-sig4437worker2 # [ 54.336633] nix-daemon[842]: reaped child process 950, status = succeeded4438worker2 # [ 54.414994] nix-daemon[842]: accepted connection from pid 966, user nix-grpc-daemon (trusted)4439worker2 # [ 54.439697] nix-daemon[842]: reaped child process 969, status = succeeded4440lb # [ 54.449656] niks3-server[970]: 2026/09/23 05:12:48 INFO Received uploads request method=POST path=/api/pending_closures4441worker2 # [ 54.450674] nix-grpc-daemon[870]: time=2026-09-23T05:12:48.948Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4442worker2 # [ 54.452418] nix-grpc-daemon[870]: time=2026-09-23T05:12:48.948Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4443lb # [ 54.489129] niks3-server[970]: 2026/09/23 05:12:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4444worker2 # [ 54.488818] nix-grpc-daemon[870]: time=2026-09-23T05:12:48.987Z level=INFO msg="Uploading 1 narinfos"4445lb # [ 54.501579] niks3-server[970]: 2026/09/23 05:12:49 INFO Signed narinfos id=14 count=14446lb # [ 54.511063] niks3-server[970]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4447lb # [ 54.517917] niks3-server[970]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4448lb # [ 54.521789] niks3-server[970]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4449lb # [ 54.526441] niks3-server[970]: 2026/09/23 05:12:49 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4450lb # [ 54.533054] niks3-server[970]: 2026/09/23 05:12:49 INFO Completed upload id=144451worker2 # [ 54.523186] nix-grpc-daemon[870]: time=2026-09-23T05:12:49.022Z level=INFO msg="Upload complete. (289ms)"4452lb # [ 54.536557] niks3-server[970]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4453worker2 # [ 54.536699] nix-grpc-daemon[773]: ts=2026-09-23T05:12:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14454worker2 # [ 54.556760] nix-daemon[842]: reaped child process 948, status = succeeded4455client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.68 seconds)4456client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4457client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.20 seconds)4458client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4459worker1 # [ 54.921916] nix-grpc-daemon[767]: ts=2026-09-23T05:12:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:349704460client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4461client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4462client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4463client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4464worker1 # [ 54.963902] nix-grpc-daemon[767]: ts=2026-09-23T05:12:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:349704465worker1 # [ 54.973643] nix-grpc-daemon[767]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4466worker1 # [ 55.002746] nix-grpc-daemon[767]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4467worker1 # [ 55.026073] nix-grpc-daemon[767]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4468client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.30 seconds)4469client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4470worker2 # [ 55.280246] nix-grpc-daemon[773]: ts=2026-09-23T05:12:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:591504471worker2 # [ 55.323818] nix-grpc-daemon[773]: ts=2026-09-23T05:12:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:591504472client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.31 seconds)4473client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4474client # finding garbage collector roots...4475client # removing stale temporary roots file "/nix/var/nix/temproots/788"4476client # removing stale temporary roots file "/nix/var/nix/temproots/846"4477client # removing stale temporary roots file "/nix/var/nix/temproots/867"4478client # removing stale temporary roots file "/nix/var/nix/temproots/882"4479client # removing stale temporary roots file "/nix/var/nix/temproots/838"4480client # removing stale temporary roots file "/nix/var/nix/temproots/826"4481client # removing stale temporary roots file "/nix/var/nix/temproots/859"4482client # removing stale temporary roots file "/nix/var/nix/temproots/804"4483client # deleting specified paths...4484client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4485client # deleting unused links...4486client # note: hard linking is currently saving 0.0 KiB4487worker2 # [ 55.757917] nix-grpc-daemon[773]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590644488worker2 # [ 55.768732] nix-grpc-daemon[773]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590644489worker2 # [ 55.782540] nix-grpc-daemon[870]: time=2026-09-23T05:12:50.280Z level=INFO msg="All 1 paths already cached"4490worker2 # [ 55.792649] nix-grpc-daemon[773]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:590644491worker1 # [ 55.787486] nix-grpc-daemon[767]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467464492client # copying 3 paths...4493worker2 # [ 55.810848] nix-grpc-daemon[773]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:590644494client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4495worker2 # [ 55.836634] nix-grpc-daemon[773]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:590644496worker2 # [ 55.844984] nix-grpc-daemon[773]: ts=2026-09-23T05:12:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_out=2324497worker2 # [ 55.855458] nix-grpc-daemon[773]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:590644498worker1 # [ 55.853017] nix-grpc-daemon[767]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:467464499worker2 # [ 55.861152] nix-grpc-daemon[773]: ts=2026-09-23T05:12:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_out=1764500worker1 # [ 55.861246] nix-grpc-daemon[767]: ts=2026-09-23T05:12:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=0 paths=1 nar_bytes_out=1284501client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4502client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4503client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.54 seconds)4504(finished: subtest: outputs read back through any worker carry the cache signature, in 7.03 seconds)4505subtest: two clients wanting the same drv build it once4506client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4507client # [ 56.033384] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4508client # Running as unit: dedup1.service; invocation ID: 00e76a387dee412d81d760288b05e1104509client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.06 seconds)4510client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4511worker1 # [ 56.166054] nix-grpc-daemon[767]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467464512client # [ 56.231536] nix[935]: don't know how to build these paths:4513client # [ 56.233899] nix[935]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4514worker1 # [ 56.187009] nix-grpc-daemon[767]: ts=2026-09-23T05:12:50Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04515worker1 # [ 56.194926] nix-grpc-daemon[767]: ts=2026-09-23T05:12:50Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14516worker1 # [ 56.233943] nix-grpc-daemon[767]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464517worker1 # [ 56.251194] nix-grpc-daemon[767]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464518client # [ 56.324270] nix[935]: copying 1 paths to 'lb:50051'...4519client # [ 56.330187] nix[935]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4520worker1 # [ 56.271683] nix-grpc-daemon[767]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467464521worker1 # [ 56.288581] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4522worker1 # [ 56.465196] nix-daemon[850]: accepted connection from pid 1045, user nix-grpc-daemon (trusted)4523worker1 # [ 56.484861] nix-daemon[850]: reaped child process 1048, status = succeeded4524lb # [ 56.512806] niks3-server[970]: 2026/09/23 05:12:51 INFO Received uploads request method=POST path=/api/pending_closures4525worker1 # [ 56.504951] nix-grpc-daemon[865]: time=2026-09-23T05:12:51.007Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4526worker1 # [ 56.506487] nix-grpc-daemon[865]: time=2026-09-23T05:12:51.007Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4527lb # [ 56.540797] niks3-server[970]: 2026/09/23 05:12:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4528lb # [ 56.546686] niks3-server[970]: 2026/09/23 05:12:51 INFO Signed narinfos id=15 count=14529worker1 # [ 56.527654] nix-grpc-daemon[865]: time=2026-09-23T05:12:51.030Z level=INFO msg="Uploading 1 narinfos"4530lb # [ 56.557941] niks3-server[970]: 2026/09/23 05:12:51 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4531lb # [ 56.566037] niks3-server[970]: 2026/09/23 05:12:51 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4532lb # [ 56.574764] niks3-server[970]: 2026/09/23 05:12:51 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4533worker1 # [ 56.561204] nix-grpc-daemon[865]: time=2026-09-23T05:12:51.064Z level=INFO msg="Upload complete. (253ms)"4534lb # [ 56.581838] niks3-server[970]: 2026/09/23 05:12:51 INFO Completed upload id=154535lb # [ 56.586035] niks3-server[970]: 2026/09/23 05:12:51 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4536worker1 # [ 56.568192] nix-grpc-daemon[767]: ts=2026-09-23T05:12:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=0 paths=1 nar_bytes_in=4324537worker1 # [ 56.584207] nix-grpc-daemon[767]: ts=2026-09-23T05:12:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464538worker1 # [ 56.591916] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4539worker1 # [ 56.597916] nix-daemon[850]: reaped child process 1043, status = succeeded4540client # [ 56.668967] nix[935]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4541worker1 # [ 56.615341] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4542client # [ 57.109273] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4543client # Running as unit: dedup2.service; invocation ID: 731bafb446804ed5aa6f9b2b505830c24544client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.08 seconds)4545worker2 # [ 57.267658] nix-grpc-daemon[773]: ts=2026-09-23T05:12:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590644546client # [ 57.324524] nix[951]: don't know how to build these paths:4547client # [ 57.325529] nix[951]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4548worker1 # [ 57.271968] nix-grpc-daemon[767]: ts=2026-09-23T05:12:51Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04549worker1 # [ 57.278688] nix-grpc-daemon[767]: ts=2026-09-23T05:12:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464550client # [ 57.340826] nix[951]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4551client: must succeed: sleep 24552lb # [ 60.274635] envoy[908]: conn peer=[2001:db8:1::1]:36156 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9515 tx=4377 ms=54174553lb # [ 60.279756] envoy[908]: conn peer=[2001:db8:1::1]:34304 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=834554lb # [ 60.284724] envoy[908]: conn peer=[2001:db8:1::1]:34306 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=714555lb # [ 60.290800] envoy[908]: conn peer=[2001:db8:1::1]:34294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2913 ms=1644556lb # [ 60.295847] envoy[908]: conn peer=[2001:db8:1::1]:34310 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=584557client: (finished: must succeed: sleep 2, in 2.03 seconds)4558client: waiting for success: ! systemctl is-active dedup1 dedup24559worker1 # [ 60.726455] nix-daemon[850]: reaped child process 1052, status = succeeded4560worker1 # [ 60.801919] nix-daemon[850]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)4561worker1 # [ 60.824391] nix-daemon[850]: reaped child process 1087, status = succeeded4562lb # [ 60.848838] niks3-server[970]: 2026/09/23 05:12:55 INFO Received uploads request method=POST path=/api/pending_closures4563worker1 # [ 60.844492] nix-grpc-daemon[865]: time=2026-09-23T05:12:55.344Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4564worker1 # [ 60.846170] nix-grpc-daemon[865]: time=2026-09-23T05:12:55.344Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4565lb # [ 60.903614] niks3-server[970]: 2026/09/23 05:12:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4566worker1 # [ 60.891094] nix-grpc-daemon[865]: time=2026-09-23T05:12:55.394Z level=INFO msg="Uploading 1 narinfos"4567lb # [ 60.922698] niks3-server[970]: 2026/09/23 05:12:55 INFO Signed narinfos id=16 count=14568lb # [ 60.929713] niks3-server[970]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4569lb # [ 60.937147] niks3-server[970]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4570lb # [ 60.945897] niks3-server[970]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4571lb # [ 60.956736] niks3-server[970]: 2026/09/23 05:12:55 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4572worker1 # [ 60.937884] nix-grpc-daemon[865]: time=2026-09-23T05:12:55.441Z level=INFO msg="Upload complete. (308ms)"4573lb # [ 60.966569] niks3-server[970]: 2026/09/23 05:12:55 INFO Completed upload id=164574lb # [ 60.969888] niks3-server[970]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4575worker1 # [ 60.953308] nix-grpc-daemon[767]: ts=2026-09-23T05:12:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14576worker1 # [ 60.973288] nix-grpc-daemon[767]: ts=2026-09-23T05:12:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14577worker1 # [ 60.984428] nix-daemon[850]: reaped child process 1050, status = succeeded4578client # [ 61.084462] systemd[1]: dedup1.service: Deactivated successfully.4579client # [ 61.088385] systemd[1]: dedup1.service: Consumed 179ms CPU time over 5.054s wall clock time, 41.5M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.4580client # [ 61.096559] systemd[1]: dedup2.service: Deactivated successfully.4581client # [ 61.098138] systemd[1]: dedup2.service: Consumed 204ms CPU time over 3.984s wall clock time, 37.2M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4582client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4583client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4584client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4585worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4586worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4587worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4588worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4589worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4590worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4591worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4592worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4593(finished: subtest: two clients wanting the same drv build it once, in 6.13 seconds)4594subtest: --timeout and --max-silent-time reach the builder4595worker2 # [ 62.297792] nix-grpc-daemon[773]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590644596worker1 # [ 62.303034] nix-grpc-daemon[767]: ts=2026-09-23T05:12:56Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04597worker1 # [ 62.309464] nix-grpc-daemon[767]: ts=2026-09-23T05:12:56Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14598worker2 # [ 62.359338] nix-grpc-daemon[773]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644599worker2 # [ 62.377280] nix-grpc-daemon[773]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590644600worker2 # [ 62.393930] nix-grpc-daemon[773]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590644601worker2 # [ 62.410949] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4602worker2 # [ 62.581679] nix-daemon[842]: accepted connection from pid 994, user nix-grpc-daemon (trusted)4603worker2 # [ 62.605231] nix-daemon[842]: reaped child process 997, status = succeeded4604lb # [ 62.613084] niks3-server[970]: 2026/09/23 05:12:57 INFO Received uploads request method=POST path=/api/pending_closures4605worker2 # [ 62.613412] nix-grpc-daemon[870]: time=2026-09-23T05:12:57.112Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4606worker2 # [ 62.614907] nix-grpc-daemon[870]: time=2026-09-23T05:12:57.112Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4607lb # [ 62.651092] niks3-server[970]: 2026/09/23 05:12:57 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4608lb # [ 62.663865] niks3-server[970]: 2026/09/23 05:12:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4609worker2 # [ 62.659493] nix-grpc-daemon[870]: time=2026-09-23T05:12:57.158Z level=INFO msg="Uploading 1 narinfos"4610lb # [ 62.675798] niks3-server[970]: 2026/09/23 05:12:57 INFO Signed narinfos id=17 count=14611lb # [ 62.683357] niks3-server[970]: 2026/09/23 05:12:57 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4612lb # [ 62.701115] niks3-server[970]: 2026/09/23 05:12:57 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4613lb # [ 62.709294] niks3-server[970]: 2026/09/23 05:12:57 INFO Completed upload id=174614lb # [ 62.711158] niks3-server[970]: 2026/09/23 05:12:57 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4615worker2 # [ 62.698198] nix-grpc-daemon[870]: time=2026-09-23T05:12:57.197Z level=INFO msg="Upload complete. (267ms)"4616worker2 # [ 62.711428] nix-grpc-daemon[773]: ts=2026-09-23T05:12:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=4244617worker2 # [ 62.730921] nix-grpc-daemon[773]: ts=2026-09-23T05:12:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644618worker2 # [ 62.737851] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4619worker2 # [ 62.743273] nix-daemon[842]: reaped child process 992, status = succeeded4620worker2 # [ 62.759183] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4621worker2 # [ 68.575166] nix-grpc-daemon[773]: ts=2026-09-23T05:13:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04622worker2 # [ 68.595014] nix-daemon[842]: reaped child process 999, status = succeeded4623worker2 # [ 68.596447] nix-daemon[842]: reaped child process 1001, status = succeeded4624worker1 # [ 68.857502] nix-grpc-daemon[767]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467464625worker1 # [ 68.874875] nix-grpc-daemon[767]: ts=2026-09-23T05:13:03Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04626worker1 # [ 68.882237] nix-grpc-daemon[767]: ts=2026-09-23T05:13:03Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14627worker1 # [ 68.892318] nix-grpc-daemon[767]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464628worker1 # [ 68.903885] nix-grpc-daemon[767]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464629worker1 # [ 68.919263] nix-grpc-daemon[767]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467464630worker1 # [ 68.935113] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4631worker1 # [ 69.091404] nix-daemon[850]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4632worker1 # [ 69.115049] nix-daemon[850]: reaped child process 1107, status = succeeded4633lb # [ 69.137918] niks3-server[970]: 2026/09/23 05:13:03 INFO Received uploads request method=POST path=/api/pending_closures4634worker1 # [ 69.129457] nix-grpc-daemon[865]: time=2026-09-23T05:13:03.632Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4635worker1 # [ 69.130958] nix-grpc-daemon[865]: time=2026-09-23T05:13:03.632Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4636lb # [ 69.167532] niks3-server[970]: 2026/09/23 05:13:03 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4637lb # [ 69.190152] niks3-server[970]: 2026/09/23 05:13:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4638worker1 # [ 69.178221] nix-grpc-daemon[865]: time=2026-09-23T05:13:03.681Z level=INFO msg="Uploading 1 narinfos"4639lb # [ 69.200903] niks3-server[970]: 2026/09/23 05:13:03 INFO Signed narinfos id=18 count=14640lb # [ 69.210342] niks3-server[970]: 2026/09/23 05:13:03 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4641lb # [ 69.224556] niks3-server[970]: 2026/09/23 05:13:03 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4642lb # [ 69.233006] niks3-server[970]: 2026/09/23 05:13:03 INFO Completed upload id=184643worker1 # [ 69.213241] nix-grpc-daemon[865]: time=2026-09-23T05:13:03.716Z level=INFO msg="Upload complete. (261ms)"4644lb # [ 69.237818] niks3-server[970]: 2026/09/23 05:13:03 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4645worker1 # [ 69.224358] nix-grpc-daemon[767]: ts=2026-09-23T05:13:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=0 paths=1 nar_bytes_in=4244646worker1 # [ 69.242989] nix-grpc-daemon[767]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464647worker1 # [ 69.256405] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4648worker1 # [ 69.267180] nix-daemon[850]: reaped child process 1102, status = succeeded4649worker1 # [ 69.277753] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4650lb # [ 70.276294] envoy[908]: conn peer=[2001:db8:1::1]:34318 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1536 ms=48084651lb # [ 70.276691] envoy[908]: conn peer=[2001:db8:1::1]:34326 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=37384652lb # [ 70.276827] envoy[908]: conn peer=[2001:db8:1::1]:47390 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3001 tx=954 ms=63454653worker1 # [ 75.294241] nix-grpc-daemon[767]: ts=2026-09-23T05:13:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04654worker1 # [ 75.313927] nix-daemon[850]: reaped child process 1109, status = succeeded4655worker1 # [ 75.315056] nix-daemon[850]: reaped child process 1111, status = succeeded4656(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.46 seconds)4657subtest: the first client leaving does not fail the second4658client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4659client # [ 75.646584] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4660client # Running as unit: share1.service; invocation ID: 2245124843634344a28df86e4a733d174661client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4662client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4663client # [ 75.861147] nix[1025]: don't know how to build these paths:4664client # [ 75.862305] nix[1025]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4665worker2 # [ 75.810750] nix-grpc-daemon[773]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590644666worker1 # [ 75.815553] nix-grpc-daemon[767]: ts=2026-09-23T05:13:10Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04667worker2 # [ 75.828680] nix-grpc-daemon[773]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644668worker1 # [ 75.822075] nix-grpc-daemon[767]: ts=2026-09-23T05:13:10Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14669worker2 # [ 75.846674] nix-grpc-daemon[773]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590644670client # [ 75.909274] nix[1025]: copying 1 paths to 'lb:50051'...4671client # [ 75.913093] nix[1025]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4672worker2 # [ 75.864118] nix-grpc-daemon[773]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590644673worker2 # [ 75.879331] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4674worker2 # [ 76.041820] nix-daemon[842]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4675worker2 # [ 76.065747] nix-daemon[842]: reaped child process 1027, status = succeeded4676lb # [ 76.075289] niks3-server[970]: 2026/09/23 05:13:10 INFO Received uploads request method=POST path=/api/pending_closures4677worker2 # [ 76.076451] nix-grpc-daemon[870]: time=2026-09-23T05:13:10.575Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4678worker2 # [ 76.077979] nix-grpc-daemon[870]: time=2026-09-23T05:13:10.575Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4679lb # [ 76.112352] niks3-server[970]: 2026/09/23 05:13:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4680worker2 # [ 76.110977] nix-grpc-daemon[870]: time=2026-09-23T05:13:10.610Z level=INFO msg="Uploading 1 narinfos"4681lb # [ 76.128219] niks3-server[970]: 2026/09/23 05:13:10 INFO Signed narinfos id=19 count=14682lb # [ 76.134713] niks3-server[970]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4683lb # [ 76.146703] niks3-server[970]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4684lb # [ 76.164350] niks3-server[970]: 2026/09/23 05:13:10 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4685lb # [ 76.172210] niks3-server[970]: 2026/09/23 05:13:10 INFO Completed upload id=194686worker2 # [ 76.160819] nix-grpc-daemon[870]: time=2026-09-23T05:13:10.660Z level=INFO msg="Upload complete. (262ms)"4687lb # [ 76.181741] niks3-server[970]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4688worker2 # [ 76.171952] nix-grpc-daemon[773]: ts=2026-09-23T05:13:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=4324689worker2 # [ 76.186792] nix-grpc-daemon[773]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644690worker2 # [ 76.198375] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4691worker2 # [ 76.206189] nix-daemon[842]: reaped child process 1022, status = succeeded4692client # [ 76.266556] nix[1025]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4693worker2 # [ 76.219679] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4694client # [ 76.755792] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4695client # Running as unit: share2.service; invocation ID: 10fc821103c3486dbb5551a0698eba0f4696client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds)4697worker2 # [ 76.937812] nix-grpc-daemon[773]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590644698client # [ 77.003265] nix[1041]: this derivation will be built:4699worker2 # [ 76.955440] nix-grpc-daemon[773]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:590644700client # [ 77.013621] nix[1041]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4701worker1 # [ 76.965081] nix-grpc-daemon[767]: ts=2026-09-23T05:13:11Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04702worker2 # [ 76.976909] nix-grpc-daemon[773]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644703client # [ 77.033525] nix[1041]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4704client: must succeed: sleep 2; systemctl kill -s INT share14705client # [ 80.143765] systemd[1]: share1.service: Sent signal SIGINT to main process 1025 (nix) on client request.4706client # [ 80.153235] nix[1025]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4707client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4708client: waiting for success: ! systemctl is-active share14709client # [ 80.167441] nix[1025]: error: interrupted by the user4710client # [ 80.224186] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4711client # [ 80.226304] systemd[1]: share1.service: Failed with result 'exit-code'.4712client # [ 80.229023] systemd[1]: share1.service: Consumed 181ms CPU time over 4.582s wall clock time, 41.4M memory peak, 6.3K incoming IP traffic, 8.2K outgoing IP traffic.4713lb # [ 80.275976] envoy[908]: conn peer=[2001:db8:1::1]:47396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3028 tx=952 ms=65004714lb # [ 80.276488] envoy[908]: conn peer=[2001:db8:1::1]:35120 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2990 tx=814 ms=43154715client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds)4716client: waiting for success: ! systemctl is-active share24717worker2 # [ 81.690266] nix-daemon[842]: reaped child process 1031, status = succeeded4718worker2 # [ 81.821533] nix-daemon[842]: accepted connection from pid 1065, user nix-grpc-daemon (trusted)4719worker2 # [ 81.843687] nix-daemon[842]: reaped child process 1068, status = succeeded4720lb # [ 81.853799] niks3-server[970]: 2026/09/23 05:13:16 INFO Received uploads request method=POST path=/api/pending_closures4721worker2 # [ 81.856434] nix-grpc-daemon[870]: time=2026-09-23T05:13:16.355Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4722worker2 # [ 81.858094] nix-grpc-daemon[870]: time=2026-09-23T05:13:16.355Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4723lb # [ 81.905457] niks3-server[970]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4724worker2 # [ 81.905825] nix-grpc-daemon[870]: time=2026-09-23T05:13:16.404Z level=INFO msg="Uploading 1 narinfos"4725lb # [ 81.918075] niks3-server[970]: 2026/09/23 05:13:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4726lb # [ 81.925136] niks3-server[970]: 2026/09/23 05:13:16 INFO Signed narinfos id=20 count=14727lb # [ 81.932223] niks3-server[970]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4728lb # [ 81.935424] niks3-server[970]: 2026/09/23 05:13:16 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4729worker2 # [ 81.930512] nix-grpc-daemon[870]: time=2026-09-23T05:13:16.429Z level=INFO msg="Upload complete. (296ms)"4730lb # [ 81.941431] niks3-server[970]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4731lb # [ 81.948846] niks3-server[970]: 2026/09/23 05:13:16 INFO Completed upload id=204732lb # [ 81.951750] niks3-server[970]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4733worker2 # [ 81.943920] nix-grpc-daemon[773]: ts=2026-09-23T05:13:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14734worker2 # [ 81.960384] nix-grpc-daemon[773]: ts=2026-09-23T05:13:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14735worker2 # [ 81.966552] nix-daemon[842]: reaped child process 1029, status = succeeded4736client # [ 82.069200] systemd[1]: share2.service: Deactivated successfully.4737client # [ 82.073229] systemd[1]: share2.service: Consumed 204ms CPU time over 5.313s wall clock time, 35.3M memory peak, 6.5K incoming IP traffic, 5.9K outgoing IP traffic.4738client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4739client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4740client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds)4741(finished: subtest: the first client leaving does not fail the second, in 7.15 seconds)4742subtest: build hook: nix-daemon with builders = grpc://lb4743client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4744client # [ 82.862101] systemd[1]: Started Nix Daemon.4745client # [ 82.989540] nix-daemon[1093]: accepted connection from pid 1090, user root (trusted)4746worker1 # [ 83.087941] nix-grpc-daemon[767]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:467464747worker1 # [ 83.094636] nix-grpc-daemon[767]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464748worker1 # [ 83.114338] nix-grpc-daemon[767]: ts=2026-09-23T05:13:17Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04749worker1 # [ 83.121612] nix-grpc-daemon[767]: ts=2026-09-23T05:13:17Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14750worker1 # [ 83.131007] nix-grpc-daemon[767]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464751worker1 # [ 83.142560] nix-grpc-daemon[767]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464752worker1 # [ 83.158208] nix-grpc-daemon[767]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467464753worker1 # [ 83.173833] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4754worker1 # [ 83.326468] nix-daemon[850]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)4755worker1 # [ 83.349948] nix-daemon[850]: reaped child process 1159, status = succeeded4756lb # [ 83.373253] niks3-server[970]: 2026/09/23 05:13:17 INFO Received uploads request method=POST path=/api/pending_closures4757worker1 # [ 83.364375] nix-grpc-daemon[865]: time=2026-09-23T05:13:17.866Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4758worker1 # [ 83.365902] nix-grpc-daemon[865]: time=2026-09-23T05:13:17.867Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4759lb # [ 83.403444] niks3-server[970]: 2026/09/23 05:13:17 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4760worker1 # [ 83.396988] nix-grpc-daemon[865]: time=2026-09-23T05:13:17.900Z level=INFO msg="Uploading 1 narinfos"4761lb # [ 83.418330] niks3-server[970]: 2026/09/23 05:13:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4762lb # [ 83.431048] niks3-server[970]: 2026/09/23 05:13:17 INFO Signed narinfos id=21 count=14763lb # [ 83.438570] niks3-server[970]: 2026/09/23 05:13:17 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4764lb # [ 83.449398] niks3-server[970]: 2026/09/23 05:13:17 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4765lb # [ 83.459833] niks3-server[970]: 2026/09/23 05:13:17 INFO Completed upload id=214766worker1 # [ 83.439906] nix-grpc-daemon[865]: time=2026-09-23T05:13:17.943Z level=INFO msg="Upload complete. (249ms)"4767lb # [ 83.464033] niks3-server[970]: 2026/09/23 05:13:17 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4768worker1 # [ 83.452172] nix-grpc-daemon[767]: ts=2026-09-23T05:13:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=0 paths=1 nar_bytes_in=5124769worker1 # [ 83.468392] nix-grpc-daemon[767]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464770worker1 # [ 83.474996] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4771worker1 # [ 83.480821] nix-daemon[850]: reaped child process 1154, status = succeeded4772worker1 # [ 83.492459] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4773worker1 # [ 84.776828] nix-daemon[850]: reaped child process 1163, status = succeeded4774worker1 # [ 84.872003] nix-daemon[850]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)4775worker1 # [ 84.896425] nix-daemon[850]: reaped child process 1182, status = succeeded4776lb # [ 84.913938] niks3-server[970]: 2026/09/23 05:13:19 INFO Received uploads request method=POST path=/api/pending_closures4777worker1 # [ 84.904921] nix-grpc-daemon[865]: time=2026-09-23T05:13:19.407Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4778worker1 # [ 84.906475] nix-grpc-daemon[865]: time=2026-09-23T05:13:19.407Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4779lb # [ 84.962593] niks3-server[970]: 2026/09/23 05:13:19 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4780worker1 # [ 84.953893] nix-grpc-daemon[865]: time=2026-09-23T05:13:19.456Z level=INFO msg="Uploading 1 narinfos"4781lb # [ 84.975788] niks3-server[970]: 2026/09/23 05:13:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4782lb # [ 84.985611] niks3-server[970]: 2026/09/23 05:13:19 INFO Signed narinfos id=22 count=14783lb # [ 84.991252] niks3-server[970]: 2026/09/23 05:13:19 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4784worker1 # [ 84.982534] nix-grpc-daemon[865]: time=2026-09-23T05:13:19.485Z level=INFO msg="Upload complete. (276ms)"4785lb # [ 85.003176] niks3-server[970]: 2026/09/23 05:13:19 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4786lb # [ 85.010056] niks3-server[970]: 2026/09/23 05:13:19 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4787lb # [ 85.014053] niks3-server[970]: 2026/09/23 05:13:19 INFO Completed upload id=224788worker1 # [ 84.993111] nix-grpc-daemon[767]: ts=2026-09-23T05:13:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14789lb # [ 85.016833] niks3-server[970]: 2026/09/23 05:13:19 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4790worker1 # [ 85.010427] nix-daemon[850]: reaped child process 1161, status = succeeded4791worker2 # [ 85.050178] nix-grpc-daemon[773]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:590644792worker2 # [ 85.059239] nix-grpc-daemon[773]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4793worker2 # [ 85.101734] nix-grpc-daemon[773]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:590644794worker2 # [ 85.109005] nix-grpc-daemon[773]: ts=2026-09-23T05:13:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_out=1284795worker2 # [ 85.393694] nix-grpc-daemon[773]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:590644796worker1 # [ 85.389615] nix-grpc-daemon[767]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464797worker1 # [ 85.400673] nix-grpc-daemon[865]: time=2026-09-23T05:13:19.903Z level=INFO msg="All 1 paths already cached"4798worker1 # [ 85.419915] nix-grpc-daemon[767]: ts=2026-09-23T05:13:19Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04799worker1 # [ 85.426845] nix-grpc-daemon[767]: ts=2026-09-23T05:13:19Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14800worker1 # [ 85.435902] nix-grpc-daemon[767]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464801worker1 # [ 85.447865] nix-grpc-daemon[767]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464802worker1 # [ 85.462683] nix-grpc-daemon[865]: time=2026-09-23T05:13:19.965Z level=INFO msg="All 1 paths already cached"4803worker1 # [ 85.472815] nix-grpc-daemon[767]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467464804worker1 # [ 85.487078] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4805worker1 # [ 85.649161] nix-daemon[850]: accepted connection from pid 1188, user nix-grpc-daemon (trusted)4806worker1 # [ 85.672855] nix-daemon[850]: reaped child process 1191, status = succeeded4807lb # [ 85.690275] niks3-server[970]: 2026/09/23 05:13:20 INFO Received uploads request method=POST path=/api/pending_closures4808worker1 # [ 85.680987] nix-grpc-daemon[865]: time=2026-09-23T05:13:20.183Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4809worker1 # [ 85.683239] nix-grpc-daemon[865]: time=2026-09-23T05:13:20.183Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4810lb # [ 85.727579] niks3-server[970]: 2026/09/23 05:13:20 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4811worker1 # [ 85.716463] nix-grpc-daemon[865]: time=2026-09-23T05:13:20.219Z level=INFO msg="Uploading 1 narinfos"4812lb # [ 85.743216] niks3-server[970]: 2026/09/23 05:13:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4813lb # [ 85.755791] niks3-server[970]: 2026/09/23 05:13:20 INFO Signed narinfos id=23 count=14814lb # [ 85.762087] niks3-server[970]: 2026/09/23 05:13:20 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4815worker1 # [ 85.742353] nix-grpc-daemon[865]: time=2026-09-23T05:13:20.245Z level=INFO msg="Upload complete. (238ms)"4816lb # [ 85.772396] niks3-server[970]: 2026/09/23 05:13:20 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4817worker1 # [ 85.753599] nix-grpc-daemon[767]: ts=2026-09-23T05:13:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=0 paths=1 nar_bytes_in=6244818lb # [ 85.777211] niks3-server[970]: 2026/09/23 05:13:20 INFO Completed upload id=234819lb # [ 85.779609] niks3-server[970]: 2026/09/23 05:13:20 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4820worker1 # [ 85.768666] nix-grpc-daemon[767]: ts=2026-09-23T05:13:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464821worker1 # [ 85.775309] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4822worker1 # [ 85.781519] nix-daemon[850]: reaped child process 1186, status = succeeded4823worker1 # [ 85.794961] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4824worker1 # [ 87.233352] nix-daemon[850]: reaped child process 1195, status = succeeded4825worker1 # [ 87.319657] nix-daemon[850]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)4826worker1 # [ 87.339232] nix-daemon[850]: reaped child process 1214, status = succeeded4827lb # [ 87.365700] niks3-server[970]: 2026/09/23 05:13:21 INFO Received uploads request method=POST path=/api/pending_closures4828worker1 # [ 87.356414] nix-grpc-daemon[865]: time=2026-09-23T05:13:21.859Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4829worker1 # [ 87.357895] nix-grpc-daemon[865]: time=2026-09-23T05:13:21.859Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4830lb # [ 87.406696] niks3-server[970]: 2026/09/23 05:13:21 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4831worker1 # [ 87.404261] nix-grpc-daemon[865]: time=2026-09-23T05:13:21.907Z level=INFO msg="Uploading 1 narinfos"4832lb # [ 87.423283] niks3-server[970]: 2026/09/23 05:13:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4833lb # [ 87.432376] niks3-server[970]: 2026/09/23 05:13:21 INFO Signed narinfos id=24 count=14834lb # [ 87.440576] niks3-server[970]: 2026/09/23 05:13:21 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4835lb # [ 87.448411] niks3-server[970]: 2026/09/23 05:13:21 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4836worker1 # [ 87.442278] nix-grpc-daemon[865]: time=2026-09-23T05:13:21.945Z level=INFO msg="Upload complete. (237ms)"4837lb # [ 87.461500] niks3-server[970]: 2026/09/23 05:13:21 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4838lb # [ 87.465589] niks3-server[970]: 2026/09/23 05:13:21 INFO Completed upload id=244839lb # [ 87.470684] niks3-server[970]: 2026/09/23 05:13:21 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4840worker1 # [ 87.457174] nix-grpc-daemon[767]: ts=2026-09-23T05:13:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14841worker2 # [ 87.470835] nix-grpc-daemon[773]: ts=2026-09-23T05:13:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:590644842worker1 # [ 87.473081] nix-daemon[850]: reaped child process 1193, status = succeeded4843worker2 # [ 87.483398] nix-grpc-daemon[773]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4844worker1 # [ 87.513224] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:467464845worker1 # [ 87.520619] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=0 paths=1 nar_bytes_out=1764846worker1 # [ 87.757880] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:467464847worker1 # [ 87.765549] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464848worker1 # [ 87.773382] nix-grpc-daemon[865]: time=2026-09-23T05:13:22.274Z level=INFO msg="All 2 paths already cached"4849worker1 # [ 87.786477] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04850worker1 # [ 87.793865] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14851worker1 # [ 87.833878] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464852worker1 # [ 87.851016] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464853worker1 # [ 87.866192] nix-grpc-daemon[865]: time=2026-09-23T05:13:22.369Z level=INFO msg="All 2 paths already cached"4854worker1 # [ 87.876953] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467464855worker1 # [ 87.891174] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4856worker1 # [ 88.045765] nix-daemon[850]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)4857worker1 # [ 88.067719] nix-daemon[850]: reaped child process 1223, status = succeeded4858lb # [ 88.084060] niks3-server[970]: 2026/09/23 05:13:22 INFO Received uploads request method=POST path=/api/pending_closures4859worker1 # [ 88.075198] nix-grpc-daemon[865]: time=2026-09-23T05:13:22.578Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4860worker1 # [ 88.076944] nix-grpc-daemon[865]: time=2026-09-23T05:13:22.578Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4861lb # [ 88.117094] niks3-server[970]: 2026/09/23 05:13:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4862worker1 # [ 88.104138] nix-grpc-daemon[865]: time=2026-09-23T05:13:22.607Z level=INFO msg="Uploading 1 narinfos"4863lb # [ 88.131613] niks3-server[970]: 2026/09/23 05:13:22 INFO Signed narinfos id=25 count=14864lb # [ 88.135812] niks3-server[970]: 2026/09/23 05:13:22 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4865worker1 # [ 88.132964] nix-grpc-daemon[865]: time=2026-09-23T05:13:22.636Z level=INFO msg="Upload complete. (224ms)"4866lb # [ 88.152620] niks3-server[970]: 2026/09/23 05:13:22 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4867worker1 # [ 88.140363] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=0 paths=1 nar_bytes_in=7524868lb # [ 88.159984] niks3-server[970]: 2026/09/23 05:13:22 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4869lb # [ 88.168252] niks3-server[970]: 2026/09/23 05:13:22 INFO Completed upload id=254870lb # [ 88.170739] niks3-server[970]: 2026/09/23 05:13:22 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4871worker1 # [ 88.156365] nix-grpc-daemon[767]: ts=2026-09-23T05:13:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467464872worker1 # [ 88.163009] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4873worker1 # [ 88.168189] nix-daemon[850]: reaped child process 1218, status = succeeded4874worker1 # [ 88.180419] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4875worker1 # [ 89.055864] nix-daemon[850]: reaped child process 1227, status = succeeded4876worker1 # [ 89.118826] nix-daemon[850]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)4877worker1 # [ 89.142505] nix-daemon[850]: reaped child process 1246, status = succeeded4878lb # [ 89.163784] niks3-server[970]: 2026/09/23 05:13:23 INFO Received uploads request method=POST path=/api/pending_closures4879worker1 # [ 89.244966] nix-grpc-daemon[865]: time=2026-09-23T05:13:23.745Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4880worker1 # [ 89.250773] nix-grpc-daemon[865]: time=2026-09-23T05:13:23.745Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4881lb # [ 89.302928] niks3-server[970]: 2026/09/23 05:13:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4882worker1 # [ 89.294309] nix-grpc-daemon[865]: time=2026-09-23T05:13:23.797Z level=INFO msg="Uploading 1 narinfos"4883lb # [ 89.319534] niks3-server[970]: 2026/09/23 05:13:23 INFO Signed narinfos id=26 count=14884lb # [ 89.328287] niks3-server[970]: 2026/09/23 05:13:23 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4885lb # [ 89.340986] niks3-server[970]: 2026/09/23 05:13:23 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4886worker1 # [ 89.330056] nix-grpc-daemon[865]: time=2026-09-23T05:13:23.833Z level=INFO msg="Upload complete. (337ms)"4887lb # [ 89.353523] niks3-server[970]: 2026/09/23 05:13:23 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4888lb # [ 89.359937] niks3-server[970]: 2026/09/23 05:13:23 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4889lb # [ 89.365481] niks3-server[970]: 2026/09/23 05:13:23 INFO Completed upload id=264890worker1 # [ 89.342184] nix-grpc-daemon[767]: ts=2026-09-23T05:13:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14891lb # [ 89.369073] niks3-server[970]: 2026/09/23 05:13:23 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4892worker2 # [ 89.369317] nix-grpc-daemon[773]: ts=2026-09-23T05:13:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:590644893worker1 # [ 89.364514] nix-daemon[850]: reaped child process 1225, status = succeeded4894worker2 # [ 89.383020] nix-grpc-daemon[773]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4895worker2 # [ 89.422238] nix-grpc-daemon[773]: ts=2026-09-23T05:13:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:590644896worker2 # [ 89.429433] nix-grpc-daemon[773]: ts=2026-09-23T05:13:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_out=2324897client # [ 89.602830] nix-daemon[1093]: reaped child process 1095, status = succeeded4898client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.90 seconds)4899client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4900client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4901client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4902client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4903client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4904client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4905(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.02 seconds)4906subtest: an input only one worker has still reaches the builder4907client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4908client # this derivation will be built:4909client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4910client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4911client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)4912client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4913client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4914worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4915lb # [ 90.276394] envoy[908]: conn peer=[2001:db8:1::1]:35130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1605 tx=1596 ms=50584916lb # [ 90.276695] envoy[908]: conn peer=[2001:db8:1::1]:35100 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3445 tx=1943 ms=20724917lb # [ 90.276797] envoy[908]: conn peer=[2001:db8:1::1]:35116 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=333 ms=704918lb # [ 90.276908] envoy[908]: conn peer=[2001:db8:1::1]:35132 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3914 tx=2156 ms=21534919lb # [ 90.277022] envoy[908]: conn peer=[2001:db8:1::1]:35136 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=274920lb # [ 90.277118] envoy[908]: conn peer=[2001:db8:1::1]:35146 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4352 tx=2412 ms=17204921lb # [ 90.277226] envoy[908]: conn peer=[2001:db8:1::1]:35160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=694922worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4923worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4924worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4925client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24926client # [ 90.583235] nix-daemon[1093]: accepted connection from pid 1176, user root (trusted)4927client # this derivation will be built:4928client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4929client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4930worker1 # [ 90.679124] nix-grpc-daemon[767]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:467464931worker1 # [ 90.687652] nix-grpc-daemon[767]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467464932worker1 # [ 90.851099] nix-daemon[850]: accepted connection from pid 1254, user nix-grpc-daemon (trusted)4933worker1 # [ 90.875301] nix-daemon[850]: reaped child process 1257, status = succeeded4934lb # [ 90.899963] niks3-server[970]: 2026/09/23 05:13:25 INFO Received uploads request method=POST path=/api/pending_closures4935worker1 # [ 90.894086] nix-grpc-daemon[865]: time=2026-09-23T05:13:25.396Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4936worker1 # [ 90.895878] nix-grpc-daemon[865]: time=2026-09-23T05:13:25.396Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4937lb # [ 90.932579] niks3-server[970]: 2026/09/23 05:13:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4938worker1 # [ 90.925206] nix-grpc-daemon[865]: time=2026-09-23T05:13:25.428Z level=INFO msg="Uploading 1 narinfos"4939lb # [ 90.948965] niks3-server[970]: 2026/09/23 05:13:25 INFO Signed narinfos id=27 count=14940lb # [ 90.956881] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4941lb # [ 90.969535] niks3-server[970]: 2026/09/23 05:13:25 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4942lb # [ 90.978490] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4943lb # [ 90.989960] niks3-server[970]: 2026/09/23 05:13:25 INFO Completed upload id=274944worker1 # [ 90.971328] nix-grpc-daemon[865]: time=2026-09-23T05:13:25.474Z level=INFO msg="Upload complete. (284ms)"4945lb # [ 91.002999] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4946worker1 # [ 90.999090] nix-grpc-daemon[767]: ts=2026-09-23T05:13:25Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04947worker1 # [ 91.005412] nix-grpc-daemon[767]: ts=2026-09-23T05:13:25Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14948worker2 # [ 91.056813] nix-grpc-daemon[773]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644949worker2 # [ 91.073006] nix-grpc-daemon[773]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590644950client # copying 2 paths to 'lb:50051'...4951client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4952worker2 # [ 91.093065] nix-grpc-daemon[773]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590644953client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4954worker2 # [ 91.108322] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4955worker2 # [ 91.279512] nix-daemon[842]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4956worker2 # [ 91.297745] nix-daemon[842]: reaped child process 1081, status = succeeded4957lb # [ 91.314691] niks3-server[970]: 2026/09/23 05:13:25 INFO Received uploads request method=POST path=/api/pending_closures4958lb # [ 91.324061] niks3-server[970]: 2026/09/23 05:13:25 INFO Received uploads request method=POST path=/api/pending_closures4959worker2 # [ 91.326830] nix-grpc-daemon[870]: time=2026-09-23T05:13:25.823Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4960worker2 # [ 91.329768] nix-grpc-daemon[870]: time=2026-09-23T05:13:25.824Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4961worker2 # [ 91.332503] nix-grpc-daemon[870]: time=2026-09-23T05:13:25.824Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4962lb # [ 91.372452] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4963lb # [ 91.386023] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4964worker2 # [ 91.388583] nix-grpc-daemon[870]: time=2026-09-23T05:13:25.887Z level=INFO msg="Uploading 3 narinfos"4965lb # [ 91.401392] niks3-server[970]: 2026/09/23 05:13:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4966lb # [ 91.409319] niks3-server[970]: 2026/09/23 05:13:25 INFO Signed narinfos id=28 count=14967lb # [ 91.414566] niks3-server[970]: 2026/09/23 05:13:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4968lb # [ 91.421998] niks3-server[970]: 2026/09/23 05:13:25 INFO Signed narinfos id=29 count=24969lb # [ 91.427471] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4970lb # [ 91.433130] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4971lb # [ 91.444194] niks3-server[970]: 2026/09/23 05:13:25 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4972lb # [ 91.447244] niks3-server[970]: 2026/09/23 05:13:25 INFO Completed upload id=284973lb # [ 91.453840] niks3-server[970]: 2026/09/23 05:13:25 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4974lb # [ 91.457079] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4975lb # [ 91.460918] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4976lb # [ 91.465073] niks3-server[970]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4977lb # [ 91.470618] niks3-server[970]: 2026/09/23 05:13:25 INFO Completed upload id=294978worker2 # [ 91.458210] nix-grpc-daemon[870]: time=2026-09-23T05:13:25.957Z level=INFO msg="Upload complete. (329ms)"4979worker2 # [ 91.468941] nix-grpc-daemon[773]: ts=2026-09-23T05:13:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=2 nar_bytes_in=9844980worker2 # [ 91.486892] nix-grpc-daemon[773]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590644981worker2 # [ 91.500370] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4982worker2 # [ 91.512641] nix-daemon[842]: reaped child process 1076, status = succeeded4983worker2 # [ 91.523319] nix-grpc-daemon[773]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4984client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4985worker2 # [ 91.539893] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)4986client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4987client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4988worker2 # [ 92.371549] nix-daemon[842]: reaped child process 1086, status = succeeded4989worker2 # [ 92.474257] nix-daemon[842]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)4990worker2 # [ 92.491308] nix-daemon[842]: reaped child process 1105, status = succeeded4991lb # [ 92.510018] niks3-server[970]: 2026/09/23 05:13:27 INFO Received uploads request method=POST path=/api/pending_closures4992worker2 # [ 92.511361] nix-grpc-daemon[870]: time=2026-09-23T05:13:27.009Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4993lb # [ 92.537853] niks3-server[970]: 2026/09/23 05:13:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign4994worker2 # [ 92.537068] nix-grpc-daemon[870]: time=2026-09-23T05:13:27.036Z level=INFO msg="Uploading 1 narinfos"4995lb # [ 92.549412] niks3-server[970]: 2026/09/23 05:13:27 INFO Signed narinfos id=30 count=14996lb # [ 92.558011] niks3-server[970]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4997lb # [ 92.568478] niks3-server[970]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4998worker2 # [ 92.569023] nix-grpc-daemon[870]: time=2026-09-23T05:13:27.068Z level=INFO msg="Upload complete. (208ms)"4999lb # [ 92.580969] niks3-server[970]: 2026/09/23 05:13:27 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5000lb # [ 92.586430] niks3-server[970]: 2026/09/23 05:13:27 INFO Completed upload id=305001lb # [ 92.589518] niks3-server[970]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5002worker2 # [ 92.580882] nix-grpc-daemon[773]: ts=2026-09-23T05:13:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15003worker2 # [ 92.592453] nix-daemon[842]: reaped child process 1083, status = succeeded5004client # copying 1 paths...5005worker2 # [ 92.627836] nix-grpc-daemon[773]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:590645006client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5007worker2 # [ 92.662536] nix-grpc-daemon[773]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:590645008worker2 # [ 92.669538] nix-grpc-daemon[773]: ts=2026-09-23T05:13:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_out=1205009client # [ 92.848503] nix-daemon[1093]: reaped child process 1179, status = succeeded5010client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.46 seconds)5011client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25012client # [ 93.017800] nix-daemon[1093]: accepted connection from pid 1197, user root (trusted)5013client # this derivation will be built:5014client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5015worker1 # [ 93.109260] nix-grpc-daemon[767]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:467465016client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5017worker1 # [ 93.119393] nix-grpc-daemon[767]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467465018worker1 # [ 93.131238] nix-grpc-daemon[865]: time=2026-09-23T05:13:27.633Z level=INFO msg="All 1 paths already cached"5019worker1 # [ 93.150804] nix-grpc-daemon[767]: ts=2026-09-23T05:13:27Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05020worker1 # [ 93.158875] nix-grpc-daemon[767]: ts=2026-09-23T05:13:27Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15021worker1 # [ 93.168202] nix-grpc-daemon[767]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467465022worker1 # [ 93.179396] nix-grpc-daemon[767]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467465023client # copying 1 paths to 'lb:50051'...5024client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5025worker1 # [ 93.199117] nix-grpc-daemon[767]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467465026worker1 # [ 93.214410] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5027worker1 # [ 93.234438] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)5028worker1 # [ 93.255673] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 2/5)5029worker1 # [ 93.320114] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 251 ms (attempt 3/5)5030worker1 # [ 93.573485] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 574 ms (attempt 4/5)5031worker1 # [ 94.158785] nix-grpc-daemon[767]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5032worker1 # [ 94.167960] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5033worker1 # [ 94.175570] nix-grpc-daemon[767]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5034worker1 # [ 94.343203] nix-daemon[850]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5035worker1 # [ 94.362932] nix-daemon[850]: reaped child process 1273, status = succeeded5036lb # [ 94.391368] niks3-server[970]: 2026/09/23 05:13:28 INFO Received uploads request method=POST path=/api/pending_closures5037worker1 # [ 94.383615] nix-grpc-daemon[865]: time=2026-09-23T05:13:28.886Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5038worker1 # [ 94.385959] nix-grpc-daemon[865]: time=2026-09-23T05:13:28.886Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5039lb # [ 94.425005] niks3-server[970]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5040worker1 # [ 94.419143] nix-grpc-daemon[865]: time=2026-09-23T05:13:28.922Z level=INFO msg="Uploading 1 narinfos"5041lb # [ 94.440480] niks3-server[970]: 2026/09/23 05:13:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5042lb # [ 94.452250] niks3-server[970]: 2026/09/23 05:13:28 INFO Signed narinfos id=31 count=15043lb # [ 94.457905] niks3-server[970]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5044worker1 # [ 94.448678] nix-grpc-daemon[865]: time=2026-09-23T05:13:28.951Z level=INFO msg="Upload complete. (259ms)"5045lb # [ 94.468955] niks3-server[970]: 2026/09/23 05:13:28 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5046lb # [ 94.474789] niks3-server[970]: 2026/09/23 05:13:28 INFO Completed upload id=315047lb # [ 94.478857] niks3-server[970]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5048worker1 # [ 94.460816] nix-grpc-daemon[767]: ts=2026-09-23T05:13:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=1 paths=1 nar_bytes_in=5685049worker1 # [ 94.475482] nix-grpc-daemon[767]: ts=2026-09-23T05:13:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467465050worker1 # [ 94.484008] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5051worker1 # [ 94.499165] nix-daemon[850]: reaped child process 1263, status = succeeded5052worker1 # [ 94.505424] nix-daemon[850]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5053client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5054client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5055client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5056worker1 # [ 95.972529] nix-daemon[850]: reaped child process 1277, status = succeeded5057worker1 # [ 96.104958] nix-daemon[850]: accepted connection from pid 1293, user nix-grpc-daemon (trusted)5058worker1 # [ 96.126546] nix-daemon[850]: reaped child process 1296, status = succeeded5059lb # [ 96.149680] niks3-server[970]: 2026/09/23 05:13:30 INFO Received uploads request method=POST path=/api/pending_closures5060worker1 # [ 96.142331] nix-grpc-daemon[865]: time=2026-09-23T05:13:30.643Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5061lb # [ 96.181967] niks3-server[970]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5062worker1 # [ 96.177063] nix-grpc-daemon[865]: time=2026-09-23T05:13:30.680Z level=INFO msg="Uploading 1 narinfos"5063lb # [ 96.198581] niks3-server[970]: 2026/09/23 05:13:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5064lb # [ 96.208622] niks3-server[970]: 2026/09/23 05:13:30 INFO Signed narinfos id=32 count=15065lb # [ 96.214131] niks3-server[970]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5066worker1 # [ 96.207579] nix-grpc-daemon[865]: time=2026-09-23T05:13:30.710Z level=INFO msg="Upload complete. (287ms)"5067lb # [ 96.226902] niks3-server[970]: 2026/09/23 05:13:30 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5068lb # [ 96.234552] niks3-server[970]: 2026/09/23 05:13:30 INFO Completed upload id=325069client # copying 1 paths...5070lb # [ 96.238141] niks3-server[970]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5071worker1 # [ 96.224643] nix-grpc-daemon[767]: ts=2026-09-23T05:13:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46746 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15072worker2 # [ 96.238053] nix-grpc-daemon[773]: ts=2026-09-23T05:13:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:590645073worker2 # [ 96.245879] nix-grpc-daemon[773]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5074worker1 # [ 96.240227] nix-daemon[850]: reaped child process 1275, status = succeeded5075client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5076worker2 # [ 96.288457] nix-grpc-daemon[773]: ts=2026-09-23T05:13:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:590645077worker2 # [ 96.295423] nix-grpc-daemon[773]: ts=2026-09-23T05:13:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_out=1205078client # [ 96.475813] nix-daemon[1093]: reaped child process 1200, status = succeeded5079client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.63 seconds)5080client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25081client # [ 96.644606] nix-daemon[1093]: accepted connection from pid 1218, user root (trusted)5082client # this derivation will be built:5083client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5084worker2 # [ 96.744431] nix-grpc-daemon[773]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:590645085client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5086worker2 # [ 96.752210] nix-grpc-daemon[773]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645087worker2 # [ 96.760110] nix-grpc-daemon[870]: time=2026-09-23T05:13:31.257Z level=INFO msg="All 1 paths already cached"5088worker1 # [ 96.761376] nix-grpc-daemon[767]: ts=2026-09-23T05:13:31Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05089worker1 # [ 96.768659] nix-grpc-daemon[767]: ts=2026-09-23T05:13:31Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15090worker2 # [ 96.822014] nix-grpc-daemon[773]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645091worker2 # [ 96.837802] nix-grpc-daemon[773]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590645092client # copying 1 paths to 'lb:50051'...5093worker2 # [ 96.852534] nix-grpc-daemon[870]: time=2026-09-23T05:13:31.351Z level=INFO msg="All 1 paths already cached"5094client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5095worker2 # [ 96.860142] nix-grpc-daemon[773]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590645096worker2 # [ 96.875680] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)5097worker2 # [ 97.035955] nix-daemon[842]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)5098worker2 # [ 97.058645] nix-daemon[842]: reaped child process 1113, status = succeeded5099lb # [ 97.071923] niks3-server[970]: 2026/09/23 05:13:31 INFO Received uploads request method=POST path=/api/pending_closures5100worker2 # [ 97.072776] nix-grpc-daemon[870]: time=2026-09-23T05:13:31.572Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5101worker2 # [ 97.074876] nix-grpc-daemon[870]: time=2026-09-23T05:13:31.573Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5102lb # [ 97.109290] niks3-server[970]: 2026/09/23 05:13:31 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5103worker2 # [ 97.111170] nix-grpc-daemon[870]: time=2026-09-23T05:13:31.610Z level=INFO msg="Uploading 1 narinfos"5104lb # [ 97.123392] niks3-server[970]: 2026/09/23 05:13:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5105lb # [ 97.135990] niks3-server[970]: 2026/09/23 05:13:31 INFO Signed narinfos id=33 count=15106lb # [ 97.141911] niks3-server[970]: 2026/09/23 05:13:31 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5107worker2 # [ 97.143449] nix-grpc-daemon[870]: time=2026-09-23T05:13:31.642Z level=INFO msg="Upload complete. (248ms)"5108lb # [ 97.155328] niks3-server[970]: 2026/09/23 05:13:31 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5109lb # [ 97.158752] niks3-server[970]: 2026/09/23 05:13:31 INFO Completed upload id=335110lb # [ 97.161834] niks3-server[970]: 2026/09/23 05:13:31 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5111worker2 # [ 97.157042] nix-grpc-daemon[773]: ts=2026-09-23T05:13:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_in=5685112worker2 # [ 97.172545] nix-grpc-daemon[773]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590645113worker2 # [ 97.179198] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)5114worker2 # [ 97.185390] nix-daemon[842]: reaped child process 1108, status = succeeded5115client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5116worker2 # [ 97.199940] nix-daemon[842]: accepted connection from pid 773, user nix-grpc-daemon (trusted)5117client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5118client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5119worker2 # [ 98.697094] nix-daemon[842]: reaped child process 1117, status = succeeded5120worker2 # [ 98.764971] nix-daemon[842]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5121worker2 # [ 98.786146] nix-daemon[842]: reaped child process 1136, status = succeeded5122lb # [ 98.795336] niks3-server[970]: 2026/09/23 05:13:33 INFO Received uploads request method=POST path=/api/pending_closures5123worker2 # [ 98.798804] nix-grpc-daemon[870]: time=2026-09-23T05:13:33.295Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5124lb # [ 98.828564] niks3-server[970]: 2026/09/23 05:13:33 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5125worker2 # [ 98.830870] nix-grpc-daemon[870]: time=2026-09-23T05:13:33.330Z level=INFO msg="Uploading 1 narinfos"5126lb # [ 98.840682] niks3-server[970]: 2026/09/23 05:13:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5127lb # [ 98.853153] niks3-server[970]: 2026/09/23 05:13:33 INFO Signed narinfos id=34 count=15128lb # [ 98.860178] niks3-server[970]: 2026/09/23 05:13:33 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5129worker2 # [ 98.864900] nix-grpc-daemon[870]: time=2026-09-23T05:13:33.364Z level=INFO msg="Upload complete. (253ms)"5130lb # [ 98.875349] niks3-server[970]: 2026/09/23 05:13:33 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5131lb # [ 98.882465] niks3-server[970]: 2026/09/23 05:13:33 INFO Completed upload id=345132lb # [ 98.885929] niks3-server[970]: 2026/09/23 05:13:33 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5133worker2 # [ 98.877026] nix-grpc-daemon[773]: ts=2026-09-23T05:13:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15134worker2 # [ 98.892488] nix-daemon[842]: reaped child process 1115, status = succeeded5135client # copying 1 paths...5136worker1 # [ 98.917004] nix-grpc-daemon[767]: ts=2026-09-23T05:13:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467465137worker1 # [ 98.930579] nix-grpc-daemon[767]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5138client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5139worker2 # [ 98.983833] nix-grpc-daemon[773]: ts=2026-09-23T05:13:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:590645140worker2 # [ 98.991366] nix-grpc-daemon[773]: ts=2026-09-23T05:13:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59064 duration_s=0 paths=1 nar_bytes_out=1205141client # [ 99.123152] nix-daemon[1093]: reaped child process 1221, status = succeeded5142client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.64 seconds)5143(finished: subtest: an input only one worker has still reaches the builder, in 9.39 seconds)5144subtest: upload whose reference the worker lacks is completed from the cache5145client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5146client # this derivation will be built:5147client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5148client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5149client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.34 seconds)5150client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5151client # this derivation will be built:5152client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5153client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5154client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.29 seconds)5155client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5156client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5157worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5158worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)5159worker2 # [ 100.098593] systemd[1]: Stopping Nix worker-protocol over gRPC...5160lb # [ 100.277059] envoy[908]: conn peer=[2001:db8:1::1]:35162 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3891 tx=2066 ms=20395161lb # [ 100.282214] envoy[908]: conn peer=[2001:db8:1::1]:48492 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=695162lb # [ 100.287004] envoy[908]: conn peer=[2001:db8:1::1]:48504 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3757 tx=2122 ms=32375163lb # [ 100.291952] envoy[908]: conn peer=[2001:db8:1::1]:48506 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=715164lb # [ 100.296604] envoy[908]: conn peer=[2001:db8:1::1]:48522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3758 tx=2126 ms=22635165lb # [ 100.301509] envoy[908]: conn peer=[2001:db8:1::1]:48532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=295166worker2 # [ 100.724365] nix-grpc-daemon[773]: ts=2026-09-23T05:13:35Z level=info event=signal_exit5167worker1 # [ 102.209074] nix-grpc-daemon[767]: ts=2026-09-23T05:13:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:467365168worker1 # [ 102.209407] nix-grpc-daemon[767]: ts=2026-09-23T05:13:36Z level=info event=worker_gone addr=192.168.1.4:500515169worker2 # [ 102.225192] nix-grpc-daemon[773]: ts=2026-09-23T05:13:36Z level=info event=scheduler_restarting5170worker2 # [ 102.250389] nix-grpc-daemon[773]: ts=2026-09-23T05:13:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5171worker2 # [ 102.270462] nix-daemon[842]: reaped child process 844, status = succeeded5172worker2 # [ 102.889016] nix-grpc-daemon[773]: E0000 00:00:1790140417.388181 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5173worker2 # [ 102.897965] nix-grpc-daemon[773]: E0000 00:00:1790140417.397172 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5174worker2 # [ 104.132821] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5175worker2 # [ 104.137778] systemd[1]: Stopped Nix worker-protocol over gRPC.5176worker2 # [ 104.143802] systemd[1]: nix-grpc-daemon.service: Consumed 2.792s CPU time over 1min 23.701s wall clock time, 60.6M memory peak, 80.6K incoming IP traffic, 185.8K outgoing IP traffic.5177worker2 # [ 104.159015] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5178worker2 # [ 104.168430] systemd[1]: Closed Nix worker-protocol over gRPC.5179worker2 # [ 104.170151] systemd[1]: nix-grpc-daemon.socket: Consumed 719us CPU time over 1min 33.862s wall clock time, 256K memory peak, 93.5K incoming IP traffic, 81.8K outgoing IP traffic.5180lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5181lb: must succeed: curl -sf localhost:9901/clusters5182lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5183lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5184client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25185worker1 # [ 104.459398] nix-grpc-daemon[767]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467465186client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5187worker1 # [ 104.466932] nix-grpc-daemon[767]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467465188client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)5189worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5190worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5191worker2 # [ 104.597904] systemd[1]: Listening on Nix worker-protocol over gRPC.5192worker1 # [ 104.662008] systemd[1]: Stopping Nix worker-protocol over gRPC...5193worker2 # [ 105.129556] systemd[1]: Starting scheduler-token.service...5194worker2 # [ 105.222828] systemd[1]: scheduler-token.service: Deactivated successfully.5195worker2 # [ 105.233149] systemd[1]: Finished scheduler-token.service.5196worker2 # [ 105.235119] systemd[1]: scheduler-token.service: Consumed 62ms CPU time over 97ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5197worker2 # [ 105.242507] systemd[1]: Starting Nix worker-protocol over gRPC...5198worker2 # [ 105.392575] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5199worker2 # [ 105.395149] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5200worker2 # [ 105.399288] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15201worker2 # [ 105.404056] nix-grpc-daemon[1160]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5202worker2 # [ 105.405511] nix-grpc-daemon[1160]: E0000 00:00:1790140419.903431 1160 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5203worker2 # [ 105.452094] nix-grpc-daemon[1160]: E0000 00:00:1790140419.909882 1160 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5204worker2 # [ 105.454551] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5205worker2 # [ 105.460470] nix-grpc-daemon[1160]: E0000 00:00:1790140419.920793 1200 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5206worker2 # [ 105.463347] nix-grpc-daemon[1160]: E0000 00:00:1790140419.939621 1200 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5207worker2 # [ 105.466003] nix-grpc-daemon[1160]: E0000 00:00:1790140419.939795 1200 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5208worker2 # [ 105.471559] systemd[1]: Started Nix worker-protocol over gRPC.5209worker2 # [ 105.474587] nix-grpc-daemon[1160]: E0000 00:00:1790140419.958910 1200 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5210worker2 # [ 105.477235] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:39Z level=info event=scheduler_connected addr=lb:500515211worker1 # [ 105.469967] nix-grpc-daemon[767]: ts=2026-09-23T05:13:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:467365212worker1 # [ 105.476197] nix-grpc-daemon[767]: ts=2026-09-23T05:13:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05213worker1 # [ 105.542668] nix-grpc-daemon[767]: ts=2026-09-23T05:13:40Z level=info event=signal_exit5214lb # [ 106.230876] niks3-server[970]: 2026/09/23 05:13:40 INFO lead: released remote=192.168.1.3:391185215worker1 # [ 107.203750] nix-grpc-daemon[767]: ts=2026-09-23T05:13:41Z level=info event=scheduler_restarting5216worker2 # [ 107.215873] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:41Z level=info event=scheduler_restarting addr=lb:500515217worker1 # [ 107.218512] nix-grpc-daemon[767]: ts=2026-09-23T05:13:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:467365218worker1 # [ 107.224502] nix-grpc-daemon[767]: ts=2026-09-23T05:13:41Z level=info event=worker_gone addr=192.168.1.4:500515219worker1 # [ 107.229075] nix-grpc-daemon[767]: ts=2026-09-23T05:13:41Z level=info event=scheduler_restarting addr=lb:500515220worker1 # [ 107.233936] nix-grpc-daemon[767]: ts=2026-09-23T05:13:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:467365221worker1 # [ 107.239482] nix-grpc-daemon[767]: ts=2026-09-23T05:13:41Z level=info event=worker_gone addr=192.168.1.3:500515222worker1 # [ 107.269466] nix-daemon[850]: reaped child process 852, status = succeeded5223worker2 # [ 107.415817] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:41Z level=info event=scheduler_connected addr=lb:500515224worker2 # [ 107.422382] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5225worker1 # [ 107.441695] nix-grpc-daemon[767]: ts=2026-09-23T05:13:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5226worker2 # [ 107.623568] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500515227worker2 # [ 107.629447] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5228worker1 # [ 107.731876] nix-grpc-daemon[767]: E0000 00:00:1790140422.234748 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5229worker1 # [ 107.738861] nix-grpc-daemon[767]: E0000 00:00:1790140422.235140 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5230worker2 # [ 107.830416] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500515231worker2 # [ 107.836215] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5232worker2 # [ 108.037901] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500515233worker2 # [ 108.043513] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5234worker1 # [ 108.063289] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5235worker1 # [ 108.071626] systemd[1]: Stopped Nix worker-protocol over gRPC.5236worker1 # [ 108.079975] systemd[1]: nix-grpc-daemon.service: Consumed 3.899s CPU time over 1min 27.803s wall clock time, 69.3M memory peak, 115.5K incoming IP traffic, 233.1K outgoing IP traffic.5237worker1 # [ 108.095331] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5238worker1 # [ 108.101837] systemd[1]: Closed Nix worker-protocol over gRPC.5239worker1 # [ 108.103589] systemd[1]: nix-grpc-daemon.socket: Consumed 808us CPU time over 1min 37.371s wall clock time, 256K memory peak, 125.6K incoming IP traffic, 106.6K outgoing IP traffic.5240lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5241lb: must succeed: curl -sf localhost:9901/clusters5242lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5243lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5244client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25245worker2 # [ 108.246227] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500515246worker2 # [ 108.251672] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5247worker2 # [ 108.443683] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372965248worker2 # [ 108.461248] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500515249worker2 # [ 108.470740] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5250worker2 # [ 108.481372] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5251worker2 # [ 108.497495] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5)5252worker2 # [ 108.598121] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 164 ms (attempt 2/5)5253worker2 # [ 108.656719] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500515254worker2 # [ 108.662445] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5255worker2 # [ 108.764649] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 392 ms (attempt 3/5)5256worker2 # [ 108.863480] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500515257worker2 # [ 108.869277] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5258worker2 # [ 109.070466] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500515259worker2 # [ 109.075866] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5260worker2 # [ 109.158895] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 648 ms (attempt 4/5)5261worker2 # [ 109.279268] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500515262worker2 # [ 109.284491] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5263worker2 # [ 109.480733] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500515264worker2 # [ 109.486538] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5265worker2 # [ 109.687519] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=scheduler_connected addr=lb:500515266worker2 # [ 109.692958] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5267worker2 # [ 109.810585] nix-grpc-daemon[1160]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5268worker2 # [ 109.817354] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5269client # copying 2 paths...5270client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5271worker2 # [ 109.844651] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372965272client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5273worker2 # [ 109.866583] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5274worker2 # [ 109.899606] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=niks3_push_started pid=12215275worker2 # [ 109.925830] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=scheduler_connected addr=lb:500515276worker2 # [ 109.930666] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5277worker2 # [ 110.062641] nix-grpc-daemon[1221]: time=2026-09-23T05:13:44.561Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5278worker2 # [ 110.114405] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=scheduler_connected addr=lb:500515279worker2 # [ 110.118556] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5280worker2 # [ 110.182614] nix-daemon[842]: accepted connection from pid 1234, user nix-grpc-daemon (trusted)5281worker2 # [ 110.206319] nix-daemon[842]: reaped child process 1237, status = succeeded5282lb # [ 110.224970] niks3-server[970]: 2026/09/23 05:13:44 INFO Received uploads request method=POST path=/api/pending_closures5283lb # [ 110.236502] niks3-server[970]: 2026/09/23 05:13:44 INFO Received uploads request method=POST path=/api/pending_closures5284worker2 # [ 110.238321] nix-grpc-daemon[1221]: time=2026-09-23T05:13:44.735Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5285worker2 # [ 110.239963] nix-grpc-daemon[1221]: time=2026-09-23T05:13:44.735Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5286worker2 # [ 110.244574] nix-grpc-daemon[1221]: time=2026-09-23T05:13:44.735Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5287lb # [ 110.279904] envoy[908]: conn peer=[2001:db8:1::4]:52702 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=53547 tx=3126 ms=814705288lb # [ 110.293985] envoy[908]: conn peer=[2001:db8:1::1]:43496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=235289lb # [ 110.303415] envoy[908]: conn peer=[2001:db8:1::3]:47868 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=11705 tx=3444 ms=868275290lb # [ 110.317150] niks3-server[970]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5291worker2 # [ 110.319375] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=scheduler_connected addr=lb:500515292lb # [ 110.331317] niks3-server[970]: 2026/09/23 05:13:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5293worker2 # [ 110.323331] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5294lb # [ 110.346369] niks3-server[970]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5295lb # [ 110.357444] niks3-server[970]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5296lb # [ 110.362699] niks3-server[970]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5297lb # [ 110.367726] niks3-server[970]: 2026/09/23 05:13:44 INFO Signed narinfos id=35 count=15298worker2 # [ 110.360154] nix-grpc-daemon[1221]: time=2026-09-23T05:13:44.857Z level=INFO msg="Uploading 3 narinfos"5299lb # [ 110.369883] niks3-server[970]: 2026/09/23 05:13:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5300lb # [ 110.374868] niks3-server[970]: 2026/09/23 05:13:44 INFO Signed narinfos id=36 count=25301lb # [ 110.401019] niks3-server[970]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5302lb # [ 110.406799] niks3-server[970]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5303lb # [ 110.411583] niks3-server[970]: 2026/09/23 05:13:44 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5304lb # [ 110.419727] niks3-server[970]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5305lb # [ 110.425619] niks3-server[970]: 2026/09/23 05:13:44 INFO Completed upload id=355306lb # [ 110.426895] niks3-server[970]: 2026/09/23 05:13:44 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5307lb # [ 110.429849] niks3-server[970]: 2026/09/23 05:13:44 INFO Completed upload id=365308lb # [ 110.432669] niks3-server[970]: 2026/09/23 05:13:44 INFO lead: acquired remote=192.168.1.4:467365309worker2 # [ 110.420390] nix-grpc-daemon[1221]: time=2026-09-23T05:13:44.919Z level=INFO msg="Upload complete. (353ms)"5310worker2 # [ 110.432527] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=scheduler_take_over5311worker2 # [ 110.440564] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37296 duration_s=0 paths=2 nar_bytes_in=3125312worker2 # [ 110.461562] nix-daemon[842]: reaped child process 1219, status = succeeded5313client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.27 seconds)5314worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5315worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5316worker2 # [ 110.520806] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:45Z level=info event=scheduler_connected addr=lb:500515317worker2 # [ 110.539502] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:45Z level=info event=oidc_jwks provider=mock keys=15318worker2 # [ 110.544302] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:373045319worker2 # [ 110.549321] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05320worker1 # [ 110.573303] systemd[1]: Listening on Nix worker-protocol over gRPC.5321worker1 # [ 110.583550] systemd[1]: Starting Nix worker-protocol over gRPC...5322worker1 # [ 110.730991] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5323worker1 # [ 110.733459] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5324worker1 # [ 110.737620] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15325worker1 # [ 110.742105] nix-grpc-daemon[1322]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5326worker1 # [ 110.743588] nix-grpc-daemon[1322]: E0000 00:00:1790140425.245451 1322 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5327lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5328lb: must succeed: curl -sf localhost:9901/clusters5329worker1 # [ 110.780314] nix-grpc-daemon[1322]: E0000 00:00:1790140425.252152 1322 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5330worker1 # [ 110.795192] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5331worker1 # [ 110.797680] nix-grpc-daemon[1322]: E0000 00:00:1790140425.262433 1362 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5332worker1 # [ 110.800227] nix-grpc-daemon[1322]: E0000 00:00:1790140425.279451 1362 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5333worker1 # [ 110.802460] nix-grpc-daemon[1322]: E0000 00:00:1790140425.279723 1362 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5334worker1 # [ 110.804872] systemd[1]: Started Nix worker-protocol over gRPC.5335worker1 # [ 110.805698] nix-grpc-daemon[1322]: E0000 00:00:1790140425.297319 1362 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5336worker1 # [ 110.814112] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:45Z level=info event=scheduler_connected addr=lb:500515337worker2 # [ 110.824104] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:373045338worker2 # [ 110.830818] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5340lb: must succeed: curl -sf localhost:9901/clusters5341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5342lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5343(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.84 seconds)5344subtest: a niks3 restart does not cost the leader its role5345lb: must succeed: curl -sf localhost:9901/clusters5346lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5347lb: must succeed: curl -sf localhost:9901/clusters5348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5349lb # [ 112.069673] niks3-server[970]: 2026/09/23 05:13:46 INFO Shutdown signal received, draining in-flight requests timeout=10s5350lb # [ 112.070249] systemd[1]: Stopping niks3 server...5351worker1 # [ 112.058216] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:46Z level=info event=scheduler_lock_lost status=2005352lb # [ 112.080157] niks3-server[970]: 2026/09/23 05:13:46 INFO lead: released remote=192.168.1.4:467365353worker2 # [ 112.067768] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:46Z level=info event=scheduler_lock_lost status=2005354lb # [ 112.102378] systemd[1]: niks3.service: Deactivated successfully.5355lb # [ 112.104322] systemd[1]: Stopped niks3 server.5356lb # [ 112.105055] systemd[1]: niks3.service: Consumed 689ms CPU time over 1min 28.183s wall clock time, 27.7M memory peak, 98K incoming IP traffic, 141.1K outgoing IP traffic.5357lb # [ 112.115344] systemd[1]: Starting niks3 server...5358lb # [ 112.179717] niks3-server[1111]: 2026/09/23 05:13:46 goose: no migrations to run. current version: 202609200000005359lb # [ 112.186456] niks3-server[1111]: 2026/09/23 05:13:46 OK 1_commit_pending_closure.sql (5.07ms)5360lb # [ 112.190356] niks3-server[1111]: 2026/09/23 05:13:46 OK 2_object_stats_trigger.sql (3.5ms)5361lb # [ 112.191561] niks3-server[1111]: 2026/09/23 05:13:46 goose: up to current file version: 25362lb # [ 112.193672] niks3-server[1111]: 2026/09/23 05:13:46 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5363lb # [ 112.205098] niks3-server[1111]: 2026/09/23 05:13:46 INFO Read proxy enabled — serving cache objects from S35364lb # [ 112.206524] niks3-server[1111]: 2026/09/23 05:13:46 INFO Using socket-activated listener address=0.0.0.0:57515365lb # [ 112.211947] niks3-server[1111]: 2026/09/23 05:13:46 INFO systemd watchdog enabled interval=15s5366lb # [ 112.213490] niks3-server[1111]: 2026/09/23 05:13:46 INFO Starting HTTP server address=0.0.0.0:57515367lb # [ 112.215347] systemd[1]: Started niks3 server.5368lb: waiting for unit niks3.service5369lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5370lb: must succeed: curl -sf localhost:9901/clusters5371lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5372worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5373worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5374lb: must succeed: curl -sf localhost:9901/clusters5375lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5376lb: must succeed: curl -sf localhost:9901/clusters5377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5378worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5379worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5380client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25381worker1 # [ 112.702710] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:489925382worker1 # [ 112.714431] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5383client # don't know how to build these paths:5384client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5385worker2 # [ 112.753520] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372965386worker2 # [ 112.788173] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:47Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05387worker2 # [ 112.794368] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:47Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15388worker2 # [ 112.842828] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372965389worker2 # [ 112.861163] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372965390client # copying 1 paths to 'lb:50051'...5391worker2 # [ 112.878887] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372965392client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5393worker2 # [ 112.896753] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5394worker2 # [ 113.057643] nix-daemon[842]: accepted connection from pid 1254, user nix-grpc-daemon (trusted)5395lb # [ 113.087183] niks3-server[1111]: 2026/09/23 05:13:47 INFO lead: acquired remote=192.168.1.4:395385396worker2 # [ 113.085618] nix-daemon[842]: reaped child process 1257, status = succeeded5397worker2 # [ 113.118949] nix-grpc-daemon[1221]: time=2026-09-23T05:13:47.617Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5398worker2 # [ 113.121338] nix-grpc-daemon[1221]: time=2026-09-23T05:13:47.617Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5399lb # [ 113.093436] niks3-server[1111]: 2026/09/23 05:13:47 INFO Received uploads request method=POST path=/api/pending_closures5400lb # [ 113.163444] niks3-server[1111]: 2026/09/23 05:13:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5401lb # [ 113.189619] niks3-server[1111]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5402lb # [ 113.208097] niks3-server[1111]: 2026/09/23 05:13:47 INFO Signed narinfos id=37 count=15403lb # [ 113.216385] niks3-server[1111]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5404worker2 # [ 113.203784] nix-grpc-daemon[1221]: time=2026-09-23T05:13:47.701Z level=INFO msg="Uploading 1 narinfos"5405lb # [ 113.224421] niks3-server[1111]: 2026/09/23 05:13:47 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5406lb # [ 113.232738] niks3-server[1111]: 2026/09/23 05:13:47 INFO Completed upload id=375407lb # [ 113.240390] niks3-server[1111]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5408worker2 # [ 113.223098] nix-grpc-daemon[1221]: time=2026-09-23T05:13:47.722Z level=INFO msg="Upload complete. (307ms)"5409worker2 # [ 113.238606] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37296 duration_s=0 paths=1 nar_bytes_in=5445410worker2 # [ 113.252525] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372965411worker2 # [ 113.266518] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5412worker2 # [ 113.273117] nix-daemon[842]: reaped child process 1252, status = succeeded5413client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5414client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5415worker2 # [ 113.286756] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5416client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5417client # farm-a> LOG-farm-a-after-niks3-restart5418worker2 # [ 114.126196] nix-daemon[842]: reaped child process 1263, status = succeeded5419worker2 # [ 114.231127] nix-daemon[842]: accepted connection from pid 1279, user nix-grpc-daemon (trusted)5420worker2 # [ 114.251880] nix-daemon[842]: reaped child process 1282, status = succeeded5421lb # [ 114.270095] niks3-server[1111]: 2026/09/23 05:13:48 INFO Received uploads request method=POST path=/api/pending_closures5422worker2 # [ 114.275639] nix-grpc-daemon[1221]: time=2026-09-23T05:13:48.772Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5423worker2 # [ 114.277416] nix-grpc-daemon[1221]: time=2026-09-23T05:13:48.772Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5424lb # [ 114.316973] niks3-server[1111]: 2026/09/23 05:13:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5425lb # [ 114.334826] niks3-server[1111]: 2026/09/23 05:13:48 INFO Signed narinfos id=38 count=15426worker2 # [ 114.329702] nix-grpc-daemon[1221]: time=2026-09-23T05:13:48.828Z level=INFO msg="Uploading 1 narinfos"5427lb # [ 114.348667] niks3-server[1111]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5428lb # [ 114.361347] niks3-server[1111]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5429lb # [ 114.373013] niks3-server[1111]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5430lb # [ 114.375851] niks3-server[1111]: 2026/09/23 05:13:48 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5431lb # [ 114.380465] niks3-server[1111]: 2026/09/23 05:13:48 INFO Completed upload id=385432worker2 # [ 114.370481] nix-grpc-daemon[1221]: time=2026-09-23T05:13:48.869Z level=INFO msg="Upload complete. (269ms)"5433lb # [ 114.383002] niks3-server[1111]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5434worker2 # [ 114.397280] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37296 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15435worker2 # [ 114.408585] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:48Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05436worker2 # [ 114.415876] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:48Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15437worker2 # [ 114.420576] nix-daemon[842]: reaped child process 1261, status = succeeded5438worker2 # [ 114.442762] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372965439worker2 # [ 114.459756] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372965440worker2 # [ 114.478399] nix-grpc-daemon[1221]: time=2026-09-23T05:13:48.977Z level=INFO msg="All 1 paths already cached"5441client # copying 1 paths to 'lb:50051'...5442worker2 # [ 114.490221] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372965443client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5444worker2 # [ 114.507774] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5445worker2 # [ 114.668156] nix-daemon[842]: accepted connection from pid 1288, user nix-grpc-daemon (trusted)5446worker2 # [ 114.693510] nix-daemon[842]: reaped child process 1291, status = succeeded5447lb # [ 114.700649] niks3-server[1111]: 2026/09/23 05:13:49 INFO Received uploads request method=POST path=/api/pending_closures5448worker2 # [ 114.699786] nix-grpc-daemon[1221]: time=2026-09-23T05:13:49.198Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5449worker2 # [ 114.701647] nix-grpc-daemon[1221]: time=2026-09-23T05:13:49.198Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5450lb # [ 114.728998] niks3-server[1111]: 2026/09/23 05:13:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5451worker2 # [ 114.725619] nix-grpc-daemon[1221]: time=2026-09-23T05:13:49.224Z level=INFO msg="Uploading 1 narinfos"5452lb # [ 114.740983] niks3-server[1111]: 2026/09/23 05:13:49 INFO Signed narinfos id=39 count=15453lb # [ 114.752763] niks3-server[1111]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5454lb # [ 114.772451] niks3-server[1111]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5455lb # [ 114.775925] niks3-server[1111]: 2026/09/23 05:13:49 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5456lb # [ 114.781315] niks3-server[1111]: 2026/09/23 05:13:49 INFO Completed upload id=395457worker2 # [ 114.771314] nix-grpc-daemon[1221]: time=2026-09-23T05:13:49.270Z level=INFO msg="Upload complete. (244ms)"5458lb # [ 114.785700] niks3-server[1111]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5459worker2 # [ 114.784505] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37296 duration_s=0 paths=1 nar_bytes_in=6565460worker2 # [ 114.793898] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372965461worker2 # [ 114.806705] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5462worker2 # [ 114.813947] nix-daemon[842]: reaped child process 1286, status = succeeded5463client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5464client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5465worker2 # [ 114.829194] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5466client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5467client # farm-b> LOG-farm-b-after-niks3-restart5468worker2 # [ 115.463303] nix-daemon[842]: reaped child process 1295, status = succeeded5469worker2 # [ 115.563799] nix-daemon[842]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5470worker2 # [ 115.589241] nix-daemon[842]: reaped child process 1314, status = succeeded5471lb # [ 115.597833] niks3-server[1111]: 2026/09/23 05:13:50 INFO Received uploads request method=POST path=/api/pending_closures5472worker2 # [ 115.600606] nix-grpc-daemon[1221]: time=2026-09-23T05:13:50.099Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5473worker2 # [ 115.602278] nix-grpc-daemon[1221]: time=2026-09-23T05:13:50.099Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5474lb # [ 115.648976] niks3-server[1111]: 2026/09/23 05:13:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5475worker2 # [ 115.648254] nix-grpc-daemon[1221]: time=2026-09-23T05:13:50.147Z level=INFO msg="Uploading 1 narinfos"5476lb # [ 115.661020] niks3-server[1111]: 2026/09/23 05:13:50 INFO Signed narinfos id=40 count=15477lb # [ 115.669576] niks3-server[1111]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5478lb # [ 115.677167] niks3-server[1111]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5479lb # [ 115.689253] niks3-server[1111]: 2026/09/23 05:13:50 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5480lb # [ 115.696869] niks3-server[1111]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5481lb # [ 115.702509] niks3-server[1111]: 2026/09/23 05:13:50 INFO Completed upload id=405482worker2 # [ 115.692641] nix-grpc-daemon[1221]: time=2026-09-23T05:13:50.191Z level=INFO msg="Upload complete. (252ms)"5483lb # [ 115.706363] niks3-server[1111]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5484worker2 # [ 115.717450] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37296 duration_s=0 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15485worker2 # [ 115.734660] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:50Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05486worker2 # [ 115.738143] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:50Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15487worker2 # [ 115.742284] nix-daemon[842]: reaped child process 1293, status = succeeded5488worker2 # [ 115.758821] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372965489worker2 # [ 115.774280] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372965490worker2 # [ 115.789755] nix-grpc-daemon[1221]: time=2026-09-23T05:13:50.289Z level=INFO msg="All 2 paths already cached"5491client # copying 1 paths to 'lb:50051'...5492client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5493worker2 # [ 115.801715] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372965494worker2 # [ 115.816403] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5495worker2 # [ 115.974053] nix-daemon[842]: accepted connection from pid 1320, user nix-grpc-daemon (trusted)5496worker2 # [ 115.998080] nix-daemon[842]: reaped child process 1323, status = succeeded5497lb # [ 116.009295] niks3-server[1111]: 2026/09/23 05:13:50 INFO Received uploads request method=POST path=/api/pending_closures5498worker2 # [ 116.008392] nix-grpc-daemon[1221]: time=2026-09-23T05:13:50.507Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5499worker2 # [ 116.010558] nix-grpc-daemon[1221]: time=2026-09-23T05:13:50.507Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5500worker2 # [ 116.042849] nix-grpc-daemon[1221]: time=2026-09-23T05:13:50.541Z level=INFO msg="Uploading 1 narinfos"5501lb # [ 116.046586] niks3-server[1111]: 2026/09/23 05:13:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5502lb # [ 116.059187] niks3-server[1111]: 2026/09/23 05:13:50 INFO Signed narinfos id=41 count=15503lb # [ 116.068263] niks3-server[1111]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5504worker2 # [ 116.075617] nix-grpc-daemon[1221]: time=2026-09-23T05:13:50.575Z level=INFO msg="Upload complete. (242ms)"5505lb # [ 116.085908] niks3-server[1111]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5506lb # [ 116.093809] niks3-server[1111]: 2026/09/23 05:13:50 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5507worker2 # [ 116.084345] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37296 duration_s=0 paths=1 nar_bytes_in=7845508lb # [ 116.099400] niks3-server[1111]: 2026/09/23 05:13:50 INFO Completed upload id=415509lb # [ 116.102739] niks3-server[1111]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5510worker2 # [ 116.100405] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372965511worker2 # [ 116.109718] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5512worker2 # [ 116.115203] nix-daemon[842]: reaped child process 1318, status = succeeded5513client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5514worker2 # [ 116.127200] nix-daemon[842]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5515client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5516client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5517client # farm-top> LOG-farm-top-after-niks3-restart5518worker2 # [ 117.757391] nix-daemon[842]: reaped child process 1327, status = succeeded5519worker2 # [ 117.861557] nix-daemon[842]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5520worker2 # [ 117.884744] nix-daemon[842]: reaped child process 1346, status = succeeded5521lb # [ 117.906494] niks3-server[1111]: 2026/09/23 05:13:52 INFO Received uploads request method=POST path=/api/pending_closures5522worker2 # [ 117.909093] nix-grpc-daemon[1221]: time=2026-09-23T05:13:52.407Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5523worker2 # [ 117.910667] nix-grpc-daemon[1221]: time=2026-09-23T05:13:52.407Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5524lb # [ 117.958887] niks3-server[1111]: 2026/09/23 05:13:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5525worker2 # [ 117.958236] nix-grpc-daemon[1221]: time=2026-09-23T05:13:52.457Z level=INFO msg="Uploading 1 narinfos"5526lb # [ 117.972322] niks3-server[1111]: 2026/09/23 05:13:52 INFO Signed narinfos id=42 count=15527lb # [ 117.980779] niks3-server[1111]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5528lb # [ 117.990399] niks3-server[1111]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5529lb # [ 117.997947] niks3-server[1111]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5530lb # [ 118.007893] niks3-server[1111]: 2026/09/23 05:13:52 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5531worker2 # [ 118.002022] nix-grpc-daemon[1221]: time=2026-09-23T05:13:52.501Z level=INFO msg="Upload complete. (260ms)"5532lb # [ 118.015270] niks3-server[1111]: 2026/09/23 05:13:52 INFO Completed upload id=425533lb # [ 118.018244] niks3-server[1111]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5534worker2 # [ 118.018866] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37296 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15535worker2 # [ 118.045722] nix-daemon[842]: reaped child process 1325, status = succeeded5536client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.59 seconds)5537client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5538client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.20 seconds)5539(finished: subtest: a niks3 restart does not cost the leader its role, in 6.34 seconds)5540subtest: scheduler down: the other node takes the lock and keeps it5541lb: must succeed: curl -sf localhost:9901/clusters5542lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5543lb: must succeed: curl -sf localhost:9901/clusters5544lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5545lb: must succeed: curl -sf localhost:9901/clusters5546lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5547lb: must succeed: curl -sf localhost:9901/clusters5548lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5549worker2 # [ 118.526503] systemd[1]: Stopping Nix worker-protocol over gRPC...5550worker2 # [ 119.424889] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:53Z level=info event=signal_exit5551lb # [ 120.097337] niks3-server[1111]: 2026/09/23 05:13:54 INFO lead: released remote=192.168.1.4:395385552lb # [ 120.279236] envoy[908]: conn peer=[2001:db8:1::1]:43502 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1011 tx=305 ms=20125553lb # [ 120.284574] envoy[908]: conn peer=[2001:db8:1::1]:46834 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9719 tx=4434 ms=53255554worker1 # [ 121.077225] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:55Z level=info event=scheduler_restarting addr=lb:500515555worker2 # [ 121.090151] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:55Z level=info event=scheduler_restarting5556worker2 # [ 121.095509] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:373045557worker2 # [ 121.100917] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:55Z level=info event=worker_gone addr=192.168.1.4:500515558worker2 # [ 121.105679] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:373045559worker2 # [ 121.111132] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:55Z level=info event=worker_gone addr=192.168.1.3:500515560worker2 # [ 121.115645] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:55Z level=info event=scheduler_restarting addr=lb:500515561worker2 # [ 121.141116] nix-daemon[842]: reaped child process 1213, status = succeeded5562worker1 # [ 121.276858] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:55Z level=info event=scheduler_connected addr=lb:500515563worker1 # [ 121.282609] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5564worker2 # [ 121.310698] nix-grpc-daemon[1160]: ts=2026-09-23T05:13:55Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5565worker2 # [ 121.478891] nix-grpc-daemon[1160]: E0000 00:00:1790140435.977814 1206 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5566worker2 # [ 121.486078] nix-grpc-daemon[1160]: E0000 00:00:1790140435.978749 1206 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5567worker1 # [ 121.483335] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:55Z level=info event=scheduler_connected addr=lb:500515568worker1 # [ 121.488909] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5569worker1 # [ 121.689759] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:56Z level=info event=scheduler_connected addr=lb:500515570worker1 # [ 121.695194] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5571worker1 # [ 121.896143] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:56Z level=info event=scheduler_connected addr=lb:500515572worker1 # [ 121.901568] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5573worker1 # [ 122.104706] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:56Z level=info event=scheduler_connected addr=lb:500515574worker1 # [ 122.108934] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5575worker1 # [ 122.305685] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:56Z level=info event=scheduler_connected addr=lb:500515576worker1 # [ 122.311101] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5577worker2 # [ 122.374832] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5578worker2 # [ 122.379831] systemd[1]: Stopped Nix worker-protocol over gRPC.5579worker2 # [ 122.388581] systemd[1]: nix-grpc-daemon.service: Consumed 1.411s CPU time over 17.136s wall clock time, 39.1M memory peak, 49.2K incoming IP traffic, 94.4K outgoing IP traffic.5580worker2 # [ 122.400577] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5581worker2 # [ 122.405720] systemd[1]: Closed Nix worker-protocol over gRPC.5582worker2 # [ 122.407485] systemd[1]: nix-grpc-daemon.socket: Consumed 801us CPU time over 17.814s wall clock time, 256K memory peak, 38.6K incoming IP traffic, 29K outgoing IP traffic.5583worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5584worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5585worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5586worker1 # [ 122.512361] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=scheduler_connected addr=lb:500515587worker1 # [ 122.519287] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5588worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5589worker1 # [ 122.717715] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=scheduler_connected addr=lb:500515590worker1 # [ 122.723252] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5591worker1 # [ 122.924119] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=scheduler_connected addr=lb:500515592worker1 # [ 122.929514] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5593worker1 # [ 123.064134] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=scheduler_take_over5594lb # [ 123.083624] niks3-server[1111]: 2026/09/23 05:13:57 INFO lead: acquired remote=192.168.1.3:571365595worker1 # [ 123.131568] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=scheduler_connected addr=lb:500515596worker1 # [ 123.141529] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:508305597worker1 # [ 123.147934] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05598worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5599worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5600client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25601worker1 # [ 123.835667] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:489925602client # don't know how to build these paths:5603client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5604worker1 # [ 123.846429] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489925605worker1 # [ 123.861757] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5)5606worker1 # [ 123.961498] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 158 ms (attempt 2/5)5607worker1 # [ 124.124057] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 145 ms (attempt 3/5)5608worker1 # [ 124.279826] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 470 ms (attempt 4/5)5609worker1 # [ 124.751556] nix-grpc-daemon[1322]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5610worker1 # [ 124.758694] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5611worker1 # [ 124.792138] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:59Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05612worker1 # [ 124.798609] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:59Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15613worker1 # [ 124.846022] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489925614worker1 # [ 124.863000] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489925615client # copying 1 paths to 'lb:50051'...5616worker1 # [ 124.878202] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489925617client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5618worker1 # [ 124.895683] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5619worker1 # [ 124.924149] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:59Z level=info event=niks3_push_started pid=13995620worker1 # [ 125.088841] nix-grpc-daemon[1399]: time=2026-09-23T05:13:59.591Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5621worker1 # [ 125.203906] nix-daemon[850]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5622worker1 # [ 125.228645] nix-daemon[850]: reaped child process 1415, status = succeeded5623lb # [ 125.249200] niks3-server[1111]: 2026/09/23 05:13:59 INFO Received uploads request method=POST path=/api/pending_closures5624worker1 # [ 125.243058] nix-grpc-daemon[1399]: time=2026-09-23T05:13:59.745Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5625worker1 # [ 125.247148] nix-grpc-daemon[1399]: time=2026-09-23T05:13:59.745Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5626lb # [ 125.295058] niks3-server[1111]: 2026/09/23 05:13:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5627lb # [ 125.313803] niks3-server[1111]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5628lb # [ 125.329739] niks3-server[1111]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5629lb # [ 125.346603] niks3-server[1111]: 2026/09/23 05:13:59 INFO Signed narinfos id=43 count=15630worker1 # [ 125.329819] nix-grpc-daemon[1399]: time=2026-09-23T05:13:59.831Z level=INFO msg="Uploading 1 narinfos"5631lb # [ 125.362987] niks3-server[1111]: 2026/09/23 05:13:59 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5632lb # [ 125.379544] niks3-server[1111]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5633lb # [ 125.386090] niks3-server[1111]: 2026/09/23 05:13:59 INFO Completed upload id=435634worker1 # [ 125.367683] nix-grpc-daemon[1399]: time=2026-09-23T05:13:59.870Z level=INFO msg="Upload complete. (275ms)"5635worker1 # [ 125.384217] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48992 duration_s=0 paths=1 nar_bytes_in=5205636worker1 # [ 125.400181] nix-grpc-daemon[1322]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489925637worker1 # [ 125.407827] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5638worker1 # [ 125.413810] nix-daemon[850]: reaped child process 1397, status = succeeded5639client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5640client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5641worker1 # [ 125.430251] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5642client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5643client # farm-a> LOG-farm-a-on-standby5644worker1 # [ 126.471106] nix-daemon[850]: reaped child process 1419, status = succeeded5645worker1 # [ 126.585269] nix-daemon[850]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5646worker1 # [ 126.608047] nix-daemon[850]: reaped child process 1438, status = succeeded5647lb # [ 126.625148] niks3-server[1111]: 2026/09/23 05:14:01 INFO Received uploads request method=POST path=/api/pending_closures5648worker1 # [ 126.620462] nix-grpc-daemon[1399]: time=2026-09-23T05:14:01.121Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5649worker1 # [ 126.621993] nix-grpc-daemon[1399]: time=2026-09-23T05:14:01.121Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5650lb # [ 126.666472] niks3-server[1111]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5651lb # [ 126.688102] niks3-server[1111]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5652worker1 # [ 126.683254] nix-grpc-daemon[1399]: time=2026-09-23T05:14:01.186Z level=INFO msg="Uploading 1 narinfos"5653lb # [ 126.701674] niks3-server[1111]: 2026/09/23 05:14:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5654lb # [ 126.710742] niks3-server[1111]: 2026/09/23 05:14:01 INFO Signed narinfos id=44 count=15655lb # [ 126.719204] niks3-server[1111]: 2026/09/23 05:14:01 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5656lb # [ 126.729237] niks3-server[1111]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5657lb # [ 126.734911] niks3-server[1111]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5658lb # [ 126.739308] niks3-server[1111]: 2026/09/23 05:14:01 INFO Completed upload id=445659worker1 # [ 126.720732] nix-grpc-daemon[1399]: time=2026-09-23T05:14:01.223Z level=INFO msg="Upload complete. (277ms)"5660worker1 # [ 126.743834] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48992 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15661worker1 # [ 126.760851] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:01Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05662worker1 # [ 126.766583] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:01Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15663worker1 # [ 126.770802] nix-daemon[850]: reaped child process 1417, status = succeeded5664worker1 # [ 126.786081] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489925665worker1 # [ 126.801807] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489925666worker1 # [ 126.816962] nix-grpc-daemon[1399]: time=2026-09-23T05:14:01.320Z level=INFO msg="All 1 paths already cached"5667client # copying 1 paths to 'lb:50051'...5668client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5669worker1 # [ 126.828603] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489925670worker1 # [ 126.842966] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5671worker1 # [ 127.002060] nix-daemon[850]: accepted connection from pid 1444, user nix-grpc-daemon (trusted)5672worker1 # [ 127.024870] nix-daemon[850]: reaped child process 1447, status = succeeded5673lb # [ 127.040699] niks3-server[1111]: 2026/09/23 05:14:01 INFO Received uploads request method=POST path=/api/pending_closures5674worker1 # [ 127.032248] nix-grpc-daemon[1399]: time=2026-09-23T05:14:01.535Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5675worker1 # [ 127.034379] nix-grpc-daemon[1399]: time=2026-09-23T05:14:01.535Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5676worker1 # [ 127.055431] nix-grpc-daemon[1399]: time=2026-09-23T05:14:01.558Z level=INFO msg="Uploading 1 narinfos"5677lb # [ 127.074504] niks3-server[1111]: 2026/09/23 05:14:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5678lb # [ 127.087442] niks3-server[1111]: 2026/09/23 05:14:01 INFO Signed narinfos id=45 count=15679lb # [ 127.096467] niks3-server[1111]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5680worker1 # [ 127.090932] nix-grpc-daemon[1399]: time=2026-09-23T05:14:01.594Z level=INFO msg="Upload complete. (231ms)"5681lb # [ 127.109706] niks3-server[1111]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5682lb # [ 127.117751] niks3-server[1111]: 2026/09/23 05:14:01 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5683lb # [ 127.126986] niks3-server[1111]: 2026/09/23 05:14:01 INFO Completed upload id=455684lb # [ 127.130647] niks3-server[1111]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5685worker1 # [ 127.100317] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48992 duration_s=0 paths=1 nar_bytes_in=6405686worker1 # [ 127.116993] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489925687worker1 # [ 127.123378] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5688worker1 # [ 127.132868] nix-daemon[850]: reaped child process 1442, status = succeeded5689client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5690worker1 # [ 127.143175] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5691client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5692client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5693client # farm-b> LOG-farm-b-on-standby5694worker1 # [ 128.206338] nix-daemon[850]: reaped child process 1451, status = succeeded5695worker1 # [ 128.321205] nix-daemon[850]: accepted connection from pid 1467, user nix-grpc-daemon (trusted)5696worker1 # [ 128.343441] nix-daemon[850]: reaped child process 1470, status = succeeded5697lb # [ 128.366701] niks3-server[1111]: 2026/09/23 05:14:02 INFO Received uploads request method=POST path=/api/pending_closures5698worker1 # [ 128.359612] nix-grpc-daemon[1399]: time=2026-09-23T05:14:02.861Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5699worker1 # [ 128.361661] nix-grpc-daemon[1399]: time=2026-09-23T05:14:02.861Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5700lb # [ 128.409608] niks3-server[1111]: 2026/09/23 05:14:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5701worker1 # [ 128.401416] nix-grpc-daemon[1399]: time=2026-09-23T05:14:02.904Z level=INFO msg="Uploading 1 narinfos"5702lb # [ 128.424530] niks3-server[1111]: 2026/09/23 05:14:02 INFO Signed narinfos id=46 count=15703lb # [ 128.435382] niks3-server[1111]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5704lb # [ 128.448211] niks3-server[1111]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5705lb # [ 128.451524] niks3-server[1111]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5706worker1 # [ 128.445779] nix-grpc-daemon[1399]: time=2026-09-23T05:14:02.949Z level=INFO msg="Upload complete. (292ms)"5707lb # [ 128.465882] niks3-server[1111]: 2026/09/23 05:14:02 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5708lb # [ 128.469826] niks3-server[1111]: 2026/09/23 05:14:02 INFO Completed upload id=465709lb # [ 128.473166] niks3-server[1111]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5710worker1 # [ 128.463760] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48992 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15711worker1 # [ 128.484405] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:02Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05712worker1 # [ 128.491055] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:02Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15713worker1 # [ 128.496659] nix-daemon[850]: reaped child process 1449, status = succeeded5714worker1 # [ 128.522554] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489925715worker1 # [ 128.539238] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489925716worker1 # [ 128.555363] nix-grpc-daemon[1399]: time=2026-09-23T05:14:03.058Z level=INFO msg="All 2 paths already cached"5717client # copying 1 paths to 'lb:50051'...5718client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5719worker1 # [ 128.567464] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489925720worker1 # [ 128.581759] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5721worker1 # [ 128.746893] nix-daemon[850]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)5722worker1 # [ 128.768416] nix-daemon[850]: reaped child process 1479, status = succeeded5723lb # [ 128.791854] niks3-server[1111]: 2026/09/23 05:14:03 INFO Received uploads request method=POST path=/api/pending_closures5724worker1 # [ 128.782467] nix-grpc-daemon[1399]: time=2026-09-23T05:14:03.285Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5725worker1 # [ 128.784235] nix-grpc-daemon[1399]: time=2026-09-23T05:14:03.285Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5726lb # [ 128.824287] niks3-server[1111]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5727worker1 # [ 128.816252] nix-grpc-daemon[1399]: time=2026-09-23T05:14:03.319Z level=INFO msg="Uploading 1 narinfos"5728lb # [ 128.840416] niks3-server[1111]: 2026/09/23 05:14:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5729lb # [ 128.850127] niks3-server[1111]: 2026/09/23 05:14:03 INFO Signed narinfos id=47 count=15730lb # [ 128.855770] niks3-server[1111]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5731lb # [ 128.866082] niks3-server[1111]: 2026/09/23 05:14:03 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5732lb # [ 128.872156] niks3-server[1111]: 2026/09/23 05:14:03 INFO Completed upload id=475733lb # [ 128.875120] niks3-server[1111]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5734worker1 # [ 128.850889] nix-grpc-daemon[1399]: time=2026-09-23T05:14:03.354Z level=INFO msg="Upload complete. (251ms)"5735worker1 # [ 128.860622] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48992 duration_s=0 paths=1 nar_bytes_in=7685736worker1 # [ 128.880186] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489925737worker1 # [ 128.887375] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5738worker1 # [ 128.893242] nix-daemon[850]: reaped child process 1474, status = succeeded5739client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5740worker1 # [ 128.907659] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5741client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5742client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5743client # farm-top> LOG-farm-top-on-standby5744lb # [ 130.278050] envoy[908]: conn peer=[2001:db8:1::4]:52110 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=13940 tx=1040 ms=158245745worker1 # [ 130.418954] nix-daemon[850]: reaped child process 1483, status = succeeded5746worker1 # [ 130.494192] nix-daemon[850]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5747worker1 # [ 130.516488] nix-daemon[850]: reaped child process 1502, status = succeeded5748lb # [ 130.533559] niks3-server[1111]: 2026/09/23 05:14:05 INFO Received uploads request method=POST path=/api/pending_closures5749worker1 # [ 130.528499] nix-grpc-daemon[1399]: time=2026-09-23T05:14:05.029Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5750worker1 # [ 130.530122] nix-grpc-daemon[1399]: time=2026-09-23T05:14:05.029Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5751lb # [ 130.583428] niks3-server[1111]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5752worker1 # [ 130.575623] nix-grpc-daemon[1399]: time=2026-09-23T05:14:05.078Z level=INFO msg="Uploading 1 narinfos"5753lb # [ 130.599348] niks3-server[1111]: 2026/09/23 05:14:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5754lb # [ 130.611634] niks3-server[1111]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5755worker1 # [ 130.603108] nix-grpc-daemon[1399]: time=2026-09-23T05:14:05.106Z level=INFO msg="Upload complete. (283ms)"5756lb # [ 130.623881] niks3-server[1111]: 2026/09/23 05:14:05 INFO Signed narinfos id=48 count=15757lb # [ 130.632629] niks3-server[1111]: 2026/09/23 05:14:05 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5758lb # [ 130.637696] niks3-server[1111]: 2026/09/23 05:14:05 INFO Completed upload id=485759worker1 # [ 130.617322] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48992 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15760lb # [ 130.641750] niks3-server[1111]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5761lb # [ 130.645654] niks3-server[1111]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5762worker1 # [ 130.638309] nix-daemon[850]: reaped child process 1481, status = succeeded5763client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 7.05 seconds)5764client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5765client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds)5766worker2 # [ 130.939047] systemd[1]: Listening on Nix worker-protocol over gRPC.5767worker2 # [ 130.948514] systemd[1]: Starting scheduler-token.service...5768worker2 # [ 131.018612] systemd[1]: scheduler-token.service: Deactivated successfully.5769worker2 # [ 131.022853] systemd[1]: Finished scheduler-token.service.5770worker2 # [ 131.036832] systemd[1]: scheduler-token.service: Consumed 43ms CPU time over 73ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5771worker2 # [ 131.039894] systemd[1]: Starting Nix worker-protocol over gRPC...5772worker2 # [ 131.155575] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5773worker2 # [ 131.158067] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5774worker2 # [ 131.161979] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15775worker2 # [ 131.168061] nix-grpc-daemon[1367]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5776worker2 # [ 131.169556] nix-grpc-daemon[1367]: E0000 00:00:1790140445.665868 1367 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5777lb: must succeed: curl -sf localhost:9901/clusters5778worker2 # [ 131.213610] nix-grpc-daemon[1367]: E0000 00:00:1790140445.672356 1367 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5779worker2 # [ 131.217824] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5780worker2 # [ 131.220097] nix-grpc-daemon[1367]: E0000 00:00:1790140445.680281 1408 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5781worker2 # [ 131.222646] nix-grpc-daemon[1367]: E0000 00:00:1790140445.697815 1408 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5782worker2 # [ 131.224967] nix-grpc-daemon[1367]: E0000 00:00:1790140445.697993 1408 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5783worker2 # [ 131.227322] systemd[1]: Started Nix worker-protocol over gRPC.5784worker2 # [ 131.228178] nix-grpc-daemon[1367]: E0000 00:00:1790140445.716299 1408 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5785worker2 # [ 131.236281] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:05Z level=info event=scheduler_connected addr=lb:500515786worker1 # [ 131.239463] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:05Z level=info event=oidc_jwks provider=mock keys=15787worker1 # [ 131.244383] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:508305788worker1 # [ 131.250972] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05789lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5790worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5791worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5792lb: must succeed: curl -sf localhost:9901/clusters5793lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5794lb: must succeed: curl -sf localhost:9901/clusters5795lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5796worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5797worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5798(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.27 seconds)5799subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5800client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5801client # [ 131.649817] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5802client # Running as unit: rs.service; invocation ID: 48d4e5b2bc20411782e0ed1aa9cf24a45803client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5804worker1 # [ 131.794972] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:489925805client # [ 131.862634] nix[1345]: don't know how to build these paths:5806client # [ 131.863756] nix[1345]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5807worker1 # [ 131.812891] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:06Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05808worker1 # [ 131.820242] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:06Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15809worker1 # [ 131.857660] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489925810worker1 # [ 131.873922] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489925811client # [ 131.946719] nix[1345]: copying 1 paths to 'lb:50051'...5812worker1 # [ 131.890369] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489925813client # [ 131.952112] nix[1345]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5814worker1 # [ 131.905644] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5815worker1 # [ 132.072727] nix-daemon[850]: accepted connection from pid 1515, user nix-grpc-daemon (trusted)5816worker1 # [ 132.092901] nix-daemon[850]: reaped child process 1518, status = succeeded5817lb # [ 132.118156] niks3-server[1111]: 2026/09/23 05:14:06 INFO Received uploads request method=POST path=/api/pending_closures5818worker1 # [ 132.109579] nix-grpc-daemon[1399]: time=2026-09-23T05:14:06.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5819worker1 # [ 132.111248] nix-grpc-daemon[1399]: time=2026-09-23T05:14:06.612Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5820worker1 # [ 132.138928] nix-grpc-daemon[1399]: time=2026-09-23T05:14:06.641Z level=INFO msg="Uploading 1 narinfos"5821lb # [ 132.149195] niks3-server[1111]: 2026/09/23 05:14:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5822lb # [ 132.165857] niks3-server[1111]: 2026/09/23 05:14:06 INFO Signed narinfos id=49 count=15823lb # [ 132.173491] niks3-server[1111]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5824lb # [ 132.181303] niks3-server[1111]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5825lb # [ 132.192818] niks3-server[1111]: 2026/09/23 05:14:06 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5826lb # [ 132.199024] niks3-server[1111]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5827worker1 # [ 132.180880] nix-grpc-daemon[1399]: time=2026-09-23T05:14:06.684Z level=INFO msg="Upload complete. (257ms)"5828lb # [ 132.203499] niks3-server[1111]: 2026/09/23 05:14:06 INFO Completed upload id=495829worker1 # [ 132.191800] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48992 duration_s=0 paths=1 nar_bytes_in=4165830worker1 # [ 132.208620] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489925831worker1 # [ 132.216833] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5832worker1 # [ 132.227569] nix-daemon[850]: reaped child process 1513, status = succeeded5833client # [ 132.290792] nix[1345]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5834worker1 # [ 132.235495] nix-daemon[850]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5835lb: must succeed: curl -sf localhost:9901/clusters5836lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5837lb: must succeed: curl -sf localhost:9901/clusters5838lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5839lb: must succeed: curl -sf localhost:9901/clusters5840lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5841lb: must succeed: curl -sf localhost:9901/clusters5842lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5843worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5844worker2 # [ 134.441676] systemd-journald[264]: Received client request to rotate journal, rotating.5845worker2 # [ 134.461228] systemd-journald[264]: Vacuuming done, freed 0B of archived journals from /var/log/journal/99399b7496db48009c4418bf81fcde58.5846worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.08 seconds)5847worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5848worker1 # [ 134.493068] systemd[1]: Stopping Nix worker-protocol over gRPC...5849worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5850client: must succeed: sleep 5.05851worker1 # [ 134.767141] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:09Z level=info event=draining5852client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5853worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5854worker1 # [ 139.916341] nix-daemon[850]: reaped child process 1522, status = succeeded5855worker1 # [ 140.008425] nix-daemon[850]: accepted connection from pid 1563, user nix-grpc-daemon (trusted)5856worker1 # [ 140.032124] nix-daemon[850]: reaped child process 1566, status = succeeded5857lb # [ 140.051023] niks3-server[1111]: 2026/09/23 05:14:14 INFO Received uploads request method=POST path=/api/pending_closures5858worker1 # [ 140.044480] nix-grpc-daemon[1399]: time=2026-09-23T05:14:14.545Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5859worker1 # [ 140.046246] nix-grpc-daemon[1399]: time=2026-09-23T05:14:14.545Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5860lb # [ 140.102999] niks3-server[1111]: 2026/09/23 05:14:14 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5861lb # [ 140.113028] niks3-server[1111]: 2026/09/23 05:14:14 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5862worker1 # [ 140.104765] nix-grpc-daemon[1399]: time=2026-09-23T05:14:14.607Z level=INFO msg="Uploading 1 narinfos"5863lb # [ 140.126551] niks3-server[1111]: 2026/09/23 05:14:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5864lb # [ 140.134185] niks3-server[1111]: 2026/09/23 05:14:14 INFO Signed narinfos id=50 count=15865lb # [ 140.138908] niks3-server[1111]: 2026/09/23 05:14:14 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5866lb # [ 140.149541] niks3-server[1111]: 2026/09/23 05:14:14 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5867worker1 # [ 140.129514] nix-grpc-daemon[1399]: time=2026-09-23T05:14:14.632Z level=INFO msg="Upload complete. (279ms)"5868lb # [ 140.151949] niks3-server[1111]: 2026/09/23 05:14:14 INFO Completed upload id=505869lb # [ 140.154668] niks3-server[1111]: 2026/09/23 05:14:14 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5870worker1 # [ 140.143708] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48992 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15871worker1 # [ 140.166435] nix-daemon[850]: reaped child process 1520, status = succeeded5872client # [ 140.237299] systemd[1]: rs.service: Deactivated successfully.5873client # [ 140.242512] systemd[1]: rs.service: Consumed 198ms CPU time over 8.588s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.5874lb # [ 140.278051] envoy[908]: conn peer=[2001:db8:1::1]:51674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9636 tx=4485 ms=67985875lb # [ 140.283135] envoy[908]: conn peer=[2001:db8:1::1]:58600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1540 ms=83635876worker1 # [ 140.769471] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:15Z level=info event=signal_exit5877lb # [ 141.089443] niks3-server[1111]: 2026/09/23 05:14:15 INFO lead: released remote=192.168.1.3:571365878lb # [ 141.192321] niks3-server[1111]: 2026/09/23 05:14:15 INFO lead: acquired remote=192.168.1.4:485385879worker2 # [ 141.187823] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:15Z level=info event=scheduler_take_over5880worker2 # [ 142.078375] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:16Z level=info event=scheduler_restarting addr=lb:500515881worker1 # [ 142.074777] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:16Z level=info event=scheduler_restarting5882worker1 # [ 142.080044] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:16Z level=info event=scheduler_restarting addr=lb:500515883worker1 # [ 142.086445] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:508305884worker1 # [ 142.091593] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:16Z level=info event=worker_gone addr=192.168.1.3:500515885worker1 # [ 142.096976] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:508305886worker1 # [ 142.102129] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:16Z level=info event=worker_gone addr=192.168.1.4:500515887worker1 # [ 142.129578] nix-daemon[850]: reaped child process 1375, status = succeeded5888worker1 # [ 142.133313] nix-grpc-daemon[1322]: E0000 00:00:1790140456.636349 1368 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5889worker1 # [ 142.162607] nix-grpc-daemon[1322]: E0000 00:00:1790140456.665692 1368 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5890worker2 # [ 142.281296] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:16Z level=info event=scheduler_connected addr=lb:500515891worker2 # [ 142.286737] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5892worker1 # [ 142.293305] nix-grpc-daemon[1322]: ts=2026-09-23T05:14:16Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5893worker2 # [ 142.488151] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:16Z level=info event=scheduler_connected addr=lb:500515894worker2 # [ 142.506542] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:17Z level=info event=oidc_jwks provider=mock keys=15895worker2 # [ 142.511375] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:591685896worker2 # [ 142.517147] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05897worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.26 seconds)5898client: waiting for success: ! systemctl is-active rs5899client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5900client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5901client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds)5902client: must fail: journalctl -u rs | grep -q 'reconnecting'5903client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5904lb: must succeed: curl -sf localhost:9901/clusters5905lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5906worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5907worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5908worker1 # [ 143.383726] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5909worker1 # [ 143.390565] systemd[1]: Stopped Nix worker-protocol over gRPC.5910worker1 # [ 143.397148] systemd[1]: nix-grpc-daemon.service: Consumed 1.679s CPU time over 32.805s wall clock time, 38.9M memory peak, 48.1K incoming IP traffic, 90K outgoing IP traffic.5911worker1 # [ 143.415988] systemd[1]: Starting Nix worker-protocol over gRPC...5912worker1 # [ 143.557311] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5913worker1 # [ 143.559497] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5914worker1 # [ 143.563513] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15915worker1 # [ 143.569298] nix-grpc-daemon[1573]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5916worker1 # [ 143.570819] nix-grpc-daemon[1573]: E0000 00:00:1790140458.071875 1573 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5917worker1 # [ 143.612718] nix-grpc-daemon[1573]: E0000 00:00:1790140458.078203 1573 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5918worker1 # [ 143.615663] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5919worker1 # [ 143.617697] nix-grpc-daemon[1573]: E0000 00:00:1790140458.084565 1616 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5920worker1 # [ 143.620069] nix-grpc-daemon[1573]: E0000 00:00:1790140458.100335 1616 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5921worker1 # [ 143.624131] nix-grpc-daemon[1573]: E0000 00:00:1790140458.100512 1616 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5922worker1 # [ 143.626565] nix-grpc-daemon[1573]: E0000 00:00:1790140458.115015 1616 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5923worker1 # [ 143.630540] systemd[1]: Started Nix worker-protocol over gRPC.5924worker1 # [ 143.633306] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:18Z level=info event=scheduler_connected addr=lb:500515925worker2 # [ 143.645632] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:18Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:591685926worker2 # [ 143.645958] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05927lb: must succeed: curl -sf localhost:9901/clusters5928lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5929worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5930worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5931(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.99 seconds)5932subtest: scheduler restart mid-build: build finishes, no double build5933client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5934client # [ 144.644913] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5935client # Running as unit: mid.service; invocation ID: f8af968362394d8f9117c8fa3c1a64b35936client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5937worker2 # [ 144.815114] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:591805938worker2 # [ 144.830443] nix-daemon[842]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5939client # [ 144.898292] nix[1381]: don't know how to build these paths:5940client # [ 144.900942] nix[1381]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5941worker2 # [ 144.865686] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:19Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05942worker2 # [ 144.873143] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:19Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15943worker2 # [ 144.919470] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591805944worker2 # [ 144.937339] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 1/5)5945worker2 # [ 145.033781] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 2/5)5946worker2 # [ 145.130286] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 343 ms (attempt 3/5)5947worker2 # [ 145.475445] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 4/5)5948worker2 # [ 145.490066] nix-grpc-daemon[1367]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5949worker2 # [ 145.495059] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5950worker2 # [ 145.509148] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591805951client # [ 145.572083] nix[1381]: copying 1 paths to 'lb:50051'...5952client # [ 145.577771] nix[1381]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5953worker2 # [ 145.529043] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591805954worker2 # [ 145.544725] nix-daemon[842]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5955worker2 # [ 145.581587] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:20Z level=info event=niks3_push_started pid=14965956worker2 # [ 145.730018] nix-grpc-daemon[1496]: time=2026-09-23T05:14:20.228Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5957worker2 # [ 145.847553] nix-daemon[842]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)5958lb # [ 145.887711] niks3-server[1111]: 2026/09/23 05:14:20 INFO Received uploads request method=POST path=/api/pending_closures5959worker2 # [ 145.881984] nix-daemon[842]: reaped child process 1512, status = succeeded5960worker2 # [ 145.890564] nix-grpc-daemon[1496]: time=2026-09-23T05:14:20.388Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5961worker2 # [ 145.895561] nix-grpc-daemon[1496]: time=2026-09-23T05:14:20.388Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5962lb # [ 145.928643] niks3-server[1111]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5963worker2 # [ 145.929785] nix-grpc-daemon[1496]: time=2026-09-23T05:14:20.428Z level=INFO msg="Uploading 1 narinfos"5964lb # [ 145.941412] niks3-server[1111]: 2026/09/23 05:14:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5965lb # [ 145.951479] niks3-server[1111]: 2026/09/23 05:14:20 INFO Signed narinfos id=51 count=15966lb # [ 145.957289] niks3-server[1111]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5967worker2 # [ 145.957273] nix-grpc-daemon[1496]: time=2026-09-23T05:14:20.456Z level=INFO msg="Upload complete. (223ms)"5968lb # [ 145.966827] niks3-server[1111]: 2026/09/23 05:14:20 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5969lb # [ 145.974476] niks3-server[1111]: 2026/09/23 05:14:20 INFO Completed upload id=515970worker2 # [ 145.963562] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59180 duration_s=0 paths=1 nar_bytes_in=4245971lb # [ 145.978396] niks3-server[1111]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5972worker2 # [ 145.974110] nix-grpc-daemon[1367]: ts=2026-09-23T05:14:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591805973worker2 # [ 145.978558] nix-daemon[842]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5974worker2 # [ 145.982252] nix-daemon[842]: reaped child process 1494, status = succeeded5975worker2 # [ 145.990969] nix-daemon[842]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5976client # [ 146.041603] nix[1381]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5977lb: must succeed: curl -sf localhost:9901/clusters5978lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5979lb: must succeed: curl -sf localhost:9901/clusters5980lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5981worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5982worker2 # [ 147.365755] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1367 (nix-grpc-daemon) on client request.5983worker2 # [ 147.367384] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5984lb # [ 147.401471] niks3-server[1111]: 2026/09/23 05:14:21 INFO lead: released remote=192.168.1.4:485385985worker2 # [ 147.400485] systemd[1]: Starting scheduler-token.service...5986worker2 # [ 147.401353] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5987worker2 # [ 147.402843] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5988client # [ 147.450674] nix[1381]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5989worker1 # [ 147.392380] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:21Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5990client # [ 147.457304] nix[1381]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5991worker2 # [ 147.419388] systemd[1]: nix-grpc-daemon.service: Consumed 526ms CPU time over 16.368s wall clock time, 34.7M memory peak, 15.2K incoming IP traffic, 25.3K outgoing IP traffic.5992worker2 # [ 147.451565] nix-daemon[842]: reaped child process 1486, status = succeeded5993worker2 # [ 147.453527] nix-daemon[842]: reaped child process 1518, status = succeeded5994worker2 # [ 147.460432] nix-daemon[1520]: unexpected Nix daemon error: error: interrupted by the user5995worker2 # [ 147.471687] nix-daemon[842]: reaped child process 1520, status = failed with exit code 15996worker2 # [ 147.495616] systemd[1]: scheduler-token.service: Deactivated successfully.5997worker2 # [ 147.500155] systemd[1]: Finished scheduler-token.service.5998worker2 # [ 147.500997] systemd[1]: scheduler-token.service: Consumed 26ms CPU time over 98ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5999worker2 # [ 147.502904] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6000worker2 # [ 147.506500] systemd[1]: Starting scheduler-token.service...6001worker2 # [ 147.553580] systemd[1]: scheduler-token.service: Deactivated successfully.6002worker2 # [ 147.555349] systemd[1]: Finished scheduler-token.service.6003worker2 # [ 147.558997] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 49ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6004worker2 # [ 147.567756] systemd[1]: Starting Nix worker-protocol over gRPC...6005worker1 # [ 147.592929] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:22Z level=info event=scheduler_connected addr=lb:500516006worker2 # [ 147.669843] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6007worker2 # [ 147.672195] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6008worker2 # [ 147.676293] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16009worker2 # [ 147.682911] nix-grpc-daemon[1559]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6010worker2 # [ 147.704189] nix-grpc-daemon[1559]: E0000 00:00:1790140462.180082 1559 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6011lb # [ 147.714589] niks3-server[1111]: 2026/09/23 05:14:22 INFO lead: acquired remote=192.168.1.4:549646012worker2 # [ 147.706703] nix-grpc-daemon[1559]: E0000 00:00:1790140462.182209 1559 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6013worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.40 seconds)6014client: must succeed: sleep 5.06015worker2 # [ 147.745733] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6016worker2 # [ 147.748658] nix-grpc-daemon[1559]: E0000 00:00:1790140462.202145 1602 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6017worker2 # [ 147.751590] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=scheduler_take_over6018worker2 # [ 147.752987] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:592106019worker2 # [ 147.758137] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06020worker2 # [ 147.762759] nix-grpc-daemon[1559]: E0000 00:00:1790140462.228895 1602 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6021worker2 # [ 147.766327] nix-grpc-daemon[1559]: E0000 00:00:1790140462.229087 1602 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6022worker2 # [ 147.769448] nix-grpc-daemon[1559]: E0000 00:00:1790140462.244189 1602 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6023worker2 # [ 147.772456] systemd[1]: Started Nix worker-protocol over gRPC.6024worker2 # [ 147.773788] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=scheduler_connected addr=lb:500516025worker2 # [ 147.777726] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=oidc_jwks provider=mock keys=16026worker2 # [ 147.779209] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:592106027worker2 # [ 147.781356] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06028worker2 # [ 147.914862] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06029worker2 # [ 147.921916] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:22Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16030worker1 # [ 147.970298] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599686031worker1 # [ 147.986292] nix-daemon[850]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6032worker1 # [ 148.007958] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5)6033worker1 # [ 148.027311] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 2/5)6034worker1 # [ 148.109254] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 243 ms (attempt 3/5)6035worker1 # [ 148.354935] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 252 ms (attempt 4/5)6036worker1 # [ 148.610948] nix-grpc-daemon[1573]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6037worker1 # [ 148.617991] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6038worker1 # [ 148.636526] nix-grpc-daemon[1573]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...6039worker1 # [ 148.667637] nix-daemon[850]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6040client # [ 148.747680] nix[1381]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6041worker1 # [ 148.692119] nix-daemon[850]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6042lb # [ 150.278835] envoy[908]: conn peer=[2001:db8:1::3]:53972 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2717 tx=949 ms=314726043lb # [ 150.284409] envoy[908]: conn peer=[2001:db8:1::4]:51394 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2722 tx=393 ms=161776044client: (finished: must succeed: sleep 5.0, in 5.06 seconds)6045client: waiting for success: ! systemctl is-active mid6046worker1 # [ 153.123600] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:27Z level=info event=niks3_push_started pid=16766047worker1 # [ 153.178080] nix-daemon[850]: reaped child process 1651, status = succeeded6048worker1 # [ 153.304618] nix-grpc-daemon[1676]: time=2026-09-23T05:14:27.807Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6049worker1 # [ 153.417698] nix-daemon[850]: accepted connection from pid 1689, user nix-grpc-daemon (trusted)6050worker1 # [ 153.438626] nix-daemon[850]: reaped child process 1692, status = succeeded6051lb # [ 153.461337] niks3-server[1111]: 2026/09/23 05:14:27 INFO Received uploads request method=POST path=/api/pending_closures6052worker1 # [ 153.451434] nix-grpc-daemon[1676]: time=2026-09-23T05:14:27.953Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6053worker1 # [ 153.455166] nix-grpc-daemon[1676]: time=2026-09-23T05:14:27.953Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6054lb # [ 153.499902] niks3-server[1111]: 2026/09/23 05:14:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6055lb # [ 153.529417] niks3-server[1111]: 2026/09/23 05:14:28 INFO Signed narinfos id=52 count=16056worker1 # [ 153.511659] nix-grpc-daemon[1676]: time=2026-09-23T05:14:28.014Z level=INFO msg="Uploading 1 narinfos"6057lb # [ 153.536330] niks3-server[1111]: 2026/09/23 05:14:28 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6058lb # [ 153.547185] niks3-server[1111]: 2026/09/23 05:14:28 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6059lb # [ 153.555392] niks3-server[1111]: 2026/09/23 05:14:28 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6060lb # [ 153.562210] niks3-server[1111]: 2026/09/23 05:14:28 INFO Completed upload id=526061worker1 # [ 153.541841] nix-grpc-daemon[1676]: time=2026-09-23T05:14:28.045Z level=INFO msg="Upload complete. (232ms)"6062lb # [ 153.567660] niks3-server[1111]: 2026/09/23 05:14:28 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6063lb # [ 153.571945] niks3-server[1111]: 2026/09/23 05:14:28 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6064worker1 # [ 153.552340] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59968 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16065worker1 # [ 153.573343] nix-daemon[850]: reaped child process 1649, status = succeeded6066client # [ 153.656243] systemd[1]: mid.service: Deactivated successfully.6067client # [ 153.660554] systemd[1]: mid.service: Consumed 210ms CPU time over 9.016s wall clock time, 43.5M memory peak, 8.7K incoming IP traffic, 10.5K outgoing IP traffic.6068client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)6069client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6070client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)6071(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.69 seconds)6072subtest: interrupting the client stops the build on the worker6073client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6074client # [ 154.308552] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6075client # Running as unit: intr.service; invocation ID: 8df96aefc6e84bc2add6dca953d10a0d6076client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6077worker1 # [ 154.423226] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:599686078client # [ 154.490486] nix[1417]: don't know how to build these paths:6079client # [ 154.491578] nix[1417]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6080worker2 # [ 154.459241] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:28Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06081worker2 # [ 154.459644] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:28Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16082worker2 # [ 154.518222] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866083worker2 # [ 154.533091] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6084worker2 # [ 154.558216] nix-grpc-daemon[1559]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5)6085worker2 # [ 154.576880] nix-grpc-daemon[1559]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 2/5)6086worker2 # [ 154.650675] nix-grpc-daemon[1559]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 3/5)6087worker2 # [ 154.763805] nix-grpc-daemon[1559]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 4/5)6088worker2 # [ 154.804371] nix-grpc-daemon[1559]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6089worker2 # [ 154.810480] nix-grpc-daemon[1559]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6090worker2 # [ 154.828540] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595866091client # [ 154.893576] nix[1417]: copying 1 paths to 'lb:50051'...6092worker2 # [ 154.843924] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595866093client # [ 154.898178] nix[1417]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6094worker2 # [ 154.858779] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6095worker2 # [ 154.896269] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:29Z level=info event=niks3_push_started pid=16366096worker2 # [ 155.035151] nix-grpc-daemon[1636]: time=2026-09-23T05:14:29.534Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6097worker2 # [ 155.141635] nix-daemon[842]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6098worker2 # [ 155.162845] nix-daemon[842]: reaped child process 1651, status = succeeded6099lb # [ 155.174994] niks3-server[1111]: 2026/09/23 05:14:29 INFO Received uploads request method=POST path=/api/pending_closures6100worker2 # [ 155.179141] nix-grpc-daemon[1636]: time=2026-09-23T05:14:29.677Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6101worker2 # [ 155.182612] nix-grpc-daemon[1636]: time=2026-09-23T05:14:29.677Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6102lb # [ 155.218111] niks3-server[1111]: 2026/09/23 05:14:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6103worker2 # [ 155.220982] nix-grpc-daemon[1636]: time=2026-09-23T05:14:29.719Z level=INFO msg="Uploading 1 narinfos"6104lb # [ 155.234759] niks3-server[1111]: 2026/09/23 05:14:29 INFO Signed narinfos id=53 count=16105lb # [ 155.241735] niks3-server[1111]: 2026/09/23 05:14:29 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6106lb # [ 155.252328] niks3-server[1111]: 2026/09/23 05:14:29 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6107lb # [ 155.255831] niks3-server[1111]: 2026/09/23 05:14:29 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6108worker2 # [ 155.252321] nix-grpc-daemon[1636]: time=2026-09-23T05:14:29.751Z level=INFO msg="Upload complete. (212ms)"6109lb # [ 155.263961] niks3-server[1111]: 2026/09/23 05:14:29 INFO Completed upload id=536110lb # [ 155.266221] niks3-server[1111]: 2026/09/23 05:14:29 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6111worker2 # [ 155.260111] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=0 paths=1 nar_bytes_in=4246112worker2 # [ 155.274919] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866113worker2 # [ 155.281379] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6114worker2 # [ 155.286719] nix-daemon[842]: reaped child process 1634, status = succeeded6115client # [ 155.346204] nix[1417]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6116worker2 # [ 155.299512] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6117client: must succeed: systemctl kill -s INT intr6118worker2 # [ 156.750568] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:31Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6119client # [ 156.797348] systemd[1]: intr.service: Sent signal SIGINT to main process 1417 (nix) on client request.6120client # [ 156.807087] nix[1417]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6121client # [ 156.818556] nix[1417]: error: interrupted by the user6122client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6123client # [ 156.845234] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6124client # [ 156.850811] systemd[1]: intr.service: Failed with result 'exit-code'.6125client # [ 156.853646] systemd[1]: intr.service: Consumed 169ms CPU time over 2.540s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6126worker2 # [ 156.921554] nix-daemon[1655]: unexpected Nix daemon error: error: interrupted by the user6127worker2 # [ 156.932640] nix-daemon[842]: reaped child process 1653, status = succeeded6128worker2 # [ 156.934345] nix-daemon[842]: reaped child process 1655, status = failed with exit code 16129(finished: subtest: interrupting the client stops the build on the worker, in 3.82 seconds)6130subtest: a deploy mid-build drains: the build finishes, the new generation takes over6131client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6132client # [ 158.157931] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6133client # Running as unit: sw.service; invocation ID: c59c77d087d342488b0b4e47f82ccf096134client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)6135worker1 # [ 158.304579] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:599686136client # [ 158.370332] nix[1438]: don't know how to build these paths:6137client # [ 158.371381] nix[1438]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6138worker2 # [ 158.335086] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:32Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06139worker2 # [ 158.341108] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:32Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16140worker1 # [ 158.372929] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599686141worker1 # [ 158.384773] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599686142client # [ 158.455354] nix[1438]: copying 1 paths to 'lb:50051'...6143worker1 # [ 158.396882] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599686144client # [ 158.458030] nix[1438]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6145worker1 # [ 158.412367] nix-daemon[850]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6146worker1 # [ 158.570894] nix-daemon[850]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6147worker1 # [ 158.590649] nix-daemon[850]: reaped child process 1723, status = succeeded6148lb # [ 158.615428] niks3-server[1111]: 2026/09/23 05:14:33 INFO Received uploads request method=POST path=/api/pending_closures6149worker1 # [ 158.608570] nix-grpc-daemon[1676]: time=2026-09-23T05:14:33.110Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6150worker1 # [ 158.611367] nix-grpc-daemon[1676]: time=2026-09-23T05:14:33.110Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6151lb # [ 158.647578] niks3-server[1111]: 2026/09/23 05:14:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6152worker1 # [ 158.641390] nix-grpc-daemon[1676]: time=2026-09-23T05:14:33.143Z level=INFO msg="Uploading 1 narinfos"6153lb # [ 158.663011] niks3-server[1111]: 2026/09/23 05:14:33 INFO Signed narinfos id=54 count=16154lb # [ 158.671600] niks3-server[1111]: 2026/09/23 05:14:33 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6155lb # [ 158.684388] niks3-server[1111]: 2026/09/23 05:14:33 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6156lb # [ 158.697615] niks3-server[1111]: 2026/09/23 05:14:33 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6157lb # [ 158.703553] niks3-server[1111]: 2026/09/23 05:14:33 INFO Completed upload id=546158worker1 # [ 158.678654] nix-grpc-daemon[1676]: time=2026-09-23T05:14:33.181Z level=INFO msg="Upload complete. (249ms)"6159lb # [ 158.707465] niks3-server[1111]: 2026/09/23 05:14:33 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6160worker1 # [ 158.689468] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59968 duration_s=0 paths=1 nar_bytes_in=4166161worker1 # [ 158.704335] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599686162worker1 # [ 158.711449] nix-daemon[850]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6163worker1 # [ 158.717034] nix-daemon[850]: reaped child process 1718, status = succeeded6164client # [ 158.786721] nix[1438]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6165worker1 # [ 158.731890] nix-daemon[850]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6166lb # [ 160.278711] envoy[908]: conn peer=[2001:db8:1::1]:33554 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1966 ms=87806167lb # [ 160.284557] envoy[908]: conn peer=[2001:db8:1::1]:39032 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=827 ms=23496168worker1: must succeed: systemctl reload nix-grpc-daemon.service6169worker1 # [ 160.698016] systemd[1]: Reloading Nix worker-protocol over gRPC...6170worker1 # [ 160.744119] systemd[1]: Reloaded Nix worker-protocol over gRPC.6171worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6172worker1: must succeed: pgrep -f 'read -t [9]0 x'6173worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6174client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26175client # don't know how to build these paths:6176client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6177worker2 # [ 161.024472] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:595866178worker2 # [ 161.030726] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595866179worker2 # [ 161.056146] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:35Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06180worker2 # [ 161.062042] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:35Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16181worker2 # [ 161.106925] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866182worker2 # [ 161.127190] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595866183client # copying 1 paths to 'lb:50051'...6184worker2 # [ 161.142841] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595866185client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6186worker2 # [ 161.166518] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6187worker2 # [ 161.327926] nix-daemon[842]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6188worker2 # [ 161.347591] nix-daemon[842]: reaped child process 1713, status = succeeded6189lb # [ 161.365816] niks3-server[1111]: 2026/09/23 05:14:35 INFO Received uploads request method=POST path=/api/pending_closures6190worker2 # [ 161.364349] nix-grpc-daemon[1636]: time=2026-09-23T05:14:35.862Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6191worker2 # [ 161.366136] nix-grpc-daemon[1636]: time=2026-09-23T05:14:35.862Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6192lb # [ 161.399167] niks3-server[1111]: 2026/09/23 05:14:35 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6193lb # [ 161.417243] niks3-server[1111]: 2026/09/23 05:14:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6194worker2 # [ 161.413267] nix-grpc-daemon[1636]: time=2026-09-23T05:14:35.911Z level=INFO msg="Uploading 1 narinfos"6195lb # [ 161.433505] niks3-server[1111]: 2026/09/23 05:14:35 INFO Signed narinfos id=55 count=16196lb # [ 161.447989] niks3-server[1111]: 2026/09/23 05:14:35 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6197lb # [ 161.459843] niks3-server[1111]: 2026/09/23 05:14:35 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6198worker2 # [ 161.457194] nix-grpc-daemon[1636]: time=2026-09-23T05:14:35.956Z level=INFO msg="Upload complete. (273ms)"6199lb # [ 161.469365] niks3-server[1111]: 2026/09/23 05:14:35 INFO Completed upload id=556200lb # [ 161.472233] niks3-server[1111]: 2026/09/23 05:14:35 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6201worker2 # [ 161.468314] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=0 paths=1 nar_bytes_in=5286202worker2 # [ 161.487042] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866203worker2 # [ 161.499725] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6204worker2 # [ 161.503685] nix-daemon[842]: reaped child process 1708, status = succeeded6205client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6206client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6207worker2 # [ 161.522328] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6208worker1 # [ 161.585038] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:36Z level=info event=draining6209client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6210client # farm-a> LOG-farm-a-during-drain6211worker2 # [ 163.193939] nix-daemon[842]: reaped child process 1717, status = succeeded6212worker2 # [ 163.313762] nix-daemon[842]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6213worker2 # [ 163.333338] nix-daemon[842]: reaped child process 1736, status = succeeded6214lb # [ 163.348784] niks3-server[1111]: 2026/09/23 05:14:37 INFO Received uploads request method=POST path=/api/pending_closures6215worker2 # [ 163.352478] nix-grpc-daemon[1636]: time=2026-09-23T05:14:37.850Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6216worker2 # [ 163.354421] nix-grpc-daemon[1636]: time=2026-09-23T05:14:37.850Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6217lb # [ 163.394545] niks3-server[1111]: 2026/09/23 05:14:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6218worker2 # [ 163.393083] nix-grpc-daemon[1636]: time=2026-09-23T05:14:37.892Z level=INFO msg="Uploading 1 narinfos"6219lb # [ 163.409768] niks3-server[1111]: 2026/09/23 05:14:37 INFO Signed narinfos id=56 count=16220lb # [ 163.420187] niks3-server[1111]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6221lb # [ 163.431057] niks3-server[1111]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6222lb # [ 163.438515] niks3-server[1111]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6223worker2 # [ 163.434205] nix-grpc-daemon[1636]: time=2026-09-23T05:14:37.933Z level=INFO msg="Upload complete. (298ms)"6224lb # [ 163.444811] niks3-server[1111]: 2026/09/23 05:14:37 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6225lb # [ 163.453149] niks3-server[1111]: 2026/09/23 05:14:37 INFO Completed upload id=566226lb # [ 163.456000] niks3-server[1111]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6227worker2 # [ 163.449424] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16228worker2 # [ 163.471476] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:37Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06229worker2 # [ 163.476445] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:37Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16230worker2 # [ 163.481399] nix-daemon[842]: reaped child process 1715, status = succeeded6231worker2 # [ 163.498355] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866232worker2 # [ 163.513384] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595866233worker2 # [ 163.527679] nix-grpc-daemon[1636]: time=2026-09-23T05:14:38.027Z level=INFO msg="All 1 paths already cached"6234client # copying 1 paths to 'lb:50051'...6235worker2 # [ 163.538654] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595866236client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6237worker2 # [ 163.561034] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6238worker2 # [ 163.724145] nix-daemon[842]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6239worker2 # [ 163.745326] nix-daemon[842]: reaped child process 1745, status = succeeded6240lb # [ 163.753854] niks3-server[1111]: 2026/09/23 05:14:38 INFO Received uploads request method=POST path=/api/pending_closures6241worker2 # [ 163.752266] nix-grpc-daemon[1636]: time=2026-09-23T05:14:38.251Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6242worker2 # [ 163.754832] nix-grpc-daemon[1636]: time=2026-09-23T05:14:38.251Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6243lb # [ 163.787835] niks3-server[1111]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6244worker2 # [ 163.789019] nix-grpc-daemon[1636]: time=2026-09-23T05:14:38.288Z level=INFO msg="Uploading 1 narinfos"6245lb # [ 163.804606] niks3-server[1111]: 2026/09/23 05:14:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6246lb # [ 163.815641] niks3-server[1111]: 2026/09/23 05:14:38 INFO Signed narinfos id=57 count=16247lb # [ 163.821922] niks3-server[1111]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6248lb # [ 163.828876] niks3-server[1111]: 2026/09/23 05:14:38 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6249lb # [ 163.837898] niks3-server[1111]: 2026/09/23 05:14:38 INFO Completed upload id=576250worker2 # [ 163.826306] nix-grpc-daemon[1636]: time=2026-09-23T05:14:38.325Z level=INFO msg="Upload complete. (247ms)"6251lb # [ 163.840497] niks3-server[1111]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6252worker2 # [ 163.836322] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=0 paths=1 nar_bytes_in=6406253worker2 # [ 163.856592] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866254worker2 # [ 163.863676] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6255worker2 # [ 163.870107] nix-daemon[842]: reaped child process 1740, status = succeeded6256client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6257worker2 # [ 163.883492] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6258client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6259client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6260client # farm-b> LOG-farm-b-during-drain6261worker2 # [ 165.168214] nix-daemon[842]: reaped child process 1749, status = succeeded6262worker2 # [ 165.267185] nix-daemon[842]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6263worker2 # [ 165.285167] nix-daemon[842]: reaped child process 1768, status = succeeded6264lb # [ 165.302109] niks3-server[1111]: 2026/09/23 05:14:39 INFO Received uploads request method=POST path=/api/pending_closures6265worker2 # [ 165.302465] nix-grpc-daemon[1636]: time=2026-09-23T05:14:39.800Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6266worker2 # [ 165.304093] nix-grpc-daemon[1636]: time=2026-09-23T05:14:39.800Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6267lb # [ 165.345931] niks3-server[1111]: 2026/09/23 05:14:39 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6268worker2 # [ 165.347502] nix-grpc-daemon[1636]: time=2026-09-23T05:14:39.846Z level=INFO msg="Uploading 1 narinfos"6269lb # [ 165.359516] niks3-server[1111]: 2026/09/23 05:14:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6270lb # [ 165.369469] niks3-server[1111]: 2026/09/23 05:14:39 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6271lb # [ 165.381318] niks3-server[1111]: 2026/09/23 05:14:39 INFO Signed narinfos id=58 count=16272lb # [ 165.383205] niks3-server[1111]: 2026/09/23 05:14:39 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6273lb # [ 165.388072] niks3-server[1111]: 2026/09/23 05:14:39 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6274lb # [ 165.392374] niks3-server[1111]: 2026/09/23 05:14:39 INFO Completed upload id=586275worker2 # [ 165.382513] nix-grpc-daemon[1636]: time=2026-09-23T05:14:39.881Z level=INFO msg="Upload complete. (285ms)"6276lb # [ 165.394697] niks3-server[1111]: 2026/09/23 05:14:39 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6277worker2 # [ 165.404833] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16278worker2 # [ 165.416296] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:39Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06279worker2 # [ 165.424304] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:39Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16280worker2 # [ 165.429458] nix-daemon[842]: reaped child process 1747, status = succeeded6281worker2 # [ 165.446380] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866282worker2 # [ 165.467257] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595866283worker2 # [ 165.481268] nix-grpc-daemon[1636]: time=2026-09-23T05:14:39.980Z level=INFO msg="All 2 paths already cached"6284client # copying 1 paths to 'lb:50051'...6285worker2 # [ 165.492112] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595866286client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6287worker2 # [ 165.514722] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6288worker2 # [ 165.674214] nix-daemon[842]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6289worker2 # [ 165.692820] nix-daemon[842]: reaped child process 1777, status = succeeded6290lb # [ 165.708079] niks3-server[1111]: 2026/09/23 05:14:40 INFO Received uploads request method=POST path=/api/pending_closures6291worker2 # [ 165.706396] nix-grpc-daemon[1636]: time=2026-09-23T05:14:40.205Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6292worker2 # [ 165.708228] nix-grpc-daemon[1636]: time=2026-09-23T05:14:40.205Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6293lb # [ 165.736824] niks3-server[1111]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6294lb # [ 165.747574] niks3-server[1111]: 2026/09/23 05:14:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6295worker2 # [ 165.744741] nix-grpc-daemon[1636]: time=2026-09-23T05:14:40.243Z level=INFO msg="Uploading 1 narinfos"6296lb # [ 165.762526] niks3-server[1111]: 2026/09/23 05:14:40 INFO Signed narinfos id=59 count=16297lb # [ 165.770386] niks3-server[1111]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6298lb # [ 165.782381] niks3-server[1111]: 2026/09/23 05:14:40 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6299lb # [ 165.788965] niks3-server[1111]: 2026/09/23 05:14:40 INFO Completed upload id=596300lb # [ 165.790989] niks3-server[1111]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6301worker2 # [ 165.777064] nix-grpc-daemon[1636]: time=2026-09-23T05:14:40.276Z level=INFO msg="Upload complete. (244ms)"6302worker2 # [ 165.788431] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=0 paths=1 nar_bytes_in=7686303worker2 # [ 165.804955] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866304worker2 # [ 165.812740] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6305worker2 # [ 165.824408] nix-daemon[842]: reaped child process 1772, status = succeeded6306client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6307client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6308worker2 # [ 165.833836] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6309client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6310client # farm-top> LOG-farm-top-during-drain6311worker2 # [ 166.724382] nix-daemon[842]: reaped child process 1781, status = succeeded6312worker2 # [ 166.810658] nix-daemon[842]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6313worker2 # [ 166.832136] nix-daemon[842]: reaped child process 1800, status = succeeded6314lb # [ 166.846841] niks3-server[1111]: 2026/09/23 05:14:41 INFO Received uploads request method=POST path=/api/pending_closures6315worker2 # [ 166.848751] nix-grpc-daemon[1636]: time=2026-09-23T05:14:41.346Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6316worker2 # [ 166.850655] nix-grpc-daemon[1636]: time=2026-09-23T05:14:41.346Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6317lb # [ 166.886510] niks3-server[1111]: 2026/09/23 05:14:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6318worker2 # [ 166.886246] nix-grpc-daemon[1636]: time=2026-09-23T05:14:41.385Z level=INFO msg="Uploading 1 narinfos"6319lb # [ 166.901414] niks3-server[1111]: 2026/09/23 05:14:41 INFO Signed narinfos id=60 count=16320lb # [ 166.909335] niks3-server[1111]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6321lb # [ 166.920991] niks3-server[1111]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6322worker2 # [ 166.917252] nix-grpc-daemon[1636]: time=2026-09-23T05:14:41.416Z level=INFO msg="Upload complete. (273ms)"6323lb # [ 166.929320] niks3-server[1111]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6324lb # [ 166.932695] niks3-server[1111]: 2026/09/23 05:14:41 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6325lb # [ 166.937365] niks3-server[1111]: 2026/09/23 05:14:41 INFO Completed upload id=606326lb # [ 166.940276] niks3-server[1111]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6327worker2 # [ 166.929106] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16328worker2 # [ 166.951125] nix-daemon[842]: reaped child process 1779, status = succeeded6329client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.18 seconds)6330client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6331client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6332worker1: must succeed: pkill -f 'read -t [9]0 x'6333worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6334client: waiting for success: ! systemctl is-active sw6335worker1 # [ 167.404892] nix-daemon[850]: reaped child process 1727, status = succeeded6336worker1 # [ 167.515028] nix-daemon[850]: accepted connection from pid 1768, user nix-grpc-daemon (trusted)6337worker1 # [ 167.535122] nix-daemon[850]: reaped child process 1771, status = succeeded6338lb # [ 167.556710] niks3-server[1111]: 2026/09/23 05:14:42 INFO Received uploads request method=POST path=/api/pending_closures6339worker1 # [ 167.547730] nix-grpc-daemon[1676]: time=2026-09-23T05:14:42.050Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6340worker1 # [ 167.549501] nix-grpc-daemon[1676]: time=2026-09-23T05:14:42.050Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6341lb # [ 167.597160] niks3-server[1111]: 2026/09/23 05:14:42 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6342worker1 # [ 167.589010] nix-grpc-daemon[1676]: time=2026-09-23T05:14:42.091Z level=INFO msg="Uploading 1 narinfos"6343lb # [ 167.611209] niks3-server[1111]: 2026/09/23 05:14:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6344lb # [ 167.620272] niks3-server[1111]: 2026/09/23 05:14:42 INFO Signed narinfos id=61 count=16345lb # [ 167.628568] niks3-server[1111]: 2026/09/23 05:14:42 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6346lb # [ 167.636469] niks3-server[1111]: 2026/09/23 05:14:42 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6347worker1 # [ 167.614271] nix-grpc-daemon[1676]: time=2026-09-23T05:14:42.117Z level=INFO msg="Upload complete. (264ms)"6348lb # [ 167.639371] niks3-server[1111]: 2026/09/23 05:14:42 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6349lb # [ 167.645861] niks3-server[1111]: 2026/09/23 05:14:42 INFO Completed upload id=616350lb # [ 167.648560] niks3-server[1111]: 2026/09/23 05:14:42 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6351worker1 # [ 167.626536] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59968 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16352worker1 # [ 167.645612] nix-daemon[850]: reaped child process 1725, status = succeeded6353client # [ 167.722815] systemd[1]: sw.service: Deactivated successfully.6354client # [ 167.728211] systemd[1]: sw.service: Consumed 195ms CPU time over 9.566s wall clock time, 39.2M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6355client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6356client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6357client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6358worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6359worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.06 seconds)6360lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6361lb: must succeed: curl -sf localhost:9901/clusters6362lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6363worker1 # [ 168.589731] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:43Z level=info event=signal_exit6364lb: must succeed: curl -sf localhost:9901/clusters6365lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6366lb # [ 170.278345] envoy[908]: conn peer=[2001:db8:1::1]:42220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9621 tx=4401 ms=59346367lb # [ 170.283365] envoy[908]: conn peer=[2001:db8:1::1]:39042 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1513 ms=93406368lb: must succeed: curl -sf localhost:9901/clusters6369worker2 # [ 170.602016] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:592106370worker1 # [ 170.595740] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:45Z level=info event=scheduler_restarting6371worker2 # [ 170.602613] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:45Z level=info event=worker_gone addr=192.168.1.3:500516372worker1 # [ 170.619746] nix-grpc-daemon[1573]: ts=2026-09-23T05:14:45Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6373lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6374worker1 # [ 170.642953] nix-daemon[850]: reaped child process 1642, status = succeeded6375worker1 # [ 171.463567] nix-grpc-daemon[1573]: E0000 00:00:1790140485.966611 1619 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6376worker1 # [ 171.470542] nix-grpc-daemon[1573]: E0000 00:00:1790140485.966829 1619 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6377worker1 # [ 171.611234] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6378worker1 # [ 171.631452] systemd[1]: nix-grpc-daemon.service: Consumed 864ms CPU time over 28.204s wall clock time, 35.8M memory peak, 21.9K incoming IP traffic, 28.5K outgoing IP traffic.6379lb: must succeed: curl -sf localhost:9901/clusters6380lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6381worker1 # [ 172.608809] systemd[1]: Starting Nix worker-protocol over gRPC...6382lb: must succeed: curl -sf localhost:9901/clusters6383lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6384worker1 # [ 172.776809] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6385worker1 # [ 172.778815] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6386worker1 # [ 172.783004] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16387worker1 # [ 172.788057] nix-grpc-daemon[1783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6388worker1 # [ 172.789524] nix-grpc-daemon[1783]: E0000 00:00:1790140487.291083 1783 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6389worker1 # [ 172.832075] nix-grpc-daemon[1783]: E0000 00:00:1790140487.297358 1783 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6390worker1 # [ 172.834387] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6391worker1 # [ 172.841942] nix-grpc-daemon[1783]: E0000 00:00:1790140487.309132 1823 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6392worker1 # [ 172.844799] nix-grpc-daemon[1783]: E0000 00:00:1790140487.326079 1823 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6393worker1 # [ 172.847241] nix-grpc-daemon[1783]: E0000 00:00:1790140487.326214 1823 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6394worker1 # [ 172.852886] systemd[1]: Started Nix worker-protocol over gRPC.6395worker1 # [ 172.855485] nix-grpc-daemon[1783]: E0000 00:00:1790140487.344650 1823 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6396worker1 # [ 172.858019] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:47Z level=info event=scheduler_connected addr=lb:500516397worker2 # [ 172.866312] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:592106398worker2 # [ 172.872960] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06399lb: must succeed: curl -sf localhost:9901/clusters6400lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6401lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 5.42 seconds)6402lb: must succeed: curl -sf localhost:9901/clusters6403lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6404worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6405worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6406(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.92 seconds)6407subtest: queue gauges show a burst while every build blocks6408lb: must succeed: curl -sf localhost:9901/clusters6409lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6410lb: must succeed: curl -sf localhost:9901/clusters6411lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6412worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6413worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6414client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6415client # [ 174.249351] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6416client # Running as unit: burst.service; invocation ID: 256dca65abf1442f9f957603798555506417client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6418lb: must succeed: curl -sf localhost:9901/clusters6419lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6420lb: must succeed: curl -sf localhost:9901/clusters6421lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6422worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6423client # [ 174.457519] nix[1494]: don't know how to build these paths:6424client # [ 174.458505] nix[1494]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6425client # [ 174.459551] nix[1494]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6426worker2 # [ 174.407582] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:595866427client # [ 174.460657] nix[1494]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6428client # [ 174.462150] nix[1494]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6429worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6430worker2 # [ 174.422807] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:48Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06431worker2 # [ 174.428923] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:48Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.4:50051 clients=16432worker2 # [ 174.434006] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:48Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06433worker2 # [ 174.438382] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:48Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.3:50051 clients=16434worker2 # [ 174.442925] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:48Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06435worker2 # [ 174.446888] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:48Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06436worker2 # [ 174.470643] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866437worker1 # [ 174.467909] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546438worker2 # [ 174.486592] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595866439worker1 # [ 174.482580] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6440client # [ 174.548979] nix[1494]: copying 1 paths to 'lb:50051'...6441worker2 # [ 174.500963] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595866442client # [ 174.554365] nix[1494]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6443worker2 # [ 174.516854] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6444worker1 # [ 174.507015] nix-grpc-daemon[1783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 1/5)6445worker1 # [ 174.594586] nix-grpc-daemon[1783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 2/5)6446worker2 # [ 174.664861] nix-daemon[842]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6447worker1 # [ 174.658551] nix-grpc-daemon[1783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 334 ms (attempt 3/5)6448worker2 # [ 174.685185] nix-daemon[842]: reaped child process 1818, status = succeeded6449lb # [ 174.705768] niks3-server[1111]: 2026/09/23 05:14:49 INFO Received uploads request method=POST path=/api/pending_closures6450worker2 # [ 174.704105] nix-grpc-daemon[1636]: time=2026-09-23T05:14:49.202Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6451worker2 # [ 174.705797] nix-grpc-daemon[1636]: time=2026-09-23T05:14:49.202Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6452lb # [ 174.748084] niks3-server[1111]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6453worker2 # [ 174.746290] nix-grpc-daemon[1636]: time=2026-09-23T05:14:49.245Z level=INFO msg="Uploading 1 narinfos"6454lb # [ 174.759741] niks3-server[1111]: 2026/09/23 05:14:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6455lb # [ 174.770787] niks3-server[1111]: 2026/09/23 05:14:49 INFO Signed narinfos id=62 count=16456lb # [ 174.776281] niks3-server[1111]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6457lb # [ 174.792371] niks3-server[1111]: 2026/09/23 05:14:49 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6458lb # [ 174.799833] niks3-server[1111]: 2026/09/23 05:14:49 INFO Completed upload id=626459worker2 # [ 174.789115] nix-grpc-daemon[1636]: time=2026-09-23T05:14:49.288Z level=INFO msg="Upload complete. (257ms)"6460lb # [ 174.802621] niks3-server[1111]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6461worker2 # [ 174.801006] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=0 paths=1 nar_bytes_in=4166462worker2 # [ 174.816531] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866463worker2 # [ 174.826993] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6464worker2 # [ 174.835047] nix-daemon[842]: reaped child process 1813, status = succeeded6465worker2 # [ 174.847831] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6466client # [ 174.896468] nix[1494]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6467worker1 # [ 174.994982] nix-grpc-daemon[1783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 386 ms (attempt 4/5)6468worker1 # [ 175.383687] nix-grpc-daemon[1783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6469worker1 # [ 175.391629] nix-grpc-daemon[1783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6470lb: must succeed: curl -sf localhost:9901/clusters6471worker1 # [ 175.406414] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546472client # [ 175.481340] nix[1494]: copying 1 paths to 'lb:50051'...6473worker1 # [ 175.424125] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546474client # [ 175.485469] nix[1494]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6475worker1 # [ 175.439131] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6476worker1 # [ 175.477891] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:49Z level=info event=niks3_push_started pid=18446477lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6478lb: must succeed: curl -sf localhost:9901/clusters6479lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6480worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6481worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6482worker1 # [ 175.623848] nix-grpc-daemon[1844]: time=2026-09-23T05:14:50.124Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6483worker1 # [ 175.746556] nix-daemon[850]: accepted connection from pid 1861, user nix-grpc-daemon (trusted)6484worker1 # [ 175.764241] nix-daemon[850]: reaped child process 1864, status = succeeded6485lb # [ 175.791834] niks3-server[1111]: 2026/09/23 05:14:50 INFO Received uploads request method=POST path=/api/pending_closures6486worker1 # [ 175.786515] nix-grpc-daemon[1844]: time=2026-09-23T05:14:50.288Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6487worker1 # [ 175.790853] nix-grpc-daemon[1844]: time=2026-09-23T05:14:50.288Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6488lb # [ 175.839023] niks3-server[1111]: 2026/09/23 05:14:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6489worker1 # [ 175.830563] nix-grpc-daemon[1844]: time=2026-09-23T05:14:50.332Z level=INFO msg="Uploading 1 narinfos"6490lb # [ 175.852646] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6491lb # [ 175.861785] niks3-server[1111]: 2026/09/23 05:14:50 INFO Signed narinfos id=63 count=16492lb # [ 175.868639] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6493lb # [ 175.871494] niks3-server[1111]: 2026/09/23 05:14:50 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6494worker1 # [ 175.856934] nix-grpc-daemon[1844]: time=2026-09-23T05:14:50.360Z level=INFO msg="Upload complete. (234ms)"6495lb # [ 175.878859] niks3-server[1111]: 2026/09/23 05:14:50 INFO Completed upload id=636496lb # [ 175.881336] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6497worker1 # [ 175.867940] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=4166498worker1 # [ 175.881925] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546499worker1 # [ 175.887935] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6500worker1 # [ 175.893074] nix-daemon[850]: reaped child process 1842, status = succeeded6501client # [ 175.961305] nix[1494]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6502worker1 # [ 175.905962] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6503worker2 # [ 175.916659] nix-daemon[842]: reaped child process 1822, status = succeeded6504worker2 # [ 176.018264] nix-daemon[842]: accepted connection from pid 1846, user nix-grpc-daemon (trusted)6505worker2 # [ 176.038248] nix-daemon[842]: reaped child process 1849, status = succeeded6506lb # [ 176.052934] niks3-server[1111]: 2026/09/23 05:14:50 INFO Received uploads request method=POST path=/api/pending_closures6507worker2 # [ 176.055373] nix-grpc-daemon[1636]: time=2026-09-23T05:14:50.552Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6508worker2 # [ 176.057114] nix-grpc-daemon[1636]: time=2026-09-23T05:14:50.552Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6509lb # [ 176.098084] niks3-server[1111]: 2026/09/23 05:14:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6510worker2 # [ 176.093171] nix-grpc-daemon[1636]: time=2026-09-23T05:14:50.592Z level=INFO msg="Uploading 1 narinfos"6511lb # [ 176.110205] niks3-server[1111]: 2026/09/23 05:14:50 INFO Signed narinfos id=64 count=16512lb # [ 176.115631] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6513lb # [ 176.122876] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6514worker2 # [ 176.120811] nix-grpc-daemon[1636]: time=2026-09-23T05:14:50.620Z level=INFO msg="Upload complete. (258ms)"6515lb # [ 176.131800] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6516lb # [ 176.136209] niks3-server[1111]: 2026/09/23 05:14:50 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6517lb # [ 176.140378] niks3-server[1111]: 2026/09/23 05:14:50 INFO Completed upload id=646518worker2 # [ 176.131344] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=16519lb # [ 176.145457] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6520worker2 # [ 176.148234] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:50Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=16521worker2 # [ 176.156837] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866522worker2 # [ 176.161382] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595866523worker2 # [ 176.165844] nix-daemon[842]: reaped child process 1820, status = succeeded6524client # [ 176.219417] nix[1494]: copying 1 paths to 'lb:50051'...6525worker2 # [ 176.169134] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595866526client # [ 176.221983] nix[1494]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6527worker2 # [ 176.178614] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6528worker2 # [ 176.308226] nix-daemon[842]: accepted connection from pid 1853, user nix-grpc-daemon (trusted)6529worker2 # [ 176.329003] nix-daemon[842]: reaped child process 1856, status = succeeded6530lb # [ 176.337890] niks3-server[1111]: 2026/09/23 05:14:50 INFO Received uploads request method=POST path=/api/pending_closures6531worker2 # [ 176.336432] nix-grpc-daemon[1636]: time=2026-09-23T05:14:50.835Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6532worker2 # [ 176.338017] nix-grpc-daemon[1636]: time=2026-09-23T05:14:50.835Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6533lb # [ 176.370071] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6534lb # [ 176.385219] niks3-server[1111]: 2026/09/23 05:14:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6535worker2 # [ 176.379903] nix-grpc-daemon[1636]: time=2026-09-23T05:14:50.879Z level=INFO msg="Uploading 1 narinfos"6536lb # [ 176.393996] niks3-server[1111]: 2026/09/23 05:14:50 INFO Signed narinfos id=65 count=16537lb # [ 176.399008] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6538worker2 # [ 176.397193] nix-grpc-daemon[1636]: time=2026-09-23T05:14:50.896Z level=INFO msg="Upload complete. (209ms)"6539lb # [ 176.408527] niks3-server[1111]: 2026/09/23 05:14:50 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6540lb # [ 176.412314] niks3-server[1111]: 2026/09/23 05:14:50 INFO Completed upload id=656541lb # [ 176.416079] niks3-server[1111]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6542worker2 # [ 176.404299] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=0 paths=1 nar_bytes_in=4166543worker2 # [ 176.418035] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595866544worker2 # [ 176.423701] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6545worker2 # [ 176.428280] nix-daemon[842]: reaped child process 1851, status = succeeded6546worker2 # [ 176.437845] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6547client # [ 176.487900] nix[1494]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6548worker1 # [ 178.137227] nix-daemon[850]: reaped child process 1868, status = succeeded6549worker1 # [ 178.255912] nix-daemon[850]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)6550worker1 # [ 178.273295] nix-daemon[850]: reaped child process 1896, status = succeeded6551lb # [ 178.298350] niks3-server[1111]: 2026/09/23 05:14:52 INFO Received uploads request method=POST path=/api/pending_closures6552worker1 # [ 178.290508] nix-grpc-daemon[1844]: time=2026-09-23T05:14:52.791Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6553worker1 # [ 178.291992] nix-grpc-daemon[1844]: time=2026-09-23T05:14:52.791Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6554lb # [ 178.338897] niks3-server[1111]: 2026/09/23 05:14:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6555worker1 # [ 178.330398] nix-grpc-daemon[1844]: time=2026-09-23T05:14:52.833Z level=INFO msg="Uploading 1 narinfos"6556lb # [ 178.355775] niks3-server[1111]: 2026/09/23 05:14:52 INFO Signed narinfos id=66 count=16557worker2 # [ 178.354174] nix-daemon[842]: reaped child process 1860, status = succeeded6558lb # [ 178.365489] niks3-server[1111]: 2026/09/23 05:14:52 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6559lb # [ 178.375065] niks3-server[1111]: 2026/09/23 05:14:52 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6560lb # [ 178.384112] niks3-server[1111]: 2026/09/23 05:14:52 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6561lb # [ 178.387715] niks3-server[1111]: 2026/09/23 05:14:52 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6562lb # [ 178.396231] niks3-server[1111]: 2026/09/23 05:14:52 INFO Completed upload id=666563lb # [ 178.398776] niks3-server[1111]: 2026/09/23 05:14:52 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6564worker1 # [ 178.375616] nix-grpc-daemon[1844]: time=2026-09-23T05:14:52.878Z level=INFO msg="Upload complete. (258ms)"6565worker2 # [ 178.396598] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:52Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=16566worker1 # [ 178.392602] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=16567worker1 # [ 178.407908] nix-daemon[850]: reaped child process 1866, status = succeeded6568worker2 # [ 178.433182] nix-daemon[842]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6569worker1 # [ 178.433868] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546570worker2 # [ 178.452829] nix-daemon[842]: reaped child process 1888, status = succeeded6571lb # [ 178.468000] niks3-server[1111]: 2026/09/23 05:14:52 INFO Received uploads request method=POST path=/api/pending_closures6572worker1 # [ 178.455605] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546573worker2 # [ 178.468459] nix-grpc-daemon[1636]: time=2026-09-23T05:14:52.966Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6574worker2 # [ 178.470169] nix-grpc-daemon[1636]: time=2026-09-23T05:14:52.966Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6575worker1 # [ 178.471309] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546576client # [ 178.533156] nix[1494]: copying 1 paths to 'lb:50051'...6577client # [ 178.533443] nix[1494]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6578worker1 # [ 178.492935] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6579lb # [ 178.514332] niks3-server[1111]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6580worker2 # [ 178.520367] nix-grpc-daemon[1636]: time=2026-09-23T05:14:53.019Z level=INFO msg="Uploading 1 narinfos"6581lb # [ 178.523001] niks3-server[1111]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6582lb # [ 178.533774] niks3-server[1111]: 2026/09/23 05:14:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6583lb # [ 178.541378] niks3-server[1111]: 2026/09/23 05:14:53 INFO Signed narinfos id=67 count=16584lb # [ 178.548272] niks3-server[1111]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6585worker2 # [ 178.544663] nix-grpc-daemon[1636]: time=2026-09-23T05:14:53.044Z level=INFO msg="Upload complete. (291ms)"6586lb # [ 178.555182] niks3-server[1111]: 2026/09/23 05:14:53 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6587lb # [ 178.559740] niks3-server[1111]: 2026/09/23 05:14:53 INFO Completed upload id=676588lb # [ 178.563536] niks3-server[1111]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6589worker2 # [ 178.556363] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=16590worker2 # [ 178.572716] nix-daemon[842]: reaped child process 1858, status = succeeded6591worker1 # [ 178.653844] nix-daemon[850]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)6592worker1 # [ 178.673233] nix-daemon[850]: reaped child process 1903, status = succeeded6593lb # [ 178.697815] niks3-server[1111]: 2026/09/23 05:14:53 INFO Received uploads request method=POST path=/api/pending_closures6594worker1 # [ 178.687272] nix-grpc-daemon[1844]: time=2026-09-23T05:14:53.190Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6595worker1 # [ 178.689092] nix-grpc-daemon[1844]: time=2026-09-23T05:14:53.190Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6596lb # [ 178.725257] niks3-server[1111]: 2026/09/23 05:14:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6597worker1 # [ 178.713122] nix-grpc-daemon[1844]: time=2026-09-23T05:14:53.216Z level=INFO msg="Uploading 1 narinfos"6598lb # [ 178.737917] niks3-server[1111]: 2026/09/23 05:14:53 INFO Signed narinfos id=68 count=16599lb # [ 178.747876] niks3-server[1111]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6600lb # [ 178.756853] niks3-server[1111]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6601worker1 # [ 178.750451] nix-grpc-daemon[1844]: time=2026-09-23T05:14:53.253Z level=INFO msg="Upload complete. (240ms)"6602lb # [ 178.769782] niks3-server[1111]: 2026/09/23 05:14:53 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6603lb # [ 178.779439] niks3-server[1111]: 2026/09/23 05:14:53 INFO Completed upload id=686604worker1 # [ 178.761323] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=4166605lb # [ 178.782073] niks3-server[1111]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6606worker1 # [ 178.775190] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546607worker1 # [ 178.782304] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6608worker1 # [ 178.787475] nix-daemon[850]: reaped child process 1898, status = succeeded6609client # [ 178.856917] nix[1494]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6610worker1 # [ 178.800252] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6611lb # [ 180.280359] envoy[908]: conn peer=[2001:db8:1::3]:47526 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1020 tx=482 ms=269796612worker1 # [ 180.431230] nix-daemon[850]: reaped child process 1907, status = succeeded6613worker1 # [ 180.533204] nix-daemon[850]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6614worker1 # [ 180.550799] nix-daemon[850]: reaped child process 1935, status = succeeded6615lb # [ 180.578188] niks3-server[1111]: 2026/09/23 05:14:55 INFO Received uploads request method=POST path=/api/pending_closures6616worker1 # [ 180.568406] nix-grpc-daemon[1844]: time=2026-09-23T05:14:55.070Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6617worker1 # [ 180.570098] nix-grpc-daemon[1844]: time=2026-09-23T05:14:55.070Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6618lb # [ 180.618780] niks3-server[1111]: 2026/09/23 05:14:55 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6619worker1 # [ 180.610066] nix-grpc-daemon[1844]: time=2026-09-23T05:14:55.113Z level=INFO msg="Uploading 1 narinfos"6620lb # [ 180.633500] niks3-server[1111]: 2026/09/23 05:14:55 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6621lb # [ 180.643187] niks3-server[1111]: 2026/09/23 05:14:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6622lb # [ 180.652412] niks3-server[1111]: 2026/09/23 05:14:55 INFO Signed narinfos id=69 count=16623worker1 # [ 180.633329] nix-grpc-daemon[1844]: time=2026-09-23T05:14:55.136Z level=INFO msg="Upload complete. (220ms)"6624lb # [ 180.654369] niks3-server[1111]: 2026/09/23 05:14:55 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6625lb # [ 180.660379] niks3-server[1111]: 2026/09/23 05:14:55 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6626lb # [ 180.663907] niks3-server[1111]: 2026/09/23 05:14:55 INFO Completed upload id=696627worker1 # [ 180.644894] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=16628lb # [ 180.667585] niks3-server[1111]: 2026/09/23 05:14:55 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6629worker1 # [ 180.657976] nix-daemon[850]: reaped child process 1905, status = succeeded6630client # [ 180.738904] systemd[1]: burst.service: Deactivated successfully.6631client # [ 180.744165] systemd[1]: burst.service: Consumed 236ms CPU time over 6.490s wall clock time, 57.6M memory peak, 14.1K incoming IP traffic, 18.9K outgoing IP traffic.6632client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6633client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.08 seconds)6634lb: must succeed: curl -sf localhost:9901/clusters6635lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6636lb: must succeed: curl -sf localhost:9901/clusters6637lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6638worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6639worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6640(finished: subtest: queue gauges show a burst while every build blocks, in 8.03 seconds)6641subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6642client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16643worker2 # [ 182.209881] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:595866644worker2 # [ 182.220790] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595866645worker2 # [ 182.263130] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:56Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06646worker2 # [ 182.269449] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:56Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16647worker2 # [ 182.275416] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:56Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06648worker2 # [ 182.280754] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:56Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06649worker1 # [ 182.303146] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546650worker1 # [ 182.319334] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546651worker1 # [ 182.334268] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546652worker1 # [ 182.349357] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6653worker1 # [ 182.511998] nix-daemon[850]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)6654worker1 # [ 182.532133] nix-daemon[850]: reaped child process 1946, status = succeeded6655lb # [ 182.557085] niks3-server[1111]: 2026/09/23 05:14:57 INFO Received uploads request method=POST path=/api/pending_closures6656worker1 # [ 182.546315] nix-grpc-daemon[1844]: time=2026-09-23T05:14:57.048Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6657worker1 # [ 182.548444] nix-grpc-daemon[1844]: time=2026-09-23T05:14:57.048Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6658lb # [ 182.587153] niks3-server[1111]: 2026/09/23 05:14:57 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6659worker1 # [ 182.576065] nix-grpc-daemon[1844]: time=2026-09-23T05:14:57.079Z level=INFO msg="Uploading 1 narinfos"6660lb # [ 182.595900] niks3-server[1111]: 2026/09/23 05:14:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6661lb # [ 182.603044] niks3-server[1111]: 2026/09/23 05:14:57 INFO Signed narinfos id=70 count=16662lb # [ 182.609930] niks3-server[1111]: 2026/09/23 05:14:57 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6663lb # [ 182.619039] niks3-server[1111]: 2026/09/23 05:14:57 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6664lb # [ 182.624107] niks3-server[1111]: 2026/09/23 05:14:57 INFO Completed upload id=706665lb # [ 182.628677] niks3-server[1111]: 2026/09/23 05:14:57 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6666worker1 # [ 182.605356] nix-grpc-daemon[1844]: time=2026-09-23T05:14:57.108Z level=INFO msg="Upload complete. (238ms)"6667worker1 # [ 182.616300] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=5126668worker1 # [ 182.628698] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546669worker1 # [ 182.637332] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6670worker1 # [ 182.642729] nix-daemon[850]: reaped child process 1941, status = succeeded6671worker1 # [ 182.652517] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6672worker1 # [ 183.558217] nix-daemon[850]: reaped child process 1950, status = succeeded6673worker1 # [ 183.634273] nix-daemon[850]: accepted connection from pid 1966, user nix-grpc-daemon (trusted)6674worker1 # [ 183.653517] nix-daemon[850]: reaped child process 1969, status = succeeded6675lb # [ 183.679421] niks3-server[1111]: 2026/09/23 05:14:58 INFO Received uploads request method=POST path=/api/pending_closures6676worker1 # [ 183.670127] nix-grpc-daemon[1844]: time=2026-09-23T05:14:58.172Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6677worker1 # [ 183.671737] nix-grpc-daemon[1844]: time=2026-09-23T05:14:58.172Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6678lb # [ 183.724947] niks3-server[1111]: 2026/09/23 05:14:58 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6679worker1 # [ 183.714290] nix-grpc-daemon[1844]: time=2026-09-23T05:14:58.217Z level=INFO msg="Uploading 1 narinfos"6680lb # [ 183.737874] niks3-server[1111]: 2026/09/23 05:14:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6681lb # [ 183.747882] niks3-server[1111]: 2026/09/23 05:14:58 INFO Signed narinfos id=71 count=16682lb # [ 183.756293] niks3-server[1111]: 2026/09/23 05:14:58 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6683worker1 # [ 183.743242] nix-grpc-daemon[1844]: time=2026-09-23T05:14:58.246Z level=INFO msg="Upload complete. (279ms)"6684lb # [ 183.766230] niks3-server[1111]: 2026/09/23 05:14:58 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6685lb # [ 183.772941] niks3-server[1111]: 2026/09/23 05:14:58 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6686lb # [ 183.778456] niks3-server[1111]: 2026/09/23 05:14:58 INFO Completed upload id=716687worker2 # [ 183.764298] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:58Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16688worker1 # [ 183.757262] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=16689lb # [ 183.781415] niks3-server[1111]: 2026/09/23 05:14:58 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6690worker2 # [ 183.780161] nix-grpc-daemon[1559]: ts=2026-09-23T05:14:58Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06691worker1 # [ 183.779378] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546692worker1 # [ 183.786723] nix-daemon[850]: reaped child process 1948, status = succeeded6693worker1 # [ 183.790041] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546694worker1 # [ 183.803278] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546695worker1 # [ 183.816075] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6696worker1 # [ 183.951607] nix-daemon[850]: accepted connection from pid 1973, user nix-grpc-daemon (trusted)6697worker1 # [ 183.969353] nix-daemon[850]: reaped child process 1976, status = succeeded6698lb # [ 183.993634] niks3-server[1111]: 2026/09/23 05:14:58 INFO Received uploads request method=POST path=/api/pending_closures6699worker1 # [ 183.983485] nix-grpc-daemon[1844]: time=2026-09-23T05:14:58.486Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6700worker1 # [ 183.985360] nix-grpc-daemon[1844]: time=2026-09-23T05:14:58.486Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6701lb # [ 184.022246] niks3-server[1111]: 2026/09/23 05:14:58 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6702worker1 # [ 184.021488] nix-grpc-daemon[1844]: time=2026-09-23T05:14:58.524Z level=INFO msg="Uploading 1 narinfos"6703lb # [ 184.034666] niks3-server[1111]: 2026/09/23 05:14:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6704lb # [ 184.045460] niks3-server[1111]: 2026/09/23 05:14:58 INFO Signed narinfos id=72 count=16705lb # [ 184.051668] niks3-server[1111]: 2026/09/23 05:14:58 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6706worker1 # [ 184.043447] nix-grpc-daemon[1844]: time=2026-09-23T05:14:58.546Z level=INFO msg="Upload complete. (215ms)"6707lb # [ 184.063541] niks3-server[1111]: 2026/09/23 05:14:58 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6708lb # [ 184.073699] niks3-server[1111]: 2026/09/23 05:14:58 INFO Completed upload id=726709worker1 # [ 184.052357] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=5126710lb # [ 184.077339] niks3-server[1111]: 2026/09/23 05:14:58 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6711worker1 # [ 184.071100] nix-grpc-daemon[1783]: ts=2026-09-23T05:14:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546712worker1 # [ 184.078238] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6713worker1 # [ 184.083482] nix-daemon[850]: reaped child process 1971, status = succeeded6714worker1 # [ 184.094931] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6715worker1 # [ 185.346474] nix-daemon[850]: reaped child process 1980, status = succeeded6716worker1 # [ 185.473968] nix-daemon[850]: accepted connection from pid 1996, user nix-grpc-daemon (trusted)6717worker1 # [ 185.501103] nix-daemon[850]: reaped child process 1999, status = succeeded6718lb # [ 185.521551] niks3-server[1111]: 2026/09/23 05:15:00 INFO Received uploads request method=POST path=/api/pending_closures6719worker1 # [ 185.516502] nix-grpc-daemon[1844]: time=2026-09-23T05:15:00.017Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6720worker1 # [ 185.517966] nix-grpc-daemon[1844]: time=2026-09-23T05:15:00.017Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6721lb # [ 185.598330] niks3-server[1111]: 2026/09/23 05:15:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6722worker1 # [ 185.591361] nix-grpc-daemon[1844]: time=2026-09-23T05:15:00.093Z level=INFO msg="Uploading 1 narinfos"6723lb # [ 185.617542] niks3-server[1111]: 2026/09/23 05:15:00 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6724lb # [ 185.627023] niks3-server[1111]: 2026/09/23 05:15:00 INFO Signed narinfos id=73 count=16725lb # [ 185.632697] niks3-server[1111]: 2026/09/23 05:15:00 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6726worker1 # [ 185.628644] nix-grpc-daemon[1844]: time=2026-09-23T05:15:00.131Z level=INFO msg="Upload complete. (320ms)"6727lb # [ 185.648797] niks3-server[1111]: 2026/09/23 05:15:00 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6728lb # [ 185.652519] niks3-server[1111]: 2026/09/23 05:15:00 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6729lb # [ 185.659313] niks3-server[1111]: 2026/09/23 05:15:00 INFO Completed upload id=736730lb # [ 185.663313] niks3-server[1111]: 2026/09/23 05:15:00 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6731worker2 # [ 185.651472] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:00Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16732worker1 # [ 185.641664] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=16733worker2 # [ 185.670432] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:00Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06734worker1 # [ 185.664457] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546735worker1 # [ 185.680633] nix-daemon[850]: reaped child process 1978, status = succeeded6736worker1 # [ 185.683610] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546737worker1 # [ 185.698209] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546738worker1 # [ 185.714515] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6739worker1 # [ 185.909895] nix-daemon[850]: accepted connection from pid 2003, user nix-grpc-daemon (trusted)6740worker1 # [ 185.932692] nix-daemon[850]: reaped child process 2006, status = succeeded6741lb # [ 185.956718] niks3-server[1111]: 2026/09/23 05:15:00 INFO Received uploads request method=POST path=/api/pending_closures6742worker1 # [ 185.948660] nix-grpc-daemon[1844]: time=2026-09-23T05:15:00.451Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6743worker1 # [ 185.950184] nix-grpc-daemon[1844]: time=2026-09-23T05:15:00.451Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6744lb # [ 185.999662] niks3-server[1111]: 2026/09/23 05:15:00 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6745lb # [ 186.015842] niks3-server[1111]: 2026/09/23 05:15:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6746worker1 # [ 186.005456] nix-grpc-daemon[1844]: time=2026-09-23T05:15:00.508Z level=INFO msg="Uploading 1 narinfos"6747lb # [ 186.030980] niks3-server[1111]: 2026/09/23 05:15:00 INFO Signed narinfos id=74 count=16748lb # [ 186.037767] niks3-server[1111]: 2026/09/23 05:15:00 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6749lb # [ 186.054222] niks3-server[1111]: 2026/09/23 05:15:00 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6750lb # [ 186.062110] niks3-server[1111]: 2026/09/23 05:15:00 INFO Completed upload id=746751worker1 # [ 186.042138] nix-grpc-daemon[1844]: time=2026-09-23T05:15:00.545Z level=INFO msg="Upload complete. (310ms)"6752lb # [ 186.065120] niks3-server[1111]: 2026/09/23 05:15:00 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6753worker1 # [ 186.052486] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=5126754worker1 # [ 186.076372] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546755worker1 # [ 186.089149] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6756worker1 # [ 186.096558] nix-daemon[850]: reaped child process 2001, status = succeeded6757worker1 # [ 186.119891] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6758worker1 # [ 187.849189] nix-daemon[850]: reaped child process 2010, status = succeeded6759worker1 # [ 187.970956] nix-daemon[850]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)6760worker1 # [ 187.994379] nix-daemon[850]: reaped child process 2029, status = succeeded6761lb # [ 188.023268] niks3-server[1111]: 2026/09/23 05:15:02 INFO Received uploads request method=POST path=/api/pending_closures6762worker1 # [ 188.020265] nix-grpc-daemon[1844]: time=2026-09-23T05:15:02.520Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6763worker1 # [ 188.021983] nix-grpc-daemon[1844]: time=2026-09-23T05:15:02.521Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6764lb # [ 188.083425] niks3-server[1111]: 2026/09/23 05:15:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6765worker1 # [ 188.075504] nix-grpc-daemon[1844]: time=2026-09-23T05:15:02.578Z level=INFO msg="Uploading 1 narinfos"6766lb # [ 188.096886] niks3-server[1111]: 2026/09/23 05:15:02 INFO Signed narinfos id=75 count=16767lb # [ 188.113503] niks3-server[1111]: 2026/09/23 05:15:02 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6768lb # [ 188.128349] niks3-server[1111]: 2026/09/23 05:15:02 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6769lb # [ 188.137085] niks3-server[1111]: 2026/09/23 05:15:02 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6770lb # [ 188.148647] niks3-server[1111]: 2026/09/23 05:15:02 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6771lb # [ 188.156071] niks3-server[1111]: 2026/09/23 05:15:02 INFO Completed upload id=756772worker1 # [ 188.139897] nix-grpc-daemon[1844]: time=2026-09-23T05:15:02.640Z level=INFO msg="Upload complete. (402ms)"6773lb # [ 188.161670] niks3-server[1111]: 2026/09/23 05:15:02 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6774worker2 # [ 188.159989] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:02Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16775worker1 # [ 188.157933] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=2 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=16776worker2 # [ 188.177013] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:02Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06777worker1 # [ 188.176519] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546778worker1 # [ 188.196732] nix-daemon[850]: reaped child process 2008, status = succeeded6779worker1 # [ 188.204877] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546780worker1 # [ 188.218478] nix-grpc-daemon[1844]: time=2026-09-23T05:15:02.721Z level=INFO msg="All 1 paths already cached"6781worker1 # [ 188.230203] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546782worker1 # [ 188.247674] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6783worker1 # [ 188.458943] nix-daemon[850]: accepted connection from pid 2033, user nix-grpc-daemon (trusted)6784worker1 # [ 188.479882] nix-daemon[850]: reaped child process 2036, status = succeeded6785lb # [ 188.509062] niks3-server[1111]: 2026/09/23 05:15:03 INFO Received uploads request method=POST path=/api/pending_closures6786worker1 # [ 188.501669] nix-grpc-daemon[1844]: time=2026-09-23T05:15:03.004Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6787worker1 # [ 188.503677] nix-grpc-daemon[1844]: time=2026-09-23T05:15:03.004Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6788lb # [ 188.557475] niks3-server[1111]: 2026/09/23 05:15:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6789worker1 # [ 188.546275] nix-grpc-daemon[1844]: time=2026-09-23T05:15:03.049Z level=INFO msg="Uploading 1 narinfos"6790lb # [ 188.571873] niks3-server[1111]: 2026/09/23 05:15:03 INFO Signed narinfos id=76 count=16791lb # [ 188.578863] niks3-server[1111]: 2026/09/23 05:15:03 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6792lb # [ 188.591635] niks3-server[1111]: 2026/09/23 05:15:03 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6793lb # [ 188.602751] niks3-server[1111]: 2026/09/23 05:15:03 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6794worker1 # [ 188.588698] nix-grpc-daemon[1844]: time=2026-09-23T05:15:03.091Z level=INFO msg="Upload complete. (321ms)"6795lb # [ 188.610490] niks3-server[1111]: 2026/09/23 05:15:03 INFO Completed upload id=766796lb # [ 188.612938] niks3-server[1111]: 2026/09/23 05:15:03 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6797worker1 # [ 188.600483] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=6246798worker1 # [ 188.617282] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546799worker1 # [ 188.636869] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6800worker1 # [ 188.645137] nix-daemon[850]: reaped child process 2031, status = succeeded6801worker1 # [ 188.666806] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6802worker1 # [ 189.831408] nix-daemon[850]: reaped child process 2040, status = succeeded6803worker1 # [ 189.926063] nix-daemon[850]: accepted connection from pid 2056, user nix-grpc-daemon (trusted)6804worker1 # [ 189.953775] nix-daemon[850]: reaped child process 2059, status = succeeded6805lb # [ 189.979615] niks3-server[1111]: 2026/09/23 05:15:04 INFO Received uploads request method=POST path=/api/pending_closures6806worker1 # [ 189.971857] nix-grpc-daemon[1844]: time=2026-09-23T05:15:04.473Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6807worker1 # [ 189.975401] nix-grpc-daemon[1844]: time=2026-09-23T05:15:04.473Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6808lb # [ 190.046614] niks3-server[1111]: 2026/09/23 05:15:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6809worker1 # [ 190.041009] nix-grpc-daemon[1844]: time=2026-09-23T05:15:04.543Z level=INFO msg="Uploading 1 narinfos"6810lb # [ 190.061827] niks3-server[1111]: 2026/09/23 05:15:04 INFO Signed narinfos id=77 count=16811lb # [ 190.073343] niks3-server[1111]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6812lb # [ 190.085658] niks3-server[1111]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6813lb # [ 190.094507] niks3-server[1111]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6814lb # [ 190.106345] niks3-server[1111]: 2026/09/23 05:15:04 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6815worker1 # [ 190.086968] nix-grpc-daemon[1844]: time=2026-09-23T05:15:04.590Z level=INFO msg="Upload complete. (405ms)"6816lb # [ 190.110657] niks3-server[1111]: 2026/09/23 05:15:04 INFO Completed upload id=776817lb # [ 190.113075] niks3-server[1111]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6818worker1 # [ 190.101573] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=16819worker2 # [ 190.119283] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:04Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16820worker2 # [ 190.128341] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:04Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06821worker1 # [ 190.128780] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546822worker1 # [ 190.138956] nix-daemon[850]: reaped child process 2038, status = succeeded6823worker1 # [ 190.151120] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546824worker1 # [ 190.172988] nix-grpc-daemon[1844]: time=2026-09-23T05:15:04.676Z level=INFO msg="All 1 paths already cached"6825worker1 # [ 190.187762] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546826worker1 # [ 190.204944] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6827lb # [ 190.282084] envoy[908]: conn peer=[2001:db8:1::1]:49350 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9740 tx=5201 ms=62636828worker1 # [ 190.381037] nix-daemon[850]: accepted connection from pid 2063, user nix-grpc-daemon (trusted)6829worker1 # [ 190.406491] nix-daemon[850]: reaped child process 2066, status = succeeded6830lb # [ 190.429372] niks3-server[1111]: 2026/09/23 05:15:04 INFO Received uploads request method=POST path=/api/pending_closures6831worker1 # [ 190.426413] nix-grpc-daemon[1844]: time=2026-09-23T05:15:04.929Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6832worker1 # [ 190.428390] nix-grpc-daemon[1844]: time=2026-09-23T05:15:04.929Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6833lb # [ 190.480657] niks3-server[1111]: 2026/09/23 05:15:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6834worker1 # [ 190.470540] nix-grpc-daemon[1844]: time=2026-09-23T05:15:04.973Z level=INFO msg="Uploading 1 narinfos"6835lb # [ 190.496421] niks3-server[1111]: 2026/09/23 05:15:04 INFO Signed narinfos id=78 count=16836lb # [ 190.503944] niks3-server[1111]: 2026/09/23 05:15:05 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6837lb # [ 190.514225] niks3-server[1111]: 2026/09/23 05:15:05 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6838worker1 # [ 190.516272] nix-grpc-daemon[1844]: time=2026-09-23T05:15:05.019Z level=INFO msg="Upload complete. (293ms)"6839lb # [ 190.529230] niks3-server[1111]: 2026/09/23 05:15:05 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6840lb # [ 190.540436] niks3-server[1111]: 2026/09/23 05:15:05 INFO Completed upload id=786841lb # [ 190.542486] niks3-server[1111]: 2026/09/23 05:15:05 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6842worker1 # [ 190.528504] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=6246843worker1 # [ 190.545195] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546844worker1 # [ 190.560448] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6845worker1 # [ 190.572526] nix-daemon[850]: reaped child process 2061, status = succeeded6846worker1 # [ 190.594769] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6847worker1 # [ 191.503890] nix-daemon[850]: reaped child process 2070, status = succeeded6848worker1 # [ 191.617085] nix-daemon[850]: accepted connection from pid 2086, user nix-grpc-daemon (trusted)6849worker1 # [ 191.637233] nix-daemon[850]: reaped child process 2089, status = succeeded6850lb # [ 191.662223] niks3-server[1111]: 2026/09/23 05:15:06 INFO Received uploads request method=POST path=/api/pending_closures6851worker1 # [ 191.655186] nix-grpc-daemon[1844]: time=2026-09-23T05:15:06.156Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6852worker1 # [ 191.660087] nix-grpc-daemon[1844]: time=2026-09-23T05:15:06.156Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6853worker1 # [ 191.700819] nix-grpc-daemon[1844]: time=2026-09-23T05:15:06.203Z level=INFO msg="Uploading 1 narinfos"6854lb # [ 191.710374] niks3-server[1111]: 2026/09/23 05:15:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6855lb # [ 191.725199] niks3-server[1111]: 2026/09/23 05:15:06 INFO Signed narinfos id=79 count=16856lb # [ 191.734134] niks3-server[1111]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6857lb # [ 191.745931] niks3-server[1111]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6858lb # [ 191.753018] niks3-server[1111]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6859worker1 # [ 191.742364] nix-grpc-daemon[1844]: time=2026-09-23T05:15:06.245Z level=INFO msg="Upload complete. (331ms)"6860lb # [ 191.762599] niks3-server[1111]: 2026/09/23 05:15:06 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6861lb # [ 191.766592] niks3-server[1111]: 2026/09/23 05:15:06 INFO Completed upload id=796862lb # [ 191.770285] niks3-server[1111]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6863worker2 # [ 191.761888] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:06Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16864worker1 # [ 191.757221] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=16865worker1 # [ 191.775913] nix-daemon[850]: reaped child process 2068, status = succeeded6866worker1 # [ 191.799840] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546867worker2 # [ 191.812147] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:06Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06868worker1 # [ 191.817743] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546869worker1 # [ 191.832440] nix-grpc-daemon[1844]: time=2026-09-23T05:15:06.335Z level=INFO msg="All 1 paths already cached"6870worker1 # [ 191.841633] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546871worker1 # [ 191.857036] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6872worker1 # [ 192.017712] nix-daemon[850]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)6873worker1 # [ 192.038845] nix-daemon[850]: reaped child process 2096, status = succeeded6874lb # [ 192.058951] niks3-server[1111]: 2026/09/23 05:15:06 INFO Received uploads request method=POST path=/api/pending_closures6875worker1 # [ 192.049214] nix-grpc-daemon[1844]: time=2026-09-23T05:15:06.552Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6876worker1 # [ 192.050753] nix-grpc-daemon[1844]: time=2026-09-23T05:15:06.552Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6877lb # [ 192.092709] niks3-server[1111]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6878worker1 # [ 192.084589] nix-grpc-daemon[1844]: time=2026-09-23T05:15:06.587Z level=INFO msg="Uploading 1 narinfos"6879lb # [ 192.105425] niks3-server[1111]: 2026/09/23 05:15:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6880lb # [ 192.117134] niks3-server[1111]: 2026/09/23 05:15:06 INFO Signed narinfos id=80 count=16881lb # [ 192.123272] niks3-server[1111]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6882lb # [ 192.134242] niks3-server[1111]: 2026/09/23 05:15:06 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete6883lb # [ 192.143729] niks3-server[1111]: 2026/09/23 05:15:06 INFO Completed upload id=806884worker1 # [ 192.123873] nix-grpc-daemon[1844]: time=2026-09-23T05:15:06.627Z level=INFO msg="Upload complete. (248ms)"6885lb # [ 192.147548] niks3-server[1111]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6886worker1 # [ 192.136315] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=6246887worker1 # [ 192.152362] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546888worker1 # [ 192.163978] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6889worker1 # [ 192.167102] nix-daemon[850]: reaped child process 2091, status = succeeded6890worker1 # [ 192.184317] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6891worker1 # [ 193.288541] nix-daemon[850]: reaped child process 2100, status = succeeded6892worker1 # [ 193.361379] nix-daemon[850]: accepted connection from pid 2116, user nix-grpc-daemon (trusted)6893worker1 # [ 193.378868] nix-daemon[850]: reaped child process 2119, status = succeeded6894lb # [ 193.405261] niks3-server[1111]: 2026/09/23 05:15:07 INFO Received uploads request method=POST path=/api/pending_closures6895worker1 # [ 193.396409] nix-grpc-daemon[1844]: time=2026-09-23T05:15:07.898Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6896worker1 # [ 193.398052] nix-grpc-daemon[1844]: time=2026-09-23T05:15:07.898Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6897lb # [ 193.448727] niks3-server[1111]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6898worker1 # [ 193.449322] nix-grpc-daemon[1844]: time=2026-09-23T05:15:07.952Z level=INFO msg="Uploading 1 narinfos"6899lb # [ 193.461512] niks3-server[1111]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6900lb # [ 193.473294] niks3-server[1111]: 2026/09/23 05:15:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign6901lb # [ 193.482456] niks3-server[1111]: 2026/09/23 05:15:07 INFO Signed narinfos id=81 count=16902lb # [ 193.488989] niks3-server[1111]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6903worker1 # [ 193.481657] nix-grpc-daemon[1844]: time=2026-09-23T05:15:07.984Z level=INFO msg="Upload complete. (285ms)"6904lb # [ 193.502479] niks3-server[1111]: 2026/09/23 05:15:08 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete6905lb # [ 193.510326] niks3-server[1111]: 2026/09/23 05:15:08 INFO Completed upload id=816906worker2 # [ 193.499540] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:07Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16907lb # [ 193.512831] niks3-server[1111]: 2026/09/23 05:15:08 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6908worker1 # [ 193.497157] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=16909worker2 # [ 193.513229] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:08Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06910worker1 # [ 193.515691] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546911worker1 # [ 193.523215] nix-daemon[850]: reaped child process 2098, status = succeeded6912worker1 # [ 193.531988] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546913worker1 # [ 193.545558] nix-grpc-daemon[1844]: time=2026-09-23T05:15:08.048Z level=INFO msg="All 2 paths already cached"6914worker1 # [ 193.555817] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546915worker1 # [ 193.569473] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6916worker1 # [ 193.712617] nix-daemon[850]: accepted connection from pid 2123, user nix-grpc-daemon (trusted)6917worker1 # [ 193.729424] nix-daemon[850]: reaped child process 2126, status = succeeded6918lb # [ 193.755190] niks3-server[1111]: 2026/09/23 05:15:08 INFO Received uploads request method=POST path=/api/pending_closures6919worker1 # [ 193.746389] nix-grpc-daemon[1844]: time=2026-09-23T05:15:08.249Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6920worker1 # [ 193.748097] nix-grpc-daemon[1844]: time=2026-09-23T05:15:08.249Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6921lb # [ 193.785548] niks3-server[1111]: 2026/09/23 05:15:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign6922worker1 # [ 193.775635] nix-grpc-daemon[1844]: time=2026-09-23T05:15:08.278Z level=INFO msg="Uploading 1 narinfos"6923lb # [ 193.799651] niks3-server[1111]: 2026/09/23 05:15:08 INFO Signed narinfos id=82 count=16924lb # [ 193.807025] niks3-server[1111]: 2026/09/23 05:15:08 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6925lb # [ 193.815469] niks3-server[1111]: 2026/09/23 05:15:08 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6926lb # [ 193.822804] niks3-server[1111]: 2026/09/23 05:15:08 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete6927lb # [ 193.828135] niks3-server[1111]: 2026/09/23 05:15:08 INFO Completed upload id=826928worker1 # [ 193.808186] nix-grpc-daemon[1844]: time=2026-09-23T05:15:08.311Z level=INFO msg="Upload complete. (225ms)"6929lb # [ 193.830384] niks3-server[1111]: 2026/09/23 05:15:08 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6930worker1 # [ 193.816281] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=7526931worker1 # [ 193.831667] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546932worker1 # [ 193.838928] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6933worker1 # [ 193.843777] nix-daemon[850]: reaped child process 2121, status = succeeded6934worker1 # [ 193.856292] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6935worker1 # [ 194.965015] nix-daemon[850]: reaped child process 2130, status = succeeded6936worker1 # [ 195.033748] nix-daemon[850]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)6937worker1 # [ 195.051448] nix-daemon[850]: reaped child process 2149, status = succeeded6938lb # [ 195.078384] niks3-server[1111]: 2026/09/23 05:15:09 INFO Received uploads request method=POST path=/api/pending_closures6939worker1 # [ 195.069890] nix-grpc-daemon[1844]: time=2026-09-23T05:15:09.572Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6940worker1 # [ 195.071669] nix-grpc-daemon[1844]: time=2026-09-23T05:15:09.572Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6941lb # [ 195.125485] niks3-server[1111]: 2026/09/23 05:15:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign6942worker1 # [ 195.115153] nix-grpc-daemon[1844]: time=2026-09-23T05:15:09.618Z level=INFO msg="Uploading 1 narinfos"6943lb # [ 195.139428] niks3-server[1111]: 2026/09/23 05:15:09 INFO Signed narinfos id=83 count=16944lb # [ 195.147505] niks3-server[1111]: 2026/09/23 05:15:09 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6945lb # [ 195.155987] niks3-server[1111]: 2026/09/23 05:15:09 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6946lb # [ 195.159589] niks3-server[1111]: 2026/09/23 05:15:09 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6947lb # [ 195.166266] niks3-server[1111]: 2026/09/23 05:15:09 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete6948lb # [ 195.173695] niks3-server[1111]: 2026/09/23 05:15:09 INFO Completed upload id=836949worker1 # [ 195.153827] nix-grpc-daemon[1844]: time=2026-09-23T05:15:09.657Z level=INFO msg="Upload complete. (287ms)"6950lb # [ 195.175538] niks3-server[1111]: 2026/09/23 05:15:09 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6951worker2 # [ 195.172323] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:09Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16952worker1 # [ 195.169148] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=16953worker1 # [ 195.185630] nix-daemon[850]: reaped child process 2128, status = succeeded6954worker1 # [ 195.209361] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546955worker1 # [ 195.224902] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546956worker1 # [ 195.238708] nix-grpc-daemon[1844]: time=2026-09-23T05:15:09.741Z level=INFO msg="All 2 paths already cached"6957worker1 # [ 195.248737] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544546958worker1 # [ 195.262386] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6959worker1 # [ 195.413172] nix-daemon[850]: accepted connection from pid 2153, user nix-grpc-daemon (trusted)6960worker1 # [ 195.430395] nix-daemon[850]: reaped child process 2156, status = succeeded6961lb # [ 195.455767] niks3-server[1111]: 2026/09/23 05:15:09 INFO Received uploads request method=POST path=/api/pending_closures6962worker1 # [ 195.445640] nix-grpc-daemon[1844]: time=2026-09-23T05:15:09.948Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6963worker1 # [ 195.447330] nix-grpc-daemon[1844]: time=2026-09-23T05:15:09.948Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6964worker1 # [ 195.476371] nix-grpc-daemon[1844]: time=2026-09-23T05:15:09.979Z level=INFO msg="Uploading 1 narinfos"6965lb # [ 195.484498] niks3-server[1111]: 2026/09/23 05:15:09 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6966lb # [ 195.498938] niks3-server[1111]: 2026/09/23 05:15:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign6967lb # [ 195.508932] niks3-server[1111]: 2026/09/23 05:15:10 INFO Signed narinfos id=84 count=16968worker1 # [ 195.499592] nix-grpc-daemon[1844]: time=2026-09-23T05:15:10.002Z level=INFO msg="Upload complete. (221ms)"6969lb # [ 195.519303] niks3-server[1111]: 2026/09/23 05:15:10 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6970worker1 # [ 195.508318] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=7526971lb # [ 195.530905] niks3-server[1111]: 2026/09/23 05:15:10 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete6972lb # [ 195.538123] niks3-server[1111]: 2026/09/23 05:15:10 INFO Completed upload id=846973lb # [ 195.541196] niks3-server[1111]: 2026/09/23 05:15:10 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6974worker1 # [ 195.520678] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546975worker1 # [ 195.531023] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6976worker1 # [ 195.534603] nix-daemon[850]: reaped child process 2151, status = succeeded6977worker1 # [ 195.544751] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6978worker1 # [ 196.850856] nix-daemon[850]: reaped child process 2160, status = succeeded6979worker1 # [ 196.930995] nix-daemon[850]: accepted connection from pid 2176, user nix-grpc-daemon (trusted)6980worker1 # [ 196.949050] nix-daemon[850]: reaped child process 2179, status = succeeded6981lb # [ 196.974990] niks3-server[1111]: 2026/09/23 05:15:11 INFO Received uploads request method=POST path=/api/pending_closures6982worker1 # [ 196.967607] nix-grpc-daemon[1844]: time=2026-09-23T05:15:11.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6983worker1 # [ 196.969615] nix-grpc-daemon[1844]: time=2026-09-23T05:15:11.469Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6984lb # [ 197.011534] niks3-server[1111]: 2026/09/23 05:15:11 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6985worker1 # [ 197.008752] nix-grpc-daemon[1844]: time=2026-09-23T05:15:11.511Z level=INFO msg="Uploading 1 narinfos"6986lb # [ 197.028285] niks3-server[1111]: 2026/09/23 05:15:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign6987lb # [ 197.037446] niks3-server[1111]: 2026/09/23 05:15:11 INFO Signed narinfos id=85 count=16988lb # [ 197.039980] niks3-server[1111]: 2026/09/23 05:15:11 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6989lb # [ 197.047969] niks3-server[1111]: 2026/09/23 05:15:11 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6990worker1 # [ 197.039022] nix-grpc-daemon[1844]: time=2026-09-23T05:15:11.542Z level=INFO msg="Upload complete. (276ms)"6991lb # [ 197.060104] niks3-server[1111]: 2026/09/23 05:15:11 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete6992lb # [ 197.063282] niks3-server[1111]: 2026/09/23 05:15:11 INFO Completed upload id=856993lb # [ 197.066263] niks3-server[1111]: 2026/09/23 05:15:11 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6994worker2 # [ 197.057757] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:11Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16995worker1 # [ 197.052834] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=16996worker1 # [ 197.068615] nix-daemon[850]: reaped child process 2158, status = succeeded6997worker1 # [ 197.097517] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544546998worker1 # [ 197.112432] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544546999worker1 # [ 197.126047] nix-grpc-daemon[1844]: time=2026-09-23T05:15:11.629Z level=INFO msg="All 2 paths already cached"7000worker1 # [ 197.135692] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544547001worker1 # [ 197.150509] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)7002worker1 # [ 197.302770] nix-daemon[850]: accepted connection from pid 2183, user nix-grpc-daemon (trusted)7003worker1 # [ 197.321488] nix-daemon[850]: reaped child process 2186, status = succeeded7004lb # [ 197.343603] niks3-server[1111]: 2026/09/23 05:15:11 INFO Received uploads request method=POST path=/api/pending_closures7005worker1 # [ 197.333705] nix-grpc-daemon[1844]: time=2026-09-23T05:15:11.836Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7006worker1 # [ 197.335291] nix-grpc-daemon[1844]: time=2026-09-23T05:15:11.836Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"7007lb # [ 197.374733] niks3-server[1111]: 2026/09/23 05:15:11 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls7008worker1 # [ 197.373606] nix-grpc-daemon[1844]: time=2026-09-23T05:15:11.876Z level=INFO msg="Uploading 1 narinfos"7009lb # [ 197.383897] niks3-server[1111]: 2026/09/23 05:15:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7010lb # [ 197.396947] niks3-server[1111]: 2026/09/23 05:15:11 INFO Signed narinfos id=86 count=17011lb # [ 197.404761] niks3-server[1111]: 2026/09/23 05:15:11 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst7012lb # [ 197.418608] niks3-server[1111]: 2026/09/23 05:15:11 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7013worker1 # [ 197.403946] nix-grpc-daemon[1844]: time=2026-09-23T05:15:11.907Z level=INFO msg="Upload complete. (239ms)"7014lb # [ 197.426001] niks3-server[1111]: 2026/09/23 05:15:11 INFO Completed upload id=867015lb # [ 197.430121] niks3-server[1111]: 2026/09/23 05:15:11 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo7016worker1 # [ 197.416571] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 paths=1 nar_bytes_in=7527017worker1 # [ 197.429706] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544547018worker1 # [ 197.435995] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)7019worker1 # [ 197.440996] nix-daemon[850]: reaped child process 2181, status = succeeded7020worker1 # [ 197.453547] nix-daemon[850]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)7021worker1 # [ 198.090704] nix-daemon[850]: reaped child process 2190, status = succeeded7022worker1 # [ 198.183686] nix-daemon[850]: accepted connection from pid 2206, user nix-grpc-daemon (trusted)7023worker1 # [ 198.198469] nix-daemon[850]: reaped child process 2209, status = succeeded7024lb # [ 198.226100] niks3-server[1111]: 2026/09/23 05:15:12 INFO Received uploads request method=POST path=/api/pending_closures7025worker1 # [ 198.218671] nix-grpc-daemon[1844]: time=2026-09-23T05:15:12.720Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7026worker1 # [ 198.220208] nix-grpc-daemon[1844]: time=2026-09-23T05:15:12.720Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7027lb # [ 198.265501] niks3-server[1111]: 2026/09/23 05:15:12 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7028lb # [ 198.277000] niks3-server[1111]: 2026/09/23 05:15:12 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7029worker1 # [ 198.270338] nix-grpc-daemon[1844]: time=2026-09-23T05:15:12.773Z level=INFO msg="Uploading 1 narinfos"7030lb # [ 198.290544] niks3-server[1111]: 2026/09/23 05:15:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7031lb # [ 198.300109] niks3-server[1111]: 2026/09/23 05:15:12 INFO Signed narinfos id=87 count=17032lb # [ 198.302791] niks3-server[1111]: 2026/09/23 05:15:12 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7033lb # [ 198.315209] niks3-server[1111]: 2026/09/23 05:15:12 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7034worker1 # [ 198.301107] nix-grpc-daemon[1844]: time=2026-09-23T05:15:12.804Z level=INFO msg="Upload complete. (239ms)"7035lb # [ 198.322940] niks3-server[1111]: 2026/09/23 05:15:12 INFO Completed upload id=877036lb # [ 198.325044] niks3-server[1111]: 2026/09/23 05:15:12 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7037worker1 # [ 198.312962] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54454 duration_s=0 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=17038worker1 # [ 198.326769] nix-daemon[850]: reaped child process 2188, status = succeeded7039client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.45 seconds)7040worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7041worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7042client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7043client # [ 198.629897] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7044client # Running as unit: gpu.service; invocation ID: ca7eda90d7034d30b84190d4c0e4aee37045client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)7046lb: must succeed: curl -sf localhost:9901/clusters7047lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7048lb: must succeed: curl -sf localhost:9901/clusters7049lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7050worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7051worker2 # [ 198.777667] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:595867052client # [ 198.835531] nix[1568]: don't know how to build these paths:7053client # [ 198.836555] nix[1568]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7054worker2 # [ 198.791887] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595867055worker2 # [ 198.811813] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:13Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07056client # [ 198.862361] nix[1568]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker7057worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)7058lb: must succeed: curl -sf localhost:9901/clusters7059lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7060lb: must succeed: curl -sf localhost:9901/clusters7061lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7062worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7063worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7064client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7065lb # [ 200.280703] envoy[908]: conn peer=[2001:db8:1::1]:45426 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27213 tx=12447 ms=161747066client # [ 228.869387] nix[1568]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}7067client # [ 228.883289] nix[1568]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7068client # [ 228.915286] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7069client # [ 228.923535] systemd[1]: gpu.service: Failed with result 'exit-code'.7070client # [ 228.927485] systemd[1]: gpu.service: Consumed 198ms CPU time over 30.290s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7071client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.06 seconds)7072client: must succeed: journalctl -u gpu -o cat7073client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7074lb: must succeed: curl -sf localhost:9901/clusters7075lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7076lb: must succeed: curl -sf localhost:9901/clusters7077lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7078worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7079worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7080(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.27 seconds)7081subtest: low disk drains a worker and builds go to the other7082worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7083lb # [ 230.277753] envoy[908]: conn peer=[2001:db8:1::1]:35584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=614 ms=300707084worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.19 seconds)7085worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7086worker1 # [ 230.834308] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:45Z level=info event=unhealthy reason=min_free7087worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)7088client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27089worker1 # [ 231.768276] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544547090client # don't know how to build these paths:7091client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7092worker1 # [ 231.778160] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544547093worker2 # [ 231.816429] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:46Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07094worker2 # [ 231.822085] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:46Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17095worker2 # [ 231.863378] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595867096worker2 # [ 231.878734] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595867097client # copying 1 paths to 'lb:50051'...7098client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7099worker2 # [ 231.894775] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595867100worker2 # [ 231.913491] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)7101worker2 # [ 232.086272] nix-daemon[842]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)7102worker2 # [ 232.107242] nix-daemon[842]: reaped child process 1937, status = succeeded7103lb # [ 232.125024] niks3-server[1111]: 2026/09/23 05:15:46 INFO Received uploads request method=POST path=/api/pending_closures7104worker2 # [ 232.126408] nix-grpc-daemon[1636]: time=2026-09-23T05:15:46.624Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7105worker2 # [ 232.127966] nix-grpc-daemon[1636]: time=2026-09-23T05:15:46.624Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7106lb # [ 232.161652] niks3-server[1111]: 2026/09/23 05:15:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7107worker2 # [ 232.158382] nix-grpc-daemon[1636]: time=2026-09-23T05:15:46.657Z level=INFO msg="Uploading 1 narinfos"7108lb # [ 232.177054] niks3-server[1111]: 2026/09/23 05:15:46 INFO Signed narinfos id=88 count=17109lb # [ 232.195190] niks3-server[1111]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7110lb # [ 232.203449] niks3-server[1111]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7111lb # [ 232.223246] niks3-server[1111]: 2026/09/23 05:15:46 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7112lb # [ 232.230504] niks3-server[1111]: 2026/09/23 05:15:46 INFO Completed upload id=887113worker2 # [ 232.219405] nix-grpc-daemon[1636]: time=2026-09-23T05:15:46.718Z level=INFO msg="Upload complete. (286ms)"7114lb # [ 232.233614] niks3-server[1111]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7115worker2 # [ 232.229955] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=0 paths=1 nar_bytes_in=5127116worker2 # [ 232.245058] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595867117worker2 # [ 232.258179] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)7118worker2 # [ 232.265246] nix-daemon[842]: reaped child process 1932, status = succeeded7119client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7120client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7121worker2 # [ 232.279089] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)7122client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7123client # farm-a> LOG-farm-a-drain7124worker2 # [ 233.790449] nix-daemon[842]: reaped child process 1941, status = succeeded7125worker2 # [ 233.861629] nix-daemon[842]: accepted connection from pid 1957, user nix-grpc-daemon (trusted)7126worker2 # [ 233.883021] nix-daemon[842]: reaped child process 1960, status = succeeded7127lb # [ 233.891473] niks3-server[1111]: 2026/09/23 05:15:48 INFO Received uploads request method=POST path=/api/pending_closures7128worker2 # [ 233.895165] nix-grpc-daemon[1636]: time=2026-09-23T05:15:48.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7129worker2 # [ 233.896840] nix-grpc-daemon[1636]: time=2026-09-23T05:15:48.392Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7130lb # [ 233.933827] niks3-server[1111]: 2026/09/23 05:15:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7131worker2 # [ 233.932550] nix-grpc-daemon[1636]: time=2026-09-23T05:15:48.431Z level=INFO msg="Uploading 1 narinfos"7132lb # [ 233.950108] niks3-server[1111]: 2026/09/23 05:15:48 INFO Signed narinfos id=89 count=17133lb # [ 233.956473] niks3-server[1111]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7134lb # [ 233.963401] niks3-server[1111]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7135lb # [ 233.971332] niks3-server[1111]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7136worker2 # [ 233.966782] nix-grpc-daemon[1636]: time=2026-09-23T05:15:48.466Z level=INFO msg="Upload complete. (275ms)"7137lb # [ 233.977408] niks3-server[1111]: 2026/09/23 05:15:48 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7138lb # [ 233.981460] niks3-server[1111]: 2026/09/23 05:15:48 INFO Completed upload id=897139lb # [ 233.983168] niks3-server[1111]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7140worker2 # [ 233.978486] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=17141worker2 # [ 234.000954] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:48Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07142worker2 # [ 234.006186] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:48Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17143worker2 # [ 234.010915] nix-daemon[842]: reaped child process 1939, status = succeeded7144worker2 # [ 234.030345] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595867145worker2 # [ 234.045332] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595867146worker2 # [ 234.060254] nix-grpc-daemon[1636]: time=2026-09-23T05:15:48.559Z level=INFO msg="All 1 paths already cached"7147client # copying 1 paths to 'lb:50051'...7148worker2 # [ 234.071759] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595867149client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7150worker2 # [ 234.088150] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)7151worker2 # [ 234.255254] nix-daemon[842]: accepted connection from pid 1966, user nix-grpc-daemon (trusted)7152worker2 # [ 234.272244] nix-daemon[842]: reaped child process 1969, status = succeeded7153lb # [ 234.288689] niks3-server[1111]: 2026/09/23 05:15:48 INFO Received uploads request method=POST path=/api/pending_closures7154worker2 # [ 234.287230] nix-grpc-daemon[1636]: time=2026-09-23T05:15:48.786Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7155worker2 # [ 234.289008] nix-grpc-daemon[1636]: time=2026-09-23T05:15:48.786Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7156lb # [ 234.316920] niks3-server[1111]: 2026/09/23 05:15:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7157worker2 # [ 234.313477] nix-grpc-daemon[1636]: time=2026-09-23T05:15:48.812Z level=INFO msg="Uploading 1 narinfos"7158lb # [ 234.330303] niks3-server[1111]: 2026/09/23 05:15:48 INFO Signed narinfos id=90 count=17159lb # [ 234.336741] niks3-server[1111]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7160lb # [ 234.349518] niks3-server[1111]: 2026/09/23 05:15:48 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7161worker2 # [ 234.344674] nix-grpc-daemon[1636]: time=2026-09-23T05:15:48.844Z level=INFO msg="Upload complete. (238ms)"7162lb # [ 234.359353] niks3-server[1111]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7163lb # [ 234.365300] niks3-server[1111]: 2026/09/23 05:15:48 INFO Completed upload id=907164worker2 # [ 234.356196] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=0 paths=1 nar_bytes_in=6327165lb # [ 234.368844] niks3-server[1111]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7166worker2 # [ 234.369534] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595867167worker2 # [ 234.376586] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)7168worker2 # [ 234.381875] nix-daemon[842]: reaped child process 1964, status = succeeded7169client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7170worker2 # [ 234.394675] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)7171client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7172client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7173client # farm-b> LOG-farm-b-drain7174worker2 # [ 235.300941] nix-daemon[842]: reaped child process 1973, status = succeeded7175worker2 # [ 235.372317] nix-daemon[842]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)7176worker2 # [ 235.394649] nix-daemon[842]: reaped child process 1992, status = succeeded7177lb # [ 235.406541] niks3-server[1111]: 2026/09/23 05:15:49 INFO Received uploads request method=POST path=/api/pending_closures7178worker2 # [ 235.404508] nix-grpc-daemon[1636]: time=2026-09-23T05:15:49.903Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7179worker2 # [ 235.406135] nix-grpc-daemon[1636]: time=2026-09-23T05:15:49.903Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7180lb # [ 235.447076] niks3-server[1111]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7181worker2 # [ 235.458213] nix-grpc-daemon[1636]: time=2026-09-23T05:15:49.957Z level=INFO msg="Uploading 1 narinfos"7182lb # [ 235.462267] niks3-server[1111]: 2026/09/23 05:15:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7183lb # [ 235.476387] niks3-server[1111]: 2026/09/23 05:15:49 INFO Signed narinfos id=91 count=17184lb # [ 235.482320] niks3-server[1111]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7185lb # [ 235.493229] niks3-server[1111]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7186lb # [ 235.502175] niks3-server[1111]: 2026/09/23 05:15:50 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7187lb # [ 235.512206] niks3-server[1111]: 2026/09/23 05:15:50 INFO Completed upload id=917188worker2 # [ 235.500537] nix-grpc-daemon[1636]: time=2026-09-23T05:15:49.999Z level=INFO msg="Upload complete. (296ms)"7189lb # [ 235.514518] niks3-server[1111]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7190worker2 # [ 235.511684] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=17191worker2 # [ 235.532107] nix-daemon[842]: reaped child process 1971, status = succeeded7192worker2 # [ 235.558874] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:50Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07193worker2 # [ 235.565735] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:50Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17194worker2 # [ 235.576182] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595867195worker2 # [ 235.587747] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595867196worker2 # [ 235.601094] nix-grpc-daemon[1636]: time=2026-09-23T05:15:50.100Z level=INFO msg="All 2 paths already cached"7197client # copying 1 paths to 'lb:50051'...7198client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7199worker2 # [ 235.612490] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595867200worker2 # [ 235.629534] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)7201worker2 # [ 235.783557] nix-daemon[842]: accepted connection from pid 1998, user nix-grpc-daemon (trusted)7202worker2 # [ 235.802820] nix-daemon[842]: reaped child process 2001, status = succeeded7203lb # [ 235.817213] niks3-server[1111]: 2026/09/23 05:15:50 INFO Received uploads request method=POST path=/api/pending_closures7204worker2 # [ 235.815639] nix-grpc-daemon[1636]: time=2026-09-23T05:15:50.314Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7205worker2 # [ 235.817437] nix-grpc-daemon[1636]: time=2026-09-23T05:15:50.314Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7206lb # [ 235.850340] niks3-server[1111]: 2026/09/23 05:15:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7207worker2 # [ 235.846883] nix-grpc-daemon[1636]: time=2026-09-23T05:15:50.346Z level=INFO msg="Uploading 1 narinfos"7208lb # [ 235.864529] niks3-server[1111]: 2026/09/23 05:15:50 INFO Signed narinfos id=92 count=17209lb # [ 235.870125] niks3-server[1111]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7210worker2 # [ 235.873825] nix-grpc-daemon[1636]: time=2026-09-23T05:15:50.373Z level=INFO msg="Upload complete. (227ms)"7211lb # [ 235.884739] niks3-server[1111]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7212worker2 # [ 235.882708] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=0 paths=1 nar_bytes_in=7607213lb # [ 235.894148] niks3-server[1111]: 2026/09/23 05:15:50 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7214lb # [ 235.898880] niks3-server[1111]: 2026/09/23 05:15:50 INFO Completed upload id=927215lb # [ 235.902147] niks3-server[1111]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7216worker2 # [ 235.897450] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595867217worker2 # [ 235.903897] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)7218worker2 # [ 235.909214] nix-daemon[842]: reaped child process 1996, status = succeeded7219client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7220worker2 # [ 235.920114] nix-daemon[842]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)7221client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7222client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7223client # farm-top> LOG-farm-top-drain7224worker2 # [ 236.812332] nix-daemon[842]: reaped child process 2005, status = succeeded7225worker2 # [ 236.896235] nix-daemon[842]: accepted connection from pid 2021, user nix-grpc-daemon (trusted)7226worker2 # [ 236.917681] nix-daemon[842]: reaped child process 2024, status = succeeded7227lb # [ 236.926777] niks3-server[1111]: 2026/09/23 05:15:51 INFO Received uploads request method=POST path=/api/pending_closures7228worker2 # [ 236.926573] nix-grpc-daemon[1636]: time=2026-09-23T05:15:51.425Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7229worker2 # [ 236.930292] nix-grpc-daemon[1636]: time=2026-09-23T05:15:51.425Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7230lb # [ 236.973517] niks3-server[1111]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7231worker2 # [ 236.973065] nix-grpc-daemon[1636]: time=2026-09-23T05:15:51.472Z level=INFO msg="Uploading 1 narinfos"7232lb # [ 236.987278] niks3-server[1111]: 2026/09/23 05:15:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7233lb # [ 236.995832] niks3-server[1111]: 2026/09/23 05:15:51 INFO Signed narinfos id=93 count=17234lb # [ 237.000972] niks3-server[1111]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7235lb # [ 237.003883] niks3-server[1111]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7236lb # [ 237.013768] niks3-server[1111]: 2026/09/23 05:15:51 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7237lb # [ 237.017475] niks3-server[1111]: 2026/09/23 05:15:51 INFO Completed upload id=937238worker2 # [ 237.007267] nix-grpc-daemon[1636]: time=2026-09-23T05:15:51.506Z level=INFO msg="Upload complete. (274ms)"7239lb # [ 237.020368] niks3-server[1111]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7240worker2 # [ 237.021130] nix-grpc-daemon[1559]: ts=2026-09-23T05:15:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59586 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=17241worker2 # [ 237.038306] nix-daemon[842]: reaped child process 2003, status = succeeded7242client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.55 seconds)7243client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7244client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds)7245worker1: must succeed: rm /nix/.rw-store/fill7246worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7247worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7248worker1 # [ 237.844687] nix-grpc-daemon[1783]: ts=2026-09-23T05:15:52Z level=info event=healthy reason=min_free7249worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7250(finished: subtest: low disk drains a worker and builds go to the other, in 9.37 seconds)7251(finished: run the VM test script, in 239.66 seconds)7252test script finished in 239.91s7253cleanup7254kill QemuMachine (pid 51)7255client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7256client # [2026-09-23T05:15:52Z INFO virtiofsd] Client disconnected, shutting down7257client # [2026-09-23T05:15:53Z INFO virtiofsd] Client disconnected, shutting down7258client # [2026-09-23T05:15:53Z INFO virtiofsd] Client disconnected, shutting down7259kill QemuMachine (pid 49)7260lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7261lb # [2026-09-23T05:15:53Z INFO virtiofsd] Client disconnected, shutting down7262lb # [2026-09-23T05:15:53Z INFO virtiofsd] Client disconnected, shutting down7263lb # [2026-09-23T05:15:53Z INFO virtiofsd] Client disconnected, shutting down7264kill QemuMachine (pid 50)7265worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7266worker1 # [2026-09-23T05:15:53Z INFO virtiofsd] Client disconnected, shutting down7267worker1 # [2026-09-23T05:15:53Z INFO virtiofsd] Client disconnected, shutting down7268worker1 # [2026-09-23T05:15:53Z INFO virtiofsd] Client disconnected, shutting down7269kill QemuMachine (pid 52)7270worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7271worker2 # [2026-09-23T05:15:54Z INFO virtiofsd] Client disconnected, shutting down7272worker2 # [2026-09-23T05:15:54Z INFO virtiofsd] Client disconnected, shutting down7273worker2 # [2026-09-23T05:15:54Z INFO virtiofsd] Client disconnected, shutting down7274(finished: cleanup, in 1.28 seconds)7275additionally exposed symbols:7276 client, lb, worker1, worker2,7277 vlan1,7278 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh