vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #532
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14worker1 # Disk image does not exist, creating the virtualisation disk image...15lb: QEMU running (pid 49)16worker2: QEMU running (pid 52)17client # Disk image does not exist, creating the virtualisation disk image...18client: QEMU running (pid 50)19worker1: QEMU running (pid 51)20worker2 # Formatting '/build/vm-state-worker2/tmp.0uZclR9QTt', fmt=raw size=107374182421worker1 # Formatting '/build/vm-state-worker1/tmp.pG58sVlxY7', fmt=raw size=107374182422client # Formatting '/build/vm-state-client/tmp.BLazXudYBz', fmt=raw size=107374182423worker1 # mke2fs 1.47.4 (6-Mar-2025)24worker2 # mke2fs 1.47.4 (6-Mar-2025)25worker1 # Discarding device blocks: 0/262144 done26lb # Formatting '/build/vm-state-lb/tmp.uANvhdVuxc', fmt=raw size=107374182427worker2 # Discarding device blocks: 0/262144 done28worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes29lb # mke2fs 1.47.4 (6-Mar-2025)30worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # Filesystem UUID: 6791fd92-16a0-4211-9718-3e507c5a93dc32lb # Discarding device blocks: 0/262144 done33worker2 # Filesystem UUID: 92c65040-f127-4aa2-8d3f-abf260cc7c6434worker1 # Superblock backups stored on blocks:35lb # Creating filesystem with 262144 4k blocks and 65536 inodes36client # mke2fs 1.47.4 (6-Mar-2025)37(finished: start all VMs, in 0.48 seconds)38worker2 # Superblock backups stored on blocks:39worker1 # 32768, 98304, 163840, 22937640lb # Filesystem UUID: 403c5b88-34c7-4837-963b-b9ae6e158bd541worker2 # 32768, 98304, 163840, 22937642worker1 # 43lb # Superblock backups stored on blocks:44worker2 # 45worker1 # Allocating group tables: 0/8 done46lb # 32768, 98304, 163840, 22937647worker2 # Allocating group tables: 0/8 done48worker1 # Writing inode tables: 0/8 done49lb # 50worker2 # Writing inode tables: 0/8 done51worker1 # Creating journal (8192 blocks): done52lb # Allocating group tables: 0/8 done53worker2 # Creating journal (8192 blocks): done54worker1 # Writing superblocks and filesystem accounting information: 0/8 done55lb # Writing inode tables: 0/8 done56worker2 # Writing superblocks and filesystem accounting information: 0/8 done57worker1 # 58lb # Creating journal (8192 blocks): done59worker2 # 60worker1 # Virtualisation disk image created.61lb # Writing superblocks and filesystem accounting information: 0/8 done62worker2 # Virtualisation disk image created.63worker1 # Starting virtiofs daemons...64lb # 65worker2 # Starting virtiofs daemons...66worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67lb # Virtualisation disk image created.68worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70lb # Starting virtiofs daemons...71worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...73lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...75worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)78worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79lb # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...80worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether81worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)82lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...84worker1 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether85lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether86worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)87worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...88lb # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...89worker2 # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...91lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)92worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests94lb # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests96worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests97lb # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...98worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests99worker1 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests100lb # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests101worker2 # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests102client # Discarding device blocks: 0/262144 done103client: waiting for unit network-addresses-eth1.service104client # Creating filesystem with 262144 4k blocks and 65536 inodes105client: waiting for the VM to finish booting106client # Filesystem UUID: 8e1e7877-f7b8-4126-bb0a-62d6c498188f107lb # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests108client # Superblock backups stored on blocks:109lb # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests110client # 32768, 98304, 163840, 229376111client # 112client # Allocating group tables: 0/8 done113client # Writing inode tables: 0/8 done114client # Creating journal (8192 blocks): done115client # Writing superblocks and filesystem accounting information: 0/8 done116client # 117client # Virtualisation disk image created.118client # Starting virtiofs daemons...119client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)122client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether123client # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...124client # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...125client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126client # [2026-09-23T05:32:51Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127client # [2026-09-23T05:32:51Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-23T05:32:51Z INFO virtiofsd] Client connected, servicing requests131lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133lb # [ 0.000000] KASLR enabled134lb # [ 0.000000] random: crng init done135lb # [ 0.000000] Machine model: linux,dummy-virt136lb # [ 0.000000] efi: UEFI not found.137lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140lb # [ 0.000000] Zone ranges:141lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142lb # [ 0.000000] DMA32 empty143lb # [ 0.000000] Normal empty144lb # [ 0.000000] Device empty145lb # [ 0.000000] Movable zone start for each node146lb # [ 0.000000] Early memory node ranges147lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150lb # [ 0.000000] psci: probing for conduit method from DT.151lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.152lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153lb # [ 0.000000] psci: Trusted OS migration not required154lb # [ 0.000000] psci: SMC Calling Convention v1.1155lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157lb # [ 0.000000] Detected PIPT I-cache on CPU0158lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159lb # [ 0.000000] CPU features: detected: GICv3 CPU interface160lb # [ 0.000000] CPU features: detected: Spectre-v4161lb # [ 0.000000] CPU features: detected: Spectre-BHB162lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164lb # [ 0.000000] alternatives: applying boot alternatives165lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vyrpk0yvg81ch7n98c7n14mzchdldfk4-nixos-system-lb-test/init regInfo=/nix/store/56sfh65ns0h9zv3wlb18ccjn63ky2v3j-closure-info/registration console=ttyAMA0,115200n8 console=tty0166lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/56sfh65ns0h9zv3wlb18ccjn63ky2v3j-closure-info/registration", will be passed to user space.167lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes168lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)169lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)170lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB171lb # [ 0.000000] software IO TLB: area num 1.172lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB173lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)174lb # [ 0.000000] Fallback order for Node 0: 0175lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216176lb # [ 0.000000] Policy zone: DMA177lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off178lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1179lb # [ 0.000000] allocated 3145728 bytes of page_ext180lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages181lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups182lb # [ 0.000000] rcu: Hierarchical RCU implementation.183lb # [ 0.000000] rcu: RCU event tracing is enabled.184lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.185lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.186lb # [ 0.000000] Rude variant of Tasks RCU enabled.187lb # [ 0.000000] Tracing variant of Tasks RCU enabled.188lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.189lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1190lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.191lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.192lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.193lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0194lb # [ 0.000000] GICv3: 256 SPIs implemented195lb # [ 0.000000] GICv3: 0 Extended SPIs implemented196lb # [ 0.000000] Root IRQ handler: gic_handle_irq197lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI198lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0199lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000200lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]201client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]202lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)203worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]204client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026205lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)206client # [ 0.000000] KASLR enabled207worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026208client # [ 0.000000] random: crng init done209lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000210worker2 # [ 0.000000] KASLR enabled211client # [ 0.000000] Machine model: linux,dummy-virt212worker2 # [ 0.000000] random: crng init done213client # [ 0.000000] efi: UEFI not found.214lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000215worker2 # [ 0.000000] Machine model: linux,dummy-virt216worker2 # [ 0.000000] efi: UEFI not found.217client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT218lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.219worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT220client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]221lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns222worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]223client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]224client # [ 0.000000] Zone ranges:225lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).226worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]227worker2 # [ 0.000000] Zone ranges:228client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]229client # [ 0.000000] DMA32 empty230lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns231client # [ 0.000000] Normal empty232worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]233worker2 # [ 0.000000] DMA32 empty234client # [ 0.000000] Device empty235worker2 # [ 0.000000] Normal empty236client # [ 0.000000] Movable zone start for each node237lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns238worker2 # [ 0.000000] Device empty239client # [ 0.000000] Early memory node ranges240lb # [ 0.000034] arm-pv: using stolen time PV241worker2 # [ 0.000000] Movable zone start for each node242client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]243worker2 # [ 0.000000] Early memory node ranges244lb # [ 0.000653] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)245worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]246client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]247lb # [ 0.000817] Console: colour dummy device 80x25248client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000249worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]250lb # [ 0.000824] printk: legacy console [tty0] enabled251client # [ 0.000000] psci: probing for conduit method from DT.252worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000253worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]254client # [ 0.000000] psci: PSCIv1.3 detected in firmware.255worker2 # [ 0.000000] psci: probing for conduit method from DT.256lb # [ 0.001011] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)257client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs258worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.259lb # [ 0.001018] pid_max: default: 32768 minimum: 301260client # [ 0.000000] psci: Trusted OS migration not required261worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026262worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs263worker1 # [ 0.000000] KASLR enabled264client # [ 0.000000] psci: SMC Calling Convention v1.1265lb # [ 0.001088] LSM: initializing lsm=capability,landlock,yama,bpf,ima266worker2 # [ 0.000000] psci: Trusted OS migration not required267lb # [ 0.001258] landlock: Up and running.268worker1 # [ 0.000000] random: crng init done269worker2 # [ 0.000000] psci: SMC Calling Convention v1.1270lb # [ 0.001261] Yama: becoming mindful.271worker1 # [ 0.000000] Machine model: linux,dummy-virt272client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)273worker1 # [ 0.000000] efi: UEFI not found.274lb # [ 0.001921] LSM support for eBPF active275worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)276client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296277lb # [ 0.002086] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)278worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT279client # [ 0.000000] Detected PIPT I-cache on CPU0280worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296281worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]282worker2 # [ 0.000000] Detected PIPT I-cache on CPU0283lb # [ 0.002118] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)284client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)285lb # [ 0.003947] rcu: Hierarchical SRCU implementation.286worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]287client # [ 0.000000] CPU features: detected: GICv3 CPU interface288worker1 # [ 0.000000] Zone ranges:289worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)290lb # [ 0.003952] rcu: Max phase no-delay instances is 1000.291client # [ 0.000000] CPU features: detected: Spectre-v4292worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface293worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]294lb # [ 0.005186] fsl-mc MSI: its@8080000 domain created295client # [ 0.000000] CPU features: detected: Spectre-BHB296worker1 # [ 0.000000] DMA32 empty297worker2 # [ 0.000000] CPU features: detected: Spectre-v4298lb # [ 0.005283] EFI services will not be available.299worker1 # [ 0.000000] Normal empty300client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38301worker2 # [ 0.000000] CPU features: detected: Spectre-BHB302worker1 # [ 0.000000] Device empty303lb # [ 0.005372] smp: Bringing up secondary CPUs ...304lb # [ 0.005381] smp: Brought up 1 node, 1 CPU305worker1 # [ 0.000000] Movable zone start for each node306client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23307worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38308worker1 # [ 0.000000] Early memory node ranges309lb # [ 0.005383] SMP: Total of 1 processors activated.310client # [ 0.000000] alternatives: applying boot alternatives311worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23312lb # [ 0.005389] CPU: All CPU(s) started at EL1313worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]314worker2 # [ 0.000000] alternatives: applying boot alternatives315lb # [ 0.005399] CPU features: detected: Branch Target Identification316worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]317lb # [ 0.005404] CPU features: detected: ARMv8.4 Translation Table Level318worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000319worker1 # [ 0.000000] psci: probing for conduit method from DT.320lb # [ 0.005407] CPU features: detected: Instruction cache invalidation not required for I/D coherence321worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.322lb # [ 0.005410] CPU features: detected: Data cache clean to the PoU not required for I/D coherence323worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs324client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test/init regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration console=ttyAMA0,115200n8 console=tty0325worker1 # [ 0.000000] psci: Trusted OS migration not required326worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bszflazskwamnpdm0gzh5fsvsms7kvza-nixos-system-worker2-test/init regInfo=/nix/store/0fpyr3903f58d5irnb20j9w5pnq32y4y-closure-info/registration console=ttyAMA0,115200n8 console=tty0327lb # [ 0.005414] CPU features: detected: Common not Private translations328worker1 # [ 0.000000] psci: SMC Calling Convention v1.1329lb # [ 0.005417] CPU features: detected: CRC32 instructions330client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration", will be passed to user space.331lb # [ 0.005420] CPU features: detected: Data cache clean to Point of Deep Persistence332worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)333worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0fpyr3903f58d5irnb20j9w5pnq32y4y-closure-info/registration", will be passed to user space.334client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes335worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296336lb # [ 0.005424] CPU features: detected: Data cache clean to Point of Persistence337worker1 # [ 0.000000] Detected PIPT I-cache on CPU0338client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)339lb # [ 0.005427] CPU features: detected: Data independent timing control (DIT)340worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes341client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)342worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)343worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)344lb # [ 0.005430] CPU features: detected: E0PD345worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface346worker1 # [ 0.000000] CPU features: detected: Spectre-v4347lb # [ 0.005432] CPU features: detected: Enhanced Counter Virtualization348worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)349client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB350worker1 # [ 0.000000] CPU features: detected: Spectre-BHB351client # [ 0.000000] software IO TLB: area num 1.352worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB353lb # [ 0.005435] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)354worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38355worker2 # [ 0.000000] software IO TLB: area num 1.356client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB357lb # [ 0.005439] CPU features: detected: Enhanced Virtualization Traps358worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23359worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB360lb # [ 0.005442] CPU features: detected: Fine Grained Traps361client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)362worker1 # [ 0.000000] alternatives: applying boot alternatives363client # [ 0.000000] Fallback order for Node 0: 0364worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)365lb # [ 0.005446] CPU features: detected: Generic authentication (architected QARMA5 algorithm)366worker2 # [ 0.000000] Fallback order for Node 0: 0367client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216368lb # [ 0.005451] CPU features: detected: RCpc load-acquire (LDAPR)369client # [ 0.000000] Policy zone: DMA370worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216371lb # [ 0.005453] CPU features: detected: LSE atomic instructions372worker2 # [ 0.000000] Policy zone: DMA373client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off374lb # [ 0.005456] CPU features: detected: Privileged Access Never375worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off376client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1377lb # [ 0.005459] CPU features: detected: PMUv3378client # [ 0.000000] allocated 3145728 bytes of page_ext379worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1380lb # [ 0.005462] CPU features: detected: RAS Extension Support381worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4cjy3h5garv6kdzqg4fsdb44fsjfs9vp-nixos-system-worker1-test/init regInfo=/nix/store/kilsa45kilzg2azh7zrbnmf6qgdvl019-closure-info/registration console=ttyAMA0,115200n8 console=tty0382client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages383worker2 # [ 0.000000] allocated 3145728 bytes of page_ext384lb # [ 0.005465] CPU features: detected: RASv1p1 Extension Support385client # [ 0.000000] ftrace: allocated 294 pages with 4 groups386worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages387lb # [ 0.005467] CPU features: detected: Random Number Generator388client # [ 0.000000] rcu: Hierarchical RCU implementation.389worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kilsa45kilzg2azh7zrbnmf6qgdvl019-closure-info/registration", will be passed to user space.390worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups391lb # [ 0.005470] CPU features: detected: Speculation barrier (SB)392client # [ 0.000000] rcu: RCU event tracing is enabled.393worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.394worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes395lb # [ 0.005473] CPU features: detected: Stage-2 Force Write-Back396worker2 # [ 0.000000] rcu: RCU event tracing is enabled.397client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.398worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)399client # [ 0.000000] Trampoline variant of Tasks RCU enabled.400lb # [ 0.005476] CPU features: detected: TLB range maintenance instructions401worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.402client # [ 0.000000] Rude variant of Tasks RCU enabled.403lb # [ 0.005480] CPU features: detected: Speculative Store Bypassing Safe (SSBS)404worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.405worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)406client # [ 0.000000] Tracing variant of Tasks RCU enabled.407worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.408lb # [ 0.005521] alternatives: applying system-wide alternatives409worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB410worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.411client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.412worker1 # [ 0.000000] software IO TLB: area num 1.413lb # [ 0.008518] CPU features: detected: BBM Level 2 without TLB conflict abort414client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1415worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.416worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB417worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1418client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.419lb # [ 0.008682] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)420worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)421lb # [ 0.009075] devtmpfs: initialized422worker1 # [ 0.000000] Fallback order for Node 0: 0423worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.424client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.425lb # [ 0.010952] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)426worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216427worker1 # [ 0.000000] Policy zone: DMA428worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.429client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.430lb # [ 0.010980] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).431worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off432client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0433worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.434lb # [ 0.011155] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL435client # [ 0.000000] GICv3: 256 SPIs implemented436worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1437worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0438lb # [ 0.011160] 0 pages in range for non-PLT usage439client # [ 0.000000] GICv3: 0 Extended SPIs implemented440worker1 # [ 0.000000] allocated 3145728 bytes of page_ext441worker2 # [ 0.000000] GICv3: 256 SPIs implemented442lb # [ 0.011161] 508272 pages in range for PLT usage443client # [ 0.000000] Root IRQ handler: gic_handle_irq444worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages445worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented446lb # [ 0.011267] pinctrl core: initialized pinctrl subsystem447client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI448worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups449worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq450lb # [ 0.012025] DMI not present or invalid.451client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0452worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI453worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.454lb # [ 0.015725] NET: Registered PF_NETLINK/PF_ROUTE protocol family455worker1 # [ 0.000000] rcu: RCU event tracing is enabled.456client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000457worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0458lb # [ 0.017943] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations459client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]460worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.461worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000462worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]463lb # [ 0.018116] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations464worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.465client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)466worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.467lb # [ 0.018321] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations468worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)469worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.470client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)471lb # [ 0.018342] audit: initializing netlink subsys (disabled)472worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.473client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000474worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)475lb # [ 0.018922] thermal_sys: Registered thermal governor 'fair_share'476worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1477client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000478worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000479lb # [ 0.018923] thermal_sys: Registered thermal governor 'bang_bang'480lb # [ 0.018927] thermal_sys: Registered thermal governor 'step_wise'481client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.482worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000483worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.484lb # [ 0.018930] thermal_sys: Registered thermal governor 'user_space'485worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.486client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns487worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.488lb # [ 0.018935] thermal_sys: Registered thermal governor 'power_allocator'489client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).490worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns491worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.492lb # [ 0.018960] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1493worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).494worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0495lb # [ 0.018969] cpuidle: using governor ladder496client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns497lb # [ 0.018975] cpuidle: using governor menu498worker1 # [ 0.000000] GICv3: 256 SPIs implemented499worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented500worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns501client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns502lb # [ 0.019164] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.503worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq504client # [ 0.000029] arm-pv: using stolen time PV505lb # [ 0.019181] ASID allocator initialised with 65536 entries506worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns507worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI508lb # [ 0.020439] Serial: AMBA PL011 UART driver509worker2 # [ 0.000030] arm-pv: using stolen time PV510worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0511client # [ 0.000443] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)512lb # [ 0.025972] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1513worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000514client # [ 0.000587] Console: colour dummy device 80x25515worker2 # [ 0.000510] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)516lb # [ 0.026104] printk: console [ttyAMA0] enabled517client # [ 0.000594] printk: legacy console [tty0] enabled518worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]519worker2 # [ 0.000676] Console: colour dummy device 80x25520worker2 # [ 0.000684] printk: legacy console [tty0] enabled521worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)522client # [ 0.000803] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)523client # [ 0.000810] pid_max: default: 32768 minimum: 301524worker2 # [ 0.000876] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)525worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)526client # [ 0.000889] LSM: initializing lsm=capability,landlock,yama,bpf,ima527worker2 # [ 0.000883] pid_max: default: 32768 minimum: 301528client # [ 0.001041] landlock: Up and running.529worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000530client # [ 0.001043] Yama: becoming mindful.531worker2 # [ 0.000952] LSM: initializing lsm=capability,landlock,yama,bpf,ima532client # [ 0.001537] LSM support for eBPF active533worker2 # [ 0.001090] landlock: Up and running.534worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000535worker2 # [ 0.001093] Yama: becoming mindful.536client # [ 0.001697] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)537worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.538worker2 # [ 0.001535] LSM support for eBPF active539worker2 # [ 0.001690] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)540client # [ 0.001731] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)541worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns542client # [ 0.003565] rcu: Hierarchical SRCU implementation.543worker2 # [ 0.001723] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)544worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).545client # [ 0.003570] rcu: Max phase no-delay instances is 1000.546worker2 # [ 0.003508] rcu: Hierarchical SRCU implementation.547client # [ 0.004799] fsl-mc MSI: its@8080000 domain created548worker2 # [ 0.003512] rcu: Max phase no-delay instances is 1000.549worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns550client # [ 0.004889] EFI services will not be available.551lb # [ 0.145270] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages552worker2 # [ 0.004746] fsl-mc MSI: its@8080000 domain created553client # [ 0.004966] smp: Bringing up secondary CPUs ...554lb # [ 0.145288] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page555worker2 # [ 0.004836] EFI services will not be available.556worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns557client # [ 0.004974] smp: Brought up 1 node, 1 CPU558worker1 # [ 0.000028] arm-pv: using stolen time PV559worker2 # [ 0.004910] smp: Bringing up secondary CPUs ...560lb # [ 0.145293] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages561client # [ 0.004977] SMP: Total of 1 processors activated.562worker2 # [ 0.004918] smp: Brought up 1 node, 1 CPU563client # [ 0.004982] CPU: All CPU(s) started at EL1564lb # [ 0.145297] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page565worker2 # [ 0.004921] SMP: Total of 1 processors activated.566worker1 # [ 0.000388] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)567client # [ 0.004992] CPU features: detected: Branch Target Identification568worker2 # [ 0.004926] CPU: All CPU(s) started at EL1569lb # [ 0.145302] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages570worker1 # [ 0.000556] Console: colour dummy device 80x25571client # [ 0.004997] CPU features: detected: ARMv8.4 Translation Table Level572worker2 # [ 0.004936] CPU features: detected: Branch Target Identification573lb # [ 0.145306] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page574worker1 # [ 0.000563] printk: legacy console [tty0] enabled575worker2 # [ 0.004940] CPU features: detected: ARMv8.4 Translation Table Level576lb # [ 0.145311] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages577client # [ 0.005000] CPU features: detected: Instruction cache invalidation not required for I/D coherence578worker1 # [ 0.000755] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)579lb # [ 0.145315] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page580worker1 # [ 0.000761] pid_max: default: 32768 minimum: 301581worker2 # [ 0.004943] CPU features: detected: Instruction cache invalidation not required for I/D coherence582client # [ 0.005003] CPU features: detected: Data cache clean to the PoU not required for I/D coherence583worker1 # [ 0.000839] LSM: initializing lsm=capability,landlock,yama,bpf,ima584client # [ 0.005007] CPU features: detected: Common not Private translations585worker2 # [ 0.004947] CPU features: detected: Data cache clean to the PoU not required for I/D coherence586worker1 # [ 0.001016] landlock: Up and running.587client # [ 0.005010] CPU features: detected: CRC32 instructions588worker1 # [ 0.001019] Yama: becoming mindful.589worker2 # [ 0.004951] CPU features: detected: Common not Private translations590lb # [ 0.152410] fbcon: Taking over console591worker1 # [ 0.001445] LSM support for eBPF active592client # [ 0.005013] CPU features: detected: Data cache clean to Point of Deep Persistence593worker2 # [ 0.004955] CPU features: detected: CRC32 instructions594lb # [ 0.152423] ACPI: Interpreter disabled.595worker1 # [ 0.001584] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)596client # [ 0.005017] CPU features: detected: Data cache clean to Point of Persistence597worker2 # [ 0.004958] CPU features: detected: Data cache clean to Point of Deep Persistence598worker1 # [ 0.001615] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)599client # [ 0.005019] CPU features: detected: Data independent timing control (DIT)600worker2 # [ 0.004961] CPU features: detected: Data cache clean to Point of Persistence601client # [ 0.005023] CPU features: detected: E0PD602worker1 # [ 0.003436] rcu: Hierarchical SRCU implementation.603worker2 # [ 0.004964] CPU features: detected: Data independent timing control (DIT)604worker1 # [ 0.003440] rcu: Max phase no-delay instances is 1000.605client # [ 0.005025] CPU features: detected: Enhanced Counter Virtualization606worker2 # [ 0.004967] CPU features: detected: E0PD607worker1 # [ 0.004677] fsl-mc MSI: its@8080000 domain created608client # [ 0.005028] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)609worker2 # [ 0.004970] CPU features: detected: Enhanced Counter Virtualization610worker1 # [ 0.004767] EFI services will not be available.611client # [ 0.005032] CPU features: detected: Enhanced Virtualization Traps612worker1 # [ 0.004829] smp: Bringing up secondary CPUs ...613worker2 # [ 0.004973] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)614client # [ 0.005035] CPU features: detected: Fine Grained Traps615worker1 # [ 0.004836] smp: Brought up 1 node, 1 CPU616worker2 # [ 0.004977] CPU features: detected: Enhanced Virtualization Traps617worker1 # [ 0.004839] SMP: Total of 1 processors activated.618worker2 # [ 0.004980] CPU features: detected: Fine Grained Traps619worker1 # [ 0.004844] CPU: All CPU(s) started at EL1620client # [ 0.005038] CPU features: detected: Generic authentication (architected QARMA5 algorithm)621client # [ 0.005043] CPU features: detected: RCpc load-acquire (LDAPR)622worker1 # [ 0.004854] CPU features: detected: Branch Target Identification623worker2 # [ 0.004983] CPU features: detected: Generic authentication (architected QARMA5 algorithm)624client # [ 0.005046] CPU features: detected: LSE atomic instructions625worker1 # [ 0.004858] CPU features: detected: ARMv8.4 Translation Table Level626worker2 # [ 0.004988] CPU features: detected: RCpc load-acquire (LDAPR)627client # [ 0.005049] CPU features: detected: Privileged Access Never628worker2 # [ 0.004991] CPU features: detected: LSE atomic instructions629lb # [ 0.161297] iommu: Default domain type: Translated630client # [ 0.005052] CPU features: detected: PMUv3631worker1 # [ 0.004862] CPU features: detected: Instruction cache invalidation not required for I/D coherence632worker2 # [ 0.004993] CPU features: detected: Privileged Access Never633client # [ 0.005055] CPU features: detected: RAS Extension Support634lb # [ 0.161307] iommu: DMA domain TLB invalidation policy: strict mode635worker2 # [ 0.004996] CPU features: detected: PMUv3636worker1 # [ 0.004865] CPU features: detected: Data cache clean to the PoU not required for I/D coherence637lb # [ 0.161676] SCSI subsystem initialized638client # [ 0.005057] CPU features: detected: RASv1p1 Extension Support639worker2 # [ 0.004999] CPU features: detected: RAS Extension Support640worker1 # [ 0.004869] CPU features: detected: Common not Private translations641client # [ 0.005060] CPU features: detected: Random Number Generator642worker2 # [ 0.005001] CPU features: detected: RASv1p1 Extension Support643lb # [ 0.163575] usbcore: registered new interface driver usbfs644worker1 # [ 0.004872] CPU features: detected: CRC32 instructions645client # [ 0.005063] CPU features: detected: Speculation barrier (SB)646worker2 # [ 0.005004] CPU features: detected: Random Number Generator647lb # [ 0.163606] usbcore: registered new interface driver hub648client # [ 0.005065] CPU features: detected: Stage-2 Force Write-Back649worker1 # [ 0.004875] CPU features: detected: Data cache clean to Point of Deep Persistence650worker2 # [ 0.005007] CPU features: detected: Speculation barrier (SB)651lb # [ 0.163620] usbcore: registered new device driver usb652client # [ 0.005068] CPU features: detected: TLB range maintenance instructions653worker1 # [ 0.004878] CPU features: detected: Data cache clean to Point of Persistence654lb # [ 0.163895] pps_core: LinuxPPS API ver. 1 registered655worker2 # [ 0.005009] CPU features: detected: Stage-2 Force Write-Back656client # [ 0.005073] CPU features: detected: Speculative Store Bypassing Safe (SSBS)657worker1 # [ 0.004882] CPU features: detected: Data independent timing control (DIT)658lb # [ 0.163902] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>659worker1 # [ 0.004885] CPU features: detected: E0PD660lb # [ 0.163918] PTP clock support registered661lb # [ 0.163967] EDAC MC: Ver: 3.0.0662worker1 # [ 0.004887] CPU features: detected: Enhanced Counter Virtualization663lb # [ 0.168428] scmi_core: SCMI protocol bus registered664worker1 # [ 0.004891] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)665worker1 # [ 0.004904] CPU features: detected: Enhanced Virtualization Traps666lb # [ 0.169408] FPGA manager framework667worker1 # [ 0.004907] CPU features: detected: Fine Grained Traps668lb # [ 0.170342] vgaarb: loaded669worker1 # [ 0.004911] CPU features: detected: Generic authentication (architected QARMA5 algorithm)670lb # [ 0.170956] clocksource: Switched to clocksource arch_sys_counter671worker1 # [ 0.004915] CPU features: detected: RCpc load-acquire (LDAPR)672worker1 # [ 0.004918] CPU features: detected: LSE atomic instructions673worker1 # [ 0.004921] CPU features: detected: Privileged Access Never674worker1 # [ 0.004924] CPU features: detected: PMUv3675lb # [ 0.173881] VFS: Disk quotas dquot_6.6.0676worker1 # [ 0.004927] CPU features: detected: RAS Extension Support677lb # [ 0.173915] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)678worker2 # [ 0.005012] CPU features: detected: TLB range maintenance instructions679worker1 # [ 0.004929] CPU features: detected: RASv1p1 Extension Support680client # [ 0.005110] alternatives: applying system-wide alternatives681worker2 # [ 0.005017] CPU features: detected: Speculative Store Bypassing Safe (SSBS)682worker1 # [ 0.004932] CPU features: detected: Random Number Generator683client # [ 0.007995] CPU features: detected: BBM Level 2 without TLB conflict abort684worker2 # [ 0.005055] alternatives: applying system-wide alternatives685worker1 # [ 0.004935] CPU features: detected: Speculation barrier (SB)686worker1 # [ 0.004937] CPU features: detected: Stage-2 Force Write-Back687worker2 # [ 0.007944] CPU features: detected: BBM Level 2 without TLB conflict abort688lb # [ 0.177530] netfs: FS-Cache loaded689client # [ 0.008153] Memory: 1405668K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)690worker1 # [ 0.004940] CPU features: detected: TLB range maintenance instructions691lb # [ 0.177648] pnp: PnP ACPI: disabled692client # [ 0.008524] devtmpfs: initialized693worker1 # [ 0.004944] CPU features: detected: Speculative Store Bypassing Safe (SSBS)694worker2 # [ 0.008178] Memory: 1405712K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)695client # [ 0.010310] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)696worker2 # [ 0.008539] devtmpfs: initialized697worker1 # [ 0.004982] alternatives: applying system-wide alternatives698client # [ 0.010331] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).699worker2 # [ 0.010252] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)700worker1 # [ 0.007819] CPU features: detected: BBM Level 2 without TLB conflict abort701client # [ 0.010506] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL702worker2 # [ 0.010273] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).703client # [ 0.010510] 0 pages in range for non-PLT usage704worker2 # [ 0.010449] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL705client # [ 0.010511] 508272 pages in range for PLT usage706worker1 # [ 0.007973] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)707worker1 # [ 0.008368] devtmpfs: initialized708worker2 # [ 0.010453] 0 pages in range for non-PLT usage709client # [ 0.010621] pinctrl core: initialized pinctrl subsystem710worker2 # [ 0.010455] 508272 pages in range for PLT usage711client # [ 0.011355] DMI not present or invalid.712worker1 # [ 0.010220] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)713lb # [ 0.181317] NET: Registered PF_INET protocol family714worker2 # [ 0.010560] pinctrl core: initialized pinctrl subsystem715client # [ 0.014540] NET: Registered PF_NETLINK/PF_ROUTE protocol family716worker2 # [ 0.011355] DMI not present or invalid.717lb # [ 0.181573] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)718worker1 # [ 0.010242] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).719client # [ 0.016722] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations720worker2 # [ 0.014508] NET: Registered PF_NETLINK/PF_ROUTE protocol family721worker1 # [ 0.010409] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL722client # [ 0.016897] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations723worker1 # [ 0.010413] 0 pages in range for non-PLT usage724worker2 # [ 0.016676] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations725worker1 # [ 0.010414] 508272 pages in range for PLT usage726client # [ 0.017092] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations727worker2 # [ 0.016868] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations728worker1 # [ 0.010524] pinctrl core: initialized pinctrl subsystem729client # [ 0.017113] audit: initializing netlink subsys (disabled)730worker1 # [ 0.011311] DMI not present or invalid.731worker2 # [ 0.017064] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations732client # [ 0.017673] thermal_sys: Registered thermal governor 'fair_share'733worker1 # [ 0.014415] NET: Registered PF_NETLINK/PF_ROUTE protocol family734worker2 # [ 0.017085] audit: initializing netlink subsys (disabled)735client # [ 0.017675] thermal_sys: Registered thermal governor 'bang_bang'736worker1 # [ 0.016550] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations737worker2 # [ 0.017651] thermal_sys: Registered thermal governor 'fair_share'738client # [ 0.017679] thermal_sys: Registered thermal governor 'step_wise'739worker2 # [ 0.017652] thermal_sys: Registered thermal governor 'bang_bang'740worker1 # [ 0.016718] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations741client # [ 0.017682] thermal_sys: Registered thermal governor 'user_space'742worker2 # [ 0.017656] thermal_sys: Registered thermal governor 'step_wise'743worker1 # [ 0.016917] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations744client # [ 0.017687] thermal_sys: Registered thermal governor 'power_allocator'745worker2 # [ 0.017659] thermal_sys: Registered thermal governor 'user_space'746worker1 # [ 0.016936] audit: initializing netlink subsys (disabled)747client # [ 0.017711] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1748worker2 # [ 0.017663] thermal_sys: Registered thermal governor 'power_allocator'749worker1 # [ 0.017451] thermal_sys: Registered thermal governor 'fair_share'750client # [ 0.017719] cpuidle: using governor ladder751client # [ 0.017725] cpuidle: using governor menu752worker2 # [ 0.017687] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1753worker1 # [ 0.017453] thermal_sys: Registered thermal governor 'bang_bang'754worker2 # [ 0.017695] cpuidle: using governor ladder755client # [ 0.017910] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.756worker1 # [ 0.017457] thermal_sys: Registered thermal governor 'step_wise'757worker2 # [ 0.017701] cpuidle: using governor menu758client # [ 0.017925] ASID allocator initialised with 65536 entries759worker1 # [ 0.017460] thermal_sys: Registered thermal governor 'user_space'760worker2 # [ 0.017882] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.761client # [ 0.019117] Serial: AMBA PL011 UART driver762worker1 # [ 0.017465] thermal_sys: Registered thermal governor 'power_allocator'763worker2 # [ 0.017895] ASID allocator initialised with 65536 entries764client # [ 0.024307] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1765worker2 # [ 0.019120] Serial: AMBA PL011 UART driver766worker1 # [ 0.017487] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1767client # [ 0.024420] printk: console [ttyAMA0] enabled768worker1 # [ 0.017496] cpuidle: using governor ladder769worker2 # [ 0.024303] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1770worker1 # [ 0.017502] cpuidle: using governor menu771worker2 # [ 0.024454] printk: console [ttyAMA0] enabled772client # [ 0.143867] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages773worker1 # [ 0.017682] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.774client # [ 0.143884] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page775worker1 # [ 0.017695] ASID allocator initialised with 65536 entries776worker2 # [ 0.144376] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages777worker1 # [ 0.018867] Serial: AMBA PL011 UART driver778client # [ 0.143890] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages779worker2 # [ 0.144392] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page780client # [ 0.143894] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page781worker1 # [ 0.023999] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1782worker2 # [ 0.144397] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages783worker1 # [ 0.024136] printk: console [ttyAMA0] enabled784client # [ 0.143898] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages785worker2 # [ 0.144402] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page786client # [ 0.143903] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page787worker2 # [ 0.144407] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages788client # [ 0.143907] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages789worker2 # [ 0.144411] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page790client # [ 0.143911] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page791worker2 # [ 0.144415] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages792worker2 # [ 0.144420] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page793client # [ 0.151300] fbcon: Taking over console794client # [ 0.151313] ACPI: Interpreter disabled.795worker2 # [ 0.151744] fbcon: Taking over console796worker2 # [ 0.151756] ACPI: Interpreter disabled.797worker1 # [ 0.142293] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages798worker1 # [ 0.142308] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page799worker1 # [ 0.142314] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages800worker1 # [ 0.142318] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page801worker1 # [ 0.142323] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages802client # [ 0.158371] iommu: Default domain type: Translated803worker1 # [ 0.142327] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page804client # [ 0.158380] iommu: DMA domain TLB invalidation policy: strict mode805worker1 # [ 0.142332] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages806client # [ 0.160057] SCSI subsystem initialized807worker1 # [ 0.142336] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page808worker2 # [ 0.159711] iommu: Default domain type: Translated809client # [ 0.160736] usbcore: registered new interface driver usbfs810worker2 # [ 0.159719] iommu: DMA domain TLB invalidation policy: strict mode811client # [ 0.160768] usbcore: registered new interface driver hub812worker2 # [ 0.160075] SCSI subsystem initialized813worker1 # [ 0.149759] fbcon: Taking over console814client # [ 0.160784] usbcore: registered new device driver usb815worker1 # [ 0.149773] ACPI: Interpreter disabled.816client # [ 0.161054] pps_core: LinuxPPS API ver. 1 registered817worker2 # [ 0.162110] usbcore: registered new interface driver usbfs818worker2 # [ 0.162141] usbcore: registered new interface driver hub819client # [ 0.161060] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>820worker2 # [ 0.162156] usbcore: registered new device driver usb821client # [ 0.161071] PTP clock support registered822client # [ 0.161128] EDAC MC: Ver: 3.0.0823worker2 # [ 0.162440] pps_core: LinuxPPS API ver. 1 registered824client # [ 0.165785] scmi_core: SCMI protocol bus registered825worker2 # [ 0.162446] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>826lb # [ 0.209945] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)827worker2 # [ 0.162455] PTP clock support registered828client # [ 0.166758] FPGA manager framework829worker2 # [ 0.162513] EDAC MC: Ver: 3.0.0830client # [ 0.167644] vgaarb: loaded831lb # [ 0.209983] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)832worker2 # [ 0.167195] scmi_core: SCMI protocol bus registered833client # [ 0.168255] clocksource: Switched to clocksource arch_sys_counter834lb # [ 0.210008] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)835worker2 # [ 0.168150] FPGA manager framework836lb # [ 0.210084] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)837worker2 # [ 0.169066] vgaarb: loaded838lb # [ 0.210230] TCP: Hash tables configured (established 16384 bind 16384)839worker2 # [ 0.169689] clocksource: Switched to clocksource arch_sys_counter840worker1 # [ 0.158726] iommu: Default domain type: Translated841lb # [ 0.210302] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)842worker1 # [ 0.158736] iommu: DMA domain TLB invalidation policy: strict mode843lb # [ 0.210335] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)844worker1 # [ 0.159072] SCSI subsystem initialized845worker2 # [ 0.171745] VFS: Disk quotas dquot_6.6.0846lb # [ 0.210371] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)847client # [ 0.173138] VFS: Disk quotas dquot_6.6.0848worker2 # [ 0.171779] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)849worker1 # [ 0.161085] usbcore: registered new interface driver usbfs850lb # [ 0.210469] NET: Registered PF_UNIX/PF_LOCAL protocol family851client # [ 0.173179] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)852worker1 # [ 0.161117] usbcore: registered new interface driver hub853lb # [ 0.210487] NET: Registered PF_XDP protocol family854lb # [ 0.210503] PCI: CLS 0 bytes, default 64855worker1 # [ 0.161133] usbcore: registered new device driver usb856client # [ 0.174768] netfs: FS-Cache loaded857client # [ 0.174882] pnp: PnP ACPI: disabled858lb # [ 0.210720] Trying to unpack rootfs image as initramfs...859worker1 # [ 0.161401] pps_core: LinuxPPS API ver. 1 registered860worker2 # [ 0.175358] netfs: FS-Cache loaded861worker1 # [ 0.161407] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>862worker2 # [ 0.175469] pnp: PnP ACPI: disabled863worker1 # [ 0.161417] PTP clock support registered864worker1 # [ 0.161467] EDAC MC: Ver: 3.0.0865worker1 # [ 0.166051] scmi_core: SCMI protocol bus registered866worker1 # [ 0.167038] FPGA manager framework867worker1 # [ 0.167926] vgaarb: loaded868client # [ 0.178761] NET: Registered PF_INET protocol family869worker1 # [ 0.168538] clocksource: Switched to clocksource arch_sys_counter870client # [ 0.179017] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)871lb # [ 0.228515] kvm [1]: HYP mode not available872worker2 # [ 0.179258] NET: Registered PF_INET protocol family873worker2 # [ 0.179508] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)874worker1 # [ 0.174545] VFS: Disk quotas dquot_6.6.0875worker1 # [ 0.174585] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)876worker1 # [ 0.176213] netfs: FS-Cache loaded877worker1 # [ 0.176330] pnp: PnP ACPI: disabled878worker1 # [ 0.180199] NET: Registered PF_INET protocol family879worker1 # [ 0.180448] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)880client # [ 0.208042] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881client # [ 0.208082] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882client # [ 0.208107] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)883client # [ 0.208176] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)884client # [ 0.212161] TCP: Hash tables configured (established 16384 bind 16384)885client # [ 0.212244] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)886worker2 # [ 0.208265] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)887worker2 # [ 0.208306] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)888client # [ 0.213940] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)889worker2 # [ 0.208334] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)890client # [ 0.213981] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)891worker2 # [ 0.208409] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)892client # [ 0.214079] NET: Registered PF_UNIX/PF_LOCAL protocol family893worker2 # [ 0.208552] TCP: Hash tables configured (established 16384 bind 16384)894client # [ 0.214099] NET: Registered PF_XDP protocol family895client # [ 0.214118] PCI: CLS 0 bytes, default 64896worker2 # [ 0.208634] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)897client # [ 0.214343] Trying to unpack rootfs image as initramfs...898worker2 # [ 0.208667] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)899worker2 # [ 0.208702] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker2 # [ 0.208791] NET: Registered PF_UNIX/PF_LOCAL protocol family901worker2 # [ 0.208857] NET: Registered PF_XDP protocol family902worker2 # [ 0.208874] PCI: CLS 0 bytes, default 64903worker2 # [ 0.209089] Trying to unpack rootfs image as initramfs...904worker1 # [ 0.209653] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)905worker1 # [ 0.209700] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)906worker1 # [ 0.209727] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)907worker1 # [ 0.209800] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)908client # [ 0.224439] kvm [1]: HYP mode not available909worker1 # [ 0.209943] TCP: Hash tables configured (established 16384 bind 16384)910worker1 # [ 0.210023] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)911worker1 # [ 0.210058] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)912worker2 # [ 0.227521] kvm [1]: HYP mode not available913worker1 # [ 0.210093] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914worker1 # [ 0.210182] NET: Registered PF_UNIX/PF_LOCAL protocol family915worker1 # [ 0.210224] NET: Registered PF_XDP protocol family916worker1 # [ 0.210241] PCI: CLS 0 bytes, default 64917worker1 # [ 0.210491] Trying to unpack rootfs image as initramfs...918worker1 # [ 0.226109] kvm [1]: HYP mode not available919lb # [ 0.319452] Initialise system trusted keyrings920lb # [ 0.320197] workingset: timestamp_bits=42 max_order=19 bucket_order=0921lb # [ 0.321440] squashfs: version 4.0 (2009/01/31) Phillip Lougher922lb # [ 0.322215] 9p: Installing v9fs 9p2000 file system support923lb # [ 0.342826] Key type asymmetric registered924lb # [ 0.342849] Asymmetric key parser 'x509' registered925lb # [ 0.342911] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)926lb # [ 0.351024] io scheduler mq-deadline registered927lb # [ 0.351044] io scheduler kyber registered928client # [ 0.316826] Initialise system trusted keyrings929client # [ 0.317569] workingset: timestamp_bits=42 max_order=19 bucket_order=0930lb # [ 0.363104] pl061_gpio 9030000.pl061: PL061 GPIO chip registered931client # [ 0.318828] squashfs: version 4.0 (2009/01/31) Phillip Lougher932lb # [ 0.363719] ledtrig-cpu: registered to indicate activity on CPUs933lb # [ 0.364105] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:934client # [ 0.319598] 9p: Installing v9fs 9p2000 file system support935lb # [ 0.364123] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000936lb # [ 0.364135] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000937worker2 # [ 0.322238] Initialise system trusted keyrings938lb # [ 0.364144] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000939worker2 # [ 0.322993] workingset: timestamp_bits=42 max_order=19 bucket_order=0940lb # [ 0.364172] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits941worker2 # [ 0.324272] squashfs: version 4.0 (2009/01/31) Phillip Lougher942lb # [ 0.364197] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]943lb # [ 0.364270] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00944worker2 # [ 0.325064] 9p: Installing v9fs 9p2000 file system support945lb # [ 0.364280] pci_bus 0000:00: root bus resource [bus 00-ff]946lb # [ 0.364287] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]947lb # [ 0.364292] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]948worker1 # [ 0.317053] Initialise system trusted keyrings949lb # [ 0.364297] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]950worker1 # [ 0.317800] workingset: timestamp_bits=42 max_order=19 bucket_order=0951lb # [ 0.364377] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint952worker1 # [ 0.319051] squashfs: version 4.0 (2009/01/31) Phillip Lougher953lb # [ 0.364815] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint954lb # [ 0.364998] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]955worker1 # [ 0.319833] 9p: Installing v9fs 9p2000 file system support956lb # [ 0.365015] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]957lb # [ 0.365044] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]958lb # [ 0.365061] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]959lb # [ 0.365520] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint960lb # [ 0.365700] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]961lb # [ 0.365716] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]962lb # [ 0.365745] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]963lb # [ 0.366185] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint964lb # [ 0.366364] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]965lb # [ 0.366379] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]966lb # [ 0.366409] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]967lb # [ 0.366850] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint968lb # [ 0.367053] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]969lb # [ 0.367069] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]970lb # [ 0.367099] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]971lb # [ 0.367118] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]972lb # [ 0.367599] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint973lb # [ 0.367782] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]974lb # [ 0.367812] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]975lb # [ 0.368263] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint976lb # [ 0.368447] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]977client # [ 0.348355] Key type asymmetric registered978client # [ 0.348379] Asymmetric key parser 'x509' registered979lb # [ 0.368476] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]980client # [ 0.348437] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)981lb # [ 0.368859] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint982client # [ 0.350601] io scheduler mq-deadline registered983lb # [ 0.369037] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]984client # [ 0.350611] io scheduler kyber registered985lb # [ 0.369279] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint986worker1 # [ 0.340508] Key type asymmetric registered987lb # [ 0.369469] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]988worker1 # [ 0.340528] Asymmetric key parser 'x509' registered989lb # [ 0.369499] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]990lb # [ 0.369949] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint991worker2 # [ 0.353767] Key type asymmetric registered992worker2 # [ 0.353789] Asymmetric key parser 'x509' registered993lb # [ 0.370138] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]994worker2 # [ 0.353859] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)995lb # [ 0.370167] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996worker2 # [ 0.355985] io scheduler mq-deadline registered997lb # [ 0.370620] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint998worker2 # [ 0.355996] io scheduler kyber registered999lb # [ 0.370809] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1000lb # [ 0.370840] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1001worker1 # [ 0.348653] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1002client # [ 0.360393] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1003worker1 # [ 0.349695] io scheduler mq-deadline registered1004worker1 # [ 0.349707] io scheduler kyber registered1005client # [ 0.361795] ledtrig-cpu: registered to indicate activity on CPUs1006client # [ 0.362179] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1007client # [ 0.362196] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001008client # [ 0.362209] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001009client # [ 0.362218] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001010client # [ 0.362238] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1011client # [ 0.362274] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1012lb # [ 0.419407] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1013client # [ 0.362349] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001014lb # [ 0.419696] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1015client # [ 0.362358] pci_bus 0000:00: root bus resource [bus 00-ff]1016lb # [ 0.419715] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1017client # [ 0.362364] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1018lb # [ 0.419746] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019client # [ 0.362369] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1020lb # [ 0.420220] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1021worker2 # [ 0.365816] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1022client # [ 0.362374] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1023lb # [ 0.420405] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1024worker2 # [ 0.366458] ledtrig-cpu: registered to indicate activity on CPUs1025lb # [ 0.420422] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1026client # [ 0.362457] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1027worker2 # [ 0.366847] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1028lb # [ 0.420452] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1029client # [ 0.362902] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1030worker2 # [ 0.366863] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001031lb # [ 0.421025] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1032client # [ 0.363083] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1033client # [ 0.363099] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1034lb # [ 0.421038] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1035worker2 # [ 0.366875] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001036client # [ 0.363129] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1037lb # [ 0.421043] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1038worker2 # [ 0.366883] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001039client # [ 0.363144] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1040lb # [ 0.421089] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1041worker2 # [ 0.366904] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1042client # [ 0.363589] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1043worker1 # [ 0.360672] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1044lb # [ 0.421137] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1045client # [ 0.363766] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1046worker1 # [ 0.361266] ledtrig-cpu: registered to indicate activity on CPUs1047worker2 # [ 0.366928] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1048client # [ 0.363782] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1049lb # [ 0.421186] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1050worker1 # [ 0.361622] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1051worker2 # [ 0.367013] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001052client # [ 0.363810] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1053lb # [ 0.421235] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1054worker2 # [ 0.367022] pci_bus 0000:00: root bus resource [bus 00-ff]1055worker1 # [ 0.361639] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001056worker2 # [ 0.367028] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1057lb # [ 0.421283] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1058worker1 # [ 0.361651] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001059worker2 # [ 0.367033] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1060client # [ 0.383168] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1061lb # [ 0.421331] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1062worker1 # [ 0.361659] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001063worker2 # [ 0.367039] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1064client # [ 0.383354] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1065lb # [ 0.421389] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1066client # [ 0.383370] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1067worker1 # [ 0.361679] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1068worker2 # [ 0.367114] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1069lb # [ 0.421437] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1070client # [ 0.383399] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1071worker2 # [ 0.367566] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1072worker1 # [ 0.361702] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1073client # [ 0.383860] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1074lb # [ 0.421484] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1075worker2 # [ 0.367753] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1076worker1 # [ 0.361789] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001077client # [ 0.384043] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1078worker2 # [ 0.367770] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1079lb # [ 0.421553] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1080worker1 # [ 0.361799] pci_bus 0000:00: root bus resource [bus 00-ff]1081client # [ 0.384058] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1082lb # [ 0.421600] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1083worker2 # [ 0.367801] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1084worker1 # [ 0.361805] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1085client # [ 0.384087] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086worker2 # [ 0.367818] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1087lb # [ 0.421622] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1088worker1 # [ 0.361810] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1089client # [ 0.384105] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1090lb # [ 0.421644] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1091worker1 # [ 0.361815] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1092worker2 # [ 0.368304] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1093worker2 # [ 0.368489] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1094lb # [ 0.421665] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1095worker1 # [ 0.361880] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1096worker2 # [ 0.368506] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1097lb # [ 0.421688] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1098worker1 # [ 0.362318] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1099worker2 # [ 0.368536] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100lb # [ 0.421709] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1101worker1 # [ 0.362507] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1102lb # [ 0.421732] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1103worker2 # [ 0.368990] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1104worker1 # [ 0.362525] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1105worker2 # [ 0.369174] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1106lb # [ 0.421757] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1107worker1 # [ 0.362557] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108worker2 # [ 0.369191] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1109lb # [ 0.421779] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1110worker1 # [ 0.362574] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1111worker2 # [ 0.369222] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1112lb # [ 0.421802] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1113worker1 # [ 0.363034] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1114lb # [ 0.421824] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1115worker2 # [ 0.369676] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1116worker1 # [ 0.363219] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1117worker2 # [ 0.369885] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1118lb # [ 0.421847] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1119worker1 # [ 0.363236] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1120worker2 # [ 0.369902] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1121lb # [ 0.421868] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1122worker1 # [ 0.363267] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123worker2 # [ 0.369933] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124lb # [ 0.421890] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1125worker1 # [ 0.363716] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1126worker2 # [ 0.369952] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1127lb # [ 0.421912] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1128worker1 # [ 0.363900] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1129lb # [ 0.421933] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1130worker2 # [ 0.370436] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1131worker1 # [ 0.363917] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1132lb # [ 0.421954] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1133worker2 # [ 0.370626] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1134worker1 # [ 0.363948] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135lb # [ 0.421976] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1136worker2 # [ 0.370657] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137lb # [ 0.422003] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1138worker1 # [ 0.364410] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1139worker1 # [ 0.364617] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1140worker2 # [ 0.371116] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1141lb # [ 0.422013] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1142worker2 # [ 0.371305] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1143worker1 # [ 0.364634] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1144lb # [ 0.422017] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1145client # [ 0.396654] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1146lb # [ 0.422831] pci 0000:00:07.0: enabling device (0000 -> 0002)1147worker1 # [ 0.364665] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1148worker2 # [ 0.371336] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149client # [ 0.396860] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1150worker1 # [ 0.364685] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1151worker2 # [ 0.371741] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1152client # [ 0.396890] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1153worker1 # [ 0.365146] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1154worker2 # [ 0.371924] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1155client # [ 0.397356] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1156worker1 # [ 0.365335] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1157lb # [ 0.458425] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 34767 usecs1158worker2 # [ 0.372207] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1159client # [ 0.397543] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1160worker1 # [ 0.365367] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1161worker2 # [ 0.372395] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1162client # [ 0.397573] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1163worker2 # [ 0.372426] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164worker1 # [ 0.365840] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1165client # [ 0.397961] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1166worker1 # [ 0.366028] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1167worker2 # [ 0.372891] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1168client # [ 0.398143] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1169worker1 # [ 0.366060] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1170worker2 # [ 0.373079] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1171client # [ 0.398434] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1172worker2 # [ 0.373109] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1173worker1 # [ 0.366451] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1174client # [ 0.398621] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1175worker1 # [ 0.366636] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1176worker2 # [ 0.373565] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1177client # [ 0.398651] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1178worker1 # [ 0.366914] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1179client # [ 0.399102] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1180worker1 # [ 0.367102] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1181client # [ 0.399288] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1182worker1 # [ 0.367133] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1183client # [ 0.399318] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1184worker1 # [ 0.367582] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1185client # [ 0.399780] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1186worker1 # [ 0.367770] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1187client # [ 0.399965] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1188worker1 # [ 0.367802] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1189client # [ 0.399995] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1190worker1 # [ 0.368262] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1191client # [ 0.400467] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1192worker1 # [ 0.368449] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1193client # [ 0.400727] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1194worker1 # [ 0.368480] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1195client # [ 0.400745] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1196client # [ 0.400774] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1197client # [ 0.401232] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1198worker2 # [ 0.417808] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1199client # [ 0.401423] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1200worker2 # [ 0.417851] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1201client # [ 0.401439] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1202worker2 # [ 0.418350] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1203client # [ 0.401469] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1204worker2 # [ 0.418634] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1205client # [ 0.402031] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1206worker2 # [ 0.418653] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1207client # [ 0.402042] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1208worker2 # [ 0.418684] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1209client # [ 0.402048] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1210worker2 # [ 0.419150] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1211worker2 # [ 0.419335] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1212client # [ 0.402093] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1213worker2 # [ 0.419352] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1214client # [ 0.402139] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1215worker2 # [ 0.419383] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1216client # [ 0.402185] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1217worker2 # [ 0.419974] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1218client # [ 0.402231] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1219worker2 # [ 0.419986] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1220client # [ 0.402278] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1221worker2 # [ 0.419991] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1222client # [ 0.402323] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1223worker1 # [ 0.417070] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1224worker2 # [ 0.420045] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1225worker1 # [ 0.417378] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1226client # [ 0.402369] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1227worker2 # [ 0.420095] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1228worker1 # [ 0.417397] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1229lb # [ 0.485173] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1230client # [ 0.402416] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1231worker2 # [ 0.420143] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1232worker1 # [ 0.417428] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1233client # [ 0.402463] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1234worker2 # [ 0.420193] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1235worker1 # [ 0.417895] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1236worker1 # [ 0.418080] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1237client # [ 0.402531] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1238worker2 # [ 0.420242] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1239worker1 # [ 0.418097] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1240client # [ 0.402577] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1241worker2 # [ 0.420290] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1242worker1 # [ 0.418128] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1243client # [ 0.402598] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1244worker2 # [ 0.420338] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1245worker1 # [ 0.418702] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1246client # [ 0.402619] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1247worker2 # [ 0.420386] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1248worker1 # [ 0.418714] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1249client # [ 0.402640] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1250worker2 # [ 0.420434] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1251client # [ 0.402661] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1252worker1 # [ 0.418720] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1253client # [ 0.402682] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1254worker2 # [ 0.420508] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1255worker1 # [ 0.418766] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1256client # [ 0.402703] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1257worker2 # [ 0.420554] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1258worker1 # [ 0.418815] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1259client # [ 0.402728] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1260worker2 # [ 0.420577] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1261lb # [ 0.488224] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1262worker1 # [ 0.418865] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1263client # [ 0.402750] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1264worker2 # [ 0.420600] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1265lb # [ 0.490166] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1266worker1 # [ 0.418915] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1267worker2 # [ 0.420622] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1268client # [ 0.402771] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1269lb # [ 0.492158] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1270worker2 # [ 0.420644] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1271worker1 # [ 0.418965] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1272lb # [ 0.494208] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1273client # [ 0.402793] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1274worker2 # [ 0.420666] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1275client # [ 0.402816] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1276worker1 # [ 0.419015] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1277client # [ 0.402837] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1278worker2 # [ 0.420688] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1279worker1 # [ 0.419065] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1280client # [ 0.402858] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1281worker2 # [ 0.420714] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1282worker1 # [ 0.419114] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1283client # [ 0.402879] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1284worker2 # [ 0.420736] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1285client # [ 0.402900] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1286worker1 # [ 0.419163] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1287worker2 # [ 0.420758] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1288client # [ 0.402921] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1289worker2 # [ 0.420781] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1290worker1 # [ 0.419232] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1291client # [ 0.402942] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1292worker2 # [ 0.420804] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1293worker1 # [ 0.419279] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1294client # [ 0.402967] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1295worker2 # [ 0.420825] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1296client # [ 0.402976] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1297worker1 # [ 0.419302] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1298worker2 # [ 0.420847] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1299client # [ 0.402981] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1300worker1 # [ 0.419325] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1301worker2 # [ 0.420869] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1302client # [ 0.403795] pci 0000:00:07.0: enabling device (0000 -> 0002)1303worker1 # [ 0.419348] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1304worker2 # [ 0.420890] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1305worker1 # [ 0.419371] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1306worker2 # [ 0.420912] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1307worker1 # [ 0.419394] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1308worker2 # [ 0.420934] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1309worker2 # [ 0.420961] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1310worker1 # [ 0.419417] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1311worker2 # [ 0.420971] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1312worker1 # [ 0.419444] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1313worker2 # [ 0.420976] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1314worker1 # [ 0.419467] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1315worker1 # [ 0.419491] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1316worker1 # [ 0.419514] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1317worker1 # [ 0.419538] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1318worker1 # [ 0.419561] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1319worker1 # [ 0.419584] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1320client # [ 0.460400] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55283 usecs1321lb # [ 0.512587] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1322worker1 # [ 0.419606] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1323lb # [ 0.514353] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1324worker1 # [ 0.419629] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1325worker1 # [ 0.419651] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1326worker1 # [ 0.419674] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1327worker1 # [ 0.419701] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1328worker1 # [ 0.419711] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1329lb # [ 0.517687] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1330worker1 # [ 0.419716] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1331worker1 # [ 0.420517] pci 0000:00:07.0: enabling device (0000 -> 0002)1332worker2 # [ 0.465846] pci 0000:00:07.0: enabling device (0000 -> 0002)1333worker1 # [ 0.456926] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 35562 usecs1334lb # [ 0.524293] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1335lb # [ 0.526694] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1336client # [ 0.481151] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1337client # [ 0.483369] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1338lb # [ 0.536901] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1339worker2 # [ 0.486943] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1340worker2 # [ 0.488954] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1341worker1 # [ 0.482581] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1342client # [ 0.494421] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1343lb # [ 0.549679] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1344worker2 # [ 0.499980] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1345lb # [ 0.551284] msm_serial: driver initialized1346lb # [ 0.551431] SuperH (H)SCI(F) driver initialized1347lb # [ 0.551486] STM32 USART driver initialized1348client # [ 0.500492] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1349worker1 # [ 0.488765] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1350client # [ 0.502489] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1351worker1 # [ 0.490788] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1352worker2 # [ 0.505870] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1353worker2 # [ 0.507945] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1354client # [ 0.512416] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1355client # [ 0.514161] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1356client # [ 0.515845] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1357worker1 # [ 0.500716] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1358worker1 # [ 0.502731] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1359worker1 # [ 0.505157] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1360worker2 # [ 0.517880] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1361worker2 # [ 0.519600] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1362worker1 # [ 0.506826] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1363worker2 # [ 0.521379] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1364client # [ 0.526350] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1365worker1 # [ 0.516614] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1366lb # [ 0.584205] loop: module loaded1367lb # [ 0.584375] virtio_blk virtio2: 1/0/0 default/read/poll queues1368worker1 # [ 0.518436] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1369worker2 # [ 0.531612] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1370worker1 # [ 0.520209] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1371lb # [ 0.585203] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1372client # [ 0.532361] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1373client # [ 0.534728] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1374worker2 # [ 0.534532] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1375worker2 # [ 0.536772] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1376lb # [ 0.591543] megasas: 07.734.00.00-rc11377lb # [ 0.592239] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1378worker1 # [ 0.530761] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1379client # [ 0.547487] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1380worker2 # [ 0.549883] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1381client # [ 0.550045] msm_serial: driver initialized1382client # [ 0.550192] SuperH (H)SCI(F) driver initialized1383client # [ 0.550245] STM32 USART driver initialized1384lb # [ 0.602992] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001385lb # [ 0.603017] Intel/Sharp Extended Query Table at 0x00311386lb # [ 0.604587] Using buffer write method1387lb # [ 0.604649] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1388lb # [ 0.607552] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001389worker1 # [ 0.547877] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1390lb # [ 0.607575] Intel/Sharp Extended Query Table at 0x00311391lb # [ 0.608886] Using buffer write method1392lb # [ 0.608911] Concatenating MTD devices:1393lb # [ 0.608915] (0): "0.flash"1394worker2 # [ 0.558966] msm_serial: driver initialized1395lb # [ 0.608927] (1): "0.flash"1396lb # [ 0.608930] into device "0.flash"1397worker2 # [ 0.559117] SuperH (H)SCI(F) driver initialized1398worker2 # [ 0.559172] STM32 USART driver initialized1399worker1 # [ 0.550547] msm_serial: driver initialized1400worker1 # [ 0.550692] SuperH (H)SCI(F) driver initialized1401worker1 # [ 0.550751] STM32 USART driver initialized1402client # [ 0.585243] loop: module loaded1403client # [ 0.585410] virtio_blk virtio2: 1/0/0 default/read/poll queues1404client # [ 0.586142] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1405worker2 # [ 0.588458] loop: module loaded1406worker2 # [ 0.588621] virtio_blk virtio2: 1/0/0 default/read/poll queues1407worker2 # [ 0.589345] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1408worker1 # [ 0.583636] loop: module loaded1409worker1 # [ 0.583809] virtio_blk virtio2: 1/0/0 default/read/poll queues1410worker2 # [ 0.594256] megasas: 07.734.00.00-rc11411worker1 # [ 0.584657] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1412worker2 # [ 0.594947] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1413client # [ 0.596905] megasas: 07.734.00.00-rc11414worker1 # [ 0.589155] megasas: 07.734.00.00-rc11415client # [ 0.597627] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1416worker1 # [ 0.589891] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1417client # [ 0.599749] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001418client # [ 0.599774] Intel/Sharp Extended Query Table at 0x00311419client # [ 0.601277] Using buffer write method1420client # [ 0.601352] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1421client # [ 0.602976] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001422client # [ 0.603028] Intel/Sharp Extended Query Table at 0x00311423worker2 # [ 0.605730] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001424worker2 # [ 0.605754] Intel/Sharp Extended Query Table at 0x00311425worker2 # [ 0.607341] Using buffer write method1426worker2 # [ 0.607418] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1427worker2 # [ 0.609302] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001428worker2 # [ 0.609326] Intel/Sharp Extended Query Table at 0x00311429worker2 # [ 0.610859] Using buffer write method1430worker2 # [ 0.610878] Concatenating MTD devices:1431worker2 # [ 0.610882] (0): "0.flash"1432worker2 # [ 0.610886] (1): "0.flash"1433worker1 # [ 0.601591] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001434worker1 # [ 0.601624] Intel/Sharp Extended Query Table at 0x00311435worker2 # [ 0.610897] into device "0.flash"1436worker1 # [ 0.603144] Using buffer write method1437worker1 # [ 0.603208] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1438client # [ 0.621897] Using buffer write method1439client # [ 0.621932] Concatenating MTD devices:1440client # [ 0.621936] (0): "0.flash"1441client # [ 0.621946] (1): "0.flash"1442client # [ 0.621949] into device "0.flash"1443worker1 # [ 0.618589] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001444worker1 # [ 0.618620] Intel/Sharp Extended Query Table at 0x00311445worker1 # [ 0.620495] Using buffer write method1446worker1 # [ 0.620523] Concatenating MTD devices:1447worker1 # [ 0.620527] (0): "0.flash"1448worker1 # [ 0.623581] (1): "0.flash"1449worker1 # [ 0.623592] into device "0.flash"1450lb # [ 0.856301] Freeing initrd memory: 26948K1451lb # [ 0.862300] tun: Universal TUN/TAP device driver, 1.61452lb # [ 0.865945] thunder_xcv, ver 1.01453lb # [ 0.865988] thunder_bgx, ver 1.01454lb # [ 0.866009] nicpf, ver 1.01455lb # [ 0.866579] e1000: Intel(R) PRO/1000 Network Driver1456lb # [ 0.866586] e1000: Copyright (c) 1999-2006 Intel Corporation.1457lb # [ 0.866609] e1000e: Intel(R) PRO/1000 Network Driver1458lb # [ 0.866617] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1459lb # [ 0.866646] igb: Intel(R) Gigabit Ethernet Network Driver1460lb # [ 0.866652] igb: Copyright (c) 2007-2014 Intel Corporation.1461lb # [ 0.866675] igbvf: Intel(R) Gigabit Virtual Function Network Driver1462lb # [ 0.866681] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1463lb # [ 0.866815] sky2: driver version 1.301464lb # [ 0.875648] ehci-pci 0000:00:07.0: EHCI Host Controller1465lb # [ 0.875680] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11466lb # [ 0.876005] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001467lb # [ 0.878724] usbcore: registered new interface driver usb-storage1468lb # [ 0.878829] usbcore: registered new interface driver usbserial_generic1469lb # [ 0.878843] usbserial: USB Serial support registered for generic1470lb # [ 0.881710] hv_vmbus: registering driver hyperv_keyboard1471lb # [ 0.883237] rtc-pl031 9010000.pl031: registered as rtc01472lb # [ 0.883260] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:53 UTC (1790141573)1473lb # [ 0.883558] i2c_dev: i2c /dev entries driver1474lb # [ 0.888825] sdhci: Secure Digital Host Controller Interface driver1475lb # [ 0.888835] sdhci: Copyright(c) Pierre Ossman1476lb # [ 0.889100] Synopsys Designware Multimedia Card Interface Driver1477lb # [ 0.889474] sdhci-pltfm: SDHCI platform and OF driver helper1478lb # [ 0.893626] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001479lb # [ 0.893992] hub 1-0:1.0: USB hub found1480lb # [ 0.894011] hub 1-0:1.0: 6 ports detected1481lb # [ 0.896087] hid: raw HID events driver (C) Jiri Kosina1482lb # [ 0.896331] usbcore: registered new interface driver usbhid1483lb # [ 0.896337] usbhid: USB HID core driver1484lb # [ 0.899281] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1485lb # [ 0.900820] drop_monitor: Initializing network drop monitor service1486lb # [ 0.900949] NET: Registered PF_INET6 protocol family1487lb # [ 0.903937] Segment Routing with IPv61488lb # [ 0.903968] In-situ OAM (IOAM) with IPv61489lb # [ 0.904014] NET: Registered PF_PACKET protocol family1490lb # [ 0.905687] 9pnet: Installing 9P2000 support1491lb # [ 0.905737] Key type dns_resolver registered1492client # [ 0.855319] Freeing initrd memory: 26972K1493worker2 # [ 0.858292] Freeing initrd memory: 26944K1494worker1 # [ 0.847211] Freeing initrd memory: 26976K1495client # [ 0.861546] tun: Universal TUN/TAP device driver, 1.61496lb # [ 0.912610] registered taskstats version 11497lb # [ 0.912770] Loading compiled-in X.509 certificates1498worker1 # [ 0.853150] tun: Universal TUN/TAP device driver, 1.61499worker2 # [ 0.864417] tun: Universal TUN/TAP device driver, 1.61500client # [ 0.865262] thunder_xcv, ver 1.01501lb # [ 0.921593] Demotion targets for Node 0: null1502client # [ 0.865306] thunder_bgx, ver 1.01503lb # [ 0.921692] Key type .fscrypt registered1504client # [ 0.865329] nicpf, ver 1.01505worker2 # [ 0.868012] thunder_xcv, ver 1.01506lb # [ 0.921699] Key type fscrypt-provisioning registered1507client # [ 0.865897] e1000: Intel(R) PRO/1000 Network Driver1508worker2 # [ 0.868066] thunder_bgx, ver 1.01509worker2 # [ 0.868088] nicpf, ver 1.01510lb # [ 0.921801] ima: No TPM chip found, activating TPM-bypass!1511client # [ 0.865904] e1000: Copyright (c) 1999-2006 Intel Corporation.1512worker2 # [ 0.868625] e1000: Intel(R) PRO/1000 Network Driver1513lb # [ 0.921820] ima: Allocated hash algorithm: sha11514client # [ 0.865928] e1000e: Intel(R) PRO/1000 Network Driver1515lb # [ 0.921840] ima: No architecture policies found1516worker2 # [ 0.868632] e1000: Copyright (c) 1999-2006 Intel Corporation.1517client # [ 0.865935] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1518worker2 # [ 0.868655] e1000e: Intel(R) PRO/1000 Network Driver1519worker1 # [ 0.856832] thunder_xcv, ver 1.01520client # [ 0.865964] igb: Intel(R) Gigabit Ethernet Network Driver1521worker2 # [ 0.868662] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1522worker1 # [ 0.856871] thunder_bgx, ver 1.01523lb # [ 0.925780] input: gpio-keys as /devices/platform/gpio-keys/input/input01524client # [ 0.865969] igb: Copyright (c) 2007-2014 Intel Corporation.1525worker1 # [ 0.856903] nicpf, ver 1.01526worker2 # [ 0.868692] igb: Intel(R) Gigabit Ethernet Network Driver1527worker1 # [ 0.857498] e1000: Intel(R) PRO/1000 Network Driver1528client # [ 0.865996] igbvf: Intel(R) Gigabit Virtual Function Network Driver1529worker2 # [ 0.868697] igb: Copyright (c) 2007-2014 Intel Corporation.1530worker1 # [ 0.857505] e1000: Copyright (c) 1999-2006 Intel Corporation.1531client # [ 0.866002] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1532worker2 # [ 0.868719] igbvf: Intel(R) Gigabit Virtual Function Network Driver1533client # [ 0.866135] sky2: driver version 1.301534worker1 # [ 0.857531] e1000e: Intel(R) PRO/1000 Network Driver1535worker2 # [ 0.868725] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1536client # [ 0.867705] usbcore: registered new interface driver usb-storage1537worker1 # [ 0.857539] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1538worker2 # [ 0.868854] sky2: driver version 1.301539worker1 # [ 0.857570] igb: Intel(R) Gigabit Ethernet Network Driver1540client # [ 0.867754] usbcore: registered new interface driver usbserial_generic1541worker1 # [ 0.857576] igb: Copyright (c) 2007-2014 Intel Corporation.1542client # [ 0.867768] usbserial: USB Serial support registered for generic1543client # [ 0.869016] ehci-pci 0000:00:07.0: EHCI Host Controller1544worker1 # [ 0.857605] igbvf: Intel(R) Gigabit Virtual Function Network Driver1545worker1 # [ 0.857611] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1546client # [ 0.869043] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11547worker2 # [ 0.877394] usbcore: registered new interface driver usb-storage1548worker1 # [ 0.857753] sky2: driver version 1.301549client # [ 0.869316] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001550worker2 # [ 0.877513] usbcore: registered new interface driver usbserial_generic1551worker1 # [ 0.859434] usbcore: registered new interface driver usb-storage1552worker2 # [ 0.877529] usbserial: USB Serial support registered for generic1553worker1 # [ 0.859488] usbcore: registered new interface driver usbserial_generic1554client # [ 0.880808] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001555worker2 # [ 0.878391] ehci-pci 0000:00:07.0: EHCI Host Controller1556client # [ 0.881096] hub 1-0:1.0: USB hub found1557worker1 # [ 0.859503] usbserial: USB Serial support registered for generic1558client # [ 0.881111] hub 1-0:1.0: 6 ports detected1559worker2 # [ 0.878416] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11560worker1 # [ 0.860135] hv_vmbus: registering driver hyperv_keyboard1561worker2 # [ 0.878635] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001562worker1 # [ 0.861213] ehci-pci 0000:00:07.0: EHCI Host Controller1563client # [ 0.883218] hv_vmbus: registering driver hyperv_keyboard1564worker1 # [ 0.861239] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11565worker2 # [ 0.883706] hv_vmbus: registering driver hyperv_keyboard1566worker1 # [ 0.861425] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001567client # [ 0.884815] rtc-pl031 9010000.pl031: registered as rtc01568worker2 # [ 0.885247] rtc-pl031 9010000.pl031: registered as rtc01569worker1 # [ 0.873224] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001570client # [ 0.884843] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:53 UTC (1790141573)1571worker1 # [ 0.873543] hub 1-0:1.0: USB hub found1572client # [ 0.885150] i2c_dev: i2c /dev entries driver1573worker1 # [ 0.873560] hub 1-0:1.0: 6 ports detected1574worker2 # [ 0.885274] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:53 UTC (1790141573)1575worker2 # [ 0.885569] i2c_dev: i2c /dev entries driver1576worker1 # [ 0.876739] rtc-pl031 9010000.pl031: registered as rtc01577worker1 # [ 0.876765] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:53 UTC (1790141573)1578worker1 # [ 0.877145] i2c_dev: i2c /dev entries driver1579client # [ 0.890363] sdhci: Secure Digital Host Controller Interface driver1580client # [ 0.890373] sdhci: Copyright(c) Pierre Ossman1581lb # [ 0.944140] clk: Disabling unused clocks1582client # [ 0.890638] Synopsys Designware Multimedia Card Interface Driver1583lb # [ 0.944163] PM: genpd: Disabling unused power domains1584client # [ 0.891020] sdhci-pltfm: SDHCI platform and OF driver helper1585worker2 # [ 0.890731] sdhci: Secure Digital Host Controller Interface driver1586worker2 # [ 0.890741] sdhci: Copyright(c) Pierre Ossman1587worker2 # [ 0.891007] Synopsys Designware Multimedia Card Interface Driver1588lb # [ 0.948286] Freeing unused kernel memory: 4736K1589client # [ 0.895225] hid: raw HID events driver (C) Jiri Kosina1590worker1 # [ 0.882327] sdhci: Secure Digital Host Controller Interface driver1591worker2 # [ 0.891369] sdhci-pltfm: SDHCI platform and OF driver helper1592lb # [ 0.948490] Run /init as init process1593client # [ 0.895497] usbcore: registered new interface driver usbhid1594worker1 # [ 0.882338] sdhci: Copyright(c) Pierre Ossman1595worker2 # [ 0.892858] hid: raw HID events driver (C) Jiri Kosina1596client # [ 0.895503] usbhid: USB HID core driver1597worker1 # [ 0.882595] Synopsys Designware Multimedia Card Interface Driver1598worker2 # [ 0.893104] usbcore: registered new interface driver usbhid1599worker2 # [ 0.893110] usbhid: USB HID core driver1600worker1 # [ 0.882968] sdhci-pltfm: SDHCI platform and OF driver helper1601worker1 # [ 0.884475] hid: raw HID events driver (C) Jiri Kosina1602worker2 # [ 0.898257] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001603worker2 # [ 0.898569] hub 1-0:1.0: USB hub found1604worker1 # [ 0.887980] usbcore: registered new interface driver usbhid1605worker2 # [ 0.898589] hub 1-0:1.0: 6 ports detected1606client # [ 0.898348] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1607worker1 # [ 0.887991] usbhid: USB HID core driver1608client # [ 0.899852] drop_monitor: Initializing network drop monitor service1609client # [ 0.900000] NET: Registered PF_INET6 protocol family1610worker2 # [ 0.901080] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1611client # [ 0.903058] Segment Routing with IPv61612client # [ 0.903088] In-situ OAM (IOAM) with IPv61613client # [ 0.903138] NET: Registered PF_PACKET protocol family1614worker1 # [ 0.890253] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1615worker2 # [ 0.903663] drop_monitor: Initializing network drop monitor service1616worker1 # [ 0.891735] drop_monitor: Initializing network drop monitor service1617worker2 # [ 0.903829] NET: Registered PF_INET6 protocol family1618worker1 # [ 0.891893] NET: Registered PF_INET6 protocol family1619worker2 # [ 0.905755] Segment Routing with IPv61620worker1 # [ 0.895004] Segment Routing with IPv61621worker2 # [ 0.905771] In-situ OAM (IOAM) with IPv61622worker1 # [ 0.895021] In-situ OAM (IOAM) with IPv61623worker2 # [ 0.905799] NET: Registered PF_PACKET protocol family1624worker1 # [ 0.895048] NET: Registered PF_PACKET protocol family1625client # [ 0.905000] 9pnet: Installing 9P2000 support1626client # [ 0.905048] Key type dns_resolver registered1627worker2 # [ 0.907451] 9pnet: Installing 9P2000 support1628worker2 # [ 0.907501] Key type dns_resolver registered1629client # [ 0.911891] registered taskstats version 11630worker1 # [ 0.896881] 9pnet: Installing 9P2000 support1631lb # [ 0.965142] systemd[1]: Successfully made /usr/ read-only.1632client # [ 0.912034] Loading compiled-in X.509 certificates1633worker1 # [ 0.896933] Key type dns_resolver registered1634worker1 # [ 0.903667] registered taskstats version 11635worker1 # [ 0.903805] Loading compiled-in X.509 certificates1636worker2 # [ 0.914036] registered taskstats version 11637worker2 # [ 0.914172] Loading compiled-in X.509 certificates1638client # [ 0.920636] Demotion targets for Node 0: null1639client # [ 0.920744] Key type .fscrypt registered1640client # [ 0.920750] Key type fscrypt-provisioning registered1641client # [ 0.920841] ima: No TPM chip found, activating TPM-bypass!1642client # [ 0.920860] ima: Allocated hash algorithm: sha11643client # [ 0.920880] ima: No architecture policies found1644worker2 # [ 0.922815] Demotion targets for Node 0: null1645worker2 # [ 0.922916] Key type .fscrypt registered1646worker1 # [ 0.912274] Demotion targets for Node 0: null1647worker1 # [ 0.912366] Key type .fscrypt registered1648worker2 # [ 0.922922] Key type fscrypt-provisioning registered1649client # [ 0.925000] input: gpio-keys as /devices/platform/gpio-keys/input/input01650worker1 # [ 0.912372] Key type fscrypt-provisioning registered1651worker2 # [ 0.923013] ima: No TPM chip found, activating TPM-bypass!1652worker1 # [ 0.912462] ima: No TPM chip found, activating TPM-bypass!1653worker2 # [ 0.923036] ima: Allocated hash algorithm: sha11654worker1 # [ 0.912481] ima: Allocated hash algorithm: sha11655worker2 # [ 0.923056] ima: No architecture policies found1656worker1 # [ 0.912501] ima: No architecture policies found1657worker2 # [ 0.927067] input: gpio-keys as /devices/platform/gpio-keys/input/input01658worker1 # [ 0.916591] input: gpio-keys as /devices/platform/gpio-keys/input/input01659client # [ 0.942870] clk: Disabling unused clocks1660client # [ 0.942893] PM: genpd: Disabling unused power domains1661worker2 # [ 0.945442] clk: Disabling unused clocks1662worker2 # [ 0.945464] PM: genpd: Disabling unused power domains1663client # [ 0.946979] Freeing unused kernel memory: 4736K1664worker1 # [ 0.934040] clk: Disabling unused clocks1665client # [ 0.947163] Run /init as init process1666worker1 # [ 0.934060] PM: genpd: Disabling unused power domains1667worker2 # [ 0.949557] Freeing unused kernel memory: 4736K1668worker1 # [ 0.938159] Freeing unused kernel memory: 4736K1669worker1 # [ 0.938372] Run /init as init process1670worker2 # [ 0.950327] Run /init as init process1671client # [ 0.962498] systemd[1]: Successfully made /usr/ read-only.1672worker2 # [ 0.963834] systemd[1]: Successfully made /usr/ read-only.1673worker1 # [ 0.954692] systemd[1]: Successfully made /usr/ read-only.1674lb # [ 1.139035] usb 1-1: new high-speed USB device number 2 using ehci-pci1675client # [ 1.128338] usb 1-1: new high-speed USB device number 2 using ehci-pci1676worker1 # [ 1.120611] usb 1-1: new high-speed USB device number 2 using ehci-pci1677worker2 # [ 1.145772] usb 1-1: new high-speed USB device number 2 using ehci-pci1678lb # [ 1.291383] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679lb # [ 1.299894] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1680lb # [ 1.312118] systemd[1]: Detected virtualization qemu.1681lb # [ 1.312206] systemd[1]: Detected architecture arm64.1682lb # [ 1.312232] systemd[1]: Running in initrd.1683lb # [ 1.313144] systemd[1]: Initializing machine ID from random generator.1684lb # [ 1.313445] systemd[1]: Hostname set to <lb>.1685client # [ 1.280866] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11686worker1 # [ 1.273133] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11687worker2 # [ 1.298129] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11688client # [ 1.297063] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1689client # [ 1.309163] systemd[1]: Detected virtualization qemu.1690client # [ 1.311150] systemd[1]: Detected architecture arm64.1691worker1 # [ 1.289489] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1692client # [ 1.313115] systemd[1]: Running in initrd.1693worker1 # [ 1.301758] systemd[1]: Detected virtualization qemu.1694worker1 # [ 1.303738] systemd[1]: Detected architecture arm64.1695worker2 # [ 1.304059] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1696client # [ 1.315709] systemd[1]: Initializing machine ID from random generator.1697worker1 # [ 1.305718] systemd[1]: Running in initrd.1698worker2 # [ 1.304117] systemd[1]: Detected virtualization qemu.1699client # [ 1.318699] systemd[1]: Hostname set to <client>.1700worker2 # [ 1.304186] systemd[1]: Detected architecture arm64.1701worker1 # [ 1.308306] systemd[1]: Initializing machine ID from random generator.1702worker2 # [ 1.304211] systemd[1]: Running in initrd.1703worker1 # [ 1.311336] systemd[1]: Hostname set to <worker1>.1704worker2 # [ 1.305097] systemd[1]: Initializing machine ID from random generator.1705worker2 # [ 1.305394] systemd[1]: Hostname set to <worker2>.1706lb # [ 1.379203] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707client # [ 1.368547] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708worker1 # [ 1.360783] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01709worker2 # [ 1.381953] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01710lb # [ 1.499035] usb 1-2: new high-speed USB device number 3 using ehci-pci1711client # [ 1.488314] usb 1-2: new high-speed USB device number 3 using ehci-pci1712worker1 # [ 1.484586] usb 1-2: new high-speed USB device number 3 using ehci-pci1713worker2 # [ 1.501747] usb 1-2: new high-speed USB device number 3 using ehci-pci1714lb # [ 1.624303] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715lb # [ 1.655782] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21716lb # [ 1.661404] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01717client # [ 1.617277] systemd[1]: bpf-restrict-fs: LSM BPF program attached1718worker1 # [ 1.613055] systemd[1]: bpf-restrict-fs: LSM BPF program attached1719worker2 # [ 1.630699] systemd[1]: bpf-restrict-fs: LSM BPF program attached1720client # [ 1.652319] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721worker1 # [ 1.641137] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21722client # [ 1.657859] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723worker1 # [ 1.646886] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01724worker2 # [ 1.666615] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21725worker2 # [ 1.673981] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01726lb # [ 1.743823] systemd[1]: Queued start job for default target Initrd Default Target.1727lb # [ 1.753408] systemd[1]: Created slice Slice /system/modprobe.1728lb # [ 1.754623] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1729lb # [ 1.756106] systemd[1]: Expecting device /dev/disk/by-label/nixos...1730lb # [ 1.757201] systemd[1]: Reached target Path Units.1731lb # [ 1.758038] systemd[1]: Reached target Slice Units.1732lb # [ 1.758876] systemd[1]: Reached target Swaps.1733lb # [ 1.759701] systemd[1]: Reached target Timer Units.1734lb # [ 1.760731] systemd[1]: Listening on D-Bus System Message Bus Socket.1735lb # [ 1.761987] systemd[1]: Listening on Journal Socket (/dev/log).1736lb # [ 1.763268] systemd[1]: Listening on Journal Sockets.1737lb # [ 1.763416] systemd[1]: Listening on udev Control Socket.1738lb # [ 1.763552] systemd[1]: Listening on udev Kernel Socket.1739lb # [ 1.763579] systemd[1]: Reached target Socket Units.1740lb # [ 1.768745] systemd[1]: Starting Create List of Static Device Nodes...1741lb # [ 1.769921] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1742lb # [ 1.779149] systemd[1]: Mounting Kernel Configuration File System...1743lb # [ 1.787239] systemd[1]: Starting Journal Service...1744client # [ 1.736248] systemd[1]: Queued start job for default target Initrd Default Target.1745worker1 # [ 1.730188] systemd[1]: Queued start job for default target Initrd Default Target.1746client # [ 1.746838] systemd[1]: Created slice Slice /system/modprobe.1747client # [ 1.748067] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1748worker2 # [ 1.747684] systemd[1]: Queued start job for default target Initrd Default Target.1749client # [ 1.749578] systemd[1]: Expecting device /dev/disk/by-label/nixos...1750client # [ 1.750678] systemd[1]: Reached target Path Units.1751worker1 # [ 1.739003] systemd[1]: Created slice Slice /system/modprobe.1752client # [ 1.751530] systemd[1]: Reached target Slice Units.1753client # [ 1.752433] systemd[1]: Reached target Swaps.1754worker1 # [ 1.740242] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1755client # [ 1.753242] systemd[1]: Reached target Timer Units.1756worker1 # [ 1.741873] systemd[1]: Expecting device /dev/disk/by-label/nixos...1757client # [ 1.754301] systemd[1]: Listening on D-Bus System Message Bus Socket.1758worker1 # [ 1.742991] systemd[1]: Reached target Path Units.1759worker1 # [ 1.743846] systemd[1]: Reached target Slice Units.1760client # [ 1.755594] systemd[1]: Listening on Journal Socket (/dev/log).1761worker1 # [ 1.744757] systemd[1]: Reached target Swaps.1762client # [ 1.756927] systemd[1]: Listening on Journal Sockets.1763worker2 # [ 1.756256] systemd[1]: Created slice Slice /system/modprobe.1764worker1 # [ 1.745547] systemd[1]: Reached target Timer Units.1765client # [ 1.757969] systemd[1]: Listening on udev Control Socket.1766lb # [ 1.810952] systemd[1]: Starting Load Kernel Modules...1767worker1 # [ 1.746600] systemd[1]: Listening on D-Bus System Message Bus Socket.1768worker2 # [ 1.757532] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1769client # [ 1.759038] systemd[1]: Listening on udev Kernel Socket.1770worker1 # [ 1.747858] systemd[1]: Listening on Journal Socket (/dev/log).1771lb # [ 1.811992] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1772client # [ 1.759973] systemd[1]: Reached target Socket Units.1773worker1 # [ 1.749097] systemd[1]: Listening on Journal Sockets.1774worker1 # [ 1.750132] systemd[1]: Listening on udev Control Socket.1775worker2 # [ 1.759087] systemd[1]: Expecting device /dev/disk/by-label/nixos...1776worker1 # [ 1.751207] systemd[1]: Listening on udev Kernel Socket.1777worker2 # [ 1.759126] systemd[1]: Reached target Path Units.1778worker2 # [ 1.759148] systemd[1]: Reached target Slice Units.1779worker1 # [ 1.752165] systemd[1]: Reached target Socket Units.1780client # [ 1.762627] systemd[1]: Starting Create List of Static Device Nodes...1781worker2 # [ 1.759169] systemd[1]: Reached target Swaps.1782worker2 # [ 1.759191] systemd[1]: Reached target Timer Units.1783client # [ 1.762711] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1784worker2 # [ 1.759393] systemd[1]: Listening on D-Bus System Message Bus Socket.1785worker2 # [ 1.759568] systemd[1]: Listening on Journal Socket (/dev/log).1786worker1 # [ 1.754867] systemd[1]: Starting Create List of Static Device Nodes...1787worker2 # [ 1.759720] systemd[1]: Listening on Journal Sockets.1788worker2 # [ 1.759867] systemd[1]: Listening on udev Control Socket.1789worker1 # [ 1.754950] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1790worker2 # [ 1.759958] systemd[1]: Listening on udev Kernel Socket.1791worker2 # [ 1.759982] systemd[1]: Reached target Socket Units.1792worker2 # [ 1.772049] systemd[1]: Starting Create List of Static Device Nodes...1793client # [ 1.772431] systemd[1]: Mounting Kernel Configuration File System...1794worker2 # [ 1.773328] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1795lb # [ 1.831221] systemd[1]: Starting Coldplug All udev Devices...1796worker1 # [ 1.764737] systemd[1]: Mounting Kernel Configuration File System...1797client # [ 1.780553] systemd[1]: Starting Journal Service...1798lb # [ 1.836014] systemd-journald[72]: Collecting audit messages is disabled.1799worker1 # [ 1.772817] systemd[1]: Starting Journal Service...1800worker2 # [ 1.784957] systemd[1]: Mounting Kernel Configuration File System...1801lb # [ 1.847104] systemd[1]: Finished Create List of Static Device Nodes.1802lb # [ 1.848631] systemd[1]: Mounted Kernel Configuration File System.1803lb # [ 1.859356] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1804worker2 # [ 1.814074] systemd[1]: Starting Journal Service...1805worker2 # [ 1.816617] systemd[1]: Starting Load Kernel Modules...1806client # [ 1.816539] systemd[1]: Starting Load Kernel Modules...1807client # [ 1.816659] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1808worker2 # [ 1.817569] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1809client # [ 1.820043] systemd[1]: Starting Coldplug All udev Devices...1810worker1 # [ 1.808711] systemd[1]: Starting Load Kernel Modules...1811worker1 # [ 1.808830] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1812worker1 # [ 1.812153] systemd[1]: Starting Coldplug All udev Devices...1813worker2 # [ 1.828912] systemd[1]: Starting Coldplug All udev Devices...1814client # [ 1.839656] systemd-journald[72]: Collecting audit messages is disabled.1815worker1 # [ 1.831438] systemd-journald[72]: Collecting audit messages is disabled.1816lb # [ 1.899609] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1817worker2 # [ 1.845879] systemd[1]: Finished Create List of Static Device Nodes.1818worker2 # [ 1.846620] systemd[1]: Mounted Kernel Configuration File System.1819lb # [ 1.902080] systemd[1]: Starting Create Static Device Nodes in /dev...1820worker1 # [ 1.840674] systemd[1]: Finished Create List of Static Device Nodes.1821client # [ 1.852654] systemd[1]: Finished Create List of Static Device Nodes.1822client # [ 1.853733] systemd[1]: Mounted Kernel Configuration File System.1823worker2 # [ 1.858200] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1824client # [ 1.862272] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1825worker1 # [ 1.850971] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1826client # [ 1.864381] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1827lb # [ 1.918845] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1828worker1 # [ 1.856902] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1829worker1 # [ 1.857213] systemd[1]: Mounted Kernel Configuration File System.1830client # [ 1.880420] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1831worker1 # [ 1.872608] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1832worker2 # [ 1.884369] systemd-journald[72]: Collecting audit messages is disabled.1833lb # [ 1.931103] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1834lb # [ 1.937188] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01835lb # [ 1.937433] [drm] features: -virgl +edid -resource_blob -host_visible1836lb # [ 1.937443] [drm] features: -context_init1837lb # [ 1.938229] [drm] number of scanouts: 11838lb # [ 1.938248] [drm] number of cap sets: 01839client # [ 1.889536] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01840client # [ 1.889767] [drm] features: -virgl +edid -resource_blob -host_visible1841client # [ 1.889777] [drm] features: -context_init1842client # [ 1.890483] [drm] number of scanouts: 11843client # [ 1.890501] [drm] number of cap sets: 01844worker1 # [ 1.886216] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01845worker1 # [ 1.886453] [drm] features: -virgl +edid -resource_blob -host_visible1846worker1 # [ 1.886464] [drm] features: -context_init1847worker1 # [ 1.887168] [drm] number of scanouts: 11848worker1 # [ 1.887187] [drm] number of cap sets: 01849worker2 # [ 1.902761] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1850lb # [ 1.955579] systemd[1]: Finished Create Static Device Nodes in /dev.1851lb # [ 1.955922] systemd[1]: Reached target Preparation for Local File Systems.1852lb # [ 1.955950] systemd[1]: Reached target Local File Systems.1853lb # [ 1.960019] systemd[1]: Starting Rule-based Manager for Device Events and Files...1854lb # [ 1.967468] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1855worker1 # [ 1.905449] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1856lb # [ 1.967488] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01857worker2 # [ 1.917949] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1858client # [ 1.916630] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1859worker1 # [ 1.908916] systemd[1]: Starting Create Static Device Nodes in /dev...1860worker2 # [ 1.920411] systemd[1]: Starting Create Static Device Nodes in /dev...1861client # [ 1.916649] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01862worker2 # [ 1.921788] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1863worker1 # [ 1.910047] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1864worker1 # [ 1.910065] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01865client # [ 1.924529] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1866client # [ 1.927205] systemd[1]: Starting Create Static Device Nodes in /dev...1867worker2 # [ 1.938883] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01868worker2 # [ 1.939119] [drm] features: -virgl +edid -resource_blob -host_visible1869worker2 # [ 1.939129] [drm] features: -context_init1870worker2 # [ 1.939827] [drm] number of scanouts: 11871worker2 # [ 1.939844] [drm] number of cap sets: 01872lb # [ 1.995283] Console: switching to colour frame buffer device 160x501873client # [ 1.947464] Console: switching to colour frame buffer device 160x501874worker1 # [ 1.944848] Console: switching to colour frame buffer device 160x501875worker2 # [ 1.958396] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1876worker2 # [ 1.958413] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01877lb # [ 2.027565] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1878client # [ 1.978114] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1879lb # [ 2.032401] systemd[1]: Finished Load Kernel Modules.1880client # [ 1.983278] systemd[1]: Finished Create Static Device Nodes in /dev.1881lb # [ 2.021914] systemd-modules-load[74]: Inserted module 'dm_mod'1882client # [ 1.985310] systemd[1]: Reached target Preparation for Local File Systems.1883worker1 # [ 1.973960] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1884client # [ 1.987071] systemd[1]: Reached target Local File Systems.1885worker1 # [ 1.978202] systemd[1]: Finished Create Static Device Nodes in /dev.1886lb # [ 2.043952] systemd[1]: Starting Apply Kernel Variables...1887worker1 # [ 1.978415] systemd[1]: Reached target Preparation for Local File Systems.1888worker1 # [ 1.978449] systemd[1]: Reached target Local File Systems.1889worker2 # [ 1.986022] Console: switching to colour frame buffer device 160x501890worker1 # [ 1.982539] systemd[1]: Starting Rule-based Manager for Device Events and Files...1891worker2 # [ 1.992665] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1892lb # [ 2.047225] systemd[1]: Started Journal Service.1893lb # [ 2.036226] systemd-modules-load[74]: Module 'virtio_balloon' is built in1894client # [ 1.996845] systemd[1]: Starting Rule-based Manager for Device Events and Files...1895lb # [ 2.044345] systemd-modules-load[74]: Module 'virtio_console' is built in1896worker2 # [ 2.006381] systemd[1]: Finished Create Static Device Nodes in /dev.1897worker2 # [ 2.006777] systemd[1]: Reached target Preparation for Local File Systems.1898worker2 # [ 2.006811] systemd[1]: Reached target Local File Systems.1899lb # [ 2.048346] systemd-modules-load[74]: Inserted module 'virtio_gpu'1900worker2 # [ 2.010830] systemd[1]: Starting Rule-based Manager for Device Events and Files...1901client # [ 2.012550] systemd[1]: Finished Load Kernel Modules.1902lb # [ 2.052382] systemd-modules-load[74]: Module 'virtio_rng' is built in1903worker1 # [ 2.000719] systemd[1]: Finished Load Kernel Modules.1904client # [ 2.020831] systemd[1]: Starting Apply Kernel Variables...1905worker1 # [ 2.009280] systemd[1]: Starting Apply Kernel Variables...1906lb # [ 2.064083] systemd[1]: Starting Create System Files and Directories...1907lb # [ 2.066655] systemd-udevd[79]: Using default interface naming scheme 'v261'.1908worker2 # [ 2.033989] systemd[1]: Finished Load Kernel Modules.1909worker2 # [ 2.047049] systemd[1]: Starting Apply Kernel Variables...1910client # [ 2.044719] systemd-modules-load[73]: Inserted module 'dm_mod'1911worker1 # [ 2.037457] systemd-modules-load[73]: Inserted module 'dm_mod'1912client # [ 2.048576] systemd-modules-load[73]: Module 'virtio_balloon' is built in1913worker1 # [ 2.052835] systemd[1]: Started Journal Service.1914worker2 # [ 2.064275] systemd[1]: Started Journal Service.1915client # [ 2.049788] systemd-modules-load[73]: Module 'virtio_console' is built in1916client # [ 2.050940] systemd-modules-load[73]: Inserted module 'virtio_gpu'1917lb # [ 2.105400] systemd[1]: Finished Create System Files and Directories.1918client # [ 2.051980] systemd-modules-load[73]: Module 'virtio_rng' is built in1919client # [ 2.062163] systemd[1]: Started Journal Service.1920lb # [ 2.112219] systemd[1]: Finished Apply Kernel Variables.1921worker1 # [ 2.052312] systemd-modules-load[73]: Module 'virtio_balloon' is built in1922worker1 # [ 2.053527] systemd-modules-load[73]: Module 'virtio_console' is built in1923worker2 # [ 2.064297] systemd-modules-load[73]: Inserted module 'dm_mod'1924worker1 # [ 2.054648] systemd-modules-load[73]: Inserted module 'virtio_gpu'1925worker1 # [ 2.055700] systemd-modules-load[73]: Module 'virtio_rng' is built in1926worker2 # [ 2.066615] systemd-modules-load[73]: Module 'virtio_balloon' is built in1927worker2 # [ 2.067766] systemd-modules-load[73]: Module 'virtio_console' is built in1928client # [ 2.072502] systemd[1]: Starting Create System Files and Directories...1929lb # [ 2.128876] systemd[1]: Started Rule-based Manager for Device Events and Files.1930worker2 # [ 2.080489] systemd-modules-load[73]: Inserted module 'virtio_gpu'1931worker2 # [ 2.081576] systemd-modules-load[73]: Module 'virtio_rng' is built in1932worker2 # [ 2.082838] systemd[1]: Starting Create System Files and Directories...1933worker1 # [ 2.072968] systemd[1]: Starting Create System Files and Directories...1934client # [ 2.085287] systemd[1]: Finished Apply Kernel Variables.1935worker1 # [ 2.080089] systemd[1]: Finished Apply Kernel Variables.1936client # [ 2.103450] systemd[1]: Finished Create System Files and Directories.1937worker1 # [ 2.097699] systemd-udevd[79]: Using default interface naming scheme 'v261'.1938worker2 # [ 2.108596] systemd-udevd[78]: Using default interface naming scheme 'v261'.1939worker2 # [ 2.113611] systemd[1]: Finished Apply Kernel Variables.1940client # [ 2.120618] systemd-udevd[79]: Using default interface naming scheme 'v261'.1941worker1 # [ 2.121087] systemd[1]: Finished Create System Files and Directories.1942lb # [ 2.187860] systemd[1]: Starting Virtual Console Setup...1943worker2 # [ 2.140365] systemd[1]: Finished Create System Files and Directories.1944worker1 # [ 2.139094] systemd[1]: Started Rule-based Manager for Device Events and Files.1945worker2 # [ 2.158641] systemd[1]: Started Rule-based Manager for Device Events and Files.1946client # [ 2.157819] systemd[1]: Started Rule-based Manager for Device Events and Files.1947lb # [ 2.240541] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1948lb # [ 2.244081] systemd[1]: Finished Virtual Console Setup.1949worker2 # [ 2.214113] systemd[1]: Starting Virtual Console Setup...1950worker1 # [ 2.211389] systemd[1]: Starting Virtual Console Setup...1951client # [ 2.220884] systemd[1]: Starting Virtual Console Setup...1952client # [ 2.272545] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1953client # [ 2.275991] systemd[1]: Finished Virtual Console Setup.1954worker1 # [ 2.272489] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1955worker2 # [ 2.284553] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1956worker1 # [ 2.275887] systemd[1]: Finished Virtual Console Setup.1957worker2 # [ 2.288005] systemd[1]: Finished Virtual Console Setup.1958lb # [ 2.939814] systemd[1]: Finished Coldplug All udev Devices.1959lb # [ 2.940882] systemd[1]: Reached target System Initialization.1960lb # [ 2.941671] systemd[1]: Reached target Basic System.1961worker2 # [ 2.963637] systemd[1]: Finished Coldplug All udev Devices.1962worker2 # [ 2.964664] systemd[1]: Reached target System Initialization.1963worker2 # [ 2.965471] systemd[1]: Reached target Basic System.1964worker1 # [ 2.970240] systemd[1]: Finished Coldplug All udev Devices.1965worker1 # [ 2.971212] systemd[1]: Reached target System Initialization.1966worker1 # [ 2.976082] systemd[1]: Reached target Basic System.1967client # [ 3.009511] systemd[1]: Finished Coldplug All udev Devices.1968client # [ 3.011149] systemd[1]: Reached target System Initialization.1969client # [ 3.012006] systemd[1]: Reached target Basic System.1970lb # [ 3.107692] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1971lb # [ 3.137345] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972lb # [ 3.143643] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1973worker2 # [ 3.111837] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1974worker1 # [ 3.151474] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1975worker2 # [ 3.165727] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976worker2 # [ 3.170914] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1977lb # [ 3.227810] systemd[1]: Found device /dev/disk/by-label/nixos.1978lb # [ 3.233685] systemd[1]: Reached target Initrd Root Device.1979lb # [ 3.240148] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1980worker1 # [ 3.197272] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1981worker1 # [ 3.202283] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1982client # [ 3.218104] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1983client # [ 3.224335] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1984worker2 # [ 3.228110] systemd[1]: Found device /dev/disk/by-label/nixos.1985client # [ 3.228144] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1986worker2 # [ 3.230630] systemd[1]: Reached target Initrd Root Device.1987worker2 # [ 3.234909] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1988lb # [ 3.291719] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1989lb # [ 3.300093] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1990lb # [ 3.308367] systemd[1]: Mounting /sysroot...1991worker1 # [ 3.260474] systemd[1]: Found device /dev/disk/by-label/nixos.1992worker1 # [ 3.263916] systemd[1]: Reached target Initrd Root Device.1993worker1 # [ 3.268798] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1994worker2 # [ 3.293014] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1995lb # [ 3.358909] EXT4-fs (vda): mounted filesystem 403c5b88-34c7-4837-963b-b9ae6e158bd5 r/w with ordered data mode. Quota mode: none.1996lb # [ 3.349395] systemd[1]: Mounted /sysroot.1997worker2 # [ 3.300089] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1998lb # [ 3.355286] systemd[1]: Reached target Initrd Root File System.1999lb # [ 3.358163] systemd[1]: Starting Mountpoints Configured in the Real Root...2000client # [ 3.305509] systemd[1]: Found device /dev/disk/by-label/nixos.
loading 2,227 hidden lines…
4228worker2 # [ 75.991755] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.619Z level=INFO msg="All 1 paths already cached"4229client # copying 1 paths to 'lb:50051'...4230client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4231worker2 # [ 76.005113] nix-grpc-daemon[772]: ts=2026-09-23T05:34:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406184232worker2 # [ 76.020139] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4233worker2 # [ 76.206535] nix-daemon[844]: accepted connection from pid 912, user nix-grpc-daemon (trusted)4234worker2 # [ 76.227732] nix-daemon[844]: reaped child process 915, status = succeeded4235lb # [ 76.286415] niks3-server[1016]: 2026/09/23 05:34:08 INFO Received uploads request method=POST path=/api/pending_closures4236worker2 # [ 76.244115] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.872Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4237worker2 # [ 76.245607] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.872Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4238lb # [ 76.320449] niks3-server[1016]: 2026/09/23 05:34:08 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4239worker2 # [ 76.280112] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.907Z level=INFO msg="Uploading 1 narinfos"4240lb # [ 76.333084] niks3-server[1016]: 2026/09/23 05:34:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4241lb # [ 76.345706] niks3-server[1016]: 2026/09/23 05:34:08 INFO Signed narinfos id=11 count=14242lb # [ 76.351683] niks3-server[1016]: 2026/09/23 05:34:08 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4243lb # [ 76.365293] niks3-server[1016]: 2026/09/23 05:34:08 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4244lb # [ 76.371062] niks3-server[1016]: 2026/09/23 05:34:09 INFO Completed upload id=114245worker2 # [ 76.318887] nix-grpc-daemon[869]: time=2026-09-23T05:34:08.947Z level=INFO msg="Upload complete. (281ms)"4246lb # [ 76.377361] niks3-server[1016]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4247worker2 # [ 76.328348] nix-grpc-daemon[772]: ts=2026-09-23T05:34:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=6244248worker2 # [ 76.343924] nix-grpc-daemon[772]: ts=2026-09-23T05:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184249worker2 # [ 76.350308] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4250worker2 # [ 76.356490] nix-daemon[844]: reaped child process 910, status = succeeded4251client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4252worker2 # [ 76.369618] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4253client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4254client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4255client # farm-b> LOG-farm-b-sig4256worker2 # [ 77.868595] nix-daemon[844]: reaped child process 919, status = succeeded4257worker2 # [ 77.936348] nix-daemon[844]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4258worker2 # [ 77.953960] nix-daemon[844]: reaped child process 938, status = succeeded4259lb # [ 78.013611] niks3-server[1016]: 2026/09/23 05:34:10 INFO Received uploads request method=POST path=/api/pending_closures4260worker2 # [ 77.973057] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.600Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4261worker2 # [ 77.974606] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.600Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4262lb # [ 78.052904] niks3-server[1016]: 2026/09/23 05:34:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4263worker2 # [ 78.012182] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.640Z level=INFO msg="Uploading 1 narinfos"4264lb # [ 78.069985] niks3-server[1016]: 2026/09/23 05:34:10 INFO Signed narinfos id=12 count=14265lb # [ 78.078273] niks3-server[1016]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4266lb # [ 78.090841] niks3-server[1016]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4267worker2 # [ 78.050556] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.678Z level=INFO msg="Upload complete. (267ms)"4268lb # [ 78.102889] niks3-server[1016]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4269lb # [ 78.106670] niks3-server[1016]: 2026/09/23 05:34:10 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4270lb # [ 78.111633] niks3-server[1016]: 2026/09/23 05:34:10 INFO Completed upload id=124271lb # [ 78.115017] niks3-server[1016]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4272worker1 # [ 78.061084] nix-grpc-daemon[769]: ts=2026-09-23T05:34:10Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04273worker2 # [ 78.061106] nix-grpc-daemon[772]: ts=2026-09-23T05:34:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14274worker2 # [ 78.080332] nix-daemon[844]: reaped child process 917, status = succeeded4275worker1 # [ 78.066931] nix-grpc-daemon[769]: ts=2026-09-23T05:34:10Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14276worker2 # [ 78.116914] nix-grpc-daemon[772]: ts=2026-09-23T05:34:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184277worker2 # [ 78.132311] nix-grpc-daemon[772]: ts=2026-09-23T05:34:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406184278worker2 # [ 78.145702] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.773Z level=INFO msg="All 2 paths already cached"4279client # copying 1 paths to 'lb:50051'...4280client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4281worker2 # [ 78.156615] nix-grpc-daemon[772]: ts=2026-09-23T05:34:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406184282worker2 # [ 78.170163] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4283worker2 # [ 78.324328] nix-daemon[844]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4284worker2 # [ 78.342951] nix-daemon[844]: reaped child process 945, status = succeeded4285lb # [ 78.401414] niks3-server[1016]: 2026/09/23 05:34:11 INFO Received uploads request method=POST path=/api/pending_closures4286worker2 # [ 78.358104] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.986Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4287worker2 # [ 78.360264] nix-grpc-daemon[869]: time=2026-09-23T05:34:10.988Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4288lb # [ 78.435267] niks3-server[1016]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4289worker2 # [ 78.390905] nix-grpc-daemon[869]: time=2026-09-23T05:34:11.018Z level=INFO msg="Uploading 1 narinfos"4290lb # [ 78.449484] niks3-server[1016]: 2026/09/23 05:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4291lb # [ 78.462066] niks3-server[1016]: 2026/09/23 05:34:11 INFO Signed narinfos id=13 count=14292worker2 # [ 78.415672] nix-grpc-daemon[869]: time=2026-09-23T05:34:11.043Z level=INFO msg="Upload complete. (229ms)"4293lb # [ 78.468236] niks3-server[1016]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4294lb # [ 78.471999] niks3-server[1016]: 2026/09/23 05:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4295lb # [ 78.478578] niks3-server[1016]: 2026/09/23 05:34:11 INFO Completed upload id=134296lb # [ 78.482418] niks3-server[1016]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4297worker2 # [ 78.424345] nix-grpc-daemon[772]: ts=2026-09-23T05:34:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=7524298worker2 # [ 78.438912] nix-grpc-daemon[772]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184299worker2 # [ 78.444819] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4300worker2 # [ 78.449719] nix-daemon[844]: reaped child process 940, status = succeeded4301client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4302worker2 # [ 78.462495] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4303client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4304lb # [ 79.392837] envoy[907]: conn peer=[2001:db8:1::1]:60042 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9536 tx=4414 ms=61674305lb # [ 79.397961] envoy[907]: conn peer=[2001:db8:1::1]:60056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=564306lb # [ 79.402656] envoy[907]: conn peer=[2001:db8:1::1]:41948 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1895 ms=21594307client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4308client # farm-top> LOG-farm-top-sig4309worker2 # [ 80.152730] nix-daemon[844]: reaped child process 949, status = succeeded4310worker2 # [ 80.248942] nix-daemon[844]: accepted connection from pid 965, user nix-grpc-daemon (trusted)4311worker2 # [ 80.267067] nix-daemon[844]: reaped child process 968, status = succeeded4312lb # [ 80.326736] niks3-server[1016]: 2026/09/23 05:34:12 INFO Received uploads request method=POST path=/api/pending_closures4313worker2 # [ 80.287291] nix-grpc-daemon[869]: time=2026-09-23T05:34:12.913Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4314worker2 # [ 80.289157] nix-grpc-daemon[869]: time=2026-09-23T05:34:12.913Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4315worker2 # [ 80.324505] nix-grpc-daemon[869]: time=2026-09-23T05:34:12.952Z level=INFO msg="Uploading 1 narinfos"4316lb # [ 80.366336] niks3-server[1016]: 2026/09/23 05:34:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4317lb # [ 80.382609] niks3-server[1016]: 2026/09/23 05:34:13 INFO Signed narinfos id=14 count=14318lb # [ 80.389857] niks3-server[1016]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4319lb # [ 80.400305] niks3-server[1016]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4320lb # [ 80.403429] niks3-server[1016]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4321worker2 # [ 80.355385] nix-grpc-daemon[869]: time=2026-09-23T05:34:12.983Z level=INFO msg="Upload complete. (280ms)"4322lb # [ 80.411107] niks3-server[1016]: 2026/09/23 05:34:13 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4323lb # [ 80.415031] niks3-server[1016]: 2026/09/23 05:34:13 INFO Completed upload id=144324lb # [ 80.418501] niks3-server[1016]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4325worker2 # [ 80.369170] nix-grpc-daemon[772]: ts=2026-09-23T05:34:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14326worker2 # [ 80.386190] nix-daemon[844]: reaped child process 947, status = succeeded4327client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.31 seconds)4328client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4329client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4330client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4331worker1 # [ 80.729119] nix-grpc-daemon[769]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:427284332client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4333client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4334client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4335client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4336worker1 # [ 80.776869] nix-grpc-daemon[769]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:427284337worker1 # [ 80.786310] nix-grpc-daemon[769]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4338worker1 # [ 80.803266] nix-grpc-daemon[769]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4339worker1 # [ 80.819337] nix-grpc-daemon[769]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4340client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.26 seconds)4341client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4342worker2 # [ 81.050397] nix-grpc-daemon[772]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:408284343worker2 # [ 81.092924] nix-grpc-daemon[772]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:408284344client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.29 seconds)4345client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4346client # finding garbage collector roots...4347client # removing stale temporary roots file "/nix/var/nix/temproots/905"4348client # removing stale temporary roots file "/nix/var/nix/temproots/864"4349client # removing stale temporary roots file "/nix/var/nix/temproots/885"4350client # removing stale temporary roots file "/nix/var/nix/temproots/926"4351client # removing stale temporary roots file "/nix/var/nix/temproots/918"4352client # removing stale temporary roots file "/nix/var/nix/temproots/849"4353client # removing stale temporary roots file "/nix/var/nix/temproots/897"4354client # removing stale temporary roots file "/nix/var/nix/temproots/941"4355client # deleting specified paths...4356client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4357client # deleting unused links...4358client # note: hard linking is currently saving 0.0 KiB4359worker1 # [ 81.479706] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507324360worker1 # [ 81.486532] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324361worker1 # [ 81.501596] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.135Z level=INFO msg="All 1 paths already cached"4362worker2 # [ 81.518417] nix-grpc-daemon[772]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406184363worker1 # [ 81.510162] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:507324364client # copying 3 paths...4365worker1 # [ 81.524957] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:507324366client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4367worker2 # [ 81.561484] nix-grpc-daemon[772]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406184368worker2 # [ 81.569208] nix-grpc-daemon[772]: ts=2026-09-23T05:34:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_out=2324369worker1 # [ 81.571093] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324370worker1 # [ 81.579910] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=1764371worker1 # [ 81.589790] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324372worker1 # [ 81.594912] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=1284373client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4374client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4375client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.54 seconds)4376(finished: subtest: outputs read back through any worker carry the cache signature, in 7.60 seconds)4377subtest: two clients wanting the same drv build it once4378client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4379client # [ 81.743113] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4380client # Running as unit: dedup1.service; invocation ID: e384347af7454a1e9eff4cadf0c2e3ff4381client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.06 seconds)4382client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4383client # [ 81.926800] nix[994]: don't know how to build these paths:4384client # [ 81.927800] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4385worker1 # [ 81.917019] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507324386worker1 # [ 81.930517] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04387worker1 # [ 81.937880] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14388worker1 # [ 81.978167] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324389worker1 # [ 81.993356] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324390client # [ 82.016124] nix[994]: copying 1 paths to 'lb:50051'...4391worker1 # [ 82.007716] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324392client # [ 82.018425] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4393worker1 # [ 82.030226] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4394worker1 # [ 82.197872] nix-daemon[855]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)4395worker1 # [ 82.219712] nix-daemon[855]: reaped child process 1050, status = succeeded4396lb # [ 82.286787] niks3-server[1016]: 2026/09/23 05:34:14 INFO Received uploads request method=POST path=/api/pending_closures4397worker1 # [ 82.235573] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.870Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4398worker1 # [ 82.237337] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.870Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4399lb # [ 82.317972] niks3-server[1016]: 2026/09/23 05:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4400worker1 # [ 82.266640] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.901Z level=INFO msg="Uploading 1 narinfos"4401lb # [ 82.331996] niks3-server[1016]: 2026/09/23 05:34:14 INFO Signed narinfos id=15 count=14402lb # [ 82.340804] niks3-server[1016]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4403worker1 # [ 82.294004] nix-grpc-daemon[869]: time=2026-09-23T05:34:14.929Z level=INFO msg="Upload complete. (243ms)"4404lb # [ 82.356971] niks3-server[1016]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4405lb # [ 82.366315] niks3-server[1016]: 2026/09/23 05:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4406worker1 # [ 82.304531] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=4324407lb # [ 82.370962] niks3-server[1016]: 2026/09/23 05:34:14 INFO Completed upload id=154408lb # [ 82.375650] niks3-server[1016]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4409worker1 # [ 82.316278] nix-grpc-daemon[769]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324410worker1 # [ 82.326857] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4411worker1 # [ 82.329758] nix-daemon[855]: reaped child process 1045, status = succeeded4412client # [ 82.348169] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4413worker1 # [ 82.343518] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4414client # [ 82.830496] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4415client # Running as unit: dedup2.service; invocation ID: e3cdea24b11544de9131a083f34115dd4416client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.10 seconds)4417worker2 # [ 83.033364] nix-grpc-daemon[772]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406184418client # [ 83.037096] nix[1010]: don't know how to build these paths:4419client # [ 83.038147] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4420worker1 # [ 83.041345] nix-grpc-daemon[769]: ts=2026-09-23T05:34:15Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04421client # [ 83.057374] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4422worker1 # [ 83.048772] nix-grpc-daemon[769]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324423client: must succeed: sleep 24424client: (finished: must succeed: sleep 2, in 2.05 seconds)4425client: waiting for success: ! systemctl is-active dedup1 dedup24426worker1 # [ 86.408756] nix-daemon[855]: reaped child process 1054, status = succeeded4427worker1 # [ 86.502253] nix-daemon[855]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)4428worker1 # [ 86.518863] nix-daemon[855]: reaped child process 1089, status = succeeded4429lb # [ 86.591574] niks3-server[1016]: 2026/09/23 05:34:19 INFO Received uploads request method=POST path=/api/pending_closures4430worker1 # [ 86.540397] nix-grpc-daemon[869]: time=2026-09-23T05:34:19.172Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4431worker1 # [ 86.542041] nix-grpc-daemon[869]: time=2026-09-23T05:34:19.173Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4432lb # [ 86.647464] niks3-server[1016]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4433worker1 # [ 86.594877] nix-grpc-daemon[869]: time=2026-09-23T05:34:19.230Z level=INFO msg="Uploading 1 narinfos"4434lb # [ 86.660877] niks3-server[1016]: 2026/09/23 05:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4435lb # [ 86.670433] niks3-server[1016]: 2026/09/23 05:34:19 INFO Signed narinfos id=16 count=14436lb # [ 86.676841] niks3-server[1016]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4437lb # [ 86.688344] niks3-server[1016]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4438lb # [ 86.691855] niks3-server[1016]: 2026/09/23 05:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4439lb # [ 86.698558] niks3-server[1016]: 2026/09/23 05:34:19 INFO Completed upload id=164440worker1 # [ 86.635794] nix-grpc-daemon[869]: time=2026-09-23T05:34:19.271Z level=INFO msg="Upload complete. (279ms)"4441lb # [ 86.701959] niks3-server[1016]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4442worker1 # [ 86.653137] nix-grpc-daemon[769]: ts=2026-09-23T05:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14443worker1 # [ 86.670195] nix-grpc-daemon[769]: ts=2026-09-23T05:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14444worker1 # [ 86.678033] nix-daemon[855]: reaped child process 1052, status = succeeded4445client # [ 86.691907] systemd[1]: dedup1.service: Deactivated successfully.4446client # [ 86.697189] systemd[1]: dedup1.service: Consumed 171ms CPU time over 4.950s wall clock time, 41.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4447client # [ 86.736640] systemd[1]: dedup2.service: Deactivated successfully.4448client # [ 86.739516] systemd[1]: dedup2.service: Consumed 182ms CPU time over 3.909s wall clock time, 33.3M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4449client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4450client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4451client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4452worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4453worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4454worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4455worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4456worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4457worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4458worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4459worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4460(finished: subtest: two clients wanting the same drv build it once, in 6.09 seconds)4461subtest: --timeout and --max-silent-time reach the builder4462worker2 # [ 88.016971] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406184463worker1 # [ 88.020286] nix-grpc-daemon[769]: ts=2026-09-23T05:34:20Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04464worker2 # [ 88.031697] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184465worker1 # [ 88.020630] nix-grpc-daemon[769]: ts=2026-09-23T05:34:20Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14466worker2 # [ 88.047024] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406184467worker2 # [ 88.061453] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406184468worker2 # [ 88.076788] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4469worker2 # [ 88.238982] nix-daemon[844]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4470worker2 # [ 88.260612] nix-daemon[844]: reaped child process 996, status = succeeded4471lb # [ 88.311755] niks3-server[1016]: 2026/09/23 05:34:20 INFO Received uploads request method=POST path=/api/pending_closures4472worker2 # [ 88.269961] nix-grpc-daemon[869]: time=2026-09-23T05:34:20.897Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4473worker2 # [ 88.272119] nix-grpc-daemon[869]: time=2026-09-23T05:34:20.899Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4474lb # [ 88.346061] niks3-server[1016]: 2026/09/23 05:34:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4475worker2 # [ 88.302869] nix-grpc-daemon[869]: time=2026-09-23T05:34:20.930Z level=INFO msg="Uploading 1 narinfos"4476lb # [ 88.360096] niks3-server[1016]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4477lb # [ 88.370623] niks3-server[1016]: 2026/09/23 05:34:20 INFO Signed narinfos id=17 count=14478lb # [ 88.382320] niks3-server[1016]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4479worker2 # [ 88.336282] nix-grpc-daemon[869]: time=2026-09-23T05:34:20.964Z level=INFO msg="Upload complete. (243ms)"4480lb # [ 88.393833] niks3-server[1016]: 2026/09/23 05:34:21 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4481worker2 # [ 88.344235] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=4244482lb # [ 88.399696] niks3-server[1016]: 2026/09/23 05:34:21 INFO Completed upload id=174483lb # [ 88.402720] niks3-server[1016]: 2026/09/23 05:34:21 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4484worker2 # [ 88.360106] nix-grpc-daemon[772]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184485worker2 # [ 88.370556] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4486worker2 # [ 88.373473] nix-daemon[844]: reaped child process 991, status = succeeded4487worker2 # [ 88.387717] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4488lb # [ 89.390562] envoy[907]: conn peer=[2001:db8:1::1]:41950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9515 tx=4378 ms=60854489lb # [ 89.395818] envoy[907]: conn peer=[2001:db8:1::1]:33534 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=1224490lb # [ 89.400961] envoy[907]: conn peer=[2001:db8:1::1]:33556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=884491lb # [ 89.405718] envoy[907]: conn peer=[2001:db8:1::1]:33550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=375 ms=1074492lb # [ 89.410360] envoy[907]: conn peer=[2001:db8:1::1]:33518 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2915 ms=1964493lb # [ 89.415168] envoy[907]: conn peer=[2001:db8:1::1]:33564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1539 ms=47474494lb # [ 89.420784] envoy[907]: conn peer=[2001:db8:1::1]:33566 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=36954495worker2 # [ 94.804845] nix-grpc-daemon[772]: ts=2026-09-23T05:34:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04496worker2 # [ 94.845452] nix-daemon[844]: reaped child process 998, status = succeeded4497worker2 # [ 94.847893] nix-daemon[844]: reaped child process 1000, status = succeeded4498worker1 # [ 95.039255] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507324499worker1 # [ 95.053949] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04500worker1 # [ 95.063747] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14501worker1 # [ 95.072795] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324502worker1 # [ 95.080290] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324503worker1 # [ 95.090344] nix-grpc-daemon[769]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324504worker1 # [ 95.105050] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4505worker1 # [ 95.256056] nix-daemon[855]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)4506worker1 # [ 95.273537] nix-daemon[855]: reaped child process 1109, status = succeeded4507lb # [ 95.343183] niks3-server[1016]: 2026/09/23 05:34:27 INFO Received uploads request method=POST path=/api/pending_closures4508worker1 # [ 95.291682] nix-grpc-daemon[869]: time=2026-09-23T05:34:27.927Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4509worker1 # [ 95.293693] nix-grpc-daemon[869]: time=2026-09-23T05:34:27.928Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4510lb # [ 95.383233] niks3-server[1016]: 2026/09/23 05:34:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4511worker1 # [ 95.332691] nix-grpc-daemon[869]: time=2026-09-23T05:34:27.968Z level=INFO msg="Uploading 1 narinfos"4512lb # [ 95.400965] niks3-server[1016]: 2026/09/23 05:34:28 INFO Signed narinfos id=18 count=14513lb # [ 95.407437] niks3-server[1016]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4514lb # [ 95.414893] niks3-server[1016]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4515worker1 # [ 95.364175] nix-grpc-daemon[869]: time=2026-09-23T05:34:27.999Z level=INFO msg="Upload complete. (244ms)"4516lb # [ 95.425994] niks3-server[1016]: 2026/09/23 05:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4517lb # [ 95.430801] niks3-server[1016]: 2026/09/23 05:34:28 INFO Completed upload id=184518lb # [ 95.433748] niks3-server[1016]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4519worker1 # [ 95.376130] nix-grpc-daemon[769]: ts=2026-09-23T05:34:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=4244520worker1 # [ 95.391283] nix-grpc-daemon[769]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324521worker1 # [ 95.398508] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4522worker1 # [ 95.404245] nix-daemon[855]: reaped child process 1104, status = succeeded4523worker1 # [ 95.419307] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4524lb # [ 99.386161] envoy[907]: conn peer=[2001:db8:1::1]:33574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=953 ms=68194525worker1 # [ 101.034100] nix-grpc-daemon[769]: ts=2026-09-23T05:34:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04526worker1 # [ 101.051620] nix-daemon[855]: reaped child process 1111, status = succeeded4527worker1 # [ 101.052918] nix-daemon[855]: reaped child process 1113, status = succeeded4528(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.48 seconds)4529subtest: the first client leaving does not fail the second4530client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4531client # [ 101.330822] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4532client # Running as unit: share1.service; invocation ID: c198c844f6e246449ac97f7a576ed51e4533client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4534client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4535worker1 # [ 101.537979] nix-grpc-daemon[769]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507324536client # [ 101.553224] nix[1083]: don't know how to build these paths:4537client # [ 101.554192] nix[1083]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4538worker1 # [ 101.556845] nix-grpc-daemon[769]: ts=2026-09-23T05:34:34Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04539worker1 # [ 101.563158] nix-grpc-daemon[769]: ts=2026-09-23T05:34:34Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14540worker2 # [ 101.578257] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184541worker2 # [ 101.593751] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406184542client # [ 101.603484] nix[1083]: copying 1 paths to 'lb:50051'...4543client # [ 101.606410] nix[1083]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4544worker2 # [ 101.609644] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406184545worker2 # [ 101.624359] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4546worker2 # [ 101.781870] nix-daemon[844]: accepted connection from pid 1023, user nix-grpc-daemon (trusted)4547worker2 # [ 101.799919] nix-daemon[844]: reaped child process 1026, status = succeeded4548lb # [ 101.859267] niks3-server[1016]: 2026/09/23 05:34:34 INFO Received uploads request method=POST path=/api/pending_closures4549worker2 # [ 101.817314] nix-grpc-daemon[869]: time=2026-09-23T05:34:34.445Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4550worker2 # [ 101.818855] nix-grpc-daemon[869]: time=2026-09-23T05:34:34.445Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4551lb # [ 101.886441] niks3-server[1016]: 2026/09/23 05:34:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4552worker2 # [ 101.842236] nix-grpc-daemon[869]: time=2026-09-23T05:34:34.470Z level=INFO msg="Uploading 1 narinfos"4553lb # [ 101.900095] niks3-server[1016]: 2026/09/23 05:34:34 INFO Signed narinfos id=19 count=14554lb # [ 101.909291] niks3-server[1016]: 2026/09/23 05:34:34 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4555lb # [ 101.920073] niks3-server[1016]: 2026/09/23 05:34:34 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4556lb # [ 101.932117] niks3-server[1016]: 2026/09/23 05:34:34 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4557lb # [ 101.941906] niks3-server[1016]: 2026/09/23 05:34:34 INFO Completed upload id=194558lb # [ 101.946623] niks3-server[1016]: 2026/09/23 05:34:34 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4559worker2 # [ 101.889398] nix-grpc-daemon[869]: time=2026-09-23T05:34:34.517Z level=INFO msg="Upload complete. (248ms)"4560worker2 # [ 101.900535] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=4324561worker2 # [ 101.918342] nix-grpc-daemon[772]: ts=2026-09-23T05:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184562worker2 # [ 101.928787] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4563worker2 # [ 101.932355] nix-daemon[844]: reaped child process 1021, status = succeeded4564client # [ 101.942304] nix[1083]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4565worker2 # [ 101.947885] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4566client # [ 102.445314] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4567client # Running as unit: share2.service; invocation ID: 8d0a91c46f4741c49c0a9a3d6e81ad724568client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds)4569worker1 # [ 102.667510] nix-grpc-daemon[769]: ts=2026-09-23T05:34:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507324570client # [ 102.683275] nix[1099]: don't know how to build these paths:4571client # [ 102.685137] nix[1099]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4572worker2 # [ 102.698674] nix-grpc-daemon[772]: ts=2026-09-23T05:34:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184573worker1 # [ 102.685524] nix-grpc-daemon[769]: ts=2026-09-23T05:34:35Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04574client # [ 102.700540] nix[1099]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4575client: must succeed: sleep 2; systemctl kill -s INT share14576client # [ 105.817467] systemd[1]: share1.service: Sent signal SIGINT to main process 1083 (nix) on client request.4577client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.08 seconds)4578client: waiting for success: ! systemctl is-active share14579client # [ 105.827663] nix[1083]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4580client # [ 105.830238] nix[1083]: error: interrupted by the user4581client # [ 105.899444] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4582client # [ 105.908633] systemd[1]: share1.service: Failed with result 'exit-code'.4583client # [ 105.910370] systemd[1]: share1.service: Consumed 179ms CPU time over 4.575s wall clock time, 43.4M memory peak, 6.1K incoming IP traffic, 8K outgoing IP traffic.4584client: (finished: waiting for success: ! systemctl is-active share1, in 1.14 seconds)4585client: waiting for success: ! systemctl is-active share24586worker2 # [ 107.399457] nix-daemon[844]: reaped child process 1030, status = succeeded4587worker2 # [ 107.526853] nix-daemon[844]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4588worker2 # [ 107.545928] nix-daemon[844]: reaped child process 1067, status = succeeded4589lb # [ 107.603798] niks3-server[1016]: 2026/09/23 05:34:40 INFO Received uploads request method=POST path=/api/pending_closures4590worker2 # [ 107.563356] nix-grpc-daemon[869]: time=2026-09-23T05:34:40.189Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4591worker2 # [ 107.565040] nix-grpc-daemon[869]: time=2026-09-23T05:34:40.189Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4592lb # [ 107.642449] niks3-server[1016]: 2026/09/23 05:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4593worker2 # [ 107.598134] nix-grpc-daemon[869]: time=2026-09-23T05:34:40.226Z level=INFO msg="Uploading 1 narinfos"4594lb # [ 107.657084] niks3-server[1016]: 2026/09/23 05:34:40 INFO Signed narinfos id=20 count=14595lb # [ 107.663448] niks3-server[1016]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4596lb # [ 107.669133] niks3-server[1016]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4597lb # [ 107.674549] niks3-server[1016]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4598lb # [ 107.683659] niks3-server[1016]: 2026/09/23 05:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4599lb # [ 107.690132] niks3-server[1016]: 2026/09/23 05:34:40 INFO Completed upload id=204600worker2 # [ 107.637754] nix-grpc-daemon[869]: time=2026-09-23T05:34:40.266Z level=INFO msg="Upload complete. (276ms)"4601lb # [ 107.692902] niks3-server[1016]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4602worker2 # [ 107.649202] nix-grpc-daemon[772]: ts=2026-09-23T05:34:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14603worker2 # [ 107.668374] nix-grpc-daemon[772]: ts=2026-09-23T05:34:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14604worker2 # [ 107.676485] nix-daemon[844]: reaped child process 1028, status = succeeded4605client # [ 107.725759] systemd[1]: share2.service: Deactivated successfully.4606client # [ 107.730890] systemd[1]: share2.service: Consumed 202ms CPU time over 5.281s wall clock time, 35.3M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4607client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4608client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4609client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4610(finished: subtest: the first client leaving does not fail the second, in 7.13 seconds)4611subtest: build hook: nix-daemon with builders = grpc://lb4612client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4613client # [ 108.537951] systemd[1]: Started Nix Daemon.4614client # [ 108.651677] nix-daemon[1151]: accepted connection from pid 1148, user root (trusted)4615worker1 # [ 108.792583] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:507324616worker1 # [ 108.798860] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324617worker1 # [ 108.816134] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04618worker1 # [ 108.823586] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14619worker1 # [ 108.832998] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324620worker1 # [ 108.844612] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324621worker1 # [ 108.859642] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324622worker1 # [ 108.874766] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4623worker1 # [ 109.035783] nix-daemon[855]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4624worker1 # [ 109.054531] nix-daemon[855]: reaped child process 1161, status = succeeded4625lb # [ 109.125292] niks3-server[1016]: 2026/09/23 05:34:41 INFO Received uploads request method=POST path=/api/pending_closures4626worker1 # [ 109.073311] nix-grpc-daemon[869]: time=2026-09-23T05:34:41.708Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4627worker1 # [ 109.075430] nix-grpc-daemon[869]: time=2026-09-23T05:34:41.708Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4628lb # [ 109.155865] niks3-server[1016]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4629worker1 # [ 109.107034] nix-grpc-daemon[869]: time=2026-09-23T05:34:41.742Z level=INFO msg="Uploading 1 narinfos"4630lb # [ 109.170219] niks3-server[1016]: 2026/09/23 05:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4631lb # [ 109.183119] niks3-server[1016]: 2026/09/23 05:34:41 INFO Signed narinfos id=21 count=14632lb # [ 109.190882] niks3-server[1016]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4633worker1 # [ 109.140125] nix-grpc-daemon[869]: time=2026-09-23T05:34:41.775Z level=INFO msg="Upload complete. (250ms)"4634lb # [ 109.204801] niks3-server[1016]: 2026/09/23 05:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4635lb # [ 109.212917] niks3-server[1016]: 2026/09/23 05:34:41 INFO Completed upload id=214636worker1 # [ 109.152336] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=5124637lb # [ 109.218180] niks3-server[1016]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4638worker1 # [ 109.168151] nix-grpc-daemon[769]: ts=2026-09-23T05:34:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324639worker1 # [ 109.175329] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4640worker1 # [ 109.181677] nix-daemon[855]: reaped child process 1156, status = succeeded4641worker1 # [ 109.198125] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4642lb # [ 109.391201] envoy[907]: conn peer=[2001:db8:1::1]:43720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=951 ms=60564643lb # [ 109.396332] envoy[907]: conn peer=[2001:db8:1::1]:46206 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2986 tx=773 ms=42954644lb # [ 109.401105] envoy[907]: conn peer=[2001:db8:1::1]:46218 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=50304645worker1 # [ 110.063908] nix-daemon[855]: reaped child process 1165, status = succeeded4646worker1 # [ 110.173481] nix-daemon[855]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)4647worker1 # [ 110.191909] nix-daemon[855]: reaped child process 1184, status = succeeded4648lb # [ 110.253724] niks3-server[1016]: 2026/09/23 05:34:42 INFO Received uploads request method=POST path=/api/pending_closures4649worker1 # [ 110.204431] nix-grpc-daemon[869]: time=2026-09-23T05:34:42.837Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4650worker1 # [ 110.205995] nix-grpc-daemon[869]: time=2026-09-23T05:34:42.837Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4651lb # [ 110.298061] niks3-server[1016]: 2026/09/23 05:34:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4652worker1 # [ 110.244817] nix-grpc-daemon[869]: time=2026-09-23T05:34:42.880Z level=INFO msg="Uploading 1 narinfos"4653lb # [ 110.312751] niks3-server[1016]: 2026/09/23 05:34:42 INFO Signed narinfos id=22 count=14654lb # [ 110.315444] niks3-server[1016]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4655lb # [ 110.335854] niks3-server[1016]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4656worker1 # [ 110.279562] nix-grpc-daemon[869]: time=2026-09-23T05:34:42.915Z level=INFO msg="Upload complete. (270ms)"4657lb # [ 110.343426] niks3-server[1016]: 2026/09/23 05:34:42 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4658lb # [ 110.348701] niks3-server[1016]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4659lb # [ 110.353973] niks3-server[1016]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4660lb # [ 110.358133] niks3-server[1016]: 2026/09/23 05:34:42 INFO Completed upload id=224661worker1 # [ 110.293086] nix-grpc-daemon[769]: ts=2026-09-23T05:34:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14662worker1 # [ 110.308950] nix-daemon[855]: reaped child process 1163, status = succeeded4663worker2 # [ 110.346457] nix-grpc-daemon[772]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406184664worker2 # [ 110.360789] nix-grpc-daemon[772]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4665worker2 # [ 110.401763] nix-grpc-daemon[772]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406184666worker2 # [ 110.408561] nix-grpc-daemon[772]: ts=2026-09-23T05:34:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_out=1284667worker1 # [ 110.698451] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:507324668worker2 # [ 110.712861] nix-grpc-daemon[772]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406184669worker2 # [ 110.724825] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.352Z level=INFO msg="All 1 paths already cached"4670worker1 # [ 110.732729] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04671worker1 # [ 110.740084] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14672worker1 # [ 110.749358] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324673worker1 # [ 110.760908] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324674worker1 # [ 110.775586] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.411Z level=INFO msg="All 1 paths already cached"4675worker1 # [ 110.786016] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324676worker1 # [ 110.800442] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4677worker1 # [ 110.966250] nix-daemon[855]: accepted connection from pid 1190, user nix-grpc-daemon (trusted)4678worker1 # [ 110.987633] nix-daemon[855]: reaped child process 1193, status = succeeded4679lb # [ 111.046853] niks3-server[1016]: 2026/09/23 05:34:43 INFO Received uploads request method=POST path=/api/pending_closures4680worker1 # [ 110.994336] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.629Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4681worker1 # [ 110.995801] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.629Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4682lb # [ 111.078184] niks3-server[1016]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4683worker1 # [ 111.028206] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.663Z level=INFO msg="Uploading 1 narinfos"4684lb # [ 111.093855] niks3-server[1016]: 2026/09/23 05:34:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4685lb # [ 111.105130] niks3-server[1016]: 2026/09/23 05:34:43 INFO Signed narinfos id=23 count=14686lb # [ 111.115541] niks3-server[1016]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4687worker1 # [ 111.061706] nix-grpc-daemon[869]: time=2026-09-23T05:34:43.697Z level=INFO msg="Upload complete. (244ms)"4688lb # [ 111.126501] niks3-server[1016]: 2026/09/23 05:34:43 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4689lb # [ 111.133026] niks3-server[1016]: 2026/09/23 05:34:43 INFO Completed upload id=234690worker1 # [ 111.070646] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=6244691lb # [ 111.136861] niks3-server[1016]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4692worker1 # [ 111.088253] nix-grpc-daemon[769]: ts=2026-09-23T05:34:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324693worker1 # [ 111.095591] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4694worker1 # [ 111.101032] nix-daemon[855]: reaped child process 1188, status = succeeded4695worker1 # [ 111.114779] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4696worker1 # [ 112.208483] nix-daemon[855]: reaped child process 1197, status = succeeded4697worker1 # [ 112.296269] nix-daemon[855]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)4698worker1 # [ 112.314925] nix-daemon[855]: reaped child process 1216, status = succeeded4699lb # [ 112.381638] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received uploads request method=POST path=/api/pending_closures4700worker1 # [ 112.332431] nix-grpc-daemon[869]: time=2026-09-23T05:34:44.964Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4701worker1 # [ 112.334116] nix-grpc-daemon[869]: time=2026-09-23T05:34:44.964Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4702lb # [ 112.425441] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4703worker1 # [ 112.371485] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.006Z level=INFO msg="Uploading 1 narinfos"4704lb # [ 112.438744] niks3-server[1016]: 2026/09/23 05:34:45 INFO Signed narinfos id=24 count=14705lb # [ 112.446841] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4706lb # [ 112.456793] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4707lb # [ 112.461164] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4708lb # [ 112.468882] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4709worker1 # [ 112.406462] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.042Z level=INFO msg="Upload complete. (280ms)"4710lb # [ 112.471964] niks3-server[1016]: 2026/09/23 05:34:45 INFO Completed upload id=244711lb # [ 112.475322] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4712worker1 # [ 112.420719] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14713worker1 # [ 112.439265] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:507324714worker1 # [ 112.445371] nix-daemon[855]: reaped child process 1195, status = succeeded4715worker2 # [ 112.479918] nix-grpc-daemon[772]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406184716worker2 # [ 112.487529] nix-grpc-daemon[772]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4717worker2 # [ 112.505556] nix-grpc-daemon[772]: ts=2026-09-23T05:34:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_out=1764718worker2 # [ 112.788882] nix-grpc-daemon[772]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:406184719worker2 # [ 112.796290] nix-grpc-daemon[772]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406184720worker2 # [ 112.804319] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.430Z level=INFO msg="All 2 paths already cached"4721worker1 # [ 112.806610] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04722worker1 # [ 112.814696] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14723worker1 # [ 112.823857] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324724worker1 # [ 112.835029] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324725worker1 # [ 112.849816] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.485Z level=INFO msg="All 2 paths already cached"4726worker1 # [ 112.859959] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324727worker1 # [ 112.874684] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4728worker1 # [ 113.027239] nix-daemon[855]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)4729worker1 # [ 113.046128] nix-daemon[855]: reaped child process 1225, status = succeeded4730lb # [ 113.114059] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received uploads request method=POST path=/api/pending_closures4731worker1 # [ 113.062382] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.697Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4732worker1 # [ 113.063953] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.697Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4733lb # [ 113.142692] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4734worker1 # [ 113.086644] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.722Z level=INFO msg="Uploading 1 narinfos"4735lb # [ 113.155331] niks3-server[1016]: 2026/09/23 05:34:45 INFO Signed narinfos id=25 count=14736lb # [ 113.164575] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4737lb # [ 113.173771] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4738worker1 # [ 113.126677] nix-grpc-daemon[869]: time=2026-09-23T05:34:45.762Z level=INFO msg="Upload complete. (235ms)"4739lb # [ 113.190397] niks3-server[1016]: 2026/09/23 05:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4740lb # [ 113.200712] niks3-server[1016]: 2026/09/23 05:34:45 INFO Completed upload id=254741worker1 # [ 113.136392] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=7524742lb # [ 113.203749] niks3-server[1016]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4743worker1 # [ 113.154986] nix-grpc-daemon[769]: ts=2026-09-23T05:34:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324744worker1 # [ 113.164363] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4745worker1 # [ 113.167527] nix-daemon[855]: reaped child process 1220, status = succeeded4746worker1 # [ 113.181290] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4747worker1 # [ 114.426926] nix-daemon[855]: reaped child process 1229, status = succeeded4748worker1 # [ 114.518390] nix-daemon[855]: accepted connection from pid 1245, user nix-grpc-daemon (trusted)4749worker1 # [ 114.533510] nix-daemon[855]: reaped child process 1248, status = succeeded4750lb # [ 114.603178] niks3-server[1016]: 2026/09/23 05:34:47 INFO Received uploads request method=POST path=/api/pending_closures4751worker1 # [ 114.551740] nix-grpc-daemon[869]: time=2026-09-23T05:34:47.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4752worker1 # [ 114.553386] nix-grpc-daemon[869]: time=2026-09-23T05:34:47.186Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4753worker1 # [ 114.592595] nix-grpc-daemon[869]: time=2026-09-23T05:34:47.227Z level=INFO msg="Uploading 1 narinfos"4754lb # [ 114.642944] niks3-server[1016]: 2026/09/23 05:34:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4755lb # [ 114.658069] niks3-server[1016]: 2026/09/23 05:34:47 INFO Signed narinfos id=26 count=14756lb # [ 114.667451] niks3-server[1016]: 2026/09/23 05:34:47 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4757lb # [ 114.677795] niks3-server[1016]: 2026/09/23 05:34:47 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4758lb # [ 114.688692] niks3-server[1016]: 2026/09/23 05:34:47 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4759worker1 # [ 114.632268] nix-grpc-daemon[869]: time=2026-09-23T05:34:47.267Z level=INFO msg="Upload complete. (238ms)"4760lb # [ 114.697300] niks3-server[1016]: 2026/09/23 05:34:47 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4761lb # [ 114.703344] niks3-server[1016]: 2026/09/23 05:34:47 INFO Completed upload id=264762lb # [ 114.707018] niks3-server[1016]: 2026/09/23 05:34:47 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4763worker1 # [ 114.646219] nix-grpc-daemon[769]: ts=2026-09-23T05:34:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14764worker1 # [ 114.665640] nix-grpc-daemon[769]: ts=2026-09-23T05:34:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:507324765worker1 # [ 114.671211] nix-daemon[855]: reaped child process 1227, status = succeeded4766worker1 # [ 114.692533] nix-grpc-daemon[769]: ts=2026-09-23T05:34:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324767worker1 # [ 114.699244] nix-grpc-daemon[769]: ts=2026-09-23T05:34:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=2324768client # [ 114.836573] nix-daemon[1151]: reaped child process 1153, status = succeeded4769client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.46 seconds)4770client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4771client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4772client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4773client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4774client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4775client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4776(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.58 seconds)4777subtest: an input only one worker has still reaches the builder4778client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4779client # this derivation will be built:4780client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4781client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4782client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)4783client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4784client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.11 seconds)4785worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4786worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)4787worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4788worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4789client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24790client # [ 115.801444] nix-daemon[1151]: accepted connection from pid 1233, user root (trusted)4791client # this derivation will be built:4792client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4793client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4794worker2 # [ 115.952340] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:406184795worker2 # [ 115.958871] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406184796client # copying 1 paths...4797client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4798worker1 # [ 115.966776] nix-grpc-daemon[769]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324799worker1 # [ 115.980320] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4800worker1 # [ 116.157086] nix-daemon[855]: accepted connection from pid 1257, user nix-grpc-daemon (trusted)4801worker1 # [ 116.178078] nix-daemon[855]: reaped child process 1260, status = succeeded4802lb # [ 116.239574] niks3-server[1016]: 2026/09/23 05:34:48 INFO Received uploads request method=POST path=/api/pending_closures4803worker1 # [ 116.188274] nix-grpc-daemon[869]: time=2026-09-23T05:34:48.823Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4804worker1 # [ 116.189984] nix-grpc-daemon[869]: time=2026-09-23T05:34:48.823Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4805worker1 # [ 116.215743] nix-grpc-daemon[869]: time=2026-09-23T05:34:48.851Z level=INFO msg="Uploading 1 narinfos"4806lb # [ 116.269480] niks3-server[1016]: 2026/09/23 05:34:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4807lb # [ 116.285431] niks3-server[1016]: 2026/09/23 05:34:48 INFO Signed narinfos id=27 count=14808lb # [ 116.291787] niks3-server[1016]: 2026/09/23 05:34:48 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4809worker1 # [ 116.244488] nix-grpc-daemon[869]: time=2026-09-23T05:34:48.880Z level=INFO msg="Upload complete. (249ms)"4810lb # [ 116.306353] niks3-server[1016]: 2026/09/23 05:34:48 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4811lb # [ 116.310494] niks3-server[1016]: 2026/09/23 05:34:48 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4812lb # [ 116.315036] niks3-server[1016]: 2026/09/23 05:34:48 INFO Completed upload id=274813worker1 # [ 116.256144] nix-grpc-daemon[769]: ts=2026-09-23T05:34:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_in=1204814lb # [ 116.318558] niks3-server[1016]: 2026/09/23 05:34:48 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4815worker1 # [ 116.269586] nix-grpc-daemon[769]: ts=2026-09-23T05:34:48Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04816worker2 # [ 116.283301] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184817worker1 # [ 116.276800] nix-grpc-daemon[769]: ts=2026-09-23T05:34:48Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14818worker1 # [ 116.281669] nix-daemon[855]: reaped child process 1255, status = succeeded4819worker2 # [ 116.298253] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406184820client # copying 2 paths to 'lb:50051'...4821worker2 # [ 116.317193] nix-grpc-daemon[772]: ts=2026-09-23T05:34:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406184822client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4823client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4824worker2 # [ 116.334113] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4825worker2 # [ 116.498073] nix-daemon[844]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)4826worker2 # [ 116.515495] nix-daemon[844]: reaped child process 1079, status = succeeded4827lb # [ 116.575171] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received uploads request method=POST path=/api/pending_closures4828lb # [ 116.584165] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received uploads request method=POST path=/api/pending_closures4829worker2 # [ 116.542726] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.168Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4830worker2 # [ 116.545605] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.168Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4831worker2 # [ 116.548211] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.168Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4832lb # [ 116.616597] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4833lb # [ 116.636839] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4834lb # [ 116.648279] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4835worker2 # [ 116.602946] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.230Z level=INFO msg="Uploading 3 narinfos"4836lb # [ 116.657324] niks3-server[1016]: 2026/09/23 05:34:49 INFO Signed narinfos id=29 count=24837lb # [ 116.661879] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4838lb # [ 116.668035] niks3-server[1016]: 2026/09/23 05:34:49 INFO Signed narinfos id=28 count=14839lb # [ 116.669742] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4840lb # [ 116.677560] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4841lb # [ 116.690491] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4842lb # [ 116.698214] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4843lb # [ 116.701622] niks3-server[1016]: 2026/09/23 05:34:49 INFO Completed upload id=284844lb # [ 116.703468] niks3-server[1016]: 2026/09/23 05:34:49 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4845lb # [ 116.708382] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4846worker2 # [ 116.656412] nix-grpc-daemon[869]: time=2026-09-23T05:34:49.284Z level=INFO msg="Upload complete. (304ms)"4847lb # [ 116.710732] niks3-server[1016]: 2026/09/23 05:34:49 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4848lb # [ 116.713240] niks3-server[1016]: 2026/09/23 05:34:49 INFO Completed upload id=294849worker2 # [ 116.667344] nix-grpc-daemon[772]: ts=2026-09-23T05:34:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=2 nar_bytes_in=9844850worker2 # [ 116.680688] nix-grpc-daemon[772]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184851worker2 # [ 116.696195] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4852worker2 # [ 116.703686] nix-daemon[844]: reaped child process 1074, status = succeeded4853worker2 # [ 116.720953] nix-grpc-daemon[772]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4854client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4855client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4856worker2 # [ 116.741542] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4857client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4858worker2 # [ 117.846019] nix-daemon[844]: reaped child process 1084, status = succeeded4859worker2 # [ 117.916144] nix-daemon[844]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)4860worker2 # [ 117.933306] nix-daemon[844]: reaped child process 1103, status = succeeded4861lb # [ 117.991886] niks3-server[1016]: 2026/09/23 05:34:50 INFO Received uploads request method=POST path=/api/pending_closures4862worker2 # [ 117.950511] nix-grpc-daemon[869]: time=2026-09-23T05:34:50.577Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4863lb # [ 118.019955] niks3-server[1016]: 2026/09/23 05:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign4864worker2 # [ 117.979383] nix-grpc-daemon[869]: time=2026-09-23T05:34:50.607Z level=INFO msg="Uploading 1 narinfos"4865lb # [ 118.035345] niks3-server[1016]: 2026/09/23 05:34:50 INFO Signed narinfos id=30 count=14866lb # [ 118.043234] niks3-server[1016]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4867lb # [ 118.054593] niks3-server[1016]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4868worker2 # [ 118.008139] nix-grpc-daemon[869]: time=2026-09-23T05:34:50.636Z level=INFO msg="Upload complete. (253ms)"4869lb # [ 118.060840] niks3-server[1016]: 2026/09/23 05:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete4870lb # [ 118.070292] niks3-server[1016]: 2026/09/23 05:34:50 INFO Completed upload id=304871lb # [ 118.073562] niks3-server[1016]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4872worker2 # [ 118.019968] nix-grpc-daemon[772]: ts=2026-09-23T05:34:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=14873worker2 # [ 118.033629] nix-daemon[844]: reaped child process 1081, status = succeeded4874client # copying 1 paths...4875worker2 # [ 118.066243] nix-grpc-daemon[772]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406184876client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4877worker1 # [ 118.091245] nix-grpc-daemon[769]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324878worker1 # [ 118.099243] nix-grpc-daemon[769]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4879worker1 # [ 118.115264] nix-grpc-daemon[769]: ts=2026-09-23T05:34:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=1204880client # [ 118.248509] nix-daemon[1151]: reaped child process 1236, status = succeeded4881client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.65 seconds)4882client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24883client # [ 118.422474] nix-daemon[1151]: accepted connection from pid 1255, user root (trusted)4884client # this derivation will be built:4885client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4886worker1 # [ 118.560678] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:507324887client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4888worker2 # [ 118.577258] nix-grpc-daemon[772]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406184889worker2 # [ 118.586810] nix-grpc-daemon[869]: time=2026-09-23T05:34:51.215Z level=INFO msg="All 1 paths already cached"4890worker1 # [ 118.590172] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04891worker1 # [ 118.601083] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14892worker1 # [ 118.610369] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324893worker1 # [ 118.618806] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324894client # copying 1 paths to 'lb:50051'...4895client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4896worker1 # [ 118.632775] nix-grpc-daemon[769]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507324897worker1 # [ 118.647663] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4898worker1 # [ 118.667251] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5)4899worker1 # [ 118.737592] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 2/5)4900worker1 # [ 118.906440] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 210 ms (attempt 3/5)4901worker1 # [ 119.119495] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 370 ms (attempt 4/5)4902lb # [ 119.389797] envoy[907]: conn peer=[2001:db8:1::1]:46230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=333 ms=714903lb # [ 119.394681] envoy[907]: conn peer=[2001:db8:1::1]:46222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3420 tx=1943 ms=16674904lb # [ 119.399531] envoy[907]: conn peer=[2001:db8:1::1]:46234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3913 tx=2155 ms=18574905lb # [ 119.404395] envoy[907]: conn peer=[2001:db8:1::1]:55264 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=914906lb # [ 119.408955] envoy[907]: conn peer=[2001:db8:1::1]:55276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=684907lb # [ 119.413534] envoy[907]: conn peer=[2001:db8:1::1]:55266 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4337 tx=2390 ms=19804908lb # [ 119.418367] envoy[907]: conn peer=[2001:db8:1::1]:55286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4278 tx=2076 ms=22334909lb # [ 119.423174] envoy[907]: conn peer=[2001:db8:1::1]:55300 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=904910worker1 # [ 119.491157] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4911worker1 # [ 119.497359] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4912worker1 # [ 119.505439] nix-grpc-daemon[769]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4913worker1 # [ 119.682017] nix-daemon[855]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)4914worker1 # [ 119.702963] nix-daemon[855]: reaped child process 1277, status = succeeded4915lb # [ 119.769856] niks3-server[1016]: 2026/09/23 05:34:52 INFO Received uploads request method=POST path=/api/pending_closures4916worker1 # [ 119.717269] nix-grpc-daemon[869]: time=2026-09-23T05:34:52.352Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4917worker1 # [ 119.719523] nix-grpc-daemon[869]: time=2026-09-23T05:34:52.352Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4918lb # [ 119.800413] niks3-server[1016]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4919lb # [ 119.812811] niks3-server[1016]: 2026/09/23 05:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign4920worker1 # [ 119.757025] nix-grpc-daemon[869]: time=2026-09-23T05:34:52.392Z level=INFO msg="Uploading 1 narinfos"4921lb # [ 119.825185] niks3-server[1016]: 2026/09/23 05:34:52 INFO Signed narinfos id=31 count=14922lb # [ 119.831542] niks3-server[1016]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4923lb # [ 119.846799] niks3-server[1016]: 2026/09/23 05:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete4924worker1 # [ 119.784467] nix-grpc-daemon[869]: time=2026-09-23T05:34:52.420Z level=INFO msg="Upload complete. (259ms)"4925lb # [ 119.850462] niks3-server[1016]: 2026/09/23 05:34:52 INFO Completed upload id=314926lb # [ 119.853407] niks3-server[1016]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4927worker1 # [ 119.796668] nix-grpc-daemon[769]: ts=2026-09-23T05:34:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 paths=1 nar_bytes_in=5684928worker1 # [ 119.808737] nix-grpc-daemon[769]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507324929worker1 # [ 119.822918] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4930worker1 # [ 119.828572] nix-daemon[855]: reaped child process 1267, status = succeeded4931worker1 # [ 119.840545] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4932client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4933client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4934client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4935worker1 # [ 120.938401] nix-daemon[855]: reaped child process 1281, status = succeeded4936worker1 # [ 121.003954] nix-daemon[855]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)4937worker1 # [ 121.024380] nix-daemon[855]: reaped child process 1300, status = succeeded4938lb # [ 121.085928] niks3-server[1016]: 2026/09/23 05:34:53 INFO Received uploads request method=POST path=/api/pending_closures4939worker1 # [ 121.034574] nix-grpc-daemon[869]: time=2026-09-23T05:34:53.668Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4940lb # [ 121.111928] niks3-server[1016]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4941lb # [ 121.127220] niks3-server[1016]: 2026/09/23 05:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign4942worker1 # [ 121.073674] nix-grpc-daemon[869]: time=2026-09-23T05:34:53.709Z level=INFO msg="Uploading 1 narinfos"4943lb # [ 121.141948] niks3-server[1016]: 2026/09/23 05:34:53 INFO Signed narinfos id=32 count=14944lb # [ 121.147635] niks3-server[1016]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4945worker1 # [ 121.099441] nix-grpc-daemon[869]: time=2026-09-23T05:34:53.735Z level=INFO msg="Upload complete. (246ms)"4946lb # [ 121.163062] niks3-server[1016]: 2026/09/23 05:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete4947lb # [ 121.171552] niks3-server[1016]: 2026/09/23 05:34:53 INFO Completed upload id=324948worker1 # [ 121.109755] nix-grpc-daemon[769]: ts=2026-09-23T05:34:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=14949lb # [ 121.176191] niks3-server[1016]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4950worker1 # [ 121.124336] nix-daemon[855]: reaped child process 1279, status = succeeded4951client # copying 1 paths...4952worker1 # [ 121.129861] nix-grpc-daemon[769]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:507324953client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4954worker1 # [ 121.159625] nix-grpc-daemon[769]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:507324955worker1 # [ 121.166448] nix-grpc-daemon[769]: ts=2026-09-23T05:34:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50732 duration_s=0 paths=1 nar_bytes_out=1204956client # [ 121.302505] nix-daemon[1151]: reaped child process 1258, status = succeeded4957client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.05 seconds)4958client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24959client # [ 121.473973] nix-daemon[1151]: accepted connection from pid 1277, user root (trusted)4960client # this derivation will be built:4961client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4962worker1 # [ 121.613021] nix-grpc-daemon[769]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:507324963client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4964worker1 # [ 121.620469] nix-grpc-daemon[769]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507324965worker1 # [ 121.627586] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.261Z level=INFO msg="All 1 paths already cached"4966worker1 # [ 121.638952] nix-grpc-daemon[769]: ts=2026-09-23T05:34:54Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04967worker2 # [ 121.653089] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184968worker1 # [ 121.645625] nix-grpc-daemon[769]: ts=2026-09-23T05:34:54Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14969worker2 # [ 121.668453] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406184970client # copying 1 paths to 'lb:50051'...4971worker2 # [ 121.685101] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.310Z level=INFO msg="All 1 paths already cached"4972client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4973worker2 # [ 121.690425] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406184974worker2 # [ 121.705773] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4975worker2 # [ 121.864195] nix-daemon[844]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4976worker2 # [ 121.882550] nix-daemon[844]: reaped child process 1110, status = succeeded4977lb # [ 121.940792] niks3-server[1016]: 2026/09/23 05:34:54 INFO Received uploads request method=POST path=/api/pending_closures4978worker2 # [ 121.897641] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.525Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4979worker2 # [ 121.899111] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.525Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4980lb # [ 121.970594] niks3-server[1016]: 2026/09/23 05:34:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign4981worker2 # [ 121.925089] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.553Z level=INFO msg="Uploading 1 narinfos"4982lb # [ 121.985991] niks3-server[1016]: 2026/09/23 05:34:54 INFO Signed narinfos id=33 count=14983lb # [ 121.993881] niks3-server[1016]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4984worker2 # [ 121.957958] nix-grpc-daemon[869]: time=2026-09-23T05:34:54.586Z level=INFO msg="Upload complete. (233ms)"4985lb # [ 122.010969] niks3-server[1016]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4986lb # [ 122.016223] niks3-server[1016]: 2026/09/23 05:34:54 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete4987worker2 # [ 121.968272] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_in=5684988lb # [ 122.022428] niks3-server[1016]: 2026/09/23 05:34:54 INFO Completed upload id=334989lb # [ 122.028701] niks3-server[1016]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4990worker2 # [ 121.982981] nix-grpc-daemon[772]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406184991worker2 # [ 121.989773] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4992worker2 # [ 121.994609] nix-daemon[844]: reaped child process 1105, status = succeeded4993client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4994worker2 # [ 122.007789] nix-daemon[844]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4995client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4996client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4997worker2 # [ 123.316890] nix-daemon[844]: reaped child process 1114, status = succeeded4998worker2 # [ 123.385602] nix-daemon[844]: accepted connection from pid 1130, user nix-grpc-daemon (trusted)4999worker2 # [ 123.403143] nix-daemon[844]: reaped child process 1133, status = succeeded5000lb # [ 123.467087] niks3-server[1016]: 2026/09/23 05:34:56 INFO Received uploads request method=POST path=/api/pending_closures5001worker2 # [ 123.423460] nix-grpc-daemon[869]: time=2026-09-23T05:34:56.050Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5002lb # [ 123.494259] niks3-server[1016]: 2026/09/23 05:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5003worker2 # [ 123.448276] nix-grpc-daemon[869]: time=2026-09-23T05:34:56.076Z level=INFO msg="Uploading 1 narinfos"5004lb # [ 123.508213] niks3-server[1016]: 2026/09/23 05:34:56 INFO Signed narinfos id=34 count=15005lb # [ 123.516141] niks3-server[1016]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5006lb # [ 123.525506] niks3-server[1016]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5007worker2 # [ 123.483515] nix-grpc-daemon[869]: time=2026-09-23T05:34:56.111Z level=INFO msg="Upload complete. (264ms)"5008lb # [ 123.535940] niks3-server[1016]: 2026/09/23 05:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5009lb # [ 123.545019] niks3-server[1016]: 2026/09/23 05:34:56 INFO Completed upload id=345010client # copying 1 paths...5011lb # [ 123.548952] niks3-server[1016]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5012worker2 # [ 123.497005] nix-grpc-daemon[772]: ts=2026-09-23T05:34:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15013worker2 # [ 123.512411] nix-grpc-daemon[772]: ts=2026-09-23T05:34:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406185014worker2 # [ 123.518265] nix-daemon[844]: reaped child process 1112, status = succeeded5015client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5016worker2 # [ 123.544852] nix-grpc-daemon[772]: ts=2026-09-23T05:34:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406185017worker2 # [ 123.551477] nix-grpc-daemon[772]: ts=2026-09-23T05:34:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40618 duration_s=0 paths=1 nar_bytes_out=1205018client # [ 123.673447] nix-daemon[1151]: reaped child process 1280, status = succeeded5019client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.37 seconds)5020(finished: subtest: an input only one worker has still reaches the builder, in 8.71 seconds)5021subtest: upload whose reference the worker lacks is completed from the cache5022client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5023client # this derivation will be built:5024client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5025client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5026client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5027client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5028client # this derivation will be built:5029client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5030client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5031client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)5032client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5033client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5034worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5035worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)5036worker2 # [ 124.665818] systemd[1]: Stopping Nix worker-protocol over gRPC...5037worker2 # [ 124.700961] nix-grpc-daemon[772]: ts=2026-09-23T05:34:57Z level=info event=signal_exit5038worker2 # [ 125.720295] nix-grpc-daemon[772]: ts=2026-09-23T05:34:58Z level=info event=scheduler_restarting5039worker1 # [ 125.755338] nix-grpc-daemon[769]: ts=2026-09-23T05:34:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:455205040worker1 # [ 125.763154] nix-grpc-daemon[769]: ts=2026-09-23T05:34:58Z level=info event=worker_gone addr=192.168.1.4:500515041worker2 # [ 125.791151] nix-grpc-daemon[772]: ts=2026-09-23T05:34:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5042worker2 # [ 125.811290] nix-daemon[844]: reaped child process 846, status = succeeded5043worker2 # [ 125.850481] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5044worker2 # [ 125.856705] systemd[1]: Stopped Nix worker-protocol over gRPC.5045worker2 # [ 125.858998] systemd[1]: nix-grpc-daemon.service: Consumed 2.746s CPU time over 1min 46.387s wall clock time, 61.6M memory peak, 71.2K incoming IP traffic, 137.2K outgoing IP traffic.5046worker2 # [ 125.877045] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5047lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux5048lb: must succeed: curl -sf localhost:9901/clusters5049worker2 # [ 125.882820] systemd[1]: Closed Nix worker-protocol over gRPC.5050worker2 # [ 125.884523] systemd[1]: nix-grpc-daemon.socket: Consumed 539us CPU time over 1min 55.754s wall clock time, 256K memory peak, 91.7K incoming IP traffic, 80.7K outgoing IP traffic.5051lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5052lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.07 seconds)5053client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25054worker1 # [ 126.143514] nix-grpc-daemon[769]: ts=2026-09-23T05:34:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507325055worker1 # [ 126.152425] nix-grpc-daemon[769]: ts=2026-09-23T05:34:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:507325056client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5057client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.23 seconds)5058worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5059worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5060worker2 # [ 126.286235] systemd[1]: Listening on Nix worker-protocol over gRPC.5061worker1 # [ 126.346885] systemd[1]: Stopping Nix worker-protocol over gRPC...5062worker1 # [ 126.602866] nix-grpc-daemon[769]: ts=2026-09-23T05:34:59Z level=info event=signal_exit5063lb # [ 126.777334] niks3-server[1016]: 2026/09/23 05:34:59 INFO lead: released remote=192.168.1.3:522345064worker2 # [ 126.930658] systemd[1]: Starting scheduler-token.service...5065worker2 # [ 127.009844] systemd[1]: scheduler-token.service: Deactivated successfully.5066worker2 # [ 127.015891] systemd[1]: Finished scheduler-token.service.5067worker2 # [ 127.022931] systemd[1]: scheduler-token.service: Consumed 55ms CPU time over 83ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5068worker2 # [ 127.030244] systemd[1]: Starting Nix worker-protocol over gRPC...5069worker2 # [ 127.160595] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5070worker2 # [ 127.162478] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5071worker2 # [ 127.166293] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15072worker2 # [ 127.172057] nix-grpc-daemon[1157]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5073worker2 # [ 127.173498] nix-grpc-daemon[1157]: E0000 00:00:1790141699.799150 1157 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5074lb # [ 127.237781] niks3-server[1016]: 2026/09/23 05:34:59 INFO lead: acquired remote=192.168.1.4:522845075worker2 # [ 127.216100] nix-grpc-daemon[1157]: E0000 00:00:1790141699.805345 1157 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5076worker2 # [ 127.218363] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5077worker2 # [ 127.219948] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=scheduler_take_over5078worker2 # [ 127.224808] nix-grpc-daemon[1157]: E0000 00:00:1790141699.814636 1200 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5079worker2 # [ 127.227689] nix-grpc-daemon[1157]: E0000 00:00:1790141699.832012 1200 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5080worker2 # [ 127.230370] nix-grpc-daemon[1157]: E0000 00:00:1790141699.832144 1200 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5081worker2 # [ 127.235579] systemd[1]: Started Nix worker-protocol over gRPC.5082worker2 # [ 127.238400] nix-grpc-daemon[1157]: E0000 00:00:1790141699.852197 1200 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5083worker2 # [ 127.240955] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=scheduler_connected addr=lb:500515084worker2 # [ 127.243386] nix-grpc-daemon[1157]: ts=2026-09-23T05:34:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5085worker2 # [ 127.439597] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:500515086worker2 # [ 127.445061] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5087worker2 # [ 127.646193] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:500515088worker2 # [ 127.651612] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5089worker1 # [ 127.717828] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=scheduler_restarting5090lb # [ 127.778187] envoy[907]: [2026-09-23 05:35:00.406][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5091worker1 # [ 127.726922] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=scheduler_restarting addr=lb:500515092worker1 # [ 127.731306] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:455205093worker1 # [ 127.736333] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=worker_gone addr=192.168.1.3:500515094worker1 # [ 127.753163] nix-grpc-daemon[769]: E0000 00:00:1790141700.388595 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true5095worker1 # [ 127.772759] nix-daemon[855]: reaped child process 857, status = succeeded5096worker1 # [ 127.818737] nix-grpc-daemon[769]: E0000 00:00:1790141700.454253 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5097worker2 # [ 127.852786] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:500515098worker2 # [ 127.858812] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5099worker1 # [ 127.944891] nix-grpc-daemon[769]: ts=2026-09-23T05:35:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5100worker2 # [ 128.059677] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:500515101worker2 # [ 128.064918] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5102worker2 # [ 128.221980] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:595145103worker2 # [ 128.265817] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:500515104worker2 # [ 128.282525] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=oidc_jwks provider=mock keys=15105worker2 # [ 128.287303] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:595285106worker2 # [ 128.300200] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05107lb # [ 129.388867] envoy[907]: conn peer=[2001:db8:1::1]:55314 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3756 tx=2147 ms=26655108lb # [ 129.393905] envoy[907]: conn peer=[2001:db8:1::1]:57624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=695109lb # [ 129.399509] envoy[907]: conn peer=[2001:db8:1::1]:57638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3770 tx=2101 ms=19855110lb # [ 129.404869] envoy[907]: conn peer=[2001:db8:1::1]:57640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=685111lb # [ 129.409879] envoy[907]: conn peer=192.168.1.4:59178 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=17036 tx=1569 ms=905955112lb # [ 129.414851] envoy[907]: conn peer=[2001:db8:1::1]:57654 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=285113lb # [ 129.419621] envoy[907]: conn peer=[2001:db8:1::3]:43278 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5116 tx=1842 ms=922605114worker1 # [ 129.794373] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5115worker1 # [ 129.799125] systemd[1]: Stopped Nix worker-protocol over gRPC.5116worker1 # [ 129.807402] systemd[1]: nix-grpc-daemon.service: Consumed 3.944s CPU time over 1min 50.465s wall clock time, 71.8M memory peak, 106.6K incoming IP traffic, 214.8K outgoing IP traffic.5117worker1 # [ 129.820883] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5118worker1 # [ 129.827792] systemd[1]: Closed Nix worker-protocol over gRPC.5119worker1 # [ 129.830229] systemd[1]: nix-grpc-daemon.socket: Consumed 541us CPU time over 1min 59.836s wall clock time, 256K memory peak, 306.9K incoming IP traffic, 114.3K outgoing IP traffic.5120lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5121lb: must succeed: curl -sf localhost:9901/clusters5122lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5123lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.07 seconds)5124client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25125worker2 # [ 130.145585] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595465126worker2 # [ 130.158879] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5127worker2 # [ 130.185655] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5)5128worker2 # [ 130.206119] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)5129worker2 # [ 130.364095] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 131 ms (attempt 3/5)5130worker2 # [ 130.496958] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 691 ms (attempt 4/5)5131worker2 # [ 131.199802] nix-grpc-daemon[1157]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5132worker2 # [ 131.206359] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5133client # copying 2 paths...5134client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5135worker2 # [ 131.231058] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595465136client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5137worker2 # [ 131.249797] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5138worker2 # [ 131.282776] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:03Z level=info event=niks3_push_started pid=12185139worker2 # [ 131.429710] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.057Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5140worker2 # [ 131.539633] nix-daemon[844]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5141worker2 # [ 131.557417] nix-daemon[844]: reaped child process 1234, status = succeeded5142lb # [ 131.616585] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received uploads request method=POST path=/api/pending_closures5143lb # [ 131.625727] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received uploads request method=POST path=/api/pending_closures5144worker2 # [ 131.583184] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.211Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5145worker2 # [ 131.588613] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.211Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5146worker2 # [ 131.591003] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.211Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5147lb # [ 131.683202] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5148worker2 # [ 131.642687] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.270Z level=INFO msg="Uploading 3 narinfos"5149lb # [ 131.695394] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5150lb # [ 131.705418] niks3-server[1016]: 2026/09/23 05:35:04 INFO Signed narinfos id=35 count=15151lb # [ 131.712895] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5152lb # [ 131.715328] niks3-server[1016]: 2026/09/23 05:35:04 INFO Signed narinfos id=36 count=25153lb # [ 131.725160] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5154lb # [ 131.727676] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5155lb # [ 131.745717] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5156lb # [ 131.750765] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5157lb # [ 131.754395] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5158lb # [ 131.756342] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5159lb # [ 131.759390] niks3-server[1016]: 2026/09/23 05:35:04 INFO Completed upload id=355160lb # [ 131.761742] niks3-server[1016]: 2026/09/23 05:35:04 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5161lb # [ 131.763450] niks3-server[1016]: 2026/09/23 05:35:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5162lb # [ 131.766855] niks3-server[1016]: 2026/09/23 05:35:04 INFO Completed upload id=365163worker2 # [ 131.715187] nix-grpc-daemon[1218]: time=2026-09-23T05:35:04.343Z level=INFO msg="Upload complete. (281ms)"5164worker2 # [ 131.728620] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=0 paths=2 nar_bytes_in=3125165worker2 # [ 131.745335] nix-daemon[844]: reaped child process 1216, status = succeeded5166client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.83 seconds)5167worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5168worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5169worker1 # [ 131.855177] systemd[1]: Listening on Nix worker-protocol over gRPC.5170worker1 # [ 131.868406] systemd[1]: Starting Nix worker-protocol over gRPC...5171worker1 # [ 132.018499] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5172worker1 # [ 132.020840] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5173worker1 # [ 132.024970] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15174worker1 # [ 132.030509] nix-grpc-daemon[1325]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5175worker1 # [ 132.031955] nix-grpc-daemon[1325]: E0000 00:00:1790141704.665579 1325 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5176lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5177lb: must succeed: curl -sf localhost:9901/clusters5178worker1 # [ 132.068166] nix-grpc-daemon[1325]: E0000 00:00:1790141704.671872 1325 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5179worker1 # [ 132.070464] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5180worker1 # [ 132.081545] nix-grpc-daemon[1325]: E0000 00:00:1790141704.681660 1368 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5181worker1 # [ 132.084581] nix-grpc-daemon[1325]: E0000 00:00:1790141704.696703 1368 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5182worker1 # [ 132.086859] nix-grpc-daemon[1325]: E0000 00:00:1790141704.696846 1368 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5183worker1 # [ 132.093585] systemd[1]: Started Nix worker-protocol over gRPC.5184worker1 # [ 132.096295] nix-grpc-daemon[1325]: E0000 00:00:1790141704.716262 1368 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5185worker1 # [ 132.098810] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=scheduler_connected addr=lb:500515186worker2 # [ 132.109783] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:04Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:595285187worker2 # [ 132.115774] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05188lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5189lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.08 seconds)5190(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 8.44 seconds)5191subtest: a niks3 restart does not cost the leader its role5192lb: must succeed: curl -sf localhost:9901/clusters5193lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5194lb: must succeed: curl -sf localhost:9901/clusters5195lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5196worker1 # [ 132.241603] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:04Z level=info event=scheduler_lock_lost status=2005197lb # [ 132.307777] niks3-server[1016]: 2026/09/23 05:35:04 INFO Shutdown signal received, draining in-flight requests timeout=10s5198worker2 # [ 132.260921] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:04Z level=info event=scheduler_lock_lost status=2005199lb # [ 132.323457] niks3-server[1016]: 2026/09/23 05:35:04 INFO lead: released remote=192.168.1.4:522845200lb # [ 132.328931] systemd[1]: Stopping niks3 server...5201lb # [ 132.333530] systemd[1]: niks3.service: Deactivated successfully.5202lb # [ 132.334462] systemd[1]: Stopped niks3 server.5203lb # [ 132.335114] systemd[1]: niks3.service: Consumed 625ms CPU time over 1min 43.795s wall clock time, 27.7M memory peak, 92.8K incoming IP traffic, 135.8K outgoing IP traffic.5204lb # [ 132.344249] systemd[1]: Starting niks3 server...5205lb # [ 132.408457] niks3-server[1211]: 2026/09/23 05:35:05 goose: no migrations to run. current version: 202609200000005206lb # [ 132.414186] niks3-server[1211]: 2026/09/23 05:35:05 OK 1_commit_pending_closure.sql (6.51ms)5207lb # [ 132.417788] niks3-server[1211]: 2026/09/23 05:35:05 OK 2_object_stats_trigger.sql (3.25ms)5208lb # [ 132.418927] niks3-server[1211]: 2026/09/23 05:35:05 goose: up to current file version: 25209lb # [ 132.420560] niks3-server[1211]: 2026/09/23 05:35:05 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5210lb # [ 132.430927] niks3-server[1211]: 2026/09/23 05:35:05 INFO Read proxy enabled — serving cache objects from S35211lb # [ 132.436663] niks3-server[1211]: 2026/09/23 05:35:05 INFO Using socket-activated listener address=0.0.0.0:57515212lb # [ 132.437984] niks3-server[1211]: 2026/09/23 05:35:05 INFO systemd watchdog enabled interval=15s5213lb # [ 132.439075] niks3-server[1211]: 2026/09/23 05:35:05 INFO Starting HTTP server address=0.0.0.0:57515214lb # [ 132.440466] systemd[1]: Started niks3 server.5215lb: waiting for unit niks3.service5216lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5217lb: must succeed: curl -sf localhost:9901/clusters5218lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5219worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5220worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5221lb: must succeed: curl -sf localhost:9901/clusters5222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5223lb: must succeed: curl -sf localhost:9901/clusters5224lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5225worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5226worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5227client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25228worker2 # [ 132.861065] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:595465229client # don't know how to build these paths:5230client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5231worker2 # [ 132.872739] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595465232worker2 # [ 132.902945] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05233worker2 # [ 132.910299] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15234worker2 # [ 132.948750] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595465235worker2 # [ 132.964317] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595465236client # copying 1 paths to 'lb:50051'...5237client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5238worker2 # [ 132.980105] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595465239worker2 # [ 132.994838] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5240worker2 # [ 133.157213] nix-daemon[844]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5241worker2 # [ 133.177061] nix-daemon[844]: reaped child process 1254, status = succeeded5242lb # [ 133.232328] niks3-server[1211]: 2026/09/23 05:35:05 INFO Received uploads request method=POST path=/api/pending_closures5243worker2 # [ 133.194717] nix-grpc-daemon[1218]: time=2026-09-23T05:35:05.822Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5244worker2 # [ 133.196668] nix-grpc-daemon[1218]: time=2026-09-23T05:35:05.822Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5245lb # [ 133.274270] niks3-server[1211]: 2026/09/23 05:35:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5246lb # [ 133.304660] niks3-server[1211]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5247lb # [ 133.320315] niks3-server[1211]: 2026/09/23 05:35:05 INFO lead: acquired remote=192.168.1.4:394285248lb # [ 133.331423] niks3-server[1211]: 2026/09/23 05:35:05 INFO Signed narinfos id=37 count=15249worker2 # [ 133.283459] nix-grpc-daemon[1218]: time=2026-09-23T05:35:05.910Z level=INFO msg="Uploading 1 narinfos"5250lb # [ 133.347556] niks3-server[1211]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5251lb # [ 133.358795] niks3-server[1211]: 2026/09/23 05:35:05 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5252lb # [ 133.367484] niks3-server[1211]: 2026/09/23 05:35:05 INFO Completed upload id=375253worker2 # [ 133.315116] nix-grpc-daemon[1218]: time=2026-09-23T05:35:05.943Z level=INFO msg="Upload complete. (303ms)"5254lb # [ 133.374594] niks3-server[1211]: 2026/09/23 05:35:06 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5255worker2 # [ 133.329337] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=0 paths=1 nar_bytes_in=5445256worker2 # [ 133.346995] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595465257worker2 # [ 133.357704] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5258worker2 # [ 133.367176] nix-daemon[844]: reaped child process 1249, status = succeeded5259client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5260client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5261worker2 # [ 133.374839] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5262client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5263client # farm-a> LOG-farm-a-after-niks3-restart5264worker2 # [ 134.828428] nix-daemon[844]: reaped child process 1260, status = succeeded5265worker2 # [ 134.923419] nix-daemon[844]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5266worker2 # [ 134.939436] nix-daemon[844]: reaped child process 1279, status = succeeded5267lb # [ 134.999186] niks3-server[1211]: 2026/09/23 05:35:07 INFO Received uploads request method=POST path=/api/pending_closures5268worker2 # [ 134.960384] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.588Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5269worker2 # [ 134.961942] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.588Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5270lb # [ 135.050618] niks3-server[1211]: 2026/09/23 05:35:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5271worker2 # [ 135.008112] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.636Z level=INFO msg="Uploading 1 narinfos"5272lb # [ 135.063517] niks3-server[1211]: 2026/09/23 05:35:07 INFO Signed narinfos id=38 count=15273lb # [ 135.073716] niks3-server[1211]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5274lb # [ 135.082043] niks3-server[1211]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5275lb # [ 135.097644] niks3-server[1211]: 2026/09/23 05:35:07 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5276lb # [ 135.104734] niks3-server[1211]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5277lb # [ 135.111355] niks3-server[1211]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5278lb # [ 135.123508] niks3-server[1211]: 2026/09/23 05:35:07 INFO Completed upload id=385279worker2 # [ 135.072060] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.700Z level=INFO msg="Upload complete. (286ms)"5280worker2 # [ 135.093314] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15281worker2 # [ 135.112248] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05282worker2 # [ 135.116691] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15283worker2 # [ 135.120940] nix-daemon[844]: reaped child process 1258, status = succeeded5284worker2 # [ 135.136417] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595465285worker2 # [ 135.151466] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595465286worker2 # [ 135.166864] nix-grpc-daemon[1218]: time=2026-09-23T05:35:07.795Z level=INFO msg="All 1 paths already cached"5287client # copying 1 paths to 'lb:50051'...5288client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5289worker2 # [ 135.177759] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595465290worker2 # [ 135.192078] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5291worker2 # [ 135.344789] nix-daemon[844]: accepted connection from pid 1285, user nix-grpc-daemon (trusted)5292worker2 # [ 135.364869] nix-daemon[844]: reaped child process 1288, status = succeeded5293lb # [ 135.415196] niks3-server[1211]: 2026/09/23 05:35:08 INFO Received uploads request method=POST path=/api/pending_closures5294worker2 # [ 135.375932] nix-grpc-daemon[1218]: time=2026-09-23T05:35:08.003Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5295worker2 # [ 135.377893] nix-grpc-daemon[1218]: time=2026-09-23T05:35:08.003Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5296lb # [ 135.449713] niks3-server[1211]: 2026/09/23 05:35:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5297worker2 # [ 135.408822] nix-grpc-daemon[1218]: time=2026-09-23T05:35:08.036Z level=INFO msg="Uploading 1 narinfos"5298lb # [ 135.468065] niks3-server[1211]: 2026/09/23 05:35:08 INFO Signed narinfos id=39 count=15299lb # [ 135.471032] niks3-server[1211]: 2026/09/23 05:35:08 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5300worker2 # [ 135.432733] nix-grpc-daemon[1218]: time=2026-09-23T05:35:08.061Z level=INFO msg="Upload complete. (225ms)"5301lb # [ 135.486753] niks3-server[1211]: 2026/09/23 05:35:08 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5302lb # [ 135.493568] niks3-server[1211]: 2026/09/23 05:35:08 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5303lb # [ 135.496814] niks3-server[1211]: 2026/09/23 05:35:08 INFO Completed upload id=395304worker2 # [ 135.441956] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=0 paths=1 nar_bytes_in=6565305lb # [ 135.498956] niks3-server[1211]: 2026/09/23 05:35:08 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5306worker2 # [ 135.454957] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595465307worker2 # [ 135.460618] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5308worker2 # [ 135.465195] nix-daemon[844]: reaped child process 1283, status = succeeded5309client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5310worker2 # [ 135.477799] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5311client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5312client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5313client # farm-b> LOG-farm-b-after-niks3-restart5314worker2 # [ 136.309032] nix-daemon[844]: reaped child process 1292, status = succeeded5315worker2 # [ 136.404152] nix-daemon[844]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5316worker2 # [ 136.422879] nix-daemon[844]: reaped child process 1311, status = succeeded5317lb # [ 136.482742] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received uploads request method=POST path=/api/pending_closures5318worker2 # [ 136.441367] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.068Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5319worker2 # [ 136.443011] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.068Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5320lb # [ 136.525379] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5321worker2 # [ 136.482121] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.110Z level=INFO msg="Uploading 1 narinfos"5322lb # [ 136.539816] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5323lb # [ 136.548226] niks3-server[1211]: 2026/09/23 05:35:09 INFO Signed narinfos id=40 count=15324lb # [ 136.550455] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5325lb # [ 136.563323] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5326lb # [ 136.568387] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5327worker2 # [ 136.516215] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.144Z level=INFO msg="Upload complete. (229ms)"5328lb # [ 136.571011] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5329lb # [ 136.574945] niks3-server[1211]: 2026/09/23 05:35:09 INFO Completed upload id=405330worker2 # [ 136.532991] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15331worker2 # [ 136.546771] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05332worker2 # [ 136.550441] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15333worker2 # [ 136.554206] nix-daemon[844]: reaped child process 1290, status = succeeded5334worker2 # [ 136.576685] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595465335worker2 # [ 136.590910] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:595465336worker2 # [ 136.605824] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.234Z level=INFO msg="All 2 paths already cached"5337client # copying 1 paths to 'lb:50051'...5338client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5339worker2 # [ 136.616780] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:595465340worker2 # [ 136.634266] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5341worker2 # [ 136.777830] nix-daemon[844]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5342worker2 # [ 136.796202] nix-daemon[844]: reaped child process 1320, status = succeeded5343lb # [ 136.854318] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received uploads request method=POST path=/api/pending_closures5344worker2 # [ 136.814844] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.442Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5345worker2 # [ 136.817026] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.442Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5346lb # [ 136.885460] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5347worker2 # [ 136.848229] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.476Z level=INFO msg="Uploading 1 narinfos"5348lb # [ 136.899926] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5349lb # [ 136.910848] niks3-server[1211]: 2026/09/23 05:35:09 INFO Signed narinfos id=41 count=15350lb # [ 136.917490] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5351lb # [ 136.930288] niks3-server[1211]: 2026/09/23 05:35:09 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5352worker2 # [ 136.884204] nix-grpc-daemon[1218]: time=2026-09-23T05:35:09.512Z level=INFO msg="Upload complete. (238ms)"5353lb # [ 136.938147] niks3-server[1211]: 2026/09/23 05:35:09 INFO Completed upload id=415354lb # [ 136.941025] niks3-server[1211]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5355worker2 # [ 136.897218] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=0 paths=1 nar_bytes_in=7845356worker2 # [ 136.912594] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:595465357worker2 # [ 136.920380] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5358worker2 # [ 136.925733] nix-daemon[844]: reaped child process 1315, status = succeeded5359client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5360client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5361worker2 # [ 136.939466] nix-daemon[844]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5362client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5363client # farm-top> LOG-farm-top-after-niks3-restart5364worker2 # [ 138.212412] nix-daemon[844]: reaped child process 1324, status = succeeded5365worker2 # [ 138.311225] nix-daemon[844]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5366worker2 # [ 138.328996] nix-daemon[844]: reaped child process 1343, status = succeeded5367lb # [ 138.390393] niks3-server[1211]: 2026/09/23 05:35:11 INFO Received uploads request method=POST path=/api/pending_closures5368worker2 # [ 138.348928] nix-grpc-daemon[1218]: time=2026-09-23T05:35:10.976Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5369worker2 # [ 138.350554] nix-grpc-daemon[1218]: time=2026-09-23T05:35:10.976Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5370lb # [ 138.431713] niks3-server[1211]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5371worker2 # [ 138.393587] nix-grpc-daemon[1218]: time=2026-09-23T05:35:11.021Z level=INFO msg="Uploading 1 narinfos"5372lb # [ 138.449311] niks3-server[1211]: 2026/09/23 05:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5373lb # [ 138.459090] niks3-server[1211]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5374lb # [ 138.471290] niks3-server[1211]: 2026/09/23 05:35:11 INFO Signed narinfos id=42 count=15375worker2 # [ 138.424173] nix-grpc-daemon[1218]: time=2026-09-23T05:35:11.052Z level=INFO msg="Upload complete. (273ms)"5376lb # [ 138.477724] niks3-server[1211]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5377lb # [ 138.484738] niks3-server[1211]: 2026/09/23 05:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5378lb # [ 138.488569] niks3-server[1211]: 2026/09/23 05:35:11 INFO Completed upload id=425379lb # [ 138.491987] niks3-server[1211]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5380worker2 # [ 138.437983] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59546 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15381worker2 # [ 138.456731] nix-daemon[844]: reaped child process 1322, status = succeeded5382client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.86 seconds)5383client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5384client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5385(finished: subtest: a niks3 restart does not cost the leader its role, in 6.54 seconds)5386subtest: scheduler down: the other node takes the lock and keeps it5387lb: must succeed: curl -sf localhost:9901/clusters5388lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5389lb: must succeed: curl -sf localhost:9901/clusters5390lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5391lb: must succeed: curl -sf localhost:9901/clusters5392lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5393lb: must succeed: curl -sf localhost:9901/clusters5394lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5395worker2 # [ 138.912679] systemd[1]: Stopping Nix worker-protocol over gRPC...5396worker2 # [ 139.187926] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:11Z level=info event=signal_exit5397lb # [ 139.324253] niks3-server[1211]: 2026/09/23 05:35:11 INFO lead: released remote=192.168.1.4:394285398lb # [ 139.389689] envoy[907]: conn peer=[2001:db8:1::1]:57656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1004 tx=305 ms=15985399lb # [ 139.395222] envoy[907]: conn peer=[2001:db8:1::1]:48604 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9710 tx=4426 ms=56045400worker2 # [ 140.273296] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=scheduler_restarting5401lb # [ 140.323137] envoy[907]: [2026-09-23 05:35:12.954][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5402worker2 # [ 140.283241] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:595285403worker1 # [ 140.273790] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:12Z level=info event=scheduler_restarting addr=lb:500515404worker2 # [ 140.292974] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=worker_gone addr=192.168.1.3:500515405worker2 # [ 140.299464] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=scheduler_restarting addr=lb:500515406worker2 # [ 140.304153] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:595285407worker2 # [ 140.309456] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:12Z level=info event=worker_gone addr=192.168.1.4:500515408worker2 # [ 140.332529] nix-daemon[844]: reaped child process 1210, status = succeeded5409worker2 # [ 140.351437] nix-grpc-daemon[1157]: E0000 00:00:1790141712.979607 1203 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5410worker1 # [ 140.473397] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:500515411worker1 # [ 140.478729] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5412worker2 # [ 140.511502] nix-grpc-daemon[1157]: ts=2026-09-23T05:35:13Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5413worker1 # [ 140.679387] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:500515414worker1 # [ 140.684822] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5415worker1 # [ 140.885443] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:500515416worker1 # [ 140.890587] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5417worker1 # [ 141.091514] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:500515418worker1 # [ 141.096828] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5419worker2 # [ 141.234099] nix-grpc-daemon[1157]: E0000 00:00:1790141713.862201 1203 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5420worker1 # [ 141.299260] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_connected addr=lb:500515421worker1 # [ 141.303530] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5422worker1 # [ 141.500066] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:500515423worker1 # [ 141.505396] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5424worker1 # [ 141.706135] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:500515425worker1 # [ 141.711456] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5426worker1 # [ 141.912329] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:500515427worker1 # [ 141.917506] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5428worker1 # [ 142.118193] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:500515429worker1 # [ 142.123407] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5430worker1 # [ 142.325965] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_connected addr=lb:500515431worker1 # [ 142.330189] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5432worker1 # [ 142.526585] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:500515433worker1 # [ 142.531991] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5434worker2 # [ 142.546298] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5435worker2 # [ 142.547958] systemd[1]: Stopped Nix worker-protocol over gRPC.5436worker2 # [ 142.556511] systemd[1]: nix-grpc-daemon.service: Consumed 1.348s CPU time over 15.517s wall clock time, 39.5M memory peak, 47.5K incoming IP traffic, 83.3K outgoing IP traffic.5437worker2 # [ 142.571332] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5438worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5439worker2 # [ 142.585225] systemd[1]: Closed Nix worker-protocol over gRPC.5440worker2 # [ 142.586869] systemd[1]: nix-grpc-daemon.socket: Consumed 844us CPU time over 16.301s wall clock time, 256K memory peak, 270.7K incoming IP traffic, 31.6K outgoing IP traffic.5441worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5442worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5443worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5444worker1 # [ 142.732683] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:500515445worker1 # [ 142.737884] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5446worker1 # [ 142.938506] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:500515447worker1 # [ 142.943646] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5448worker1 # [ 143.144277] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:500515449worker1 # [ 143.149254] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5450lb # [ 143.312853] niks3-server[1211]: 2026/09/23 05:35:15 INFO lead: acquired remote=192.168.1.3:359645451worker1 # [ 143.251708] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_take_over5452worker1 # [ 143.351466] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_connected addr=lb:500515453worker1 # [ 143.355632] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5454worker1 # [ 143.552122] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_connected addr=lb:500515455worker1 # [ 143.557208] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5456worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5457worker1 # [ 143.758257] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_connected addr=lb:500515458worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5459worker1 # [ 143.765498] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5460worker1 # [ 143.964906] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=scheduler_connected addr=lb:500515461worker1 # [ 143.974138] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:504425462worker1 # [ 143.979441] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05463worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5464worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5465client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25466worker1 # [ 145.050418] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504585467worker1 # [ 145.063767] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5468client # don't know how to build these paths:5469client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5470worker1 # [ 145.085510] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504585471worker1 # [ 145.106929] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 1/5)5472worker1 # [ 145.213998] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 2/5)5473worker1 # [ 145.225883] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 371 ms (attempt 3/5)5474worker1 # [ 145.606896] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 4/5)5475worker1 # [ 145.727795] nix-grpc-daemon[1325]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5476worker1 # [ 145.734862] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5477worker1 # [ 145.765917] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05478worker1 # [ 145.773144] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15479worker1 # [ 145.811382] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504585480worker1 # [ 145.827986] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504585481client # copying 1 paths to 'lb:50051'...5482client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5483worker1 # [ 145.842714] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504585484worker1 # [ 145.857600] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5485worker1 # [ 145.891994] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=info event=niks3_push_started pid=14055486worker1 # [ 146.037723] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.672Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5487worker1 # [ 146.147534] nix-daemon[855]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5488worker1 # [ 146.166099] nix-daemon[855]: reaped child process 1421, status = succeeded5489lb # [ 146.237873] niks3-server[1211]: 2026/09/23 05:35:18 INFO Received uploads request method=POST path=/api/pending_closures5490worker1 # [ 146.188069] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.822Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5491worker1 # [ 146.189589] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.822Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5492worker1 # [ 146.224094] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.858Z level=INFO msg="Uploading 1 narinfos"5493lb # [ 146.277354] niks3-server[1211]: 2026/09/23 05:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5494lb # [ 146.288811] niks3-server[1211]: 2026/09/23 05:35:18 INFO Signed narinfos id=43 count=15495lb # [ 146.298717] niks3-server[1211]: 2026/09/23 05:35:18 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5496lb # [ 146.312897] niks3-server[1211]: 2026/09/23 05:35:18 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5497lb # [ 146.325578] niks3-server[1211]: 2026/09/23 05:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5498worker1 # [ 146.271456] nix-grpc-daemon[1405]: time=2026-09-23T05:35:18.907Z level=INFO msg="Upload complete. (229ms)"5499lb # [ 146.335709] niks3-server[1211]: 2026/09/23 05:35:18 INFO Completed upload id=435500lb # [ 146.345284] niks3-server[1211]: 2026/09/23 05:35:18 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5501worker1 # [ 146.281086] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 paths=1 nar_bytes_in=5205502worker1 # [ 146.296267] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504585503worker1 # [ 146.305411] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5504worker1 # [ 146.310801] nix-daemon[855]: reaped child process 1403, status = succeeded5505client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5506worker1 # [ 146.323699] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5507client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5508client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5509client # farm-a> LOG-farm-a-on-standby5510worker1 # [ 147.620352] nix-daemon[855]: reaped child process 1425, status = succeeded5511worker1 # [ 147.706311] nix-daemon[855]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5512worker1 # [ 147.722917] nix-daemon[855]: reaped child process 1444, status = succeeded5513lb # [ 147.792946] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received uploads request method=POST path=/api/pending_closures5514worker1 # [ 147.743553] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.376Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5515worker1 # [ 147.746773] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.376Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5516worker1 # [ 147.781671] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.416Z level=INFO msg="Uploading 1 narinfos"5517lb # [ 147.834204] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5518lb # [ 147.856672] niks3-server[1211]: 2026/09/23 05:35:20 INFO Signed narinfos id=44 count=15519lb # [ 147.859593] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5520lb # [ 147.883205] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5521worker1 # [ 147.830493] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.466Z level=INFO msg="Upload complete. (292ms)"5522lb # [ 147.894555] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5523lb # [ 147.900868] niks3-server[1211]: 2026/09/23 05:35:20 INFO Completed upload id=445524lb # [ 147.903523] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5525lb # [ 147.907435] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5526worker1 # [ 147.853332] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15527worker1 # [ 147.869639] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05528worker1 # [ 147.873749] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15529worker1 # [ 147.877832] nix-daemon[855]: reaped child process 1423, status = succeeded5530worker1 # [ 147.898610] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504585531worker1 # [ 147.913435] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504585532worker1 # [ 147.927032] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.562Z level=INFO msg="All 1 paths already cached"5533client # copying 1 paths to 'lb:50051'...5534client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5535worker1 # [ 147.937668] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504585536worker1 # [ 147.953697] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5537worker1 # [ 148.104894] nix-daemon[855]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)5538worker1 # [ 148.124583] nix-daemon[855]: reaped child process 1453, status = succeeded5539lb # [ 148.185155] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received uploads request method=POST path=/api/pending_closures5540worker1 # [ 148.132859] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.768Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5541worker1 # [ 148.134420] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.768Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5542lb # [ 148.216073] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5543worker1 # [ 148.162138] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.797Z level=INFO msg="Uploading 1 narinfos"5544lb # [ 148.230241] niks3-server[1211]: 2026/09/23 05:35:20 INFO Signed narinfos id=45 count=15545lb # [ 148.238326] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5546lb # [ 148.254986] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5547worker1 # [ 148.203979] nix-grpc-daemon[1405]: time=2026-09-23T05:35:20.839Z level=INFO msg="Upload complete. (234ms)"5548lb # [ 148.266250] niks3-server[1211]: 2026/09/23 05:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5549lb # [ 148.270474] niks3-server[1211]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5550lb # [ 148.277050] niks3-server[1211]: 2026/09/23 05:35:20 INFO Completed upload id=455551worker1 # [ 148.215682] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 paths=1 nar_bytes_in=6405552worker1 # [ 148.231122] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504585553worker1 # [ 148.237387] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5554worker1 # [ 148.242563] nix-daemon[855]: reaped child process 1448, status = succeeded5555client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5556worker1 # [ 148.254103] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5557client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5558client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5559client # farm-b> LOG-farm-b-on-standby5560worker1 # [ 148.910219] nix-daemon[855]: reaped child process 1457, status = succeeded5561worker1 # [ 149.026587] nix-daemon[855]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5562worker1 # [ 149.041661] nix-daemon[855]: reaped child process 1476, status = succeeded5563lb # [ 149.112720] niks3-server[1211]: 2026/09/23 05:35:21 INFO Received uploads request method=POST path=/api/pending_closures5564worker1 # [ 149.062571] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.697Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5565worker1 # [ 149.064450] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.697Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5566lb # [ 149.159180] niks3-server[1211]: 2026/09/23 05:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5567worker1 # [ 149.104654] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.740Z level=INFO msg="Uploading 1 narinfos"5568lb # [ 149.171475] niks3-server[1211]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5569lb # [ 149.180934] niks3-server[1211]: 2026/09/23 05:35:21 INFO Signed narinfos id=46 count=15570lb # [ 149.183186] niks3-server[1211]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5571lb # [ 149.194062] niks3-server[1211]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5572lb # [ 149.198135] niks3-server[1211]: 2026/09/23 05:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5573worker1 # [ 149.135350] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.770Z level=INFO msg="Upload complete. (269ms)"5574lb # [ 149.201513] niks3-server[1211]: 2026/09/23 05:35:21 INFO Completed upload id=465575lb # [ 149.203206] niks3-server[1211]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5576worker1 # [ 149.156461] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15577worker1 # [ 149.170278] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05578worker1 # [ 149.174161] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15579worker1 # [ 149.178223] nix-daemon[855]: reaped child process 1455, status = succeeded5580worker1 # [ 149.198594] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504585581worker1 # [ 149.213373] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504585582worker1 # [ 149.228693] nix-grpc-daemon[1405]: time=2026-09-23T05:35:21.863Z level=INFO msg="All 2 paths already cached"5583client # copying 1 paths to 'lb:50051'...5584client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5585worker1 # [ 149.240574] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504585586worker1 # [ 149.254557] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5587lb # [ 149.391100] envoy[907]: conn peer=[2001:db8:1::4]:55842 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5513 tx=665 ms=132625588worker1 # [ 149.404856] nix-daemon[855]: accepted connection from pid 1482, user nix-grpc-daemon (trusted)5589worker1 # [ 149.422236] nix-daemon[855]: reaped child process 1485, status = succeeded5590lb # [ 149.491324] niks3-server[1211]: 2026/09/23 05:35:22 INFO Received uploads request method=POST path=/api/pending_closures5591worker1 # [ 149.439553] nix-grpc-daemon[1405]: time=2026-09-23T05:35:22.074Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5592worker1 # [ 149.441308] nix-grpc-daemon[1405]: time=2026-09-23T05:35:22.074Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5593worker1 # [ 149.468765] nix-grpc-daemon[1405]: time=2026-09-23T05:35:22.104Z level=INFO msg="Uploading 1 narinfos"5594lb # [ 149.522225] niks3-server[1211]: 2026/09/23 05:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5595lb # [ 149.536967] niks3-server[1211]: 2026/09/23 05:35:22 INFO Signed narinfos id=47 count=15596lb # [ 149.539847] niks3-server[1211]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5597lb # [ 149.556482] niks3-server[1211]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5598worker1 # [ 149.498875] nix-grpc-daemon[1405]: time=2026-09-23T05:35:22.134Z level=INFO msg="Upload complete. (229ms)"5599lb # [ 149.561612] niks3-server[1211]: 2026/09/23 05:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5600lb # [ 149.568198] niks3-server[1211]: 2026/09/23 05:35:22 INFO Completed upload id=475601lb # [ 149.571308] niks3-server[1211]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5602worker1 # [ 149.508263] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 paths=1 nar_bytes_in=7685603worker1 # [ 149.521872] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504585604worker1 # [ 149.530441] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5605worker1 # [ 149.533595] nix-daemon[855]: reaped child process 1480, status = succeeded5606client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5607worker1 # [ 149.546738] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5608client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5609client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5610client # farm-top> LOG-farm-top-on-standby5611worker1 # [ 150.622237] nix-daemon[855]: reaped child process 1489, status = succeeded5612worker1 # [ 150.706150] nix-daemon[855]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)5613worker1 # [ 150.723010] nix-daemon[855]: reaped child process 1508, status = succeeded5614lb # [ 150.793875] niks3-server[1211]: 2026/09/23 05:35:23 INFO Received uploads request method=POST path=/api/pending_closures5615worker1 # [ 150.743199] nix-grpc-daemon[1405]: time=2026-09-23T05:35:23.376Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5616worker1 # [ 150.744997] nix-grpc-daemon[1405]: time=2026-09-23T05:35:23.376Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5617lb # [ 150.832468] niks3-server[1211]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5618worker1 # [ 150.781759] nix-grpc-daemon[1405]: time=2026-09-23T05:35:23.417Z level=INFO msg="Uploading 1 narinfos"5619lb # [ 150.845439] niks3-server[1211]: 2026/09/23 05:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5620lb # [ 150.854971] niks3-server[1211]: 2026/09/23 05:35:23 INFO Signed narinfos id=48 count=15621lb # [ 150.864615] niks3-server[1211]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5622worker1 # [ 150.816304] nix-grpc-daemon[1405]: time=2026-09-23T05:35:23.451Z level=INFO msg="Upload complete. (259ms)"5623lb # [ 150.878268] niks3-server[1211]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5624lb # [ 150.883996] niks3-server[1211]: 2026/09/23 05:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5625lb # [ 150.889036] niks3-server[1211]: 2026/09/23 05:35:23 INFO Completed upload id=485626lb # [ 150.892822] niks3-server[1211]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5627worker1 # [ 150.828743] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15628worker1 # [ 150.849109] nix-daemon[855]: reaped child process 1487, status = succeeded5629client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.06 seconds)5630client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5631client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5632worker2 # [ 151.150833] systemd[1]: Listening on Nix worker-protocol over gRPC.5633worker2 # [ 151.159011] systemd[1]: Starting scheduler-token.service...5634worker2 # [ 151.230674] systemd[1]: scheduler-token.service: Deactivated successfully.5635worker2 # [ 151.236250] systemd[1]: Finished scheduler-token.service.5636worker2 # [ 151.237723] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 75ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5637worker2 # [ 151.254411] systemd[1]: Starting Nix worker-protocol over gRPC...5638worker2 # [ 151.365345] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:23Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5639worker2 # [ 151.367550] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:23Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5640worker2 # [ 151.371393] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:23Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15641worker2 # [ 151.376051] nix-grpc-daemon[1364]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5642worker2 # [ 151.377507] nix-grpc-daemon[1364]: E0000 00:00:1790141724.004290 1364 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5643lb: must succeed: curl -sf localhost:9901/clusters5644worker2 # [ 151.424110] nix-grpc-daemon[1364]: E0000 00:00:1790141724.010363 1364 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5645worker2 # [ 151.426404] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5646worker2 # [ 151.431268] nix-grpc-daemon[1364]: E0000 00:00:1790141724.018918 1407 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5647worker2 # [ 151.434220] nix-grpc-daemon[1364]: E0000 00:00:1790141724.038953 1407 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5648worker2 # [ 151.436707] nix-grpc-daemon[1364]: E0000 00:00:1790141724.039088 1407 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5649worker2 # [ 151.439087] systemd[1]: Started Nix worker-protocol over gRPC.5650worker2 # [ 151.439880] nix-grpc-daemon[1364]: E0000 00:00:1790141724.058796 1407 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5651worker2 # [ 151.447975] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:24Z level=info event=scheduler_connected addr=lb:500515652worker1 # [ 151.448137] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=oidc_jwks provider=mock keys=15653worker1 # [ 151.452909] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:504425654worker1 # [ 151.458248] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05655lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5656worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5657worker1 # [ 151.537750] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:384165658worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5659lb: must succeed: curl -sf localhost:9901/clusters5660lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5661lb: must succeed: curl -sf localhost:9901/clusters5662lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5663worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5664worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5665(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.05 seconds)5666subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5667client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5668client # [ 151.789428] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5669client # Running as unit: rs.service; invocation ID: d82ceea537064fe3a9067e1eb904666a5670client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5671worker1 # [ 151.987198] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504585672client # [ 152.003848] nix[1404]: don't know how to build these paths:5673client # [ 152.005238] nix[1404]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5674worker1 # [ 152.003372] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05675worker1 # [ 152.010443] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15676worker1 # [ 152.049952] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504585677worker1 # [ 152.065838] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504585678client # [ 152.088275] nix[1404]: copying 1 paths to 'lb:50051'...5679worker1 # [ 152.080580] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504585680client # [ 152.090559] nix[1404]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5681worker1 # [ 152.097208] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5682worker1 # [ 152.258247] nix-daemon[855]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)5683worker1 # [ 152.275796] nix-daemon[855]: reaped child process 1524, status = succeeded5684lb # [ 152.348905] niks3-server[1211]: 2026/09/23 05:35:24 INFO Received uploads request method=POST path=/api/pending_closures5685worker1 # [ 152.297338] nix-grpc-daemon[1405]: time=2026-09-23T05:35:24.932Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5686worker1 # [ 152.299454] nix-grpc-daemon[1405]: time=2026-09-23T05:35:24.932Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5687lb # [ 152.378964] niks3-server[1211]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5688worker1 # [ 152.330113] nix-grpc-daemon[1405]: time=2026-09-23T05:35:24.965Z level=INFO msg="Uploading 1 narinfos"5689lb # [ 152.395052] niks3-server[1211]: 2026/09/23 05:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5690lb # [ 152.403738] niks3-server[1211]: 2026/09/23 05:35:25 INFO Signed narinfos id=49 count=15691lb # [ 152.411696] niks3-server[1211]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5692worker1 # [ 152.354358] nix-grpc-daemon[1405]: time=2026-09-23T05:35:24.989Z level=INFO msg="Upload complete. (240ms)"5693lb # [ 152.419227] niks3-server[1211]: 2026/09/23 05:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5694lb # [ 152.423948] niks3-server[1211]: 2026/09/23 05:35:25 INFO Completed upload id=495695lb # [ 152.427104] niks3-server[1211]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5696worker1 # [ 152.364292] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=0 paths=1 nar_bytes_in=4165697worker1 # [ 152.380310] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504585698worker1 # [ 152.388512] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5699worker1 # [ 152.391407] nix-daemon[855]: reaped child process 1519, status = succeeded5700worker1 # [ 152.402406] nix-daemon[855]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5701client # [ 152.409483] nix[1404]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5702lb: must succeed: curl -sf localhost:9901/clusters5703lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5704lb: must succeed: curl -sf localhost:9901/clusters5705lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5706lb: must succeed: curl -sf localhost:9901/clusters5707lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5708lb: must succeed: curl -sf localhost:9901/clusters5709lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5710worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5711worker2 # [ 154.596056] systemd-journald[272]: Received client request to rotate journal, rotating.5712worker2 # [ 154.621810] systemd-journald[272]: Vacuuming done, freed 0B of archived journals from /var/log/journal/5bab130533d944929e2e9da27f3fa86e.5713worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5714worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5715worker1 # [ 154.658721] systemd[1]: Stopping Nix worker-protocol over gRPC...5716worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5717client: must succeed: sleep 5.05718worker1 # [ 155.046737] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:27Z level=info event=draining5719lb # [ 159.391413] envoy[907]: conn peer=[2001:db8:1::1]:35466 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9576 tx=4373 ms=58125720client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5721worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5722worker1 # [ 160.069140] nix-daemon[855]: reaped child process 1528, status = succeeded5723worker1 # [ 160.193224] nix-daemon[855]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)5724worker1 # [ 160.209312] nix-daemon[855]: reaped child process 1572, status = succeeded5725lb # [ 160.278595] niks3-server[1211]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures5726worker1 # [ 160.231771] nix-grpc-daemon[1405]: time=2026-09-23T05:35:32.863Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5727worker1 # [ 160.233722] nix-grpc-daemon[1405]: time=2026-09-23T05:35:32.863Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5728lb # [ 160.315012] niks3-server[1211]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5729worker1 # [ 160.272819] nix-grpc-daemon[1405]: time=2026-09-23T05:35:32.908Z level=INFO msg="Uploading 1 narinfos"5730lb # [ 160.335400] niks3-server[1211]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5731lb # [ 160.345228] niks3-server[1211]: 2026/09/23 05:35:32 INFO Signed narinfos id=50 count=15732lb # [ 160.352191] niks3-server[1211]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5733lb # [ 160.355895] niks3-server[1211]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5734lb # [ 160.363842] niks3-server[1211]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5735lb # [ 160.374468] niks3-server[1211]: 2026/09/23 05:35:33 INFO Completed upload id=505736lb # [ 160.378173] niks3-server[1211]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5737worker1 # [ 160.312271] nix-grpc-daemon[1405]: time=2026-09-23T05:35:32.947Z level=INFO msg="Upload complete. (292ms)"5738worker1 # [ 160.325395] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50458 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15739worker1 # [ 160.345965] nix-daemon[855]: reaped child process 1526, status = succeeded5740client # [ 160.367346] systemd[1]: rs.service: Deactivated successfully.5741client # [ 160.372136] systemd[1]: rs.service: Consumed 193ms CPU time over 8.579s wall clock time, 41.2M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.5742worker1 # [ 161.049535] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:33Z level=info event=signal_exit5743lb # [ 161.317309] niks3-server[1211]: 2026/09/23 05:35:33 INFO lead: released remote=192.168.1.3:359645744lb # [ 161.443333] niks3-server[1211]: 2026/09/23 05:35:34 INFO lead: acquired remote=192.168.1.4:386885745worker2 # [ 161.396885] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:34Z level=info event=scheduler_take_over5746worker2 # [ 162.267513] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:34Z level=info event=scheduler_restarting addr=lb:500515747worker1 # [ 162.259974] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=scheduler_restarting5748lb # [ 162.320159] envoy[907]: [2026-09-23 05:35:34.948][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5749worker1 # [ 162.262345] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=scheduler_restarting addr=lb:500515750worker1 # [ 162.262477] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:504425751worker1 # [ 162.262590] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=worker_gone addr=192.168.1.4:500515752worker1 # [ 162.262701] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:504425753worker1 # [ 162.262810] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:34Z level=info event=worker_gone addr=192.168.1.3:500515754worker1 # [ 162.288583] nix-grpc-daemon[1325]: E0000 00:00:1790141734.923592 1371 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5755worker1 # [ 162.319853] nix-daemon[855]: reaped child process 1395, status = succeeded5756worker2 # [ 162.470083] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:35Z level=info event=scheduler_connected addr=lb:500515757worker1 # [ 162.461126] nix-grpc-daemon[1325]: E0000 00:00:1790141735.096601 1371 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5758worker1 # [ 162.488795] nix-grpc-daemon[1325]: ts=2026-09-23T05:35:35Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5759worker1 # [ 162.867761] nix-grpc-daemon[1325]: E0000 00:00:1790141735.503247 1371 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5760worker1 # [ 163.166154] nix-grpc-daemon[1325]: E0000 00:00:1790141735.801513 1371 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5761worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.24 seconds)5762client: waiting for success: ! systemctl is-active rs5763client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5764client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5765client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5766client: must fail: journalctl -u rs | grep -q 'reconnecting'5767client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5768lb: must succeed: curl -sf localhost:9901/clusters5769lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5770worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5771worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5772worker1 # [ 163.596499] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5773worker1 # [ 163.603102] systemd[1]: Stopped Nix worker-protocol over gRPC.5774worker2 # [ 163.617222] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=oidc_jwks provider=mock keys=15775worker1 # [ 163.608901] systemd[1]: nix-grpc-daemon.service: Consumed 1.632s CPU time over 31.735s wall clock time, 39.3M memory peak, 49.6K incoming IP traffic, 94.8K outgoing IP traffic.5776worker2 # [ 163.622485] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:475365777worker2 # [ 163.627848] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05778worker1 # [ 163.638467] systemd[1]: Starting Nix worker-protocol over gRPC...5779worker1 # [ 163.775788] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5780worker1 # [ 163.778272] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5781worker1 # [ 163.782222] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15782worker1 # [ 163.788062] nix-grpc-daemon[1579]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5783worker1 # [ 163.789515] nix-grpc-daemon[1579]: E0000 00:00:1790141736.422353 1579 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5784worker1 # [ 163.828057] nix-grpc-daemon[1579]: E0000 00:00:1790141736.428568 1579 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5785worker1 # [ 163.830376] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5786worker1 # [ 163.833820] nix-grpc-daemon[1579]: E0000 00:00:1790141736.434608 1620 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5787worker1 # [ 163.836692] nix-grpc-daemon[1579]: E0000 00:00:1790141736.449833 1620 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5788worker1 # [ 163.839135] nix-grpc-daemon[1579]: E0000 00:00:1790141736.449967 1620 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5789worker1 # [ 163.844725] systemd[1]: Started Nix worker-protocol over gRPC.5790worker1 # [ 163.847381] nix-grpc-daemon[1579]: E0000 00:00:1790141736.468522 1620 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5791worker1 # [ 163.849985] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:36Z level=info event=scheduler_connected addr=lb:500515792worker2 # [ 163.858161] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:475365793worker2 # [ 163.864383] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05794worker2 # [ 164.144961] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:36Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:475445795lb: must succeed: curl -sf localhost:9901/clusters5796lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5797worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5798worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5799(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.92 seconds)5800subtest: scheduler restart mid-build: build finishes, no double build5801client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5802client # [ 164.704970] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5803client # Running as unit: mid.service; invocation ID: faca667cf90e4b389da40e980f95d5f75804client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5805worker2 # [ 164.919064] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:475565806worker2 # [ 164.933297] nix-daemon[844]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5807client # [ 164.949854] nix[1440]: don't know how to build these paths:5808client # [ 164.951836] nix[1440]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5809worker2 # [ 164.965694] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:37Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05810worker2 # [ 164.972529] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:37Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15811worker2 # [ 165.020562] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475565812worker2 # [ 165.035671] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5)5813worker2 # [ 165.062799] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 189 ms (attempt 2/5)5814worker2 # [ 165.261095] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 290 ms (attempt 3/5)5815worker2 # [ 165.553078] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 641 ms (attempt 4/5)5816worker2 # [ 166.197470] nix-grpc-daemon[1364]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5817worker2 # [ 166.201757] nix-grpc-daemon[1364]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5818worker2 # [ 166.222417] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:475565819client # [ 166.236824] nix[1440]: copying 1 paths to 'lb:50051'...5820worker2 # [ 166.237452] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:475565821client # [ 166.237074] nix[1440]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5822worker2 # [ 166.253057] nix-daemon[844]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5823worker2 # [ 166.286570] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:38Z level=info event=niks3_push_started pid=14975824worker2 # [ 166.435935] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.063Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5825worker2 # [ 166.541358] nix-daemon[844]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5826worker2 # [ 166.560853] nix-daemon[844]: reaped child process 1513, status = succeeded5827lb # [ 166.613896] niks3-server[1211]: 2026/09/23 05:35:39 INFO Received uploads request method=POST path=/api/pending_closures5828worker2 # [ 166.571042] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.198Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5829worker2 # [ 166.574861] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.198Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5830lb # [ 166.659932] niks3-server[1211]: 2026/09/23 05:35:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5831lb # [ 166.678295] niks3-server[1211]: 2026/09/23 05:35:39 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5832worker2 # [ 166.648164] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.276Z level=INFO msg="Uploading 1 narinfos"5833lb # [ 166.700779] niks3-server[1211]: 2026/09/23 05:35:39 INFO Signed narinfos id=51 count=15834lb # [ 166.708206] niks3-server[1211]: 2026/09/23 05:35:39 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5835lb # [ 166.721248] niks3-server[1211]: 2026/09/23 05:35:39 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5836lb # [ 166.731258] niks3-server[1211]: 2026/09/23 05:35:39 INFO Completed upload id=515837worker2 # [ 166.679744] nix-grpc-daemon[1497]: time=2026-09-23T05:35:39.307Z level=INFO msg="Upload complete. (240ms)"5838lb # [ 166.740097] niks3-server[1211]: 2026/09/23 05:35:39 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5839worker2 # [ 166.692670] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47556 duration_s=0 paths=1 nar_bytes_in=4245840worker2 # [ 166.708469] nix-grpc-daemon[1364]: ts=2026-09-23T05:35:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:475565841worker2 # [ 166.715382] nix-daemon[844]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5842worker2 # [ 166.724323] nix-daemon[844]: reaped child process 1495, status = succeeded5843worker2 # [ 166.732204] nix-daemon[844]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5844client # [ 166.729012] nix[1440]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5845lb: must succeed: curl -sf localhost:9901/clusters5846lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5847lb: must succeed: curl -sf localhost:9901/clusters5848lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5849worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5850worker2 # [ 168.581421] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1364 (nix-grpc-daemon) on client request.5851worker2 # [ 168.583166] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5852client # [ 168.588625] nix[1440]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5853worker1 # [ 168.580490] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5854worker2 # [ 168.598079] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5855lb # [ 168.648901] niks3-server[1211]: 2026/09/23 05:35:41 INFO lead: released remote=192.168.1.4:386885856client # [ 168.588967] nix[1440]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5857worker2 # [ 168.599597] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5858lb # [ 168.654937] envoy[907]: [2026-09-23 05:35:41.284][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5859worker2 # [ 168.611175] systemd[1]: nix-grpc-daemon.service: Consumed 507ms CPU time over 17.350s wall clock time, 35.1M memory peak, 15K incoming IP traffic, 23.5K outgoing IP traffic.5860worker2 # [ 168.625828] systemd[1]: Starting scheduler-token.service...5861worker2 # [ 168.649323] nix-daemon[844]: reaped child process 1515, status = succeeded5862worker2 # [ 168.651628] nix-daemon[844]: reaped child process 1483, status = succeeded5863worker2 # [ 168.657645] nix-daemon[1517]: unexpected Nix daemon error: error: interrupted by the user5864worker2 # [ 168.668465] nix-daemon[844]: reaped child process 1517, status = failed with exit code 15865worker2 # [ 168.691254] systemd[1]: scheduler-token.service: Deactivated successfully.5866worker2 # [ 168.694228] systemd[1]: Finished scheduler-token.service.5867worker2 # [ 168.695002] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5868worker2 # [ 168.700554] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5869worker2 # [ 168.705592] systemd[1]: Starting scheduler-token.service...5870worker2 # [ 168.745266] systemd[1]: scheduler-token.service: Deactivated successfully.5871worker2 # [ 168.748070] systemd[1]: Finished scheduler-token.service.5872worker2 # [ 168.748910] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 47ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5873worker2 # [ 168.753071] systemd[1]: Starting Nix worker-protocol over gRPC...5874worker1 # [ 168.780571] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:500515875lb # [ 168.862086] niks3-server[1211]: 2026/09/23 05:35:41 INFO lead: acquired remote=192.168.1.3:428385876worker1 # [ 168.804346] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_take_over5877worker2 # [ 168.853627] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5878worker2 # [ 168.855799] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5879worker2 # [ 168.859831] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15880worker2 # [ 168.866489] nix-grpc-daemon[1560]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5881worker2 # [ 168.867937] nix-grpc-daemon[1560]: E0000 00:00:1790141741.492713 1560 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5882worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.35 seconds)5883client: must succeed: sleep 5.05884worker1 # [ 168.875843] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5885worker2 # [ 168.892801] nix-grpc-daemon[1560]: E0000 00:00:1790141741.494700 1560 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5886worker2 # [ 168.895577] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5887worker2 # [ 168.922990] nix-grpc-daemon[1560]: E0000 00:00:1790141741.514120 1600 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5888worker2 # [ 168.925952] systemd[1]: Started Nix worker-protocol over gRPC.5889worker2 # [ 168.926909] nix-grpc-daemon[1560]: E0000 00:00:1790141741.536056 1600 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5890worker2 # [ 168.929358] nix-grpc-daemon[1560]: E0000 00:00:1790141741.536194 1600 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5891worker2 # [ 168.934352] nix-grpc-daemon[1560]: E0000 00:00:1790141741.550562 1600 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5892worker2 # [ 168.939114] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:500515893worker2 # [ 168.941016] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5894worker1 # [ 169.076827] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:500515895client # [ 169.092254] nix[1440]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting5896worker1 # [ 169.082226] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5897worker2 # [ 169.142216] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:500515898worker2 # [ 169.148793] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5899worker1 # [ 169.282836] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:500515900worker1 # [ 169.288188] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5901lb # [ 169.389616] envoy[907]: conn peer=[2001:db8:1::1]:44124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1515 ms=83505902lb # [ 169.394928] envoy[907]: conn peer=[2001:db8:1::3]:60940 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3797 tx=1219 ms=303775903lb # [ 169.399952] envoy[907]: conn peer=[2001:db8:1::4]:50528 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1948 tx=328 ms=171535904worker2 # [ 169.349823] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_connected addr=lb:500515905worker2 # [ 169.356283] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5906worker1 # [ 169.488917] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:500515907worker1 # [ 169.494246] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5908worker2 # [ 169.557328] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:500515909worker2 # [ 169.563646] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5910worker1 # [ 169.696626] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:500515911worker1 # [ 169.700681] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5912worker2 # [ 169.764847] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:500515913worker2 # [ 169.769950] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5914worker1 # [ 169.896914] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:500515915worker1 # [ 169.902002] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5916worker2 # [ 169.972674] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:500515917worker2 # [ 169.976743] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5918client # [ 170.095840] nix[1440]: scheduler at lb:50051: no healthy upstream, reconnecting5919worker1 # [ 170.102997] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:500515920worker1 # [ 170.108226] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5921worker2 # [ 170.173161] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:500515922worker2 # [ 170.178158] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5923worker1 # [ 170.309025] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_connected addr=lb:500515924worker1 # [ 170.314228] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5925worker2 # [ 170.379168] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:500515926worker2 # [ 170.384362] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5927worker1 # [ 170.514748] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:500515928worker1 # [ 170.520188] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5929worker2 # [ 170.585375] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:500515930worker2 # [ 170.590372] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5931worker1 # [ 170.727126] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:500515932worker1 # [ 170.735840] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:352425933worker1 # [ 170.740920] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05934worker2 # [ 170.791286] nix-grpc-daemon[1560]: ts=2026-09-23T05:35:43Z level=info event=scheduler_connected addr=lb:500515935worker1 # [ 170.791029] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=oidc_jwks provider=mock keys=15936worker1 # [ 170.795666] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:352425937worker1 # [ 170.801938] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05938worker1 # [ 172.101830] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:44Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05939worker1 # [ 172.102127] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:44Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15940worker1 # [ 172.160136] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506885941worker1 # [ 172.174809] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5942worker1 # [ 172.198010] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 1/5)5943worker1 # [ 172.232829] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 2/5)5944worker1 # [ 172.328864] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 257 ms (attempt 3/5)5945worker1 # [ 172.587958] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 769 ms (attempt 4/5)5946worker1 # [ 173.074904] nix-grpc-daemon[1579]: ts=2026-09-23T05:35:45Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:507045947worker1 # [ 173.361149] nix-grpc-daemon[1579]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5948worker1 # [ 173.369622] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5949worker1 # [ 173.378911] nix-grpc-daemon[1579]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5950worker1 # [ 173.408777] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5951worker1 # [ 173.428174] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5952client # [ 173.436336] nix[1440]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5953client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5954client: waiting for success: ! systemctl is-active mid5955worker1 # [ 264.948495] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:17Z level=info event=niks3_push_started pid=16885956worker1 # [ 265.032375] nix-daemon[855]: reaped child process 1663, status = succeeded5957worker1 # [ 265.107938] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.743Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5958worker1 # [ 265.214236] nix-daemon[855]: accepted connection from pid 1700, user nix-grpc-daemon (trusted)5959worker1 # [ 265.230653] nix-daemon[855]: reaped child process 1703, status = succeeded5960lb # [ 265.301593] niks3-server[1211]: 2026/09/23 05:37:17 INFO Received uploads request method=POST path=/api/pending_closures5961worker1 # [ 265.258349] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.892Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5962worker1 # [ 265.259832] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.892Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5963lb # [ 265.378167] niks3-server[1211]: 2026/09/23 05:37:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5964worker1 # [ 265.323641] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.957Z level=INFO msg="Uploading 1 narinfos"5965lb # [ 265.387715] niks3-server[1211]: 2026/09/23 05:37:18 INFO Signed narinfos id=52 count=15966lb # [ 265.399774] niks3-server[1211]: 2026/09/23 05:37:18 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5967lb # [ 265.407857] niks3-server[1211]: 2026/09/23 05:37:18 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5968worker1 # [ 265.351663] nix-grpc-daemon[1688]: time=2026-09-23T05:37:17.987Z level=INFO msg="Upload complete. (239ms)"5969lb # [ 265.415443] niks3-server[1211]: 2026/09/23 05:37:18 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5970lb # [ 265.419097] niks3-server[1211]: 2026/09/23 05:37:18 INFO Completed upload id=525971lb # [ 265.420858] niks3-server[1211]: 2026/09/23 05:37:18 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5972lb # [ 265.425275] niks3-server[1211]: 2026/09/23 05:37:18 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5973worker1 # [ 265.372380] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50688 duration_s=93 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15974worker1 # [ 265.388604] nix-daemon[855]: reaped child process 1661, status = succeeded5975client # [ 265.436266] systemd[1]: mid.service: Deactivated successfully.5976client # [ 265.440700] systemd[1]: mid.service: Consumed 273ms CPU time over 1min 40.734s wall clock time, 41.3M memory peak, 4K written to disk, 10.7K incoming IP traffic, 12.2K outgoing IP traffic.5977client: (finished: waiting for success: ! systemctl is-active mid, in 92.26 seconds)5978client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5979client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5980(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 101.90 seconds)5981subtest: interrupting the client stops the build on the worker5982client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5983client # [ 266.583038] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5984client # Running as unit: intr.service; invocation ID: 7cd8f1b1e6b44cf8b51567e069602f7f5985client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)5986worker2 # [ 266.793652] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:606965987worker2 # [ 266.807426] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)5988client # [ 266.821915] nix[1905]: don't know how to build these paths:5989client # [ 266.823228] nix[1905]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5990worker1 # [ 266.825088] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:19Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05991worker1 # [ 266.831316] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:19Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15992worker2 # [ 266.882023] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606965993worker2 # [ 266.902332] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5)5994worker2 # [ 266.925560] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)5995worker2 # [ 266.943603] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 295 ms (attempt 3/5)5996worker2 # [ 267.240592] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 474 ms (attempt 4/5)5997worker2 # [ 267.719620] nix-grpc-daemon[1560]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5998worker2 # [ 267.725033] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5999worker2 # [ 267.744459] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606966000client # [ 267.757398] nix[1905]: copying 1 paths to 'lb:50051'...6001worker2 # [ 267.759476] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606966002client # [ 267.760556] nix[1905]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6003worker2 # [ 267.780963] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6004worker2 # [ 267.813551] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=info event=niks3_push_started pid=16386005worker2 # [ 267.999493] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.627Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6006worker2 # [ 268.108451] nix-daemon[844]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6007worker2 # [ 268.128837] nix-daemon[844]: reaped child process 1657, status = succeeded6008lb # [ 268.180796] niks3-server[1211]: 2026/09/23 05:37:20 INFO Received uploads request method=POST path=/api/pending_closures6009worker2 # [ 268.140097] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.767Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6010worker2 # [ 268.141559] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.767Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6011lb # [ 268.218490] niks3-server[1211]: 2026/09/23 05:37:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6012worker2 # [ 268.180759] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.808Z level=INFO msg="Uploading 1 narinfos"6013lb # [ 268.233527] niks3-server[1211]: 2026/09/23 05:37:20 INFO Signed narinfos id=53 count=16014lb # [ 268.240343] niks3-server[1211]: 2026/09/23 05:37:20 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6015worker2 # [ 268.205556] nix-grpc-daemon[1638]: time=2026-09-23T05:37:20.833Z level=INFO msg="Upload complete. (202ms)"6016lb # [ 268.253162] niks3-server[1211]: 2026/09/23 05:37:20 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6017lb # [ 268.260807] niks3-server[1211]: 2026/09/23 05:37:20 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6018lb # [ 268.265507] niks3-server[1211]: 2026/09/23 05:37:20 INFO Completed upload id=536019worker2 # [ 268.212126] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=4246020lb # [ 268.268533] niks3-server[1211]: 2026/09/23 05:37:20 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6021worker2 # [ 268.224423] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966022worker2 # [ 268.227813] nix-daemon[844]: reaped child process 1636, status = succeeded6023worker2 # [ 268.234489] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6024worker2 # [ 268.244753] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6025client # [ 268.243127] nix[1905]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6026lb # [ 269.392833] envoy[907]: conn peer=[2001:db8:1::1]:43630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4454 tx=2266 ms=1004966027client: must succeed: systemctl kill -s INT intr6028client # [ 270.375178] systemd[1]: intr.service: Sent signal SIGINT to main process 1905 (nix) on client request.6029worker2 # [ 270.387475] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:23Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6030client # [ 270.384583] nix[1905]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6031client # [ 270.396757] nix[1905]: error: interrupted by the user6032client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6033client # [ 270.423726] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6034client # [ 270.428953] systemd[1]: intr.service: Failed with result 'exit-code'.6035client # [ 270.431094] systemd[1]: intr.service: Consumed 198ms CPU time over 3.844s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6036worker2 # [ 270.471261] nix-daemon[1661]: unexpected Nix daemon error: error: interrupted by the user6037worker2 # [ 270.476683] nix-daemon[844]: reaped child process 1659, status = succeeded6038worker2 # [ 270.482098] nix-daemon[844]: reaped child process 1661, status = failed with exit code 16039(finished: subtest: interrupting the client stops the build on the worker, in 3.96 seconds)6040subtest: a deploy mid-build drains: the build finishes, the new generation takes over6041client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6042client # [ 270.570300] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6043client # Running as unit: sw.service; invocation ID: 1f9e5e12dd924c92a6ac092a9707eeaa6044client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds)6045worker1 # [ 270.763824] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506886046client # [ 270.780565] nix[1926]: don't know how to build these paths:6047client # [ 270.781515] nix[1926]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6048worker1 # [ 270.783841] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06049worker1 # [ 270.791311] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16050worker1 # [ 270.830703] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506886051worker1 # [ 270.853231] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506886052client # [ 270.876142] nix[1926]: copying 1 paths to 'lb:50051'...6053worker1 # [ 270.870100] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506886054client # [ 270.880930] nix[1926]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6055worker1 # [ 270.885195] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6056worker1 # [ 271.043919] nix-daemon[855]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6057worker1 # [ 271.062560] nix-daemon[855]: reaped child process 1740, status = succeeded6058lb # [ 271.129760] niks3-server[1211]: 2026/09/23 05:37:23 INFO Received uploads request method=POST path=/api/pending_closures6059worker1 # [ 271.078498] nix-grpc-daemon[1688]: time=2026-09-23T05:37:23.713Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6060worker1 # [ 271.080103] nix-grpc-daemon[1688]: time=2026-09-23T05:37:23.713Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6061lb # [ 271.162870] niks3-server[1211]: 2026/09/23 05:37:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6062worker1 # [ 271.111556] nix-grpc-daemon[1688]: time=2026-09-23T05:37:23.746Z level=INFO msg="Uploading 1 narinfos"6063lb # [ 271.178784] niks3-server[1211]: 2026/09/23 05:37:23 INFO Signed narinfos id=54 count=16064lb # [ 271.188509] niks3-server[1211]: 2026/09/23 05:37:23 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6065lb # [ 271.199322] niks3-server[1211]: 2026/09/23 05:37:23 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6066lb # [ 271.209312] niks3-server[1211]: 2026/09/23 05:37:23 INFO Completed upload id=546067worker1 # [ 271.142841] nix-grpc-daemon[1688]: time=2026-09-23T05:37:23.778Z level=INFO msg="Upload complete. (239ms)"6068worker1 # [ 271.151888] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50688 duration_s=0 paths=1 nar_bytes_in=4166069lb # [ 271.214867] niks3-server[1211]: 2026/09/23 05:37:23 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6070lb # [ 271.224891] niks3-server[1211]: 2026/09/23 05:37:23 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6071worker1 # [ 271.164444] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506886072worker1 # [ 271.174492] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6073worker1 # [ 271.179939] nix-daemon[855]: reaped child process 1735, status = succeeded6074client # [ 271.199204] nix[1926]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6075worker1 # [ 271.194882] nix-daemon[855]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6076worker1: must succeed: systemctl reload nix-grpc-daemon.service6077worker1 # [ 273.172280] systemd[1]: Reloading Nix worker-protocol over gRPC...6078worker1 # [ 273.211820] systemd[1]: Reloaded Nix worker-protocol over gRPC.6079worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.09 seconds)6080worker1: must succeed: pgrep -f 'read -t [9]0 x'6081worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6082client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26083client # don't know how to build these paths:6084client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6085worker2 # [ 273.496930] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:606966086worker2 # [ 273.504692] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606966087worker1 # [ 273.520782] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:26Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06088worker1 # [ 273.524053] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:26Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16089worker2 # [ 273.591209] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966090worker2 # [ 273.605900] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606966091client # copying 1 paths to 'lb:50051'...6092worker2 # [ 273.621831] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606966093client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6094worker2 # [ 273.645154] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6095worker2 # [ 273.810974] nix-daemon[844]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6096worker2 # [ 273.832185] nix-daemon[844]: reaped child process 1711, status = succeeded6097lb # [ 273.885767] niks3-server[1211]: 2026/09/23 05:37:26 INFO Received uploads request method=POST path=/api/pending_closures6098worker2 # [ 273.846803] nix-grpc-daemon[1638]: time=2026-09-23T05:37:26.474Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6099worker2 # [ 273.848708] nix-grpc-daemon[1638]: time=2026-09-23T05:37:26.474Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6100worker1 # [ 273.840503] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:26Z level=info event=draining6101lb # [ 273.927672] niks3-server[1211]: 2026/09/23 05:37:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6102worker2 # [ 273.884583] nix-grpc-daemon[1638]: time=2026-09-23T05:37:26.512Z level=INFO msg="Uploading 1 narinfos"6103lb # [ 273.940971] niks3-server[1211]: 2026/09/23 05:37:26 INFO Signed narinfos id=55 count=16104lb # [ 273.949886] niks3-server[1211]: 2026/09/23 05:37:26 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6105lb # [ 273.958466] niks3-server[1211]: 2026/09/23 05:37:26 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6106lb # [ 273.968596] niks3-server[1211]: 2026/09/23 05:37:26 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6107lb # [ 273.977076] niks3-server[1211]: 2026/09/23 05:37:26 INFO Completed upload id=556108worker2 # [ 273.924790] nix-grpc-daemon[1638]: time=2026-09-23T05:37:26.553Z level=INFO msg="Upload complete. (262ms)"6109lb # [ 273.984170] niks3-server[1211]: 2026/09/23 05:37:26 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6110worker2 # [ 273.936357] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=5286111worker2 # [ 273.952334] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966112worker2 # [ 273.962081] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6113worker2 # [ 273.968723] nix-daemon[844]: reaped child process 1706, status = succeeded6114client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6115client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6116worker2 # [ 273.984196] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6117client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6118client # farm-a> LOG-farm-a-during-drain6119worker2 # [ 274.618011] nix-daemon[844]: reaped child process 1715, status = succeeded6120worker2 # [ 274.733628] nix-daemon[844]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6121worker2 # [ 274.750180] nix-daemon[844]: reaped child process 1734, status = succeeded6122lb # [ 274.809061] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received uploads request method=POST path=/api/pending_closures6123worker2 # [ 274.768074] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.395Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6124worker2 # [ 274.769604] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.395Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6125lb # [ 274.852362] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6126worker2 # [ 274.807124] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.435Z level=INFO msg="Uploading 1 narinfos"6127lb # [ 274.869418] niks3-server[1211]: 2026/09/23 05:37:27 INFO Signed narinfos id=56 count=16128lb # [ 274.876330] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6129worker2 # [ 274.837126] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.465Z level=INFO msg="Upload complete. (242ms)"6130lb # [ 274.889453] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6131lb # [ 274.898598] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6132worker2 # [ 274.847391] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16133worker1 # [ 274.842503] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:27Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06134lb # [ 274.904571] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6135worker1 # [ 274.844381] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:27Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16136lb # [ 274.907691] niks3-server[1211]: 2026/09/23 05:37:27 INFO Completed upload id=566137lb # [ 274.910778] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6138worker2 # [ 274.864522] nix-daemon[844]: reaped child process 1713, status = succeeded6139worker2 # [ 274.900267] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966140worker2 # [ 274.914392] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606966141client # copying 1 paths to 'lb:50051'...6142worker2 # [ 274.930192] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.556Z level=INFO msg="All 1 paths already cached"6143worker2 # [ 274.935540] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606966144client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6145worker2 # [ 274.957000] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6146worker2 # [ 275.107623] nix-daemon[844]: accepted connection from pid 1738, user nix-grpc-daemon (trusted)6147worker2 # [ 275.123808] nix-daemon[844]: reaped child process 1741, status = succeeded6148lb # [ 275.181992] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received uploads request method=POST path=/api/pending_closures6149worker2 # [ 275.139366] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.767Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6150worker2 # [ 275.141008] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.767Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6151worker2 # [ 275.177195] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.805Z level=INFO msg="Uploading 1 narinfos"6152lb # [ 275.221736] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6153lb # [ 275.233942] niks3-server[1211]: 2026/09/23 05:37:27 INFO Signed narinfos id=57 count=16154lb # [ 275.240455] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6155lb # [ 275.247406] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6156lb # [ 275.255291] niks3-server[1211]: 2026/09/23 05:37:27 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6157worker2 # [ 275.203149] nix-grpc-daemon[1638]: time=2026-09-23T05:37:27.831Z level=INFO msg="Upload complete. (229ms)"6158lb # [ 275.263582] niks3-server[1211]: 2026/09/23 05:37:27 INFO Completed upload id=576159lb # [ 275.267619] niks3-server[1211]: 2026/09/23 05:37:27 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6160worker2 # [ 275.212616] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=6406161worker2 # [ 275.226607] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966162worker2 # [ 275.234929] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6163worker2 # [ 275.238161] nix-daemon[844]: reaped child process 1736, status = succeeded6164client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6165worker2 # [ 275.248136] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6166client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6167client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6168client # farm-b> LOG-farm-b-during-drain6169worker2 # [ 276.113357] nix-daemon[844]: reaped child process 1745, status = succeeded6170worker2 # [ 276.224933] nix-daemon[844]: accepted connection from pid 1761, user nix-grpc-daemon (trusted)6171worker2 # [ 276.242336] nix-daemon[844]: reaped child process 1764, status = succeeded6172lb # [ 276.300823] niks3-server[1211]: 2026/09/23 05:37:28 INFO Received uploads request method=POST path=/api/pending_closures6173worker2 # [ 276.259424] nix-grpc-daemon[1638]: time=2026-09-23T05:37:28.886Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6174worker2 # [ 276.261275] nix-grpc-daemon[1638]: time=2026-09-23T05:37:28.886Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6175lb # [ 276.336234] niks3-server[1211]: 2026/09/23 05:37:28 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6176worker2 # [ 276.298035] nix-grpc-daemon[1638]: time=2026-09-23T05:37:28.926Z level=INFO msg="Uploading 1 narinfos"6177lb # [ 276.349850] niks3-server[1211]: 2026/09/23 05:37:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6178lb # [ 276.362738] niks3-server[1211]: 2026/09/23 05:37:28 INFO Signed narinfos id=58 count=16179lb # [ 276.369738] niks3-server[1211]: 2026/09/23 05:37:28 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6180lb # [ 276.378364] niks3-server[1211]: 2026/09/23 05:37:28 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6181lb # [ 276.389975] niks3-server[1211]: 2026/09/23 05:37:29 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6182worker2 # [ 276.338421] nix-grpc-daemon[1638]: time=2026-09-23T05:37:28.966Z level=INFO msg="Upload complete. (278ms)"6183lb # [ 276.393582] niks3-server[1211]: 2026/09/23 05:37:29 INFO Completed upload id=586184lb # [ 276.396173] niks3-server[1211]: 2026/09/23 05:37:29 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6185worker1 # [ 276.344080] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:28Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06186worker2 # [ 276.349135] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16187worker1 # [ 276.345907] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:28Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16188worker2 # [ 276.371281] nix-daemon[844]: reaped child process 1743, status = succeeded6189worker2 # [ 276.400101] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966190worker2 # [ 276.414686] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606966191worker2 # [ 276.429693] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.057Z level=INFO msg="All 2 paths already cached"6192client # copying 1 paths to 'lb:50051'...6193client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6194worker2 # [ 276.440776] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606966195worker2 # [ 276.457836] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6196worker2 # [ 276.615219] nix-daemon[844]: accepted connection from pid 1768, user nix-grpc-daemon (trusted)6197worker2 # [ 276.634016] nix-daemon[844]: reaped child process 1771, status = succeeded6198lb # [ 276.690872] niks3-server[1211]: 2026/09/23 05:37:29 INFO Received uploads request method=POST path=/api/pending_closures6199worker2 # [ 276.647351] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.275Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6200worker2 # [ 276.649422] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.277Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6201lb # [ 276.723474] niks3-server[1211]: 2026/09/23 05:37:29 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6202worker2 # [ 276.682661] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.310Z level=INFO msg="Uploading 1 narinfos"6203lb # [ 276.738821] niks3-server[1211]: 2026/09/23 05:37:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6204lb # [ 276.749209] niks3-server[1211]: 2026/09/23 05:37:29 INFO Signed narinfos id=59 count=16205lb # [ 276.755600] niks3-server[1211]: 2026/09/23 05:37:29 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6206lb # [ 276.766389] niks3-server[1211]: 2026/09/23 05:37:29 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6207lb # [ 276.772163] niks3-server[1211]: 2026/09/23 05:37:29 INFO Completed upload id=596208worker2 # [ 276.720133] nix-grpc-daemon[1638]: time=2026-09-23T05:37:29.348Z level=INFO msg="Upload complete. (245ms)"6209lb # [ 276.776117] niks3-server[1211]: 2026/09/23 05:37:29 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6210worker2 # [ 276.732428] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=7686211worker2 # [ 276.747204] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966212worker2 # [ 276.756579] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6213worker2 # [ 276.759541] nix-daemon[844]: reaped child process 1766, status = succeeded6214client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6215worker2 # [ 276.771074] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6216client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6217client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6218client # farm-top> LOG-farm-top-during-drain6219worker2 # [ 277.878928] nix-daemon[844]: reaped child process 1775, status = succeeded6220worker2 # [ 277.952088] nix-daemon[844]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)6221worker2 # [ 277.969755] nix-daemon[844]: reaped child process 1794, status = succeeded6222lb # [ 278.026904] niks3-server[1211]: 2026/09/23 05:37:30 INFO Received uploads request method=POST path=/api/pending_closures6223worker2 # [ 277.986571] nix-grpc-daemon[1638]: time=2026-09-23T05:37:30.613Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6224worker2 # [ 277.988351] nix-grpc-daemon[1638]: time=2026-09-23T05:37:30.613Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6225lb # [ 278.068757] niks3-server[1211]: 2026/09/23 05:37:30 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6226worker2 # [ 278.030890] nix-grpc-daemon[1638]: time=2026-09-23T05:37:30.658Z level=INFO msg="Uploading 1 narinfos"6227lb # [ 278.085316] niks3-server[1211]: 2026/09/23 05:37:30 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6228lb # [ 278.094219] niks3-server[1211]: 2026/09/23 05:37:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6229lb # [ 278.102948] niks3-server[1211]: 2026/09/23 05:37:30 INFO Signed narinfos id=60 count=16230worker2 # [ 278.053648] nix-grpc-daemon[1638]: time=2026-09-23T05:37:30.681Z level=INFO msg="Upload complete. (269ms)"6231lb # [ 278.107930] niks3-server[1211]: 2026/09/23 05:37:30 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6232lb # [ 278.110660] niks3-server[1211]: 2026/09/23 05:37:30 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6233lb # [ 278.114889] niks3-server[1211]: 2026/09/23 05:37:30 INFO Completed upload id=606234lb # [ 278.118054] niks3-server[1211]: 2026/09/23 05:37:30 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6235worker2 # [ 278.068250] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16236worker2 # [ 278.081440] nix-daemon[844]: reaped child process 1773, status = succeeded6237client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 4.84 seconds)6238client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6239client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6240worker1: must succeed: pkill -f 'read -t [9]0 x'6241worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6242client: waiting for success: ! systemctl is-active sw6243worker1 # [ 278.460965] nix-daemon[855]: reaped child process 1744, status = succeeded6244worker1 # [ 278.549654] nix-daemon[855]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)6245worker1 # [ 278.566327] nix-daemon[855]: reaped child process 1794, status = succeeded6246lb # [ 278.635714] niks3-server[1211]: 2026/09/23 05:37:31 INFO Received uploads request method=POST path=/api/pending_closures6247worker1 # [ 278.583643] nix-grpc-daemon[1688]: time=2026-09-23T05:37:31.218Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6248worker1 # [ 278.585308] nix-grpc-daemon[1688]: time=2026-09-23T05:37:31.218Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6249lb # [ 278.673271] niks3-server[1211]: 2026/09/23 05:37:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6250worker1 # [ 278.617088] nix-grpc-daemon[1688]: time=2026-09-23T05:37:31.252Z level=INFO msg="Uploading 1 narinfos"6251lb # [ 278.689040] niks3-server[1211]: 2026/09/23 05:37:31 INFO Signed narinfos id=61 count=16252lb # [ 278.691836] niks3-server[1211]: 2026/09/23 05:37:31 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6253worker1 # [ 278.644560] nix-grpc-daemon[1688]: time=2026-09-23T05:37:31.280Z level=INFO msg="Upload complete. (234ms)"6254lb # [ 278.706714] niks3-server[1211]: 2026/09/23 05:37:31 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6255lb # [ 278.715278] niks3-server[1211]: 2026/09/23 05:37:31 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6256lb # [ 278.720632] niks3-server[1211]: 2026/09/23 05:37:31 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6257worker1 # [ 278.656850] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50688 duration_s=7 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16258lb # [ 278.724704] niks3-server[1211]: 2026/09/23 05:37:31 INFO Completed upload id=616259lb # [ 278.726601] niks3-server[1211]: 2026/09/23 05:37:31 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6260worker1 # [ 278.672557] nix-daemon[855]: reaped child process 1742, status = succeeded6261client # [ 278.702486] systemd[1]: sw.service: Deactivated successfully.6262client # [ 278.707540] systemd[1]: sw.service: Consumed 188ms CPU time over 8.132s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6263worker1 # [ 278.842743] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:31Z level=info event=signal_exit6264lb # [ 279.387093] envoy[907]: conn peer=[2001:db8:1::1]:59936 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=829 ms=36276265client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6266client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6267client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)6268worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6269lb # [ 279.392048] envoy[907]: conn peer=[2001:db8:1::1]:47682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9591 tx=4407 ms=45886270lb # [ 279.392174] envoy[907]: conn peer=[2001:db8:1::1]:59950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3004 tx=1518 ms=79106271lb # [ 279.864230] niks3-server[1211]: 2026/09/23 05:37:32 INFO lead: released remote=192.168.1.3:428386272worker1 # [ 280.805529] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=scheduler_restarting6273worker2 # [ 280.817714] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_restarting addr=lb:500516274lb # [ 280.866826] envoy[907]: [2026-09-23 05:37:33.496][907][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6275worker1 # [ 280.816698] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:352426276worker1 # [ 280.827720] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=worker_gone addr=192.168.1.4:500516277worker1 # [ 280.833090] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=scheduler_restarting addr=lb:500516278worker1 # [ 280.837650] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:352426279worker1 # [ 280.843849] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=worker_gone addr=192.168.1.3:500516280worker1 # [ 280.868277] nix-daemon[855]: reaped child process 1654, status = succeeded6281worker1 # [ 280.891698] nix-grpc-daemon[1579]: E0000 00:00:1790141853.527058 1625 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6282worker1 # [ 280.905025] nix-grpc-daemon[1579]: E0000 00:00:1790141853.540450 1625 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6283worker2 # [ 281.018680] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_connected addr=lb:500516284worker2 # [ 281.023966] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6285worker1 # [ 281.045627] nix-grpc-daemon[1579]: ts=2026-09-23T05:37:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6286worker2 # [ 281.224877] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_connected addr=lb:500516287worker2 # [ 281.229889] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6288worker2 # [ 281.430844] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:500516289worker2 # [ 281.435980] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6290worker2 # [ 281.636947] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:500516291worker2 # [ 281.641951] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6292worker2 # [ 281.844769] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:500516293worker2 # [ 281.848860] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6294worker1 # [ 281.923276] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6295worker1 # [ 281.928326] systemd[1]: nix-grpc-daemon.service: Consumed 1.054s CPU time over 1min 58.298s wall clock time, 38.2M memory peak, 30.4K incoming IP traffic, 38.7K outgoing IP traffic.6296worker1 # [ 281.951513] systemd[1]: Starting Nix worker-protocol over gRPC...6297worker2 # [ 282.045248] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:500516298worker2 # [ 282.050368] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6299worker1 # [ 282.099911] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6300worker1 # [ 282.102324] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6301worker1 # [ 282.106361] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16302worker1 # [ 282.112057] nix-grpc-daemon[1808]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6303worker1 # [ 282.113475] nix-grpc-daemon[1808]: E0000 00:00:1790141854.746710 1808 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6304worker1 # [ 282.123673] nix-grpc-daemon[1808]: E0000 00:00:1790141854.753249 1808 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6305lb # [ 282.186406] niks3-server[1211]: 2026/09/23 05:37:34 INFO lead: acquired remote=192.168.1.3:569246306worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.66 seconds)6307lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6308lb: must succeed: curl -sf localhost:9901/clusters6309worker1 # [ 282.140188] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6310worker1 # [ 282.141893] systemd[1]: Started Nix worker-protocol over gRPC.6311worker1 # [ 282.142688] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=scheduler_take_over6312worker1 # [ 282.143772] nix-grpc-daemon[1808]: E0000 00:00:1790141854.766115 1848 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6313worker1 # [ 282.168285] nix-grpc-daemon[1808]: E0000 00:00:1790141854.788476 1848 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6314worker1 # [ 282.171053] nix-grpc-daemon[1808]: E0000 00:00:1790141854.788635 1848 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6315worker1 # [ 282.173667] nix-grpc-daemon[1808]: E0000 00:00:1790141854.802934 1848 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6316worker1 # [ 282.184269] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:500516317worker1 # [ 282.186144] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6318lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6319worker2 # [ 282.251443] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_connected addr=lb:500516320worker2 # [ 282.256618] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6321worker1 # [ 282.385836] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:500516322worker1 # [ 282.390972] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6323worker2 # [ 282.457642] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:500516324worker2 # [ 282.462720] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6325worker1 # [ 282.591693] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:500516326worker1 # [ 282.597058] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6327worker2 # [ 282.663600] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:500516328worker2 # [ 282.668717] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6329worker1 # [ 282.797586] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:500516330worker1 # [ 282.802633] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6331worker2 # [ 282.871180] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:500516332worker2 # [ 282.875349] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6333worker1 # [ 283.003251] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:500516334worker1 # [ 283.011689] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:407706335worker1 # [ 283.017562] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06336worker2 # [ 283.071579] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:35Z level=info event=scheduler_connected addr=lb:500516337worker1 # [ 283.072136] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=oidc_jwks provider=mock keys=16338worker1 # [ 283.077418] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:407706339worker1 # [ 283.082750] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06340lb: must succeed: curl -sf localhost:9901/clusters6341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6342lb: must succeed: curl -sf localhost:9901/clusters6343lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6344lb: must succeed: curl -sf localhost:9901/clusters6345lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6346worker1 # [ 286.008751] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:38Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:407866347lb: must succeed: curl -sf localhost:9901/clusters6348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6349lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 4.34 seconds)6350lb: must succeed: curl -sf localhost:9901/clusters6351lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6352worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6353worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6354(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.09 seconds)6355subtest: queue gauges show a burst while every build blocks6356lb: must succeed: curl -sf localhost:9901/clusters6357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6358lb: must succeed: curl -sf localhost:9901/clusters6359lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6360worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6361worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6362client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6363client # [ 286.841397] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6364client # Running as unit: burst.service; invocation ID: 7c9c3de658c74a1daba61813ecf27f7f6365client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6366lb: must succeed: curl -sf localhost:9901/clusters6367lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6368lb: must succeed: curl -sf localhost:9901/clusters6369lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6370worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6371client # [ 287.044814] nix[1982]: don't know how to build these paths:6372client # [ 287.045763] nix[1982]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6373worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6374client # [ 287.046810] nix[1982]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6375client # [ 287.047789] nix[1982]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6376worker2 # [ 287.046368] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:606966377client # [ 287.049359] nix[1982]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6378worker1 # [ 287.054554] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=want client=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06379worker1 # [ 287.060047] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16380worker1 # [ 287.065922] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=want client=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06381worker1 # [ 287.071220] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16382worker1 # [ 287.076580] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=want client=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06383worker1 # [ 287.080750] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=want client=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06384worker2 # [ 287.108773] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966385worker1 # [ 287.103067] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986386worker2 # [ 287.125068] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606966387worker1 # [ 287.117753] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6388client # [ 287.136261] nix[1982]: copying 1 paths to 'lb:50051'...6389worker2 # [ 287.139954] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606966390client # [ 287.141188] nix[1982]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6391worker1 # [ 287.140687] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 1/5)6392worker2 # [ 287.155050] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6393worker1 # [ 287.147595] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 2/5)6394worker1 # [ 287.202186] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 370 ms (attempt 3/5)6395worker2 # [ 287.323101] nix-daemon[844]: accepted connection from pid 1798, user nix-grpc-daemon (trusted)6396worker2 # [ 287.345382] nix-daemon[844]: reaped child process 1801, status = succeeded6397lb # [ 287.399360] niks3-server[1211]: 2026/09/23 05:37:40 INFO Received uploads request method=POST path=/api/pending_closures6398worker2 # [ 287.358575] nix-grpc-daemon[1638]: time=2026-09-23T05:37:39.986Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6399worker2 # [ 287.360463] nix-grpc-daemon[1638]: time=2026-09-23T05:37:39.986Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6400lb # [ 287.433020] niks3-server[1211]: 2026/09/23 05:37:40 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6401worker2 # [ 287.394205] nix-grpc-daemon[1638]: time=2026-09-23T05:37:40.022Z level=INFO msg="Uploading 1 narinfos"6402lb # [ 287.448338] niks3-server[1211]: 2026/09/23 05:37:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6403lb # [ 287.456207] niks3-server[1211]: 2026/09/23 05:37:40 INFO Signed narinfos id=62 count=16404lb # [ 287.465845] niks3-server[1211]: 2026/09/23 05:37:40 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6405lb # [ 287.480735] niks3-server[1211]: 2026/09/23 05:37:40 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6406lb # [ 287.486874] niks3-server[1211]: 2026/09/23 05:37:40 INFO Completed upload id=626407worker2 # [ 287.434794] nix-grpc-daemon[1638]: time=2026-09-23T05:37:40.063Z level=INFO msg="Upload complete. (261ms)"6408lb # [ 287.498436] niks3-server[1211]: 2026/09/23 05:37:40 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6409worker2 # [ 287.445367] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=4166410worker2 # [ 287.460509] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966411worker2 # [ 287.474871] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6412worker2 # [ 287.480396] nix-daemon[844]: reaped child process 1796, status = succeeded6413client # [ 287.489557] nix[1982]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6414worker2 # [ 287.495174] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6415worker1 # [ 287.574232] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 320 ms (attempt 4/5)6416worker1 # [ 287.899325] nix-grpc-daemon[1808]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6417worker1 # [ 287.904482] nix-grpc-daemon[1808]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6418worker1 # [ 287.923677] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986419client # [ 287.947545] nix[1982]: copying 1 paths to 'lb:50051'...6420worker1 # [ 287.939377] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986421client # [ 287.950552] nix[1982]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6422worker1 # [ 287.956339] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6423worker1 # [ 287.998900] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:40Z level=info event=niks3_push_started pid=18806424lb: must succeed: curl -sf localhost:9901/clusters6425lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6426lb: must succeed: curl -sf localhost:9901/clusters6427worker1 # [ 288.134055] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.769Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6428lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6429worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6430worker1 # [ 288.257713] nix-daemon[855]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6431worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.12 seconds)6432worker1 # [ 288.291233] nix-daemon[855]: reaped child process 1898, status = succeeded6433lb # [ 288.351623] niks3-server[1211]: 2026/09/23 05:37:40 INFO Received uploads request method=POST path=/api/pending_closures6434worker1 # [ 288.300482] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.934Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6435worker1 # [ 288.301948] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.934Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6436worker1 # [ 288.337677] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.972Z level=INFO msg="Uploading 1 narinfos"6437lb # [ 288.392720] niks3-server[1211]: 2026/09/23 05:37:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6438lb # [ 288.406953] niks3-server[1211]: 2026/09/23 05:37:41 INFO Signed narinfos id=63 count=16439worker1 # [ 288.361686] nix-grpc-daemon[1880]: time=2026-09-23T05:37:40.997Z level=INFO msg="Upload complete. (224ms)"6440lb # [ 288.413788] niks3-server[1211]: 2026/09/23 05:37:41 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6441lb # [ 288.427827] niks3-server[1211]: 2026/09/23 05:37:41 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6442worker1 # [ 288.366868] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=4166443worker1 # [ 288.371363] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986444lb # [ 288.432753] niks3-server[1211]: 2026/09/23 05:37:41 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6445lb # [ 288.436128] niks3-server[1211]: 2026/09/23 05:37:41 INFO Completed upload id=636446worker1 # [ 288.378365] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6447lb # [ 288.438666] niks3-server[1211]: 2026/09/23 05:37:41 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6448worker1 # [ 288.381324] nix-daemon[855]: reaped child process 1878, status = succeeded6449worker1 # [ 288.388125] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6450client # [ 288.397008] nix[1982]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6451lb # [ 289.392271] envoy[907]: conn peer=[2001:db8:1::3]:52016 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2431 tx=1013 ms=1171886452worker2 # [ 289.797749] nix-daemon[844]: reaped child process 1805, status = succeeded6453worker2 # [ 289.871680] nix-daemon[844]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6454worker2 # [ 289.890236] nix-daemon[844]: reaped child process 1833, status = succeeded6455lb # [ 289.948925] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received uploads request method=POST path=/api/pending_closures6456worker2 # [ 289.908429] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.534Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6457worker2 # [ 289.910706] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.534Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6458lb # [ 289.990606] niks3-server[1211]: 2026/09/23 05:37:42 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6459worker2 # [ 289.950988] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.579Z level=INFO msg="Uploading 1 narinfos"6460lb # [ 290.007006] niks3-server[1211]: 2026/09/23 05:37:42 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6461lb # [ 290.016547] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6462lb # [ 290.029475] niks3-server[1211]: 2026/09/23 05:37:42 INFO Signed narinfos id=64 count=16463worker2 # [ 289.976480] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.604Z level=INFO msg="Upload complete. (269ms)"6464worker1 # [ 289.974827] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:42Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=16465lb # [ 290.035774] niks3-server[1211]: 2026/09/23 05:37:42 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6466lb # [ 290.040491] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6467lb # [ 290.044845] niks3-server[1211]: 2026/09/23 05:37:42 INFO Completed upload id=646468lb # [ 290.047367] niks3-server[1211]: 2026/09/23 05:37:42 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6469worker2 # [ 289.989646] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=2 outputs=16470worker2 # [ 290.005367] nix-daemon[844]: reaped child process 1803, status = succeeded6471worker2 # [ 290.033120] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966472worker2 # [ 290.048572] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606966473client # [ 290.061611] nix[1982]: copying 1 paths to 'lb:50051'...6474worker2 # [ 290.062847] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606966475client # [ 290.067003] nix[1982]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6476worker2 # [ 290.082759] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6477worker2 # [ 290.244892] nix-daemon[844]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6478worker2 # [ 290.267350] nix-daemon[844]: reaped child process 1840, status = succeeded6479lb # [ 290.317457] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received uploads request method=POST path=/api/pending_closures6480worker2 # [ 290.274111] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.902Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6481worker2 # [ 290.275546] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.902Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6482lb # [ 290.359379] niks3-server[1211]: 2026/09/23 05:37:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6483worker2 # [ 290.315672] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.943Z level=INFO msg="Uploading 1 narinfos"6484lb # [ 290.374916] niks3-server[1211]: 2026/09/23 05:37:42 INFO Signed narinfos id=65 count=16485lb # [ 290.382372] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6486lb # [ 290.393011] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6487lb # [ 290.398565] niks3-server[1211]: 2026/09/23 05:37:43 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6488worker2 # [ 290.353090] nix-grpc-daemon[1638]: time=2026-09-23T05:37:42.981Z level=INFO msg="Upload complete. (254ms)"6489lb # [ 290.405149] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6490lb # [ 290.409173] niks3-server[1211]: 2026/09/23 05:37:43 INFO Completed upload id=656491worker2 # [ 290.363421] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=4166492worker2 # [ 290.380352] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606966493worker2 # [ 290.387495] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6494worker2 # [ 290.393373] nix-daemon[844]: reaped child process 1835, status = succeeded6495client # [ 290.402637] nix[1982]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6496worker2 # [ 290.408461] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6497worker1 # [ 290.844873] nix-daemon[855]: reaped child process 1906, status = succeeded6498worker1 # [ 290.956445] nix-daemon[855]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6499worker1 # [ 290.976201] nix-daemon[855]: reaped child process 1934, status = succeeded6500lb # [ 291.037841] niks3-server[1211]: 2026/09/23 05:37:43 INFO Received uploads request method=POST path=/api/pending_closures6501worker1 # [ 290.987174] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.620Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6502worker1 # [ 290.989830] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.620Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6503lb # [ 291.070000] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6504lb # [ 291.092422] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6505worker1 # [ 291.040737] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.675Z level=INFO msg="Uploading 1 narinfos"6506lb # [ 291.102218] niks3-server[1211]: 2026/09/23 05:37:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6507lb # [ 291.108195] niks3-server[1211]: 2026/09/23 05:37:43 INFO Signed narinfos id=66 count=16508lb # [ 291.110366] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6509lb # [ 291.121869] niks3-server[1211]: 2026/09/23 05:37:43 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6510lb # [ 291.126501] niks3-server[1211]: 2026/09/23 05:37:43 INFO Completed upload id=666511worker1 # [ 291.064526] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.700Z level=INFO msg="Upload complete. (266ms)"6512lb # [ 291.129576] niks3-server[1211]: 2026/09/23 05:37:43 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6513worker1 # [ 291.077003] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=1 outputs=16514worker1 # [ 291.092343] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=16515worker1 # [ 291.097667] nix-daemon[855]: reaped child process 1904, status = succeeded6516worker1 # [ 291.118354] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986517worker1 # [ 291.133714] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986518client # [ 291.157181] nix[1982]: copying 1 paths to 'lb:50051'...6519worker1 # [ 291.148106] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986520client # [ 291.157419] nix[1982]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6521worker1 # [ 291.163955] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6522worker1 # [ 291.311259] nix-daemon[855]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)6523worker1 # [ 291.330519] nix-daemon[855]: reaped child process 1941, status = succeeded6524lb # [ 291.396078] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received uploads request method=POST path=/api/pending_closures6525worker1 # [ 291.343028] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6526worker1 # [ 291.344750] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.978Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6527worker1 # [ 291.363275] nix-grpc-daemon[1880]: time=2026-09-23T05:37:43.998Z level=INFO msg="Uploading 1 narinfos"6528lb # [ 291.425666] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6529lb # [ 291.438217] niks3-server[1211]: 2026/09/23 05:37:44 INFO Signed narinfos id=67 count=16530lb # [ 291.446115] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6531lb # [ 291.454569] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6532worker1 # [ 291.406140] nix-grpc-daemon[1880]: time=2026-09-23T05:37:44.041Z level=INFO msg="Upload complete. (227ms)"6533lb # [ 291.468157] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6534lb # [ 291.472756] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6535lb # [ 291.478467] niks3-server[1211]: 2026/09/23 05:37:44 INFO Completed upload id=676536worker1 # [ 291.416383] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=4166537worker1 # [ 291.432067] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986538worker1 # [ 291.441865] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6539worker1 # [ 291.445113] nix-daemon[855]: reaped child process 1936, status = succeeded6540client # [ 291.463976] nix[1982]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6541worker1 # [ 291.457889] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6542worker2 # [ 292.046408] nix-daemon[844]: reaped child process 1844, status = succeeded6543worker2 # [ 292.149117] nix-daemon[844]: accepted connection from pid 1869, user nix-grpc-daemon (trusted)6544worker2 # [ 292.165843] nix-daemon[844]: reaped child process 1872, status = succeeded6545lb # [ 292.226304] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received uploads request method=POST path=/api/pending_closures6546worker2 # [ 292.184358] nix-grpc-daemon[1638]: time=2026-09-23T05:37:44.811Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6547worker2 # [ 292.185819] nix-grpc-daemon[1638]: time=2026-09-23T05:37:44.811Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6548lb # [ 292.270738] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6549lb # [ 292.288391] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6550lb # [ 292.298866] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6551worker2 # [ 292.255494] nix-grpc-daemon[1638]: time=2026-09-23T05:37:44.883Z level=INFO msg="Uploading 1 narinfos"6552lb # [ 292.307110] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6553lb # [ 292.315093] niks3-server[1211]: 2026/09/23 05:37:44 INFO Signed narinfos id=68 count=16554lb # [ 292.325108] niks3-server[1211]: 2026/09/23 05:37:44 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6555lb # [ 292.336338] niks3-server[1211]: 2026/09/23 05:37:44 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6556lb # [ 292.350098] niks3-server[1211]: 2026/09/23 05:37:44 INFO Completed upload id=686557worker2 # [ 292.298562] nix-grpc-daemon[1638]: time=2026-09-23T05:37:44.926Z level=INFO msg="Upload complete. (278ms)"6558worker2 # [ 292.309121] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=3 outputs=16559worker2 # [ 292.329130] nix-daemon[844]: reaped child process 1842, status = succeeded6560worker1 # [ 293.083178] nix-daemon[855]: reaped child process 1945, status = succeeded6561worker1 # [ 293.181141] nix-daemon[855]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)6562worker1 # [ 293.199190] nix-daemon[855]: reaped child process 1973, status = succeeded6563lb # [ 293.268709] niks3-server[1211]: 2026/09/23 05:37:45 INFO Received uploads request method=POST path=/api/pending_closures6564worker1 # [ 293.219003] nix-grpc-daemon[1880]: time=2026-09-23T05:37:45.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6565worker1 # [ 293.220627] nix-grpc-daemon[1880]: time=2026-09-23T05:37:45.853Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6566lb # [ 293.310515] niks3-server[1211]: 2026/09/23 05:37:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6567worker1 # [ 293.259255] nix-grpc-daemon[1880]: time=2026-09-23T05:37:45.894Z level=INFO msg="Uploading 1 narinfos"6568lb # [ 293.327436] niks3-server[1211]: 2026/09/23 05:37:45 INFO Signed narinfos id=69 count=16569lb # [ 293.338416] niks3-server[1211]: 2026/09/23 05:37:45 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6570lb # [ 293.349112] niks3-server[1211]: 2026/09/23 05:37:45 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6571lb # [ 293.362897] niks3-server[1211]: 2026/09/23 05:37:45 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6572worker1 # [ 293.307372] nix-grpc-daemon[1880]: time=2026-09-23T05:37:45.942Z level=INFO msg="Upload complete. (236ms)"6573lb # [ 293.369384] niks3-server[1211]: 2026/09/23 05:37:45 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6574lb # [ 293.374195] niks3-server[1211]: 2026/09/23 05:37:45 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6575lb # [ 293.380332] niks3-server[1211]: 2026/09/23 05:37:46 INFO Completed upload id=696576worker1 # [ 293.319380] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=4 outputs=16577worker1 # [ 293.339233] nix-daemon[855]: reaped child process 1943, status = succeeded6578client # [ 293.377989] systemd[1]: burst.service: Deactivated successfully.6579client # [ 293.383109] systemd[1]: burst.service: Consumed 227ms CPU time over 6.537s wall clock time, 53.6M memory peak, 14.4K incoming IP traffic, 19.2K outgoing IP traffic.6580client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6581client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6582lb: must succeed: curl -sf localhost:9901/clusters6583lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6584lb: must succeed: curl -sf localhost:9901/clusters6585lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6586worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6587worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6588(finished: subtest: queue gauges show a burst while every build blocks, in 8.07 seconds)6589subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6590client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16591worker2 # [ 294.891543] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:606966592worker2 # [ 294.899492] nix-grpc-daemon[1560]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606966593worker1 # [ 294.934333] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=want client=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06594worker1 # [ 294.934631] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16595worker1 # [ 294.934833] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=want client=2 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06596worker1 # [ 294.934956] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=want client=2 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06597worker1 # [ 294.978710] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986598worker1 # [ 294.993680] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986599worker1 # [ 295.008517] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986600worker1 # [ 295.023525] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6601worker1 # [ 295.182549] nix-daemon[855]: accepted connection from pid 1987, user nix-grpc-daemon (trusted)6602worker1 # [ 295.201685] nix-daemon[855]: reaped child process 1990, status = succeeded6603lb # [ 295.269844] niks3-server[1211]: 2026/09/23 05:37:47 INFO Received uploads request method=POST path=/api/pending_closures6604worker1 # [ 295.218286] nix-grpc-daemon[1880]: time=2026-09-23T05:37:47.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6605worker1 # [ 295.219977] nix-grpc-daemon[1880]: time=2026-09-23T05:37:47.853Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6606lb # [ 295.299785] niks3-server[1211]: 2026/09/23 05:37:47 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6607worker1 # [ 295.249694] nix-grpc-daemon[1880]: time=2026-09-23T05:37:47.885Z level=INFO msg="Uploading 1 narinfos"6608lb # [ 295.313339] niks3-server[1211]: 2026/09/23 05:37:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6609lb # [ 295.323528] niks3-server[1211]: 2026/09/23 05:37:47 INFO Signed narinfos id=70 count=16610lb # [ 295.330211] niks3-server[1211]: 2026/09/23 05:37:47 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6611worker1 # [ 295.277118] nix-grpc-daemon[1880]: time=2026-09-23T05:37:47.912Z level=INFO msg="Upload complete. (236ms)"6612lb # [ 295.340524] niks3-server[1211]: 2026/09/23 05:37:47 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6613lb # [ 295.347869] niks3-server[1211]: 2026/09/23 05:37:47 INFO Completed upload id=706614lb # [ 295.351939] niks3-server[1211]: 2026/09/23 05:37:47 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6615worker1 # [ 295.288334] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=5126616worker1 # [ 295.300459] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986617worker1 # [ 295.310928] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6618worker1 # [ 295.314203] nix-daemon[855]: reaped child process 1985, status = succeeded6619worker1 # [ 295.327716] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6620worker1 # [ 296.421023] nix-daemon[855]: reaped child process 1994, status = succeeded6621worker1 # [ 296.505285] nix-daemon[855]: accepted connection from pid 2010, user nix-grpc-daemon (trusted)6622worker1 # [ 296.524375] nix-daemon[855]: reaped child process 2013, status = succeeded6623lb # [ 296.590078] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received uploads request method=POST path=/api/pending_closures6624worker1 # [ 296.539647] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.173Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6625worker1 # [ 296.541564] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.173Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6626worker1 # [ 296.576656] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.212Z level=INFO msg="Uploading 1 narinfos"6627lb # [ 296.631373] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6628lb # [ 296.647408] niks3-server[1211]: 2026/09/23 05:37:49 INFO Signed narinfos id=71 count=16629lb # [ 296.653803] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6630lb # [ 296.662009] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6631lb # [ 296.671277] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6632worker1 # [ 296.616084] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.251Z level=INFO msg="Upload complete. (277ms)"6633lb # [ 296.679037] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6634lb # [ 296.683467] niks3-server[1211]: 2026/09/23 05:37:49 INFO Completed upload id=716635lb # [ 296.687744] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6636worker1 # [ 296.629124] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=5 outputs=16637worker1 # [ 296.653097] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16638worker1 # [ 296.661873] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986639worker1 # [ 296.668313] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=want client=2 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06640worker1 # [ 296.674733] nix-daemon[855]: reaped child process 1992, status = succeeded6641worker1 # [ 296.678687] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986642worker1 # [ 296.686343] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986643worker1 # [ 296.696295] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6644worker1 # [ 296.828832] nix-daemon[855]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6645worker1 # [ 296.847113] nix-daemon[855]: reaped child process 2022, status = succeeded6646lb # [ 296.912394] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received uploads request method=POST path=/api/pending_closures6647worker1 # [ 296.858873] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.494Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6648worker1 # [ 296.860407] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.494Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6649lb # [ 296.940579] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6650worker1 # [ 296.884116] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.519Z level=INFO msg="Uploading 1 narinfos"6651lb # [ 296.953737] niks3-server[1211]: 2026/09/23 05:37:49 INFO Signed narinfos id=72 count=16652lb # [ 296.961281] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6653lb # [ 296.973481] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6654worker1 # [ 296.916411] nix-grpc-daemon[1880]: time=2026-09-23T05:37:49.552Z level=INFO msg="Upload complete. (209ms)"6655lb # [ 296.979249] niks3-server[1211]: 2026/09/23 05:37:49 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6656lb # [ 296.984408] niks3-server[1211]: 2026/09/23 05:37:49 INFO Completed upload id=726657worker1 # [ 296.924257] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=5126658lb # [ 296.987522] niks3-server[1211]: 2026/09/23 05:37:49 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6659worker1 # [ 296.937170] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986660worker1 # [ 296.943039] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6661worker1 # [ 296.947815] nix-daemon[855]: reaped child process 2017, status = succeeded6662worker1 # [ 296.958516] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6663worker1 # [ 297.799258] nix-daemon[855]: reaped child process 2026, status = succeeded6664worker1 # [ 297.923088] nix-daemon[855]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)6665worker1 # [ 297.938815] nix-daemon[855]: reaped child process 2045, status = succeeded6666lb # [ 298.010271] niks3-server[1211]: 2026/09/23 05:37:50 INFO Received uploads request method=POST path=/api/pending_closures6667worker1 # [ 297.958984] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.592Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6668worker1 # [ 297.960678] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.592Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6669lb # [ 298.041715] niks3-server[1211]: 2026/09/23 05:37:50 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6670worker1 # [ 297.998606] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.634Z level=INFO msg="Uploading 1 narinfos"6671lb # [ 298.060498] niks3-server[1211]: 2026/09/23 05:37:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6672lb # [ 298.072496] niks3-server[1211]: 2026/09/23 05:37:50 INFO Signed narinfos id=73 count=16673lb # [ 298.079826] niks3-server[1211]: 2026/09/23 05:37:50 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6674lb # [ 298.095062] niks3-server[1211]: 2026/09/23 05:37:50 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6675worker1 # [ 298.038911] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.674Z level=INFO msg="Upload complete. (268ms)"6676lb # [ 298.104203] niks3-server[1211]: 2026/09/23 05:37:50 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6677lb # [ 298.111726] niks3-server[1211]: 2026/09/23 05:37:50 INFO Completed upload id=736678lb # [ 298.117464] niks3-server[1211]: 2026/09/23 05:37:50 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6679worker1 # [ 298.057177] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=6 outputs=16680worker1 # [ 298.079802] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16681worker1 # [ 298.085791] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986682worker1 # [ 298.090475] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=want client=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06683worker1 # [ 298.096209] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986684worker1 # [ 298.108348] nix-daemon[855]: reaped child process 2024, status = succeeded6685worker1 # [ 298.110548] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6686worker1 # [ 298.119509] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986687worker1 # [ 298.261802] nix-daemon[855]: accepted connection from pid 2051, user nix-grpc-daemon (trusted)6688worker1 # [ 298.279298] nix-daemon[855]: reaped child process 2054, status = succeeded6689lb # [ 298.345074] niks3-server[1211]: 2026/09/23 05:37:50 INFO Received uploads request method=POST path=/api/pending_closures6690worker1 # [ 298.291649] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.927Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6691worker1 # [ 298.293286] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.927Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6692lb # [ 298.375175] niks3-server[1211]: 2026/09/23 05:37:51 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6693worker1 # [ 298.323904] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.959Z level=INFO msg="Uploading 1 narinfos"6694lb # [ 298.389800] niks3-server[1211]: 2026/09/23 05:37:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6695lb # [ 298.401685] niks3-server[1211]: 2026/09/23 05:37:51 INFO Signed narinfos id=74 count=16696lb # [ 298.407548] niks3-server[1211]: 2026/09/23 05:37:51 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6697worker1 # [ 298.356371] nix-grpc-daemon[1880]: time=2026-09-23T05:37:50.992Z level=INFO msg="Upload complete. (232ms)"6698lb # [ 298.421067] niks3-server[1211]: 2026/09/23 05:37:51 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6699worker1 # [ 298.364252] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=5126700lb # [ 298.428583] niks3-server[1211]: 2026/09/23 05:37:51 INFO Completed upload id=746701lb # [ 298.432915] niks3-server[1211]: 2026/09/23 05:37:51 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6702worker1 # [ 298.379495] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986703worker1 # [ 298.388103] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6704worker1 # [ 298.391025] nix-daemon[855]: reaped child process 2049, status = succeeded6705worker1 # [ 298.401634] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6706lb # [ 299.390869] envoy[907]: conn peer=[2001:db8:1::1]:34334 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9649 tx=5203 ms=63146707worker1 # [ 299.664308] nix-daemon[855]: reaped child process 2058, status = succeeded6708worker1 # [ 299.749235] nix-daemon[855]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6709worker1 # [ 299.767842] nix-daemon[855]: reaped child process 2077, status = succeeded6710lb # [ 299.829639] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received uploads request method=POST path=/api/pending_closures6711worker1 # [ 299.776689] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.412Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6712worker1 # [ 299.778162] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.412Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6713worker1 # [ 299.820626] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.456Z level=INFO msg="Uploading 1 narinfos"6714lb # [ 299.873210] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6715lb # [ 299.891768] niks3-server[1211]: 2026/09/23 05:37:52 INFO Signed narinfos id=75 count=16716lb # [ 299.899588] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6717lb # [ 299.913399] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6718worker1 # [ 299.859036] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.494Z level=INFO msg="Upload complete. (245ms)"6719lb # [ 299.923831] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6720lb # [ 299.929395] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6721lb # [ 299.935450] niks3-server[1211]: 2026/09/23 05:37:52 INFO Completed upload id=756722lb # [ 299.939307] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6723worker1 # [ 299.876117] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=7 outputs=16724worker1 # [ 299.895620] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16725worker1 # [ 299.901255] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986726worker1 # [ 299.906838] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=want client=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06727worker1 # [ 299.912519] nix-daemon[855]: reaped child process 2056, status = succeeded6728worker1 # [ 299.916113] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986729worker1 # [ 299.921280] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.555Z level=INFO msg="All 1 paths already cached"6730worker1 # [ 299.925200] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986731worker1 # [ 299.938228] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6732worker1 # [ 300.065121] nix-daemon[855]: accepted connection from pid 2083, user nix-grpc-daemon (trusted)6733worker1 # [ 300.082111] nix-daemon[855]: reaped child process 2086, status = succeeded6734lb # [ 300.150948] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received uploads request method=POST path=/api/pending_closures6735worker1 # [ 300.097882] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.733Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6736worker1 # [ 300.099376] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.733Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6737lb # [ 300.181217] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6738worker1 # [ 300.124782] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.760Z level=INFO msg="Uploading 1 narinfos"6739lb # [ 300.192943] niks3-server[1211]: 2026/09/23 05:37:52 INFO Signed narinfos id=76 count=16740lb # [ 300.201920] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6741worker1 # [ 300.151251] nix-grpc-daemon[1880]: time=2026-09-23T05:37:52.786Z level=INFO msg="Upload complete. (205ms)"6742lb # [ 300.213697] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6743worker1 # [ 300.157539] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=6246744lb # [ 300.221959] niks3-server[1211]: 2026/09/23 05:37:52 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6745lb # [ 300.225904] niks3-server[1211]: 2026/09/23 05:37:52 INFO Completed upload id=766746lb # [ 300.228349] niks3-server[1211]: 2026/09/23 05:37:52 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6747worker1 # [ 300.170988] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986748worker1 # [ 300.178868] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6749worker1 # [ 300.181594] nix-daemon[855]: reaped child process 2081, status = succeeded6750worker1 # [ 300.194666] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6751worker1 # [ 301.654894] nix-daemon[855]: reaped child process 2090, status = succeeded6752worker1 # [ 301.788461] nix-daemon[855]: accepted connection from pid 2106, user nix-grpc-daemon (trusted)6753worker1 # [ 301.807416] nix-daemon[855]: reaped child process 2109, status = succeeded6754lb # [ 301.876174] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received uploads request method=POST path=/api/pending_closures6755worker1 # [ 301.824448] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.457Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6756worker1 # [ 301.826240] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.457Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6757lb # [ 301.920770] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6758worker1 # [ 301.866946] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.502Z level=INFO msg="Uploading 1 narinfos"6759lb # [ 301.936921] niks3-server[1211]: 2026/09/23 05:37:54 INFO Signed narinfos id=77 count=16760lb # [ 301.939500] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6761lb # [ 301.955304] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6762lb # [ 301.959828] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6763worker1 # [ 301.893490] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.529Z level=INFO msg="Upload complete. (288ms)"6764lb # [ 301.963472] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6765lb # [ 301.967154] niks3-server[1211]: 2026/09/23 05:37:54 INFO Completed upload id=776766lb # [ 301.970512] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6767worker1 # [ 301.913405] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=8 outputs=16768worker1 # [ 301.928311] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16769worker1 # [ 301.935295] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986770worker1 # [ 301.940662] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=want client=2 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06771worker1 # [ 301.945402] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986772worker1 # [ 301.950256] nix-daemon[855]: reaped child process 2088, status = succeeded6773worker1 # [ 301.952185] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.584Z level=INFO msg="All 1 paths already cached"6774worker1 # [ 301.966665] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986775worker1 # [ 301.972745] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6776worker1 # [ 302.095235] nix-daemon[855]: accepted connection from pid 2115, user nix-grpc-daemon (trusted)6777worker1 # [ 302.113477] nix-daemon[855]: reaped child process 2118, status = succeeded6778lb # [ 302.180592] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received uploads request method=POST path=/api/pending_closures6779worker1 # [ 302.127986] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.763Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6780worker1 # [ 302.129932] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.763Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6781lb # [ 302.210455] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6782worker1 # [ 302.155349] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.790Z level=INFO msg="Uploading 1 narinfos"6783lb # [ 302.226182] niks3-server[1211]: 2026/09/23 05:37:54 INFO Signed narinfos id=78 count=16784lb # [ 302.233565] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6785lb # [ 302.240556] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6786worker1 # [ 302.188700] nix-grpc-daemon[1880]: time=2026-09-23T05:37:54.824Z level=INFO msg="Upload complete. (213ms)"6787lb # [ 302.251956] niks3-server[1211]: 2026/09/23 05:37:54 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6788lb # [ 302.258649] niks3-server[1211]: 2026/09/23 05:37:54 INFO Completed upload id=786789worker1 # [ 302.196307] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=6246790lb # [ 302.262097] niks3-server[1211]: 2026/09/23 05:37:54 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6791worker1 # [ 302.210224] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986792worker1 # [ 302.219778] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6793worker1 # [ 302.224649] nix-daemon[855]: reaped child process 2113, status = succeeded6794worker1 # [ 302.238806] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6795worker1 # [ 303.699346] nix-daemon[855]: reaped child process 2122, status = succeeded6796worker1 # [ 303.826122] nix-daemon[855]: accepted connection from pid 2138, user nix-grpc-daemon (trusted)6797worker1 # [ 303.841870] nix-daemon[855]: reaped child process 2141, status = succeeded6798lb # [ 303.910951] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received uploads request method=POST path=/api/pending_closures6799worker1 # [ 303.864474] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.495Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6800worker1 # [ 303.866318] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.495Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6801lb # [ 303.964556] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6802worker1 # [ 303.914300] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.549Z level=INFO msg="Uploading 1 narinfos"6803lb # [ 303.979146] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6804lb # [ 303.987403] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6805lb # [ 303.998450] niks3-server[1211]: 2026/09/23 05:37:56 INFO Signed narinfos id=79 count=16806worker1 # [ 303.935264] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.570Z level=INFO msg="Upload complete. (286ms)"6807lb # [ 304.004831] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6808lb # [ 304.008939] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6809lb # [ 304.013988] niks3-server[1211]: 2026/09/23 05:37:56 INFO Completed upload id=796810lb # [ 304.016000] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6811worker1 # [ 303.950752] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=9 outputs=16812worker1 # [ 303.972603] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16813worker1 # [ 303.980800] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986814worker1 # [ 303.988946] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=want client=2 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06815worker1 # [ 303.996437] nix-daemon[855]: reaped child process 2120, status = succeeded6816worker1 # [ 304.000673] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986817worker1 # [ 304.007098] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.640Z level=INFO msg="All 1 paths already cached"6818worker1 # [ 304.011314] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986819worker1 # [ 304.024101] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6820worker1 # [ 304.155386] nix-daemon[855]: accepted connection from pid 2147, user nix-grpc-daemon (trusted)6821worker1 # [ 304.170621] nix-daemon[855]: reaped child process 2150, status = succeeded6822lb # [ 304.239703] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received uploads request method=POST path=/api/pending_closures6823worker1 # [ 304.186361] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.821Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6824worker1 # [ 304.187816] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.821Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6825lb # [ 304.268490] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6826worker1 # [ 304.215825] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.851Z level=INFO msg="Uploading 1 narinfos"6827lb # [ 304.283295] niks3-server[1211]: 2026/09/23 05:37:56 INFO Signed narinfos id=80 count=16828lb # [ 304.291546] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6829worker1 # [ 304.240086] nix-grpc-daemon[1880]: time=2026-09-23T05:37:56.875Z level=INFO msg="Upload complete. (205ms)"6830lb # [ 304.302045] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6831worker1 # [ 304.247979] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=6246832lb # [ 304.310871] niks3-server[1211]: 2026/09/23 05:37:56 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete6833lb # [ 304.314700] niks3-server[1211]: 2026/09/23 05:37:56 INFO Completed upload id=806834lb # [ 304.317237] niks3-server[1211]: 2026/09/23 05:37:56 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6835worker1 # [ 304.259448] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986836worker1 # [ 304.267826] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6837worker1 # [ 304.270633] nix-daemon[855]: reaped child process 2145, status = succeeded6838worker1 # [ 304.282722] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6839worker1 # [ 305.549053] nix-daemon[855]: reaped child process 2154, status = succeeded6840worker1 # [ 305.657998] nix-daemon[855]: accepted connection from pid 2170, user nix-grpc-daemon (trusted)6841worker1 # [ 305.676384] nix-daemon[855]: reaped child process 2173, status = succeeded6842lb # [ 305.737994] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received uploads request method=POST path=/api/pending_closures6843worker1 # [ 305.688420] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.322Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6844worker1 # [ 305.690039] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.322Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6845lb # [ 305.783319] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6846worker1 # [ 305.730516] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.365Z level=INFO msg="Uploading 1 narinfos"6847lb # [ 305.795311] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6848lb # [ 305.803894] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign6849lb # [ 305.811924] niks3-server[1211]: 2026/09/23 05:37:58 INFO Signed narinfos id=81 count=16850worker1 # [ 305.749953] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.385Z level=INFO msg="Upload complete. (258ms)"6851lb # [ 305.816220] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6852lb # [ 305.821913] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete6853lb # [ 305.825725] niks3-server[1211]: 2026/09/23 05:37:58 INFO Completed upload id=816854lb # [ 305.827230] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6855worker1 # [ 305.762515] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=10 outputs=16856worker1 # [ 305.783261] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16857worker1 # [ 305.789868] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986858worker1 # [ 305.795339] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=want client=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06859worker1 # [ 305.800627] nix-daemon[855]: reaped child process 2152, status = succeeded6860worker1 # [ 305.802714] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986861worker1 # [ 305.807826] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.442Z level=INFO msg="All 2 paths already cached"6862worker1 # [ 305.811653] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986863worker1 # [ 305.825888] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6864worker1 # [ 305.944239] nix-daemon[855]: accepted connection from pid 2179, user nix-grpc-daemon (trusted)6865worker1 # [ 305.961087] nix-daemon[855]: reaped child process 2182, status = succeeded6866lb # [ 306.028676] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received uploads request method=POST path=/api/pending_closures6867worker1 # [ 305.975833] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.611Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6868worker1 # [ 305.977507] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.611Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6869lb # [ 306.060352] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6870worker1 # [ 306.008167] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.643Z level=INFO msg="Uploading 1 narinfos"6871lb # [ 306.070040] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign6872lb # [ 306.080842] niks3-server[1211]: 2026/09/23 05:37:58 INFO Signed narinfos id=82 count=16873lb # [ 306.086645] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6874worker1 # [ 306.036031] nix-grpc-daemon[1880]: time=2026-09-23T05:37:58.671Z level=INFO msg="Upload complete. (205ms)"6875lb # [ 306.097996] niks3-server[1211]: 2026/09/23 05:37:58 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete6876lb # [ 306.107290] niks3-server[1211]: 2026/09/23 05:37:58 INFO Completed upload id=826877worker1 # [ 306.044227] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=7526878lb # [ 306.109932] niks3-server[1211]: 2026/09/23 05:37:58 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6879worker1 # [ 306.057257] nix-grpc-daemon[1808]: ts=2026-09-23T05:37:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986880worker1 # [ 306.065195] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6881worker1 # [ 306.068566] nix-daemon[855]: reaped child process 2177, status = succeeded6882worker1 # [ 306.079538] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6883worker1 # [ 307.583585] nix-daemon[855]: reaped child process 2186, status = succeeded6884worker1 # [ 307.655309] nix-daemon[855]: accepted connection from pid 2202, user nix-grpc-daemon (trusted)6885worker1 # [ 307.673342] nix-daemon[855]: reaped child process 2205, status = succeeded6886lb # [ 307.741935] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received uploads request method=POST path=/api/pending_closures6887worker1 # [ 307.688404] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.322Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6888worker1 # [ 307.690110] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.323Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6889lb # [ 307.785900] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6890worker1 # [ 307.736503] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.371Z level=INFO msg="Uploading 1 narinfos"6891lb # [ 307.801738] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6892lb # [ 307.811240] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign6893lb # [ 307.821246] niks3-server[1211]: 2026/09/23 05:38:00 INFO Signed narinfos id=83 count=16894worker1 # [ 307.758615] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.394Z level=INFO msg="Upload complete. (269ms)"6895lb # [ 307.823440] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6896lb # [ 307.829957] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete6897lb # [ 307.835461] niks3-server[1211]: 2026/09/23 05:38:00 INFO Completed upload id=836898lb # [ 307.838660] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6899worker1 # [ 307.774289] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=11 outputs=16900worker1 # [ 307.791326] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16901worker1 # [ 307.797079] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986902worker1 # [ 307.800570] nix-daemon[855]: reaped child process 2184, status = succeeded6903worker1 # [ 307.818130] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986904worker1 # [ 307.831284] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.466Z level=INFO msg="All 2 paths already cached"6905worker1 # [ 307.841285] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986906worker1 # [ 307.855399] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6907worker1 # [ 308.004220] nix-daemon[855]: accepted connection from pid 2209, user nix-grpc-daemon (trusted)6908worker1 # [ 308.020197] nix-daemon[855]: reaped child process 2213, status = succeeded6909lb # [ 308.090005] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received uploads request method=POST path=/api/pending_closures6910worker1 # [ 308.039686] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.674Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6911worker1 # [ 308.041862] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.675Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6912worker1 # [ 308.065937] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.701Z level=INFO msg="Uploading 1 narinfos"6913lb # [ 308.122192] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign6914lb # [ 308.133625] niks3-server[1211]: 2026/09/23 05:38:00 INFO Signed narinfos id=84 count=16915lb # [ 308.144444] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6916worker1 # [ 308.096155] nix-grpc-daemon[1880]: time=2026-09-23T05:38:00.731Z level=INFO msg="Upload complete. (227ms)"6917lb # [ 308.160610] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6918lb # [ 308.167519] niks3-server[1211]: 2026/09/23 05:38:00 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete6919lb # [ 308.175987] niks3-server[1211]: 2026/09/23 05:38:00 INFO Completed upload id=846920worker1 # [ 308.104353] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=7526921lb # [ 308.178761] niks3-server[1211]: 2026/09/23 05:38:00 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6922worker1 # [ 308.121136] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986923worker1 # [ 308.128531] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6924worker1 # [ 308.132540] nix-daemon[855]: reaped child process 2207, status = succeeded6925worker1 # [ 308.144145] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6926worker1 # [ 308.995336] nix-daemon[855]: reaped child process 2217, status = succeeded6927worker1 # [ 309.102092] nix-daemon[855]: accepted connection from pid 2233, user nix-grpc-daemon (trusted)6928worker1 # [ 309.120422] nix-daemon[855]: reaped child process 2236, status = succeeded6929lb # [ 309.187707] niks3-server[1211]: 2026/09/23 05:38:01 INFO Received uploads request method=POST path=/api/pending_closures6930worker1 # [ 309.137414] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.771Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6931worker1 # [ 309.138984] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.771Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6932lb # [ 309.223977] niks3-server[1211]: 2026/09/23 05:38:01 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6933lb # [ 309.235952] niks3-server[1211]: 2026/09/23 05:38:01 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6934lb # [ 309.248174] niks3-server[1211]: 2026/09/23 05:38:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign6935worker1 # [ 309.191814] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.827Z level=INFO msg="Uploading 1 narinfos"6936lb # [ 309.258365] niks3-server[1211]: 2026/09/23 05:38:01 INFO Signed narinfos id=85 count=16937lb # [ 309.265492] niks3-server[1211]: 2026/09/23 05:38:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6938worker1 # [ 309.215482] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.851Z level=INFO msg="Upload complete. (261ms)"6939lb # [ 309.278279] niks3-server[1211]: 2026/09/23 05:38:01 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete6940lb # [ 309.286314] niks3-server[1211]: 2026/09/23 05:38:01 INFO Completed upload id=856941lb # [ 309.289651] niks3-server[1211]: 2026/09/23 05:38:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6942worker1 # [ 309.229313] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=12 outputs=16943worker1 # [ 309.248263] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16944worker1 # [ 309.255342] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986945worker1 # [ 309.259913] nix-daemon[855]: reaped child process 2215, status = succeeded6946worker1 # [ 309.274254] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407986947worker1 # [ 309.286907] nix-grpc-daemon[1880]: time=2026-09-23T05:38:01.922Z level=INFO msg="All 2 paths already cached"6948worker1 # [ 309.296928] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407986949worker1 # [ 309.313387] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6950worker1 # [ 309.449266] nix-daemon[855]: accepted connection from pid 2240, user nix-grpc-daemon (trusted)6951worker1 # [ 309.466925] nix-daemon[855]: reaped child process 2243, status = succeeded6952lb # [ 309.534496] niks3-server[1211]: 2026/09/23 05:38:02 INFO Received uploads request method=POST path=/api/pending_closures6953worker1 # [ 309.480952] nix-grpc-daemon[1880]: time=2026-09-23T05:38:02.116Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6954worker1 # [ 309.482476] nix-grpc-daemon[1880]: time=2026-09-23T05:38:02.116Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6955lb # [ 309.562498] niks3-server[1211]: 2026/09/23 05:38:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign6956worker1 # [ 309.508050] nix-grpc-daemon[1880]: time=2026-09-23T05:38:02.143Z level=INFO msg="Uploading 1 narinfos"6957lb # [ 309.576188] niks3-server[1211]: 2026/09/23 05:38:02 INFO Signed narinfos id=86 count=16958lb # [ 309.579373] niks3-server[1211]: 2026/09/23 05:38:02 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6959lb # [ 309.597843] niks3-server[1211]: 2026/09/23 05:38:02 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete6960worker1 # [ 309.540220] nix-grpc-daemon[1880]: time=2026-09-23T05:38:02.175Z level=INFO msg="Upload complete. (216ms)"6961lb # [ 309.602170] niks3-server[1211]: 2026/09/23 05:38:02 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6962lb # [ 309.605991] niks3-server[1211]: 2026/09/23 05:38:02 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6963lb # [ 309.616831] niks3-server[1211]: 2026/09/23 05:38:02 INFO Completed upload id=866964worker1 # [ 309.548153] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=0 paths=1 nar_bytes_in=7526965worker1 # [ 309.562464] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407986966worker1 # [ 309.571671] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6967worker1 # [ 309.575037] nix-daemon[855]: reaped child process 2238, status = succeeded6968worker1 # [ 309.585074] nix-daemon[855]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6969worker1 # [ 310.492957] nix-daemon[855]: reaped child process 2247, status = succeeded6970worker1 # [ 310.562918] nix-daemon[855]: accepted connection from pid 2263, user nix-grpc-daemon (trusted)6971worker1 # [ 310.577930] nix-daemon[855]: reaped child process 2266, status = succeeded6972lb # [ 310.649075] niks3-server[1211]: 2026/09/23 05:38:03 INFO Received uploads request method=POST path=/api/pending_closures6973worker1 # [ 310.599439] nix-grpc-daemon[1880]: time=2026-09-23T05:38:03.232Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6974worker1 # [ 310.601350] nix-grpc-daemon[1880]: time=2026-09-23T05:38:03.232Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6975lb # [ 310.691071] niks3-server[1211]: 2026/09/23 05:38:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign6976worker1 # [ 310.637835] nix-grpc-daemon[1880]: time=2026-09-23T05:38:03.273Z level=INFO msg="Uploading 1 narinfos"6977lb # [ 310.706634] niks3-server[1211]: 2026/09/23 05:38:03 INFO Signed narinfos id=87 count=16978lb # [ 310.712992] niks3-server[1211]: 2026/09/23 05:38:03 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6979lb # [ 310.721193] niks3-server[1211]: 2026/09/23 05:38:03 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6980lb # [ 310.730778] niks3-server[1211]: 2026/09/23 05:38:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6981worker1 # [ 310.668492] nix-grpc-daemon[1880]: time=2026-09-23T05:38:03.304Z level=INFO msg="Upload complete. (271ms)"6982lb # [ 310.734452] niks3-server[1211]: 2026/09/23 05:38:03 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete6983lb # [ 310.738313] niks3-server[1211]: 2026/09/23 05:38:03 INFO Completed upload id=876984lb # [ 310.740201] niks3-server[1211]: 2026/09/23 05:38:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6985worker1 # [ 310.681108] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40798 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=13 outputs=16986worker1 # [ 310.697596] nix-daemon[855]: reaped child process 2245, status = succeeded6987client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.13 seconds)6988worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6989worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6990client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6991client # [ 310.924904] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6992client # Running as unit: gpu.service; invocation ID: 2a609e6589e44209a703c306f76fad596993client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6994lb: must succeed: curl -sf localhost:9901/clusters6995lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6996lb: must succeed: curl -sf localhost:9901/clusters6997lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6998worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6999worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7000client # [ 311.137232] nix[2056]: don't know how to build these paths:7001client # [ 311.138348] nix[2056]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7002worker2 # [ 311.138824] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:606967003worker1 # [ 311.134083] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407987004client # [ 311.167270] nix[2056]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker7005worker1 # [ 311.158300] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:03Z level=debug event=want client=3 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07006lb: must succeed: curl -sf localhost:9901/clusters7007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7008lb: must succeed: curl -sf localhost:9901/clusters7009lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7010worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7011worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7012client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7013lb # [ 314.877359] postgres[786]: [786] LOG: checkpoint starting: time7014lb # [ 319.392580] envoy[907]: conn peer=[2001:db8:1::1]:48498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27140 tx=12300 ms=158627015client # [ 341.173525] nix[2056]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}7016client # [ 341.186227] nix[2056]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7017client # [ 341.224841] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7018client # [ 341.231964] systemd[1]: gpu.service: Failed with result 'exit-code'.7019client # [ 341.236355] systemd[1]: gpu.service: Consumed 204ms CPU time over 30.305s wall clock time, 37.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7020client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.09 seconds)7021client: must succeed: journalctl -u gpu -o cat7022client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7023lb: must succeed: curl -sf localhost:9901/clusters7024lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7025lb: must succeed: curl -sf localhost:9901/clusters7026lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7027worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7028worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7029(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.94 seconds)7030subtest: low disk drains a worker and builds go to the other7031worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7032worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.16 seconds)7033worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7034worker1 # [ 343.155685] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:35Z level=info event=unhealthy reason=min_free7035worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)7036client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27037client # don't know how to build these paths:7038client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7039worker2 # [ 344.094103] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:606967040worker2 # [ 344.102749] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606967041worker1 # [ 344.115277] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:36Z level=debug event=want client=4 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07042worker1 # [ 344.115697] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:36Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17043worker2 # [ 344.171895] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606967044worker2 # [ 344.187471] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606967045client # copying 1 paths to 'lb:50051'...7046client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7047worker2 # [ 344.202965] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606967048worker2 # [ 344.218010] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)7049worker2 # [ 344.383093] nix-daemon[844]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)7050worker2 # [ 344.402906] nix-daemon[844]: reaped child process 1888, status = succeeded7051lb # [ 344.461276] niks3-server[1211]: 2026/09/23 05:38:37 INFO Received uploads request method=POST path=/api/pending_closures7052worker2 # [ 344.422577] nix-grpc-daemon[1638]: time=2026-09-23T05:38:37.050Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7053worker2 # [ 344.424288] nix-grpc-daemon[1638]: time=2026-09-23T05:38:37.050Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7054lb # [ 344.496133] niks3-server[1211]: 2026/09/23 05:38:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7055worker2 # [ 344.453402] nix-grpc-daemon[1638]: time=2026-09-23T05:38:37.081Z level=INFO msg="Uploading 1 narinfos"7056lb # [ 344.510426] niks3-server[1211]: 2026/09/23 05:38:37 INFO Signed narinfos id=88 count=17057lb # [ 344.523482] niks3-server[1211]: 2026/09/23 05:38:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7058lb # [ 344.532103] niks3-server[1211]: 2026/09/23 05:38:37 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7059lb # [ 344.551707] niks3-server[1211]: 2026/09/23 05:38:37 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7060lb # [ 344.562491] niks3-server[1211]: 2026/09/23 05:38:37 INFO Completed upload id=887061worker2 # [ 344.509575] nix-grpc-daemon[1638]: time=2026-09-23T05:38:37.137Z level=INFO msg="Upload complete. (274ms)"7062lb # [ 344.566465] niks3-server[1211]: 2026/09/23 05:38:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7063worker2 # [ 344.520353] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=5127064worker2 # [ 344.537514] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606967065worker2 # [ 344.550513] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)7066worker2 # [ 344.555007] nix-daemon[844]: reaped child process 1883, status = succeeded7067client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7068client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7069worker2 # [ 344.569698] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)7070client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7071client # farm-a> LOG-farm-a-drain7072worker2 # [ 345.827984] nix-daemon[844]: reaped child process 1892, status = succeeded7073worker2 # [ 345.939661] nix-daemon[844]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)7074worker2 # [ 345.960852] nix-daemon[844]: reaped child process 1911, status = succeeded7075lb # [ 346.014617] niks3-server[1211]: 2026/09/23 05:38:38 INFO Received uploads request method=POST path=/api/pending_closures7076worker2 # [ 345.973827] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.600Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7077worker2 # [ 345.975491] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.600Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7078lb # [ 346.052148] niks3-server[1211]: 2026/09/23 05:38:38 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7079worker2 # [ 346.011100] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.639Z level=INFO msg="Uploading 1 narinfos"7080lb # [ 346.064117] niks3-server[1211]: 2026/09/23 05:38:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7081lb # [ 346.067826] niks3-server[1211]: 2026/09/23 05:38:38 INFO Signed narinfos id=89 count=17082lb # [ 346.078680] niks3-server[1211]: 2026/09/23 05:38:38 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7083lb # [ 346.088352] niks3-server[1211]: 2026/09/23 05:38:38 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7084lb # [ 346.093745] niks3-server[1211]: 2026/09/23 05:38:38 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7085worker2 # [ 346.041695] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.669Z level=INFO msg="Upload complete. (257ms)"7086lb # [ 346.096980] niks3-server[1211]: 2026/09/23 05:38:38 INFO Completed upload id=897087lb # [ 346.100980] niks3-server[1211]: 2026/09/23 05:38:38 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7088worker2 # [ 346.053143] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=14 outputs=17089worker1 # [ 346.050170] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:38Z level=debug event=want client=4 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07090worker1 # [ 346.056162] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:38Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17091worker2 # [ 346.072875] nix-daemon[844]: reaped child process 1890, status = succeeded7092worker2 # [ 346.104582] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606967093worker2 # [ 346.119604] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606967094worker2 # [ 346.133697] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.761Z level=INFO msg="All 1 paths already cached"7095client # copying 1 paths to 'lb:50051'...7096client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7097worker2 # [ 346.144119] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606967098worker2 # [ 346.157144] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)7099worker2 # [ 346.313803] nix-daemon[844]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)7100worker2 # [ 346.329346] nix-daemon[844]: reaped child process 1918, status = succeeded7101lb # [ 346.387781] niks3-server[1211]: 2026/09/23 05:38:39 INFO Received uploads request method=POST path=/api/pending_closures7102worker2 # [ 346.345985] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.973Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7103worker2 # [ 346.347689] nix-grpc-daemon[1638]: time=2026-09-23T05:38:38.974Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7104lb # [ 346.418239] niks3-server[1211]: 2026/09/23 05:38:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7105worker2 # [ 346.372322] nix-grpc-daemon[1638]: time=2026-09-23T05:38:39.000Z level=INFO msg="Uploading 1 narinfos"7106lb # [ 346.432614] niks3-server[1211]: 2026/09/23 05:38:39 INFO Signed narinfos id=90 count=17107lb # [ 346.439147] niks3-server[1211]: 2026/09/23 05:38:39 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7108lb # [ 346.451593] niks3-server[1211]: 2026/09/23 05:38:39 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7109worker2 # [ 346.405657] nix-grpc-daemon[1638]: time=2026-09-23T05:38:39.033Z level=INFO msg="Upload complete. (232ms)"7110lb # [ 346.458878] niks3-server[1211]: 2026/09/23 05:38:39 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7111lb # [ 346.469085] niks3-server[1211]: 2026/09/23 05:38:39 INFO Completed upload id=907112worker2 # [ 346.414805] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=6327113lb # [ 346.473318] niks3-server[1211]: 2026/09/23 05:38:39 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7114worker2 # [ 346.430089] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606967115worker2 # [ 346.439274] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)7116worker2 # [ 346.442201] nix-daemon[844]: reaped child process 1913, status = succeeded7117client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7118worker2 # [ 346.453139] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)7119client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7120client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7121client # farm-b> LOG-farm-b-drain7122worker2 # [ 348.081109] nix-daemon[844]: reaped child process 1922, status = succeeded7123worker2 # [ 348.179882] nix-daemon[844]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)7124worker2 # [ 348.199837] nix-daemon[844]: reaped child process 1941, status = succeeded7125lb # [ 348.251786] niks3-server[1211]: 2026/09/23 05:38:40 INFO Received uploads request method=POST path=/api/pending_closures7126worker2 # [ 348.212363] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.838Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7127worker2 # [ 348.213791] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.838Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7128lb # [ 348.294591] niks3-server[1211]: 2026/09/23 05:38:40 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7129worker2 # [ 348.255607] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.883Z level=INFO msg="Uploading 1 narinfos"7130lb # [ 348.308391] niks3-server[1211]: 2026/09/23 05:38:40 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7131lb # [ 348.320355] niks3-server[1211]: 2026/09/23 05:38:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7132lb # [ 348.327106] niks3-server[1211]: 2026/09/23 05:38:40 INFO Signed narinfos id=91 count=17133worker2 # [ 348.281914] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.910Z level=INFO msg="Upload complete. (216ms)"7134lb # [ 348.336915] niks3-server[1211]: 2026/09/23 05:38:40 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7135lb # [ 348.342281] niks3-server[1211]: 2026/09/23 05:38:40 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7136worker2 # [ 348.292853] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=15 outputs=17137lb # [ 348.350982] niks3-server[1211]: 2026/09/23 05:38:40 INFO Completed upload id=917138lb # [ 348.353993] niks3-server[1211]: 2026/09/23 05:38:40 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7139worker2 # [ 348.305810] nix-daemon[844]: reaped child process 1920, status = succeeded7140worker1 # [ 348.300756] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:40Z level=debug event=want client=4 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07141worker2 # [ 348.312906] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606967142worker1 # [ 348.306552] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:40Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17143worker2 # [ 348.324957] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606967144worker2 # [ 348.336811] nix-grpc-daemon[1638]: time=2026-09-23T05:38:40.965Z level=INFO msg="All 2 paths already cached"7145client # copying 1 paths to 'lb:50051'...7146client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7147worker2 # [ 348.346836] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606967148worker2 # [ 348.358609] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)7149worker2 # [ 348.498030] nix-daemon[844]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)7150worker2 # [ 348.516581] nix-daemon[844]: reaped child process 1948, status = succeeded7151lb # [ 348.574086] niks3-server[1211]: 2026/09/23 05:38:41 INFO Received uploads request method=POST path=/api/pending_closures7152worker2 # [ 348.530570] nix-grpc-daemon[1638]: time=2026-09-23T05:38:41.158Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7153worker2 # [ 348.532143] nix-grpc-daemon[1638]: time=2026-09-23T05:38:41.158Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7154lb # [ 348.603180] niks3-server[1211]: 2026/09/23 05:38:41 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7155worker2 # [ 348.559548] nix-grpc-daemon[1638]: time=2026-09-23T05:38:41.187Z level=INFO msg="Uploading 1 narinfos"7156lb # [ 348.617354] niks3-server[1211]: 2026/09/23 05:38:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7157lb # [ 348.627222] niks3-server[1211]: 2026/09/23 05:38:41 INFO Signed narinfos id=92 count=17158lb # [ 348.634966] niks3-server[1211]: 2026/09/23 05:38:41 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7159worker2 # [ 348.589557] nix-grpc-daemon[1638]: time=2026-09-23T05:38:41.217Z level=INFO msg="Upload complete. (218ms)"7160lb # [ 348.643714] niks3-server[1211]: 2026/09/23 05:38:41 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7161lb # [ 348.648227] niks3-server[1211]: 2026/09/23 05:38:41 INFO Completed upload id=927162worker2 # [ 348.596255] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=7607163lb # [ 348.651464] niks3-server[1211]: 2026/09/23 05:38:41 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7164worker2 # [ 348.609760] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606967165worker2 # [ 348.618866] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)7166worker2 # [ 348.621272] nix-daemon[844]: reaped child process 1943, status = succeeded7167client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7168client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7169worker2 # [ 348.634439] nix-daemon[844]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)7170lb # [ 349.393247] envoy[907]: conn peer=[2001:db8:1::1]:38452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300677171client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7172client # farm-top> LOG-farm-top-drain7173worker2 # [ 349.689164] nix-daemon[844]: reaped child process 1952, status = succeeded7174worker2 # [ 349.805980] nix-daemon[844]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)7175worker2 # [ 349.824755] nix-daemon[844]: reaped child process 1971, status = succeeded7176lb # [ 349.875720] niks3-server[1211]: 2026/09/23 05:38:42 INFO Received uploads request method=POST path=/api/pending_closures7177worker2 # [ 349.833788] nix-grpc-daemon[1638]: time=2026-09-23T05:38:42.460Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7178worker2 # [ 349.835218] nix-grpc-daemon[1638]: time=2026-09-23T05:38:42.460Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7179lb # [ 349.915777] niks3-server[1211]: 2026/09/23 05:38:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7180worker2 # [ 349.870073] nix-grpc-daemon[1638]: time=2026-09-23T05:38:42.498Z level=INFO msg="Uploading 1 narinfos"7181lb # [ 349.931723] niks3-server[1211]: 2026/09/23 05:38:42 INFO Signed narinfos id=93 count=17182lb # [ 349.938060] niks3-server[1211]: 2026/09/23 05:38:42 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7183lb # [ 349.948525] niks3-server[1211]: 2026/09/23 05:38:42 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7184worker2 # [ 349.903475] nix-grpc-daemon[1638]: time=2026-09-23T05:38:42.531Z level=INFO msg="Upload complete. (258ms)"7185lb # [ 349.957052] niks3-server[1211]: 2026/09/23 05:38:42 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7186lb # [ 349.961508] niks3-server[1211]: 2026/09/23 05:38:42 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7187lb # [ 349.966843] niks3-server[1211]: 2026/09/23 05:38:42 INFO Completed upload id=937188lb # [ 349.970892] niks3-server[1211]: 2026/09/23 05:38:42 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7189worker2 # [ 349.917197] nix-grpc-daemon[1560]: ts=2026-09-23T05:38:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=16 outputs=17190worker2 # [ 349.932938] nix-daemon[844]: reaped child process 1950, status = succeeded7191client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.10 seconds)7192client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7193client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.17 seconds)7194worker1: must succeed: rm /nix/.rw-store/fill7195worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7196worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7197worker1 # [ 351.158787] nix-grpc-daemon[1808]: ts=2026-09-23T05:38:43Z level=info event=healthy reason=min_free7198worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7199(finished: subtest: low disk drains a worker and builds go to the other, in 9.87 seconds)7200(finished: run the VM test script, in 352.37 seconds)7201test script finished in 352.63s7202cleanup7203kill QemuMachine (pid 50)7204client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7205client # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down7206client # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down7207client # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down7208kill QemuMachine (pid 49)7209lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7210lb # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down7211lb # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down7212lb # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down7213kill QemuMachine (pid 51)7214worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7215worker1 # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down7216worker1 # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down7217worker1 # [2026-09-23T05:38:44Z INFO virtiofsd] Client disconnected, shutting down7218kill QemuMachine (pid 52)7219worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7220worker2 # [2026-09-23T05:38:45Z INFO virtiofsd] Client disconnected, shutting down7221worker2 # [2026-09-23T05:38:45Z INFO virtiofsd] Client disconnected, shutting down7222worker2 # [2026-09-23T05:38:45Z INFO virtiofsd] Client disconnected, shutting down7223(finished: cleanup, in 0.96 seconds)7224additionally exposed symbols:7225 client, lb, worker1, worker2,7226 vlan1,7227 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh