vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #506
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13worker2 # Disk image does not exist, creating the virtualisation disk image...14client: QEMU running (pid 49)15worker1 # Disk image does not exist, creating the virtualisation disk image...16worker2 # Formatting '/build/vm-state-worker2/tmp.75q5RoNCJF', fmt=raw size=107374182417worker1: QEMU running (pid 51)18lb: QEMU running (pid 50)19worker1 # Formatting '/build/vm-state-worker1/tmp.BiIw0ILslU', fmt=raw size=107374182420worker2 # mke2fs 1.47.4 (6-Mar-2025)21worker2: QEMU running (pid 52)22worker1 # mke2fs 1.47.4 (6-Mar-2025)23worker2 # Discarding device blocks: 0/262144 done24client # Formatting '/build/vm-state-client/tmp.dYEeWhmAGw', fmt=raw size=107374182425lb # Disk image does not exist, creating the virtualisation disk image...26worker1 # Discarding device blocks: 0/262144 done27worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes28lb # Formatting '/build/vm-state-lb/tmp.g8FN8AXiat', fmt=raw size=107374182429worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes30worker2 # Filesystem UUID: 796090bf-a266-4652-b1df-ba99deb215a931lb # mke2fs 1.47.4 (6-Mar-2025)32worker1 # Filesystem UUID: 02bba081-2a3f-409c-887f-61df826885b833worker2 # Superblock backups stored on blocks:34lb # Discarding device blocks: 0/262144 done35worker1 # Superblock backups stored on blocks:36worker2 # 32768, 98304, 163840, 22937637lb # Creating filesystem with 262144 4k blocks and 65536 inodes38worker1 # 32768, 98304, 163840, 22937639worker2 # 40lb # Filesystem UUID: 9744c9d1-513d-4127-943b-b41381ac5c9441worker1 # 42worker2 # Allocating group tables: 0/8 done43lb # Superblock backups stored on blocks:44worker1 # Allocating group tables: 0/8 done45worker2 # Writing inode tables: 0/8 done46lb # 32768, 98304, 163840, 22937647worker1 # Writing inode tables: 0/8 done48worker2 # Creating journal (8192 blocks): done49lb # 50worker1 # Creating journal (8192 blocks): done51worker2 # Writing superblocks and filesystem accounting information: 0/8 done52lb # Allocating group tables: 0/8 done53worker1 # Writing superblocks and filesystem accounting information: 0/8 done54worker2 # 55lb # Writing inode tables: 0/8 done56worker1 # 57worker2 # Virtualisation disk image created.58lb # Creating journal (8192 blocks): done59worker1 # Virtualisation disk image created.60worker2 # Starting virtiofs daemons...61lb # Writing superblocks and filesystem accounting information: 0/8 done62worker1 # Starting virtiofs daemons...63worker2 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)64lb # 65worker1 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)66worker2 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether67lb # Virtualisation disk image created.68worker1 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether69worker2 # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...70lb # Starting virtiofs daemons...71worker1 # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...72worker2 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)73lb # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74worker1 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75worker2 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether76lb # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77worker1 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78worker2 # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...79lb # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...80worker1 # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...81worker2 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)82lb # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83worker1 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84worker2 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether85lb # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86worker1 # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87worker2 # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...88lb # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker1 # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...90worker2 # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests91lb # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker1 # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests93worker2 # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests94lb # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...95worker1 # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests96worker2 # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests97lb # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...98worker1 # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests99(finished: start all VMs, in 0.49 seconds)100client # mke2fs 1.47.4 (6-Mar-2025)101client: waiting for unit network-addresses-eth1.service102client # Discarding device blocks: 0/262144 done103client: waiting for the VM to finish booting104client # Creating filesystem with 262144 4k blocks and 65536 inodes105lb # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests106client # Filesystem UUID: d7d2503c-f441-47fe-97b4-53cf4293554e107lb # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests108client # Superblock backups stored on blocks:109lb # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests110client # 32768, 98304, 163840, 229376111client # 112client # Allocating group tables: 0/8 done113client # Writing inode tables: 0/8 done114client # Creating journal (8192 blocks): done115client # Writing superblocks and filesystem accounting information: 0/8 done116client # 117client # Virtualisation disk image created.118client # Starting virtiofs daemons...119client # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...122client # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123client # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124client # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...125client # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126client # [2026-09-22T16:42:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127client # [2026-09-22T16:42:00Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-22T16:42:00Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133client # [ 0.000000] KASLR enabled134client # [ 0.000000] random: crng init done135client # [ 0.000000] Machine model: linux,dummy-virt136client # [ 0.000000] efi: UEFI not found.137client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140client # [ 0.000000] Zone ranges:141client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] DMA32 empty143client # [ 0.000000] Normal empty144client # [ 0.000000] Device empty145client # [ 0.000000] Movable zone start for each node146client # [ 0.000000] Early memory node ranges147client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]149client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]150client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000151worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026152worker2 # [ 0.000000] KASLR enabled153client # [ 0.000000] psci: probing for conduit method from DT.154worker2 # [ 0.000000] random: crng init done155client # [ 0.000000] psci: PSCIv1.3 detected in firmware.156worker2 # [ 0.000000] Machine model: linux,dummy-virt157client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs158worker2 # [ 0.000000] efi: UEFI not found.159client # [ 0.000000] psci: Trusted OS migration not required160worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT161client # [ 0.000000] psci: SMC Calling Convention v1.1162worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]163client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)164worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]165worker2 # [ 0.000000] Zone ranges:166client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296167client # [ 0.000000] Detected PIPT I-cache on CPU0168worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]169worker2 # [ 0.000000] DMA32 empty170worker2 # [ 0.000000] Normal empty171client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)172worker2 # [ 0.000000] Device empty173client # [ 0.000000] CPU features: detected: GICv3 CPU interface174worker2 # [ 0.000000] Movable zone start for each node175worker2 # [ 0.000000] Early memory node ranges176client # [ 0.000000] CPU features: detected: Spectre-v4177client # [ 0.000000] CPU features: detected: Spectre-BHB178worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]179client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38180worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]181client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23182worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000183client # [ 0.000000] alternatives: applying boot alternatives184worker2 # [ 0.000000] psci: probing for conduit method from DT.185worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.186worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs187worker2 # [ 0.000000] psci: Trusted OS migration not required188worker2 # [ 0.000000] psci: SMC Calling Convention v1.1189worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)190client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test/init regInfo=/nix/store/bvnhgdmv5qvfh2jgky9d95hczimkx039-closure-info/registration console=ttyAMA0,115200n8 console=tty0191worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296192worker2 # [ 0.000000] Detected PIPT I-cache on CPU0193client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bvnhgdmv5qvfh2jgky9d95hczimkx039-closure-info/registration", will be passed to user space.194worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)195worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface196client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes197worker2 # [ 0.000000] CPU features: detected: Spectre-v4198client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)199worker2 # [ 0.000000] CPU features: detected: Spectre-BHB200worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38201client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)202worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23203client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB204worker2 # [ 0.000000] alternatives: applying boot alternatives205client # [ 0.000000] software IO TLB: area num 1.206client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB207client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)208client # [ 0.000000] Fallback order for Node 0: 0209client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216210client # [ 0.000000] Policy zone: DMA211worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/j24yd38mc0ckic4mmr56r3ka0bx8c52x-nixos-system-worker2-test/init regInfo=/nix/store/jw51lbh9icv07w9h8il9pf744i6nicyg-closure-info/registration console=ttyAMA0,115200n8 console=tty0212client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off213client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1214worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/jw51lbh9icv07w9h8il9pf744i6nicyg-closure-info/registration", will be passed to user space.215client # [ 0.000000] allocated 3145728 bytes of page_ext216client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages217worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes218client # [ 0.000000] ftrace: allocated 294 pages with 4 groups219worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)220client # [ 0.000000] rcu: Hierarchical RCU implementation.221client # [ 0.000000] rcu: RCU event tracing is enabled.222worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)223client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.224worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB225client # [ 0.000000] Trampoline variant of Tasks RCU enabled.226worker2 # [ 0.000000] software IO TLB: area num 1.227client # [ 0.000000] Rude variant of Tasks RCU enabled.228worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB229lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]230client # [ 0.000000] Tracing variant of Tasks RCU enabled.231worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)232client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.233worker2 # [ 0.000000] Fallback order for Node 0: 0234lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026235lb # [ 0.000000] KASLR enabled236client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1237worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216238lb # [ 0.000000] random: crng init done239worker2 # [ 0.000000] Policy zone: DMA240lb # [ 0.000000] Machine model: linux,dummy-virt241client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.242lb # [ 0.000000] efi: UEFI not found.243worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off244worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1245client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.246lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT247worker2 # [ 0.000000] allocated 3145728 bytes of page_ext248lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]249client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.250worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages251lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]252worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups253client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0254lb # [ 0.000000] Zone ranges:255client # [ 0.000000] GICv3: 256 SPIs implemented256worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.257lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]258client # [ 0.000000] GICv3: 0 Extended SPIs implemented259worker2 # [ 0.000000] rcu: RCU event tracing is enabled.260lb # [ 0.000000] DMA32 empty261client # [ 0.000000] Root IRQ handler: gic_handle_irq262lb # [ 0.000000] Normal empty263worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.264lb # [ 0.000000] Device empty265client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI266lb # [ 0.000000] Movable zone start for each node267worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.268client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0269lb # [ 0.000000] Early memory node ranges270worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.271client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000272worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.273lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]274client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]275worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.276lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]277lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000278client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)279worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1280lb # [ 0.000000] psci: probing for conduit method from DT.281lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.282worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.283client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)284lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs285client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000286worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.287lb # [ 0.000000] psci: Trusted OS migration not required288lb # [ 0.000000] psci: SMC Calling Convention v1.1289client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000290worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.291client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.292lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)293worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0294worker2 # [ 0.000000] GICv3: 256 SPIs implemented295lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296296client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns297worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented298lb # [ 0.000000] Detected PIPT I-cache on CPU0299worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq300client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).301worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI302lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)303worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0304lb # [ 0.000000] CPU features: detected: GICv3 CPU interface305client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns306lb # [ 0.000000] CPU features: detected: Spectre-v4307worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000308lb # [ 0.000000] CPU features: detected: Spectre-BHB309worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]310client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns311client # [ 0.000034] arm-pv: using stolen time PV312lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38313worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)314lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23315client # [ 0.000658] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)316lb # [ 0.000000] alternatives: applying boot alternatives317client # [ 0.000844] Console: colour dummy device 80x25318worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)319client # [ 0.000851] printk: legacy console [tty0] enabled320worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000321worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000322client # [ 0.001047] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)323client # [ 0.001053] pid_max: default: 32768 minimum: 301324worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.325client # [ 0.001142] LSM: initializing lsm=capability,landlock,yama,bpf,ima326lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x1r0h6bvr8y12qwv2wbzlward6v8k83n-nixos-system-lb-test/init regInfo=/nix/store/5dxqycspslg1fi731232y39406hzw9qc-closure-info/registration console=ttyAMA0,115200n8 console=tty0327client # [ 0.001307] landlock: Up and running.328worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns329client # [ 0.001310] Yama: becoming mindful.330worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).331client # [ 0.001867] LSM support for eBPF active332lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/5dxqycspslg1fi731232y39406hzw9qc-closure-info/registration", will be passed to user space.333worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]334client # [ 0.002016] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)335worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns336client # [ 0.002050] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)337worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns338lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes339worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026340client # [ 0.003884] rcu: Hierarchical SRCU implementation.341worker2 # [ 0.000033] arm-pv: using stolen time PV342worker1 # [ 0.000000] KASLR enabled343client # [ 0.003889] rcu: Max phase no-delay instances is 1000.344lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)345worker1 # [ 0.000000] random: crng init done346worker1 # [ 0.000000] Machine model: linux,dummy-virt347client # [ 0.005124] fsl-mc MSI: its@8080000 domain created348worker2 # [ 0.000513] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)349lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)350worker1 # [ 0.000000] efi: UEFI not found.351client # [ 0.005211] EFI services will not be available.352worker2 # [ 0.000712] Console: colour dummy device 80x25353client # [ 0.005286] smp: Bringing up secondary CPUs ...354lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB355worker2 # [ 0.000720] printk: legacy console [tty0] enabled356worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT357client # [ 0.005293] smp: Brought up 1 node, 1 CPU358lb # [ 0.000000] software IO TLB: area num 1.359client # [ 0.005296] SMP: Total of 1 processors activated.360worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]361lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB362worker2 # [ 0.000915] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)363client # [ 0.005301] CPU: All CPU(s) started at EL1364worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]365worker2 # [ 0.000923] pid_max: default: 32768 minimum: 301366lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)367worker1 # [ 0.000000] Zone ranges:368client # [ 0.005312] CPU features: detected: Branch Target Identification369lb # [ 0.000000] Fallback order for Node 0: 0370worker2 # [ 0.000995] LSM: initializing lsm=capability,landlock,yama,bpf,ima371client # [ 0.005316] CPU features: detected: ARMv8.4 Translation Table Level372worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]373worker2 # [ 0.001147] landlock: Up and running.374worker1 # [ 0.000000] DMA32 empty375lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216376worker2 # [ 0.001149] Yama: becoming mindful.377worker1 # [ 0.000000] Normal empty378lb # [ 0.000000] Policy zone: DMA379worker2 # [ 0.001687] LSM support for eBPF active380client # [ 0.005319] CPU features: detected: Instruction cache invalidation not required for I/D coherence381worker1 # [ 0.000000] Device empty382lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off383worker1 # [ 0.000000] Movable zone start for each node384worker2 # [ 0.001834] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)385client # [ 0.005323] CPU features: detected: Data cache clean to the PoU not required for I/D coherence386worker1 # [ 0.000000] Early memory node ranges387lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1388worker2 # [ 0.001872] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)389client # [ 0.005327] CPU features: detected: Common not Private translations390worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]391lb # [ 0.000000] allocated 3145728 bytes of page_ext392client # [ 0.005330] CPU features: detected: CRC32 instructions393worker2 # [ 0.003734] rcu: Hierarchical SRCU implementation.394lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages395worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]396worker2 # [ 0.003739] rcu: Max phase no-delay instances is 1000.397lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups398client # [ 0.005333] CPU features: detected: Data cache clean to Point of Deep Persistence399worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000400worker2 # [ 0.004976] fsl-mc MSI: its@8080000 domain created401lb # [ 0.000000] rcu: Hierarchical RCU implementation.402worker1 # [ 0.000000] psci: probing for conduit method from DT.403worker2 # [ 0.005067] EFI services will not be available.404client # [ 0.005336] CPU features: detected: Data cache clean to Point of Persistence405lb # [ 0.000000] rcu: RCU event tracing is enabled.406worker2 # [ 0.005149] smp: Bringing up secondary CPUs ...407worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.408client # [ 0.005339] CPU features: detected: Data independent timing control (DIT)409worker2 # [ 0.005156] smp: Brought up 1 node, 1 CPU410lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.411worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs412worker2 # [ 0.005160] SMP: Total of 1 processors activated.413lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.414worker1 # [ 0.000000] psci: Trusted OS migration not required415client # [ 0.005343] CPU features: detected: E0PD416worker2 # [ 0.005165] CPU: All CPU(s) started at EL1417lb # [ 0.000000] Rude variant of Tasks RCU enabled.418worker1 # [ 0.000000] psci: SMC Calling Convention v1.1419lb # [ 0.000000] Tracing variant of Tasks RCU enabled.420client # [ 0.005345] CPU features: detected: Enhanced Counter Virtualization421worker2 # [ 0.005176] CPU features: detected: Branch Target Identification422worker2 # [ 0.005180] CPU features: detected: ARMv8.4 Translation Table Level423worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)424client # [ 0.005348] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)425lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.426client # [ 0.005352] CPU features: detected: Enhanced Virtualization Traps427worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296428lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1429worker2 # [ 0.005183] CPU features: detected: Instruction cache invalidation not required for I/D coherence430client # [ 0.005355] CPU features: detected: Fine Grained Traps431lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.432worker1 # [ 0.000000] Detected PIPT I-cache on CPU0433worker2 # [ 0.005187] CPU features: detected: Data cache clean to the PoU not required for I/D coherence434client # [ 0.005358] CPU features: detected: Generic authentication (architected QARMA5 algorithm)435worker2 # [ 0.005191] CPU features: detected: Common not Private translations436lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.437worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)438client # [ 0.005363] CPU features: detected: RCpc load-acquire (LDAPR)439worker2 # [ 0.005195] CPU features: detected: CRC32 instructions440worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface441client # [ 0.005366] CPU features: detected: LSE atomic instructions442lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.443worker2 # [ 0.005197] CPU features: detected: Data cache clean to Point of Deep Persistence444worker1 # [ 0.000000] CPU features: detected: Spectre-v4445client # [ 0.005369] CPU features: detected: Privileged Access Never446lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0447worker1 # [ 0.000000] CPU features: detected: Spectre-BHB448client # [ 0.005372] CPU features: detected: PMUv3449worker2 # [ 0.005201] CPU features: detected: Data cache clean to Point of Persistence450lb # [ 0.000000] GICv3: 256 SPIs implemented451client # [ 0.005375] CPU features: detected: RAS Extension Support452worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38453lb # [ 0.000000] GICv3: 0 Extended SPIs implemented454worker2 # [ 0.005204] CPU features: detected: Data independent timing control (DIT)455client # [ 0.005378] CPU features: detected: RASv1p1 Extension Support456lb # [ 0.000000] Root IRQ handler: gic_handle_irq457worker2 # [ 0.005207] CPU features: detected: E0PD458worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23459client # [ 0.005380] CPU features: detected: Random Number Generator460lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI461worker1 # [ 0.000000] alternatives: applying boot alternatives462worker2 # [ 0.005210] CPU features: detected: Enhanced Counter Virtualization463lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0464client # [ 0.005383] CPU features: detected: Speculation barrier (SB)465worker2 # [ 0.005213] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)466lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000467client # [ 0.005385] CPU features: detected: Stage-2 Force Write-Back468lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]469worker2 # [ 0.005217] CPU features: detected: Enhanced Virtualization Traps470client # [ 0.005388] CPU features: detected: TLB range maintenance instructions471worker2 # [ 0.005220] CPU features: detected: Fine Grained Traps472lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)473client # [ 0.005393] CPU features: detected: Speculative Store Bypassing Safe (SSBS)474worker2 # [ 0.005223] CPU features: detected: Generic authentication (architected QARMA5 algorithm)475client # [ 0.005433] alternatives: applying system-wide alternatives476lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)477worker2 # [ 0.005228] CPU features: detected: RCpc load-acquire (LDAPR)478worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yms5bvxhm2a2zidkzddvi0xxaqsl6a35-nixos-system-worker1-test/init regInfo=/nix/store/vnq6mm2mwl8jxprixhfy00r13gsa7l45-closure-info/registration console=ttyAMA0,115200n8 console=tty0479client # [ 0.008353] CPU features: detected: BBM Level 2 without TLB conflict abort480worker2 # [ 0.005231] CPU features: detected: LSE atomic instructions481lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000482worker2 # [ 0.005234] CPU features: detected: Privileged Access Never483lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000484worker2 # [ 0.005236] CPU features: detected: PMUv3485worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/vnq6mm2mwl8jxprixhfy00r13gsa7l45-closure-info/registration", will be passed to user space.486client # [ 0.008574] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)487lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.488client # [ 0.008938] devtmpfs: initialized489worker2 # [ 0.005239] CPU features: detected: RAS Extension Support490worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes491worker2 # [ 0.005242] CPU features: detected: RASv1p1 Extension Support492client # [ 0.010724] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)493lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns494worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)495worker2 # [ 0.005245] CPU features: detected: Random Number Generator496client # [ 0.010745] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).497lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).498worker2 # [ 0.005247] CPU features: detected: Speculation barrier (SB)499worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)500client # [ 0.010959] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL501worker2 # [ 0.005250] CPU features: detected: Stage-2 Force Write-Back502lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns503client # [ 0.010964] 0 pages in range for non-PLT usage504worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB505worker2 # [ 0.005253] CPU features: detected: TLB range maintenance instructions506client # [ 0.010965] 508272 pages in range for PLT usage507worker1 # [ 0.000000] software IO TLB: area num 1.508lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns509worker2 # [ 0.005258] CPU features: detected: Speculative Store Bypassing Safe (SSBS)510client # [ 0.011071] pinctrl core: initialized pinctrl subsystem511worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB512lb # [ 0.000031] arm-pv: using stolen time PV513client # [ 0.011862] DMI not present or invalid.514worker2 # [ 0.005298] alternatives: applying system-wide alternatives515client # [ 0.015330] NET: Registered PF_NETLINK/PF_ROUTE protocol family516worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)517worker2 # [ 0.008273] CPU features: detected: BBM Level 2 without TLB conflict abort518lb # [ 0.000446] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)519worker1 # [ 0.000000] Fallback order for Node 0: 0520lb # [ 0.000612] Console: colour dummy device 80x25521client # [ 0.017526] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations522lb # [ 0.000619] printk: legacy console [tty0] enabled523worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216524client # [ 0.017701] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations525worker1 # [ 0.000000] Policy zone: DMA526worker2 # [ 0.008430] Memory: 1405648K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)527worker2 # [ 0.008864] devtmpfs: initialized528worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off529lb # [ 0.000835] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)530client # [ 0.017901] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations531worker2 # [ 0.010659] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)532lb # [ 0.000841] pid_max: default: 32768 minimum: 301533client # [ 0.017923] audit: initializing netlink subsys (disabled)534worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1535lb # [ 0.000919] LSM: initializing lsm=capability,landlock,yama,bpf,ima536client # [ 0.018525] thermal_sys: Registered thermal governor 'fair_share'537worker1 # [ 0.000000] allocated 3145728 bytes of page_ext538worker2 # [ 0.010681] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).539lb # [ 0.001058] landlock: Up and running.540client # [ 0.018527] thermal_sys: Registered thermal governor 'bang_bang'541worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages542lb # [ 0.001061] Yama: becoming mindful.543worker2 # [ 0.010862] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL544lb # [ 0.001546] LSM support for eBPF active545worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups546worker2 # [ 0.010867] 0 pages in range for non-PLT usage547client # [ 0.018531] thermal_sys: Registered thermal governor 'step_wise'548worker2 # [ 0.010868] 508272 pages in range for PLT usage549worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.550lb # [ 0.001685] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)551client # [ 0.018534] thermal_sys: Registered thermal governor 'user_space'552worker2 # [ 0.010984] pinctrl core: initialized pinctrl subsystem553worker1 # [ 0.000000] rcu: RCU event tracing is enabled.554client # [ 0.018539] thermal_sys: Registered thermal governor 'power_allocator'555worker2 # [ 0.011761] DMI not present or invalid.556lb # [ 0.001722] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)557worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.558worker2 # [ 0.015325] NET: Registered PF_NETLINK/PF_ROUTE protocol family559lb # [ 0.003517] rcu: Hierarchical SRCU implementation.560client # [ 0.018564] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1561worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.562lb # [ 0.003522] rcu: Max phase no-delay instances is 1000.563client # [ 0.018573] cpuidle: using governor ladder564worker2 # [ 0.017523] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations565worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.566lb # [ 0.004755] fsl-mc MSI: its@8080000 domain created567client # [ 0.018579] cpuidle: using governor menu568worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.569lb # [ 0.004848] EFI services will not be available.570worker2 # [ 0.017715] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations571client # [ 0.018768] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.572lb # [ 0.004912] smp: Bringing up secondary CPUs ...573worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.574client # [ 0.018783] ASID allocator initialised with 65536 entries575lb # [ 0.004920] smp: Brought up 1 node, 1 CPU576worker2 # [ 0.017912] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations577client # [ 0.020080] Serial: AMBA PL011 UART driver578lb # [ 0.004923] SMP: Total of 1 processors activated.579worker2 # [ 0.017934] audit: initializing netlink subsys (disabled)580worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1581lb # [ 0.004928] CPU: All CPU(s) started at EL1582worker2 # [ 0.018510] thermal_sys: Registered thermal governor 'fair_share'583client # [ 0.025436] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1584worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.585client # [ 0.025594] printk: console [ttyAMA0] enabled586lb # [ 0.004938] CPU features: detected: Branch Target Identification587worker2 # [ 0.018512] thermal_sys: Registered thermal governor 'bang_bang'588lb # [ 0.004942] CPU features: detected: ARMv8.4 Translation Table Level589worker2 # [ 0.018516] thermal_sys: Registered thermal governor 'step_wise'590worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.591worker2 # [ 0.018518] thermal_sys: Registered thermal governor 'user_space'592lb # [ 0.004945] CPU features: detected: Instruction cache invalidation not required for I/D coherence593worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.594worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0595lb # [ 0.004949] CPU features: detected: Data cache clean to the PoU not required for I/D coherence596worker2 # [ 0.018524] thermal_sys: Registered thermal governor 'power_allocator'597worker1 # [ 0.000000] GICv3: 256 SPIs implemented598lb # [ 0.004953] CPU features: detected: Common not Private translations599worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented600worker2 # [ 0.018549] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1601lb # [ 0.004956] CPU features: detected: CRC32 instructions602worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq603worker2 # [ 0.018557] cpuidle: using governor ladder604worker2 # [ 0.018563] cpuidle: using governor menu605worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI606lb # [ 0.004959] CPU features: detected: Data cache clean to Point of Deep Persistence607worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0608worker2 # [ 0.018751] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.609lb # [ 0.004963] CPU features: detected: Data cache clean to Point of Persistence610worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000611worker2 # [ 0.018767] ASID allocator initialised with 65536 entries612lb # [ 0.004966] CPU features: detected: Data independent timing control (DIT)613worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]614worker2 # [ 0.020008] Serial: AMBA PL011 UART driver615lb # [ 0.004969] CPU features: detected: E0PD616worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)617lb # [ 0.004971] CPU features: detected: Enhanced Counter Virtualization618worker2 # [ 0.025335] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1619client # [ 0.144765] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages620worker2 # [ 0.025483] printk: console [ttyAMA0] enabled621lb # [ 0.004974] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)622client # [ 0.144780] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page623worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)624lb # [ 0.004978] CPU features: detected: Enhanced Virtualization Traps625client # [ 0.144786] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages626worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000627lb # [ 0.004981] CPU features: detected: Fine Grained Traps628client # [ 0.144790] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page629worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000630lb # [ 0.004984] CPU features: detected: Generic authentication (architected QARMA5 algorithm)631client # [ 0.144795] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages632worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.633lb # [ 0.004989] CPU features: detected: RCpc load-acquire (LDAPR)634client # [ 0.144799] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page635lb # [ 0.004991] CPU features: detected: LSE atomic instructions636worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns637client # [ 0.144804] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages638lb # [ 0.004994] CPU features: detected: Privileged Access Never639worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).640client # [ 0.144808] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page641lb # [ 0.004997] CPU features: detected: PMUv3642lb # [ 0.005000] CPU features: detected: RAS Extension Support643worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns644lb # [ 0.005003] CPU features: detected: RASv1p1 Extension Support645lb # [ 0.005005] CPU features: detected: Random Number Generator646worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns647worker2 # [ 0.142664] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages648client # [ 0.152037] fbcon: Taking over console649worker1 # [ 0.000029] arm-pv: using stolen time PV650client # [ 0.152051] ACPI: Interpreter disabled.651lb # [ 0.005008] CPU features: detected: Speculation barrier (SB)652worker2 # [ 0.142678] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page653lb # [ 0.005010] CPU features: detected: Stage-2 Force Write-Back654worker1 # [ 0.000405] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)655worker2 # [ 0.142684] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages656worker1 # [ 0.000581] Console: colour dummy device 80x25657worker2 # [ 0.142688] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page658worker1 # [ 0.000589] printk: legacy console [tty0] enabled659worker2 # [ 0.142693] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages660worker2 # [ 0.142697] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page661worker1 # [ 0.000781] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)662worker1 # [ 0.000787] pid_max: default: 32768 minimum: 301663worker2 # [ 0.142702] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages664worker1 # [ 0.000858] LSM: initializing lsm=capability,landlock,yama,bpf,ima665worker2 # [ 0.142706] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page666worker1 # [ 0.000998] landlock: Up and running.667worker1 # [ 0.001001] Yama: becoming mindful.668worker1 # [ 0.001441] LSM support for eBPF active669worker1 # [ 0.001569] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)670worker2 # [ 0.149938] fbcon: Taking over console671worker2 # [ 0.149952] ACPI: Interpreter disabled.672worker1 # [ 0.001600] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)673client # [ 0.161037] iommu: Default domain type: Translated674worker1 # [ 0.003475] rcu: Hierarchical SRCU implementation.675client # [ 0.161046] iommu: DMA domain TLB invalidation policy: strict mode676worker1 # [ 0.003480] rcu: Max phase no-delay instances is 1000.677client # [ 0.161387] SCSI subsystem initialized678lb # [ 0.005013] CPU features: detected: TLB range maintenance instructions679worker1 # [ 0.004717] fsl-mc MSI: its@8080000 domain created680lb # [ 0.005018] CPU features: detected: Speculative Store Bypassing Safe (SSBS)681worker1 # [ 0.004806] EFI services will not be available.682client # [ 0.163391] usbcore: registered new interface driver usbfs683worker1 # [ 0.004873] smp: Bringing up secondary CPUs ...684lb # [ 0.005057] alternatives: applying system-wide alternatives685worker1 # [ 0.004880] smp: Brought up 1 node, 1 CPU686client # [ 0.163423] usbcore: registered new interface driver hub687lb # [ 0.007916] CPU features: detected: BBM Level 2 without TLB conflict abort688client # [ 0.163439] usbcore: registered new device driver usb689worker1 # [ 0.004884] SMP: Total of 1 processors activated.690worker1 # [ 0.004889] CPU: All CPU(s) started at EL1691client # [ 0.163700] pps_core: LinuxPPS API ver. 1 registered692worker1 # [ 0.004900] CPU features: detected: Branch Target Identification693lb # [ 0.008151] Memory: 1405656K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)694client # [ 0.163706] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>695lb # [ 0.008513] devtmpfs: initialized696worker1 # [ 0.004905] CPU features: detected: ARMv8.4 Translation Table Level697client # [ 0.163723] PTP clock support registered698client # [ 0.163780] EDAC MC: Ver: 3.0.0699lb # [ 0.010266] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)700worker1 # [ 0.004908] CPU features: detected: Instruction cache invalidation not required for I/D coherence701client # [ 0.168306] scmi_core: SCMI protocol bus registered702lb # [ 0.010288] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).703worker2 # [ 0.158741] iommu: Default domain type: Translated704worker1 # [ 0.004911] CPU features: detected: Data cache clean to the PoU not required for I/D coherence705lb # [ 0.010462] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL706client # [ 0.169269] FPGA manager framework707worker2 # [ 0.158751] iommu: DMA domain TLB invalidation policy: strict mode708client # [ 0.170147] vgaarb: loaded709lb # [ 0.010467] 0 pages in range for non-PLT usage710worker2 # [ 0.159123] SCSI subsystem initialized711worker1 # [ 0.004915] CPU features: detected: Common not Private translations712lb # [ 0.010468] 508272 pages in range for PLT usage713worker1 # [ 0.004918] CPU features: detected: CRC32 instructions714client # [ 0.170740] clocksource: Switched to clocksource arch_sys_counter715lb # [ 0.010580] pinctrl core: initialized pinctrl subsystem716worker2 # [ 0.161023] usbcore: registered new interface driver usbfs717lb # [ 0.011300] DMI not present or invalid.718worker1 # [ 0.004921] CPU features: detected: Data cache clean to Point of Deep Persistence719worker2 # [ 0.161054] usbcore: registered new interface driver hub720lb # [ 0.014645] NET: Registered PF_NETLINK/PF_ROUTE protocol family721worker2 # [ 0.161069] usbcore: registered new device driver usb722worker1 # [ 0.004925] CPU features: detected: Data cache clean to Point of Persistence723lb # [ 0.016800] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations724worker2 # [ 0.161326] pps_core: LinuxPPS API ver. 1 registered725worker1 # [ 0.004928] CPU features: detected: Data independent timing control (DIT)726worker1 # [ 0.004931] CPU features: detected: E0PD727lb # [ 0.016983] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations728worker2 # [ 0.161332] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>729worker2 # [ 0.161342] PTP clock support registered730worker1 # [ 0.004933] CPU features: detected: Enhanced Counter Virtualization731lb # [ 0.017175] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations732worker2 # [ 0.161396] EDAC MC: Ver: 3.0.0733lb # [ 0.017197] audit: initializing netlink subsys (disabled)734worker1 # [ 0.004937] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)735client # [ 0.175762] VFS: Disk quotas dquot_6.6.0736worker2 # [ 0.165845] scmi_core: SCMI protocol bus registered737lb # [ 0.017780] thermal_sys: Registered thermal governor 'fair_share'738worker2 # [ 0.166782] FPGA manager framework739worker1 # [ 0.004940] CPU features: detected: Enhanced Virtualization Traps740client # [ 0.175799] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)741lb # [ 0.017782] thermal_sys: Registered thermal governor 'bang_bang'742worker1 # [ 0.004943] CPU features: detected: Fine Grained Traps743worker2 # [ 0.167629] vgaarb: loaded744client # [ 0.177436] netfs: FS-Cache loaded745lb # [ 0.017786] thermal_sys: Registered thermal governor 'step_wise'746client # [ 0.177547] pnp: PnP ACPI: disabled747worker1 # [ 0.004947] CPU features: detected: Generic authentication (architected QARMA5 algorithm)748worker2 # [ 0.168231] clocksource: Switched to clocksource arch_sys_counter749lb # [ 0.017788] thermal_sys: Registered thermal governor 'user_space'750worker1 # [ 0.004952] CPU features: detected: RCpc load-acquire (LDAPR)751lb # [ 0.017793] thermal_sys: Registered thermal governor 'power_allocator'752worker1 # [ 0.004955] CPU features: detected: LSE atomic instructions753lb # [ 0.017816] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1754worker1 # [ 0.004957] CPU features: detected: Privileged Access Never755lb # [ 0.017825] cpuidle: using governor ladder756worker1 # [ 0.004960] CPU features: detected: PMUv3757lb # [ 0.017831] cpuidle: using governor menu758worker2 # [ 0.171846] VFS: Disk quotas dquot_6.6.0759lb # [ 0.018005] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.760worker2 # [ 0.171882] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)761lb # [ 0.018020] ASID allocator initialised with 65536 entries762client # [ 0.181454] NET: Registered PF_INET protocol family763lb # [ 0.019197] Serial: AMBA PL011 UART driver764client # [ 0.181718] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)765lb # [ 0.024415] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1766lb # [ 0.024562] printk: console [ttyAMA0] enabled767worker2 # [ 0.175536] netfs: FS-Cache loaded768worker2 # [ 0.175653] pnp: PnP ACPI: disabled769lb # [ 0.143670] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages770lb # [ 0.143685] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page771lb # [ 0.143690] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages772lb # [ 0.143694] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page773lb # [ 0.143698] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages774worker1 # [ 0.004963] CPU features: detected: RAS Extension Support775lb # [ 0.143703] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page776worker2 # [ 0.179419] NET: Registered PF_INET protocol family777worker1 # [ 0.004966] CPU features: detected: RASv1p1 Extension Support778lb # [ 0.143707] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages779worker1 # [ 0.004968] CPU features: detected: Random Number Generator780worker2 # [ 0.179674] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)781lb # [ 0.143711] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page782worker1 # [ 0.004971] CPU features: detected: Speculation barrier (SB)783worker1 # [ 0.004974] CPU features: detected: Stage-2 Force Write-Back784worker1 # [ 0.004976] CPU features: detected: TLB range maintenance instructions785lb # [ 0.151010] fbcon: Taking over console786worker1 # [ 0.004981] CPU features: detected: Speculative Store Bypassing Safe (SSBS)787lb # [ 0.151024] ACPI: Interpreter disabled.788worker1 # [ 0.005018] alternatives: applying system-wide alternatives789worker1 # [ 0.007887] CPU features: detected: BBM Level 2 without TLB conflict abort790worker1 # [ 0.008057] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)791worker1 # [ 0.008420] devtmpfs: initialized792worker1 # [ 0.010202] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)793worker1 # [ 0.010229] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).794worker1 # [ 0.010414] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL795worker1 # [ 0.010419] 0 pages in range for non-PLT usage796worker1 # [ 0.010420] 508272 pages in range for PLT usage797lb # [ 0.159720] iommu: Default domain type: Translated798worker1 # [ 0.010511] pinctrl core: initialized pinctrl subsystem799worker1 # [ 0.011260] DMI not present or invalid.800lb # [ 0.159731] iommu: DMA domain TLB invalidation policy: strict mode801worker1 # [ 0.014479] NET: Registered PF_NETLINK/PF_ROUTE protocol family802lb # [ 0.161472] SCSI subsystem initialized803worker1 # [ 0.016655] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations804lb # [ 0.162121] usbcore: registered new interface driver usbfs805worker1 # [ 0.016825] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations806lb # [ 0.162144] usbcore: registered new interface driver hub807lb # [ 0.162168] usbcore: registered new device driver usb808worker1 # [ 0.017022] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations809lb # [ 0.162469] pps_core: LinuxPPS API ver. 1 registered810worker1 # [ 0.017040] audit: initializing netlink subsys (disabled)811worker1 # [ 0.017557] thermal_sys: Registered thermal governor 'fair_share'812lb # [ 0.162476] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>813lb # [ 0.162486] PTP clock support registered814worker1 # [ 0.017559] thermal_sys: Registered thermal governor 'bang_bang'815lb # [ 0.162527] EDAC MC: Ver: 3.0.0816worker1 # [ 0.017562] thermal_sys: Registered thermal governor 'step_wise'817lb # [ 0.167141] scmi_core: SCMI protocol bus registered818worker1 # [ 0.017565] thermal_sys: Registered thermal governor 'user_space'819lb # [ 0.168083] FPGA manager framework820worker1 # [ 0.017571] thermal_sys: Registered thermal governor 'power_allocator'821lb # [ 0.169020] vgaarb: loaded822worker1 # [ 0.017594] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1823worker1 # [ 0.017604] cpuidle: using governor ladder824worker1 # [ 0.017610] cpuidle: using governor menu825lb # [ 0.169635] clocksource: Switched to clocksource arch_sys_counter826lb # [ 0.170195] VFS: Disk quotas dquot_6.6.0827worker1 # [ 0.017806] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.828lb # [ 0.170245] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)829worker1 # [ 0.017823] ASID allocator initialised with 65536 entries830worker1 # [ 0.019040] Serial: AMBA PL011 UART driver831lb # [ 0.172558] netfs: FS-Cache loaded832lb # [ 0.173116] pnp: PnP ACPI: disabled833worker1 # [ 0.024180] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1834worker1 # [ 0.024332] printk: console [ttyAMA0] enabled835worker1 # [ 0.149234] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages836worker1 # [ 0.149253] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page837worker2 # [ 0.208059] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)838worker1 # [ 0.149258] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages839client # [ 0.210139] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)840worker2 # [ 0.208100] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)841worker1 # [ 0.149262] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page842client # [ 0.210177] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)843worker2 # [ 0.208124] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)844worker1 # [ 0.149267] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages845client # [ 0.210202] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)846worker1 # [ 0.149271] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page847worker2 # [ 0.208191] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)848worker1 # [ 0.149275] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages849client # [ 0.210270] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)850worker2 # [ 0.212125] TCP: Hash tables configured (established 16384 bind 16384)851worker1 # [ 0.149280] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page852client # [ 0.210414] TCP: Hash tables configured (established 16384 bind 16384)853worker2 # [ 0.212206] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)854client # [ 0.210485] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)855lb # [ 0.178580] NET: Registered PF_INET protocol family856client # [ 0.210518] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)857lb # [ 0.178837] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)858worker1 # [ 0.156633] fbcon: Taking over console859client # [ 0.210554] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)860worker1 # [ 0.156647] ACPI: Interpreter disabled.861client # [ 0.210648] NET: Registered PF_UNIX/PF_LOCAL protocol family862client # [ 0.210673] NET: Registered PF_XDP protocol family863client # [ 0.210689] PCI: CLS 0 bytes, default 64864client # [ 0.210820] Trying to unpack rootfs image as initramfs...865worker2 # [ 0.213847] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)866worker2 # [ 0.213888] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)867worker2 # [ 0.213989] NET: Registered PF_UNIX/PF_LOCAL protocol family868worker1 # [ 0.164627] iommu: Default domain type: Translated869worker2 # [ 0.214030] NET: Registered PF_XDP protocol family870worker2 # [ 0.214047] PCI: CLS 0 bytes, default 64871worker1 # [ 0.164637] iommu: DMA domain TLB invalidation policy: strict mode872worker2 # [ 0.214256] Trying to unpack rootfs image as initramfs...873worker1 # [ 0.166366] SCSI subsystem initialized874client # [ 0.228460] kvm [1]: HYP mode not available875worker1 # [ 0.167073] usbcore: registered new interface driver usbfs876worker1 # [ 0.167105] usbcore: registered new interface driver hub877worker1 # [ 0.167121] usbcore: registered new device driver usb878worker1 # [ 0.167398] pps_core: LinuxPPS API ver. 1 registered879worker2 # [ 0.224247] kvm [1]: HYP mode not available880worker1 # [ 0.167405] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>881worker1 # [ 0.167415] PTP clock support registered882worker1 # [ 0.167469] EDAC MC: Ver: 3.0.0883worker1 # [ 0.172388] scmi_core: SCMI protocol bus registered884worker1 # [ 0.173407] FPGA manager framework885worker1 # [ 0.174369] vgaarb: loaded886worker1 # [ 0.175005] clocksource: Switched to clocksource arch_sys_counter887worker1 # [ 0.177130] VFS: Disk quotas dquot_6.6.0888worker1 # [ 0.177167] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)889worker1 # [ 0.180942] netfs: FS-Cache loaded890worker1 # [ 0.181060] pnp: PnP ACPI: disabled891worker1 # [ 0.184896] NET: Registered PF_INET protocol family892worker1 # [ 0.185161] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)893lb # [ 0.207182] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)894lb # [ 0.207221] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)895lb # [ 0.207248] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)896lb # [ 0.207318] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)897lb # [ 0.207462] TCP: Hash tables configured (established 16384 bind 16384)898lb # [ 0.207537] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)899lb # [ 0.207570] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)900lb # [ 0.207604] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)901lb # [ 0.207717] NET: Registered PF_UNIX/PF_LOCAL protocol family902lb # [ 0.207737] NET: Registered PF_XDP protocol family903lb # [ 0.207754] PCI: CLS 0 bytes, default 64904lb # [ 0.207979] Trying to unpack rootfs image as initramfs...905lb # [ 0.223221] kvm [1]: HYP mode not available906worker1 # [ 0.213617] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)907worker1 # [ 0.213659] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)908worker1 # [ 0.213686] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909worker1 # [ 0.213757] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910worker1 # [ 0.213899] TCP: Hash tables configured (established 16384 bind 16384)911worker1 # [ 0.213980] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912worker1 # [ 0.214014] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913worker1 # [ 0.214050] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914worker1 # [ 0.214139] NET: Registered PF_UNIX/PF_LOCAL protocol family915worker1 # [ 0.214158] NET: Registered PF_XDP protocol family916worker1 # [ 0.214175] PCI: CLS 0 bytes, default 64917worker1 # [ 0.214420] Trying to unpack rootfs image as initramfs...918worker1 # [ 0.233144] kvm [1]: HYP mode not available919client # [ 0.312738] Initialise system trusted keyrings920client # [ 0.318783] workingset: timestamp_bits=42 max_order=19 bucket_order=0921client # [ 0.320131] squashfs: version 4.0 (2009/01/31) Phillip Lougher922client # [ 0.320904] 9p: Installing v9fs 9p2000 file system support923worker2 # [ 0.316780] Initialise system trusted keyrings924worker2 # [ 0.317497] workingset: timestamp_bits=42 max_order=19 bucket_order=0925worker2 # [ 0.318724] squashfs: version 4.0 (2009/01/31) Phillip Lougher926worker2 # [ 0.319466] 9p: Installing v9fs 9p2000 file system support927client # [ 0.341543] Key type asymmetric registered928client # [ 0.341560] Asymmetric key parser 'x509' registered929client # [ 0.341620] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)930client # [ 0.346781] io scheduler mq-deadline registered931client # [ 0.346795] io scheduler kyber registered932lb # [ 0.306287] Initialise system trusted keyrings933worker2 # [ 0.340147] Key type asymmetric registered934worker2 # [ 0.340166] Asymmetric key parser 'x509' registered935lb # [ 0.313681] workingset: timestamp_bits=42 max_order=19 bucket_order=0936lb # [ 0.315001] squashfs: version 4.0 (2009/01/31) Phillip Lougher937worker2 # [ 0.348279] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)938worker2 # [ 0.349285] io scheduler mq-deadline registered939worker2 # [ 0.349296] io scheduler kyber registered940lb # [ 0.315761] 9p: Installing v9fs 9p2000 file system support941client # [ 0.358887] pl061_gpio 9030000.pl061: PL061 GPIO chip registered942client # [ 0.359521] ledtrig-cpu: registered to indicate activity on CPUs943client # [ 0.359897] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:944client # [ 0.359913] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000945client # [ 0.359924] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000946client # [ 0.359932] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000947client # [ 0.359951] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits948client # [ 0.359974] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]949client # [ 0.360044] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00950client # [ 0.360054] pci_bus 0000:00: root bus resource [bus 00-ff]951client # [ 0.360060] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]952client # [ 0.360065] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]953client # [ 0.360071] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]954worker2 # [ 0.360366] pl061_gpio 9030000.pl061: PL061 GPIO chip registered955lb # [ 0.336371] Key type asymmetric registered956worker2 # [ 0.360972] ledtrig-cpu: registered to indicate activity on CPUs957lb # [ 0.336387] Asymmetric key parser 'x509' registered958client # [ 0.360181] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint959worker2 # [ 0.361324] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:960lb # [ 0.336440] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)961client # [ 0.360612] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint962lb # [ 0.338544] io scheduler mq-deadline registered963worker2 # [ 0.361340] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000964client # [ 0.360794] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]965lb # [ 0.338554] io scheduler kyber registered966client # [ 0.360810] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]967worker2 # [ 0.361365] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000968client # [ 0.360839] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]969worker2 # [ 0.361375] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000970client # [ 0.360855] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]971worker2 # [ 0.361395] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits972client # [ 0.361297] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint973client # [ 0.361476] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]974worker2 # [ 0.361419] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]975worker1 # [ 0.317430] Initialise system trusted keyrings976client # [ 0.361491] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]977worker2 # [ 0.361490] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00978worker1 # [ 0.318178] workingset: timestamp_bits=42 max_order=19 bucket_order=0979client # [ 0.361521] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]980worker2 # [ 0.361500] pci_bus 0000:00: root bus resource [bus 00-ff]981worker2 # [ 0.361506] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]982worker1 # [ 0.319445] squashfs: version 4.0 (2009/01/31) Phillip Lougher983client # [ 0.361966] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint984client # [ 0.362144] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]985worker2 # [ 0.361511] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]986client # [ 0.362160] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]987worker2 # [ 0.361516] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]988client # [ 0.362189] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]989worker2 # [ 0.361616] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint990client # [ 0.362627] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint991worker1 # [ 0.320226] 9p: Installing v9fs 9p2000 file system support992worker2 # [ 0.362046] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint993client # [ 0.362826] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]994worker2 # [ 0.362229] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]995client # [ 0.362841] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]996worker2 # [ 0.362246] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]997client # [ 0.362870] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]998worker2 # [ 0.362276] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]999client # [ 0.362889] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1000worker2 # [ 0.362292] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1001lb # [ 0.351569] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1002client # [ 0.363338] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1003worker2 # [ 0.362748] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1004client # [ 0.363521] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1005worker2 # [ 0.362928] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1006lb # [ 0.352829] ledtrig-cpu: registered to indicate activity on CPUs1007client # [ 0.363550] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1008worker2 # [ 0.362944] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1009lb # [ 0.353201] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1010worker2 # [ 0.362973] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1011client # [ 0.363999] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1012client # [ 0.364187] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1013lb # [ 0.353217] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001014worker2 # [ 0.363411] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1015worker2 # [ 0.363589] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1016client # [ 0.364216] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1017lb # [ 0.353229] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001018worker2 # [ 0.363605] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1019client # [ 0.364594] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1020lb # [ 0.353237] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001021worker2 # [ 0.363634] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1022client # [ 0.364771] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1023worker2 # [ 0.364094] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1024lb # [ 0.353268] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1025client # [ 0.365040] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1026worker2 # [ 0.364296] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1027client # [ 0.365222] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1028lb # [ 0.353288] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1029worker2 # [ 0.364312] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1030client # [ 0.365250] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1031lb # [ 0.353354] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001032worker2 # [ 0.364342] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033lb # [ 0.353364] pci_bus 0000:00: root bus resource [bus 00-ff]1034client # [ 0.365688] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1035worker2 # [ 0.364360] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1036lb # [ 0.353370] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1037client # [ 0.365869] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1038worker2 # [ 0.364809] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1039lb # [ 0.353375] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1040client # [ 0.365898] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1041worker2 # [ 0.364991] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1042lb # [ 0.353380] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1043client # [ 0.366338] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1044worker2 # [ 0.365021] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1045lb # [ 0.353435] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1046client # [ 0.366520] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1047worker2 # [ 0.365470] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1048client # [ 0.366549] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049worker2 # [ 0.365654] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1050lb # [ 0.365503] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1051worker2 # [ 0.365683] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1052worker2 # [ 0.366061] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1053worker2 # [ 0.366238] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1054worker2 # [ 0.366526] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1055lb # [ 0.366671] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1056worker2 # [ 0.366710] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1057lb # [ 0.366691] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1058worker2 # [ 0.366739] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059lb # [ 0.366721] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1060worker2 # [ 0.367180] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1061lb # [ 0.366737] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1062worker2 # [ 0.367363] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1063lb # [ 0.367184] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1064worker2 # [ 0.367392] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065lb # [ 0.367365] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1066worker1 # [ 0.348214] Key type asymmetric registered1067worker2 # [ 0.367838] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1068lb # [ 0.367381] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1069worker1 # [ 0.348234] Asymmetric key parser 'x509' registered1070worker2 # [ 0.368019] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1071lb # [ 0.367410] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1072worker1 # [ 0.348303] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1073worker2 # [ 0.368049] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074lb # [ 0.367862] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1075worker1 # [ 0.350445] io scheduler mq-deadline registered1076lb # [ 0.368042] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1077worker1 # [ 0.350455] io scheduler kyber registered1078lb # [ 0.368058] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1079client # [ 0.415087] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1080client # [ 0.415372] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1081lb # [ 0.368088] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1082client # [ 0.415390] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1083lb # [ 0.368547] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1084client # [ 0.415420] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085lb # [ 0.368729] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1086lb # [ 0.368745] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1087client # [ 0.415885] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1088lb # [ 0.368774] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089client # [ 0.416069] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1090client # [ 0.416086] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1091lb # [ 0.368793] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1092client # [ 0.416125] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1093lb # [ 0.369244] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1094client # [ 0.416696] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1095lb # [ 0.369431] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1096client # [ 0.416707] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1097lb # [ 0.369460] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098client # [ 0.416712] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1099client # [ 0.416758] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1100client # [ 0.416804] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1101client # [ 0.416851] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1102worker2 # [ 0.416604] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1103worker2 # [ 0.416898] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1104client # [ 0.416898] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1105worker2 # [ 0.416916] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1106client # [ 0.416944] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1107worker2 # [ 0.416946] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108client # [ 0.416992] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1109worker2 # [ 0.417406] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1110worker2 # [ 0.417585] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1111client # [ 0.417039] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1112worker2 # [ 0.417601] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1113client # [ 0.417085] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1114worker2 # [ 0.417630] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115client # [ 0.417131] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1116worker2 # [ 0.418188] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1117client # [ 0.417206] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1118worker2 # [ 0.418200] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1119client # [ 0.417259] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1120worker2 # [ 0.418206] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1121worker1 # [ 0.363677] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1122client # [ 0.417281] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1123worker1 # [ 0.364234] ledtrig-cpu: registered to indicate activity on CPUs1124worker2 # [ 0.418250] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1125client # [ 0.417302] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1126worker1 # [ 0.364597] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1127worker2 # [ 0.418297] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1128client # [ 0.417324] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1129client # [ 0.417345] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1130worker2 # [ 0.418351] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1131worker1 # [ 0.364614] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001132client # [ 0.417367] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1133worker2 # [ 0.418398] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1134worker1 # [ 0.364627] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001135client # [ 0.417388] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1136worker2 # [ 0.418445] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1137worker1 # [ 0.364635] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001138client # [ 0.417414] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1139lb # [ 0.394005] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1140worker2 # [ 0.418491] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1141lb # [ 0.394207] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1142client # [ 0.417436] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1143worker1 # [ 0.364656] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1144worker2 # [ 0.418537] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1145client # [ 0.417458] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1146lb # [ 0.394269] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147worker1 # [ 0.364677] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1148worker2 # [ 0.418583] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1149client # [ 0.417480] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1150lb # [ 0.394696] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1151worker1 # [ 0.364757] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001152lb # [ 0.394879] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1153client # [ 0.417502] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1154worker2 # [ 0.418628] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1155worker1 # [ 0.364766] pci_bus 0000:00: root bus resource [bus 00-ff]1156client # [ 0.417524] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1157lb # [ 0.395137] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1158worker2 # [ 0.418696] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1159worker1 # [ 0.364772] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1160client # [ 0.417544] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1161lb # [ 0.395323] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1162worker2 # [ 0.418741] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1163worker1 # [ 0.364778] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1164client # [ 0.417565] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1165lb # [ 0.395353] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1166worker2 # [ 0.418762] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1167client # [ 0.417586] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1168worker1 # [ 0.364783] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1169worker2 # [ 0.418783] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1170lb # [ 0.395800] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1171client # [ 0.417608] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1172worker1 # [ 0.364875] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1173worker2 # [ 0.418805] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1174lb # [ 0.395986] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1175client # [ 0.417629] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1176client # [ 0.417655] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1177worker2 # [ 0.418826] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1178worker1 # [ 0.365321] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1179lb # [ 0.396015] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180client # [ 0.417665] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1181worker2 # [ 0.418847] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1182worker1 # [ 0.365511] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1183lb # [ 0.396474] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1184worker1 # [ 0.365528] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1185worker2 # [ 0.418868] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1186client # [ 0.417669] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1187lb # [ 0.396660] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1188client # [ 0.418488] pci 0000:00:07.0: enabling device (0000 -> 0002)1189worker1 # [ 0.365558] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1190worker2 # [ 0.418893] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1191lb # [ 0.396690] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1192worker1 # [ 0.365575] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1193worker2 # [ 0.418914] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1194lb # [ 0.397142] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1195worker2 # [ 0.418936] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1196lb # [ 0.397390] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1197worker1 # [ 0.366042] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1198lb # [ 0.397407] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1199worker1 # [ 0.366230] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1200worker2 # [ 0.418958] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1201worker1 # [ 0.366246] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1202worker2 # [ 0.418980] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1203lb # [ 0.397436] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1204worker2 # [ 0.419001] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1205worker1 # [ 0.366276] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1206lb # [ 0.397905] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1207worker2 # [ 0.419022] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1208lb # [ 0.398088] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1209worker1 # [ 0.366738] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1210worker2 # [ 0.419042] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1211lb # [ 0.398104] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1212worker1 # [ 0.366926] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1213worker2 # [ 0.419063] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1214lb # [ 0.398134] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1215worker1 # [ 0.366942] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1216client # [ 0.458862] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 39434 usecs1217worker2 # [ 0.419084] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1218lb # [ 0.398735] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1219worker1 # [ 0.366972] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1220worker2 # [ 0.419105] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1221lb # [ 0.398745] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1222worker2 # [ 0.419130] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1223worker1 # [ 0.367461] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1224worker2 # [ 0.419139] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1225lb # [ 0.398751] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1226worker1 # [ 0.367650] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1227worker2 # [ 0.419144] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1228worker1 # [ 0.367665] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1229lb # [ 0.398800] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1230worker2 # [ 0.419933] pci 0000:00:07.0: enabling device (0000 -> 0002)1231worker1 # [ 0.367696] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1232lb # [ 0.398849] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1233worker1 # [ 0.367714] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1234lb # [ 0.398896] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1235worker2 # [ 0.455200] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 34445 usecs1236worker1 # [ 0.368188] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1237lb # [ 0.398943] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1238worker1 # [ 0.368379] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1239lb # [ 0.398990] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1240worker1 # [ 0.368410] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1241lb # [ 0.399036] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1242worker1 # [ 0.368884] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1243lb # [ 0.399082] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1244worker1 # [ 0.369076] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1245lb # [ 0.399129] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1246worker1 # [ 0.369107] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1247lb # [ 0.399174] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1248worker1 # [ 0.369505] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1249worker1 # [ 0.369692] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1250lb # [ 0.399246] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1251lb # [ 0.399291] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1252worker1 # [ 0.369972] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1253worker1 # [ 0.370163] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1254lb # [ 0.399313] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1255lb # [ 0.399334] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1256worker1 # [ 0.370194] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1257lb # [ 0.399356] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1258worker1 # [ 0.370657] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1259lb # [ 0.399378] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1260worker1 # [ 0.370849] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1261lb # [ 0.399399] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1262worker1 # [ 0.370879] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1263lb # [ 0.399421] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1264lb # [ 0.399446] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1265lb # [ 0.399468] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1266lb # [ 0.399490] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1267lb # [ 0.399513] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1268lb # [ 0.399535] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1269client # [ 0.479707] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1270lb # [ 0.399556] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1271lb # [ 0.399577] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1272client # [ 0.481814] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1273lb # [ 0.399599] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1274lb # [ 0.399620] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1275lb # [ 0.399641] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1276lb # [ 0.399663] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1277lb # [ 0.399689] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1278lb # [ 0.399698] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1279lb # [ 0.399702] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1280lb # [ 0.400505] pci 0000:00:07.0: enabling device (0000 -> 0002)1281worker2 # [ 0.478558] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1282worker1 # [ 0.419458] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1283worker1 # [ 0.419672] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1284client # [ 0.492943] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1285worker1 # [ 0.419703] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1286worker1 # [ 0.420183] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1287worker1 # [ 0.420453] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1288worker1 # [ 0.420472] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1289worker1 # [ 0.420502] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1290worker2 # [ 0.484881] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1291worker1 # [ 0.420987] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1292worker2 # [ 0.486874] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1293worker1 # [ 0.421175] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1294worker1 # [ 0.421191] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1295worker1 # [ 0.421221] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1296worker1 # [ 0.421792] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1297worker1 # [ 0.421803] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1298client # [ 0.498942] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1299client # [ 0.501024] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1300worker1 # [ 0.421808] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1301worker1 # [ 0.421855] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1302worker1 # [ 0.421902] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1303worker1 # [ 0.421951] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1304worker1 # [ 0.422000] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1305worker1 # [ 0.422048] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1306worker2 # [ 0.496784] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1307worker1 # [ 0.422097] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1308worker2 # [ 0.498794] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1309worker1 # [ 0.422145] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1310worker1 # [ 0.422192] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1311worker1 # [ 0.422239] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1312worker1 # [ 0.422308] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1313client # [ 0.510917] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1314lb # [ 0.472565] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1315worker1 # [ 0.422355] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1316client # [ 0.512855] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1317worker1 # [ 0.422377] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1318client # [ 0.514579] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1319worker1 # [ 0.422399] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1320worker1 # [ 0.422421] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1321worker1 # [ 0.422443] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1322worker1 # [ 0.422464] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1323worker1 # [ 0.422486] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1324worker1 # [ 0.422512] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1325worker1 # [ 0.422534] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1326worker1 # [ 0.422557] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1327lb # [ 0.478797] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1328worker1 # [ 0.422579] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1329worker1 # [ 0.422602] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1330lb # [ 0.480911] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1331worker1 # [ 0.422624] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1332worker1 # [ 0.422645] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1333client # [ 0.524925] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1334worker2 # [ 0.508408] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1335worker1 # [ 0.422666] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1336worker2 # [ 0.510645] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1337worker1 # [ 0.422688] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1338worker1 # [ 0.422710] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1339worker1 # [ 0.422731] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1340worker2 # [ 0.512387] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1341worker1 # [ 0.422758] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1342worker2 # [ 0.514178] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1343worker1 # [ 0.422767] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1344worker2 # [ 0.516008] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1345worker1 # [ 0.422772] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1346worker1 # [ 0.423634] pci 0000:00:07.0: enabling device (0000 -> 0002)1347lb # [ 0.491056] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1348client # [ 0.530909] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1349lb # [ 0.493011] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1350client # [ 0.533149] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1351worker2 # [ 0.529695] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1352client # [ 0.545928] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1353lb # [ 0.504279] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1354lb # [ 0.506770] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1355lb # [ 0.508669] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1356client # [ 0.551871] msm_serial: driver initialized1357client # [ 0.552023] SuperH (H)SCI(F) driver initialized1358client # [ 0.552077] STM32 USART driver initialized1359worker2 # [ 0.545459] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1360worker2 # [ 0.547952] msm_serial: driver initialized1361worker2 # [ 0.548087] SuperH (H)SCI(F) driver initialized1362worker1 # [ 0.489350] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1363worker2 # [ 0.548140] STM32 USART driver initialized1364worker1 # [ 0.492607] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1365lb # [ 0.518635] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1366worker1 # [ 0.494598] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1367lb # [ 0.520552] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1368lb # [ 0.524291] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1369worker1 # [ 0.504385] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1370worker1 # [ 0.506429] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1371lb # [ 0.538859] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1372lb # [ 0.541332] msm_serial: driver initialized1373lb # [ 0.541486] SuperH (H)SCI(F) driver initialized1374lb # [ 0.541541] STM32 USART driver initialized1375client # [ 0.584566] loop: module loaded1376client # [ 0.584730] virtio_blk virtio2: 1/0/0 default/read/poll queues1377worker1 # [ 0.517865] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1378client # [ 0.585535] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1379worker2 # [ 0.578981] loop: module loaded1380worker1 # [ 0.520697] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1381worker2 # [ 0.579174] virtio_blk virtio2: 1/0/0 default/read/poll queues1382worker1 # [ 0.522457] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1383worker2 # [ 0.579961] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1384client # [ 0.591294] megasas: 07.734.00.00-rc11385client # [ 0.591985] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1386client # [ 0.594084] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001387client # [ 0.594106] Intel/Sharp Extended Query Table at 0x00311388worker2 # [ 0.584762] megasas: 07.734.00.00-rc11389worker2 # [ 0.585413] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1390worker2 # [ 0.587917] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001391worker2 # [ 0.587936] Intel/Sharp Extended Query Table at 0x00311392worker1 # [ 0.532498] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1393worker1 # [ 0.534396] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1394client # [ 0.603643] Using buffer write method1395worker1 # [ 0.538208] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1396client # [ 0.603710] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1397client # [ 0.606503] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001398client # [ 0.606525] Intel/Sharp Extended Query Table at 0x00311399worker2 # [ 0.597170] Using buffer write method1400worker2 # [ 0.597232] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1401worker2 # [ 0.600159] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402worker2 # [ 0.600181] Intel/Sharp Extended Query Table at 0x00311403client # [ 0.615609] Using buffer write method1404client # [ 0.615634] Concatenating MTD devices:1405client # [ 0.615638] (0): "0.flash"1406client # [ 0.615646] (1): "0.flash"1407lb # [ 0.574201] loop: module loaded1408client # [ 0.615650] into device "0.flash"1409lb # [ 0.574491] virtio_blk virtio2: 1/0/0 default/read/poll queues1410lb # [ 0.575241] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1411worker2 # [ 0.613469] Using buffer write method1412worker2 # [ 0.613500] Concatenating MTD devices:1413worker2 # [ 0.613504] (0): "0.flash"1414worker2 # [ 0.613512] (1): "0.flash"1415worker1 # [ 0.556261] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1416worker2 # [ 0.613516] into device "0.flash"1417lb # [ 0.582190] megasas: 07.734.00.00-rc11418lb # [ 0.582911] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1419lb # [ 0.584938] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001420lb # [ 0.584960] Intel/Sharp Extended Query Table at 0x00311421worker1 # [ 0.564078] msm_serial: driver initialized1422lb # [ 0.586611] Using buffer write method1423worker1 # [ 0.564221] SuperH (H)SCI(F) driver initialized1424worker1 # [ 0.564275] STM32 USART driver initialized1425lb # [ 0.586666] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1426lb # [ 0.588409] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001427lb # [ 0.588429] Intel/Sharp Extended Query Table at 0x00311428lb # [ 0.607092] Using buffer write method1429lb # [ 0.607124] Concatenating MTD devices:1430lb # [ 0.607128] (0): "0.flash"1431lb # [ 0.607132] (1): "0.flash"1432lb # [ 0.607140] into device "0.flash"1433worker1 # [ 0.593197] loop: module loaded1434worker1 # [ 0.593385] virtio_blk virtio2: 1/0/0 default/read/poll queues1435worker1 # [ 0.594141] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1436worker1 # [ 0.607574] megasas: 07.734.00.00-rc11437worker1 # [ 0.608258] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1438worker1 # [ 0.610355] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001439worker1 # [ 0.610379] Intel/Sharp Extended Query Table at 0x00311440worker1 # [ 0.611960] Using buffer write method1441worker1 # [ 0.612028] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1442worker1 # [ 0.613678] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001443worker1 # [ 0.613734] Intel/Sharp Extended Query Table at 0x00311444worker1 # [ 0.623119] Using buffer write method1445worker1 # [ 0.623142] Concatenating MTD devices:1446worker1 # [ 0.623146] (0): "0.flash"1447worker1 # [ 0.623150] (1): "0.flash"1448worker1 # [ 0.623160] into device "0.flash"1449client # [ 0.854281] Freeing initrd memory: 26972K1450client # [ 0.860435] tun: Universal TUN/TAP device driver, 1.61451worker2 # [ 0.849556] Freeing initrd memory: 26944K1452worker2 # [ 0.855454] tun: Universal TUN/TAP device driver, 1.61453client # [ 0.864047] thunder_xcv, ver 1.01454client # [ 0.864099] thunder_bgx, ver 1.01455client # [ 0.864123] nicpf, ver 1.01456client # [ 0.864709] e1000: Intel(R) PRO/1000 Network Driver1457client # [ 0.864716] e1000: Copyright (c) 1999-2006 Intel Corporation.1458worker2 # [ 0.858954] thunder_xcv, ver 1.01459client # [ 0.864741] e1000e: Intel(R) PRO/1000 Network Driver1460worker2 # [ 0.858997] thunder_bgx, ver 1.01461worker2 # [ 0.859020] nicpf, ver 1.01462client # [ 0.864749] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1463worker2 # [ 0.859574] e1000: Intel(R) PRO/1000 Network Driver1464client # [ 0.864779] igb: Intel(R) Gigabit Ethernet Network Driver1465worker2 # [ 0.859581] e1000: Copyright (c) 1999-2006 Intel Corporation.1466client # [ 0.864785] igb: Copyright (c) 2007-2014 Intel Corporation.1467worker2 # [ 0.859605] e1000e: Intel(R) PRO/1000 Network Driver1468client # [ 0.864809] igbvf: Intel(R) Gigabit Virtual Function Network Driver1469worker2 # [ 0.859612] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1470client # [ 0.864816] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1471worker2 # [ 0.859642] igb: Intel(R) Gigabit Ethernet Network Driver1472client # [ 0.864951] sky2: driver version 1.301473worker2 # [ 0.859647] igb: Copyright (c) 2007-2014 Intel Corporation.1474client # [ 0.866555] usbcore: registered new interface driver usb-storage1475worker2 # [ 0.859670] igbvf: Intel(R) Gigabit Virtual Function Network Driver1476client # [ 0.866602] usbcore: registered new interface driver usbserial_generic1477worker2 # [ 0.859676] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1478client # [ 0.866617] usbserial: USB Serial support registered for generic1479worker2 # [ 0.859805] sky2: driver version 1.301480client # [ 0.867396] ehci-pci 0000:00:07.0: EHCI Host Controller1481client # [ 0.867419] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11482client # [ 0.867619] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001483worker2 # [ 0.868936] ehci-pci 0000:00:07.0: EHCI Host Controller1484worker2 # [ 0.868964] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11485client # [ 0.879386] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001486worker2 # [ 0.869231] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001487client # [ 0.879728] hub 1-0:1.0: USB hub found1488client # [ 0.879753] hub 1-0:1.0: 6 ports detected1489lb # [ 0.839357] Freeing initrd memory: 26948K1490worker2 # [ 0.871782] usbcore: registered new interface driver usb-storage1491client # [ 0.882340] hv_vmbus: registering driver hyperv_keyboard1492worker2 # [ 0.871874] usbcore: registered new interface driver usbserial_generic1493worker2 # [ 0.871888] usbserial: USB Serial support registered for generic1494client # [ 0.883892] rtc-pl031 9010000.pl031: registered as rtc01495worker2 # [ 0.874702] hv_vmbus: registering driver hyperv_keyboard1496worker2 # [ 0.876155] rtc-pl031 9010000.pl031: registered as rtc01497client # [ 0.883917] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:42:02 UTC (1790095322)1498client # [ 0.884233] i2c_dev: i2c /dev entries driver1499worker2 # [ 0.876180] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:42:02 UTC (1790095322)1500lb # [ 0.845176] tun: Universal TUN/TAP device driver, 1.61501worker2 # [ 0.878189] i2c_dev: i2c /dev entries driver1502client # [ 0.889563] sdhci: Secure Digital Host Controller Interface driver1503lb # [ 0.848852] thunder_xcv, ver 1.01504client # [ 0.889574] sdhci: Copyright(c) Pierre Ossman1505lb # [ 0.848893] thunder_bgx, ver 1.01506lb # [ 0.848915] nicpf, ver 1.01507client # [ 0.889841] Synopsys Designware Multimedia Card Interface Driver1508lb # [ 0.849444] e1000: Intel(R) PRO/1000 Network Driver1509client # [ 0.890219] sdhci-pltfm: SDHCI platform and OF driver helper1510lb # [ 0.849451] e1000: Copyright (c) 1999-2006 Intel Corporation.1511lb # [ 0.849475] e1000e: Intel(R) PRO/1000 Network Driver1512worker2 # [ 0.881867] sdhci: Secure Digital Host Controller Interface driver1513lb # [ 0.849482] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1514worker2 # [ 0.881877] sdhci: Copyright(c) Pierre Ossman1515client # [ 0.894451] hid: raw HID events driver (C) Jiri Kosina1516lb # [ 0.849510] igb: Intel(R) Gigabit Ethernet Network Driver1517worker2 # [ 0.882136] Synopsys Designware Multimedia Card Interface Driver1518client # [ 0.894703] usbcore: registered new interface driver usbhid1519client # [ 0.894710] usbhid: USB HID core driver1520lb # [ 0.849516] igb: Copyright (c) 2007-2014 Intel Corporation.1521worker2 # [ 0.882507] sdhci-pltfm: SDHCI platform and OF driver helper1522worker2 # [ 0.883970] hid: raw HID events driver (C) Jiri Kosina1523lb # [ 0.849537] igbvf: Intel(R) Gigabit Virtual Function Network Driver1524worker2 # [ 0.884206] usbcore: registered new interface driver usbhid1525lb # [ 0.849543] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1526worker2 # [ 0.884212] usbhid: USB HID core driver1527worker2 # [ 0.888734] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001528client # [ 0.897546] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1529worker2 # [ 0.889728] hub 1-0:1.0: USB hub found1530worker2 # [ 0.890207] hub 1-0:1.0: 6 ports detected1531lb # [ 0.856310] sky2: driver version 1.301532client # [ 0.900152] drop_monitor: Initializing network drop monitor service1533lb # [ 0.858315] ehci-pci 0000:00:07.0: EHCI Host Controller1534client # [ 0.900289] NET: Registered PF_INET6 protocol family1535lb # [ 0.858345] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11536client # [ 0.902281] Segment Routing with IPv61537worker2 # [ 0.892144] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1538lb # [ 0.858609] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001539client # [ 0.902302] In-situ OAM (IOAM) with IPv61540client # [ 0.902331] NET: Registered PF_PACKET protocol family1541lb # [ 0.861619] usbcore: registered new interface driver usb-storage1542worker2 # [ 0.894740] drop_monitor: Initializing network drop monitor service1543worker2 # [ 0.894868] NET: Registered PF_INET6 protocol family1544lb # [ 0.862492] usbcore: registered new interface driver usbserial_generic1545lb # [ 0.862513] usbserial: USB Serial support registered for generic1546worker2 # [ 0.896777] Segment Routing with IPv61547client # [ 0.904125] 9pnet: Installing 9P2000 support1548worker2 # [ 0.896794] In-situ OAM (IOAM) with IPv61549lb # [ 0.863103] hv_vmbus: registering driver hyperv_keyboard1550client # [ 0.904171] Key type dns_resolver registered1551worker2 # [ 0.896821] NET: Registered PF_PACKET protocol family1552lb # [ 0.866067] rtc-pl031 9010000.pl031: registered as rtc01553worker2 # [ 0.898443] 9pnet: Installing 9P2000 support1554lb # [ 0.866091] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:42:02 UTC (1790095322)1555worker2 # [ 0.898490] Key type dns_resolver registered1556lb # [ 0.866406] i2c_dev: i2c /dev entries driver1557client # [ 0.911097] registered taskstats version 11558client # [ 0.911239] Loading compiled-in X.509 certificates1559lb # [ 0.871567] sdhci: Secure Digital Host Controller Interface driver1560lb # [ 0.871578] sdhci: Copyright(c) Pierre Ossman1561lb # [ 0.871839] Synopsys Designware Multimedia Card Interface Driver1562lb # [ 0.872200] sdhci-pltfm: SDHCI platform and OF driver helper1563worker2 # [ 0.905280] registered taskstats version 11564worker2 # [ 0.905447] Loading compiled-in X.509 certificates1565lb # [ 0.876317] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001566lb # [ 0.876610] hub 1-0:1.0: USB hub found1567lb # [ 0.876630] hub 1-0:1.0: 6 ports detected1568client # [ 0.919744] Demotion targets for Node 0: null1569client # [ 0.919840] Key type .fscrypt registered1570client # [ 0.919846] Key type fscrypt-provisioning registered1571client # [ 0.919938] ima: No TPM chip found, activating TPM-bypass!1572lb # [ 0.878796] hid: raw HID events driver (C) Jiri Kosina1573client # [ 0.919957] ima: Allocated hash algorithm: sha11574lb # [ 0.879032] usbcore: registered new interface driver usbhid1575client # [ 0.919978] ima: No architecture policies found1576lb # [ 0.879038] usbhid: USB HID core driver1577worker2 # [ 0.914076] Demotion targets for Node 0: null1578client # [ 0.923946] input: gpio-keys as /devices/platform/gpio-keys/input/input01579worker2 # [ 0.914180] Key type .fscrypt registered1580worker2 # [ 0.914186] Key type fscrypt-provisioning registered1581worker2 # [ 0.914280] ima: No TPM chip found, activating TPM-bypass!1582lb # [ 0.881923] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1583worker2 # [ 0.914299] ima: Allocated hash algorithm: sha11584lb # [ 0.883424] drop_monitor: Initializing network drop monitor service1585worker2 # [ 0.914319] ima: No architecture policies found1586lb # [ 0.883582] NET: Registered PF_INET6 protocol family1587worker2 # [ 0.918342] input: gpio-keys as /devices/platform/gpio-keys/input/input01588lb # [ 0.886595] Segment Routing with IPv61589lb # [ 0.886612] In-situ OAM (IOAM) with IPv61590lb # [ 0.886640] NET: Registered PF_PACKET protocol family1591worker1 # [ 0.863645] Freeing initrd memory: 26976K1592lb # [ 0.888266] 9pnet: Installing 9P2000 support1593lb # [ 0.888311] Key type dns_resolver registered1594worker1 # [ 0.869588] tun: Universal TUN/TAP device driver, 1.61595lb # [ 0.894731] registered taskstats version 11596lb # [ 0.894883] Loading compiled-in X.509 certificates1597worker1 # [ 0.873277] thunder_xcv, ver 1.01598client # [ 0.941643] clk: Disabling unused clocks1599worker1 # [ 0.873320] thunder_bgx, ver 1.01600worker1 # [ 0.873343] nicpf, ver 1.01601client # [ 0.941666] PM: genpd: Disabling unused power domains1602worker1 # [ 0.873915] e1000: Intel(R) PRO/1000 Network Driver1603worker1 # [ 0.873922] e1000: Copyright (c) 1999-2006 Intel Corporation.1604worker1 # [ 0.873947] e1000e: Intel(R) PRO/1000 Network Driver1605lb # [ 0.903609] Demotion targets for Node 0: null1606worker2 # [ 0.936157] clk: Disabling unused clocks1607worker1 # [ 0.873955] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1608client # [ 0.945745] Freeing unused kernel memory: 4736K1609lb # [ 0.903716] Key type .fscrypt registered1610worker2 # [ 0.936178] PM: genpd: Disabling unused power domains1611client # [ 0.945938] Run /init as init process1612worker1 # [ 0.873985] igb: Intel(R) Gigabit Ethernet Network Driver1613lb # [ 0.903724] Key type fscrypt-provisioning registered1614worker1 # [ 0.873990] igb: Copyright (c) 2007-2014 Intel Corporation.1615lb # [ 0.903818] ima: No TPM chip found, activating TPM-bypass!1616lb # [ 0.903838] ima: Allocated hash algorithm: sha11617worker1 # [ 0.874013] igbvf: Intel(R) Gigabit Virtual Function Network Driver1618lb # [ 0.903859] ima: No architecture policies found1619worker1 # [ 0.874019] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1620worker1 # [ 0.874153] sky2: driver version 1.301621lb # [ 0.907754] input: gpio-keys as /devices/platform/gpio-keys/input/input01622worker2 # [ 0.940229] Freeing unused kernel memory: 4736K1623worker2 # [ 0.940440] Run /init as init process1624worker1 # [ 0.883748] ehci-pci 0000:00:07.0: EHCI Host Controller1625worker1 # [ 0.883779] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11626worker1 # [ 0.883991] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001627worker1 # [ 0.886652] usbcore: registered new interface driver usb-storage1628worker1 # [ 0.886751] usbcore: registered new interface driver usbserial_generic1629worker1 # [ 0.886765] usbserial: USB Serial support registered for generic1630worker1 # [ 0.889782] hv_vmbus: registering driver hyperv_keyboard1631worker1 # [ 0.891373] rtc-pl031 9010000.pl031: registered as rtc01632client # [ 0.961004] systemd[1]: Successfully made /usr/ read-only.1633worker1 # [ 0.891398] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:42:02 UTC (1790095322)1634worker1 # [ 0.891698] i2c_dev: i2c /dev entries driver1635worker1 # [ 0.895082] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001636worker1 # [ 0.896135] hub 1-0:1.0: USB hub found1637worker1 # [ 0.896658] hub 1-0:1.0: 6 ports detected1638worker2 # [ 0.954413] systemd[1]: Successfully made /usr/ read-only.1639lb # [ 0.924698] clk: Disabling unused clocks1640lb # [ 0.924719] PM: genpd: Disabling unused power domains1641worker1 # [ 0.899922] sdhci: Secure Digital Host Controller Interface driver1642lb # [ 0.928767] Freeing unused kernel memory: 4736K1643worker1 # [ 0.899933] sdhci: Copyright(c) Pierre Ossman1644lb # [ 0.928958] Run /init as init process1645worker1 # [ 0.900196] Synopsys Designware Multimedia Card Interface Driver1646worker1 # [ 0.900571] sdhci-pltfm: SDHCI platform and OF driver helper1647worker1 # [ 0.902068] hid: raw HID events driver (C) Jiri Kosina1648worker1 # [ 0.902314] usbcore: registered new interface driver usbhid1649worker1 # [ 0.902321] usbhid: USB HID core driver1650worker1 # [ 0.908049] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1651worker1 # [ 0.909606] drop_monitor: Initializing network drop monitor service1652worker1 # [ 0.909752] NET: Registered PF_INET6 protocol family1653worker1 # [ 0.912930] Segment Routing with IPv61654worker1 # [ 0.912962] In-situ OAM (IOAM) with IPv61655worker1 # [ 0.912995] NET: Registered PF_PACKET protocol family1656worker1 # [ 0.914713] 9pnet: Installing 9P2000 support1657worker1 # [ 0.914757] Key type dns_resolver registered1658lb # [ 0.943756] systemd[1]: Successfully made /usr/ read-only.1659worker1 # [ 0.921712] registered taskstats version 11660worker1 # [ 0.921865] Loading compiled-in X.509 certificates1661worker1 # [ 0.930318] Demotion targets for Node 0: null1662worker1 # [ 0.930416] Key type .fscrypt registered1663worker1 # [ 0.930422] Key type fscrypt-provisioning registered1664worker1 # [ 0.930507] ima: No TPM chip found, activating TPM-bypass!1665worker1 # [ 0.930527] ima: Allocated hash algorithm: sha11666worker1 # [ 0.930548] ima: No architecture policies found1667worker1 # [ 0.934794] input: gpio-keys as /devices/platform/gpio-keys/input/input01668worker1 # [ 0.952681] clk: Disabling unused clocks1669worker1 # [ 0.952702] PM: genpd: Disabling unused power domains1670worker1 # [ 0.956873] Freeing unused kernel memory: 4736K1671worker1 # [ 0.957069] Run /init as init process1672worker1 # [ 0.973177] systemd[1]: Successfully made /usr/ read-only.1673client # [ 1.126815] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker2 # [ 1.136308] usb 1-1: new high-speed USB device number 2 using ehci-pci1675lb # [ 1.125708] usb 1-1: new high-speed USB device number 2 using ehci-pci1676worker1 # [ 1.143077] usb 1-1: new high-speed USB device number 2 using ehci-pci1677client # [ 1.279348] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11678worker2 # [ 1.288846] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679client # [ 1.295550] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1680client # [ 1.307807] systemd[1]: Detected virtualization qemu.1681client # [ 1.309897] systemd[1]: Detected architecture arm64.1682client # [ 1.311887] systemd[1]: Running in initrd.1683worker2 # [ 1.295032] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1684client # [ 1.314482] systemd[1]: Initializing machine ID from random generator.1685worker2 # [ 1.307394] systemd[1]: Detected virtualization qemu.1686client # [ 1.317483] systemd[1]: Hostname set to <client>.1687worker2 # [ 1.309528] systemd[1]: Detected architecture arm64.1688worker2 # [ 1.311432] systemd[1]: Running in initrd.1689lb # [ 1.278115] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11690worker2 # [ 1.314046] systemd[1]: Initializing machine ID from random generator.1691worker2 # [ 1.317013] systemd[1]: Hostname set to <worker2>.1692lb # [ 1.284130] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1693lb # [ 1.296575] systemd[1]: Detected virtualization qemu.1694lb # [ 1.298714] systemd[1]: Detected architecture arm64.1695lb # [ 1.298751] systemd[1]: Running in initrd.1696lb # [ 1.299650] systemd[1]: Initializing machine ID from random generator.1697lb # [ 1.299921] systemd[1]: Hostname set to <lb>.1698worker1 # [ 1.295546] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11699client # [ 1.366993] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01700worker1 # [ 1.308292] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1701worker1 # [ 1.320688] systemd[1]: Detected virtualization qemu.1702worker1 # [ 1.322804] systemd[1]: Detected architecture arm64.1703worker1 # [ 1.324915] systemd[1]: Running in initrd.1704worker2 # [ 1.384443] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705worker1 # [ 1.327424] systemd[1]: Initializing machine ID from random generator.1706worker1 # [ 1.330226] systemd[1]: Hostname set to <worker1>.1707lb # [ 1.361892] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708worker1 # [ 1.379280] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01709client # [ 1.490819] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker2 # [ 1.504306] usb 1-2: new high-speed USB device number 3 using ehci-pci1711lb # [ 1.481716] usb 1-2: new high-speed USB device number 3 using ehci-pci1712worker1 # [ 1.503091] usb 1-2: new high-speed USB device number 3 using ehci-pci1713client # [ 1.624276] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714worker2 # [ 1.624601] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715lb # [ 1.600937] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716client # [ 1.651569] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21717client # [ 1.657222] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01718worker2 # [ 1.660868] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719worker2 # [ 1.665866] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01720lb # [ 1.639228] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721lb # [ 1.639514] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722worker1 # [ 1.636740] systemd[1]: bpf-restrict-fs: LSM BPF program attached1723worker1 # [ 1.663765] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21724worker1 # [ 1.669283] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01725client # [ 1.743450] systemd[1]: Queued start job for default target Initrd Default Target.1726worker2 # [ 1.742419] systemd[1]: Queued start job for default target Initrd Default Target.1727client # [ 1.753295] systemd[1]: Created slice Slice /system/modprobe.1728client # [ 1.754496] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1729lb # [ 1.713220] systemd[1]: Queued start job for default target Initrd Default Target.1730client # [ 1.755948] systemd[1]: Expecting device /dev/disk/by-label/nixos...1731client # [ 1.757045] systemd[1]: Reached target Path Units.1732client # [ 1.757887] systemd[1]: Reached target Slice Units.1733client # [ 1.758868] systemd[1]: Reached target Swaps.1734client # [ 1.759638] systemd[1]: Reached target Timer Units.1735worker2 # [ 1.750854] systemd[1]: Created slice Slice /system/modprobe.1736client # [ 1.760688] systemd[1]: Listening on D-Bus System Message Bus Socket.1737client # [ 1.761962] systemd[1]: Listening on Journal Socket (/dev/log).1738worker2 # [ 1.752225] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1739worker2 # [ 1.753777] systemd[1]: Expecting device /dev/disk/by-label/nixos...1740client # [ 1.763173] systemd[1]: Listening on Journal Sockets.1741worker2 # [ 1.755074] systemd[1]: Reached target Path Units.1742client # [ 1.763322] systemd[1]: Listening on udev Control Socket.1743worker2 # [ 1.755996] systemd[1]: Reached target Slice Units.1744lb # [ 1.724077] systemd[1]: Created slice Slice /system/modprobe.1745client # [ 1.763451] systemd[1]: Listening on udev Kernel Socket.1746worker2 # [ 1.756998] systemd[1]: Reached target Swaps.1747client # [ 1.763474] systemd[1]: Reached target Socket Units.1748worker2 # [ 1.757835] systemd[1]: Reached target Timer Units.1749lb # [ 1.725313] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1750lb # [ 1.726778] systemd[1]: Expecting device /dev/disk/by-label/nixos...1751client # [ 1.768704] systemd[1]: Starting Create List of Static Device Nodes...1752worker2 # [ 1.758971] systemd[1]: Listening on D-Bus System Message Bus Socket.1753lb # [ 1.727870] systemd[1]: Reached target Path Units.1754worker2 # [ 1.760371] systemd[1]: Listening on Journal Socket (/dev/log).1755lb # [ 1.728713] systemd[1]: Reached target Slice Units.1756client # [ 1.769883] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1757lb # [ 1.729575] systemd[1]: Reached target Swaps.1758worker2 # [ 1.761635] systemd[1]: Listening on Journal Sockets.1759lb # [ 1.730450] systemd[1]: Reached target Timer Units.1760worker2 # [ 1.762763] systemd[1]: Listening on udev Control Socket.1761lb # [ 1.731492] systemd[1]: Listening on D-Bus System Message Bus Socket.1762worker2 # [ 1.763918] systemd[1]: Listening on udev Kernel Socket.1763lb # [ 1.732767] systemd[1]: Listening on Journal Socket (/dev/log).1764worker2 # [ 1.765029] systemd[1]: Reached target Socket Units.1765lb # [ 1.734094] systemd[1]: Listening on Journal Sockets.1766worker2 # [ 1.767810] systemd[1]: Starting Create List of Static Device Nodes...1767lb # [ 1.734254] systemd[1]: Listening on udev Control Socket.1768lb # [ 1.734373] systemd[1]: Listening on udev Kernel Socket.1769worker2 # [ 1.769188] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1770lb # [ 1.734397] systemd[1]: Reached target Socket Units.1771client # [ 1.778910] systemd[1]: Mounting Kernel Configuration File System...1772lb # [ 1.739509] systemd[1]: Starting Create List of Static Device Nodes...1773lb # [ 1.740688] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1774client # [ 1.787033] systemd[1]: Starting Journal Service...1775worker2 # [ 1.777046] systemd[1]: Mounting Kernel Configuration File System...1776lb # [ 1.749805] systemd[1]: Mounting Kernel Configuration File System...1777worker2 # [ 1.788551] systemd[1]: Starting Journal Service...1778lb # [ 1.757963] systemd[1]: Starting Journal Service...1779client # [ 1.818917] systemd[1]: Starting Load Kernel Modules...1780client # [ 1.819039] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1781worker2 # [ 1.812386] systemd[1]: Starting Load Kernel Modules...1782worker1 # [ 1.754654] systemd[1]: Queued start job for default target Initrd Default Target.1783lb # [ 1.782062] systemd[1]: Starting Load Kernel Modules...1784lb # [ 1.782183] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1785worker2 # [ 1.816489] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1786worker1 # [ 1.765396] systemd[1]: Created slice Slice /system/modprobe.1787worker1 # [ 1.766621] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1788worker1 # [ 1.768101] systemd[1]: Expecting device /dev/disk/by-label/nixos...1789client # [ 1.835135] systemd[1]: Starting Coldplug All udev Devices...1790worker1 # [ 1.769229] systemd[1]: Reached target Path Units.1791worker1 # [ 1.770076] systemd[1]: Reached target Slice Units.1792worker2 # [ 1.828433] systemd[1]: Starting Coldplug All udev Devices...1793worker1 # [ 1.770991] systemd[1]: Reached target Swaps.1794worker1 # [ 1.771799] systemd[1]: Reached target Timer Units.1795worker1 # [ 1.772852] systemd[1]: Listening on D-Bus System Message Bus Socket.1796worker1 # [ 1.774116] systemd[1]: Listening on Journal Socket (/dev/log).1797worker1 # [ 1.775331] systemd[1]: Listening on Journal Sockets.1798worker2 # [ 1.835211] systemd-journald[72]: Collecting audit messages is disabled.1799worker1 # [ 1.775472] systemd[1]: Listening on udev Control Socket.1800worker1 # [ 1.775599] systemd[1]: Listening on udev Kernel Socket.1801worker1 # [ 1.775622] systemd[1]: Reached target Socket Units.1802lb # [ 1.805904] systemd[1]: Starting Coldplug All udev Devices...1803worker1 # [ 1.780757] systemd[1]: Starting Create List of Static Device Nodes...1804worker1 # [ 1.781941] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1805client # [ 1.851810] systemd-journald[72]: Collecting audit messages is disabled.1806client # [ 1.855685] systemd[1]: Finished Create List of Static Device Nodes.1807client # [ 1.856486] systemd[1]: Mounted Kernel Configuration File System.1808worker1 # [ 1.791162] systemd[1]: Mounting Kernel Configuration File System...1809worker2 # [ 1.852514] systemd[1]: Finished Create List of Static Device Nodes.1810worker2 # [ 1.853203] systemd[1]: Mounted Kernel Configuration File System.1811worker1 # [ 1.799311] systemd[1]: Starting Journal Service...1812client # [ 1.868337] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1813lb # [ 1.823503] systemd-journald[72]: Collecting audit messages is disabled.1814lb # [ 1.828113] systemd[1]: Finished Create List of Static Device Nodes.1815lb # [ 1.828763] systemd[1]: Mounted Kernel Configuration File System.1816worker2 # [ 1.865524] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1817client # [ 1.878815] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1818lb # [ 1.838154] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1819client # [ 1.886855] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1820worker1 # [ 1.823492] systemd[1]: Starting Load Kernel Modules...1821worker1 # [ 1.824509] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1822lb # [ 1.850079] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1823client # [ 1.895957] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01824client # [ 1.896199] [drm] features: -virgl +edid -resource_blob -host_visible1825client # [ 1.896209] [drm] features: -context_init1826client # [ 1.896889] [drm] number of scanouts: 11827client # [ 1.896908] [drm] number of cap sets: 01828lb # [ 1.861696] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1829worker1 # [ 1.842988] systemd[1]: Starting Coldplug All udev Devices...1830worker2 # [ 1.897742] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1831lb # [ 1.868869] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01832lb # [ 1.869101] [drm] features: -virgl +edid -resource_blob -host_visible1833lb # [ 1.869111] [drm] features: -context_init1834worker1 # [ 1.852963] systemd-journald[72]: Collecting audit messages is disabled.1835client # [ 1.923980] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1836client # [ 1.923999] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01837worker2 # [ 1.920861] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1838worker2 # [ 1.922983] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1839client # [ 1.932289] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1840worker2 # [ 1.925401] systemd[1]: Starting Create Static Device Nodes in /dev...1841lb # [ 1.894236] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1842lb # [ 1.896599] systemd[1]: Starting Create Static Device Nodes in /dev...1843worker1 # [ 1.871340] systemd[1]: Finished Create List of Static Device Nodes.1844worker1 # [ 1.872045] systemd[1]: Mounted Kernel Configuration File System.1845lb # [ 1.901950] [drm] number of scanouts: 11846lb # [ 1.901979] [drm] number of cap sets: 01847worker1 # [ 1.875071] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1848client # [ 1.943199] systemd[1]: Starting Create Static Device Nodes in /dev...1849worker2 # [ 1.937489] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01850worker2 # [ 1.937726] [drm] features: -virgl +edid -resource_blob -host_visible1851worker2 # [ 1.937735] [drm] features: -context_init1852worker2 # [ 1.938429] [drm] number of scanouts: 11853worker2 # [ 1.938447] [drm] number of cap sets: 01854lb # [ 1.903487] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1855lb # [ 1.903501] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01856client # [ 1.947947] Console: switching to colour frame buffer device 160x501857worker2 # [ 1.956587] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1858worker2 # [ 1.956605] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01859lb # [ 1.926172] Console: switching to colour frame buffer device 160x501860worker1 # [ 1.904583] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1861client # [ 1.980069] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1862worker1 # [ 1.915627] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1863client # [ 1.983324] systemd[1]: Finished Create Static Device Nodes in /dev.1864client # [ 1.983670] systemd[1]: Reached target Preparation for Local File Systems.1865client # [ 1.983708] systemd[1]: Reached target Local File Systems.1866worker1 # [ 1.919328] systemd[1]: Starting Create Static Device Nodes in /dev...1867lb # [ 1.946354] systemd[1]: Finished Create Static Device Nodes in /dev.1868lb # [ 1.946667] systemd[1]: Reached target Preparation for Local File Systems.1869lb # [ 1.946692] systemd[1]: Reached target Local File Systems.1870worker1 # [ 1.923127] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1871client # [ 1.988539] systemd[1]: Starting Rule-based Manager for Device Events and Files...1872worker2 # [ 1.980709] systemd[1]: Finished Create Static Device Nodes in /dev.1873worker2 # [ 1.981019] systemd[1]: Reached target Preparation for Local File Systems.1874worker2 # [ 1.981047] systemd[1]: Reached target Local File Systems.1875lb # [ 1.950481] systemd[1]: Starting Rule-based Manager for Device Events and Files...1876worker2 # [ 1.984880] systemd[1]: Starting Rule-based Manager for Device Events and Files...1877worker1 # [ 1.932292] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01878client # [ 1.999259] systemd[1]: Finished Load Kernel Modules.1879worker1 # [ 1.932525] [drm] features: -virgl +edid -resource_blob -host_visible1880worker1 # [ 1.932544] [drm] features: -context_init1881worker1 # [ 1.933252] [drm] number of scanouts: 11882worker1 # [ 1.933269] [drm] number of cap sets: 01883lb # [ 1.962280] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1884worker2 # [ 1.992460] Console: switching to colour frame buffer device 160x501885worker2 # [ 1.998848] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1886client # [ 2.022398] systemd[1]: Starting Apply Kernel Variables...1887lb # [ 1.981884] systemd[1]: Finished Load Kernel Modules.1888worker1 # [ 1.951355] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1889worker1 # [ 1.951370] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01890lb # [ 1.984948] systemd[1]: Starting Apply Kernel Variables...1891worker2 # [ 2.008513] systemd-modules-load[74]: Inserted module 'dm_mod'1892worker2 # [ 2.021883] systemd[1]: Finished Load Kernel Modules.1893worker2 # [ 2.023758] systemd[1]: Started Journal Service.1894worker2 # [ 2.014415] systemd-modules-load[74]: Module 'virtio_balloon' is built in1895worker2 # [ 2.015537] systemd-modules-load[74]: Module 'virtio_console' is built in1896client # [ 2.028379] systemd-modules-load[73]: Inserted module 'dm_mod'1897worker1 # [ 1.975586] systemd[1]: Finished Create Static Device Nodes in /dev.1898worker1 # [ 1.975900] systemd[1]: Reached target Preparation for Local File Systems.1899client # [ 2.044249] systemd[1]: Started Journal Service.1900worker1 # [ 1.975927] systemd[1]: Reached target Local File Systems.1901worker2 # [ 2.024346] systemd-modules-load[74]: Inserted module 'virtio_gpu'1902worker2 # [ 2.025331] systemd-modules-load[74]: Module 'virtio_rng' is built in1903client # [ 2.032595] systemd-modules-load[73]: Module 'virtio_balloon' is built in1904worker2 # [ 2.026345] systemd[1]: Starting Apply Kernel Variables...1905client # [ 2.033707] systemd-modules-load[73]: Module 'virtio_console' is built in1906client # [ 2.034765] systemd-modules-load[73]: Inserted module 'virtio_gpu'1907client # [ 2.035762] systemd-modules-load[73]: Module 'virtio_rng' is built in1908worker1 # [ 1.979974] systemd[1]: Starting Rule-based Manager for Device Events and Files...1909lb # [ 1.996384] systemd-modules-load[74]: Inserted module 'dm_mod'1910lb # [ 2.011093] systemd[1]: Started Journal Service.1911lb # [ 2.004301] systemd-modules-load[74]: Module 'virtio_balloon' is built in1912worker1 # [ 1.986329] Console: switching to colour frame buffer device 160x501913lb # [ 2.005424] systemd-modules-load[74]: Module 'virtio_console' is built in1914lb # [ 2.006604] systemd-modules-load[74]: Inserted module 'virtio_gpu'1915lb # [ 2.007587] systemd-modules-load[74]: Module 'virtio_rng' is built in1916client # [ 2.056383] systemd[1]: Starting Create System Files and Directories...1917worker2 # [ 2.052941] systemd[1]: Starting Create System Files and Directories...1918lb # [ 2.021537] systemd[1]: Starting Create System Files and Directories...1919client # [ 2.064074] systemd[1]: Finished Apply Kernel Variables.1920worker1 # [ 2.015661] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1921lb # [ 2.034673] systemd-udevd[79]: Using default interface naming scheme 'v261'.1922client # [ 2.078040] systemd-udevd[79]: Using default interface naming scheme 'v261'.1923worker2 # [ 2.076345] systemd-udevd[78]: Using default interface naming scheme 'v261'.1924lb # [ 2.044327] systemd[1]: Finished Apply Kernel Variables.1925worker1 # [ 2.031690] systemd[1]: Finished Load Kernel Modules.1926worker1 # [ 2.034686] systemd[1]: Starting Apply Kernel Variables...1927client # [ 2.093193] systemd[1]: Finished Create System Files and Directories.1928lb # [ 2.057852] systemd[1]: Finished Create System Files and Directories.1929worker2 # [ 2.096206] systemd[1]: Finished Apply Kernel Variables.1930worker1 # [ 2.051168] systemd[1]: Started Journal Service.1931worker1 # [ 2.039022] systemd-modules-load[74]: Inserted module 'dm_mod'1932worker1 # [ 2.041446] systemd-modules-load[74]: Module 'virtio_balloon' is built in1933worker1 # [ 2.042618] systemd-modules-load[74]: Module 'virtio_console' is built in1934client # [ 2.110542] systemd[1]: Started Rule-based Manager for Device Events and Files.1935worker1 # [ 2.043732] systemd-modules-load[74]: Inserted module 'virtio_gpu'1936worker2 # [ 2.104991] systemd[1]: Finished Create System Files and Directories.1937lb # [ 2.077102] systemd[1]: Started Rule-based Manager for Device Events and Files.1938worker2 # [ 2.111687] systemd[1]: Started Rule-based Manager for Device Events and Files.1939worker1 # [ 2.056377] systemd-modules-load[74]: Module 'virtio_rng' is built in1940worker1 # [ 2.064107] systemd[1]: Starting Create System Files and Directories...1941worker1 # [ 2.077486] systemd-udevd[78]: Using default interface naming scheme 'v261'.1942worker1 # [ 2.081039] systemd[1]: Finished Apply Kernel Variables.1943client # [ 2.168090] systemd[1]: Starting Virtual Console Setup...1944lb # [ 2.128683] systemd[1]: Starting Virtual Console Setup...1945worker1 # [ 2.105076] systemd[1]: Finished Create System Files and Directories.1946worker2 # [ 2.169215] systemd[1]: Starting Virtual Console Setup...1947worker1 # [ 2.121651] systemd[1]: Started Rule-based Manager for Device Events and Files.1948client # [ 2.212418] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1949client # [ 2.215819] systemd[1]: Finished Virtual Console Setup.1950lb # [ 2.180441] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1951worker2 # [ 2.216430] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1952lb # [ 2.183689] systemd[1]: Finished Virtual Console Setup.1953worker2 # [ 2.219646] systemd[1]: Finished Virtual Console Setup.1954worker1 # [ 2.172613] systemd[1]: Starting Virtual Console Setup...1955worker1 # [ 2.224439] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1956worker1 # [ 2.227724] systemd[1]: Finished Virtual Console Setup.1957worker2 # [ 2.811615] systemd[1]: Finished Coldplug All udev Devices.1958worker2 # [ 2.812999] systemd[1]: Reached target System Initialization.1959worker2 # [ 2.814174] systemd[1]: Reached target Basic System.1960lb # [ 2.783264] systemd[1]: Finished Coldplug All udev Devices.1961lb # [ 2.784264] systemd[1]: Reached target System Initialization.1962lb # [ 2.785237] systemd[1]: Reached target Basic System.1963client # [ 2.836159] systemd[1]: Finished Coldplug All udev Devices.1964client # [ 2.837731] systemd[1]: Reached target System Initialization.1965client # [ 2.838829] systemd[1]: Reached target Basic System.1966worker1 # [ 2.856385] systemd[1]: Finished Coldplug All udev Devices.1967worker1 # [ 2.857978] systemd[1]: Reached target System Initialization.1968worker1 # [ 2.858839] systemd[1]: Reached target Basic System.1969worker2 # [ 2.971545] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line.1970client # [ 2.987330] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1971lb # [ 2.950545] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972lb # [ 2.955369] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1973client # [ 3.006817] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1974client # [ 3.012237] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1975worker2 # [ 3.003779] (udev-worker)[103]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976lb # [ 2.976361] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1977worker2 # [ 3.010451] (udev-worker)[103]: Network interface NamePolicy= disabled on kernel command line.1978worker1 # [ 2.983982] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1979worker2 # [ 3.057647] systemd[1]: Found device /dev/disk/by-label/nixos.1980worker2 # [ 3.060140] systemd[1]: Reached target Initrd Root Device.1981lb # [ 3.029091] systemd[1]: Found device /dev/disk/by-label/nixos.1982worker2 # [ 3.062842] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1983lb # [ 3.032364] systemd[1]: Reached target Initrd Root Device.1984lb # [ 3.036153] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985client # [ 3.087987] systemd[1]: Found device /dev/disk/by-label/nixos.1986client # [ 3.090753] systemd[1]: Reached target Initrd Root Device.1987client # [ 3.096149] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1988worker1 # [ 3.039105] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1989worker1 # [ 3.044726] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1990worker2 # [ 3.112374] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1991worker2 # [ 3.117625] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1992lb # [ 3.088812] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1993worker2 # [ 3.123607] systemd[1]: Mounting /sysroot...1994lb # [ 3.094732] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1995client # [ 3.145530] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1996lb # [ 3.108205] systemd[1]: Mounting /sysroot...1997client # [ 3.152111] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1998client # [ 3.154611] systemd[1]: Mounting /sysroot...1999worker2 # [ 3.166461] EXT4-fs (vda): mounted filesystem 796090bf-a266-4652-b1df-ba99deb215a9 r/w with ordered data mode. Quota mode: none.2000worker2 # [ 3.159323] systemd[1]: Mounted /sysroot.
loading 2,267 hidden lines…
4268lb # [ 50.503790] niks3-server[992]: 2026/09/22 16:42:52 INFO Completed upload id=64269worker1 # [ 50.477152] nix-grpc-daemon[866]: time=2026-09-22T16:42:52.100Z level=INFO msg="Upload complete. (300ms)"4270lb # [ 50.506507] niks3-server[992]: 2026/09/22 16:42:52 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4271worker1 # [ 50.489202] nix-grpc-daemon[768]: ts=2026-09-22T16:42:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14272worker1 # [ 50.512075] nix-daemon[851]: reaped child process 949, status = succeeded4273client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.53 seconds)4274client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4275client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.18 seconds)4276client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4277client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4278client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4279client # copying 3 paths...4280client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4281client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4282client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4283client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)4284(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.05 seconds)4285subtest: repeat is answered Cached by the scheduler without building4286worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4287worker1 # finding garbage collector roots...4288worker1 # deleting specified paths...4289worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4290worker1 # deleting unused links...4291worker1 # note: hard linking is currently saving 0.0 KiB4292worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds)4293worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4294worker2 # finding garbage collector roots...4295worker2 # deleting specified paths...4296worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4297worker2 # deleting unused links...4298worker2 # note: hard linking is currently saving 0.0 KiB4299worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.25 seconds)4300client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24301worker2 # [ 51.824925] nix-grpc-daemon[769]: ts=2026-09-22T16:42:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:459464302client # don't know how to build these paths:4303client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4304worker2 # [ 51.834674] nix-grpc-daemon[769]: ts=2026-09-22T16:42:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459464305worker1 # [ 51.805714] nix-grpc-daemon[768]: ts=2026-09-22T16:42:53Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14306client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.30 seconds)4307client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4308client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.15 seconds)4309lb: must succeed: curl -sf localhost:9901/clusters4310lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4311lb: must succeed: curl -sf localhost:9901/clusters4312lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4313worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4314worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4315(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.18 seconds)4316subtest: inputs already in the cache are not sent to the scheduler4317lb: must succeed: curl -sf localhost:9901/clusters4318lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4319lb: must succeed: curl -sf localhost:9901/clusters4320lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4321worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4322worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4323client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24324client # don't know how to build these paths:4325client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4326worker1 # [ 52.627011] nix-grpc-daemon[768]: ts=2026-09-22T16:42:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582724327worker1 # [ 52.634807] nix-grpc-daemon[768]: ts=2026-09-22T16:42:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724328worker1 # [ 52.645521] nix-grpc-daemon[866]: time=2026-09-22T16:42:54.268Z level=INFO msg="All 2 paths already cached"4329worker1 # [ 52.661414] nix-grpc-daemon[768]: ts=2026-09-22T16:42:54Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04330worker1 # [ 52.668408] nix-grpc-daemon[768]: ts=2026-09-22T16:42:54Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14331worker1 # [ 52.714534] nix-grpc-daemon[768]: ts=2026-09-22T16:42:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724332worker1 # [ 52.730520] nix-grpc-daemon[768]: ts=2026-09-22T16:42:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724333worker1 # [ 52.745786] nix-grpc-daemon[866]: time=2026-09-22T16:42:54.369Z level=INFO msg="All 2 paths already cached"4334client # copying 1 paths to 'lb:50051'...4335client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4336worker1 # [ 52.757421] nix-grpc-daemon[768]: ts=2026-09-22T16:42:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582724337worker1 # [ 52.775565] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4338worker1 # [ 52.934172] nix-daemon[851]: accepted connection from pid 999, user nix-grpc-daemon (trusted)4339worker1 # [ 52.954825] nix-daemon[851]: reaped child process 1002, status = succeeded4340lb # [ 52.987431] niks3-server[992]: 2026/09/22 16:42:54 INFO Received uploads request method=POST path=/api/pending_closures4341worker1 # [ 52.970582] nix-grpc-daemon[866]: time=2026-09-22T16:42:54.593Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4342worker1 # [ 52.972505] nix-grpc-daemon[866]: time=2026-09-22T16:42:54.593Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4343lb # [ 53.016159] niks3-server[992]: 2026/09/22 16:42:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4344worker1 # [ 52.998956] nix-grpc-daemon[866]: time=2026-09-22T16:42:54.622Z level=INFO msg="Uploading 1 narinfos"4345lb # [ 53.031559] niks3-server[992]: 2026/09/22 16:42:54 INFO Signed narinfos id=7 count=14346lb # [ 53.039935] niks3-server[992]: 2026/09/22 16:42:54 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4347lb # [ 53.054784] niks3-server[992]: 2026/09/22 16:42:54 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4348worker1 # [ 53.034399] nix-grpc-daemon[866]: time=2026-09-22T16:42:54.657Z level=INFO msg="Upload complete. (240ms)"4349lb # [ 53.062297] niks3-server[992]: 2026/09/22 16:42:54 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4350lb # [ 53.071428] niks3-server[992]: 2026/09/22 16:42:54 INFO Completed upload id=74351worker1 # [ 53.044826] nix-grpc-daemon[768]: ts=2026-09-22T16:42:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_in=7524352lb # [ 53.076763] niks3-server[992]: 2026/09/22 16:42:54 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4353worker1 # [ 53.060530] nix-grpc-daemon[768]: ts=2026-09-22T16:42:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724354worker1 # [ 53.067871] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4355worker1 # [ 53.074033] nix-daemon[851]: reaped child process 997, status = succeeded4356client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4357client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4358worker1 # [ 53.089114] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4359client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4360client # farm-top> LOG-farm-top-t1b4361worker1 # [ 54.196587] nix-daemon[851]: reaped child process 1006, status = succeeded4362worker1 # [ 54.271825] nix-daemon[851]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)4363worker1 # [ 54.291677] nix-daemon[851]: reaped child process 1025, status = succeeded4364lb # [ 54.325682] niks3-server[992]: 2026/09/22 16:42:55 INFO Received uploads request method=POST path=/api/pending_closures4365worker1 # [ 54.308467] nix-grpc-daemon[866]: time=2026-09-22T16:42:55.931Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4366worker1 # [ 54.313712] nix-grpc-daemon[866]: time=2026-09-22T16:42:55.931Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4367lb # [ 54.370203] niks3-server[992]: 2026/09/22 16:42:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4368worker1 # [ 54.356176] nix-grpc-daemon[866]: time=2026-09-22T16:42:55.979Z level=INFO msg="Uploading 1 narinfos"4369lb # [ 54.390355] niks3-server[992]: 2026/09/22 16:42:56 INFO Signed narinfos id=8 count=14370lb # [ 54.396326] niks3-server[992]: 2026/09/22 16:42:56 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4371lb # [ 54.404970] niks3-server[992]: 2026/09/22 16:42:56 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4372lb # [ 54.409877] niks3-server[992]: 2026/09/22 16:42:56 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4373lb # [ 54.417672] niks3-server[992]: 2026/09/22 16:42:56 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4374lb # [ 54.421935] niks3-server[992]: 2026/09/22 16:42:56 INFO Completed upload id=84375worker1 # [ 54.395304] nix-grpc-daemon[866]: time=2026-09-22T16:42:56.018Z level=INFO msg="Upload complete. (294ms)"4376lb # [ 54.424968] niks3-server[992]: 2026/09/22 16:42:56 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4377worker1 # [ 54.409256] nix-grpc-daemon[768]: ts=2026-09-22T16:42:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14378worker1 # [ 54.428827] nix-daemon[851]: reaped child process 1004, status = succeeded4379client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.03 seconds)4380client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4381client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds)4382lb: must succeed: curl -sf localhost:9901/clusters4383lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4384lb: must succeed: curl -sf localhost:9901/clusters4385lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4386worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4387worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4388(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.54 seconds)4389subtest: outputs read back through any worker carry the cache signature4390client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24391worker1 # [ 55.011738] nix-grpc-daemon[768]: ts=2026-09-22T16:42:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582724392client # don't know how to build these paths:4393client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4394worker1 # [ 55.023283] nix-grpc-daemon[768]: ts=2026-09-22T16:42:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724395worker1 # [ 55.047186] nix-grpc-daemon[768]: ts=2026-09-22T16:42:56Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04396worker2 # [ 55.113478] nix-grpc-daemon[769]: ts=2026-09-22T16:42:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464397worker1 # [ 55.053677] nix-grpc-daemon[768]: ts=2026-09-22T16:42:56Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14398worker2 # [ 55.131695] nix-grpc-daemon[769]: ts=2026-09-22T16:42:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459464399client # copying 1 paths to 'lb:50051'...4400client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4401worker2 # [ 55.147964] nix-grpc-daemon[769]: ts=2026-09-22T16:42:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459464402worker2 # [ 55.164182] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4403worker2 # [ 55.194026] nix-grpc-daemon[769]: ts=2026-09-22T16:42:56Z level=info event=niks3_push_started pid=8654404worker2 # [ 55.358256] nix-grpc-daemon[865]: time=2026-09-22T16:42:56.993Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4405worker2 # [ 55.465305] nix-daemon[838]: accepted connection from pid 877, user nix-grpc-daemon (trusted)4406worker2 # [ 55.488143] nix-daemon[838]: reaped child process 880, status = succeeded4407lb # [ 55.459162] niks3-server[992]: 2026/09/22 16:42:57 INFO Received uploads request method=POST path=/api/pending_closures4408worker2 # [ 55.502843] nix-grpc-daemon[865]: time=2026-09-22T16:42:57.137Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4409worker2 # [ 55.506647] nix-grpc-daemon[865]: time=2026-09-22T16:42:57.137Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4410lb # [ 55.494944] niks3-server[992]: 2026/09/22 16:42:57 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4411worker2 # [ 55.539132] nix-grpc-daemon[865]: time=2026-09-22T16:42:57.174Z level=INFO msg="Uploading 1 narinfos"4412lb # [ 55.507947] niks3-server[992]: 2026/09/22 16:42:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4413lb # [ 55.517231] niks3-server[992]: 2026/09/22 16:42:57 INFO Signed narinfos id=9 count=14414lb # [ 55.527770] niks3-server[992]: 2026/09/22 16:42:57 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4415worker2 # [ 55.570825] nix-grpc-daemon[865]: time=2026-09-22T16:42:57.206Z level=INFO msg="Upload complete. (208ms)"4416lb # [ 55.537227] niks3-server[992]: 2026/09/22 16:42:57 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4417worker2 # [ 55.579282] nix-grpc-daemon[769]: ts=2026-09-22T16:42:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_in=5124418lb # [ 55.548546] niks3-server[992]: 2026/09/22 16:42:57 INFO Completed upload id=94419lb # [ 55.551072] niks3-server[992]: 2026/09/22 16:42:57 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4420worker2 # [ 55.593962] nix-grpc-daemon[769]: ts=2026-09-22T16:42:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464421worker2 # [ 55.599764] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4422worker2 # [ 55.605045] nix-daemon[838]: reaped child process 863, status = succeeded4423client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4424worker2 # [ 55.617711] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4425client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4426client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4427client # farm-a> LOG-farm-a-sig4428worker2 # [ 57.308183] nix-daemon[838]: reaped child process 884, status = succeeded4429worker2 # [ 57.397349] nix-daemon[838]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4430worker2 # [ 57.417211] nix-daemon[838]: reaped child process 904, status = succeeded4431lb # [ 57.390205] niks3-server[992]: 2026/09/22 16:42:59 INFO Received uploads request method=POST path=/api/pending_closures4432worker2 # [ 57.436565] nix-grpc-daemon[865]: time=2026-09-22T16:42:59.071Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4433worker2 # [ 57.438683] nix-grpc-daemon[865]: time=2026-09-22T16:42:59.071Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4434lb # [ 57.442752] niks3-server[992]: 2026/09/22 16:42:59 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4435worker2 # [ 57.487931] nix-grpc-daemon[865]: time=2026-09-22T16:42:59.123Z level=INFO msg="Uploading 1 narinfos"4436lb # [ 57.454318] niks3-server[992]: 2026/09/22 16:42:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4437lb # [ 57.463481] niks3-server[992]: 2026/09/22 16:42:59 INFO Signed narinfos id=10 count=14438lb # [ 57.470313] niks3-server[992]: 2026/09/22 16:42:59 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4439lb # [ 57.476432] niks3-server[992]: 2026/09/22 16:42:59 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4440lb # [ 57.482990] niks3-server[992]: 2026/09/22 16:42:59 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4441lb # [ 57.490591] niks3-server[992]: 2026/09/22 16:42:59 INFO Completed upload id=104442worker2 # [ 57.523681] nix-grpc-daemon[865]: time=2026-09-22T16:42:59.159Z level=INFO msg="Upload complete. (295ms)"4443lb # [ 57.492463] niks3-server[992]: 2026/09/22 16:42:59 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4444worker2 # [ 57.537294] nix-grpc-daemon[769]: ts=2026-09-22T16:42:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14445worker1 # [ 57.480953] nix-grpc-daemon[768]: ts=2026-09-22T16:42:59Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04446worker1 # [ 57.481295] nix-grpc-daemon[768]: ts=2026-09-22T16:42:59Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14447worker2 # [ 57.557916] nix-daemon[838]: reaped child process 882, status = succeeded4448worker2 # [ 57.586639] nix-grpc-daemon[769]: ts=2026-09-22T16:42:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464449worker2 # [ 57.603170] nix-grpc-daemon[769]: ts=2026-09-22T16:42:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459464450worker2 # [ 57.617651] nix-grpc-daemon[865]: time=2026-09-22T16:42:59.253Z level=INFO msg="All 1 paths already cached"4451client # copying 1 paths to 'lb:50051'...4452client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4453worker2 # [ 57.629082] nix-grpc-daemon[769]: ts=2026-09-22T16:42:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459464454worker2 # [ 57.642900] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4455worker2 # [ 57.799174] nix-daemon[838]: accepted connection from pid 908, user nix-grpc-daemon (trusted)4456worker2 # [ 57.820875] nix-daemon[838]: reaped child process 911, status = succeeded4457lb # [ 57.791448] niks3-server[992]: 2026/09/22 16:42:59 INFO Received uploads request method=POST path=/api/pending_closures4458worker2 # [ 57.834253] nix-grpc-daemon[865]: time=2026-09-22T16:42:59.469Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4459worker2 # [ 57.835769] nix-grpc-daemon[865]: time=2026-09-22T16:42:59.469Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4460lb # [ 57.820619] niks3-server[992]: 2026/09/22 16:42:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4461worker2 # [ 57.860890] nix-grpc-daemon[865]: time=2026-09-22T16:42:59.496Z level=INFO msg="Uploading 1 narinfos"4462lb # [ 57.830260] niks3-server[992]: 2026/09/22 16:42:59 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4463lb # [ 57.837489] niks3-server[992]: 2026/09/22 16:42:59 INFO Signed narinfos id=11 count=14464lb # [ 57.843460] niks3-server[992]: 2026/09/22 16:42:59 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4465worker2 # [ 57.880721] nix-grpc-daemon[865]: time=2026-09-22T16:42:59.516Z level=INFO msg="Upload complete. (221ms)"4466lb # [ 57.847315] niks3-server[992]: 2026/09/22 16:42:59 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4467lb # [ 57.851173] niks3-server[992]: 2026/09/22 16:42:59 INFO Completed upload id=114468lb # [ 57.854267] niks3-server[992]: 2026/09/22 16:42:59 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4469worker2 # [ 57.888155] nix-grpc-daemon[769]: ts=2026-09-22T16:42:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_in=6244470worker2 # [ 57.901718] nix-grpc-daemon[769]: ts=2026-09-22T16:42:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464471worker2 # [ 57.908605] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4472worker2 # [ 57.913326] nix-daemon[838]: reaped child process 906, status = succeeded4473client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4474worker2 # [ 57.926674] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4475client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4476client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4477client # farm-b> LOG-farm-b-sig4478worker2 # [ 59.158815] nix-daemon[838]: reaped child process 915, status = succeeded4479lb # [ 59.213572] envoy[899]: conn peer=[2001:db8:1::1]:52730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9575 tx=4383 ms=63144480worker2 # [ 59.251597] nix-daemon[838]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4481lb # [ 59.218595] envoy[899]: conn peer=[2001:db8:1::1]:50750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=481 ms=694482lb # [ 59.223532] envoy[899]: conn peer=[2001:db8:1::1]:50760 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1919 ms=17934483worker2 # [ 59.269995] nix-daemon[838]: reaped child process 934, status = succeeded4484lb # [ 59.245265] niks3-server[992]: 2026/09/22 16:43:00 INFO Received uploads request method=POST path=/api/pending_closures4485worker2 # [ 59.287929] nix-grpc-daemon[865]: time=2026-09-22T16:43:00.922Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4486worker2 # [ 59.289645] nix-grpc-daemon[865]: time=2026-09-22T16:43:00.922Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4487lb # [ 59.281393] niks3-server[992]: 2026/09/22 16:43:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4488worker2 # [ 59.321404] nix-grpc-daemon[865]: time=2026-09-22T16:43:00.957Z level=INFO msg="Uploading 1 narinfos"4489lb # [ 59.296582] niks3-server[992]: 2026/09/22 16:43:00 INFO Signed narinfos id=12 count=14490lb # [ 59.304138] niks3-server[992]: 2026/09/22 16:43:00 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4491lb # [ 59.307768] niks3-server[992]: 2026/09/22 16:43:00 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4492lb # [ 59.319905] niks3-server[992]: 2026/09/22 16:43:00 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4493worker2 # [ 59.361694] nix-grpc-daemon[865]: time=2026-09-22T16:43:00.997Z level=INFO msg="Upload complete. (225ms)"4494lb # [ 59.323869] niks3-server[992]: 2026/09/22 16:43:00 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4495lb # [ 59.334049] niks3-server[992]: 2026/09/22 16:43:00 INFO Completed upload id=124496lb # [ 59.337713] niks3-server[992]: 2026/09/22 16:43:00 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4497worker2 # [ 59.372923] nix-grpc-daemon[769]: ts=2026-09-22T16:43:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14498worker1 # [ 59.319035] nix-grpc-daemon[768]: ts=2026-09-22T16:43:00Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04499worker2 # [ 59.386941] nix-daemon[838]: reaped child process 913, status = succeeded4500worker1 # [ 59.325256] nix-grpc-daemon[768]: ts=2026-09-22T16:43:00Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14501worker2 # [ 59.426243] nix-grpc-daemon[769]: ts=2026-09-22T16:43:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464502worker2 # [ 59.442251] nix-grpc-daemon[769]: ts=2026-09-22T16:43:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459464503worker2 # [ 59.456440] nix-grpc-daemon[865]: time=2026-09-22T16:43:01.092Z level=INFO msg="All 2 paths already cached"4504client # copying 1 paths to 'lb:50051'...4505client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4506worker2 # [ 59.467887] nix-grpc-daemon[769]: ts=2026-09-22T16:43:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459464507worker2 # [ 59.481876] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4508worker2 # [ 59.635982] nix-daemon[838]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4509worker2 # [ 59.654971] nix-daemon[838]: reaped child process 941, status = succeeded4510lb # [ 59.625472] niks3-server[992]: 2026/09/22 16:43:01 INFO Received uploads request method=POST path=/api/pending_closures4511worker2 # [ 59.668191] nix-grpc-daemon[865]: time=2026-09-22T16:43:01.304Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4512worker2 # [ 59.669788] nix-grpc-daemon[865]: time=2026-09-22T16:43:01.304Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4513lb # [ 59.659122] niks3-server[992]: 2026/09/22 16:43:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4514worker2 # [ 59.700228] nix-grpc-daemon[865]: time=2026-09-22T16:43:01.335Z level=INFO msg="Uploading 1 narinfos"4515lb # [ 59.672166] niks3-server[992]: 2026/09/22 16:43:01 INFO Signed narinfos id=13 count=14516lb # [ 59.679677] niks3-server[992]: 2026/09/22 16:43:01 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4517lb # [ 59.696670] niks3-server[992]: 2026/09/22 16:43:01 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4518lb # [ 59.701714] niks3-server[992]: 2026/09/22 16:43:01 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4519lb # [ 59.707959] niks3-server[992]: 2026/09/22 16:43:01 INFO Completed upload id=134520worker2 # [ 59.741058] nix-grpc-daemon[865]: time=2026-09-22T16:43:01.376Z level=INFO msg="Upload complete. (242ms)"4521lb # [ 59.711163] niks3-server[992]: 2026/09/22 16:43:01 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4522worker2 # [ 59.751635] nix-grpc-daemon[769]: ts=2026-09-22T16:43:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_in=7524523worker2 # [ 59.768772] nix-grpc-daemon[769]: ts=2026-09-22T16:43:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464524worker2 # [ 59.775519] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4525worker2 # [ 59.787364] nix-daemon[838]: reaped child process 936, status = succeeded4526client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4527client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4528worker2 # [ 59.795890] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4529client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4530client # farm-top> LOG-farm-top-sig4531worker2 # [ 60.456717] nix-daemon[838]: reaped child process 945, status = succeeded4532worker2 # [ 60.552847] nix-daemon[838]: accepted connection from pid 961, user nix-grpc-daemon (trusted)4533worker2 # [ 60.575018] nix-daemon[838]: reaped child process 964, status = succeeded4534lb # [ 60.541388] niks3-server[992]: 2026/09/22 16:43:02 INFO Received uploads request method=POST path=/api/pending_closures4535worker2 # [ 60.586409] nix-grpc-daemon[865]: time=2026-09-22T16:43:02.220Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4536worker2 # [ 60.587894] nix-grpc-daemon[865]: time=2026-09-22T16:43:02.220Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4537lb # [ 60.579634] niks3-server[992]: 2026/09/22 16:43:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4538worker2 # [ 60.623053] nix-grpc-daemon[865]: time=2026-09-22T16:43:02.258Z level=INFO msg="Uploading 1 narinfos"4539lb # [ 60.594754] niks3-server[992]: 2026/09/22 16:43:02 INFO Signed narinfos id=14 count=14540lb # [ 60.602286] niks3-server[992]: 2026/09/22 16:43:02 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4541lb # [ 60.610404] niks3-server[992]: 2026/09/22 16:43:02 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4542lb # [ 60.615199] niks3-server[992]: 2026/09/22 16:43:02 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4543lb # [ 60.623633] niks3-server[992]: 2026/09/22 16:43:02 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4544lb # [ 60.627864] niks3-server[992]: 2026/09/22 16:43:02 INFO Completed upload id=144545worker2 # [ 60.661759] nix-grpc-daemon[865]: time=2026-09-22T16:43:02.297Z level=INFO msg="Upload complete. (255ms)"4546lb # [ 60.631658] niks3-server[992]: 2026/09/22 16:43:02 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4547worker2 # [ 60.673037] nix-grpc-daemon[769]: ts=2026-09-22T16:43:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14548worker2 # [ 60.693676] nix-daemon[838]: reaped child process 943, status = succeeded4549client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.87 seconds)4550client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4551client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4552client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4553worker1 # [ 60.985543] nix-grpc-daemon[768]: ts=2026-09-22T16:43:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:387304554client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4555client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4556client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4557client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4558worker1 # [ 61.032493] nix-grpc-daemon[768]: ts=2026-09-22T16:43:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:387304559worker1 # [ 61.042376] nix-grpc-daemon[768]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4560worker1 # [ 61.059594] nix-grpc-daemon[768]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4561worker1 # [ 61.074119] nix-grpc-daemon[768]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4562client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.27 seconds)4563client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4564worker2 # [ 61.362595] nix-grpc-daemon[769]: ts=2026-09-22T16:43:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:328904565worker2 # [ 61.406738] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:328904566client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.30 seconds)4567client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4568client # finding garbage collector roots...4569client # removing stale temporary roots file "/nix/var/nix/temproots/853"4570client # removing stale temporary roots file "/nix/var/nix/temproots/923"4571client # removing stale temporary roots file "/nix/var/nix/temproots/902"4572client # removing stale temporary roots file "/nix/var/nix/temproots/946"4573client # removing stale temporary roots file "/nix/var/nix/temproots/931"4574client # removing stale temporary roots file "/nix/var/nix/temproots/890"4575client # removing stale temporary roots file "/nix/var/nix/temproots/869"4576client # removing stale temporary roots file "/nix/var/nix/temproots/910"4577client # deleting specified paths...4578client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4579client # deleting unused links...4580client # note: hard linking is currently saving 0.0 KiB4581worker1 # [ 61.761746] nix-grpc-daemon[768]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582724582worker2 # [ 61.830043] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459464583worker2 # [ 61.838039] nix-grpc-daemon[865]: time=2026-09-22T16:43:03.471Z level=INFO msg="All 1 paths already cached"4584worker1 # [ 61.779857] nix-grpc-daemon[768]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582724585worker1 # [ 61.788798] nix-grpc-daemon[768]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582724586worker1 # [ 61.797149] nix-grpc-daemon[768]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582724587client # copying 3 paths...4588worker2 # [ 61.871049] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:459464589client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4590worker2 # [ 61.898297] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:459464591worker2 # [ 61.905783] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_out=2324592worker2 # [ 61.914579] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:459464593worker2 # [ 61.921536] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_out=1764594worker2 # [ 61.930387] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:459464595worker2 # [ 61.935423] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_out=1284596client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4597client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4598client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.55 seconds)4599(finished: subtest: outputs read back through any worker carry the cache signature, in 7.18 seconds)4600subtest: two clients wanting the same drv build it once4601client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4602client # [ 62.090587] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4603client # Running as unit: dedup1.service; invocation ID: 02930c49bb2c4ca79114591bc98b5fc94604client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4605client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4606worker2 # [ 62.277649] nix-grpc-daemon[769]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:459464607client # [ 62.292576] nix[999]: don't know how to build these paths:4608client # [ 62.293532] nix[999]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4609worker1 # [ 62.238134] nix-grpc-daemon[768]: ts=2026-09-22T16:43:03Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04610worker1 # [ 62.243560] nix-grpc-daemon[768]: ts=2026-09-22T16:43:03Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14611worker1 # [ 62.281297] nix-grpc-daemon[768]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724612worker1 # [ 62.296973] nix-grpc-daemon[768]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724613client # [ 62.380059] nix[999]: copying 1 paths to 'lb:50051'...4614worker1 # [ 62.311705] nix-grpc-daemon[768]: ts=2026-09-22T16:43:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582724615client # [ 62.382519] nix[999]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4616worker1 # [ 62.327088] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4617worker1 # [ 62.498704] nix-daemon[851]: accepted connection from pid 1045, user nix-grpc-daemon (trusted)4618worker1 # [ 62.517654] nix-daemon[851]: reaped child process 1048, status = succeeded4619lb # [ 62.550829] niks3-server[992]: 2026/09/22 16:43:04 INFO Received uploads request method=POST path=/api/pending_closures4620worker1 # [ 62.536274] nix-grpc-daemon[866]: time=2026-09-22T16:43:04.159Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4621worker1 # [ 62.537858] nix-grpc-daemon[866]: time=2026-09-22T16:43:04.159Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4622lb # [ 62.584626] niks3-server[992]: 2026/09/22 16:43:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4623worker1 # [ 62.567309] nix-grpc-daemon[866]: time=2026-09-22T16:43:04.190Z level=INFO msg="Uploading 1 narinfos"4624lb # [ 62.596304] niks3-server[992]: 2026/09/22 16:43:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4625lb # [ 62.605761] niks3-server[992]: 2026/09/22 16:43:04 INFO Signed narinfos id=15 count=14626lb # [ 62.614103] niks3-server[992]: 2026/09/22 16:43:04 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4627lb # [ 62.621422] niks3-server[992]: 2026/09/22 16:43:04 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4628worker1 # [ 62.593306] nix-grpc-daemon[866]: time=2026-09-22T16:43:04.216Z level=INFO msg="Upload complete. (248ms)"4629lb # [ 62.629566] niks3-server[992]: 2026/09/22 16:43:04 INFO Completed upload id=154630lb # [ 62.631754] niks3-server[992]: 2026/09/22 16:43:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4631worker1 # [ 62.603595] nix-grpc-daemon[768]: ts=2026-09-22T16:43:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_in=4324632worker1 # [ 62.618178] nix-grpc-daemon[768]: ts=2026-09-22T16:43:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724633worker1 # [ 62.624572] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4634worker1 # [ 62.629423] nix-daemon[851]: reaped child process 1043, status = succeeded4635client # [ 62.707302] nix[999]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4636worker1 # [ 62.642478] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4637client # [ 63.186942] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4638client # Running as unit: dedup2.service; invocation ID: 59f5305048514f50a2f36c2e8fcc05444639client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4640worker2 # [ 63.425818] nix-grpc-daemon[769]: ts=2026-09-22T16:43:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:459464641client # [ 63.440408] nix[1015]: don't know how to build these paths:4642client # [ 63.442470] nix[1015]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4643worker1 # [ 63.383619] nix-grpc-daemon[768]: ts=2026-09-22T16:43:05Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04644client # [ 63.459245] nix[1015]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4645worker1 # [ 63.390884] nix-grpc-daemon[768]: ts=2026-09-22T16:43:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724646client: must succeed: sleep 24647client: (finished: must succeed: sleep 2, in 2.05 seconds)4648client: waiting for success: ! systemctl is-active dedup1 dedup24649worker1 # [ 66.742217] nix-daemon[851]: reaped child process 1052, status = succeeded4650worker1 # [ 66.809259] nix-daemon[851]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)4651worker1 # [ 66.828579] nix-daemon[851]: reaped child process 1087, status = succeeded4652lb # [ 66.861385] niks3-server[992]: 2026/09/22 16:43:08 INFO Received uploads request method=POST path=/api/pending_closures4653worker1 # [ 66.844445] nix-grpc-daemon[866]: time=2026-09-22T16:43:08.467Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4654worker1 # [ 66.849444] nix-grpc-daemon[866]: time=2026-09-22T16:43:08.469Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4655lb # [ 66.909547] niks3-server[992]: 2026/09/22 16:43:08 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4656worker1 # [ 66.894506] nix-grpc-daemon[866]: time=2026-09-22T16:43:08.517Z level=INFO msg="Uploading 1 narinfos"4657lb # [ 66.921856] niks3-server[992]: 2026/09/22 16:43:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4658lb # [ 66.932101] niks3-server[992]: 2026/09/22 16:43:08 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4659lb # [ 66.935530] niks3-server[992]: 2026/09/22 16:43:08 INFO Signed narinfos id=16 count=14660lb # [ 66.947470] niks3-server[992]: 2026/09/22 16:43:08 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4661lb # [ 66.954171] niks3-server[992]: 2026/09/22 16:43:08 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4662worker1 # [ 66.926909] nix-grpc-daemon[866]: time=2026-09-22T16:43:08.550Z level=INFO msg="Upload complete. (270ms)"4663lb # [ 66.957518] niks3-server[992]: 2026/09/22 16:43:08 INFO Completed upload id=164664lb # [ 66.959062] niks3-server[992]: 2026/09/22 16:43:08 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4665worker1 # [ 66.941149] nix-grpc-daemon[768]: ts=2026-09-22T16:43:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14666worker1 # [ 66.960325] nix-grpc-daemon[768]: ts=2026-09-22T16:43:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14667worker1 # [ 66.967357] nix-daemon[851]: reaped child process 1050, status = succeeded4668client # [ 67.048647] systemd[1]: dedup1.service: Deactivated successfully.4669client # [ 67.059385] systemd[1]: dedup1.service: Consumed 177ms CPU time over 4.964s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.4670client # [ 67.098968] systemd[1]: dedup2.service: Deactivated successfully.4671client # [ 67.102090] systemd[1]: dedup2.service: Consumed 219ms CPU time over 3.912s wall clock time, 35.2M memory peak, 6.1K incoming IP traffic, 5.6K outgoing IP traffic.4672client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4673client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4674client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4675worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4676worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4677worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4678worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4679worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4680worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4681worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4682worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4683(finished: subtest: two clients wanting the same drv build it once, in 6.11 seconds)4684subtest: --timeout and --max-silent-time reach the builder4685worker2 # [ 68.361717] nix-grpc-daemon[769]: ts=2026-09-22T16:43:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:459464686worker1 # [ 68.318807] nix-grpc-daemon[768]: ts=2026-09-22T16:43:09Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04687worker1 # [ 68.324800] nix-grpc-daemon[768]: ts=2026-09-22T16:43:09Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14688worker2 # [ 68.426044] nix-grpc-daemon[769]: ts=2026-09-22T16:43:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464689worker2 # [ 68.442149] nix-grpc-daemon[769]: ts=2026-09-22T16:43:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459464690worker2 # [ 68.457776] nix-grpc-daemon[769]: ts=2026-09-22T16:43:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459464691worker2 # [ 68.472835] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4692worker2 # [ 68.636067] nix-daemon[838]: accepted connection from pid 989, user nix-grpc-daemon (trusted)4693worker2 # [ 68.656961] nix-daemon[838]: reaped child process 992, status = succeeded4694lb # [ 68.628902] niks3-server[992]: 2026/09/22 16:43:10 INFO Received uploads request method=POST path=/api/pending_closures4695worker2 # [ 68.672746] nix-grpc-daemon[865]: time=2026-09-22T16:43:10.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4696worker2 # [ 68.674223] nix-grpc-daemon[865]: time=2026-09-22T16:43:10.307Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4697lb # [ 68.663854] niks3-server[992]: 2026/09/22 16:43:10 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4698worker2 # [ 68.709267] nix-grpc-daemon[865]: time=2026-09-22T16:43:10.344Z level=INFO msg="Uploading 1 narinfos"4699lb # [ 68.679672] niks3-server[992]: 2026/09/22 16:43:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4700lb # [ 68.690501] niks3-server[992]: 2026/09/22 16:43:10 INFO Signed narinfos id=17 count=14701lb # [ 68.699093] niks3-server[992]: 2026/09/22 16:43:10 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4702lb # [ 68.707506] niks3-server[992]: 2026/09/22 16:43:10 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4703lb # [ 68.711848] niks3-server[992]: 2026/09/22 16:43:10 INFO Completed upload id=174704worker2 # [ 68.739041] nix-grpc-daemon[865]: time=2026-09-22T16:43:10.374Z level=INFO msg="Upload complete. (248ms)"4705lb # [ 68.714789] niks3-server[992]: 2026/09/22 16:43:10 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4706worker2 # [ 68.749576] nix-grpc-daemon[769]: ts=2026-09-22T16:43:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_in=4244707worker2 # [ 68.767975] nix-grpc-daemon[769]: ts=2026-09-22T16:43:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464708worker2 # [ 68.774974] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4709worker2 # [ 68.780336] nix-daemon[838]: reaped child process 987, status = succeeded4710worker2 # [ 68.793752] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4711lb # [ 69.212969] envoy[899]: conn peer=[2001:db8:1::1]:50762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9498 tx=4357 ms=56144712lb # [ 69.218560] envoy[899]: conn peer=[2001:db8:1::1]:57648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2952 ms=1834713lb # [ 69.224084] envoy[899]: conn peer=[2001:db8:1::1]:57656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=1104714lb # [ 69.231494] envoy[899]: conn peer=[2001:db8:1::1]:57672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=794715lb # [ 69.240148] envoy[899]: conn peer=[2001:db8:1::1]:57668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=954716lb # [ 69.244813] envoy[899]: conn peer=[2001:db8:1::1]:57674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1539 ms=47364717lb # [ 69.249616] envoy[899]: conn peer=[2001:db8:1::1]:57676 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=36374718worker2 # [ 74.609874] nix-grpc-daemon[769]: ts=2026-09-22T16:43:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04719worker2 # [ 74.648989] nix-daemon[838]: reaped child process 994, status = succeeded4720worker2 # [ 74.651751] nix-daemon[838]: reaped child process 996, status = succeeded4721worker1 # [ 74.785667] nix-grpc-daemon[768]: ts=2026-09-22T16:43:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582724722worker1 # [ 74.799154] nix-grpc-daemon[768]: ts=2026-09-22T16:43:16Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04723worker1 # [ 74.809265] nix-grpc-daemon[768]: ts=2026-09-22T16:43:16Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14724worker1 # [ 74.818550] nix-grpc-daemon[768]: ts=2026-09-22T16:43:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724725worker1 # [ 74.826208] nix-grpc-daemon[768]: ts=2026-09-22T16:43:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724726worker1 # [ 74.836668] nix-grpc-daemon[768]: ts=2026-09-22T16:43:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582724727worker1 # [ 74.853332] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4728worker1 # [ 75.003567] nix-daemon[851]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4729worker1 # [ 75.024233] nix-daemon[851]: reaped child process 1107, status = succeeded4730lb # [ 75.057206] niks3-server[992]: 2026/09/22 16:43:16 INFO Received uploads request method=POST path=/api/pending_closures4731worker1 # [ 75.040624] nix-grpc-daemon[866]: time=2026-09-22T16:43:16.663Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4732worker1 # [ 75.042211] nix-grpc-daemon[866]: time=2026-09-22T16:43:16.663Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4733lb # [ 75.090503] niks3-server[992]: 2026/09/22 16:43:16 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4734worker1 # [ 75.075086] nix-grpc-daemon[866]: time=2026-09-22T16:43:16.698Z level=INFO msg="Uploading 1 narinfos"4735lb # [ 75.103066] niks3-server[992]: 2026/09/22 16:43:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4736lb # [ 75.114066] niks3-server[992]: 2026/09/22 16:43:16 INFO Signed narinfos id=18 count=14737lb # [ 75.120625] niks3-server[992]: 2026/09/22 16:43:16 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4738lb # [ 75.132910] niks3-server[992]: 2026/09/22 16:43:16 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4739lb # [ 75.140248] niks3-server[992]: 2026/09/22 16:43:16 INFO Completed upload id=184740worker1 # [ 75.112789] nix-grpc-daemon[866]: time=2026-09-22T16:43:16.736Z level=INFO msg="Upload complete. (246ms)"4741lb # [ 75.144231] niks3-server[992]: 2026/09/22 16:43:16 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4742worker1 # [ 75.125021] nix-grpc-daemon[768]: ts=2026-09-22T16:43:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_in=4244743worker1 # [ 75.136318] nix-grpc-daemon[768]: ts=2026-09-22T16:43:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724744worker1 # [ 75.148664] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4745worker1 # [ 75.154060] nix-daemon[851]: reaped child process 1102, status = succeeded4746worker1 # [ 75.167753] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4747lb # [ 79.214045] envoy[899]: conn peer=[2001:db8:1::1]:57684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3014 tx=954 ms=62674748worker1 # [ 81.388761] nix-grpc-daemon[768]: ts=2026-09-22T16:43:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04749worker1 # [ 81.435046] nix-daemon[851]: reaped child process 1109, status = succeeded4750worker1 # [ 81.438207] nix-daemon[851]: reaped child process 1111, status = succeeded4751(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.55 seconds)4752subtest: the first client leaving does not fail the second4753client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4754client # [ 81.762542] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4755client # Running as unit: share1.service; invocation ID: 7cb25cef48a74e989a1741f2c4f7291a4756client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4757client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4758worker2 # [ 81.979857] nix-grpc-daemon[769]: ts=2026-09-22T16:43:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:459464759client # [ 81.994960] nix[1088]: don't know how to build these paths:4760client # [ 81.996054] nix[1088]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4761worker1 # [ 81.940337] nix-grpc-daemon[768]: ts=2026-09-22T16:43:23Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04762worker1 # [ 81.946806] nix-grpc-daemon[768]: ts=2026-09-22T16:43:23Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14763worker2 # [ 82.010298] nix-grpc-daemon[769]: ts=2026-09-22T16:43:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464764worker2 # [ 82.026492] nix-grpc-daemon[769]: ts=2026-09-22T16:43:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459464765client # [ 82.048490] nix[1088]: copying 1 paths to 'lb:50051'...4766client # [ 82.051432] nix[1088]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4767worker2 # [ 82.041979] nix-grpc-daemon[769]: ts=2026-09-22T16:43:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459464768worker2 # [ 82.058585] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4769worker2 # [ 82.227195] nix-daemon[838]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4770worker2 # [ 82.249362] nix-daemon[838]: reaped child process 1022, status = succeeded4771lb # [ 82.220742] niks3-server[992]: 2026/09/22 16:43:23 INFO Received uploads request method=POST path=/api/pending_closures4772worker2 # [ 82.264389] nix-grpc-daemon[865]: time=2026-09-22T16:43:23.899Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4773worker2 # [ 82.265906] nix-grpc-daemon[865]: time=2026-09-22T16:43:23.899Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4774lb # [ 82.261537] niks3-server[992]: 2026/09/22 16:43:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4775worker2 # [ 82.303803] nix-grpc-daemon[865]: time=2026-09-22T16:43:23.939Z level=INFO msg="Uploading 1 narinfos"4776lb # [ 82.274444] niks3-server[992]: 2026/09/22 16:43:23 INFO Signed narinfos id=19 count=14777lb # [ 82.282447] niks3-server[992]: 2026/09/22 16:43:23 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4778lb # [ 82.292668] niks3-server[992]: 2026/09/22 16:43:23 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4779lb # [ 82.301043] niks3-server[992]: 2026/09/22 16:43:23 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4780lb # [ 82.309478] niks3-server[992]: 2026/09/22 16:43:23 INFO Completed upload id=194781worker2 # [ 82.341971] nix-grpc-daemon[865]: time=2026-09-22T16:43:23.977Z level=INFO msg="Upload complete. (261ms)"4782lb # [ 82.313149] niks3-server[992]: 2026/09/22 16:43:23 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4783worker2 # [ 82.352403] nix-grpc-daemon[769]: ts=2026-09-22T16:43:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_in=4324784worker2 # [ 82.371860] nix-grpc-daemon[769]: ts=2026-09-22T16:43:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464785worker2 # [ 82.379850] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4786worker2 # [ 82.386344] nix-daemon[838]: reaped child process 1017, status = succeeded4787client # [ 82.405906] nix[1088]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4788worker2 # [ 82.402705] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4789client # [ 82.877336] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4790client # Running as unit: share2.service; invocation ID: ee9d15c1f619481681c7b345ee3091574791client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds)4792worker2 # [ 83.097594] nix-grpc-daemon[769]: ts=2026-09-22T16:43:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:459464793client # [ 83.117648] nix[1104]: this derivation will be built:4794worker1 # [ 83.052708] nix-grpc-daemon[768]: ts=2026-09-22T16:43:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582724795worker1 # [ 83.066312] nix-grpc-daemon[768]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' from 'http://lb:5751'...4796client # [ 83.158447] nix[1104]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4797worker1 # [ 83.102206] nix-grpc-daemon[768]: ts=2026-09-22T16:43:24Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04798worker2 # [ 83.166775] nix-grpc-daemon[769]: ts=2026-09-22T16:43:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459464799client # [ 83.182050] nix[1104]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4800client: must succeed: sleep 2; systemctl kill -s INT share14801client # [ 86.254815] systemd[1]: share1.service: Sent signal SIGINT to main process 1088 (nix) on client request.4802client # [ 86.261434] nix[1088]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4803client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4804client: waiting for success: ! systemctl is-active share14805client # [ 86.277835] nix[1088]: error: interrupted by the user4806client # [ 86.335479] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4807client # [ 86.338198] systemd[1]: share1.service: Failed with result 'exit-code'.4808client # [ 86.341174] systemd[1]: share1.service: Consumed 191ms CPU time over 4.576s wall clock time, 39.4M memory peak, 6.2K incoming IP traffic, 8.3K outgoing IP traffic.4809client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds)4810client: waiting for success: ! systemctl is-active share24811worker2 # [ 87.910611] nix-daemon[838]: reaped child process 1026, status = succeeded4812worker2 # [ 87.990751] nix-daemon[838]: accepted connection from pid 1060, user nix-grpc-daemon (trusted)4813worker2 # [ 88.009000] nix-daemon[838]: reaped child process 1063, status = succeeded4814lb # [ 87.983071] niks3-server[992]: 2026/09/22 16:43:29 INFO Received uploads request method=POST path=/api/pending_closures4815worker2 # [ 88.028209] nix-grpc-daemon[865]: time=2026-09-22T16:43:29.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4816worker2 # [ 88.029912] nix-grpc-daemon[865]: time=2026-09-22T16:43:29.661Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4817lb # [ 88.023404] niks3-server[992]: 2026/09/22 16:43:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4818worker2 # [ 88.065408] nix-grpc-daemon[865]: time=2026-09-22T16:43:29.700Z level=INFO msg="Uploading 1 narinfos"4819lb # [ 88.038014] niks3-server[992]: 2026/09/22 16:43:29 INFO Signed narinfos id=20 count=14820lb # [ 88.044369] niks3-server[992]: 2026/09/22 16:43:29 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4821lb # [ 88.047934] niks3-server[992]: 2026/09/22 16:43:29 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4822lb # [ 88.054246] niks3-server[992]: 2026/09/22 16:43:29 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4823lb # [ 88.063156] niks3-server[992]: 2026/09/22 16:43:29 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4824worker2 # [ 88.096338] nix-grpc-daemon[865]: time=2026-09-22T16:43:29.732Z level=INFO msg="Upload complete. (277ms)"4825lb # [ 88.069273] niks3-server[992]: 2026/09/22 16:43:29 INFO Completed upload id=204826lb # [ 88.071876] niks3-server[992]: 2026/09/22 16:43:29 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4827worker2 # [ 88.109170] nix-grpc-daemon[769]: ts=2026-09-22T16:43:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14828worker2 # [ 88.126026] nix-grpc-daemon[769]: ts=2026-09-22T16:43:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14829worker2 # [ 88.133284] nix-daemon[838]: reaped child process 1024, status = succeeded4830client # [ 88.193208] systemd[1]: share2.service: Deactivated successfully.4831client # [ 88.198316] systemd[1]: share2.service: Consumed 207ms CPU time over 5.317s wall clock time, 35.3M memory peak, 6.6K incoming IP traffic, 5.9K outgoing IP traffic.4832client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4833client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4834client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4835(finished: subtest: the first client leaving does not fail the second, in 7.12 seconds)4836subtest: build hook: nix-daemon with builders = grpc://lb4837client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4838client # [ 88.953575] systemd[1]: Started Nix Daemon.4839client # [ 89.071280] nix-daemon[1156]: accepted connection from pid 1153, user root (trusted)4840worker2 # [ 89.218154] nix-grpc-daemon[769]: ts=2026-09-22T16:43:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:459464841worker1 # [ 89.162738] nix-grpc-daemon[768]: ts=2026-09-22T16:43:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724842worker1 # [ 89.182400] nix-grpc-daemon[768]: ts=2026-09-22T16:43:30Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04843worker1 # [ 89.190403] nix-grpc-daemon[768]: ts=2026-09-22T16:43:30Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14844lb # [ 89.221103] envoy[899]: conn peer=[2001:db8:1::1]:49980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3028 tx=976 ms=66654845lb # [ 89.221404] envoy[899]: conn peer=[2001:db8:1::1]:56052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3003 tx=789 ms=42964846worker1 # [ 89.200337] nix-grpc-daemon[768]: ts=2026-09-22T16:43:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724847lb # [ 89.221525] envoy[899]: conn peer=[2001:db8:1::1]:56062 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1605 tx=1599 ms=50674848worker1 # [ 89.219167] nix-grpc-daemon[768]: ts=2026-09-22T16:43:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724849worker1 # [ 89.235197] nix-grpc-daemon[768]: ts=2026-09-22T16:43:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582724850worker1 # [ 89.250497] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4851worker1 # [ 89.420205] nix-daemon[851]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)4852worker1 # [ 89.438382] nix-daemon[851]: reaped child process 1160, status = succeeded4853lb # [ 89.474858] niks3-server[992]: 2026/09/22 16:43:31 INFO Received uploads request method=POST path=/api/pending_closures4854worker1 # [ 89.457707] nix-grpc-daemon[866]: time=2026-09-22T16:43:31.079Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4855worker1 # [ 89.459296] nix-grpc-daemon[866]: time=2026-09-22T16:43:31.080Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4856lb # [ 89.507591] niks3-server[992]: 2026/09/22 16:43:31 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4857worker1 # [ 89.492138] nix-grpc-daemon[866]: time=2026-09-22T16:43:31.115Z level=INFO msg="Uploading 1 narinfos"4858lb # [ 89.521724] niks3-server[992]: 2026/09/22 16:43:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4859lb # [ 89.533096] niks3-server[992]: 2026/09/22 16:43:31 INFO Signed narinfos id=21 count=14860lb # [ 89.535760] niks3-server[992]: 2026/09/22 16:43:31 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4861lb # [ 89.548316] niks3-server[992]: 2026/09/22 16:43:31 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4862worker1 # [ 89.521087] nix-grpc-daemon[866]: time=2026-09-22T16:43:31.144Z level=INFO msg="Upload complete. (253ms)"4863lb # [ 89.551267] niks3-server[992]: 2026/09/22 16:43:31 INFO Completed upload id=214864lb # [ 89.554700] niks3-server[992]: 2026/09/22 16:43:31 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4865worker1 # [ 89.532589] nix-grpc-daemon[768]: ts=2026-09-22T16:43:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_in=5124866worker1 # [ 89.546160] nix-grpc-daemon[768]: ts=2026-09-22T16:43:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724867worker1 # [ 89.552647] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4868worker1 # [ 89.557518] nix-daemon[851]: reaped child process 1155, status = succeeded4869worker1 # [ 89.569488] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4870worker1 # [ 90.869080] nix-daemon[851]: reaped child process 1164, status = succeeded4871worker1 # [ 90.952823] nix-daemon[851]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)4872worker1 # [ 90.972407] nix-daemon[851]: reaped child process 1183, status = succeeded4873lb # [ 91.005038] niks3-server[992]: 2026/09/22 16:43:32 INFO Received uploads request method=POST path=/api/pending_closures4874worker1 # [ 90.988453] nix-grpc-daemon[866]: time=2026-09-22T16:43:32.610Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4875worker1 # [ 90.990212] nix-grpc-daemon[866]: time=2026-09-22T16:43:32.610Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4876lb # [ 91.050380] niks3-server[992]: 2026/09/22 16:43:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4877worker1 # [ 91.031981] nix-grpc-daemon[866]: time=2026-09-22T16:43:32.655Z level=INFO msg="Uploading 1 narinfos"4878lb # [ 91.064800] niks3-server[992]: 2026/09/22 16:43:32 INFO Signed narinfos id=22 count=14879lb # [ 91.073447] niks3-server[992]: 2026/09/22 16:43:32 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4880lb # [ 91.081545] niks3-server[992]: 2026/09/22 16:43:32 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4881lb # [ 91.092213] niks3-server[992]: 2026/09/22 16:43:32 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4882worker1 # [ 91.074932] nix-grpc-daemon[866]: time=2026-09-22T16:43:32.698Z level=INFO msg="Upload complete. (295ms)"4883lb # [ 91.101906] niks3-server[992]: 2026/09/22 16:43:32 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4884lb # [ 91.106331] niks3-server[992]: 2026/09/22 16:43:32 INFO Completed upload id=224885lb # [ 91.111426] niks3-server[992]: 2026/09/22 16:43:32 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4886worker1 # [ 91.089262] nix-grpc-daemon[768]: ts=2026-09-22T16:43:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14887worker1 # [ 91.106958] nix-daemon[851]: reaped child process 1162, status = succeeded4888worker2 # [ 91.195399] nix-grpc-daemon[769]: ts=2026-09-22T16:43:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:459464889worker2 # [ 91.209032] nix-grpc-daemon[769]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4890worker2 # [ 91.245880] nix-grpc-daemon[769]: ts=2026-09-22T16:43:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:459464891worker2 # [ 91.252433] nix-grpc-daemon[769]: ts=2026-09-22T16:43:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_out=1284892worker2 # [ 91.536092] nix-grpc-daemon[769]: ts=2026-09-22T16:43:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:459464893worker2 # [ 91.543837] nix-grpc-daemon[769]: ts=2026-09-22T16:43:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459464894worker2 # [ 91.551163] nix-grpc-daemon[865]: time=2026-09-22T16:43:33.185Z level=INFO msg="All 1 paths already cached"4895worker1 # [ 91.502461] nix-grpc-daemon[768]: ts=2026-09-22T16:43:33Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04896worker1 # [ 91.512605] nix-grpc-daemon[768]: ts=2026-09-22T16:43:33Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14897worker1 # [ 91.521480] nix-grpc-daemon[768]: ts=2026-09-22T16:43:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724898worker1 # [ 91.529316] nix-grpc-daemon[768]: ts=2026-09-22T16:43:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724899worker1 # [ 91.539951] nix-grpc-daemon[866]: time=2026-09-22T16:43:33.163Z level=INFO msg="All 1 paths already cached"4900worker1 # [ 91.550667] nix-grpc-daemon[768]: ts=2026-09-22T16:43:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582724901worker1 # [ 91.565044] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4902worker1 # [ 91.723388] nix-daemon[851]: accepted connection from pid 1189, user nix-grpc-daemon (trusted)4903worker1 # [ 91.744237] nix-daemon[851]: reaped child process 1192, status = succeeded4904lb # [ 91.769118] niks3-server[992]: 2026/09/22 16:43:33 INFO Received uploads request method=POST path=/api/pending_closures4905worker1 # [ 91.751621] nix-grpc-daemon[866]: time=2026-09-22T16:43:33.374Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4906worker1 # [ 91.753639] nix-grpc-daemon[866]: time=2026-09-22T16:43:33.374Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4907worker1 # [ 91.771218] nix-grpc-daemon[866]: time=2026-09-22T16:43:33.394Z level=INFO msg="Uploading 1 narinfos"4908lb # [ 91.798144] niks3-server[992]: 2026/09/22 16:43:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4909lb # [ 91.810649] niks3-server[992]: 2026/09/22 16:43:33 INFO Signed narinfos id=23 count=14910lb # [ 91.820189] niks3-server[992]: 2026/09/22 16:43:33 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4911worker1 # [ 91.808829] nix-grpc-daemon[866]: time=2026-09-22T16:43:33.431Z level=INFO msg="Upload complete. (226ms)"4912lb # [ 91.835819] niks3-server[992]: 2026/09/22 16:43:33 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4913lb # [ 91.843142] niks3-server[992]: 2026/09/22 16:43:33 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4914lb # [ 91.850835] niks3-server[992]: 2026/09/22 16:43:33 INFO Completed upload id=234915worker1 # [ 91.820122] nix-grpc-daemon[768]: ts=2026-09-22T16:43:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_in=6244916lb # [ 91.856177] niks3-server[992]: 2026/09/22 16:43:33 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4917worker1 # [ 91.832393] nix-grpc-daemon[768]: ts=2026-09-22T16:43:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724918worker1 # [ 91.841757] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4919worker1 # [ 91.844467] nix-daemon[851]: reaped child process 1187, status = succeeded4920worker1 # [ 91.858023] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4921worker1 # [ 92.967907] nix-daemon[851]: reaped child process 1196, status = succeeded4922worker1 # [ 93.041069] nix-daemon[851]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)4923worker1 # [ 93.058593] nix-daemon[851]: reaped child process 1215, status = succeeded4924lb # [ 93.094161] niks3-server[992]: 2026/09/22 16:43:34 INFO Received uploads request method=POST path=/api/pending_closures4925worker1 # [ 93.076456] nix-grpc-daemon[866]: time=2026-09-22T16:43:34.698Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4926worker1 # [ 93.078304] nix-grpc-daemon[866]: time=2026-09-22T16:43:34.698Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4927lb # [ 93.143491] niks3-server[992]: 2026/09/22 16:43:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4928worker1 # [ 93.124761] nix-grpc-daemon[866]: time=2026-09-22T16:43:34.747Z level=INFO msg="Uploading 1 narinfos"4929lb # [ 93.157767] niks3-server[992]: 2026/09/22 16:43:34 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4930lb # [ 93.166683] niks3-server[992]: 2026/09/22 16:43:34 INFO Signed narinfos id=24 count=14931worker1 # [ 93.145686] nix-grpc-daemon[866]: time=2026-09-22T16:43:34.769Z level=INFO msg="Upload complete. (276ms)"4932lb # [ 93.173823] niks3-server[992]: 2026/09/22 16:43:34 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4933lb # [ 93.181013] niks3-server[992]: 2026/09/22 16:43:34 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4934lb # [ 93.185088] niks3-server[992]: 2026/09/22 16:43:34 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4935lb # [ 93.189101] niks3-server[992]: 2026/09/22 16:43:34 INFO Completed upload id=244936lb # [ 93.191080] niks3-server[992]: 2026/09/22 16:43:34 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4937worker1 # [ 93.161142] nix-grpc-daemon[768]: ts=2026-09-22T16:43:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14938worker1 # [ 93.174908] nix-daemon[851]: reaped child process 1194, status = succeeded4939worker1 # [ 93.202206] nix-grpc-daemon[768]: ts=2026-09-22T16:43:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582724940worker2 # [ 93.294196] nix-grpc-daemon[769]: ts=2026-09-22T16:43:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:459464941worker2 # [ 93.306845] nix-grpc-daemon[769]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4942worker2 # [ 93.323295] nix-grpc-daemon[769]: ts=2026-09-22T16:43:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_out=1764943worker1 # [ 93.543690] nix-grpc-daemon[768]: ts=2026-09-22T16:43:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:582724944worker1 # [ 93.551169] nix-grpc-daemon[768]: ts=2026-09-22T16:43:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724945worker1 # [ 93.558843] nix-grpc-daemon[866]: time=2026-09-22T16:43:35.180Z level=INFO msg="All 2 paths already cached"4946worker1 # [ 93.571183] nix-grpc-daemon[768]: ts=2026-09-22T16:43:35Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04947worker1 # [ 93.580645] nix-grpc-daemon[768]: ts=2026-09-22T16:43:35Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14948worker1 # [ 93.589843] nix-grpc-daemon[768]: ts=2026-09-22T16:43:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724949worker1 # [ 93.598692] nix-grpc-daemon[768]: ts=2026-09-22T16:43:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582724950worker1 # [ 93.609279] nix-grpc-daemon[866]: time=2026-09-22T16:43:35.232Z level=INFO msg="All 2 paths already cached"4951worker1 # [ 93.619972] nix-grpc-daemon[768]: ts=2026-09-22T16:43:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582724952worker1 # [ 93.635807] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4953worker1 # [ 93.785982] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)4954worker1 # [ 93.806763] nix-daemon[851]: reaped child process 1224, status = succeeded4955lb # [ 93.831366] niks3-server[992]: 2026/09/22 16:43:35 INFO Received uploads request method=POST path=/api/pending_closures4956worker1 # [ 93.813871] nix-grpc-daemon[866]: time=2026-09-22T16:43:35.437Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4957worker1 # [ 93.815384] nix-grpc-daemon[866]: time=2026-09-22T16:43:35.437Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4958lb # [ 93.861537] niks3-server[992]: 2026/09/22 16:43:35 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4959worker1 # [ 93.843137] nix-grpc-daemon[866]: time=2026-09-22T16:43:35.466Z level=INFO msg="Uploading 1 narinfos"4960lb # [ 93.873422] niks3-server[992]: 2026/09/22 16:43:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4961lb # [ 93.883931] niks3-server[992]: 2026/09/22 16:43:35 INFO Signed narinfos id=25 count=14962lb # [ 93.893468] niks3-server[992]: 2026/09/22 16:43:35 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4963worker1 # [ 93.871874] nix-grpc-daemon[866]: time=2026-09-22T16:43:35.495Z level=INFO msg="Upload complete. (222ms)"4964lb # [ 93.899627] niks3-server[992]: 2026/09/22 16:43:35 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4965lb # [ 93.904114] niks3-server[992]: 2026/09/22 16:43:35 INFO Completed upload id=254966worker1 # [ 93.880138] nix-grpc-daemon[768]: ts=2026-09-22T16:43:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_in=7524967lb # [ 93.909599] niks3-server[992]: 2026/09/22 16:43:35 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4968worker1 # [ 93.893305] nix-grpc-daemon[768]: ts=2026-09-22T16:43:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582724969worker1 # [ 93.899290] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4970worker1 # [ 93.904116] nix-daemon[851]: reaped child process 1219, status = succeeded4971worker1 # [ 93.916593] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4972worker1 # [ 94.821926] nix-daemon[851]: reaped child process 1228, status = succeeded4973worker1 # [ 94.892605] nix-daemon[851]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)4974worker1 # [ 94.913613] nix-daemon[851]: reaped child process 1247, status = succeeded4975lb # [ 94.945241] niks3-server[992]: 2026/09/22 16:43:36 INFO Received uploads request method=POST path=/api/pending_closures4976worker1 # [ 94.929723] nix-grpc-daemon[866]: time=2026-09-22T16:43:36.551Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4977worker1 # [ 94.931477] nix-grpc-daemon[866]: time=2026-09-22T16:43:36.551Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4978worker1 # [ 94.968537] nix-grpc-daemon[866]: time=2026-09-22T16:43:36.591Z level=INFO msg="Uploading 1 narinfos"4979lb # [ 94.987367] niks3-server[992]: 2026/09/22 16:43:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4980lb # [ 95.003265] niks3-server[992]: 2026/09/22 16:43:36 INFO Signed narinfos id=26 count=14981lb # [ 95.010993] niks3-server[992]: 2026/09/22 16:43:36 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4982lb # [ 95.021200] niks3-server[992]: 2026/09/22 16:43:36 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4983worker1 # [ 94.999197] nix-grpc-daemon[866]: time=2026-09-22T16:43:36.622Z level=INFO msg="Upload complete. (273ms)"4984lb # [ 95.028639] niks3-server[992]: 2026/09/22 16:43:36 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4985lb # [ 95.033989] niks3-server[992]: 2026/09/22 16:43:36 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4986lb # [ 95.038206] niks3-server[992]: 2026/09/22 16:43:36 INFO Completed upload id=264987lb # [ 95.040791] niks3-server[992]: 2026/09/22 16:43:36 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4988worker1 # [ 95.013174] nix-grpc-daemon[768]: ts=2026-09-22T16:43:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14989worker1 # [ 95.031067] nix-daemon[851]: reaped child process 1226, status = succeeded4990worker2 # [ 95.118969] nix-grpc-daemon[769]: ts=2026-09-22T16:43:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:459464991worker2 # [ 95.132460] nix-grpc-daemon[769]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4992worker1 # [ 95.110617] nix-grpc-daemon[768]: ts=2026-09-22T16:43:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:582724993worker1 # [ 95.118064] nix-grpc-daemon[768]: ts=2026-09-22T16:43:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_out=2324994client # [ 95.270405] nix-daemon[1156]: reaped child process 1158, status = succeeded4995client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.48 seconds)4996client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4997client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4998client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4999client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)5000client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5001client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5002(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.60 seconds)5003subtest: an input only one worker has still reaches the builder5004client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only5005client # this derivation will be built:5006client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv5007client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...5008client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds)5009client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure5010client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)5011worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5012worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)5013worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only5014worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)5015client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&25016client # [ 96.228479] nix-daemon[1156]: accepted connection from pid 1239, user root (trusted)5017client # this derivation will be built:5018client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5019client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5020worker2 # [ 96.375074] nix-grpc-daemon[769]: ts=2026-09-22T16:43:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:459465021worker2 # [ 96.382767] nix-grpc-daemon[769]: ts=2026-09-22T16:43:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459465022client # copying 1 paths...5023client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...5024worker2 # [ 96.393636] nix-grpc-daemon[769]: ts=2026-09-22T16:43:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459465025worker2 # [ 96.408742] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5026worker2 # [ 96.580590] nix-daemon[838]: accepted connection from pid 1073, user nix-grpc-daemon (trusted)5027worker2 # [ 96.602067] nix-daemon[838]: reaped child process 1076, status = succeeded5028lb # [ 96.573813] niks3-server[992]: 2026/09/22 16:43:38 INFO Received uploads request method=POST path=/api/pending_closures5029worker2 # [ 96.619310] nix-grpc-daemon[865]: time=2026-09-22T16:43:38.254Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5030worker2 # [ 96.621771] nix-grpc-daemon[865]: time=2026-09-22T16:43:38.254Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"5031lb # [ 96.608546] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls5032worker2 # [ 96.654378] nix-grpc-daemon[865]: time=2026-09-22T16:43:38.289Z level=INFO msg="Uploading 1 narinfos"5033lb # [ 96.621803] niks3-server[992]: 2026/09/22 16:43:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5034lb # [ 96.634778] niks3-server[992]: 2026/09/22 16:43:38 INFO Signed narinfos id=27 count=15035lb # [ 96.640357] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5036lb # [ 96.647662] niks3-server[992]: 2026/09/22 16:43:38 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5037lb # [ 96.655212] niks3-server[992]: 2026/09/22 16:43:38 INFO Completed upload id=275038worker2 # [ 96.687219] nix-grpc-daemon[865]: time=2026-09-22T16:43:38.323Z level=INFO msg="Upload complete. (263ms)"5039lb # [ 96.657391] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo5040worker2 # [ 96.700359] nix-grpc-daemon[769]: ts=2026-09-22T16:43:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_in=1205041worker2 # [ 96.716639] nix-daemon[838]: reaped child process 1071, status = succeeded5042worker1 # [ 96.652951] nix-grpc-daemon[768]: ts=2026-09-22T16:43:38Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=05043worker1 # [ 96.653372] nix-grpc-daemon[768]: ts=2026-09-22T16:43:38Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15044worker2 # [ 96.758329] nix-grpc-daemon[769]: ts=2026-09-22T16:43:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459465045worker2 # [ 96.774582] nix-grpc-daemon[769]: ts=2026-09-22T16:43:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459465046client # copying 2 paths to 'lb:50051'...5047client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...5048worker2 # [ 96.794041] nix-grpc-daemon[769]: ts=2026-09-22T16:43:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459465049client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...5050worker2 # [ 96.807497] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5051worker2 # [ 96.962811] nix-daemon[838]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)5052worker2 # [ 96.982965] nix-daemon[838]: reaped child process 1083, status = succeeded5053lb # [ 96.952925] niks3-server[992]: 2026/09/22 16:43:38 INFO Received uploads request method=POST path=/api/pending_closures5054lb # [ 96.962212] niks3-server[992]: 2026/09/22 16:43:38 INFO Received uploads request method=POST path=/api/pending_closures5055worker2 # [ 97.006541] nix-grpc-daemon[865]: time=2026-09-22T16:43:38.640Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5056worker2 # [ 97.009631] nix-grpc-daemon[865]: time=2026-09-22T16:43:38.640Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"5057worker2 # [ 97.012336] nix-grpc-daemon[865]: time=2026-09-22T16:43:38.640Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"5058lb # [ 97.008621] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst5059lb # [ 97.020833] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls5060worker2 # [ 97.065727] nix-grpc-daemon[865]: time=2026-09-22T16:43:38.700Z level=INFO msg="Uploading 3 narinfos"5061lb # [ 97.031577] niks3-server[992]: 2026/09/22 16:43:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5062lb # [ 97.042867] niks3-server[992]: 2026/09/22 16:43:38 INFO Signed narinfos id=28 count=15063lb # [ 97.049016] niks3-server[992]: 2026/09/22 16:43:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5064lb # [ 97.056646] niks3-server[992]: 2026/09/22 16:43:38 INFO Signed narinfos id=29 count=25065lb # [ 97.061542] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls5066lb # [ 97.070689] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst5067lb # [ 97.080362] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo5068lb # [ 97.091009] niks3-server[992]: 2026/09/22 16:43:38 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5069lb # [ 97.097007] niks3-server[992]: 2026/09/22 16:43:38 INFO Completed upload id=285070lb # [ 97.100072] niks3-server[992]: 2026/09/22 16:43:38 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5071lb # [ 97.104191] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5072worker2 # [ 97.138012] nix-grpc-daemon[865]: time=2026-09-22T16:43:38.773Z level=INFO msg="Upload complete. (311ms)"5073lb # [ 97.106950] niks3-server[992]: 2026/09/22 16:43:38 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5074lb # [ 97.109823] niks3-server[992]: 2026/09/22 16:43:38 INFO Completed upload id=295075worker2 # [ 97.149345] nix-grpc-daemon[769]: ts=2026-09-22T16:43:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=2 nar_bytes_in=9845076worker2 # [ 97.164620] nix-grpc-daemon[769]: ts=2026-09-22T16:43:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459465077worker2 # [ 97.177574] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5078worker2 # [ 97.188348] nix-daemon[838]: reaped child process 1078, status = succeeded5079client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...5080worker2 # [ 97.200631] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5081client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5082client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5083worker2 # [ 98.080222] nix-daemon[838]: reaped child process 1087, status = succeeded5084worker2 # [ 98.189991] nix-daemon[838]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)5085worker2 # [ 98.209131] nix-daemon[838]: reaped child process 1106, status = succeeded5086lb # [ 98.181886] niks3-server[992]: 2026/09/22 16:43:39 INFO Received uploads request method=POST path=/api/pending_closures5087worker2 # [ 98.228630] nix-grpc-daemon[865]: time=2026-09-22T16:43:39.862Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5088worker2 # [ 98.259914] nix-grpc-daemon[865]: time=2026-09-22T16:43:39.895Z level=INFO msg="Uploading 1 narinfos"5089lb # [ 98.219671] niks3-server[992]: 2026/09/22 16:43:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5090lb # [ 98.231836] niks3-server[992]: 2026/09/22 16:43:39 INFO Signed narinfos id=30 count=15091lb # [ 98.239115] niks3-server[992]: 2026/09/22 16:43:39 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5092worker2 # [ 98.287955] nix-grpc-daemon[865]: time=2026-09-22T16:43:39.923Z level=INFO msg="Upload complete. (274ms)"5093lb # [ 98.254817] niks3-server[992]: 2026/09/22 16:43:39 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5094lb # [ 98.262596] niks3-server[992]: 2026/09/22 16:43:39 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5095lb # [ 98.267681] niks3-server[992]: 2026/09/22 16:43:39 INFO Completed upload id=305096lb # [ 98.270891] niks3-server[992]: 2026/09/22 16:43:39 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5097worker2 # [ 98.301178] nix-grpc-daemon[769]: ts=2026-09-22T16:43:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15098worker2 # [ 98.317913] nix-daemon[838]: reaped child process 1085, status = succeeded5099client # copying 1 paths...5100worker1 # [ 98.290481] nix-grpc-daemon[768]: ts=2026-09-22T16:43:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582725101worker1 # [ 98.298457] nix-grpc-daemon[768]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...5102client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5103worker1 # [ 98.341556] nix-grpc-daemon[768]: ts=2026-09-22T16:43:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:582725104worker1 # [ 98.348562] nix-grpc-daemon[768]: ts=2026-09-22T16:43:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_out=1205105client # [ 98.544916] nix-daemon[1156]: reaped child process 1242, status = succeeded5106client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.51 seconds)5107client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25108client # [ 98.720219] nix-daemon[1156]: accepted connection from pid 1260, user root (trusted)5109client # this derivation will be built:5110client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5111worker2 # [ 98.856647] nix-grpc-daemon[769]: ts=2026-09-22T16:43:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:459465112client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5113worker1 # [ 98.800635] nix-grpc-daemon[768]: ts=2026-09-22T16:43:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582725114worker1 # [ 98.811170] nix-grpc-daemon[866]: time=2026-09-22T16:43:40.433Z level=INFO msg="All 1 paths already cached"5115worker1 # [ 98.827316] nix-grpc-daemon[768]: ts=2026-09-22T16:43:40Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05116worker1 # [ 98.836537] nix-grpc-daemon[768]: ts=2026-09-22T16:43:40Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15117worker1 # [ 98.845028] nix-grpc-daemon[768]: ts=2026-09-22T16:43:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582725118worker1 # [ 98.852816] nix-grpc-daemon[768]: ts=2026-09-22T16:43:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582725119client # copying 1 paths to 'lb:50051'...5120client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5121worker1 # [ 98.864894] nix-grpc-daemon[768]: ts=2026-09-22T16:43:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582725122worker1 # [ 98.880252] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5123worker1 # [ 98.899598] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5)5124worker1 # [ 98.950827] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 2/5)5125worker1 # [ 99.143720] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 3/5)5126worker1 # [ 99.175501] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 4/5)5127lb # [ 99.216711] envoy[899]: conn peer=[2001:db8:1::1]:56078 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3440 tx=1966 ms=20935128worker1 # [ 99.272818] nix-grpc-daemon[768]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5129worker1 # [ 99.280203] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5130worker1 # [ 99.286872] nix-grpc-daemon[768]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5131worker1 # [ 99.456587] nix-daemon[851]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)5132worker1 # [ 99.478585] nix-daemon[851]: reaped child process 1269, status = succeeded5133lb # [ 99.221653] envoy[899]: conn peer=[2001:db8:1::1]:36298 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=665134lb # [ 99.221849] envoy[899]: conn peer=[2001:db8:1::1]:36302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3903 tx=2171 ms=18455135lb # [ 99.221949] envoy[899]: conn peer=[2001:db8:1::1]:36306 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=945136lb # [ 99.222043] envoy[899]: conn peer=[2001:db8:1::1]:36308 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2390 ms=15905137lb # [ 99.222135] envoy[899]: conn peer=[2001:db8:1::1]:36310 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=295138lb # [ 99.222243] envoy[899]: conn peer=[2001:db8:1::1]:36332 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=725139lb # [ 99.222352] envoy[899]: conn peer=[2001:db8:1::1]:36316 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4236 tx=2056 ms=20985140lb # [ 99.509229] niks3-server[992]: 2026/09/22 16:43:41 INFO Received uploads request method=POST path=/api/pending_closures5141worker1 # [ 99.522895] nix-grpc-daemon[866]: time=2026-09-22T16:43:41.145Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5142worker1 # [ 99.525926] nix-grpc-daemon[866]: time=2026-09-22T16:43:41.145Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5143lb # [ 99.565926] niks3-server[992]: 2026/09/22 16:43:41 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5144worker1 # [ 99.554567] nix-grpc-daemon[866]: time=2026-09-22T16:43:41.177Z level=INFO msg="Uploading 1 narinfos"5145lb # [ 99.581174] niks3-server[992]: 2026/09/22 16:43:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5146lb # [ 99.591934] niks3-server[992]: 2026/09/22 16:43:41 INFO Signed narinfos id=31 count=15147lb # [ 99.598710] niks3-server[992]: 2026/09/22 16:43:41 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5148lb # [ 99.608738] niks3-server[992]: 2026/09/22 16:43:41 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5149lb # [ 99.615318] niks3-server[992]: 2026/09/22 16:43:41 INFO Completed upload id=315150worker1 # [ 99.588204] nix-grpc-daemon[866]: time=2026-09-22T16:43:41.211Z level=INFO msg="Upload complete. (283ms)"5151lb # [ 99.618188] niks3-server[992]: 2026/09/22 16:43:41 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5152worker1 # [ 99.599914] nix-grpc-daemon[768]: ts=2026-09-22T16:43:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_in=5685153worker1 # [ 99.616547] nix-grpc-daemon[768]: ts=2026-09-22T16:43:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582725154worker1 # [ 99.632521] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5155worker1 # [ 99.636552] nix-daemon[851]: reaped child process 1259, status = succeeded5156client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5157client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5158worker1 # [ 99.654492] nix-daemon[851]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5159client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5160worker1 # [ 101.127469] nix-daemon[851]: reaped child process 1273, status = succeeded5161worker1 # [ 101.248762] nix-daemon[851]: accepted connection from pid 1289, user nix-grpc-daemon (trusted)5162worker1 # [ 101.266235] nix-daemon[851]: reaped child process 1292, status = succeeded5163lb # [ 101.301105] niks3-server[992]: 2026/09/22 16:43:42 INFO Received uploads request method=POST path=/api/pending_closures5164worker1 # [ 101.284412] nix-grpc-daemon[866]: time=2026-09-22T16:43:42.906Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5165lb # [ 101.329783] niks3-server[992]: 2026/09/22 16:43:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5166worker1 # [ 101.311030] nix-grpc-daemon[866]: time=2026-09-22T16:43:42.934Z level=INFO msg="Uploading 1 narinfos"5167lb # [ 101.343589] niks3-server[992]: 2026/09/22 16:43:42 INFO Signed narinfos id=32 count=15168lb # [ 101.352443] niks3-server[992]: 2026/09/22 16:43:42 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5169lb # [ 101.363531] niks3-server[992]: 2026/09/22 16:43:42 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5170worker1 # [ 101.342139] nix-grpc-daemon[866]: time=2026-09-22T16:43:42.965Z level=INFO msg="Upload complete. (273ms)"5171lb # [ 101.371774] niks3-server[992]: 2026/09/22 16:43:43 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5172lb # [ 101.375322] niks3-server[992]: 2026/09/22 16:43:43 INFO Completed upload id=325173lb # [ 101.378496] niks3-server[992]: 2026/09/22 16:43:43 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5174worker1 # [ 101.357100] nix-grpc-daemon[768]: ts=2026-09-22T16:43:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15175worker1 # [ 101.371442] nix-daemon[851]: reaped child process 1271, status = succeeded5176client # copying 1 paths...5177worker1 # [ 101.402448] nix-grpc-daemon[768]: ts=2026-09-22T16:43:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582725178client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5179worker1 # [ 101.433334] nix-grpc-daemon[768]: ts=2026-09-22T16:43:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:582725180worker1 # [ 101.440283] nix-grpc-daemon[768]: ts=2026-09-22T16:43:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58272 duration_s=0 paths=1 nar_bytes_out=1205181client # [ 101.632611] nix-daemon[1156]: reaped child process 1263, status = succeeded5182client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.09 seconds)5183client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25184client # [ 101.808569] nix-daemon[1156]: accepted connection from pid 1282, user root (trusted)5185client # this derivation will be built:5186client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5187client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5188worker2 # [ 101.950980] nix-grpc-daemon[769]: ts=2026-09-22T16:43:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:459465189worker2 # [ 101.958733] nix-grpc-daemon[769]: ts=2026-09-22T16:43:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459465190worker2 # [ 101.966848] nix-grpc-daemon[865]: time=2026-09-22T16:43:43.600Z level=INFO msg="All 1 paths already cached"5191worker1 # [ 101.917082] nix-grpc-daemon[768]: ts=2026-09-22T16:43:43Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05192worker1 # [ 101.923805] nix-grpc-daemon[768]: ts=2026-09-22T16:43:43Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15193worker2 # [ 101.987106] nix-grpc-daemon[769]: ts=2026-09-22T16:43:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459465194worker2 # [ 102.001579] nix-grpc-daemon[769]: ts=2026-09-22T16:43:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459465195worker2 # [ 102.015447] nix-grpc-daemon[865]: time=2026-09-22T16:43:43.651Z level=INFO msg="All 1 paths already cached"5196client # copying 1 paths to 'lb:50051'...5197client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5198worker2 # [ 102.025998] nix-grpc-daemon[769]: ts=2026-09-22T16:43:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459465199worker2 # [ 102.041300] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5200worker2 # [ 102.202840] nix-daemon[838]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)5201worker2 # [ 102.220698] nix-daemon[838]: reaped child process 1113, status = succeeded5202lb # [ 102.192325] niks3-server[992]: 2026/09/22 16:43:43 INFO Received uploads request method=POST path=/api/pending_closures5203worker2 # [ 102.235031] nix-grpc-daemon[865]: time=2026-09-22T16:43:43.870Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5204worker2 # [ 102.237196] nix-grpc-daemon[865]: time=2026-09-22T16:43:43.870Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5205lb # [ 102.224190] niks3-server[992]: 2026/09/22 16:43:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5206worker2 # [ 102.264230] nix-grpc-daemon[865]: time=2026-09-22T16:43:43.899Z level=INFO msg="Uploading 1 narinfos"5207lb # [ 102.238898] niks3-server[992]: 2026/09/22 16:43:43 INFO Signed narinfos id=33 count=15208lb # [ 102.245894] niks3-server[992]: 2026/09/22 16:43:43 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5209worker2 # [ 102.291772] nix-grpc-daemon[865]: time=2026-09-22T16:43:43.927Z level=INFO msg="Upload complete. (233ms)"5210lb # [ 102.258297] niks3-server[992]: 2026/09/22 16:43:43 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5211worker2 # [ 102.300353] nix-grpc-daemon[769]: ts=2026-09-22T16:43:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_in=5685212lb # [ 102.268854] niks3-server[992]: 2026/09/22 16:43:43 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5213lb # [ 102.273601] niks3-server[992]: 2026/09/22 16:43:43 INFO Completed upload id=335214lb # [ 102.277223] niks3-server[992]: 2026/09/22 16:43:43 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5215worker2 # [ 102.316488] nix-grpc-daemon[769]: ts=2026-09-22T16:43:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459465216worker2 # [ 102.323215] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5217worker2 # [ 102.332442] nix-daemon[838]: reaped child process 1108, status = succeeded5218client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5219worker2 # [ 102.342111] nix-daemon[838]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5220client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5221client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5222worker2 # [ 104.000481] nix-daemon[838]: reaped child process 1117, status = succeeded5223worker2 # [ 104.131246] nix-daemon[838]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5224worker2 # [ 104.151876] nix-daemon[838]: reaped child process 1136, status = succeeded5225lb # [ 104.122597] niks3-server[992]: 2026/09/22 16:43:45 INFO Received uploads request method=POST path=/api/pending_closures5226worker2 # [ 104.167707] nix-grpc-daemon[865]: time=2026-09-22T16:43:45.802Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5227worker2 # [ 104.198225] nix-grpc-daemon[865]: time=2026-09-22T16:43:45.833Z level=INFO msg="Uploading 1 narinfos"5228lb # [ 104.151332] niks3-server[992]: 2026/09/22 16:43:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5229lb # [ 104.167897] niks3-server[992]: 2026/09/22 16:43:45 INFO Signed narinfos id=34 count=15230lb # [ 104.175375] niks3-server[992]: 2026/09/22 16:43:45 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5231lb # [ 104.182034] niks3-server[992]: 2026/09/22 16:43:45 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5232lb # [ 104.193092] niks3-server[992]: 2026/09/22 16:43:45 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5233lb # [ 104.198354] niks3-server[992]: 2026/09/22 16:43:45 INFO Completed upload id=345234worker2 # [ 104.231651] nix-grpc-daemon[865]: time=2026-09-22T16:43:45.867Z level=INFO msg="Upload complete. (279ms)"5235lb # [ 104.203274] niks3-server[992]: 2026/09/22 16:43:45 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5236worker2 # [ 104.245101] nix-grpc-daemon[769]: ts=2026-09-22T16:43:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15237worker2 # [ 104.261619] nix-daemon[838]: reaped child process 1115, status = succeeded5238client # copying 1 paths...5239worker2 # [ 104.291065] nix-grpc-daemon[769]: ts=2026-09-22T16:43:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:459465240client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5241worker2 # [ 104.324191] nix-grpc-daemon[769]: ts=2026-09-22T16:43:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:459465242worker2 # [ 104.331010] nix-grpc-daemon[769]: ts=2026-09-22T16:43:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45946 duration_s=0 paths=1 nar_bytes_out=1205243client # [ 104.463406] nix-daemon[1156]: reaped child process 1285, status = succeeded5244client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.83 seconds)5245(finished: subtest: an input only one worker has still reaches the builder, in 9.06 seconds)5246subtest: upload whose reference the worker lacks is completed from the cache5247client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5248client # this derivation will be built:5249client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5250client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5251client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5252client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5253client # this derivation will be built:5254client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5255client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5256client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)5257client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5258client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5259worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5260worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5261worker2 # [ 105.465261] systemd[1]: Stopping Nix worker-protocol over gRPC...5262worker2 # [ 105.595824] nix-grpc-daemon[769]: ts=2026-09-22T16:43:47Z level=info event=signal_exit5263worker2 # [ 106.764506] nix-grpc-daemon[769]: ts=2026-09-22T16:43:48Z level=info event=scheduler_restarting5264worker2 # [ 106.792238] nix-grpc-daemon[769]: E0000 00:00:1790095428.427810 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5265worker1 # [ 106.745544] nix-grpc-daemon[768]: ts=2026-09-22T16:43:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:582705266worker1 # [ 106.750825] nix-grpc-daemon[768]: ts=2026-09-22T16:43:48Z level=info event=worker_gone addr=192.168.1.4:500515267worker2 # [ 106.840748] nix-grpc-daemon[769]: ts=2026-09-22T16:43:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5268worker2 # [ 106.857026] nix-daemon[838]: reaped child process 840, status = succeeded5269worker2 # [ 107.751429] nix-grpc-daemon[769]: E0000 00:00:1790095429.387161 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5270worker2 # [ 108.214193] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5271worker2 # [ 108.220850] systemd[1]: Stopped Nix worker-protocol over gRPC.5272worker2 # [ 108.223520] systemd[1]: nix-grpc-daemon.service: Consumed 2.836s CPU time over 1min 28.846s wall clock time, 60.6M memory peak, 85.9K incoming IP traffic, 211.2K outgoing IP traffic.5273worker2 # [ 108.242208] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5274lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5275lb: must succeed: curl -sf localhost:9901/clusters5276worker2 # [ 108.254341] systemd[1]: Closed Nix worker-protocol over gRPC.5277worker2 # [ 108.256010] systemd[1]: nix-grpc-daemon.socket: Consumed 634us CPU time over 1min 37.790s wall clock time, 256K memory peak, 95.5K incoming IP traffic, 83.6K outgoing IP traffic.5278lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5279lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5280client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25281worker1 # [ 108.495465] nix-grpc-daemon[768]: ts=2026-09-22T16:43:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582725282worker1 # [ 108.504792] nix-grpc-daemon[768]: ts=2026-09-22T16:43:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582725283client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5284client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.28 seconds)5285worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5286worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5287worker2 # [ 108.710205] systemd[1]: Listening on Nix worker-protocol over gRPC.5288worker1 # [ 108.721435] systemd[1]: Stopping Nix worker-protocol over gRPC...5289worker2 # [ 109.210373] systemd[1]: Starting scheduler-token.service...5290lb # [ 109.215676] envoy[899]: conn peer=[2001:db8:1::1]:36346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2124 ms=27005291lb # [ 109.220872] envoy[899]: conn peer=[2001:db8:1::1]:46664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=665292lb # [ 109.225817] envoy[899]: conn peer=[2001:db8:1::1]:46680 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3754 tx=2100 ms=24395293lb # [ 109.230803] envoy[899]: conn peer=[2001:db8:1::1]:46696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=705294lb # [ 109.236642] envoy[899]: conn peer=[2001:db8:1::4]:54856 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=69383 tx=3687 ms=871895295lb # [ 109.241888] envoy[899]: conn peer=[2001:db8:1::1]:46708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=315296worker2 # [ 109.302328] systemd[1]: scheduler-token.service: Deactivated successfully.5297worker2 # [ 109.313246] systemd[1]: Finished scheduler-token.service.5298worker2 # [ 109.315374] systemd[1]: scheduler-token.service: Consumed 58ms CPU time over 96ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5299worker2 # [ 109.323114] systemd[1]: Starting Nix worker-protocol over gRPC...5300worker2 # [ 109.459215] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5301worker2 # [ 109.461674] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5302worker2 # [ 109.465559] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15303worker2 # [ 109.472077] nix-grpc-daemon[1160]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5304worker2 # [ 109.473536] nix-grpc-daemon[1160]: E0000 00:00:1790095431.106115 1160 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5305worker2 # [ 109.516072] nix-grpc-daemon[1160]: E0000 00:00:1790095431.112450 1160 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5306worker2 # [ 109.518419] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5307worker2 # [ 109.525547] nix-grpc-daemon[1160]: E0000 00:00:1790095431.123615 1199 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5308worker2 # [ 109.528472] nix-grpc-daemon[1160]: E0000 00:00:1790095431.141967 1199 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5309worker2 # [ 109.530901] nix-grpc-daemon[1160]: E0000 00:00:1790095431.142118 1199 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5310worker1 # [ 109.469433] nix-grpc-daemon[768]: ts=2026-09-22T16:43:51Z level=info event=signal_exit5311worker2 # [ 109.536083] systemd[1]: Started Nix worker-protocol over gRPC.5312worker2 # [ 109.538852] nix-grpc-daemon[1160]: E0000 00:00:1790095431.160738 1199 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5313worker2 # [ 109.541322] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:51Z level=info event=scheduler_connected addr=lb:500515314worker1 # [ 109.479925] nix-grpc-daemon[768]: ts=2026-09-22T16:43:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:582705315worker1 # [ 109.486588] nix-grpc-daemon[768]: ts=2026-09-22T16:43:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05316lb # [ 109.729272] niks3-server[992]: 2026/09/22 16:43:51 INFO lead: released remote=192.168.1.3:465945317worker1 # [ 110.701278] nix-grpc-daemon[768]: ts=2026-09-22T16:43:52Z level=info event=scheduler_restarting5318worker2 # [ 110.763971] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:52Z level=info event=scheduler_restarting addr=lb:500515319worker1 # [ 110.712489] nix-grpc-daemon[768]: ts=2026-09-22T16:43:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:582705320worker1 # [ 110.718770] nix-grpc-daemon[768]: ts=2026-09-22T16:43:52Z level=info event=worker_gone addr=192.168.1.4:500515321worker1 # [ 110.723325] nix-grpc-daemon[768]: ts=2026-09-22T16:43:52Z level=info event=scheduler_restarting addr=lb:500515322worker1 # [ 110.728491] nix-grpc-daemon[768]: ts=2026-09-22T16:43:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:582705323worker1 # [ 110.734154] nix-grpc-daemon[768]: ts=2026-09-22T16:43:52Z level=info event=worker_gone addr=192.168.1.3:500515324worker1 # [ 110.762941] nix-daemon[851]: reaped child process 853, status = succeeded5325worker2 # [ 110.963918] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:52Z level=info event=scheduler_connected addr=lb:500515326worker2 # [ 110.969284] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5327worker1 # [ 110.937474] nix-grpc-daemon[768]: ts=2026-09-22T16:43:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5328worker2 # [ 111.170495] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:52Z level=info event=scheduler_connected addr=lb:500515329worker2 # [ 111.175761] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5330worker2 # [ 111.376862] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_connected addr=lb:500515331worker2 # [ 111.382056] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5332worker2 # [ 111.583012] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_connected addr=lb:500515333worker2 # [ 111.588216] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5334worker1 # [ 111.643205] nix-grpc-daemon[768]: E0000 00:00:1790095433.266332 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5335worker1 # [ 111.643617] nix-grpc-daemon[768]: E0000 00:00:1790095433.266645 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5336worker2 # [ 111.791151] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_connected addr=lb:500515337worker2 # [ 111.795410] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5338worker2 # [ 111.992192] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_connected addr=lb:500515339worker2 # [ 111.997424] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5340worker1 # [ 112.094432] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5341worker1 # [ 112.104366] systemd[1]: Stopped Nix worker-protocol over gRPC.5342worker1 # [ 112.107073] systemd[1]: nix-grpc-daemon.service: Consumed 3.903s CPU time over 1min 32.908s wall clock time, 69.9M memory peak, 114.8K incoming IP traffic, 233.8K outgoing IP traffic.5343worker1 # [ 112.123901] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5344worker1 # [ 112.132210] systemd[1]: Closed Nix worker-protocol over gRPC.5345worker1 # [ 112.134021] systemd[1]: nix-grpc-daemon.socket: Consumed 675us CPU time over 1min 41.662s wall clock time, 256K memory peak, 130.1K incoming IP traffic, 111.7K outgoing IP traffic.5346lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5347worker2 # [ 112.198540] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_connected addr=lb:500515348lb: must succeed: curl -sf localhost:9901/clusters5349worker2 # [ 112.203623] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5350lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5351lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5352client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25353worker2 # [ 112.404841] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_connected addr=lb:500515354worker2 # [ 112.409877] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5355worker2 # [ 112.501064] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500645356worker2 # [ 112.517237] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5357worker2 # [ 112.544285] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)5358worker2 # [ 112.589620] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)5359worker2 # [ 112.610887] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_connected addr=lb:500515360worker2 # [ 112.616004] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5361worker2 # [ 112.790748] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 272 ms (attempt 3/5)5362worker2 # [ 112.817148] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_connected addr=lb:500515363worker2 # [ 112.822148] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker2 # [ 113.023152] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_connected addr=lb:500515365worker2 # [ 113.028268] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5366worker2 # [ 113.063825] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 422 ms (attempt 4/5)5367worker2 # [ 113.229339] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_connected addr=lb:500515368worker2 # [ 113.234565] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5369worker2 # [ 113.435653] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_connected addr=lb:500515370worker2 # [ 113.440592] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5371worker2 # [ 113.496241] nix-grpc-daemon[1160]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5372worker2 # [ 113.503372] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5373client # copying 2 paths...5374client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5375worker2 # [ 113.529834] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500645376client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5377worker2 # [ 113.549162] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5378worker2 # [ 113.580836] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=niks3_push_started pid=12205379worker2 # [ 113.644968] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_connected addr=lb:500515380worker2 # [ 113.650382] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5381worker2 # [ 113.743297] nix-grpc-daemon[1220]: time=2026-09-22T16:43:55.378Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5382worker2 # [ 113.848450] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_connected addr=lb:500515383worker2 # [ 113.852931] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker2 # [ 113.854586] nix-daemon[838]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)5385worker2 # [ 113.872773] nix-daemon[838]: reaped child process 1236, status = succeeded5386lb # [ 113.845412] niks3-server[992]: 2026/09/22 16:43:55 INFO Received uploads request method=POST path=/api/pending_closures5387lb # [ 113.856423] niks3-server[992]: 2026/09/22 16:43:55 INFO Received uploads request method=POST path=/api/pending_closures5388worker2 # [ 113.900080] nix-grpc-daemon[1220]: time=2026-09-22T16:43:55.534Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5389worker2 # [ 113.901554] nix-grpc-daemon[1220]: time=2026-09-22T16:43:55.534Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5390worker2 # [ 113.903264] nix-grpc-daemon[1220]: time=2026-09-22T16:43:55.534Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5391lb # [ 113.916687] niks3-server[992]: 2026/09/22 16:43:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5392worker2 # [ 113.961539] nix-grpc-daemon[1220]: time=2026-09-22T16:43:55.595Z level=INFO msg="Uploading 3 narinfos"5393lb # [ 113.933623] niks3-server[992]: 2026/09/22 16:43:55 INFO Signed narinfos id=35 count=15394lb # [ 113.935561] niks3-server[992]: 2026/09/22 16:43:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5395lb # [ 113.949298] niks3-server[992]: 2026/09/22 16:43:55 INFO Signed narinfos id=36 count=25396lb # [ 113.950878] niks3-server[992]: 2026/09/22 16:43:55 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5397lb # [ 113.964851] niks3-server[992]: 2026/09/22 16:43:55 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5398lb # [ 113.966876] niks3-server[992]: 2026/09/22 16:43:55 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5399lb # [ 113.973553] niks3-server[992]: 2026/09/22 16:43:55 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5400lb # [ 113.982197] niks3-server[992]: 2026/09/22 16:43:55 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5401lb # [ 113.985053] niks3-server[992]: 2026/09/22 16:43:55 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5402lb # [ 113.987686] niks3-server[992]: 2026/09/22 16:43:55 INFO Completed upload id=355403lb # [ 113.991496] niks3-server[992]: 2026/09/22 16:43:55 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5404lb # [ 113.994663] niks3-server[992]: 2026/09/22 16:43:55 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5405worker2 # [ 114.028655] nix-grpc-daemon[1220]: time=2026-09-22T16:43:55.664Z level=INFO msg="Upload complete. (281ms)"5406lb # [ 113.998273] niks3-server[992]: 2026/09/22 16:43:55 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5407lb # [ 114.000485] niks3-server[992]: 2026/09/22 16:43:55 INFO Completed upload id=365408worker2 # [ 114.040337] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50064 duration_s=0 paths=2 nar_bytes_in=3125409worker2 # [ 114.052987] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_connected addr=lb:500515410worker2 # [ 114.063673] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5411worker2 # [ 114.069287] nix-daemon[838]: reaped child process 1218, status = succeeded5412client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.80 seconds)5413worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5414worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5415worker1 # [ 114.123075] systemd[1]: Listening on Nix worker-protocol over gRPC.5416worker1 # [ 114.133997] systemd[1]: Starting Nix worker-protocol over gRPC...5417worker2 # [ 114.254182] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_connected addr=lb:500515418worker2 # [ 114.259433] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5419worker1 # [ 114.281746] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5420worker1 # [ 114.284104] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5421worker1 # [ 114.288277] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15422worker1 # [ 114.294736] nix-grpc-daemon[1317]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5423worker1 # [ 114.304357] nix-grpc-daemon[1317]: E0000 00:00:1790095435.916094 1317 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5424lb # [ 114.332698] niks3-server[992]: 2026/09/22 16:43:55 INFO lead: acquired remote=192.168.1.3:541345425worker1 # [ 114.312160] nix-grpc-daemon[1317]: E0000 00:00:1790095435.918126 1317 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5426worker1 # [ 114.314531] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5427lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5428lb: must succeed: curl -sf localhost:9901/clusters5429worker1 # [ 114.344662] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:55Z level=info event=scheduler_take_over5430worker1 # [ 114.346341] nix-grpc-daemon[1317]: E0000 00:00:1790095435.932014 1359 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5431worker1 # [ 114.349063] systemd[1]: Started Nix worker-protocol over gRPC.5432worker1 # [ 114.349889] nix-grpc-daemon[1317]: E0000 00:00:1790095435.951838 1359 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5433worker1 # [ 114.352221] nix-grpc-daemon[1317]: E0000 00:00:1790095435.952577 1359 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5434worker1 # [ 114.357875] nix-grpc-daemon[1317]: E0000 00:00:1790095435.967174 1359 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5435worker1 # [ 114.362286] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:55Z level=info event=scheduler_connected addr=lb:500515436worker1 # [ 114.363578] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437worker2 # [ 114.460504] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:56Z level=info event=scheduler_connected addr=lb:500515438lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5439worker2 # [ 114.465661] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5440worker1 # [ 114.563424] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=scheduler_connected addr=lb:500515441worker1 # [ 114.568625] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5442worker2 # [ 114.666757] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:56Z level=info event=scheduler_connected addr=lb:500515443worker2 # [ 114.671963] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5444worker1 # [ 114.769318] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=scheduler_connected addr=lb:500515445worker1 # [ 114.774606] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5446worker2 # [ 114.874979] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:56Z level=info event=scheduler_connected addr=lb:500515447worker2 # [ 114.879071] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5448worker1 # [ 114.975407] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=scheduler_connected addr=lb:500515449worker1 # [ 114.980884] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5450worker2 # [ 115.075561] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:56Z level=info event=scheduler_connected addr=lb:500515451worker2 # [ 115.080738] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5452worker1 # [ 115.181554] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=scheduler_connected addr=lb:500515453worker1 # [ 115.189891] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:539165454worker1 # [ 115.195816] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05455worker2 # [ 115.281756] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:56Z level=info event=scheduler_connected addr=lb:500515456worker1 # [ 115.230714] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=oidc_jwks provider=mock keys=15457worker1 # [ 115.235296] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:539165458worker1 # [ 115.240734] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:56Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05459lb: must succeed: curl -sf localhost:9901/clusters5460lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5461lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds)5462(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.07 seconds)5463subtest: a niks3 restart does not cost the leader its role5464lb: must succeed: curl -sf localhost:9901/clusters5465lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5466lb: must succeed: curl -sf localhost:9901/clusters5467lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5468lb # [ 115.624699] systemd[1]: Stopping niks3 server...5469lb # [ 115.626992] niks3-server[992]: 2026/09/22 16:43:57 INFO Shutdown signal received, draining in-flight requests timeout=10s5470worker2 # [ 115.664824] nix-grpc-daemon[1160]: ts=2026-09-22T16:43:57Z level=info event=scheduler_lock_lost status=2005471worker1 # [ 115.605259] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:57Z level=info event=scheduler_lock_lost status=2005472lb # [ 115.650642] niks3-server[992]: 2026/09/22 16:43:57 INFO lead: released remote=192.168.1.3:541345473lb # [ 115.655022] systemd[1]: niks3.service: Deactivated successfully.5474lb # [ 115.657087] systemd[1]: Stopped niks3 server.5475lb # [ 115.657802] systemd[1]: niks3.service: Consumed 647ms CPU time over 1min 29.181s wall clock time, 27.7M memory peak, 95.5K incoming IP traffic, 138.9K outgoing IP traffic.5476lb # [ 115.667100] systemd[1]: Starting niks3 server...5477lb # [ 115.733247] niks3-server[1133]: 2026/09/22 16:43:57 goose: no migrations to run. current version: 202609200000005478lb # [ 115.740949] niks3-server[1133]: 2026/09/22 16:43:57 OK 1_commit_pending_closure.sql (5.8ms)5479lb # [ 115.744727] niks3-server[1133]: 2026/09/22 16:43:57 OK 2_object_stats_trigger.sql (3.34ms)5480lb # [ 115.745866] niks3-server[1133]: 2026/09/22 16:43:57 goose: up to current file version: 25481lb # [ 115.747543] niks3-server[1133]: 2026/09/22 16:43:57 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5482lb # [ 115.760757] niks3-server[1133]: 2026/09/22 16:43:57 INFO Read proxy enabled — serving cache objects from S35483lb # [ 115.762094] niks3-server[1133]: 2026/09/22 16:43:57 INFO Using socket-activated listener address=0.0.0.0:57515484lb # [ 115.767460] niks3-server[1133]: 2026/09/22 16:43:57 INFO systemd watchdog enabled interval=15s5485lb # [ 115.769428] niks3-server[1133]: 2026/09/22 16:43:57 INFO Starting HTTP server address=0.0.0.0:57515486lb # [ 115.772319] systemd[1]: Started niks3 server.5487lb: waiting for unit niks3.service5488lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5489lb: must succeed: curl -sf localhost:9901/clusters5490lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5491worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5492worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5493lb: must succeed: curl -sf localhost:9901/clusters5494lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5495lb: must succeed: curl -sf localhost:9901/clusters5496lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5497worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5498worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5499client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25500worker1 # [ 116.260147] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:539305501worker1 # [ 116.278477] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5502client # don't know how to build these paths:5503client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5504worker1 # [ 116.301489] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:539305505worker1 # [ 116.322291] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5)5506worker1 # [ 116.399110] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 180 ms (attempt 2/5)5507worker1 # [ 116.581365] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 199 ms (attempt 3/5)5508lb # [ 116.637941] niks3-server[1133]: 2026/09/22 16:43:58 INFO lead: acquired remote=192.168.1.3:541425509worker1 # [ 116.790946] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 521 ms (attempt 4/5)5510worker1 # [ 117.314132] nix-grpc-daemon[1317]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5511worker1 # [ 117.321536] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5512worker1 # [ 117.368158] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:58Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05513worker1 # [ 117.374922] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:58Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=15514worker1 # [ 117.422347] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:539305515worker1 # [ 117.437699] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:539305516client # copying 1 paths to 'lb:50051'...5517client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5518worker1 # [ 117.452666] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:539305519worker1 # [ 117.468709] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5520worker1 # [ 117.507858] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:59Z level=info event=niks3_push_started pid=13895521worker1 # [ 117.640806] nix-grpc-daemon[1389]: time=2026-09-22T16:43:59.263Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5522worker1 # [ 117.749110] nix-daemon[851]: accepted connection from pid 1401, user nix-grpc-daemon (trusted)5523worker1 # [ 117.765886] nix-daemon[851]: reaped child process 1404, status = succeeded5524lb # [ 117.801898] niks3-server[1133]: 2026/09/22 16:43:59 INFO Received uploads request method=POST path=/api/pending_closures5525worker1 # [ 117.790524] nix-grpc-daemon[1389]: time=2026-09-22T16:43:59.412Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5526worker1 # [ 117.794537] nix-grpc-daemon[1389]: time=2026-09-22T16:43:59.412Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5527lb # [ 117.845020] niks3-server[1133]: 2026/09/22 16:43:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5528lb # [ 117.886172] niks3-server[1133]: 2026/09/22 16:43:59 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5529lb # [ 117.899456] niks3-server[1133]: 2026/09/22 16:43:59 INFO Signed narinfos id=37 count=15530worker1 # [ 117.876108] nix-grpc-daemon[1389]: time=2026-09-22T16:43:59.498Z level=INFO msg="Uploading 1 narinfos"5531lb # [ 117.908876] niks3-server[1133]: 2026/09/22 16:43:59 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5532lb # [ 117.920829] niks3-server[1133]: 2026/09/22 16:43:59 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5533lb # [ 117.928164] niks3-server[1133]: 2026/09/22 16:43:59 INFO Completed upload id=375534worker1 # [ 117.901645] nix-grpc-daemon[1389]: time=2026-09-22T16:43:59.524Z level=INFO msg="Upload complete. (256ms)"5535lb # [ 117.936151] niks3-server[1133]: 2026/09/22 16:43:59 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5536worker1 # [ 117.914609] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53930 duration_s=0 paths=1 nar_bytes_in=5445537worker1 # [ 117.932603] nix-grpc-daemon[1317]: ts=2026-09-22T16:43:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:539305538worker1 # [ 117.943275] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5539worker1 # [ 117.947092] nix-daemon[851]: reaped child process 1387, status = succeeded5540client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5541worker1 # [ 117.958745] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5542client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5543client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5544client # farm-a> LOG-farm-a-after-niks3-restart5545lb # [ 119.221148] envoy[899]: conn peer=[2001:db8:1::3]:43416 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=14269 tx=3960 ms=914325546lb # [ 119.226329] envoy[899]: conn peer=[2001:db8:1::1]:40022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=282 ms=15535547worker1 # [ 119.250431] nix-daemon[851]: reaped child process 1408, status = succeeded5548worker1 # [ 119.342779] nix-daemon[851]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)5549worker1 # [ 119.363500] nix-daemon[851]: reaped child process 1427, status = succeeded5550lb # [ 119.395035] niks3-server[1133]: 2026/09/22 16:44:01 INFO Received uploads request method=POST path=/api/pending_closures5551worker1 # [ 119.383020] nix-grpc-daemon[1389]: time=2026-09-22T16:44:01.003Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5552worker1 # [ 119.386238] nix-grpc-daemon[1389]: time=2026-09-22T16:44:01.004Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5553lb # [ 119.435312] niks3-server[1133]: 2026/09/22 16:44:01 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5554lb # [ 119.451521] niks3-server[1133]: 2026/09/22 16:44:01 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5555worker1 # [ 119.436551] nix-grpc-daemon[1389]: time=2026-09-22T16:44:01.059Z level=INFO msg="Uploading 1 narinfos"5556lb # [ 119.463918] niks3-server[1133]: 2026/09/22 16:44:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5557lb # [ 119.471399] niks3-server[1133]: 2026/09/22 16:44:01 INFO Signed narinfos id=38 count=15558lb # [ 119.475801] niks3-server[1133]: 2026/09/22 16:44:01 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5559lb # [ 119.485761] niks3-server[1133]: 2026/09/22 16:44:01 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5560lb # [ 119.491482] niks3-server[1133]: 2026/09/22 16:44:01 INFO Completed upload id=385561worker1 # [ 119.463604] nix-grpc-daemon[1389]: time=2026-09-22T16:44:01.087Z level=INFO msg="Upload complete. (290ms)"5562lb # [ 119.494930] niks3-server[1133]: 2026/09/22 16:44:01 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5563worker1 # [ 119.489071] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53930 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15564worker1 # [ 119.500386] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:01Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05565worker1 # [ 119.507417] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:01Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=15566worker1 # [ 119.511746] nix-daemon[851]: reaped child process 1406, status = succeeded5567worker1 # [ 119.529883] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:539305568worker1 # [ 119.545745] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:539305569worker1 # [ 119.560745] nix-grpc-daemon[1389]: time=2026-09-22T16:44:01.184Z level=INFO msg="All 1 paths already cached"5570client # copying 1 paths to 'lb:50051'...5571client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5572worker1 # [ 119.571338] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:539305573worker1 # [ 119.584875] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5574worker1 # [ 119.733253] nix-daemon[851]: accepted connection from pid 1433, user nix-grpc-daemon (trusted)5575worker1 # [ 119.750818] nix-daemon[851]: reaped child process 1436, status = succeeded5576lb # [ 119.784485] niks3-server[1133]: 2026/09/22 16:44:01 INFO Received uploads request method=POST path=/api/pending_closures5577worker1 # [ 119.769983] nix-grpc-daemon[1389]: time=2026-09-22T16:44:01.393Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5578worker1 # [ 119.771632] nix-grpc-daemon[1389]: time=2026-09-22T16:44:01.393Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5579lb # [ 119.818085] niks3-server[1133]: 2026/09/22 16:44:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5580worker1 # [ 119.796864] nix-grpc-daemon[1389]: time=2026-09-22T16:44:01.420Z level=INFO msg="Uploading 1 narinfos"5581lb # [ 119.834132] niks3-server[1133]: 2026/09/22 16:44:01 INFO Signed narinfos id=39 count=15582lb # [ 119.842958] niks3-server[1133]: 2026/09/22 16:44:01 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5583lb # [ 119.851936] niks3-server[1133]: 2026/09/22 16:44:01 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5584worker1 # [ 119.840066] nix-grpc-daemon[1389]: time=2026-09-22T16:44:01.463Z level=INFO msg="Upload complete. (239ms)"5585lb # [ 119.867398] niks3-server[1133]: 2026/09/22 16:44:01 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5586lb # [ 119.872534] niks3-server[1133]: 2026/09/22 16:44:01 INFO Completed upload id=395587lb # [ 119.875985] niks3-server[1133]: 2026/09/22 16:44:01 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5588worker1 # [ 119.852369] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53930 duration_s=0 paths=1 nar_bytes_in=6565589worker1 # [ 119.868600] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:539305590worker1 # [ 119.879911] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5591worker1 # [ 119.883163] nix-daemon[851]: reaped child process 1431, status = succeeded5592client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5593worker1 # [ 119.895699] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5594client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5595client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5596client # farm-b> LOG-farm-b-after-niks3-restart5597worker1 # [ 121.376692] nix-daemon[851]: reaped child process 1440, status = succeeded5598worker1 # [ 121.481100] nix-daemon[851]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5599worker1 # [ 121.499217] nix-daemon[851]: reaped child process 1460, status = succeeded5600lb # [ 121.534338] niks3-server[1133]: 2026/09/22 16:44:03 INFO Received uploads request method=POST path=/api/pending_closures5601worker1 # [ 121.518821] nix-grpc-daemon[1389]: time=2026-09-22T16:44:03.141Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5602worker1 # [ 121.520756] nix-grpc-daemon[1389]: time=2026-09-22T16:44:03.141Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5603worker1 # [ 121.559973] nix-grpc-daemon[1389]: time=2026-09-22T16:44:03.183Z level=INFO msg="Uploading 1 narinfos"5604lb # [ 121.575054] niks3-server[1133]: 2026/09/22 16:44:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5605lb # [ 121.593948] niks3-server[1133]: 2026/09/22 16:44:03 INFO Signed narinfos id=40 count=15606lb # [ 121.602516] niks3-server[1133]: 2026/09/22 16:44:03 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5607lb # [ 121.615660] niks3-server[1133]: 2026/09/22 16:44:03 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5608lb # [ 121.628535] niks3-server[1133]: 2026/09/22 16:44:03 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5609worker1 # [ 121.608372] nix-grpc-daemon[1389]: time=2026-09-22T16:44:03.231Z level=INFO msg="Upload complete. (297ms)"5610lb # [ 121.638575] niks3-server[1133]: 2026/09/22 16:44:03 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5611lb # [ 121.644257] niks3-server[1133]: 2026/09/22 16:44:03 INFO Completed upload id=405612lb # [ 121.646532] niks3-server[1133]: 2026/09/22 16:44:03 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5613worker1 # [ 121.621337] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53930 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15614worker1 # [ 121.645590] nix-daemon[851]: reaped child process 1438, status = succeeded5615worker1 # [ 121.649457] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:03Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05616worker1 # [ 121.654880] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:03Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=15617worker1 # [ 121.698280] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:539305618worker1 # [ 121.714901] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:539305619worker1 # [ 121.731579] nix-grpc-daemon[1389]: time=2026-09-22T16:44:03.354Z level=INFO msg="All 2 paths already cached"5620client # copying 1 paths to 'lb:50051'...5621client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5622worker1 # [ 121.744126] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:539305623worker1 # [ 121.758829] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5624worker1 # [ 121.917081] nix-daemon[851]: accepted connection from pid 1466, user nix-grpc-daemon (trusted)5625worker1 # [ 121.939700] nix-daemon[851]: reaped child process 1469, status = succeeded5626lb # [ 121.965031] niks3-server[1133]: 2026/09/22 16:44:03 INFO Received uploads request method=POST path=/api/pending_closures5627worker1 # [ 121.949148] nix-grpc-daemon[1389]: time=2026-09-22T16:44:03.572Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5628worker1 # [ 121.950688] nix-grpc-daemon[1389]: time=2026-09-22T16:44:03.572Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5629lb # [ 122.000937] niks3-server[1133]: 2026/09/22 16:44:03 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5630worker1 # [ 121.994214] nix-grpc-daemon[1389]: time=2026-09-22T16:44:03.617Z level=INFO msg="Uploading 1 narinfos"5631lb # [ 122.013344] niks3-server[1133]: 2026/09/22 16:44:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5632lb # [ 122.026222] niks3-server[1133]: 2026/09/22 16:44:03 INFO Signed narinfos id=41 count=15633lb # [ 122.031428] niks3-server[1133]: 2026/09/22 16:44:03 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5634lb # [ 122.040341] niks3-server[1133]: 2026/09/22 16:44:03 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5635lb # [ 122.048307] niks3-server[1133]: 2026/09/22 16:44:03 INFO Completed upload id=415636worker1 # [ 122.021215] nix-grpc-daemon[1389]: time=2026-09-22T16:44:03.644Z level=INFO msg="Upload complete. (245ms)"5637lb # [ 122.051236] niks3-server[1133]: 2026/09/22 16:44:03 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5638worker1 # [ 122.032821] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53930 duration_s=0 paths=1 nar_bytes_in=7845639worker1 # [ 122.047648] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:539305640worker1 # [ 122.054173] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5641worker1 # [ 122.059319] nix-daemon[851]: reaped child process 1464, status = succeeded5642client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5643worker1 # [ 122.071018] nix-daemon[851]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5644client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5645client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5646client # farm-top> LOG-farm-top-after-niks3-restart5647worker1 # [ 123.742232] nix-daemon[851]: reaped child process 1473, status = succeeded5648worker1 # [ 123.860192] nix-daemon[851]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5649worker1 # [ 123.876627] nix-daemon[851]: reaped child process 1492, status = succeeded5650lb # [ 123.912713] niks3-server[1133]: 2026/09/22 16:44:05 INFO Received uploads request method=POST path=/api/pending_closures5651worker1 # [ 123.899589] nix-grpc-daemon[1389]: time=2026-09-22T16:44:05.520Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5652worker1 # [ 123.901884] nix-grpc-daemon[1389]: time=2026-09-22T16:44:05.520Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5653lb # [ 123.955383] niks3-server[1133]: 2026/09/22 16:44:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5654worker1 # [ 123.938609] nix-grpc-daemon[1389]: time=2026-09-22T16:44:05.561Z level=INFO msg="Uploading 1 narinfos"5655lb # [ 123.972813] niks3-server[1133]: 2026/09/22 16:44:05 INFO Signed narinfos id=42 count=15656lb # [ 123.980500] niks3-server[1133]: 2026/09/22 16:44:05 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5657lb # [ 123.992348] niks3-server[1133]: 2026/09/22 16:44:05 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5658worker1 # [ 123.965260] nix-grpc-daemon[1389]: time=2026-09-22T16:44:05.588Z level=INFO msg="Upload complete. (280ms)"5659lb # [ 123.995120] niks3-server[1133]: 2026/09/22 16:44:05 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5660lb # [ 124.004871] niks3-server[1133]: 2026/09/22 16:44:05 INFO Completed upload id=425661lb # [ 124.006760] niks3-server[1133]: 2026/09/22 16:44:05 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5662worker1 # [ 123.979207] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53930 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15663lb # [ 124.011424] niks3-server[1133]: 2026/09/22 16:44:05 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5664worker1 # [ 123.999561] nix-daemon[851]: reaped child process 1471, status = succeeded5665client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 8.01 seconds)5666client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5667client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5668(finished: subtest: a niks3 restart does not cost the leader its role, in 8.73 seconds)5669subtest: scheduler down: the other node takes the lock and keeps it5670lb: must succeed: curl -sf localhost:9901/clusters5671lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5672lb: must succeed: curl -sf localhost:9901/clusters5673lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5674lb: must succeed: curl -sf localhost:9901/clusters5675lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5676lb: must succeed: curl -sf localhost:9901/clusters5677lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5678worker1 # [ 124.447840] systemd[1]: Stopping Nix worker-protocol over gRPC...5679worker1 # [ 125.306313] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:06Z level=info event=signal_exit5680lb # [ 125.650237] niks3-server[1133]: 2026/09/22 16:44:07 INFO lead: released remote=192.168.1.3:541425681lb # [ 126.638818] niks3-server[1133]: 2026/09/22 16:44:08 INFO lead: acquired remote=192.168.1.4:383305682worker2 # [ 126.674806] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:08Z level=info event=scheduler_take_over5683worker2 # [ 126.682103] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:08Z level=info event=scheduler_restarting addr=lb:500515684worker1 # [ 126.629129] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:08Z level=info event=scheduler_restarting5685worker1 # [ 126.629547] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:08Z level=info event=scheduler_restarting addr=lb:500515686worker1 # [ 126.629676] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:539165687worker1 # [ 126.629788] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:08Z level=info event=worker_gone addr=192.168.1.3:500515688worker1 # [ 126.629919] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:539165689worker1 # [ 126.630030] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:08Z level=info event=worker_gone addr=192.168.1.4:500515690worker1 # [ 126.677298] nix-daemon[851]: reaped child process 1377, status = succeeded5691worker2 # [ 126.883451] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:08Z level=info event=scheduler_connected addr=lb:500515692worker2 # [ 126.888727] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5693worker1 # [ 126.843121] nix-grpc-daemon[1317]: ts=2026-09-22T16:44:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5694worker2 # [ 127.089866] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:08Z level=info event=scheduler_connected addr=lb:500515695worker2 # [ 127.095097] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5696worker1 # [ 127.141398] nix-grpc-daemon[1317]: E0000 00:00:1790095448.764643 1362 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5697worker1 # [ 127.148194] nix-grpc-daemon[1317]: E0000 00:00:1790095448.764848 1362 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5698worker2 # [ 127.296072] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:08Z level=info event=scheduler_connected addr=lb:500515699worker2 # [ 127.301243] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5700worker2 # [ 127.502256] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:09Z level=info event=scheduler_connected addr=lb:500515701worker2 # [ 127.507122] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5702worker2 # [ 127.709566] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:09Z level=info event=scheduler_connected addr=lb:500515703worker2 # [ 127.736138] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:09Z level=info event=oidc_jwks provider=mock keys=15704worker2 # [ 127.741738] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:442325705worker2 # [ 127.746983] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05706worker1 # [ 127.916796] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5707worker1 # [ 127.924262] systemd[1]: Stopped Nix worker-protocol over gRPC.5708worker1 # [ 127.926862] systemd[1]: nix-grpc-daemon.service: Consumed 1.276s CPU time over 13.787s wall clock time, 37.8M memory peak, 36.7K incoming IP traffic, 59.2K outgoing IP traffic.5709worker1 # [ 127.943242] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5710worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5711worker1 # [ 127.956133] systemd[1]: Closed Nix worker-protocol over gRPC.5712worker1 # [ 127.957812] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 13.831s wall clock time, 256K memory peak, 34.8K incoming IP traffic, 25.3K outgoing IP traffic.5713worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5714worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5715worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5716client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25717worker2 # [ 128.367705] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500645718client # don't know how to build these paths:5719client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5720worker2 # [ 128.377834] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500645721worker2 # [ 128.405249] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:10Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05722worker2 # [ 128.412450] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:10Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15723worker2 # [ 128.449924] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500645724worker2 # [ 128.466349] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500645725client # copying 1 paths to 'lb:50051'...5726client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5727worker2 # [ 128.482683] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500645728worker2 # [ 128.498580] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5729worker2 # [ 128.649969] nix-daemon[838]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5730worker2 # [ 128.670224] nix-daemon[838]: reaped child process 1259, status = succeeded5731lb # [ 128.642529] niks3-server[1133]: 2026/09/22 16:44:10 INFO Received uploads request method=POST path=/api/pending_closures5732worker2 # [ 128.687673] nix-grpc-daemon[1220]: time=2026-09-22T16:44:10.322Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5733worker2 # [ 128.690518] nix-grpc-daemon[1220]: time=2026-09-22T16:44:10.322Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5734lb # [ 128.684874] niks3-server[1133]: 2026/09/22 16:44:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5735worker2 # [ 128.727494] nix-grpc-daemon[1220]: time=2026-09-22T16:44:10.361Z level=INFO msg="Uploading 1 narinfos"5736lb # [ 128.696471] niks3-server[1133]: 2026/09/22 16:44:10 INFO Signed narinfos id=43 count=15737lb # [ 128.704753] niks3-server[1133]: 2026/09/22 16:44:10 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5738lb # [ 128.714530] niks3-server[1133]: 2026/09/22 16:44:10 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5739lb # [ 128.726866] niks3-server[1133]: 2026/09/22 16:44:10 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5740lb # [ 128.735591] niks3-server[1133]: 2026/09/22 16:44:10 INFO Completed upload id=435741worker2 # [ 128.768000] nix-grpc-daemon[1220]: time=2026-09-22T16:44:10.403Z level=INFO msg="Upload complete. (252ms)"5742lb # [ 128.742585] niks3-server[1133]: 2026/09/22 16:44:10 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5743worker2 # [ 128.782195] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50064 duration_s=0 paths=1 nar_bytes_in=5205744worker2 # [ 128.796540] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500645745worker2 # [ 128.808643] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5746worker2 # [ 128.811866] nix-daemon[838]: reaped child process 1254, status = succeeded5747client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5748worker2 # [ 128.823625] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5749client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5750lb # [ 129.220969] envoy[899]: conn peer=[2001:db8:1::1]:40030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9779 tx=4454 ms=77425751lb # [ 129.226426] envoy[899]: conn peer=[2001:db8:1::3]:40956 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1605 tx=620 ms=124775752client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5753client # farm-a> LOG-farm-a-on-standby5754worker2 # [ 130.716475] nix-daemon[838]: reaped child process 1263, status = succeeded5755worker2 # [ 130.813664] nix-daemon[838]: accepted connection from pid 1279, user nix-grpc-daemon (trusted)5756worker2 # [ 130.829886] nix-daemon[838]: reaped child process 1282, status = succeeded5757lb # [ 130.805857] niks3-server[1133]: 2026/09/22 16:44:12 INFO Received uploads request method=POST path=/api/pending_closures5758worker2 # [ 130.848474] nix-grpc-daemon[1220]: time=2026-09-22T16:44:12.482Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5759worker2 # [ 130.850296] nix-grpc-daemon[1220]: time=2026-09-22T16:44:12.483Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5760lb # [ 130.845896] niks3-server[1133]: 2026/09/22 16:44:12 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5761lb # [ 130.861915] niks3-server[1133]: 2026/09/22 16:44:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5762lb # [ 130.876324] niks3-server[1133]: 2026/09/22 16:44:12 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5763lb # [ 130.883990] niks3-server[1133]: 2026/09/22 16:44:12 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5764worker2 # [ 130.917572] nix-grpc-daemon[1220]: time=2026-09-22T16:44:12.553Z level=INFO msg="Uploading 1 narinfos"5765lb # [ 130.890364] niks3-server[1133]: 2026/09/22 16:44:12 INFO Signed narinfos id=44 count=15766lb # [ 130.896116] niks3-server[1133]: 2026/09/22 16:44:12 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5767lb # [ 130.909249] niks3-server[1133]: 2026/09/22 16:44:12 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5768lb # [ 130.917771] niks3-server[1133]: 2026/09/22 16:44:12 INFO Completed upload id=445769worker2 # [ 130.951721] nix-grpc-daemon[1220]: time=2026-09-22T16:44:12.587Z level=INFO msg="Upload complete. (314ms)"5770worker2 # [ 130.973433] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50064 duration_s=2 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15771worker2 # [ 130.992419] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:12Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05772worker2 # [ 130.996944] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:12Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15773worker2 # [ 131.001256] nix-daemon[838]: reaped child process 1261, status = succeeded5774worker2 # [ 131.018122] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500645775worker2 # [ 131.033146] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500645776worker2 # [ 131.047984] nix-grpc-daemon[1220]: time=2026-09-22T16:44:12.683Z level=INFO msg="All 1 paths already cached"5777client # copying 1 paths to 'lb:50051'...5778client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5779worker2 # [ 131.058684] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500645780worker2 # [ 131.073646] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5781worker2 # [ 131.241258] nix-daemon[838]: accepted connection from pid 1288, user nix-grpc-daemon (trusted)5782worker2 # [ 131.262229] nix-daemon[838]: reaped child process 1291, status = succeeded5783lb # [ 131.231596] niks3-server[1133]: 2026/09/22 16:44:12 INFO Received uploads request method=POST path=/api/pending_closures5784worker2 # [ 131.276058] nix-grpc-daemon[1220]: time=2026-09-22T16:44:12.911Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5785worker2 # [ 131.277573] nix-grpc-daemon[1220]: time=2026-09-22T16:44:12.912Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5786lb # [ 131.274524] niks3-server[1133]: 2026/09/22 16:44:12 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5787worker2 # [ 131.314561] nix-grpc-daemon[1220]: time=2026-09-22T16:44:12.950Z level=INFO msg="Uploading 1 narinfos"5788lb # [ 131.286583] niks3-server[1133]: 2026/09/22 16:44:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5789lb # [ 131.299893] niks3-server[1133]: 2026/09/22 16:44:12 INFO Signed narinfos id=45 count=15790worker2 # [ 131.338283] nix-grpc-daemon[1220]: time=2026-09-22T16:44:12.974Z level=INFO msg="Upload complete. (248ms)"5791lb # [ 131.304763] niks3-server[1133]: 2026/09/22 16:44:12 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5792lb # [ 131.311499] niks3-server[1133]: 2026/09/22 16:44:12 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5793lb # [ 131.315756] niks3-server[1133]: 2026/09/22 16:44:12 INFO Completed upload id=455794lb # [ 131.318522] niks3-server[1133]: 2026/09/22 16:44:12 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5795worker2 # [ 131.348334] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50064 duration_s=0 paths=1 nar_bytes_in=6405796worker2 # [ 131.363961] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500645797worker2 # [ 131.370676] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5798worker2 # [ 131.375922] nix-daemon[838]: reaped child process 1286, status = succeeded5799client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5800worker2 # [ 131.389361] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5801client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5802client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5803client # farm-b> LOG-farm-b-on-standby5804worker2 # [ 132.472130] nix-daemon[838]: reaped child process 1295, status = succeeded5805worker2 # [ 132.578198] nix-daemon[838]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5806worker2 # [ 132.596807] nix-daemon[838]: reaped child process 1314, status = succeeded5807lb # [ 132.570282] niks3-server[1133]: 2026/09/22 16:44:14 INFO Received uploads request method=POST path=/api/pending_closures5808worker2 # [ 132.612393] nix-grpc-daemon[1220]: time=2026-09-22T16:44:14.247Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5809worker2 # [ 132.614031] nix-grpc-daemon[1220]: time=2026-09-22T16:44:14.247Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5810lb # [ 132.614104] niks3-server[1133]: 2026/09/22 16:44:14 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5811lb # [ 132.626563] niks3-server[1133]: 2026/09/22 16:44:14 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5812worker2 # [ 132.666522] nix-grpc-daemon[1220]: time=2026-09-22T16:44:14.302Z level=INFO msg="Uploading 1 narinfos"5813lb # [ 132.637486] niks3-server[1133]: 2026/09/22 16:44:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5814lb # [ 132.647613] niks3-server[1133]: 2026/09/22 16:44:14 INFO Signed narinfos id=46 count=15815lb # [ 132.653985] niks3-server[1133]: 2026/09/22 16:44:14 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5816worker2 # [ 132.687415] nix-grpc-daemon[1220]: time=2026-09-22T16:44:14.323Z level=INFO msg="Upload complete. (287ms)"5817lb # [ 132.659162] niks3-server[1133]: 2026/09/22 16:44:14 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5818lb # [ 132.664820] niks3-server[1133]: 2026/09/22 16:44:14 INFO Completed upload id=465819lb # [ 132.667749] niks3-server[1133]: 2026/09/22 16:44:14 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5820worker2 # [ 132.700800] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50064 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15821worker2 # [ 132.725418] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:14Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05822worker2 # [ 132.730082] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:14Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15823worker2 # [ 132.734934] nix-daemon[838]: reaped child process 1293, status = succeeded5824worker2 # [ 132.765997] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500645825worker2 # [ 132.781478] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500645826worker2 # [ 132.796603] nix-grpc-daemon[1220]: time=2026-09-22T16:44:14.432Z level=INFO msg="All 2 paths already cached"5827client # copying 1 paths to 'lb:50051'...5828client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5829worker2 # [ 132.808118] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500645830worker2 # [ 132.822069] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5831worker2 # [ 132.976165] nix-daemon[838]: accepted connection from pid 1320, user nix-grpc-daemon (trusted)5832worker2 # [ 132.995094] nix-daemon[838]: reaped child process 1323, status = succeeded5833lb # [ 132.965439] niks3-server[1133]: 2026/09/22 16:44:14 INFO Received uploads request method=POST path=/api/pending_closures5834worker2 # [ 133.009372] nix-grpc-daemon[1220]: time=2026-09-22T16:44:14.644Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5835worker2 # [ 133.011453] nix-grpc-daemon[1220]: time=2026-09-22T16:44:14.644Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5836lb # [ 132.997333] niks3-server[1133]: 2026/09/22 16:44:14 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5837worker2 # [ 133.043736] nix-grpc-daemon[1220]: time=2026-09-22T16:44:14.679Z level=INFO msg="Uploading 1 narinfos"5838lb # [ 133.012257] niks3-server[1133]: 2026/09/22 16:44:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5839lb # [ 133.021705] niks3-server[1133]: 2026/09/22 16:44:14 INFO Signed narinfos id=47 count=15840worker2 # [ 133.066300] nix-grpc-daemon[1220]: time=2026-09-22T16:44:14.702Z level=INFO msg="Upload complete. (227ms)"5841lb # [ 133.033426] niks3-server[1133]: 2026/09/22 16:44:14 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5842lb # [ 133.038626] niks3-server[1133]: 2026/09/22 16:44:14 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5843worker2 # [ 133.076300] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50064 duration_s=0 paths=1 nar_bytes_in=7685844lb # [ 133.046872] niks3-server[1133]: 2026/09/22 16:44:14 INFO Completed upload id=475845lb # [ 133.049407] niks3-server[1133]: 2026/09/22 16:44:14 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5846worker2 # [ 133.091449] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500645847worker2 # [ 133.098138] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5848worker2 # [ 133.103258] nix-daemon[838]: reaped child process 1318, status = succeeded5849client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5850worker2 # [ 133.116952] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5851client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5852client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5853client # farm-top> LOG-farm-top-on-standby5854worker2 # [ 134.393039] nix-daemon[838]: reaped child process 1327, status = succeeded5855worker2 # [ 134.502382] nix-daemon[838]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5856worker2 # [ 134.521849] nix-daemon[838]: reaped child process 1346, status = succeeded5857lb # [ 134.494668] niks3-server[1133]: 2026/09/22 16:44:16 INFO Received uploads request method=POST path=/api/pending_closures5858worker2 # [ 134.538729] nix-grpc-daemon[1220]: time=2026-09-22T16:44:16.173Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5859worker2 # [ 134.541595] nix-grpc-daemon[1220]: time=2026-09-22T16:44:16.173Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5860lb # [ 134.535607] niks3-server[1133]: 2026/09/22 16:44:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5861worker2 # [ 134.579736] nix-grpc-daemon[1220]: time=2026-09-22T16:44:16.215Z level=INFO msg="Uploading 1 narinfos"5862lb # [ 134.551678] niks3-server[1133]: 2026/09/22 16:44:16 INFO Signed narinfos id=48 count=15863lb # [ 134.562713] niks3-server[1133]: 2026/09/22 16:44:16 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5864lb # [ 134.577553] niks3-server[1133]: 2026/09/22 16:44:16 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5865lb # [ 134.588650] niks3-server[1133]: 2026/09/22 16:44:16 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5866worker2 # [ 134.619653] nix-grpc-daemon[1220]: time=2026-09-22T16:44:16.255Z level=INFO msg="Upload complete. (292ms)"5867lb # [ 134.595298] niks3-server[1133]: 2026/09/22 16:44:16 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5868lb # [ 134.600514] niks3-server[1133]: 2026/09/22 16:44:16 INFO Completed upload id=485869worker2 # [ 134.633509] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50064 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15870lb # [ 134.604553] niks3-server[1133]: 2026/09/22 16:44:16 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5871worker2 # [ 134.654258] nix-daemon[838]: reaped child process 1325, status = succeeded5872client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.53 seconds)5873client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5874client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds)5875worker1 # [ 134.883192] systemd[1]: Listening on Nix worker-protocol over gRPC.5876worker1 # [ 134.892178] systemd[1]: Starting Nix worker-protocol over gRPC...5877worker1 # [ 135.041990] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5878worker1 # [ 135.044478] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5879worker1 # [ 135.048330] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15880worker1 # [ 135.054848] nix-grpc-daemon[1510]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5881worker1 # [ 135.072788] nix-grpc-daemon[1510]: E0000 00:00:1790095456.676296 1510 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5882lb: must succeed: curl -sf localhost:9901/clusters5883worker1 # [ 135.105990] nix-grpc-daemon[1510]: E0000 00:00:1790095456.678252 1510 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5884worker1 # [ 135.109803] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5885worker1 # [ 135.111952] nix-grpc-daemon[1510]: E0000 00:00:1790095456.691816 1550 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5886worker1 # [ 135.114828] nix-grpc-daemon[1510]: E0000 00:00:1790095456.714324 1550 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5887worker1 # [ 135.117267] nix-grpc-daemon[1510]: E0000 00:00:1790095456.714465 1550 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5888worker1 # [ 135.119812] nix-grpc-daemon[1510]: E0000 00:00:1790095456.732565 1550 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5889worker1 # [ 135.125351] systemd[1]: Started Nix worker-protocol over gRPC.5890worker1 # [ 135.128034] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:16Z level=info event=scheduler_connected addr=lb:500515891worker2 # [ 135.189335] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:442325892worker2 # [ 135.195770] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05893lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5894worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5895worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5896lb: must succeed: curl -sf localhost:9901/clusters5897lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5898lb: must succeed: curl -sf localhost:9901/clusters5899lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5900worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5901worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5902(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.20 seconds)5903subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5904client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5905client # [ 135.542940] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5906client # Running as unit: rs.service; invocation ID: 6cf947bc1e1a4110851cb263862dff865907client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5908worker1 # [ 135.693085] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457985909worker1 # [ 135.707948] nix-daemon[851]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5910client # [ 135.795045] nix[1409]: don't know how to build these paths:5911client # [ 135.797624] nix[1409]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5912worker2 # [ 135.800208] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:17Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05913worker2 # [ 135.805754] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:17Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15914worker2 # [ 135.848139] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500645915worker2 # [ 135.865459] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500645916client # [ 135.892421] nix[1409]: copying 1 paths to 'lb:50051'...5917worker2 # [ 135.885605] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500645918client # [ 135.894757] nix[1409]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5919worker2 # [ 135.903370] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5920worker2 # [ 136.072551] nix-daemon[838]: accepted connection from pid 1359, user nix-grpc-daemon (trusted)5921worker2 # [ 136.093377] nix-daemon[838]: reaped child process 1362, status = succeeded5922lb # [ 136.066079] niks3-server[1133]: 2026/09/22 16:44:17 INFO Received uploads request method=POST path=/api/pending_closures5923worker2 # [ 136.107341] nix-grpc-daemon[1220]: time=2026-09-22T16:44:17.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5924worker2 # [ 136.109731] nix-grpc-daemon[1220]: time=2026-09-22T16:44:17.745Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5925worker2 # [ 136.142754] nix-grpc-daemon[1220]: time=2026-09-22T16:44:17.778Z level=INFO msg="Uploading 1 narinfos"5926lb # [ 136.099998] niks3-server[1133]: 2026/09/22 16:44:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5927lb # [ 136.113209] niks3-server[1133]: 2026/09/22 16:44:17 INFO Signed narinfos id=49 count=15928lb # [ 136.121922] niks3-server[1133]: 2026/09/22 16:44:17 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5929lb # [ 136.137467] niks3-server[1133]: 2026/09/22 16:44:17 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5930lb # [ 136.152716] niks3-server[1133]: 2026/09/22 16:44:17 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5931worker2 # [ 136.192278] nix-grpc-daemon[1220]: time=2026-09-22T16:44:17.828Z level=INFO msg="Upload complete. (272ms)"5932lb # [ 136.163712] niks3-server[1133]: 2026/09/22 16:44:17 INFO Completed upload id=495933worker2 # [ 136.204681] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50064 duration_s=0 paths=1 nar_bytes_in=4165934lb # [ 136.176289] niks3-server[1133]: 2026/09/22 16:44:17 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5935worker2 # [ 136.216570] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500645936worker2 # [ 136.232660] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5937worker2 # [ 136.244217] nix-daemon[838]: reaped child process 1357, status = succeeded5938client # [ 136.258239] nix[1409]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5939worker2 # [ 136.255030] nix-daemon[838]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5940lb: must succeed: curl -sf localhost:9901/clusters5941lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5942lb: must succeed: curl -sf localhost:9901/clusters5943lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5944lb: must succeed: curl -sf localhost:9901/clusters5945lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5946lb: must succeed: curl -sf localhost:9901/clusters5947lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5948worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5949worker1 # [ 138.288792] systemd-journald[270]: Received client request to rotate journal, rotating.5950worker1 # [ 138.315221] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/8ce2175fab9b4bbca526a273d7aada41.5951worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5952worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5953worker2 # [ 138.422023] systemd[1]: Stopping Nix worker-protocol over gRPC...5954worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5955client: must succeed: sleep 5.05956worker2 # [ 138.494539] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:20Z level=info event=draining5957lb # [ 139.219437] envoy[899]: conn peer=[2001:db8:1::1]:51434 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9606 tx=4393 ms=62815958client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5959worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5960worker2 # [ 143.899644] nix-daemon[838]: reaped child process 1366, status = succeeded5961worker2 # [ 144.027473] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5962worker2 # [ 144.044861] nix-daemon[838]: reaped child process 1410, status = succeeded5963lb # [ 144.018435] niks3-server[1133]: 2026/09/22 16:44:25 INFO Received uploads request method=POST path=/api/pending_closures5964worker2 # [ 144.064110] nix-grpc-daemon[1220]: time=2026-09-22T16:44:25.698Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5965worker2 # [ 144.065963] nix-grpc-daemon[1220]: time=2026-09-22T16:44:25.698Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5966lb # [ 144.063357] niks3-server[1133]: 2026/09/22 16:44:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5967worker2 # [ 144.105778] nix-grpc-daemon[1220]: time=2026-09-22T16:44:25.741Z level=INFO msg="Uploading 1 narinfos"5968lb # [ 144.078023] niks3-server[1133]: 2026/09/22 16:44:25 INFO Signed narinfos id=50 count=15969lb # [ 144.084879] niks3-server[1133]: 2026/09/22 16:44:25 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5970lb # [ 144.092805] niks3-server[1133]: 2026/09/22 16:44:25 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5971lb # [ 144.101341] niks3-server[1133]: 2026/09/22 16:44:25 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5972lb # [ 144.112444] niks3-server[1133]: 2026/09/22 16:44:25 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5973worker2 # [ 144.150886] nix-grpc-daemon[1220]: time=2026-09-22T16:44:25.786Z level=INFO msg="Upload complete. (278ms)"5974lb # [ 144.118477] niks3-server[1133]: 2026/09/22 16:44:25 INFO Completed upload id=505975lb # [ 144.123135] niks3-server[1133]: 2026/09/22 16:44:25 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5976worker2 # [ 144.167027] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50064 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15977worker2 # [ 144.186785] nix-daemon[838]: reaped child process 1364, status = succeeded5978client # [ 144.211511] systemd[1]: rs.service: Deactivated successfully.5979client # [ 144.216201] systemd[1]: rs.service: Consumed 203ms CPU time over 8.669s wall clock time, 41.2M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.5980worker2 # [ 144.496850] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:26Z level=info event=signal_exit5981lb # [ 144.644630] niks3-server[1133]: 2026/09/22 16:44:26 INFO lead: released remote=192.168.1.4:383305982lb # [ 145.106512] niks3-server[1133]: 2026/09/22 16:44:26 INFO lead: acquired remote=192.168.1.3:441045983worker1 # [ 145.079647] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:26Z level=info event=scheduler_take_over5984worker1 # [ 145.617490] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:27Z level=info event=scheduler_restarting addr=lb:500515985worker2 # [ 145.684601] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:27Z level=info event=scheduler_restarting5986worker2 # [ 145.688963] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:27Z level=info event=scheduler_restarting addr=lb:500515987worker2 # [ 145.689131] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:442325988worker2 # [ 145.689276] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:27Z level=info event=worker_gone addr=192.168.1.4:500515989worker2 # [ 145.689405] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:442325990worker2 # [ 145.689515] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:27Z level=info event=worker_gone addr=192.168.1.3:500515991worker2 # [ 145.712738] nix-daemon[838]: reaped child process 1212, status = succeeded5992worker1 # [ 145.824369] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:27Z level=info event=scheduler_connected addr=lb:500515993worker1 # [ 145.835914] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5994worker2 # [ 145.899881] nix-grpc-daemon[1160]: ts=2026-09-22T16:44:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5995worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.21 seconds)5996client: waiting for success: ! systemctl is-active rs5997client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5998client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5999client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)6000client: must fail: journalctl -u rs | grep -q 'reconnecting'6001client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6002lb: must succeed: curl -sf localhost:9901/clusters6003worker1 # [ 146.034120] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:27Z level=info event=scheduler_connected addr=lb:500516004worker1 # [ 146.039489] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6005lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6006worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6007worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6008worker1 # [ 146.241755] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:27Z level=info event=scheduler_connected addr=lb:500516009worker1 # [ 146.250430] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:572526010worker1 # [ 146.255991] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06011worker2 # [ 146.549433] nix-grpc-daemon[1160]: E0000 00:00:1790095468.185156 1205 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6012worker2 # [ 146.578886] nix-grpc-daemon[1160]: E0000 00:00:1790095468.214712 1205 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6013lb: must succeed: curl -sf localhost:9901/clusters6014lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6015worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6016worker2 # [ 147.337332] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6017worker2 # [ 147.343755] systemd[1]: Stopped Nix worker-protocol over gRPC.6018worker2 # [ 147.348862] systemd[1]: nix-grpc-daemon.service: Consumed 1.838s CPU time over 38.015s wall clock time, 39.6M memory peak, 60K incoming IP traffic, 131.5K outgoing IP traffic.6019worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6020worker2 # [ 147.369097] systemd[1]: Starting scheduler-token.service...6021worker2 # [ 147.447149] systemd[1]: scheduler-token.service: Deactivated successfully.6022worker2 # [ 147.451345] systemd[1]: Finished scheduler-token.service.6023worker2 # [ 147.457096] systemd[1]: scheduler-token.service: Consumed 47ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6024worker2 # [ 147.463561] systemd[1]: Starting Nix worker-protocol over gRPC...6025worker2 # [ 147.599193] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6026worker2 # [ 147.601559] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6027worker2 # [ 147.605355] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16028worker2 # [ 147.610640] nix-grpc-daemon[1420]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6029worker2 # [ 147.636134] nix-grpc-daemon[1420]: E0000 00:00:1790095469.245815 1420 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6030worker2 # [ 147.638532] nix-grpc-daemon[1420]: E0000 00:00:1790095469.249367 1420 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6031worker2 # [ 147.655381] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:29Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6032worker2 # [ 147.657697] nix-grpc-daemon[1420]: E0000 00:00:1790095469.255426 1462 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6033worker2 # [ 147.660258] nix-grpc-daemon[1420]: E0000 00:00:1790095469.270850 1462 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6034worker2 # [ 147.665793] nix-grpc-daemon[1420]: E0000 00:00:1790095469.271006 1462 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6035worker2 # [ 147.670186] systemd[1]: Started Nix worker-protocol over gRPC.6036worker2 # [ 147.671124] nix-grpc-daemon[1420]: E0000 00:00:1790095469.290559 1462 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6037worker2 # [ 147.673791] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:29Z level=info event=scheduler_connected addr=lb:500516038worker1 # [ 147.619868] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:29Z level=info event=oidc_jwks provider=mock keys=16039worker1 # [ 147.624466] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:572526040worker1 # [ 147.629536] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06041lb: must succeed: curl -sf localhost:9901/clusters6042lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6043worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6044worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6045(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.02 seconds)6046subtest: scheduler restart mid-build: build finishes, no double build6047client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid6048client # [ 148.560924] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.6049client # Running as unit: mid.service; invocation ID: 651b6833b38e4f5a989071ffa9c4fc7e6050client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)6051worker2 # [ 148.763380] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396046052worker2 # [ 148.777914] nix-daemon[838]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)6053client # [ 148.803425] nix[1445]: don't know how to build these paths:6054client # [ 148.805346] nix[1445]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6055worker1 # [ 148.745320] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:30Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06056worker1 # [ 148.751436] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:30Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16057worker1 # [ 148.798550] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457986058worker1 # [ 148.814493] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5)6059worker1 # [ 148.827745] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 2/5)6060worker1 # [ 148.857353] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 3/5)6061lb # [ 149.223580] envoy[899]: conn peer=[2001:db8:1::1]:50370 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1519 ms=84336062worker1 # [ 149.196428] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 148 ms (attempt 4/5)6063lb # [ 149.224113] envoy[899]: conn peer=[2001:db8:1::4]:45384 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=22549 tx=1481 ms=363516064worker1 # [ 149.347211] nix-grpc-daemon[1510]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6065worker1 # [ 149.352819] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6066worker1 # [ 149.371584] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457986067client # [ 149.452872] nix[1445]: copying 1 paths to 'lb:50051'...6068client # [ 149.457862] nix[1445]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...6069worker1 # [ 149.391283] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457986070worker1 # [ 149.406773] nix-daemon[851]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)6071worker1 # [ 149.438955] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:31Z level=info event=niks3_push_started pid=16356072worker1 # [ 149.598161] nix-grpc-daemon[1635]: time=2026-09-22T16:44:31.221Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6073worker1 # [ 149.713065] nix-daemon[851]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6074worker1 # [ 149.743656] nix-daemon[851]: reaped child process 1654, status = succeeded6075lb # [ 149.772635] niks3-server[1133]: 2026/09/22 16:44:31 INFO Received uploads request method=POST path=/api/pending_closures6076worker1 # [ 149.755509] nix-grpc-daemon[1635]: time=2026-09-22T16:44:31.377Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6077worker1 # [ 149.758777] nix-grpc-daemon[1635]: time=2026-09-22T16:44:31.377Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"6078lb # [ 149.815506] niks3-server[1133]: 2026/09/22 16:44:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6079lb # [ 149.834418] niks3-server[1133]: 2026/09/22 16:44:31 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst6080lb # [ 149.848399] niks3-server[1133]: 2026/09/22 16:44:31 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls6081worker1 # [ 149.828360] nix-grpc-daemon[1635]: time=2026-09-22T16:44:31.451Z level=INFO msg="Uploading 1 narinfos"6082lb # [ 149.855623] niks3-server[1133]: 2026/09/22 16:44:31 INFO Signed narinfos id=51 count=16083lb # [ 149.868694] niks3-server[1133]: 2026/09/22 16:44:31 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6084lb # [ 149.881406] niks3-server[1133]: 2026/09/22 16:44:31 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo6085lb # [ 149.898104] niks3-server[1133]: 2026/09/22 16:44:31 INFO Completed upload id=516086worker1 # [ 149.871665] nix-grpc-daemon[1635]: time=2026-09-22T16:44:31.494Z level=INFO msg="Upload complete. (269ms)"6087worker1 # [ 149.888195] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45798 duration_s=0 paths=1 nar_bytes_in=4246088worker1 # [ 149.898345] nix-grpc-daemon[1510]: ts=2026-09-22T16:44:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457986089worker1 # [ 149.912160] nix-daemon[851]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)6090worker1 # [ 149.919813] nix-daemon[851]: reaped child process 1633, status = succeeded6091worker1 # [ 149.932533] nix-daemon[851]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)6092client # [ 149.999902] nix[1445]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6093lb: must succeed: curl -sf localhost:9901/clusters6094lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6095lb: must succeed: curl -sf localhost:9901/clusters6096lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6097worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6098worker1 # [ 152.425747] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1510 (nix-grpc-daemon) on client request.6099worker1 # [ 152.427334] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6100lb # [ 152.467340] niks3-server[1133]: 2026/09/22 16:44:34 INFO lead: released remote=192.168.1.3:441046101worker1 # [ 152.455968] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6102worker1 # [ 152.457588] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6103worker1 # [ 152.458831] systemd[1]: nix-grpc-daemon.service: Consumed 518ms CPU time over 17.559s wall clock time, 34.5M memory peak, 15.7K incoming IP traffic, 26.4K outgoing IP traffic.6104client # [ 152.524444] nix[1445]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6105worker2 # [ 152.516688] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:34Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6106client # [ 152.524743] nix[1445]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6107worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)6108client: must succeed: sleep 5.06109worker1 # [ 152.491193] nix-daemon[851]: reaped child process 1657, status = succeeded6110worker1 # [ 152.492883] nix-daemon[851]: reaped child process 1569, status = succeeded6111worker1 # [ 152.501148] nix-daemon[1659]: unexpected Nix daemon error: error: interrupted by the user6112worker1 # [ 152.506112] nix-daemon[851]: reaped child process 1659, status = failed with exit code 16113lb # [ 152.588138] niks3-server[1133]: 2026/09/22 16:44:34 INFO lead: acquired remote=192.168.1.4:412726114worker2 # [ 152.622187] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:34Z level=info event=scheduler_take_over6115worker1 # [ 152.581719] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6116worker1 # [ 152.588906] systemd[1]: Starting Nix worker-protocol over gRPC...6117worker2 # [ 152.717227] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:34Z level=info event=scheduler_connected addr=lb:500516118worker1 # [ 152.721571] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6119worker1 # [ 152.723832] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6120worker1 # [ 152.727753] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16121worker1 # [ 152.733285] nix-grpc-daemon[1696]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6122worker1 # [ 152.734789] nix-grpc-daemon[1696]: E0000 00:00:1790095474.356024 1696 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6123worker2 # [ 152.813079] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6124worker1 # [ 152.772139] nix-grpc-daemon[1696]: E0000 00:00:1790095474.362102 1696 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6125worker1 # [ 152.774495] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6126worker1 # [ 152.783619] nix-grpc-daemon[1696]: E0000 00:00:1790095474.370376 1735 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6127worker1 # [ 152.786734] nix-grpc-daemon[1696]: E0000 00:00:1790095474.387217 1735 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6128worker1 # [ 152.789421] nix-grpc-daemon[1696]: E0000 00:00:1790095474.387471 1735 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6129worker1 # [ 152.794737] systemd[1]: Started Nix worker-protocol over gRPC.6130worker1 # [ 152.797812] nix-grpc-daemon[1696]: E0000 00:00:1790095474.406436 1735 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6131worker1 # [ 152.800772] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=scheduler_connected addr=lb:500516132worker1 # [ 152.802096] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6133worker2 # [ 153.014518] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:34Z level=info event=scheduler_connected addr=lb:500516134client # [ 153.028401] nix[1445]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting6135worker2 # [ 153.019878] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6136worker1 # [ 153.001759] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=scheduler_connected addr=lb:500516137worker1 # [ 153.008484] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6138worker2 # [ 153.220831] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:34Z level=info event=scheduler_connected addr=lb:500516139worker2 # [ 153.226093] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6140worker1 # [ 153.209341] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=scheduler_connected addr=lb:500516141worker1 # [ 153.214521] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6142worker2 # [ 153.427016] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516143worker2 # [ 153.432189] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6144worker1 # [ 153.415594] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516145worker1 # [ 153.421170] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6146worker2 # [ 153.635109] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516147worker2 # [ 153.639336] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6148worker1 # [ 153.621985] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516149worker1 # [ 153.627630] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6150worker2 # [ 153.835862] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516151worker2 # [ 153.841144] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6152worker1 # [ 153.830578] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516153worker1 # [ 153.835103] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6154client # [ 154.032260] nix[1445]: scheduler at lb:50051: no healthy upstream, reconnecting6155worker2 # [ 154.042007] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516156worker2 # [ 154.047352] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6157worker1 # [ 154.031039] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516158worker1 # [ 154.036521] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6159worker2 # [ 154.248602] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516160worker2 # [ 154.253703] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6161worker1 # [ 154.237311] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_connected addr=lb:500516162worker1 # [ 154.242668] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6163worker2 # [ 154.454715] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:36Z level=info event=scheduler_connected addr=lb:500516164worker2 # [ 154.481905] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:36Z level=info event=oidc_jwks provider=mock keys=16165worker2 # [ 154.486383] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:332726166worker2 # [ 154.491679] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06167worker1 # [ 154.443536] nix-grpc-daemon[1696]: ts=2026-09-22T16:44:36Z level=info event=scheduler_connected addr=lb:500516168worker2 # [ 154.505954] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:332726169worker2 # [ 154.511068] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06170worker2 # [ 156.038679] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:37Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06171worker2 # [ 156.039040] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:37Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=16172worker2 # [ 156.082190] nix-grpc-daemon[1420]: ts=2026-09-22T16:44:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396046173worker2 # [ 156.098175] nix-grpc-daemon[1420]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 1/5)6174worker2 # [ 156.174564] nix-grpc-daemon[1420]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 110 ms (attempt 2/5)6175worker2 # [ 156.286466] nix-grpc-daemon[1420]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 256 ms (attempt 3/5)6176worker2 # [ 156.544508] nix-grpc-daemon[1420]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 497 ms (attempt 4/5)6177worker2 # [ 157.045399] nix-grpc-daemon[1420]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6178worker2 # [ 157.051481] nix-grpc-daemon[1420]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6179worker2 # [ 157.067429] nix-grpc-daemon[1420]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...6180worker2 # [ 157.095844] nix-daemon[838]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)6181client # [ 157.121352] nix[1445]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6182worker2 # [ 157.118279] nix-daemon[838]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)6183client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6184client: waiting for success: ! systemctl is-active mid6185lb # [ 159.222018] envoy[899]: conn peer=[2001:db8:1::3]:35812 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=936 tx=404 ms=173406186worker2 # [ 248.405998] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:10Z level=info event=niks3_push_started pid=15286187worker2 # [ 248.497713] nix-daemon[838]: reaped child process 1503, status = succeeded6188worker2 # [ 248.572520] nix-grpc-daemon[1528]: time=2026-09-22T16:46:10.208Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6189worker2 # [ 248.677445] nix-daemon[838]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6190worker2 # [ 248.696930] nix-daemon[838]: reaped child process 1544, status = succeeded6191lb # [ 248.668571] niks3-server[1133]: 2026/09/22 16:46:10 INFO Received uploads request method=POST path=/api/pending_closures6192worker2 # [ 248.714704] nix-grpc-daemon[1528]: time=2026-09-22T16:46:10.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6193worker2 # [ 248.718317] nix-grpc-daemon[1528]: time=2026-09-22T16:46:10.349Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6194lb # [ 248.736895] niks3-server[1133]: 2026/09/22 16:46:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6195worker2 # [ 248.776085] nix-grpc-daemon[1528]: time=2026-09-22T16:46:10.411Z level=INFO msg="Uploading 1 narinfos"6196lb # [ 248.744785] niks3-server[1133]: 2026/09/22 16:46:10 INFO Signed narinfos id=52 count=16197lb # [ 248.753476] niks3-server[1133]: 2026/09/22 16:46:10 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6198lb # [ 248.761652] niks3-server[1133]: 2026/09/22 16:46:10 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6199lb # [ 248.768851] niks3-server[1133]: 2026/09/22 16:46:10 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6200lb # [ 248.778033] niks3-server[1133]: 2026/09/22 16:46:10 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6201lb # [ 248.784368] niks3-server[1133]: 2026/09/22 16:46:10 INFO Completed upload id=526202lb # [ 248.787567] niks3-server[1133]: 2026/09/22 16:46:10 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6203worker2 # [ 248.815485] nix-grpc-daemon[1528]: time=2026-09-22T16:46:10.451Z level=INFO msg="Upload complete. (239ms)"6204worker2 # [ 248.832588] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39604 duration_s=92 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16205worker2 # [ 248.851809] nix-daemon[838]: reaped child process 1501, status = succeeded6206client # [ 248.874992] systemd[1]: mid.service: Deactivated successfully.6207client # [ 248.880550] systemd[1]: mid.service: Consumed 260ms CPU time over 1min 40.315s wall clock time, 43.5M memory peak, 4K written to disk, 9.1K incoming IP traffic, 11.2K outgoing IP traffic.6208client: (finished: waiting for success: ! systemctl is-active mid, in 91.13 seconds)6209client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6210client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)6211(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 100.56 seconds)6212subtest: interrupting the client stops the build on the worker6213client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6214client # [ 249.089766] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6215client # Running as unit: intr.service; invocation ID: 65b6e2882cd6499b9a5cf58047ec18916216client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6217worker2 # [ 249.262206] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396046218client # [ 249.277555] nix[1906]: don't know how to build these paths:6219client # [ 249.279397] nix[1906]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6220lb # [ 249.236983] envoy[899]: conn peer=[2001:db8:1::1]:39964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4288 tx=2089 ms=1000866221worker2 # [ 249.284171] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:10Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06222worker2 # [ 249.289982] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:10Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=16223worker1 # [ 249.283963] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626224worker1 # [ 249.298318] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6225worker1 # [ 249.324322] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 1/5)6226worker1 # [ 249.417106] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 103 ms (attempt 2/5)6227worker1 # [ 249.522011] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 96 ms (attempt 3/5)6228worker1 # [ 249.659945] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 575 ms (attempt 4/5)6229worker1 # [ 250.239151] nix-grpc-daemon[1696]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6230worker1 # [ 250.247791] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6231worker1 # [ 250.264348] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626232client # [ 250.341274] nix[1906]: copying 1 paths to 'lb:50051'...6233worker1 # [ 250.273625] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626234client # [ 250.346020] nix[1906]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6235worker1 # [ 250.287179] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6236worker1 # [ 250.315171] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:11Z level=info event=niks3_push_started pid=17726237worker1 # [ 250.446501] nix-grpc-daemon[1772]: time=2026-09-22T16:46:12.069Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6238worker1 # [ 250.558379] nix-daemon[851]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6239worker1 # [ 250.576823] nix-daemon[851]: reaped child process 1788, status = succeeded6240lb # [ 250.611627] niks3-server[1133]: 2026/09/22 16:46:12 INFO Received uploads request method=POST path=/api/pending_closures6241worker1 # [ 250.595493] nix-grpc-daemon[1772]: time=2026-09-22T16:46:12.217Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6242worker1 # [ 250.599522] nix-grpc-daemon[1772]: time=2026-09-22T16:46:12.217Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6243lb # [ 250.654807] niks3-server[1133]: 2026/09/22 16:46:12 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6244worker1 # [ 250.638824] nix-grpc-daemon[1772]: time=2026-09-22T16:46:12.261Z level=INFO msg="Uploading 1 narinfos"6245lb # [ 250.666222] niks3-server[1133]: 2026/09/22 16:46:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6246lb # [ 250.677038] niks3-server[1133]: 2026/09/22 16:46:12 INFO Signed narinfos id=53 count=16247lb # [ 250.679538] niks3-server[1133]: 2026/09/22 16:46:12 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6248lb # [ 250.693313] niks3-server[1133]: 2026/09/22 16:46:12 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6249lb # [ 250.697374] niks3-server[1133]: 2026/09/22 16:46:12 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6250lb # [ 250.702291] niks3-server[1133]: 2026/09/22 16:46:12 INFO Completed upload id=536251worker1 # [ 250.675592] nix-grpc-daemon[1772]: time=2026-09-22T16:46:12.299Z level=INFO msg="Upload complete. (225ms)"6252worker1 # [ 250.684354] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=4246253worker1 # [ 250.705097] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626254worker1 # [ 250.709347] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6255worker1 # [ 250.719827] nix-daemon[851]: reaped child process 1770, status = succeeded6256client # [ 250.792295] nix[1906]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6257worker1 # [ 250.727292] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6258client: must succeed: systemctl kill -s INT intr6259worker1 # [ 252.703628] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:14Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6260client # [ 252.769272] systemd[1]: intr.service: Sent signal SIGINT to main process 1906 (nix) on client request.6261client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)6262client # [ 252.779922] nix[1906]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6263client # [ 252.782844] nix[1906]: error: interrupted by the user6264client # [ 252.818717] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6265client # [ 252.829773] systemd[1]: intr.service: Failed with result 'exit-code'.6266client # [ 252.832294] systemd[1]: intr.service: Consumed 176ms CPU time over 3.738s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6267worker1 # [ 252.772588] nix-daemon[1792]: unexpected Nix daemon error: error: interrupted by the user6268worker1 # [ 252.781446] nix-daemon[851]: reaped child process 1790, status = succeeded6269worker1 # [ 252.783636] nix-daemon[851]: reaped child process 1792, status = failed with exit code 16270(finished: subtest: interrupting the client stops the build on the worker, in 3.87 seconds)6271subtest: a deploy mid-build drains: the build finishes, the new generation takes over6272client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6273client # [ 252.983741] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6274client # Running as unit: sw.service; invocation ID: ae200e3a6cec40ec8feca3bee8085b0a6275client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds)6276worker1 # [ 253.107846] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:431626277client # [ 253.183298] nix[1927]: don't know how to build these paths:6278client # [ 253.184319] nix[1927]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6279worker2 # [ 253.188972] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:14Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06280worker2 # [ 253.194225] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:14Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=16281worker2 # [ 253.234366] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396046282worker2 # [ 253.253989] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396046283client # [ 253.276205] nix[1927]: copying 1 paths to 'lb:50051'...6284worker2 # [ 253.269457] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396046285client # [ 253.278517] nix[1927]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6286worker2 # [ 253.286002] nix-daemon[838]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)6287worker2 # [ 253.454697] nix-daemon[838]: accepted connection from pid 1578, user nix-grpc-daemon (trusted)6288worker2 # [ 253.473375] nix-daemon[838]: reaped child process 1581, status = succeeded6289lb # [ 253.449936] niks3-server[1133]: 2026/09/22 16:46:15 INFO Received uploads request method=POST path=/api/pending_closures6290worker2 # [ 253.492053] nix-grpc-daemon[1528]: time=2026-09-22T16:46:15.126Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6291worker2 # [ 253.493756] nix-grpc-daemon[1528]: time=2026-09-22T16:46:15.126Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6292worker2 # [ 253.526537] nix-grpc-daemon[1528]: time=2026-09-22T16:46:15.161Z level=INFO msg="Uploading 1 narinfos"6293lb # [ 253.480173] niks3-server[1133]: 2026/09/22 16:46:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6294lb # [ 253.501362] niks3-server[1133]: 2026/09/22 16:46:15 INFO Signed narinfos id=54 count=16295lb # [ 253.512319] niks3-server[1133]: 2026/09/22 16:46:15 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6296lb # [ 253.523186] niks3-server[1133]: 2026/09/22 16:46:15 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6297worker2 # [ 253.562537] nix-grpc-daemon[1528]: time=2026-09-22T16:46:15.198Z level=INFO msg="Upload complete. (259ms)"6298lb # [ 253.534281] niks3-server[1133]: 2026/09/22 16:46:15 INFO Completed upload id=546299lb # [ 253.539441] niks3-server[1133]: 2026/09/22 16:46:15 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6300lb # [ 253.542351] niks3-server[1133]: 2026/09/22 16:46:15 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6301worker2 # [ 253.572397] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39604 duration_s=0 paths=1 nar_bytes_in=4166302worker2 # [ 253.592903] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396046303worker2 # [ 253.604767] nix-daemon[838]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)6304worker2 # [ 253.608447] nix-daemon[838]: reaped child process 1576, status = succeeded6305client # [ 253.627580] nix[1927]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6306worker2 # [ 253.624827] nix-daemon[838]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)6307worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26308worker2 # Checking switch inhibitors... done6309worker2 # [ 255.972695] nixos[1617]: switching to system configuration /nix/store/v5mnmfs27v4kkcfdrk24lffriqdnfny8-nixos-system-worker2-test6310worker2 # [ 255.974591] systemd[1]: Stopped target Local File Systems.6311worker2 # activating the configuration...6312worker2 # [ 255.980308] systemd[1]: Stopped target Remote File Systems.6313worker2 # setting up /etc...6314worker2 # [ 256.525830] systemd[1]: Reload requested from client PID 1617 ('.switch-to-conf') (unit backdoor.service)...6315worker2 # [ 256.527386] systemd[1]: Reloading...6316lb # [ 259.237274] envoy[899]: conn peer=[2001:db8:1::1]:51700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=849 ms=35246317worker2 # [ 261.472403] systemd[1]: Reloading finished in 4938 ms.6318worker2 # restarting sysinit-reactivation.target6319worker2 # [ 261.584989] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6320worker2 # [ 261.710129] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6321worker2 # [ 261.712749] systemd[1]: Reached target Reactivate sysinit units.6322worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6323worker2 # [ 261.718748] systemd[1]: Reloading D-Bus System Message Bus...6324worker2 # [ 261.719914] dbus-broker-launch[491]: Caught SIGHUP, trigger reload.6325worker2 # [ 261.731125] systemd[1]: Reloading Nix worker-protocol over gRPC...6326worker2 # [ 261.742314] dbus-broker-launch[491]: Looking up NSS user entry for 'systemd-timesync'...6327worker2 # [ 261.749134] dbus-broker-launch[491]: NSS returned no entry for 'systemd-timesync'6328worker2 # [ 261.750149] dbus-broker-launch[491]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6329worker2 # [ 261.767287] systemd[1]: Reloaded Nix worker-protocol over gRPC.6330worker2 # [ 261.782685] systemd[1]: Reloaded D-Bus System Message Bus.6331worker2 # [ 261.794436] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6332worker2 # [ 261.795749] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6333worker2 # [ 261.808211] systemd[1]: Reached target Remote File Systems.6334worker2 # [ 261.809033] systemd[1]: Starting linger-users.service...6335worker2 # [ 261.809781] systemd[1]: Reached target Local File Systems.6336worker2 # [ 261.810504] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6337worker2 # [ 261.826097] systemd[1]: Starting Create SUID/SGID Wrappers...6338worker2 # [ 261.827548] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6339worker2 # [ 261.832576] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676340worker2 # [ 261.834699] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6341worker2 # [ 261.848229] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6342worker2 # [ 261.852079] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6343worker2 # [ 261.874504] systemd[1]: linger-users.service: Deactivated successfully.6344worker2 # [ 261.877694] systemd[1]: Finished linger-users.service.6345worker2 # [ 262.164929] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6346worker2 # [ 262.172146] systemd[1]: Finished Create SUID/SGID Wrappers.6347worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6348worker2 # [ 262.423245] nixos[1617]: finished switching to system configuration /nix/store/v5mnmfs27v4kkcfdrk24lffriqdnfny8-nixos-system-worker2-test6349worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.89 seconds)6350worker2: must succeed: pgrep -f 'read -t [9]0 x'6351worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6352client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26353worker2 # [ 262.657416] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:24Z level=info event=draining6354client # don't know how to build these paths:6355client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6356worker2 # [ 262.716002] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396046357worker1 # [ 262.653635] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:431626358worker2 # [ 262.736093] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:24Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06359worker2 # [ 262.738010] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:24Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16360worker1 # [ 262.725333] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626361worker1 # [ 262.740907] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626362client # copying 1 paths to 'lb:50051'...6363worker1 # [ 262.757905] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626364client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6365worker1 # [ 262.782805] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6366worker1 # [ 262.952641] nix-daemon[851]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)6367worker1 # [ 262.971733] nix-daemon[851]: reaped child process 1842, status = succeeded6368lb # [ 263.003123] niks3-server[1133]: 2026/09/22 16:46:24 INFO Received uploads request method=POST path=/api/pending_closures6369worker1 # [ 262.990289] nix-grpc-daemon[1772]: time=2026-09-22T16:46:24.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6370worker1 # [ 262.992292] nix-grpc-daemon[1772]: time=2026-09-22T16:46:24.612Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6371lb # [ 263.042326] niks3-server[1133]: 2026/09/22 16:46:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6372worker1 # [ 263.025732] nix-grpc-daemon[1772]: time=2026-09-22T16:46:24.648Z level=INFO msg="Uploading 1 narinfos"6373lb # [ 263.056530] niks3-server[1133]: 2026/09/22 16:46:24 INFO Signed narinfos id=55 count=16374lb # [ 263.065375] niks3-server[1133]: 2026/09/22 16:46:24 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6375lb # [ 263.079701] niks3-server[1133]: 2026/09/22 16:46:24 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6376worker1 # [ 263.060071] nix-grpc-daemon[1772]: time=2026-09-22T16:46:24.683Z level=INFO msg="Upload complete. (260ms)"6377lb # [ 263.090376] niks3-server[1133]: 2026/09/22 16:46:24 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6378lb # [ 263.101832] niks3-server[1133]: 2026/09/22 16:46:24 INFO Completed upload id=556379worker1 # [ 263.073432] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=5286380lb # [ 263.105836] niks3-server[1133]: 2026/09/22 16:46:24 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6381worker1 # [ 263.088278] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626382worker1 # [ 263.096391] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6383worker1 # [ 263.102270] nix-daemon[851]: reaped child process 1837, status = succeeded6384client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6385client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6386worker1 # [ 263.118371] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6387client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6388client # farm-a> LOG-farm-a-during-drain6389worker1 # [ 264.188969] nix-daemon[851]: reaped child process 1846, status = succeeded6390worker1 # [ 264.259484] nix-daemon[851]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6391worker1 # [ 264.280838] nix-daemon[851]: reaped child process 1865, status = succeeded6392lb # [ 264.311315] niks3-server[1133]: 2026/09/22 16:46:25 INFO Received uploads request method=POST path=/api/pending_closures6393worker1 # [ 264.294331] nix-grpc-daemon[1772]: time=2026-09-22T16:46:25.916Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6394worker1 # [ 264.295842] nix-grpc-daemon[1772]: time=2026-09-22T16:46:25.916Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6395lb # [ 264.340209] niks3-server[1133]: 2026/09/22 16:46:25 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6396lb # [ 264.353709] niks3-server[1133]: 2026/09/22 16:46:26 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6397worker1 # [ 264.334603] nix-grpc-daemon[1772]: time=2026-09-22T16:46:25.957Z level=INFO msg="Uploading 1 narinfos"6398lb # [ 264.362500] niks3-server[1133]: 2026/09/22 16:46:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6399lb # [ 264.367835] niks3-server[1133]: 2026/09/22 16:46:26 INFO Signed narinfos id=56 count=16400lb # [ 264.375061] niks3-server[1133]: 2026/09/22 16:46:26 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6401lb # [ 264.380566] niks3-server[1133]: 2026/09/22 16:46:26 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6402worker1 # [ 264.353722] nix-grpc-daemon[1772]: time=2026-09-22T16:46:25.977Z level=INFO msg="Upload complete. (223ms)"6403lb # [ 264.382817] niks3-server[1133]: 2026/09/22 16:46:26 INFO Completed upload id=566404lb # [ 264.386427] niks3-server[1133]: 2026/09/22 16:46:26 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6405worker2 # [ 264.426377] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:26Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06406worker1 # [ 264.362008] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16407worker2 # [ 264.428215] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:26Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16408worker1 # [ 264.377831] nix-daemon[851]: reaped child process 1844, status = succeeded6409worker1 # [ 264.413479] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626410worker1 # [ 264.434204] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626411worker1 # [ 264.448628] nix-grpc-daemon[1772]: time=2026-09-22T16:46:26.071Z level=INFO msg="All 1 paths already cached"6412client # copying 1 paths to 'lb:50051'...6413worker1 # [ 264.459873] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626414client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6415worker1 # [ 264.482439] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6416worker1 # [ 264.639847] nix-daemon[851]: accepted connection from pid 1869, user nix-grpc-daemon (trusted)6417worker1 # [ 264.657783] nix-daemon[851]: reaped child process 1872, status = succeeded6418lb # [ 264.691567] niks3-server[1133]: 2026/09/22 16:46:26 INFO Received uploads request method=POST path=/api/pending_closures6419worker1 # [ 264.674055] nix-grpc-daemon[1772]: time=2026-09-22T16:46:26.297Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6420worker1 # [ 264.676412] nix-grpc-daemon[1772]: time=2026-09-22T16:46:26.299Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6421lb # [ 264.725927] niks3-server[1133]: 2026/09/22 16:46:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6422worker1 # [ 264.706033] nix-grpc-daemon[1772]: time=2026-09-22T16:46:26.329Z level=INFO msg="Uploading 1 narinfos"6423lb # [ 264.740320] niks3-server[1133]: 2026/09/22 16:46:26 INFO Signed narinfos id=57 count=16424lb # [ 264.743434] niks3-server[1133]: 2026/09/22 16:46:26 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6425lb # [ 264.758117] niks3-server[1133]: 2026/09/22 16:46:26 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6426worker1 # [ 264.736663] nix-grpc-daemon[1772]: time=2026-09-22T16:46:26.360Z level=INFO msg="Upload complete. (237ms)"6427lb # [ 264.768170] niks3-server[1133]: 2026/09/22 16:46:26 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6428lb # [ 264.774026] niks3-server[1133]: 2026/09/22 16:46:26 INFO Completed upload id=576429worker1 # [ 264.748305] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=6406430lb # [ 264.778343] niks3-server[1133]: 2026/09/22 16:46:26 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6431worker1 # [ 264.763436] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626432worker1 # [ 264.773601] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6433worker1 # [ 264.777057] nix-daemon[851]: reaped child process 1867, status = succeeded6434client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6435client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6436worker1 # [ 264.792787] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6437client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6438client # farm-b> LOG-farm-b-during-drain6439worker1 # [ 265.866307] nix-daemon[851]: reaped child process 1876, status = succeeded6440worker1 # [ 265.975411] nix-daemon[851]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6441worker1 # [ 265.998934] nix-daemon[851]: reaped child process 1895, status = succeeded6442lb # [ 266.025757] niks3-server[1133]: 2026/09/22 16:46:27 INFO Received uploads request method=POST path=/api/pending_closures6443worker1 # [ 266.008491] nix-grpc-daemon[1772]: time=2026-09-22T16:46:27.631Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6444worker1 # [ 266.010012] nix-grpc-daemon[1772]: time=2026-09-22T16:46:27.631Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6445lb # [ 266.067251] niks3-server[1133]: 2026/09/22 16:46:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6446worker1 # [ 266.047915] nix-grpc-daemon[1772]: time=2026-09-22T16:46:27.671Z level=INFO msg="Uploading 1 narinfos"6447lb # [ 266.085357] niks3-server[1133]: 2026/09/22 16:46:27 INFO Signed narinfos id=58 count=16448lb # [ 266.094021] niks3-server[1133]: 2026/09/22 16:46:27 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6449lb # [ 266.107121] niks3-server[1133]: 2026/09/22 16:46:27 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6450worker1 # [ 266.093376] nix-grpc-daemon[1772]: time=2026-09-22T16:46:27.716Z level=INFO msg="Upload complete. (289ms)"6451lb # [ 266.121357] niks3-server[1133]: 2026/09/22 16:46:27 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6452lb # [ 266.128209] niks3-server[1133]: 2026/09/22 16:46:27 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6453lb # [ 266.135424] niks3-server[1133]: 2026/09/22 16:46:27 INFO Completed upload id=586454worker2 # [ 266.174069] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:27Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06455worker2 # [ 266.176154] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:27Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16456lb # [ 266.141096] niks3-server[1133]: 2026/09/22 16:46:27 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6457worker1 # [ 266.105159] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16458worker1 # [ 266.127952] nix-daemon[851]: reaped child process 1874, status = succeeded6459worker1 # [ 266.161989] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626460worker1 # [ 266.179211] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626461worker1 # [ 266.194756] nix-grpc-daemon[1772]: time=2026-09-22T16:46:27.818Z level=INFO msg="All 2 paths already cached"6462client # copying 1 paths to 'lb:50051'...6463worker1 # [ 266.206568] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626464client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6465worker1 # [ 266.229671] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6466worker1 # [ 266.397455] nix-daemon[851]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6467worker1 # [ 266.419023] nix-daemon[851]: reaped child process 1902, status = succeeded6468lb # [ 266.447079] niks3-server[1133]: 2026/09/22 16:46:28 INFO Received uploads request method=POST path=/api/pending_closures6469worker1 # [ 266.429301] nix-grpc-daemon[1772]: time=2026-09-22T16:46:28.052Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6470worker1 # [ 266.430864] nix-grpc-daemon[1772]: time=2026-09-22T16:46:28.052Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6471lb # [ 266.479407] niks3-server[1133]: 2026/09/22 16:46:28 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6472worker1 # [ 266.464100] nix-grpc-daemon[1772]: time=2026-09-22T16:46:28.087Z level=INFO msg="Uploading 1 narinfos"6473lb # [ 266.491587] niks3-server[1133]: 2026/09/22 16:46:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6474lb # [ 266.504948] niks3-server[1133]: 2026/09/22 16:46:28 INFO Signed narinfos id=59 count=16475lb # [ 266.507412] niks3-server[1133]: 2026/09/22 16:46:28 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6476lb # [ 266.515521] niks3-server[1133]: 2026/09/22 16:46:28 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6477lb # [ 266.525143] niks3-server[1133]: 2026/09/22 16:46:28 INFO Completed upload id=596478worker1 # [ 266.498598] nix-grpc-daemon[1772]: time=2026-09-22T16:46:28.122Z level=INFO msg="Upload complete. (250ms)"6479lb # [ 266.530545] niks3-server[1133]: 2026/09/22 16:46:28 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6480worker1 # [ 266.508330] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=7686481worker1 # [ 266.528722] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626482worker1 # [ 266.535719] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6483worker1 # [ 266.541701] nix-daemon[851]: reaped child process 1897, status = succeeded6484client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6485client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6486worker1 # [ 266.555822] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6487client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6488client # farm-top> LOG-farm-top-during-drain6489worker1 # [ 267.433115] nix-daemon[851]: reaped child process 1906, status = succeeded6490worker1 # [ 267.540528] nix-daemon[851]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6491worker1 # [ 267.559743] nix-daemon[851]: reaped child process 1925, status = succeeded6492lb # [ 267.591331] niks3-server[1133]: 2026/09/22 16:46:29 INFO Received uploads request method=POST path=/api/pending_closures6493worker1 # [ 267.575553] nix-grpc-daemon[1772]: time=2026-09-22T16:46:29.197Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6494worker1 # [ 267.577503] nix-grpc-daemon[1772]: time=2026-09-22T16:46:29.197Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6495lb # [ 267.629395] niks3-server[1133]: 2026/09/22 16:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6496worker1 # [ 267.610484] nix-grpc-daemon[1772]: time=2026-09-22T16:46:29.233Z level=INFO msg="Uploading 1 narinfos"6497lb # [ 267.643195] niks3-server[1133]: 2026/09/22 16:46:29 INFO Signed narinfos id=60 count=16498lb # [ 267.651946] niks3-server[1133]: 2026/09/22 16:46:29 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6499lb # [ 267.659714] niks3-server[1133]: 2026/09/22 16:46:29 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6500lb # [ 267.670147] niks3-server[1133]: 2026/09/22 16:46:29 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6501lb # [ 267.674380] niks3-server[1133]: 2026/09/22 16:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6502worker1 # [ 267.651689] nix-grpc-daemon[1772]: time=2026-09-22T16:46:29.275Z level=INFO msg="Upload complete. (284ms)"6503lb # [ 267.680733] niks3-server[1133]: 2026/09/22 16:46:29 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6504lb # [ 267.686798] niks3-server[1133]: 2026/09/22 16:46:29 INFO Completed upload id=606505worker1 # [ 267.665108] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16506worker1 # [ 267.683963] nix-daemon[851]: reaped child process 1904, status = succeeded6507client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.28 seconds)6508client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6509client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6510worker2: must succeed: pkill -f 'read -t [9]0 x'6511worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6512client: waiting for success: ! systemctl is-active sw6513worker2 # [ 268.080571] nix-daemon[838]: reaped child process 1585, status = succeeded6514worker2 # [ 268.167953] nix-daemon[838]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6515worker2 # [ 268.188266] nix-daemon[838]: reaped child process 1767, status = succeeded6516lb # [ 268.159210] niks3-server[1133]: 2026/09/22 16:46:29 INFO Received uploads request method=POST path=/api/pending_closures6517worker2 # [ 268.202782] nix-grpc-daemon[1528]: time=2026-09-22T16:46:29.837Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6518worker2 # [ 268.204326] nix-grpc-daemon[1528]: time=2026-09-22T16:46:29.837Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6519lb # [ 268.195126] niks3-server[1133]: 2026/09/22 16:46:29 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6520lb # [ 268.209110] niks3-server[1133]: 2026/09/22 16:46:29 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6521worker2 # [ 268.252117] nix-grpc-daemon[1528]: time=2026-09-22T16:46:29.887Z level=INFO msg="Uploading 1 narinfos"6522lb # [ 268.220881] niks3-server[1133]: 2026/09/22 16:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6523lb # [ 268.229492] niks3-server[1133]: 2026/09/22 16:46:29 INFO Signed narinfos id=61 count=16524lb # [ 268.235488] niks3-server[1133]: 2026/09/22 16:46:29 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6525lb # [ 268.243742] niks3-server[1133]: 2026/09/22 16:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6526worker2 # [ 268.283166] nix-grpc-daemon[1528]: time=2026-09-22T16:46:29.919Z level=INFO msg="Upload complete. (228ms)"6527lb # [ 268.252978] niks3-server[1133]: 2026/09/22 16:46:29 INFO Completed upload id=616528lb # [ 268.256941] niks3-server[1133]: 2026/09/22 16:46:29 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6529worker2 # [ 268.297614] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39604 duration_s=14 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16530worker2 # [ 268.313823] nix-daemon[838]: reaped child process 1583, status = succeeded6531client # [ 268.340578] systemd[1]: sw.service: Deactivated successfully.6532client # [ 268.344751] systemd[1]: sw.service: Consumed 193ms CPU time over 15.359s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6533worker2 # [ 268.660109] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:30Z level=info event=signal_exit6534client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6535client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6536client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6537worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6538worker2 # [ 269.246854] systemd[1]: Starting scheduler-token.service...6539lb # [ 269.237049] envoy[899]: conn peer=[2001:db8:1::1]:33982 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9595 tx=4420 ms=50346540worker2 # [ 269.318081] systemd[1]: scheduler-token.service: Deactivated successfully.6541worker2 # [ 269.323085] systemd[1]: Finished scheduler-token.service.6542worker2 # [ 269.324916] systemd[1]: scheduler-token.service: Consumed 40ms CPU time over 75ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6543lb # [ 269.241952] envoy[899]: conn peer=[2001:db8:1::1]:60568 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1519 ms=151446544lb # [ 269.591504] niks3-server[1133]: 2026/09/22 16:46:31 INFO lead: released remote=192.168.1.4:412726545worker2 # [ 270.626099] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:32Z level=info event=scheduler_restarting6546worker1 # [ 270.567249] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:32Z level=info event=scheduler_restarting addr=lb:500516547worker2 # [ 270.638031] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:332726548worker2 # [ 270.643133] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:32Z level=info event=worker_gone addr=192.168.1.3:500516549worker2 # [ 270.647784] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:32Z level=info event=scheduler_restarting addr=lb:500516550worker2 # [ 270.652498] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:332726551worker2 # [ 270.657507] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:32Z level=info event=worker_gone addr=192.168.1.4:500516552worker2 # [ 270.680344] nix-daemon[838]: reaped child process 1476, status = succeeded6553worker1 # [ 270.766899] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:32Z level=info event=scheduler_connected addr=lb:500516554worker1 # [ 270.772352] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6555worker2 # [ 270.854459] nix-grpc-daemon[1420]: ts=2026-09-22T16:46:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6556worker1 # [ 270.972992] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:32Z level=info event=scheduler_connected addr=lb:500516557worker1 # [ 270.978257] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6558worker1 # [ 271.179127] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:32Z level=info event=scheduler_connected addr=lb:500516559worker1 # [ 271.184492] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6560worker1 # [ 271.385119] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_connected addr=lb:500516561worker1 # [ 271.390315] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6562worker2 # [ 271.536941] nix-grpc-daemon[1420]: E0000 00:00:1790095593.172609 1465 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6563worker2 # [ 271.537282] nix-grpc-daemon[1420]: E0000 00:00:1790095593.172824 1465 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6564worker1 # [ 271.592851] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_connected addr=lb:500516565worker1 # [ 271.597033] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6566worker2 # [ 271.762675] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6567worker2 # [ 271.771377] systemd[1]: nix-grpc-daemon.service: Consumed 1.011s CPU time over 2min 4.303s wall clock time, 37M memory peak, 28.7K incoming IP traffic, 43.6K outgoing IP traffic.6568worker2 # [ 271.792330] systemd[1]: Starting Nix worker-protocol over gRPC...6569worker1 # [ 271.793591] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_connected addr=lb:500516570worker1 # [ 271.798670] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6571worker2 # [ 271.940229] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6572worker2 # [ 271.943303] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6573lb # [ 271.924502] niks3-server[1133]: 2026/09/22 16:46:33 INFO lead: acquired remote=192.168.1.4:577286574worker2 # [ 271.968212] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16575worker2 # [ 271.970334] nix-grpc-daemon[1784]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6576worker2 # [ 271.971709] nix-grpc-daemon[1784]: E0000 00:00:1790095593.581526 1784 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6577worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.83 seconds)6578lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6579lb: must succeed: curl -sf localhost:9901/clusters6580worker2 # [ 271.996497] nix-grpc-daemon[1784]: E0000 00:00:1790095593.583509 1784 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6581worker2 # [ 271.999300] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6582worker2 # [ 272.001236] nix-grpc-daemon[1784]: E0000 00:00:1790095593.591453 1824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6583worker2 # [ 272.003527] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:33Z level=info event=scheduler_take_over6584worker2 # [ 272.004786] systemd[1]: Started Nix worker-protocol over gRPC.6585worker2 # [ 272.005606] nix-grpc-daemon[1784]: E0000 00:00:1790095593.614730 1824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6586worker2 # [ 272.007984] nix-grpc-daemon[1784]: E0000 00:00:1790095593.614934 1824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6587worker2 # [ 272.013587] nix-grpc-daemon[1784]: E0000 00:00:1790095593.631576 1824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6588worker2 # [ 272.017956] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:33Z level=info event=scheduler_connected addr=lb:500516589worker2 # [ 272.019675] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6590worker1 # [ 271.999473] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_connected addr=lb:500516591lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6592worker1 # [ 272.004770] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6593worker2 # [ 272.221058] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:33Z level=info event=scheduler_connected addr=lb:500516594worker2 # [ 272.226268] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6595worker1 # [ 272.205604] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_connected addr=lb:500516596worker1 # [ 272.210851] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6597worker2 # [ 272.427470] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=scheduler_connected addr=lb:500516598worker2 # [ 272.432817] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6599worker1 # [ 272.411517] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:34Z level=info event=scheduler_connected addr=lb:500516600worker1 # [ 272.416749] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6601worker2 # [ 272.633813] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=scheduler_connected addr=lb:500516602worker2 # [ 272.638968] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6603worker1 # [ 272.619204] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:34Z level=info event=scheduler_connected addr=lb:500516604worker1 # [ 272.623485] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6605worker2 # [ 272.839964] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=scheduler_connected addr=lb:500516606worker2 # [ 272.856075] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=oidc_jwks provider=mock keys=16607worker2 # [ 272.860509] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:427566608worker2 # [ 272.865947] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06609worker1 # [ 272.819779] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:34Z level=info event=scheduler_connected addr=lb:500516610worker2 # [ 272.882189] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:427566611worker2 # [ 272.887234] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06612lb: must succeed: curl -sf localhost:9901/clusters6613lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6614lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds)6615lb: must succeed: curl -sf localhost:9901/clusters6616lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6617worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6618worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6619(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 20.34 seconds)6620subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6621client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16622worker2 # [ 273.479372] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427706623worker2 # [ 273.494283] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)6624worker2 # [ 273.515356] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427706625worker2 # [ 273.530031] nix-grpc-daemon[1784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)6626worker2 # [ 273.574371] nix-grpc-daemon[1784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 2/5)6627worker2 # [ 273.647288] nix-grpc-daemon[1784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 355 ms (attempt 3/5)6628worker2 # [ 274.011863] nix-grpc-daemon[1784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 236 ms (attempt 4/5)6629worker2 # [ 274.250030] nix-grpc-daemon[1784]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6630worker2 # [ 274.257505] nix-grpc-daemon[1784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6631worker2 # [ 274.302204] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:35Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06632worker2 # [ 274.308108] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:35Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16633worker2 # [ 274.313661] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:35Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06634worker2 # [ 274.318673] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:35Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06635worker1 # [ 274.291213] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626636worker1 # [ 274.306099] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626637worker1 # [ 274.320112] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626638worker1 # [ 274.335086] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6639worker1 # [ 274.498405] nix-daemon[851]: accepted connection from pid 1929, user nix-grpc-daemon (trusted)6640worker1 # [ 274.515850] nix-daemon[851]: reaped child process 1932, status = succeeded6641lb # [ 274.549946] niks3-server[1133]: 2026/09/22 16:46:36 INFO Received uploads request method=POST path=/api/pending_closures6642worker1 # [ 274.533129] nix-grpc-daemon[1772]: time=2026-09-22T16:46:36.155Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6643worker1 # [ 274.535181] nix-grpc-daemon[1772]: time=2026-09-22T16:46:36.155Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6644lb # [ 274.582045] niks3-server[1133]: 2026/09/22 16:46:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6645worker1 # [ 274.563353] nix-grpc-daemon[1772]: time=2026-09-22T16:46:36.186Z level=INFO msg="Uploading 1 narinfos"6646lb # [ 274.595100] niks3-server[1133]: 2026/09/22 16:46:36 INFO Signed narinfos id=62 count=16647lb # [ 274.603949] niks3-server[1133]: 2026/09/22 16:46:36 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6648lb # [ 274.613172] niks3-server[1133]: 2026/09/22 16:46:36 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6649lb # [ 274.632122] niks3-server[1133]: 2026/09/22 16:46:36 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6650lb # [ 274.639765] niks3-server[1133]: 2026/09/22 16:46:36 INFO Completed upload id=626651worker1 # [ 274.613209] nix-grpc-daemon[1772]: time=2026-09-22T16:46:36.236Z level=INFO msg="Upload complete. (260ms)"6652lb # [ 274.643438] niks3-server[1133]: 2026/09/22 16:46:36 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6653worker1 # [ 274.624651] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=5126654worker1 # [ 274.646522] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626655worker1 # [ 274.651595] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6656worker1 # [ 274.660319] nix-daemon[851]: reaped child process 1927, status = succeeded6657worker1 # [ 274.671604] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6658worker1 # [ 276.113941] nix-daemon[851]: reaped child process 1936, status = succeeded6659worker1 # [ 276.238931] nix-daemon[851]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)6660worker1 # [ 276.255536] nix-daemon[851]: reaped child process 1955, status = succeeded6661lb # [ 276.289659] niks3-server[1133]: 2026/09/22 16:46:37 INFO Received uploads request method=POST path=/api/pending_closures6662worker1 # [ 276.275420] nix-grpc-daemon[1772]: time=2026-09-22T16:46:37.896Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6663worker1 # [ 276.277157] nix-grpc-daemon[1772]: time=2026-09-22T16:46:37.896Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6664lb # [ 276.334754] niks3-server[1133]: 2026/09/22 16:46:37 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6665lb # [ 276.350839] niks3-server[1133]: 2026/09/22 16:46:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6666lb # [ 276.364295] niks3-server[1133]: 2026/09/22 16:46:37 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6667lb # [ 276.372292] niks3-server[1133]: 2026/09/22 16:46:38 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6668worker1 # [ 276.345229] nix-grpc-daemon[1772]: time=2026-09-22T16:46:37.968Z level=INFO msg="Uploading 1 narinfos"6669lb # [ 276.378475] niks3-server[1133]: 2026/09/22 16:46:38 INFO Signed narinfos id=63 count=16670lb # [ 276.383694] niks3-server[1133]: 2026/09/22 16:46:38 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6671lb # [ 276.399235] niks3-server[1133]: 2026/09/22 16:46:38 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6672lb # [ 276.406014] niks3-server[1133]: 2026/09/22 16:46:38 INFO Completed upload id=636673worker1 # [ 276.379381] nix-grpc-daemon[1772]: time=2026-09-22T16:46:38.002Z level=INFO msg="Upload complete. (296ms)"6674worker2 # [ 276.453287] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:38Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16675worker1 # [ 276.393148] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=16676worker2 # [ 276.466953] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:38Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06677worker1 # [ 276.412900] nix-daemon[851]: reaped child process 1934, status = succeeded6678worker1 # [ 276.445782] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626679worker1 # [ 276.460960] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626680worker1 # [ 276.475610] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626681worker1 # [ 276.489796] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6682worker1 # [ 276.656642] nix-daemon[851]: accepted connection from pid 1959, user nix-grpc-daemon (trusted)6683worker1 # [ 276.676678] nix-daemon[851]: reaped child process 1962, status = succeeded6684lb # [ 276.708184] niks3-server[1133]: 2026/09/22 16:46:38 INFO Received uploads request method=POST path=/api/pending_closures6685worker1 # [ 276.692266] nix-grpc-daemon[1772]: time=2026-09-22T16:46:38.315Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6686worker1 # [ 276.695206] nix-grpc-daemon[1772]: time=2026-09-22T16:46:38.318Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6687lb # [ 276.741513] niks3-server[1133]: 2026/09/22 16:46:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6688worker1 # [ 276.723124] nix-grpc-daemon[1772]: time=2026-09-22T16:46:38.346Z level=INFO msg="Uploading 1 narinfos"6689lb # [ 276.759873] niks3-server[1133]: 2026/09/22 16:46:38 INFO Signed narinfos id=64 count=16690lb # [ 276.768207] niks3-server[1133]: 2026/09/22 16:46:38 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6691lb # [ 276.775828] niks3-server[1133]: 2026/09/22 16:46:38 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6692lb # [ 276.783487] niks3-server[1133]: 2026/09/22 16:46:38 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6693worker1 # [ 276.763533] nix-grpc-daemon[1772]: time=2026-09-22T16:46:38.386Z level=INFO msg="Upload complete. (256ms)"6694lb # [ 276.793183] niks3-server[1133]: 2026/09/22 16:46:38 INFO Completed upload id=646695lb # [ 276.795649] niks3-server[1133]: 2026/09/22 16:46:38 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6696worker1 # [ 276.777306] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=5126697worker1 # [ 276.788806] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626698worker1 # [ 276.804470] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6699worker1 # [ 276.807570] nix-daemon[851]: reaped child process 1957, status = succeeded6700worker1 # [ 276.823168] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6701worker1 # [ 277.896875] nix-daemon[851]: reaped child process 1966, status = succeeded6702worker1 # [ 278.007942] nix-daemon[851]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6703worker1 # [ 278.030435] nix-daemon[851]: reaped child process 1985, status = succeeded6704lb # [ 278.057556] niks3-server[1133]: 2026/09/22 16:46:39 INFO Received uploads request method=POST path=/api/pending_closures6705worker1 # [ 278.040798] nix-grpc-daemon[1772]: time=2026-09-22T16:46:39.663Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6706worker1 # [ 278.046845] nix-grpc-daemon[1772]: time=2026-09-22T16:46:39.666Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6707lb # [ 278.106317] niks3-server[1133]: 2026/09/22 16:46:39 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6708worker1 # [ 278.090224] nix-grpc-daemon[1772]: time=2026-09-22T16:46:39.713Z level=INFO msg="Uploading 1 narinfos"6709lb # [ 278.119776] niks3-server[1133]: 2026/09/22 16:46:39 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6710lb # [ 278.128504] niks3-server[1133]: 2026/09/22 16:46:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6711lb # [ 278.131858] niks3-server[1133]: 2026/09/22 16:46:39 INFO Signed narinfos id=65 count=16712lb # [ 278.143520] niks3-server[1133]: 2026/09/22 16:46:39 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6713worker1 # [ 278.116610] nix-grpc-daemon[1772]: time=2026-09-22T16:46:39.740Z level=INFO msg="Upload complete. (281ms)"6714lb # [ 278.147019] niks3-server[1133]: 2026/09/22 16:46:39 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6715lb # [ 278.154236] niks3-server[1133]: 2026/09/22 16:46:39 INFO Completed upload id=656716lb # [ 278.157055] niks3-server[1133]: 2026/09/22 16:46:39 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6717worker2 # [ 278.190832] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:39Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16718worker1 # [ 278.128765] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=16719worker2 # [ 278.198246] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:39Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06720worker1 # [ 278.148442] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626721worker1 # [ 278.160413] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626722worker1 # [ 278.166543] nix-daemon[851]: reaped child process 1964, status = succeeded6723worker1 # [ 278.174413] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626724worker1 # [ 278.186621] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6725worker1 # [ 278.324272] nix-daemon[851]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6726worker1 # [ 278.341898] nix-daemon[851]: reaped child process 1992, status = succeeded6727lb # [ 278.376077] niks3-server[1133]: 2026/09/22 16:46:40 INFO Received uploads request method=POST path=/api/pending_closures6728worker1 # [ 278.357745] nix-grpc-daemon[1772]: time=2026-09-22T16:46:39.980Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6729worker1 # [ 278.359325] nix-grpc-daemon[1772]: time=2026-09-22T16:46:39.980Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6730worker1 # [ 278.386997] nix-grpc-daemon[1772]: time=2026-09-22T16:46:40.010Z level=INFO msg="Uploading 1 narinfos"6731lb # [ 278.407708] niks3-server[1133]: 2026/09/22 16:46:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6732lb # [ 278.419267] niks3-server[1133]: 2026/09/22 16:46:40 INFO Signed narinfos id=66 count=16733lb # [ 278.429581] niks3-server[1133]: 2026/09/22 16:46:40 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6734worker1 # [ 278.418643] nix-grpc-daemon[1772]: time=2026-09-22T16:46:40.042Z level=INFO msg="Upload complete. (219ms)"6735lb # [ 278.446589] niks3-server[1133]: 2026/09/22 16:46:40 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6736lb # [ 278.456456] niks3-server[1133]: 2026/09/22 16:46:40 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6737lb # [ 278.461077] niks3-server[1133]: 2026/09/22 16:46:40 INFO Completed upload id=666738lb # [ 278.463857] niks3-server[1133]: 2026/09/22 16:46:40 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6739worker1 # [ 278.428598] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=5126740worker1 # [ 278.447901] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626741worker1 # [ 278.454015] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6742worker1 # [ 278.460339] nix-daemon[851]: reaped child process 1987, status = succeeded6743worker1 # [ 278.467662] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6744lb # [ 279.234838] envoy[899]: conn peer=[2001:db8:1::4]:44486 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8420 tx=849 ms=1231766745worker1 # [ 279.756745] nix-daemon[851]: reaped child process 1996, status = succeeded6746worker1 # [ 279.852459] nix-daemon[851]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)6747worker1 # [ 279.873219] nix-daemon[851]: reaped child process 2015, status = succeeded6748lb # [ 279.905329] niks3-server[1133]: 2026/09/22 16:46:41 INFO Received uploads request method=POST path=/api/pending_closures6749worker1 # [ 279.888440] nix-grpc-daemon[1772]: time=2026-09-22T16:46:41.511Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6750worker1 # [ 279.890120] nix-grpc-daemon[1772]: time=2026-09-22T16:46:41.511Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6751lb # [ 279.951193] niks3-server[1133]: 2026/09/22 16:46:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6752worker1 # [ 279.934309] nix-grpc-daemon[1772]: time=2026-09-22T16:46:41.557Z level=INFO msg="Uploading 1 narinfos"6753lb # [ 279.965343] niks3-server[1133]: 2026/09/22 16:46:41 INFO Signed narinfos id=67 count=16754lb # [ 279.972756] niks3-server[1133]: 2026/09/22 16:46:41 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6755lb # [ 279.979928] niks3-server[1133]: 2026/09/22 16:46:41 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6756lb # [ 279.989660] niks3-server[1133]: 2026/09/22 16:46:41 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6757lb # [ 279.994673] niks3-server[1133]: 2026/09/22 16:46:41 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6758worker1 # [ 279.967032] nix-grpc-daemon[1772]: time=2026-09-22T16:46:41.590Z level=INFO msg="Upload complete. (287ms)"6759lb # [ 279.997742] niks3-server[1133]: 2026/09/22 16:46:41 INFO Completed upload id=676760lb # [ 280.000842] niks3-server[1133]: 2026/09/22 16:46:41 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6761worker2 # [ 280.036819] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:41Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16762worker2 # [ 280.049905] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:41Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06763worker1 # [ 279.981193] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=16764worker1 # [ 280.000705] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626765worker1 # [ 280.012115] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626766worker1 # [ 280.018499] nix-daemon[851]: reaped child process 1994, status = succeeded6767worker1 # [ 280.026374] nix-grpc-daemon[1772]: time=2026-09-22T16:46:41.649Z level=INFO msg="All 1 paths already cached"6768worker1 # [ 280.035526] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626769worker1 # [ 280.046975] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6770worker1 # [ 280.188840] nix-daemon[851]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6771worker1 # [ 280.207311] nix-daemon[851]: reaped child process 2022, status = succeeded6772lb # [ 280.237472] niks3-server[1133]: 2026/09/22 16:46:41 INFO Received uploads request method=POST path=/api/pending_closures6773worker1 # [ 280.219803] nix-grpc-daemon[1772]: time=2026-09-22T16:46:41.842Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6774worker1 # [ 280.221516] nix-grpc-daemon[1772]: time=2026-09-22T16:46:41.843Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6775lb # [ 280.267232] niks3-server[1133]: 2026/09/22 16:46:41 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6776worker1 # [ 280.249656] nix-grpc-daemon[1772]: time=2026-09-22T16:46:41.873Z level=INFO msg="Uploading 1 narinfos"6777lb # [ 280.281915] niks3-server[1133]: 2026/09/22 16:46:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6778lb # [ 280.292872] niks3-server[1133]: 2026/09/22 16:46:41 INFO Signed narinfos id=68 count=16779worker1 # [ 280.274719] nix-grpc-daemon[1772]: time=2026-09-22T16:46:41.898Z level=INFO msg="Upload complete. (214ms)"6780lb # [ 280.300851] niks3-server[1133]: 2026/09/22 16:46:41 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6781lb # [ 280.310395] niks3-server[1133]: 2026/09/22 16:46:41 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6782worker1 # [ 280.284296] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=6246783lb # [ 280.314734] niks3-server[1133]: 2026/09/22 16:46:41 INFO Completed upload id=686784lb # [ 280.316976] niks3-server[1133]: 2026/09/22 16:46:41 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6785worker1 # [ 280.298553] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626786worker1 # [ 280.304969] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6787worker1 # [ 280.310219] nix-daemon[851]: reaped child process 2017, status = succeeded6788worker1 # [ 280.323343] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6789worker1 # [ 281.396638] nix-daemon[851]: reaped child process 2026, status = succeeded6790worker1 # [ 281.477365] nix-daemon[851]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)6791worker1 # [ 281.498679] nix-daemon[851]: reaped child process 2045, status = succeeded6792lb # [ 281.526281] niks3-server[1133]: 2026/09/22 16:46:43 INFO Received uploads request method=POST path=/api/pending_closures6793worker1 # [ 281.512100] nix-grpc-daemon[1772]: time=2026-09-22T16:46:43.131Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6794worker1 # [ 281.513791] nix-grpc-daemon[1772]: time=2026-09-22T16:46:43.131Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6795lb # [ 281.568091] niks3-server[1133]: 2026/09/22 16:46:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6796worker1 # [ 281.546519] nix-grpc-daemon[1772]: time=2026-09-22T16:46:43.169Z level=INFO msg="Uploading 1 narinfos"6797lb # [ 281.580340] niks3-server[1133]: 2026/09/22 16:46:43 INFO Signed narinfos id=69 count=16798lb # [ 281.587482] niks3-server[1133]: 2026/09/22 16:46:43 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6799lb # [ 281.599266] niks3-server[1133]: 2026/09/22 16:46:43 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6800lb # [ 281.606621] niks3-server[1133]: 2026/09/22 16:46:43 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6801worker1 # [ 281.584634] nix-grpc-daemon[1772]: time=2026-09-22T16:46:43.208Z level=INFO msg="Upload complete. (250ms)"6802lb # [ 281.615093] niks3-server[1133]: 2026/09/22 16:46:43 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6803lb # [ 281.621969] niks3-server[1133]: 2026/09/22 16:46:43 INFO Completed upload id=696804worker2 # [ 281.657202] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:43Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16805worker1 # [ 281.597036] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=16806lb # [ 281.626411] niks3-server[1133]: 2026/09/22 16:46:43 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6807worker1 # [ 281.614491] nix-daemon[851]: reaped child process 2024, status = succeeded6808worker1 # [ 281.640453] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626809worker2 # [ 281.703869] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:43Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06810worker1 # [ 281.656198] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626811worker1 # [ 281.670837] nix-grpc-daemon[1772]: time=2026-09-22T16:46:43.294Z level=INFO msg="All 1 paths already cached"6812worker1 # [ 281.678848] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626813worker1 # [ 281.691503] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6814worker1 # [ 281.840153] nix-daemon[851]: accepted connection from pid 2049, user nix-grpc-daemon (trusted)6815worker1 # [ 281.862098] nix-daemon[851]: reaped child process 2052, status = succeeded6816lb # [ 281.886886] niks3-server[1133]: 2026/09/22 16:46:43 INFO Received uploads request method=POST path=/api/pending_closures6817worker1 # [ 281.870328] nix-grpc-daemon[1772]: time=2026-09-22T16:46:43.493Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6818worker1 # [ 281.871861] nix-grpc-daemon[1772]: time=2026-09-22T16:46:43.493Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6819lb # [ 281.918270] niks3-server[1133]: 2026/09/22 16:46:43 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6820worker1 # [ 281.901583] nix-grpc-daemon[1772]: time=2026-09-22T16:46:43.524Z level=INFO msg="Uploading 1 narinfos"6821lb # [ 281.931347] niks3-server[1133]: 2026/09/22 16:46:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6822lb # [ 281.942136] niks3-server[1133]: 2026/09/22 16:46:43 INFO Signed narinfos id=70 count=16823lb # [ 281.952181] niks3-server[1133]: 2026/09/22 16:46:43 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6824worker1 # [ 281.936308] nix-grpc-daemon[1772]: time=2026-09-22T16:46:43.559Z level=INFO msg="Upload complete. (230ms)"6825lb # [ 281.964599] niks3-server[1133]: 2026/09/22 16:46:43 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6826lb # [ 281.970062] niks3-server[1133]: 2026/09/22 16:46:43 INFO Completed upload id=706827lb # [ 281.973907] niks3-server[1133]: 2026/09/22 16:46:43 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6828worker1 # [ 281.946934] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=6246829worker1 # [ 281.962926] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626830worker1 # [ 281.969583] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6831worker1 # [ 281.974929] nix-daemon[851]: reaped child process 2047, status = succeeded6832worker1 # [ 281.989722] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6833worker1 # [ 283.093468] nix-daemon[851]: reaped child process 2056, status = succeeded6834worker1 # [ 283.161876] nix-daemon[851]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)6835worker1 # [ 283.184311] nix-daemon[851]: reaped child process 2075, status = succeeded6836lb # [ 283.215474] niks3-server[1133]: 2026/09/22 16:46:44 INFO Received uploads request method=POST path=/api/pending_closures6837worker1 # [ 283.198921] nix-grpc-daemon[1772]: time=2026-09-22T16:46:44.821Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6838worker1 # [ 283.200740] nix-grpc-daemon[1772]: time=2026-09-22T16:46:44.821Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6839lb # [ 283.257249] niks3-server[1133]: 2026/09/22 16:46:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6840worker1 # [ 283.240256] nix-grpc-daemon[1772]: time=2026-09-22T16:46:44.863Z level=INFO msg="Uploading 1 narinfos"6841lb # [ 283.271934] niks3-server[1133]: 2026/09/22 16:46:44 INFO Signed narinfos id=71 count=16842lb # [ 283.278348] niks3-server[1133]: 2026/09/22 16:46:44 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6843lb # [ 283.285985] niks3-server[1133]: 2026/09/22 16:46:44 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6844lb # [ 283.290238] niks3-server[1133]: 2026/09/22 16:46:44 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6845lb # [ 283.299466] niks3-server[1133]: 2026/09/22 16:46:44 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6846worker1 # [ 283.272831] nix-grpc-daemon[1772]: time=2026-09-22T16:46:44.896Z level=INFO msg="Upload complete. (271ms)"6847lb # [ 283.303925] niks3-server[1133]: 2026/09/22 16:46:44 INFO Completed upload id=716848lb # [ 283.306039] niks3-server[1133]: 2026/09/22 16:46:44 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6849worker2 # [ 283.345705] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:44Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16850worker1 # [ 283.285145] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16851worker2 # [ 283.358982] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:44Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06852worker1 # [ 283.305124] nix-daemon[851]: reaped child process 2054, status = succeeded6853worker1 # [ 283.336962] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626854worker1 # [ 283.352333] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626855worker1 # [ 283.366249] nix-grpc-daemon[1772]: time=2026-09-22T16:46:44.989Z level=INFO msg="All 1 paths already cached"6856worker1 # [ 283.376715] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626857worker1 # [ 283.390570] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6858worker1 # [ 283.552733] nix-daemon[851]: accepted connection from pid 2079, user nix-grpc-daemon (trusted)6859worker1 # [ 283.573386] nix-daemon[851]: reaped child process 2082, status = succeeded6860lb # [ 283.604450] niks3-server[1133]: 2026/09/22 16:46:45 INFO Received uploads request method=POST path=/api/pending_closures6861worker1 # [ 283.586425] nix-grpc-daemon[1772]: time=2026-09-22T16:46:45.209Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6862worker1 # [ 283.587986] nix-grpc-daemon[1772]: time=2026-09-22T16:46:45.209Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6863worker1 # [ 283.611635] nix-grpc-daemon[1772]: time=2026-09-22T16:46:45.234Z level=INFO msg="Uploading 1 narinfos"6864lb # [ 283.632232] niks3-server[1133]: 2026/09/22 16:46:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6865lb # [ 283.646639] niks3-server[1133]: 2026/09/22 16:46:45 INFO Signed narinfos id=72 count=16866lb # [ 283.654475] niks3-server[1133]: 2026/09/22 16:46:45 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6867worker1 # [ 283.642407] nix-grpc-daemon[1772]: time=2026-09-22T16:46:45.265Z level=INFO msg="Upload complete. (236ms)"6868lb # [ 283.669504] niks3-server[1133]: 2026/09/22 16:46:45 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6869lb # [ 283.677154] niks3-server[1133]: 2026/09/22 16:46:45 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6870worker1 # [ 283.650318] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=6246871lb # [ 283.684059] niks3-server[1133]: 2026/09/22 16:46:45 INFO Completed upload id=726872lb # [ 283.686942] niks3-server[1133]: 2026/09/22 16:46:45 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6873worker1 # [ 283.667194] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626874worker1 # [ 283.675584] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6875worker1 # [ 283.681308] nix-daemon[851]: reaped child process 2077, status = succeeded6876worker1 # [ 283.695748] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6877worker1 # [ 284.330639] nix-daemon[851]: reaped child process 2086, status = succeeded6878worker1 # [ 284.418214] nix-daemon[851]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)6879worker1 # [ 284.438668] nix-daemon[851]: reaped child process 2105, status = succeeded6880lb # [ 284.465660] niks3-server[1133]: 2026/09/22 16:46:46 INFO Received uploads request method=POST path=/api/pending_closures6881worker1 # [ 284.450780] nix-grpc-daemon[1772]: time=2026-09-22T16:46:46.072Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6882worker1 # [ 284.452421] nix-grpc-daemon[1772]: time=2026-09-22T16:46:46.072Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6883lb # [ 284.505934] niks3-server[1133]: 2026/09/22 16:46:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6884worker1 # [ 284.487521] nix-grpc-daemon[1772]: time=2026-09-22T16:46:46.110Z level=INFO msg="Uploading 1 narinfos"6885lb # [ 284.519152] niks3-server[1133]: 2026/09/22 16:46:46 INFO Signed narinfos id=73 count=16886lb # [ 284.528651] niks3-server[1133]: 2026/09/22 16:46:46 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6887lb # [ 284.537363] niks3-server[1133]: 2026/09/22 16:46:46 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6888worker1 # [ 284.525152] nix-grpc-daemon[1772]: time=2026-09-22T16:46:46.148Z level=INFO msg="Upload complete. (220ms)"6889lb # [ 284.552924] niks3-server[1133]: 2026/09/22 16:46:46 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6890lb # [ 284.559087] niks3-server[1133]: 2026/09/22 16:46:46 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6891lb # [ 284.563817] niks3-server[1133]: 2026/09/22 16:46:46 INFO Completed upload id=736892worker2 # [ 284.593888] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:46Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16893worker1 # [ 284.537055] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16894lb # [ 284.567297] niks3-server[1133]: 2026/09/22 16:46:46 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6895worker1 # [ 284.551005] nix-daemon[851]: reaped child process 2084, status = succeeded6896worker1 # [ 284.578311] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626897worker2 # [ 284.643041] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:46Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06898worker1 # [ 284.594549] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626899worker1 # [ 284.609038] nix-grpc-daemon[1772]: time=2026-09-22T16:46:46.232Z level=INFO msg="All 2 paths already cached"6900worker1 # [ 284.619225] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626901worker1 # [ 284.633092] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6902worker1 # [ 284.784153] nix-daemon[851]: accepted connection from pid 2109, user nix-grpc-daemon (trusted)6903worker1 # [ 284.803536] nix-daemon[851]: reaped child process 2112, status = succeeded6904lb # [ 284.836465] niks3-server[1133]: 2026/09/22 16:46:46 INFO Received uploads request method=POST path=/api/pending_closures6905worker1 # [ 284.818343] nix-grpc-daemon[1772]: time=2026-09-22T16:46:46.441Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6906worker1 # [ 284.819899] nix-grpc-daemon[1772]: time=2026-09-22T16:46:46.441Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6907lb # [ 284.863318] niks3-server[1133]: 2026/09/22 16:46:46 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6908worker1 # [ 284.850895] nix-grpc-daemon[1772]: time=2026-09-22T16:46:46.474Z level=INFO msg="Uploading 1 narinfos"6909lb # [ 284.877491] niks3-server[1133]: 2026/09/22 16:46:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6910lb # [ 284.891279] niks3-server[1133]: 2026/09/22 16:46:46 INFO Signed narinfos id=74 count=16911lb # [ 284.898872] niks3-server[1133]: 2026/09/22 16:46:46 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6912lb # [ 284.913490] niks3-server[1133]: 2026/09/22 16:46:46 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6913lb # [ 284.919037] niks3-server[1133]: 2026/09/22 16:46:46 INFO Completed upload id=746914lb # [ 284.922752] niks3-server[1133]: 2026/09/22 16:46:46 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6915worker1 # [ 284.891382] nix-grpc-daemon[1772]: time=2026-09-22T16:46:46.514Z level=INFO msg="Upload complete. (243ms)"6916worker1 # [ 284.900590] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=7526917worker1 # [ 284.919848] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626918worker1 # [ 284.927244] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6919worker1 # [ 284.933556] nix-daemon[851]: reaped child process 2107, status = succeeded6920worker1 # [ 284.947124] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6921worker1 # [ 286.203549] nix-daemon[851]: reaped child process 2116, status = succeeded6922worker1 # [ 286.321854] nix-daemon[851]: accepted connection from pid 2132, user nix-grpc-daemon (trusted)6923worker1 # [ 286.340925] nix-daemon[851]: reaped child process 2135, status = succeeded6924lb # [ 286.374758] niks3-server[1133]: 2026/09/22 16:46:48 INFO Received uploads request method=POST path=/api/pending_closures6925worker1 # [ 286.358698] nix-grpc-daemon[1772]: time=2026-09-22T16:46:47.980Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6926worker1 # [ 286.360492] nix-grpc-daemon[1772]: time=2026-09-22T16:46:47.980Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6927lb # [ 286.414839] niks3-server[1133]: 2026/09/22 16:46:48 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6928worker1 # [ 286.402201] nix-grpc-daemon[1772]: time=2026-09-22T16:46:48.025Z level=INFO msg="Uploading 1 narinfos"6929lb # [ 286.430486] niks3-server[1133]: 2026/09/22 16:46:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6930lb # [ 286.439970] niks3-server[1133]: 2026/09/22 16:46:48 INFO Signed narinfos id=75 count=16931lb # [ 286.442675] niks3-server[1133]: 2026/09/22 16:46:48 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6932lb # [ 286.454506] niks3-server[1133]: 2026/09/22 16:46:48 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6933worker1 # [ 286.432216] nix-grpc-daemon[1772]: time=2026-09-22T16:46:48.055Z level=INFO msg="Upload complete. (270ms)"6934lb # [ 286.459811] niks3-server[1133]: 2026/09/22 16:46:48 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6935lb # [ 286.463367] niks3-server[1133]: 2026/09/22 16:46:48 INFO Completed upload id=756936lb # [ 286.467999] niks3-server[1133]: 2026/09/22 16:46:48 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6937worker2 # [ 286.501784] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:48Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16938worker1 # [ 286.445201] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16939worker1 # [ 286.465347] nix-daemon[851]: reaped child process 2114, status = succeeded6940worker1 # [ 286.494936] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626941worker1 # [ 286.509814] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626942worker1 # [ 286.524352] nix-grpc-daemon[1772]: time=2026-09-22T16:46:48.147Z level=INFO msg="All 2 paths already cached"6943worker1 # [ 286.534268] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626944worker1 # [ 286.548148] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6945worker1 # [ 286.700917] nix-daemon[851]: accepted connection from pid 2139, user nix-grpc-daemon (trusted)6946worker1 # [ 286.719436] nix-daemon[851]: reaped child process 2142, status = succeeded6947lb # [ 286.749947] niks3-server[1133]: 2026/09/22 16:46:48 INFO Received uploads request method=POST path=/api/pending_closures6948worker1 # [ 286.731962] nix-grpc-daemon[1772]: time=2026-09-22T16:46:48.355Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6949worker1 # [ 286.733828] nix-grpc-daemon[1772]: time=2026-09-22T16:46:48.355Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6950lb # [ 286.780177] niks3-server[1133]: 2026/09/22 16:46:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6951worker1 # [ 286.759178] nix-grpc-daemon[1772]: time=2026-09-22T16:46:48.382Z level=INFO msg="Uploading 1 narinfos"6952lb # [ 286.792704] niks3-server[1133]: 2026/09/22 16:46:48 INFO Signed narinfos id=76 count=16953lb # [ 286.795824] niks3-server[1133]: 2026/09/22 16:46:48 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6954worker1 # [ 286.787861] nix-grpc-daemon[1772]: time=2026-09-22T16:46:48.411Z level=INFO msg="Upload complete. (224ms)"6955lb # [ 286.815969] niks3-server[1133]: 2026/09/22 16:46:48 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6956lb # [ 286.825162] niks3-server[1133]: 2026/09/22 16:46:48 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6957worker1 # [ 286.800270] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=7526958lb # [ 286.832721] niks3-server[1133]: 2026/09/22 16:46:48 INFO Completed upload id=766959lb # [ 286.835581] niks3-server[1133]: 2026/09/22 16:46:48 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6960worker1 # [ 286.813914] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626961worker1 # [ 286.820464] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6962worker1 # [ 286.825986] nix-daemon[851]: reaped child process 2137, status = succeeded6963worker1 # [ 286.838687] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6964worker1 # [ 288.350831] nix-daemon[851]: reaped child process 2146, status = succeeded6965worker1 # [ 288.425147] nix-daemon[851]: accepted connection from pid 2162, user nix-grpc-daemon (trusted)6966worker1 # [ 288.447481] nix-daemon[851]: reaped child process 2165, status = succeeded6967lb # [ 288.476647] niks3-server[1133]: 2026/09/22 16:46:50 INFO Received uploads request method=POST path=/api/pending_closures6968worker1 # [ 288.460420] nix-grpc-daemon[1772]: time=2026-09-22T16:46:50.082Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6969worker1 # [ 288.462136] nix-grpc-daemon[1772]: time=2026-09-22T16:46:50.082Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6970lb # [ 288.521268] niks3-server[1133]: 2026/09/22 16:46:50 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6971worker1 # [ 288.507184] nix-grpc-daemon[1772]: time=2026-09-22T16:46:50.130Z level=INFO msg="Uploading 1 narinfos"6972lb # [ 288.534935] niks3-server[1133]: 2026/09/22 16:46:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6973lb # [ 288.547114] niks3-server[1133]: 2026/09/22 16:46:50 INFO Signed narinfos id=77 count=16974lb # [ 288.556219] niks3-server[1133]: 2026/09/22 16:46:50 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6975lb # [ 288.568272] niks3-server[1133]: 2026/09/22 16:46:50 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6976lb # [ 288.572082] niks3-server[1133]: 2026/09/22 16:46:50 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6977lb # [ 288.576688] niks3-server[1133]: 2026/09/22 16:46:50 INFO Completed upload id=776978worker1 # [ 288.540889] nix-grpc-daemon[1772]: time=2026-09-22T16:46:50.164Z level=INFO msg="Upload complete. (291ms)"6979lb # [ 288.579777] niks3-server[1133]: 2026/09/22 16:46:50 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6980worker2 # [ 288.612414] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:50Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16981worker1 # [ 288.554279] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=16982worker1 # [ 288.575478] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431626983worker1 # [ 288.582252] nix-daemon[851]: reaped child process 2144, status = succeeded6984worker1 # [ 288.590363] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431626985worker1 # [ 288.603677] nix-grpc-daemon[1772]: time=2026-09-22T16:46:50.227Z level=INFO msg="All 2 paths already cached"6986worker1 # [ 288.613336] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:431626987worker1 # [ 288.626936] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6988worker1 # [ 288.773370] nix-daemon[851]: accepted connection from pid 2169, user nix-grpc-daemon (trusted)6989worker1 # [ 288.793054] nix-daemon[851]: reaped child process 2172, status = succeeded6990lb # [ 288.818483] niks3-server[1133]: 2026/09/22 16:46:50 INFO Received uploads request method=POST path=/api/pending_closures6991worker1 # [ 288.800212] nix-grpc-daemon[1772]: time=2026-09-22T16:46:50.423Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6992worker1 # [ 288.802379] nix-grpc-daemon[1772]: time=2026-09-22T16:46:50.425Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6993lb # [ 288.846874] niks3-server[1133]: 2026/09/22 16:46:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6994worker1 # [ 288.825699] nix-grpc-daemon[1772]: time=2026-09-22T16:46:50.449Z level=INFO msg="Uploading 1 narinfos"6995lb # [ 288.858897] niks3-server[1133]: 2026/09/22 16:46:50 INFO Signed narinfos id=78 count=16996lb # [ 288.867917] niks3-server[1133]: 2026/09/22 16:46:50 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6997lb # [ 288.879958] niks3-server[1133]: 2026/09/22 16:46:50 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6998worker1 # [ 288.861308] nix-grpc-daemon[1772]: time=2026-09-22T16:46:50.484Z level=INFO msg="Upload complete. (220ms)"6999lb # [ 288.889082] niks3-server[1133]: 2026/09/22 16:46:50 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7000lb # [ 288.896178] niks3-server[1133]: 2026/09/22 16:46:50 INFO Completed upload id=787001worker1 # [ 288.869596] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=0 paths=1 nar_bytes_in=7527002lb # [ 288.898518] niks3-server[1133]: 2026/09/22 16:46:50 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo7003worker1 # [ 288.883328] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:431627004worker1 # [ 288.889208] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)7005worker1 # [ 288.894085] nix-daemon[851]: reaped child process 2167, status = succeeded7006worker1 # [ 288.906320] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)7007worker1 # [ 290.147117] nix-daemon[851]: reaped child process 2176, status = succeeded7008worker1 # [ 290.272920] nix-daemon[851]: accepted connection from pid 2192, user nix-grpc-daemon (trusted)7009worker1 # [ 290.294582] nix-daemon[851]: reaped child process 2195, status = succeeded7010lb # [ 290.326526] niks3-server[1133]: 2026/09/22 16:46:51 INFO Received uploads request method=POST path=/api/pending_closures7011worker1 # [ 290.310828] nix-grpc-daemon[1772]: time=2026-09-22T16:46:51.932Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7012worker1 # [ 290.312715] nix-grpc-daemon[1772]: time=2026-09-22T16:46:51.933Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7013lb # [ 290.363409] niks3-server[1133]: 2026/09/22 16:46:52 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7014lb # [ 290.379586] niks3-server[1133]: 2026/09/22 16:46:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7015worker1 # [ 290.358536] nix-grpc-daemon[1772]: time=2026-09-22T16:46:51.981Z level=INFO msg="Uploading 1 narinfos"7016lb # [ 290.392556] niks3-server[1133]: 2026/09/22 16:46:52 INFO Signed narinfos id=79 count=17017lb # [ 290.399577] niks3-server[1133]: 2026/09/22 16:46:52 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7018lb # [ 290.410636] niks3-server[1133]: 2026/09/22 16:46:52 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7019worker1 # [ 290.383657] nix-grpc-daemon[1772]: time=2026-09-22T16:46:52.007Z level=INFO msg="Upload complete. (267ms)"7020lb # [ 290.416399] niks3-server[1133]: 2026/09/22 16:46:52 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7021lb # [ 290.422920] niks3-server[1133]: 2026/09/22 16:46:52 INFO Completed upload id=797022lb # [ 290.426130] niks3-server[1133]: 2026/09/22 16:46:52 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7023worker1 # [ 290.395675] nix-grpc-daemon[1696]: ts=2026-09-22T16:46:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43162 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=17024worker1 # [ 290.416173] nix-daemon[851]: reaped child process 2174, status = succeeded7025client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.29 seconds)7026worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7027worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7028client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7029client # [ 290.678427] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7030client # Running as unit: gpu.service; invocation ID: d9bdb6ac14284fdaa54a85d99c2ea2ec7031client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.10 seconds)7032lb: must succeed: curl -sf localhost:9901/clusters7033lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7034lb: must succeed: curl -sf localhost:9901/clusters7035lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7036worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7037client # [ 290.884197] nix[2007]: don't know how to build these paths:7038client # [ 290.885332] nix[2007]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7039worker2 # [ 290.877192] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427707040worker2 # [ 290.887262] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427707041worker2 # [ 290.909103] nix-grpc-daemon[1784]: ts=2026-09-22T16:46:52Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07042client # [ 290.917168] nix[2007]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker7043worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)7044lb: must succeed: curl -sf localhost:9901/clusters7045lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7046lb: must succeed: curl -sf localhost:9901/clusters7047lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7048worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7049worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7050client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7051lb # [ 299.230998] envoy[899]: conn peer=[2001:db8:1::1]:40960 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27183 tx=12437 ms=170037052lb # [ 315.191521] postgres[787]: [787] LOG: checkpoint starting: time7053client # [ 320.918787] nix[2007]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}7054client # [ 320.937755] nix[2007]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7055client # [ 320.970429] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7056client # [ 320.977553] systemd[1]: gpu.service: Failed with result 'exit-code'.7057client # [ 320.980725] systemd[1]: gpu.service: Consumed 200ms CPU time over 30.297s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7058client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.07 seconds)7059client: must succeed: journalctl -u gpu -o cat7060client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7061lb: must succeed: curl -sf localhost:9901/clusters7062lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7063lb: must succeed: curl -sf localhost:9901/clusters7064lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7065worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7066worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7067(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.14 seconds)7068subtest: low disk drains a worker and builds go to the other7069worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7070worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.21 seconds)7071worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7072worker1 # [ 322.810098] nix-grpc-daemon[1696]: ts=2026-09-22T16:47:24Z level=info event=unhealthy reason=min_free7073worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)7074client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27075client # don't know how to build these paths:7076client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7077worker2 # [ 323.924243] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427707078worker1 # [ 323.867808] nix-grpc-daemon[1696]: ts=2026-09-22T16:47:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:431627079worker2 # [ 323.954706] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:25Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07080worker2 # [ 323.962668] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:25Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17081worker2 # [ 324.002492] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427707082worker2 # [ 324.018781] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427707083client # copying 1 paths to 'lb:50051'...7084client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7085worker2 # [ 324.035003] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427707086worker2 # [ 324.050447] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)7087worker2 # [ 324.077840] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:25Z level=info event=niks3_push_started pid=18807088worker2 # [ 324.228994] nix-grpc-daemon[1880]: time=2026-09-22T16:47:25.864Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""7089worker2 # [ 324.333275] nix-daemon[838]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)7090worker2 # [ 324.350791] nix-daemon[838]: reaped child process 1896, status = succeeded7091lb # [ 324.325165] niks3-server[1133]: 2026/09/22 16:47:25 INFO Received uploads request method=POST path=/api/pending_closures7092worker2 # [ 324.371354] nix-grpc-daemon[1880]: time=2026-09-22T16:47:26.006Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7093worker2 # [ 324.375288] nix-grpc-daemon[1880]: time=2026-09-22T16:47:26.006Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7094lb # [ 324.372911] niks3-server[1133]: 2026/09/22 16:47:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7095worker2 # [ 324.415968] nix-grpc-daemon[1880]: time=2026-09-22T16:47:26.050Z level=INFO msg="Uploading 1 narinfos"7096lb # [ 324.384554] niks3-server[1133]: 2026/09/22 16:47:26 INFO Signed narinfos id=80 count=17097lb # [ 324.396105] niks3-server[1133]: 2026/09/22 16:47:26 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7098lb # [ 324.407822] niks3-server[1133]: 2026/09/22 16:47:26 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7099lb # [ 324.427696] niks3-server[1133]: 2026/09/22 16:47:26 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7100lb # [ 324.434861] niks3-server[1133]: 2026/09/22 16:47:26 INFO Completed upload id=807101worker2 # [ 324.467822] nix-grpc-daemon[1880]: time=2026-09-22T16:47:26.103Z level=INFO msg="Upload complete. (235ms)"7102lb # [ 324.437267] niks3-server[1133]: 2026/09/22 16:47:26 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7103worker2 # [ 324.480487] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42770 duration_s=0 paths=1 nar_bytes_in=5127104worker2 # [ 324.501745] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427707105worker2 # [ 324.505848] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)7106worker2 # [ 324.514075] nix-daemon[838]: reaped child process 1878, status = succeeded7107client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7108client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7109worker2 # [ 324.525358] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)7110client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7111client # farm-a> LOG-farm-a-drain7112worker2 # [ 325.235308] nix-daemon[838]: reaped child process 1900, status = succeeded7113worker2 # [ 325.309236] nix-daemon[838]: accepted connection from pid 1916, user nix-grpc-daemon (trusted)7114worker2 # [ 325.329127] nix-daemon[838]: reaped child process 1919, status = succeeded7115lb # [ 325.299871] niks3-server[1133]: 2026/09/22 16:47:26 INFO Received uploads request method=POST path=/api/pending_closures7116worker2 # [ 325.347431] nix-grpc-daemon[1880]: time=2026-09-22T16:47:26.980Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7117worker2 # [ 325.350325] nix-grpc-daemon[1880]: time=2026-09-22T16:47:26.980Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7118lb # [ 325.353582] niks3-server[1133]: 2026/09/22 16:47:26 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7119worker2 # [ 325.399794] nix-grpc-daemon[1880]: time=2026-09-22T16:47:27.035Z level=INFO msg="Uploading 1 narinfos"7120lb # [ 325.367149] niks3-server[1133]: 2026/09/22 16:47:27 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7121lb # [ 325.375609] niks3-server[1133]: 2026/09/22 16:47:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7122lb # [ 325.382537] niks3-server[1133]: 2026/09/22 16:47:27 INFO Signed narinfos id=81 count=17123lb # [ 325.389559] niks3-server[1133]: 2026/09/22 16:47:27 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7124worker2 # [ 325.427577] nix-grpc-daemon[1880]: time=2026-09-22T16:47:27.063Z level=INFO msg="Upload complete. (291ms)"7125lb # [ 325.396351] niks3-server[1133]: 2026/09/22 16:47:27 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7126lb # [ 325.400755] niks3-server[1133]: 2026/09/22 16:47:27 INFO Completed upload id=817127lb # [ 325.404380] niks3-server[1133]: 2026/09/22 16:47:27 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7128worker2 # [ 325.441173] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42770 duration_s=0 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=17129worker2 # [ 325.468271] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:27Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07130worker2 # [ 325.472804] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:27Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17131worker2 # [ 325.477405] nix-daemon[838]: reaped child process 1898, status = succeeded7132worker2 # [ 325.507192] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427707133worker2 # [ 325.523443] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427707134worker2 # [ 325.538964] nix-grpc-daemon[1880]: time=2026-09-22T16:47:27.174Z level=INFO msg="All 1 paths already cached"7135client # copying 1 paths to 'lb:50051'...7136client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7137worker2 # [ 325.550628] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427707138worker2 # [ 325.565694] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)7139worker2 # [ 325.722241] nix-daemon[838]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)7140worker2 # [ 325.742273] nix-daemon[838]: reaped child process 1928, status = succeeded7141lb # [ 325.712639] niks3-server[1133]: 2026/09/22 16:47:27 INFO Received uploads request method=POST path=/api/pending_closures7142worker2 # [ 325.756493] nix-grpc-daemon[1880]: time=2026-09-22T16:47:27.391Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7143worker2 # [ 325.758539] nix-grpc-daemon[1880]: time=2026-09-22T16:47:27.391Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7144lb # [ 325.747362] niks3-server[1133]: 2026/09/22 16:47:27 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7145worker2 # [ 325.792693] nix-grpc-daemon[1880]: time=2026-09-22T16:47:27.428Z level=INFO msg="Uploading 1 narinfos"7146lb # [ 325.763822] niks3-server[1133]: 2026/09/22 16:47:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7147lb # [ 325.773508] niks3-server[1133]: 2026/09/22 16:47:27 INFO Signed narinfos id=82 count=17148lb # [ 325.778507] niks3-server[1133]: 2026/09/22 16:47:27 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7149lb # [ 325.787441] niks3-server[1133]: 2026/09/22 16:47:27 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7150worker2 # [ 325.820546] nix-grpc-daemon[1880]: time=2026-09-22T16:47:27.456Z level=INFO msg="Upload complete. (238ms)"7151lb # [ 325.791880] niks3-server[1133]: 2026/09/22 16:47:27 INFO Completed upload id=827152lb # [ 325.795097] niks3-server[1133]: 2026/09/22 16:47:27 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7153worker2 # [ 325.832341] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42770 duration_s=0 paths=1 nar_bytes_in=6327154worker2 # [ 325.844613] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427707155worker2 # [ 325.856742] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)7156worker2 # [ 325.862811] nix-daemon[838]: reaped child process 1923, status = succeeded7157client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7158client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7159worker2 # [ 325.878635] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)7160client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7161client # farm-b> LOG-farm-b-drain7162worker2 # [ 326.952225] nix-daemon[838]: reaped child process 1932, status = succeeded7163worker2 # [ 327.035748] nix-daemon[838]: accepted connection from pid 1948, user nix-grpc-daemon (trusted)7164worker2 # [ 327.057213] nix-daemon[838]: reaped child process 1951, status = succeeded7165lb # [ 327.022520] niks3-server[1133]: 2026/09/22 16:47:28 INFO Received uploads request method=POST path=/api/pending_closures7166worker2 # [ 327.067295] nix-grpc-daemon[1880]: time=2026-09-22T16:47:28.702Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7167worker2 # [ 327.069077] nix-grpc-daemon[1880]: time=2026-09-22T16:47:28.702Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7168lb # [ 327.067383] niks3-server[1133]: 2026/09/22 16:47:28 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7169worker2 # [ 327.110458] nix-grpc-daemon[1880]: time=2026-09-22T16:47:28.746Z level=INFO msg="Uploading 1 narinfos"7170lb # [ 327.078641] niks3-server[1133]: 2026/09/22 16:47:28 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7171lb # [ 327.087538] niks3-server[1133]: 2026/09/22 16:47:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7172lb # [ 327.096678] niks3-server[1133]: 2026/09/22 16:47:28 INFO Signed narinfos id=83 count=17173lb # [ 327.098733] niks3-server[1133]: 2026/09/22 16:47:28 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7174worker2 # [ 327.137710] nix-grpc-daemon[1880]: time=2026-09-22T16:47:28.773Z level=INFO msg="Upload complete. (247ms)"7175lb # [ 327.105245] niks3-server[1133]: 2026/09/22 16:47:28 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7176lb # [ 327.108808] niks3-server[1133]: 2026/09/22 16:47:28 INFO Completed upload id=837177lb # [ 327.112871] niks3-server[1133]: 2026/09/22 16:47:28 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7178worker2 # [ 327.149051] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42770 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=17179worker2 # [ 327.172277] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:28Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07180worker2 # [ 327.175969] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:28Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17181worker2 # [ 327.180635] nix-daemon[838]: reaped child process 1930, status = succeeded7182worker2 # [ 327.214348] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427707183worker2 # [ 327.230809] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427707184worker2 # [ 327.245242] nix-grpc-daemon[1880]: time=2026-09-22T16:47:28.881Z level=INFO msg="All 2 paths already cached"7185client # copying 1 paths to 'lb:50051'...7186client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7187worker2 # [ 327.256791] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427707188worker2 # [ 327.270557] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)7189worker2 # [ 327.427319] nix-daemon[838]: accepted connection from pid 1957, user nix-grpc-daemon (trusted)7190worker2 # [ 327.447657] nix-daemon[838]: reaped child process 1960, status = succeeded7191lb # [ 327.419770] niks3-server[1133]: 2026/09/22 16:47:29 INFO Received uploads request method=POST path=/api/pending_closures7192worker2 # [ 327.461883] nix-grpc-daemon[1880]: time=2026-09-22T16:47:29.097Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7193worker2 # [ 327.463519] nix-grpc-daemon[1880]: time=2026-09-22T16:47:29.097Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7194worker2 # [ 327.480526] nix-grpc-daemon[1880]: time=2026-09-22T16:47:29.116Z level=INFO msg="Uploading 1 narinfos"7195lb # [ 327.446857] niks3-server[1133]: 2026/09/22 16:47:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7196lb # [ 327.456347] niks3-server[1133]: 2026/09/22 16:47:29 INFO Signed narinfos id=84 count=17197lb # [ 327.466336] niks3-server[1133]: 2026/09/22 16:47:29 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7198lb # [ 327.479454] niks3-server[1133]: 2026/09/22 16:47:29 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7199worker2 # [ 327.517191] nix-grpc-daemon[1880]: time=2026-09-22T16:47:29.153Z level=INFO msg="Upload complete. (228ms)"7200lb # [ 327.484948] niks3-server[1133]: 2026/09/22 16:47:29 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7201lb # [ 327.489989] niks3-server[1133]: 2026/09/22 16:47:29 INFO Completed upload id=847202worker2 # [ 327.525619] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42770 duration_s=0 paths=1 nar_bytes_in=7607203lb # [ 327.494005] niks3-server[1133]: 2026/09/22 16:47:29 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7204worker2 # [ 327.540364] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427707205worker2 # [ 327.546920] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)7206worker2 # [ 327.552104] nix-daemon[838]: reaped child process 1955, status = succeeded7207client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7208client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7209worker2 # [ 327.566744] nix-daemon[838]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)7210client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7211client # farm-top> LOG-farm-top-drain7212worker2 # [ 329.244311] nix-daemon[838]: reaped child process 1964, status = succeeded7213lb # [ 329.236129] envoy[899]: conn peer=[2001:db8:1::1]:58824 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=579 ms=300627214worker2 # [ 329.347162] nix-daemon[838]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)7215worker2 # [ 329.366573] nix-daemon[838]: reaped child process 1983, status = succeeded7216lb # [ 329.337258] niks3-server[1133]: 2026/09/22 16:47:30 INFO Received uploads request method=POST path=/api/pending_closures7217worker2 # [ 329.383503] nix-grpc-daemon[1880]: time=2026-09-22T16:47:31.017Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7218worker2 # [ 329.385523] nix-grpc-daemon[1880]: time=2026-09-22T16:47:31.017Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7219lb # [ 329.377395] niks3-server[1133]: 2026/09/22 16:47:31 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7220worker2 # [ 329.424323] nix-grpc-daemon[1880]: time=2026-09-22T16:47:31.059Z level=INFO msg="Uploading 1 narinfos"7221lb # [ 329.394407] niks3-server[1133]: 2026/09/22 16:47:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7222lb # [ 329.404004] niks3-server[1133]: 2026/09/22 16:47:31 INFO Signed narinfos id=85 count=17223lb # [ 329.409199] niks3-server[1133]: 2026/09/22 16:47:31 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7224lb # [ 329.419113] niks3-server[1133]: 2026/09/22 16:47:31 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7225worker2 # [ 329.457576] nix-grpc-daemon[1880]: time=2026-09-22T16:47:31.093Z level=INFO msg="Upload complete. (279ms)"7226lb # [ 329.425378] niks3-server[1133]: 2026/09/22 16:47:31 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7227lb # [ 329.429119] niks3-server[1133]: 2026/09/22 16:47:31 INFO Completed upload id=857228lb # [ 329.432817] niks3-server[1133]: 2026/09/22 16:47:31 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7229worker2 # [ 329.472812] nix-grpc-daemon[1784]: ts=2026-09-22T16:47:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42770 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=17230worker2 # [ 329.491980] nix-daemon[838]: reaped child process 1962, status = succeeded7231client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.83 seconds)7232client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7233client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7234worker1: must succeed: rm /nix/.rw-store/fill7235worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7236worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7237worker1 # [ 330.822634] nix-grpc-daemon[1696]: ts=2026-09-22T16:47:32Z level=info event=healthy reason=min_free7238worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7239(finished: subtest: low disk drains a worker and builds go to the other, in 9.65 seconds)7240(finished: run the VM test script, in 331.88 seconds)7241test script finished in 332.13s7242cleanup7243kill QemuMachine (pid 49)7244client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7245client # [2026-09-22T16:47:32Z INFO virtiofsd] Client disconnected, shutting down7246client # [2026-09-22T16:47:32Z INFO virtiofsd] Client disconnected, shutting down7247client # [2026-09-22T16:47:32Z INFO virtiofsd] Client disconnected, shutting down7248kill QemuMachine (pid 50)7249lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7250lb # [2026-09-22T16:47:33Z INFO virtiofsd] Client disconnected, shutting down7251lb # [2026-09-22T16:47:33Z INFO virtiofsd] Client disconnected, shutting down7252lb # [2026-09-22T16:47:33Z INFO virtiofsd] Client disconnected, shutting down7253kill QemuMachine (pid 51)7254worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7255worker1 # [2026-09-22T16:47:33Z INFO virtiofsd] Client disconnected, shutting down7256worker1 # [2026-09-22T16:47:33Z INFO virtiofsd] Client disconnected, shutting down7257worker1 # [2026-09-22T16:47:33Z INFO virtiofsd] Client disconnected, shutting down7258kill QemuMachine (pid 52)7259worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7260worker2 # [2026-09-22T16:47:33Z INFO virtiofsd] Client disconnected, shutting down7261worker2 # [2026-09-22T16:47:33Z INFO virtiofsd] Client disconnected, shutting down7262worker2 # [2026-09-22T16:47:33Z INFO virtiofsd] Client disconnected, shutting down7263(finished: cleanup, in 1.08 seconds)7264additionally exposed symbols:7265 client, lb, worker1, worker2,7266 vlan1,7267 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh