vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #554
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14lb: QEMU running (pid 50)15client: QEMU running (pid 49)16client # Formatting '/build/vm-state-client/tmp.IMSBbWYzp9', fmt=raw size=107374182417worker1: QEMU running (pid 51)18worker2 # Disk image does not exist, creating the virtualisation disk image...19worker1 # Disk image does not exist, creating the virtualisation disk image...20lb # Formatting '/build/vm-state-lb/tmp.Rsg3X9rR5k', fmt=raw size=107374182421worker2 # Formatting '/build/vm-state-worker2/tmp.vV31HA7IFp', fmt=raw size=107374182422worker1 # Formatting '/build/vm-state-worker1/tmp.Mx5rt263li', fmt=raw size=107374182423lb # mke2fs 1.47.4 (6-Mar-2025)24worker2 # mke2fs 1.47.4 (6-Mar-2025)25worker1 # mke2fs 1.47.4 (6-Mar-2025)26lb # Discarding device blocks: 0/262144 done27worker2 # Discarding device blocks: 0/262144 done28worker1 # Discarding device blocks: 0/262144 done29lb # Creating filesystem with 262144 4k blocks and 65536 inodes30lb # Filesystem UUID: 80d3eab8-8922-4532-88a9-c338635f91f531worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes32client # mke2fs 1.47.4 (6-Mar-2025)33worker2: QEMU running (pid 52)34worker2 # Filesystem UUID: 32fcf0a6-95eb-4478-af00-18bcbf63d3d335client # Discarding device blocks: 0/262144 done36worker2 # Superblock backups stored on blocks:37client # Creating filesystem with 262144 4k blocks and 65536 inodes38worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes39worker2 # 32768, 98304, 163840, 22937640client # Filesystem UUID: 80cc710e-c6b4-4276-acd8-4081619fb46541worker1 # Filesystem UUID: d9b10c6c-1981-4594-a993-de61abb4c08642worker2 # 43client # Superblock backups stored on blocks:44worker1 # Superblock backups stored on blocks:45worker2 # Allocating group tables: 0/8 done46client # 32768, 98304, 163840, 22937647worker1 # 32768, 98304, 163840, 22937648worker2 # Writing inode tables: 0/8 done49client # 50worker1 # 51worker2 # Creating journal (8192 blocks): done52client # Allocating group tables: 0/8 done53worker1 # Allocating group tables: 0/8 done54worker2 # Writing superblocks and filesystem accounting information: 0/8 done55client # Writing inode tables: 0/8 done56worker1 # Writing inode tables: 0/8 done57worker2 # 58client # Creating journal (8192 blocks): done59worker1 # Creating journal (8192 blocks): done60worker2 # Virtualisation disk image created.61client # Writing superblocks and filesystem accounting information: 0/8 done62worker1 # Writing superblocks and filesystem accounting information: 0/8 done63worker2 # Starting virtiofs daemons...64client # 65worker1 # 66worker2 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67client # Virtualisation disk image created.68worker1 # Virtualisation disk image created.69worker2 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70client # Starting virtiofs daemons...71worker1 # Starting virtiofs daemons...72worker2 # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...73client # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74worker1 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75worker2 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76client # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77worker1 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78worker2 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79client # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80worker1 # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...81worker2 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)82client # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83worker1 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84worker2 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether85client # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86worker1 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87worker2 # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...88client # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker1 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90worker2 # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...91client # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...92worker1 # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93worker2 # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests94client # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...95worker1 # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...96worker2 # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests97client # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...98worker1 # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...99worker2 # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests100client # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests101worker1 # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests102(finished: start all VMs, in 0.54 seconds)103lb # Superblock backups stored on blocks:104client # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests105worker1 # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests106client: waiting for unit network-addresses-eth1.service107lb # 32768, 98304, 163840, 229376108client # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests109worker1 # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests110client: waiting for the VM to finish booting111lb # 112lb # Allocating group tables: 0/8 done113lb # Writing inode tables: 0/8 done114lb # Creating journal (8192 blocks): done115lb # Writing superblocks and filesystem accounting information: 0/8 done116lb # 117lb # Virtualisation disk image created.118lb # Starting virtiofs daemons...119lb # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)122lb # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether123lb # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...124lb # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-25T14:59:20Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-25T14:59:20Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-25T14:59:20Z INFO virtiofsd] Client connected, servicing requests131worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker1 # [ 0.000000] KASLR enabled134worker1 # [ 0.000000] random: crng init done135worker1 # [ 0.000000] Machine model: linux,dummy-virt136worker1 # [ 0.000000] efi: UEFI not found.137worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140worker1 # [ 0.000000] Zone ranges:141worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142worker1 # [ 0.000000] DMA32 empty143worker1 # [ 0.000000] Normal empty144worker1 # [ 0.000000] Device empty145worker1 # [ 0.000000] Movable zone start for each node146worker1 # [ 0.000000] Early memory node ranges147worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150worker1 # [ 0.000000] psci: probing for conduit method from DT.151worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.152worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153worker1 # [ 0.000000] psci: Trusted OS migration not required154worker1 # [ 0.000000] psci: SMC Calling Convention v1.1155worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157worker1 # [ 0.000000] Detected PIPT I-cache on CPU0158worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface160worker1 # [ 0.000000] CPU features: detected: Spectre-v4161worker1 # [ 0.000000] CPU features: detected: Spectre-BHB162worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164worker1 # [ 0.000000] alternatives: applying boot alternatives165client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]166lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]167client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026168client # [ 0.000000] KASLR enabled169lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026170client # [ 0.000000] random: crng init done171lb # [ 0.000000] KASLR enabled172client # [ 0.000000] Machine model: linux,dummy-virt173lb # [ 0.000000] random: crng init done174worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r1caxc5yfxdfis0l5rpgjh307ha4z8wh-nixos-system-worker1-test/init regInfo=/nix/store/b307fgar6jj2m9w47c3b4gq7jgva1agm-closure-info/registration console=ttyAMA0,115200n8 console=tty0175client # [ 0.000000] efi: UEFI not found.176lb # [ 0.000000] Machine model: linux,dummy-virt177worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]178lb # [ 0.000000] efi: UEFI not found.179client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT180lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT181worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/b307fgar6jj2m9w47c3b4gq7jgva1agm-closure-info/registration", will be passed to user space.182client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]183worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026184client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]185worker2 # [ 0.000000] KASLR enabled186lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]187worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes188client # [ 0.000000] Zone ranges:189worker2 # [ 0.000000] random: crng init done190lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]191worker2 # [ 0.000000] Machine model: linux,dummy-virt192worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)193lb # [ 0.000000] Zone ranges:194client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]195worker2 # [ 0.000000] efi: UEFI not found.196client # [ 0.000000] DMA32 empty197client # [ 0.000000] Normal empty198lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]199worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)200client # [ 0.000000] Device empty201lb # [ 0.000000] DMA32 empty202worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT203lb # [ 0.000000] Normal empty204client # [ 0.000000] Movable zone start for each node205worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB206lb # [ 0.000000] Device empty207worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]208client # [ 0.000000] Early memory node ranges209worker1 # [ 0.000000] software IO TLB: area num 1.210lb # [ 0.000000] Movable zone start for each node211worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]212lb # [ 0.000000] Early memory node ranges213client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]214worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB215worker2 # [ 0.000000] Zone ranges:216lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]217client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]218worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]219worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)220worker2 # [ 0.000000] DMA32 empty221client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000222worker1 # [ 0.000000] Fallback order for Node 0: 0223lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]224worker2 # [ 0.000000] Normal empty225worker2 # [ 0.000000] Device empty226client # [ 0.000000] psci: probing for conduit method from DT.227lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000228worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216229worker2 # [ 0.000000] Movable zone start for each node230client # [ 0.000000] psci: PSCIv1.3 detected in firmware.231lb # [ 0.000000] psci: probing for conduit method from DT.232worker1 # [ 0.000000] Policy zone: DMA233worker2 # [ 0.000000] Early memory node ranges234client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs235lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.236worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off237worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]238client # [ 0.000000] psci: Trusted OS migration not required239lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs240client # [ 0.000000] psci: SMC Calling Convention v1.1241worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1242lb # [ 0.000000] psci: Trusted OS migration not required243worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]244worker1 # [ 0.000000] allocated 3145728 bytes of page_ext245lb # [ 0.000000] psci: SMC Calling Convention v1.1246worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000247client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)248worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages249worker2 # [ 0.000000] psci: probing for conduit method from DT.250lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)251client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296252worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups253worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.254client # [ 0.000000] Detected PIPT I-cache on CPU0255worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.256lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296257worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs258worker1 # [ 0.000000] rcu: RCU event tracing is enabled.259lb # [ 0.000000] Detected PIPT I-cache on CPU0260worker2 # [ 0.000000] psci: Trusted OS migration not required261client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)262worker2 # [ 0.000000] psci: SMC Calling Convention v1.1263worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.264lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)265client # [ 0.000000] CPU features: detected: GICv3 CPU interface266worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.267client # [ 0.000000] CPU features: detected: Spectre-v4268lb # [ 0.000000] CPU features: detected: GICv3 CPU interface269worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)270worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.271client # [ 0.000000] CPU features: detected: Spectre-BHB272lb # [ 0.000000] CPU features: detected: Spectre-v4273worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.274worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296275lb # [ 0.000000] CPU features: detected: Spectre-BHB276client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38277worker2 # [ 0.000000] Detected PIPT I-cache on CPU0278worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.279lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38280client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23281client # [ 0.000000] alternatives: applying boot alternatives282worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)283worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1284lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23285worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface286lb # [ 0.000000] alternatives: applying boot alternatives287worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.288worker2 # [ 0.000000] CPU features: detected: Spectre-v4289worker2 # [ 0.000000] CPU features: detected: Spectre-BHB290worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.291worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38292worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23293worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.294client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/jgg1bkcnnwpis3sdm7gl6kzqxlwc5ivn-nixos-system-client-test/init regInfo=/nix/store/mlssl3r6fkrygq3m0y463sc9vyikkhk0-closure-info/registration console=ttyAMA0,115200n8 console=tty0295worker2 # [ 0.000000] alternatives: applying boot alternatives296worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0297worker1 # [ 0.000000] GICv3: 256 SPIs implemented298lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hmvppg1fs8xkiccvl0m50n4dxjz92fr-nixos-system-lb-test/init regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration console=ttyAMA0,115200n8 console=tty0299worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented300worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq301client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/mlssl3r6fkrygq3m0y463sc9vyikkhk0-closure-info/registration", will be passed to user space.302worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI303lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration", will be passed to user space.304client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes305worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0306lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes307client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)308worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000309worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/azx0l63xg7ndaj4b8gzinjsgk55i1kf6-nixos-system-worker2-test/init regInfo=/nix/store/wnigwk8c8nbnp7lb6ngki07q6kjn1xls-closure-info/registration console=ttyAMA0,115200n8 console=tty0310lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)311worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]312client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)313lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)314client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB315worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)316worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/wnigwk8c8nbnp7lb6ngki07q6kjn1xls-closure-info/registration", will be passed to user space.317client # [ 0.000000] software IO TLB: area num 1.318lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB319lb # [ 0.000000] software IO TLB: area num 1.320worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes321client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB322worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)323lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB324worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000325worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)326client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)327client # [ 0.000000] Fallback order for Node 0: 0328lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)329worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000330worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)331lb # [ 0.000000] Fallback order for Node 0: 0332client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216333worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.334worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB335client # [ 0.000000] Policy zone: DMA336lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216337worker2 # [ 0.000000] software IO TLB: area num 1.338lb # [ 0.000000] Policy zone: DMA339client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off340worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns341worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB342lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off343client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1344worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).345worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)346lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1347client # [ 0.000000] allocated 3145728 bytes of page_ext348worker2 # [ 0.000000] Fallback order for Node 0: 0349lb # [ 0.000000] allocated 3145728 bytes of page_ext350client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages351worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns352lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages353worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216354client # [ 0.000000] ftrace: allocated 294 pages with 4 groups355worker2 # [ 0.000000] Policy zone: DMA356lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups357client # [ 0.000000] rcu: Hierarchical RCU implementation.358worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns359worker1 # [ 0.000037] arm-pv: using stolen time PV360lb # [ 0.000000] rcu: Hierarchical RCU implementation.361worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off362client # [ 0.000000] rcu: RCU event tracing is enabled.363lb # [ 0.000000] rcu: RCU event tracing is enabled.364worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1365client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.366worker1 # [ 0.000708] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)367worker2 # [ 0.000000] allocated 3145728 bytes of page_ext368lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.369client # [ 0.000000] Trampoline variant of Tasks RCU enabled.370worker1 # [ 0.000934] Console: colour dummy device 80x25371worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages372lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.373client # [ 0.000000] Rude variant of Tasks RCU enabled.374worker1 # [ 0.000942] printk: legacy console [tty0] enabled375lb # [ 0.000000] Rude variant of Tasks RCU enabled.376worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups377client # [ 0.000000] Tracing variant of Tasks RCU enabled.378lb # [ 0.000000] Tracing variant of Tasks RCU enabled.379worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.380client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.381worker1 # [ 0.001133] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)382worker2 # [ 0.000000] rcu: RCU event tracing is enabled.383lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.384worker1 # [ 0.001140] pid_max: default: 32768 minimum: 301385client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1386worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.387lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1388worker1 # [ 0.001228] LSM: initializing lsm=capability,landlock,yama,bpf,ima389worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.390worker1 # [ 0.001421] landlock: Up and running.391client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.392worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.393worker1 # [ 0.001425] Yama: becoming mindful.394lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.395worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.396worker1 # [ 0.002088] LSM support for eBPF active397client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.398lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.399worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.400worker1 # [ 0.002248] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)401client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.402worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1403lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.404worker1 # [ 0.002284] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)405client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0406client # [ 0.000000] GICv3: 256 SPIs implemented407lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0408worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.409client # [ 0.000000] GICv3: 0 Extended SPIs implemented410lb # [ 0.000000] GICv3: 256 SPIs implemented411worker1 # [ 0.004417] rcu: Hierarchical SRCU implementation.412client # [ 0.000000] Root IRQ handler: gic_handle_irq413lb # [ 0.000000] GICv3: 0 Extended SPIs implemented414worker1 # [ 0.004423] rcu: Max phase no-delay instances is 1000.415worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.416lb # [ 0.000000] Root IRQ handler: gic_handle_irq417client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI418worker1 # [ 0.005745] fsl-mc MSI: its@8080000 domain created419lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI420client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0421worker1 # [ 0.005832] EFI services will not be available.422worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.423lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0424worker1 # [ 0.005912] smp: Bringing up secondary CPUs ...425client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000426worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0427worker1 # [ 0.005922] smp: Brought up 1 node, 1 CPU428client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]429lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000430worker2 # [ 0.000000] GICv3: 256 SPIs implemented431worker1 # [ 0.005925] SMP: Total of 1 processors activated.432lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]433worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented434worker1 # [ 0.005930] CPU: All CPU(s) started at EL1435worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq436client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)437lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)438worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI439worker1 # [ 0.005941] CPU features: detected: Branch Target Identification440worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0441client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)442worker1 # [ 0.005946] CPU features: detected: ARMv8.4 Translation Table Level443lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)444worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000445client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000446worker1 # [ 0.005949] CPU features: detected: Instruction cache invalidation not required for I/D coherence447worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]448lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000449client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000450lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000451worker1 # [ 0.005953] CPU features: detected: Data cache clean to the PoU not required for I/D coherence452worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)453client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.454lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.455worker1 # [ 0.005957] CPU features: detected: Common not Private translations456worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)457worker1 # [ 0.005960] CPU features: detected: CRC32 instructions458client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns459lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns460worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000461client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).462worker1 # [ 0.005963] CPU features: detected: Data cache clean to Point of Deep Persistence463lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).464worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000465worker1 # [ 0.005966] CPU features: detected: Data cache clean to Point of Persistence466client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns467worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.468lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns469worker1 # [ 0.005969] CPU features: detected: Data independent timing control (DIT)470client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns471worker1 # [ 0.005973] CPU features: detected: E0PD472lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns473client # [ 0.000032] arm-pv: using stolen time PV474worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns475worker1 # [ 0.005975] CPU features: detected: Enhanced Counter Virtualization476lb # [ 0.000032] arm-pv: using stolen time PV477worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).478worker1 # [ 0.005978] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)479client # [ 0.000381] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)480lb # [ 0.000417] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)481client # [ 0.000571] Console: colour dummy device 80x25482worker1 # [ 0.005981] CPU features: detected: Enhanced Virtualization Traps483worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns484lb # [ 0.000617] Console: colour dummy device 80x25485client # [ 0.000579] printk: legacy console [tty0] enabled486worker1 # [ 0.005984] CPU features: detected: Fine Grained Traps487lb # [ 0.000627] printk: legacy console [tty0] enabled488worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns489worker1 # [ 0.005988] CPU features: detected: Generic authentication (architected QARMA5 algorithm)490worker2 # [ 0.000031] arm-pv: using stolen time PV491client # [ 0.000778] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)492lb # [ 0.000817] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)493worker1 # [ 0.005992] CPU features: detected: RCpc load-acquire (LDAPR)494client # [ 0.000784] pid_max: default: 32768 minimum: 301495lb # [ 0.000824] pid_max: default: 32768 minimum: 301496worker2 # [ 0.000404] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)497worker1 # [ 0.005995] CPU features: detected: LSE atomic instructions498client # [ 0.000871] LSM: initializing lsm=capability,landlock,yama,bpf,ima499lb # [ 0.000929] LSM: initializing lsm=capability,landlock,yama,bpf,ima500worker2 # [ 0.000573] Console: colour dummy device 80x25501client # [ 0.001055] landlock: Up and running.502worker1 # [ 0.005998] CPU features: detected: Privileged Access Never503lb # [ 0.001095] landlock: Up and running.504client # [ 0.001058] Yama: becoming mindful.505worker2 # [ 0.000581] printk: legacy console [tty0] enabled506worker1 # [ 0.006001] CPU features: detected: PMUv3507lb # [ 0.001099] Yama: becoming mindful.508client # [ 0.001532] LSM support for eBPF active509worker1 # [ 0.006004] CPU features: detected: RAS Extension Support510lb # [ 0.001532] LSM support for eBPF active511worker2 # [ 0.000784] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)512client # [ 0.001679] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)513worker1 # [ 0.006007] CPU features: detected: RASv1p1 Extension Support514lb # [ 0.001699] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)515worker2 # [ 0.000792] pid_max: default: 32768 minimum: 301516client # [ 0.001714] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)517worker1 # [ 0.006009] CPU features: detected: Random Number Generator518worker2 # [ 0.000869] LSM: initializing lsm=capability,landlock,yama,bpf,ima519lb # [ 0.001733] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)520client # [ 0.003780] rcu: Hierarchical SRCU implementation.521worker1 # [ 0.006012] CPU features: detected: Speculation barrier (SB)522worker2 # [ 0.001004] landlock: Up and running.523lb # [ 0.003709] rcu: Hierarchical SRCU implementation.524client # [ 0.003786] rcu: Max phase no-delay instances is 1000.525worker2 # [ 0.001006] Yama: becoming mindful.526worker1 # [ 0.006014] CPU features: detected: Stage-2 Force Write-Back527lb # [ 0.003715] rcu: Max phase no-delay instances is 1000.528worker2 # [ 0.001468] LSM support for eBPF active529client # [ 0.005138] fsl-mc MSI: its@8080000 domain created530worker1 # [ 0.006017] CPU features: detected: TLB range maintenance instructions531lb # [ 0.005023] fsl-mc MSI: its@8080000 domain created532client # [ 0.005229] EFI services will not be available.533worker2 # [ 0.001608] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)534lb # [ 0.005124] EFI services will not be available.535client # [ 0.005351] smp: Bringing up secondary CPUs ...536worker1 # [ 0.006021] CPU features: detected: Speculative Store Bypassing Safe (SSBS)537lb # [ 0.005220] smp: Bringing up secondary CPUs ...538client # [ 0.005360] smp: Brought up 1 node, 1 CPU539worker2 # [ 0.001644] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)540worker1 # [ 0.006063] alternatives: applying system-wide alternatives541lb # [ 0.005230] smp: Brought up 1 node, 1 CPU542client # [ 0.005363] SMP: Total of 1 processors activated.543worker2 # [ 0.003691] rcu: Hierarchical SRCU implementation.544lb # [ 0.005233] SMP: Total of 1 processors activated.545client # [ 0.005369] CPU: All CPU(s) started at EL1546worker1 # [ 0.009115] CPU features: detected: BBM Level 2 without TLB conflict abort547worker2 # [ 0.003697] rcu: Max phase no-delay instances is 1000.548lb # [ 0.005238] CPU: All CPU(s) started at EL1549client # [ 0.005380] CPU features: detected: Branch Target Identification550worker2 # [ 0.005019] fsl-mc MSI: its@8080000 domain created551lb # [ 0.005250] CPU features: detected: Branch Target Identification552worker2 # [ 0.005111] EFI services will not be available.553client # [ 0.005385] CPU features: detected: ARMv8.4 Translation Table Level554worker1 # [ 0.009359] Memory: 1405588K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)555lb # [ 0.005254] CPU features: detected: ARMv8.4 Translation Table Level556worker2 # [ 0.005232] smp: Bringing up secondary CPUs ...557worker1 # [ 0.009813] devtmpfs: initialized558worker2 # [ 0.005240] smp: Brought up 1 node, 1 CPU559client # [ 0.005387] CPU features: detected: Instruction cache invalidation not required for I/D coherence560worker1 # [ 0.011980] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)561worker2 # [ 0.005244] SMP: Total of 1 processors activated.562lb # [ 0.005257] CPU features: detected: Instruction cache invalidation not required for I/D coherence563worker2 # [ 0.005249] CPU: All CPU(s) started at EL1564client # [ 0.005391] CPU features: detected: Data cache clean to the PoU not required for I/D coherence565lb # [ 0.005261] CPU features: detected: Data cache clean to the PoU not required for I/D coherence566worker1 # [ 0.012005] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).567worker2 # [ 0.005261] CPU features: detected: Branch Target Identification568client # [ 0.005395] CPU features: detected: Common not Private translations569lb # [ 0.005264] CPU features: detected: Common not Private translations570worker1 # [ 0.012209] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL571client # [ 0.005398] CPU features: detected: CRC32 instructions572worker2 # [ 0.005264] CPU features: detected: ARMv8.4 Translation Table Level573worker1 # [ 0.012213] 0 pages in range for non-PLT usage574lb # [ 0.005268] CPU features: detected: CRC32 instructions575client # [ 0.005401] CPU features: detected: Data cache clean to Point of Deep Persistence576worker1 # [ 0.012214] 508272 pages in range for PLT usage577worker2 # [ 0.005267] CPU features: detected: Instruction cache invalidation not required for I/D coherence578lb # [ 0.005270] CPU features: detected: Data cache clean to Point of Deep Persistence579worker1 # [ 0.012349] pinctrl core: initialized pinctrl subsystem580client # [ 0.005405] CPU features: detected: Data cache clean to Point of Persistence581worker1 # [ 0.013226] DMI not present or invalid.582worker2 # [ 0.005271] CPU features: detected: Data cache clean to the PoU not required for I/D coherence583client # [ 0.005408] CPU features: detected: Data independent timing control (DIT)584worker1 # [ 0.016601] NET: Registered PF_NETLINK/PF_ROUTE protocol family585worker2 # [ 0.005275] CPU features: detected: Common not Private translations586lb # [ 0.005274] CPU features: detected: Data cache clean to Point of Persistence587client # [ 0.005411] CPU features: detected: E0PD588worker2 # [ 0.005278] CPU features: detected: CRC32 instructions589worker1 # [ 0.018883] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations590lb # [ 0.005277] CPU features: detected: Data independent timing control (DIT)591client # [ 0.005414] CPU features: detected: Enhanced Counter Virtualization592lb # [ 0.005280] CPU features: detected: E0PD593worker2 # [ 0.005281] CPU features: detected: Data cache clean to Point of Deep Persistence594worker1 # [ 0.019080] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations595client # [ 0.005417] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)596lb # [ 0.005283] CPU features: detected: Enhanced Counter Virtualization597client # [ 0.005421] CPU features: detected: Enhanced Virtualization Traps598worker1 # [ 0.019376] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations599lb # [ 0.005286] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)600worker2 # [ 0.005285] CPU features: detected: Data cache clean to Point of Persistence601client # [ 0.005424] CPU features: detected: Fine Grained Traps602worker1 # [ 0.019398] audit: initializing netlink subsys (disabled)603lb # [ 0.005290] CPU features: detected: Enhanced Virtualization Traps604worker2 # [ 0.005288] CPU features: detected: Data independent timing control (DIT)605worker1 # [ 0.019957] thermal_sys: Registered thermal governor 'fair_share'606client # [ 0.005428] CPU features: detected: Generic authentication (architected QARMA5 algorithm)607worker2 # [ 0.005292] CPU features: detected: E0PD608lb # [ 0.005292] CPU features: detected: Fine Grained Traps609worker1 # [ 0.019959] thermal_sys: Registered thermal governor 'bang_bang'610client # [ 0.005433] CPU features: detected: RCpc load-acquire (LDAPR)611worker2 # [ 0.005294] CPU features: detected: Enhanced Counter Virtualization612lb # [ 0.005295] CPU features: detected: Generic authentication (architected QARMA5 algorithm)613client # [ 0.005436] CPU features: detected: LSE atomic instructions614worker1 # [ 0.019963] thermal_sys: Registered thermal governor 'step_wise'615worker2 # [ 0.005297] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)616lb # [ 0.005300] CPU features: detected: RCpc load-acquire (LDAPR)617client # [ 0.005438] CPU features: detected: Privileged Access Never618worker1 # [ 0.019965] thermal_sys: Registered thermal governor 'user_space'619client # [ 0.005441] CPU features: detected: PMUv3620lb # [ 0.005303] CPU features: detected: LSE atomic instructions621worker2 # [ 0.005301] CPU features: detected: Enhanced Virtualization Traps622worker1 # [ 0.019971] thermal_sys: Registered thermal governor 'power_allocator'623client # [ 0.005444] CPU features: detected: RAS Extension Support624worker2 # [ 0.005304] CPU features: detected: Fine Grained Traps625lb # [ 0.005306] CPU features: detected: Privileged Access Never626lb # [ 0.005309] CPU features: detected: PMUv3627worker1 # [ 0.020004] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1628client # [ 0.005447] CPU features: detected: RASv1p1 Extension Support629worker2 # [ 0.005307] CPU features: detected: Generic authentication (architected QARMA5 algorithm)630worker1 # [ 0.020012] cpuidle: using governor ladder631lb # [ 0.005311] CPU features: detected: RAS Extension Support632client # [ 0.005450] CPU features: detected: Random Number Generator633worker1 # [ 0.020018] cpuidle: using governor menu634worker2 # [ 0.005312] CPU features: detected: RCpc load-acquire (LDAPR)635lb # [ 0.005314] CPU features: detected: RASv1p1 Extension Support636client # [ 0.005452] CPU features: detected: Speculation barrier (SB)637worker2 # [ 0.005315] CPU features: detected: LSE atomic instructions638worker1 # [ 0.020211] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.639lb # [ 0.005317] CPU features: detected: Random Number Generator640client # [ 0.005455] CPU features: detected: Stage-2 Force Write-Back641worker2 # [ 0.005318] CPU features: detected: Privileged Access Never642worker1 # [ 0.020227] ASID allocator initialised with 65536 entries643lb # [ 0.005319] CPU features: detected: Speculation barrier (SB)644client # [ 0.005458] CPU features: detected: TLB range maintenance instructions645worker2 # [ 0.005320] CPU features: detected: PMUv3646worker1 # [ 0.021478] Serial: AMBA PL011 UART driver647lb # [ 0.005322] CPU features: detected: Stage-2 Force Write-Back648worker2 # [ 0.005323] CPU features: detected: RAS Extension Support649client # [ 0.005463] CPU features: detected: Speculative Store Bypassing Safe (SSBS)650worker1 # [ 0.026781] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1651lb # [ 0.005325] CPU features: detected: TLB range maintenance instructions652worker2 # [ 0.005326] CPU features: detected: RASv1p1 Extension Support653client # [ 0.005503] alternatives: applying system-wide alternatives654worker1 # [ 0.026944] printk: console [ttyAMA0] enabled655worker2 # [ 0.005328] CPU features: detected: Random Number Generator656lb # [ 0.005329] CPU features: detected: Speculative Store Bypassing Safe (SSBS)657client # [ 0.008604] CPU features: detected: BBM Level 2 without TLB conflict abort658worker2 # [ 0.005331] CPU features: detected: Speculation barrier (SB)659lb # [ 0.005366] alternatives: applying system-wide alternatives660worker2 # [ 0.005334] CPU features: detected: Stage-2 Force Write-Back661lb # [ 0.008450] CPU features: detected: BBM Level 2 without TLB conflict abort662client # [ 0.008790] Memory: 1405640K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)663worker2 # [ 0.005336] CPU features: detected: TLB range maintenance instructions664worker2 # [ 0.005341] CPU features: detected: Speculative Store Bypassing Safe (SSBS)665client # [ 0.009212] devtmpfs: initialized666lb # [ 0.008655] Memory: 1405680K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)667lb # [ 0.009047] devtmpfs: initialized668worker2 # [ 0.005378] alternatives: applying system-wide alternatives669client # [ 0.011062] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)670worker2 # [ 0.008501] CPU features: detected: BBM Level 2 without TLB conflict abort671lb # [ 0.010970] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)672client # [ 0.011086] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).673lb # [ 0.010994] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).674client # [ 0.011282] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL675worker2 # [ 0.008673] Memory: 1405676K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)676client # [ 0.011287] 0 pages in range for non-PLT usage677lb # [ 0.011193] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL678worker2 # [ 0.009089] devtmpfs: initialized679worker1 # [ 0.151740] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages680lb # [ 0.011197] 0 pages in range for non-PLT usage681client # [ 0.011288] 508272 pages in range for PLT usage682lb # [ 0.011199] 508272 pages in range for PLT usage683client # [ 0.011387] pinctrl core: initialized pinctrl subsystem684worker1 # [ 0.151761] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page685worker2 # [ 0.010926] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)686client # [ 0.012198] DMI not present or invalid.687lb # [ 0.011317] pinctrl core: initialized pinctrl subsystem688worker1 # [ 0.151766] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages689worker2 # [ 0.010950] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).690lb # [ 0.012120] DMI not present or invalid.691client # [ 0.015451] NET: Registered PF_NETLINK/PF_ROUTE protocol family692worker1 # [ 0.151771] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page693lb # [ 0.015313] NET: Registered PF_NETLINK/PF_ROUTE protocol family694worker2 # [ 0.011138] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL695client # [ 0.018018] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations696worker2 # [ 0.011142] 0 pages in range for non-PLT usage697worker1 # [ 0.151776] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages698lb # [ 0.017689] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations699worker2 # [ 0.011143] 508272 pages in range for PLT usage700client # [ 0.018216] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations701worker1 # [ 0.151780] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page702lb # [ 0.017884] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations703worker2 # [ 0.011297] pinctrl core: initialized pinctrl subsystem704client # [ 0.018428] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations705worker2 # [ 0.012058] DMI not present or invalid.706worker1 # [ 0.151786] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages707lb # [ 0.018092] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations708client # [ 0.018451] audit: initializing netlink subsys (disabled)709worker2 # [ 0.015394] NET: Registered PF_NETLINK/PF_ROUTE protocol family710worker1 # [ 0.151790] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page711lb # [ 0.018116] audit: initializing netlink subsys (disabled)712client # [ 0.019024] thermal_sys: Registered thermal governor 'fair_share'713worker2 # [ 0.017812] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations714lb # [ 0.018753] thermal_sys: Registered thermal governor 'fair_share'715client # [ 0.019026] thermal_sys: Registered thermal governor 'bang_bang'716worker2 # [ 0.018007] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations717lb # [ 0.018755] thermal_sys: Registered thermal governor 'bang_bang'718client # [ 0.019030] thermal_sys: Registered thermal governor 'step_wise'719worker1 # [ 0.159541] fbcon: Taking over console720worker1 # [ 0.159558] ACPI: Interpreter disabled.721lb # [ 0.018759] thermal_sys: Registered thermal governor 'step_wise'722worker2 # [ 0.018208] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations723client # [ 0.019033] thermal_sys: Registered thermal governor 'user_space'724lb # [ 0.018762] thermal_sys: Registered thermal governor 'user_space'725worker2 # [ 0.018230] audit: initializing netlink subsys (disabled)726client # [ 0.019038] thermal_sys: Registered thermal governor 'power_allocator'727lb # [ 0.018767] thermal_sys: Registered thermal governor 'power_allocator'728worker2 # [ 0.018809] thermal_sys: Registered thermal governor 'fair_share'729client # [ 0.019064] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1730worker2 # [ 0.018811] thermal_sys: Registered thermal governor 'bang_bang'731client # [ 0.019073] cpuidle: using governor ladder732lb # [ 0.018802] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1733client # [ 0.019079] cpuidle: using governor menu734lb # [ 0.018811] cpuidle: using governor ladder735worker2 # [ 0.018815] thermal_sys: Registered thermal governor 'step_wise'736lb # [ 0.018816] cpuidle: using governor menu737client # [ 0.019277] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.738worker2 # [ 0.018818] thermal_sys: Registered thermal governor 'user_space'739lb # [ 0.018998] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.740client # [ 0.019294] ASID allocator initialised with 65536 entries741worker2 # [ 0.018823] thermal_sys: Registered thermal governor 'power_allocator'742client # [ 0.020542] Serial: AMBA PL011 UART driver743lb # [ 0.019013] ASID allocator initialised with 65536 entries744lb # [ 0.020257] Serial: AMBA PL011 UART driver745worker2 # [ 0.018848] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1746client # [ 0.025739] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1747worker2 # [ 0.018857] cpuidle: using governor ladder748worker2 # [ 0.018863] cpuidle: using governor menu749client # [ 0.025910] printk: console [ttyAMA0] enabled750lb # [ 0.025626] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1751lb # [ 0.025796] printk: console [ttyAMA0] enabled752worker2 # [ 0.019044] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.753worker1 # [ 0.168596] iommu: Default domain type: Translated754worker2 # [ 0.019059] ASID allocator initialised with 65536 entries755worker2 # [ 0.020316] Serial: AMBA PL011 UART driver756worker1 # [ 0.168608] iommu: DMA domain TLB invalidation policy: strict mode757worker1 # [ 0.170363] SCSI subsystem initialized758worker2 # [ 0.025493] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1759worker2 # [ 0.025623] printk: console [ttyAMA0] enabled760worker1 # [ 0.171068] usbcore: registered new interface driver usbfs761worker1 # [ 0.171108] usbcore: registered new interface driver hub762worker1 # [ 0.171124] usbcore: registered new device driver usb763worker1 # [ 0.171468] pps_core: LinuxPPS API ver. 1 registered764client # [ 0.150288] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages765worker1 # [ 0.171475] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>766worker1 # [ 0.171485] PTP clock support registered767client # [ 0.150309] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page768worker1 # [ 0.171534] EDAC MC: Ver: 3.0.0769lb # [ 0.150506] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages770client # [ 0.150315] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages771lb # [ 0.150524] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page772worker1 # [ 0.176379] scmi_core: SCMI protocol bus registered773client # [ 0.150319] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page774lb # [ 0.150530] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages775worker1 # [ 0.177376] FPGA manager framework776client # [ 0.150324] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages777lb # [ 0.150534] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page778worker1 # [ 0.178348] vgaarb: loaded779client # [ 0.150329] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page780lb # [ 0.150547] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages781worker1 # [ 0.178993] clocksource: Switched to clocksource arch_sys_counter782client # [ 0.150333] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages783lb # [ 0.150552] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page784worker2 # [ 0.151304] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages785client # [ 0.150338] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page786worker2 # [ 0.151321] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page787lb # [ 0.150556] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages788worker2 # [ 0.151327] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages789lb # [ 0.150561] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page790worker2 # [ 0.151332] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page791client # [ 0.158082] fbcon: Taking over console792client # [ 0.158098] ACPI: Interpreter disabled.793worker1 # [ 0.182914] VFS: Disk quotas dquot_6.6.0794worker2 # [ 0.151336] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages795lb # [ 0.158412] fbcon: Taking over console796worker1 # [ 0.182955] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)797worker2 # [ 0.151341] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page798lb # [ 0.158427] ACPI: Interpreter disabled.799worker2 # [ 0.151346] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages800worker2 # [ 0.151350] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page801worker1 # [ 0.186896] netfs: FS-Cache loaded802worker1 # [ 0.187644] pnp: PnP ACPI: disabled803worker2 # [ 0.159173] fbcon: Taking over console804worker2 # [ 0.159206] ACPI: Interpreter disabled.805client # [ 0.167106] iommu: Default domain type: Translated806client # [ 0.167117] iommu: DMA domain TLB invalidation policy: strict mode807lb # [ 0.167035] iommu: Default domain type: Translated808client # [ 0.167486] SCSI subsystem initialized809lb # [ 0.167046] iommu: DMA domain TLB invalidation policy: strict mode810lb # [ 0.167459] SCSI subsystem initialized811client # [ 0.169568] usbcore: registered new interface driver usbfs812client # [ 0.169600] usbcore: registered new interface driver hub813lb # [ 0.169551] usbcore: registered new interface driver usbfs814worker1 # [ 0.191096] NET: Registered PF_INET protocol family815client # [ 0.169614] usbcore: registered new device driver usb816lb # [ 0.169584] usbcore: registered new interface driver hub817client # [ 0.169873] pps_core: LinuxPPS API ver. 1 registered818worker1 # [ 0.191442] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)819lb # [ 0.169600] usbcore: registered new device driver usb820worker2 # [ 0.167828] iommu: Default domain type: Translated821lb # [ 0.169916] pps_core: LinuxPPS API ver. 1 registered822client # [ 0.169879] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>823worker2 # [ 0.167838] iommu: DMA domain TLB invalidation policy: strict mode824worker2 # [ 0.168205] SCSI subsystem initialized825client # [ 0.169888] PTP clock support registered826client # [ 0.169945] EDAC MC: Ver: 3.0.0827lb # [ 0.169923] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>828lb # [ 0.169933] PTP clock support registered829client # [ 0.174893] scmi_core: SCMI protocol bus registered830lb # [ 0.169985] EDAC MC: Ver: 3.0.0831worker2 # [ 0.170292] usbcore: registered new interface driver usbfs832worker2 # [ 0.170326] usbcore: registered new interface driver hub833lb # [ 0.174985] scmi_core: SCMI protocol bus registered834client # [ 0.175891] FPGA manager framework835worker2 # [ 0.170342] usbcore: registered new device driver usb836client # [ 0.176884] vgaarb: loaded837worker2 # [ 0.170652] pps_core: LinuxPPS API ver. 1 registered838lb # [ 0.176035] FPGA manager framework839lb # [ 0.177045] vgaarb: loaded840client # [ 0.177526] clocksource: Switched to clocksource arch_sys_counter841worker2 # [ 0.170658] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>842worker2 # [ 0.170669] PTP clock support registered843lb # [ 0.177701] clocksource: Switched to clocksource arch_sys_counter844worker2 # [ 0.170719] EDAC MC: Ver: 3.0.0845worker2 # [ 0.175637] scmi_core: SCMI protocol bus registered846worker2 # [ 0.176655] FPGA manager framework847client # [ 0.181466] VFS: Disk quotas dquot_6.6.0848worker2 # [ 0.177660] vgaarb: loaded849client # [ 0.181507] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)850lb # [ 0.181630] VFS: Disk quotas dquot_6.6.0851worker2 # [ 0.178318] clocksource: Switched to clocksource arch_sys_counter852lb # [ 0.181668] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)853client # [ 0.185443] netfs: FS-Cache loaded854client # [ 0.186049] pnp: PnP ACPI: disabled855lb # [ 0.185662] netfs: FS-Cache loaded856lb # [ 0.186280] pnp: PnP ACPI: disabled857worker2 # [ 0.183289] VFS: Disk quotas dquot_6.6.0858worker2 # [ 0.183336] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)859worker2 # [ 0.185031] netfs: FS-Cache loaded860worker2 # [ 0.185156] pnp: PnP ACPI: disabled861client # [ 0.189632] NET: Registered PF_INET protocol family862client # [ 0.189900] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)863worker2 # [ 0.189184] NET: Registered PF_INET protocol family864lb # [ 0.189844] NET: Registered PF_INET protocol family865worker2 # [ 0.189471] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)866lb # [ 0.190114] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)867worker1 # [ 0.221775] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)868worker1 # [ 0.221826] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)869worker1 # [ 0.221854] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)870worker1 # [ 0.221926] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)871worker1 # [ 0.222070] TCP: Hash tables configured (established 16384 bind 16384)872worker1 # [ 0.222161] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)873worker1 # [ 0.222197] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)874worker1 # [ 0.222233] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)875worker1 # [ 0.222326] NET: Registered PF_UNIX/PF_LOCAL protocol family876worker1 # [ 0.222347] NET: Registered PF_XDP protocol family877worker1 # [ 0.222364] PCI: CLS 0 bytes, default 64878worker1 # [ 0.222603] Trying to unpack rootfs image as initramfs...879worker1 # [ 0.241216] kvm [1]: HYP mode not available880client # [ 0.220501] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881client # [ 0.220556] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882worker2 # [ 0.220189] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)883lb # [ 0.221972] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)884client # [ 0.220585] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)885worker2 # [ 0.220238] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)886lb # [ 0.222021] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)887client # [ 0.220660] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)888worker2 # [ 0.220267] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)889lb # [ 0.222050] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)890client # [ 0.220806] TCP: Hash tables configured (established 16384 bind 16384)891lb # [ 0.222129] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)892worker2 # [ 0.220352] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)893client # [ 0.220895] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)894worker2 # [ 0.220495] TCP: Hash tables configured (established 16384 bind 16384)895lb # [ 0.222272] TCP: Hash tables configured (established 16384 bind 16384)896client # [ 0.220934] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)897lb # [ 0.222362] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)898worker2 # [ 0.220587] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)899client # [ 0.220972] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)900lb # [ 0.222400] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)901worker2 # [ 0.220624] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)902client # [ 0.221067] NET: Registered PF_UNIX/PF_LOCAL protocol family903lb # [ 0.222437] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)904worker2 # [ 0.220661] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)905client # [ 0.221089] NET: Registered PF_XDP protocol family906lb # [ 0.222554] NET: Registered PF_UNIX/PF_LOCAL protocol family907worker2 # [ 0.220797] NET: Registered PF_UNIX/PF_LOCAL protocol family908client # [ 0.221107] PCI: CLS 0 bytes, default 64909lb # [ 0.222582] NET: Registered PF_XDP protocol family910worker2 # [ 0.220838] NET: Registered PF_XDP protocol family911client # [ 0.221370] Trying to unpack rootfs image as initramfs...912lb # [ 0.222601] PCI: CLS 0 bytes, default 64913worker2 # [ 0.220856] PCI: CLS 0 bytes, default 64914lb # [ 0.222868] Trying to unpack rootfs image as initramfs...915worker2 # [ 0.221102] Trying to unpack rootfs image as initramfs...916worker2 # [ 0.236527] kvm [1]: HYP mode not available917client # [ 0.239820] kvm [1]: HYP mode not available918lb # [ 0.239880] kvm [1]: HYP mode not available919worker1 # [ 0.347667] Initialise system trusted keyrings920worker1 # [ 0.348450] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker1 # [ 0.349759] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker1 # [ 0.350562] 9p: Installing v9fs 9p2000 file system support923client # [ 0.350644] Initialise system trusted keyrings924worker2 # [ 0.346918] Initialise system trusted keyrings925client # [ 0.351501] workingset: timestamp_bits=42 max_order=19 bucket_order=0926lb # [ 0.351407] Initialise system trusted keyrings927worker2 # [ 0.347713] workingset: timestamp_bits=42 max_order=19 bucket_order=0928client # [ 0.352779] squashfs: version 4.0 (2009/01/31) Phillip Lougher929lb # [ 0.352197] workingset: timestamp_bits=42 max_order=19 bucket_order=0930worker2 # [ 0.349287] squashfs: version 4.0 (2009/01/31) Phillip Lougher931worker2 # [ 0.350097] 9p: Installing v9fs 9p2000 file system support932lb # [ 0.353586] squashfs: version 4.0 (2009/01/31) Phillip Lougher933worker1 # [ 0.379447] Key type asymmetric registered934worker1 # [ 0.379477] Asymmetric key parser 'x509' registered935worker1 # [ 0.379556] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)936client # [ 0.353606] 9p: Installing v9fs 9p2000 file system support937worker1 # [ 0.381781] io scheduler mq-deadline registered938worker1 # [ 0.381794] io scheduler kyber registered939lb # [ 0.354451] 9p: Installing v9fs 9p2000 file system support940worker1 # [ 0.395215] pl061_gpio 9030000.pl061: PL061 GPIO chip registered941worker1 # [ 0.395946] ledtrig-cpu: registered to indicate activity on CPUs942worker1 # [ 0.396356] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:943client # [ 0.381880] Key type asymmetric registered944worker1 # [ 0.396373] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000945client # [ 0.381912] Asymmetric key parser 'x509' registered946worker1 # [ 0.396386] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000947worker2 # [ 0.378941] Key type asymmetric registered948client # [ 0.381987] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)949lb # [ 0.382667] Key type asymmetric registered950worker2 # [ 0.378974] Asymmetric key parser 'x509' registered951client # [ 0.384242] io scheduler mq-deadline registered952lb # [ 0.382697] Asymmetric key parser 'x509' registered953worker1 # [ 0.396394] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000954client # [ 0.384254] io scheduler kyber registered955worker2 # [ 0.379061] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)956lb # [ 0.382773] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)957worker1 # [ 0.396435] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits958worker2 # [ 0.381274] io scheduler mq-deadline registered959lb # [ 0.384967] io scheduler mq-deadline registered960worker2 # [ 0.381286] io scheduler kyber registered961lb # [ 0.384979] io scheduler kyber registered962worker1 # [ 0.396463] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]963worker1 # [ 0.396547] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00964worker1 # [ 0.396557] pci_bus 0000:00: root bus resource [bus 00-ff]965worker1 # [ 0.396563] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]966worker1 # [ 0.396568] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]967worker1 # [ 0.396573] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]968worker1 # [ 0.396663] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint969worker1 # [ 0.397123] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint970worker1 # [ 0.397315] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]971worker1 # [ 0.397332] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]972client # [ 0.393703] pl061_gpio 9030000.pl061: PL061 GPIO chip registered973worker1 # [ 0.397363] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]974worker1 # [ 0.397379] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]975worker1 # [ 0.397850] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint976client # [ 0.395198] ledtrig-cpu: registered to indicate activity on CPUs977worker1 # [ 0.398041] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]978client # [ 0.395698] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:979worker1 # [ 0.398057] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]980worker1 # [ 0.398087] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]981client # [ 0.395718] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000982worker1 # [ 0.398549] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint983client # [ 0.395732] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000984worker1 # [ 0.398736] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]985worker1 # [ 0.398752] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]986client # [ 0.395742] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000987worker1 # [ 0.398783] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]988client # [ 0.395765] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits989worker1 # [ 0.399323] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint990client # [ 0.395792] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]991worker1 # [ 0.399510] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]992client # [ 0.395878] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00993worker1 # [ 0.399526] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]994client # [ 0.395888] pci_bus 0000:00: root bus resource [bus 00-ff]995worker1 # [ 0.399555] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996worker2 # [ 0.394710] pl061_gpio 9030000.pl061: PL061 GPIO chip registered997client # [ 0.395895] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]998lb # [ 0.398494] pl061_gpio 9030000.pl061: PL061 GPIO chip registered999worker1 # [ 0.399574] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1000worker2 # [ 0.395427] ledtrig-cpu: registered to indicate activity on CPUs1001lb # [ 0.399126] ledtrig-cpu: registered to indicate activity on CPUs1002client # [ 0.395900] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1003worker1 # [ 0.400043] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1004worker2 # [ 0.395850] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1005lb # [ 0.399518] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1006client # [ 0.395906] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1007worker1 # [ 0.400242] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1008worker2 # [ 0.395867] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001009lb # [ 0.399534] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001010client # [ 0.395966] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1011worker1 # [ 0.400272] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1012client # [ 0.396423] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1013lb # [ 0.399555] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001014worker2 # [ 0.395880] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001015worker1 # [ 0.400735] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1016client # [ 0.396620] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1017worker1 # [ 0.400926] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1018lb # [ 0.399564] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001019worker2 # [ 0.395889] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001020client # [ 0.396640] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1021worker1 # [ 0.400957] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1022worker2 # [ 0.395912] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1023lb # [ 0.399584] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1024client # [ 0.396671] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025worker1 # [ 0.401363] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1026client # [ 0.396688] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1027lb # [ 0.399611] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1028worker2 # [ 0.395936] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1029worker1 # [ 0.401552] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1030client # [ 0.397164] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1031lb # [ 0.399688] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001032worker2 # [ 0.396021] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001033worker1 # [ 0.401819] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1034client # [ 0.397354] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1035lb # [ 0.399698] pci_bus 0000:00: root bus resource [bus 00-ff]1036worker2 # [ 0.396031] pci_bus 0000:00: root bus resource [bus 00-ff]1037worker1 # [ 0.402014] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1038client # [ 0.397371] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1039lb # [ 0.399703] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1040worker2 # [ 0.396037] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1041worker1 # [ 0.402045] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042client # [ 0.397401] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1043lb # [ 0.399709] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1044worker2 # [ 0.396043] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1045worker1 # [ 0.402506] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1046lb # [ 0.399714] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1047worker2 # [ 0.396048] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1048worker1 # [ 0.402694] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1049worker2 # [ 0.396113] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1050lb # [ 0.399782] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1051worker1 # [ 0.402724] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1052worker2 # [ 0.396573] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1053lb # [ 0.400230] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1054worker2 # [ 0.396769] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1055lb # [ 0.400422] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1056worker2 # [ 0.396786] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1057lb # [ 0.400439] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1058worker2 # [ 0.396817] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059lb # [ 0.400470] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1060worker2 # [ 0.396834] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1061lb # [ 0.400486] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1062worker2 # [ 0.397321] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1063lb # [ 0.400952] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1064worker2 # [ 0.397513] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1065lb # [ 0.401143] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1066worker2 # [ 0.397529] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1067lb # [ 0.401159] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1068worker2 # [ 0.397560] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069lb # [ 0.401189] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070worker2 # [ 0.398032] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1071lb # [ 0.401661] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1072worker2 # [ 0.398224] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1073lb # [ 0.401868] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1074worker2 # [ 0.398241] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1075lb # [ 0.401884] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1076worker2 # [ 0.398272] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077lb # [ 0.401915] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078worker2 # [ 0.398767] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1079lb # [ 0.402398] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1080worker2 # [ 0.398963] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1081lb # [ 0.402668] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1082worker2 # [ 0.398980] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1083lb # [ 0.402689] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1084worker2 # [ 0.399011] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085lb # [ 0.402720] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086client # [ 0.426037] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1087worker2 # [ 0.399033] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1088lb # [ 0.402740] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1089worker1 # [ 0.451386] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1090client # [ 0.426254] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1091worker2 # [ 0.399631] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1092worker1 # [ 0.451615] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1093lb # [ 0.403231] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1094client # [ 0.426273] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1095worker2 # [ 0.399831] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1096lb # [ 0.403425] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1097worker1 # [ 0.451648] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098client # [ 0.426305] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1099worker2 # [ 0.399862] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100lb # [ 0.403457] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101worker1 # [ 0.452152] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1102client # [ 0.426782] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1103worker1 # [ 0.452441] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1104worker2 # [ 0.400369] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1105lb # [ 0.403923] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1106client # [ 0.426975] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1107worker1 # [ 0.452459] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1108worker2 # [ 0.400564] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1109lb # [ 0.404119] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1110client # [ 0.426992] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1111worker1 # [ 0.452496] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1112worker2 # [ 0.400595] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1113client # [ 0.427023] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114lb # [ 0.404149] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115client # [ 0.427043] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1116worker2 # [ 0.401002] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1117worker1 # [ 0.453012] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1118lb # [ 0.404583] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1119worker1 # [ 0.453204] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1120worker2 # [ 0.401194] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1121lb # [ 0.404773] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1122client # [ 0.427618] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1123worker1 # [ 0.453221] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1124client # [ 0.427815] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1125worker2 # [ 0.401457] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1126lb # [ 0.405031] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1127worker1 # [ 0.453251] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1128worker2 # [ 0.401655] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1129lb # [ 0.405224] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1130client # [ 0.427846] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1131worker1 # [ 0.453875] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1132worker2 # [ 0.401686] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133lb # [ 0.405255] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1134client # [ 0.428321] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1135worker1 # [ 0.453886] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1136client # [ 0.428517] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1137worker2 # [ 0.402155] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1138worker1 # [ 0.453892] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1139client # [ 0.428548] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1140worker1 # [ 0.453939] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1141client # [ 0.429000] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1142client # [ 0.429189] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1143worker1 # [ 0.453988] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1144client # [ 0.429443] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1145worker1 # [ 0.454038] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1146client # [ 0.429657] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1147worker1 # [ 0.454091] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1148client # [ 0.429688] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149worker1 # [ 0.454141] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1150client # [ 0.430165] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1151worker1 # [ 0.454191] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1152client # [ 0.430361] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1153worker1 # [ 0.454240] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1154client # [ 0.430392] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1155worker1 # [ 0.454288] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1156client # [ 0.430869] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1157client # [ 0.431065] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1158worker1 # [ 0.454336] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1159client # [ 0.431096] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1160worker1 # [ 0.454411] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1161client # [ 0.431597] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1162worker1 # [ 0.454459] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1163client # [ 0.431876] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1164worker1 # [ 0.454481] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1165worker2 # [ 0.446427] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1166client # [ 0.431894] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1167lb # [ 0.449817] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1168worker1 # [ 0.454503] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1169worker2 # [ 0.446477] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1170client # [ 0.431924] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171lb # [ 0.450042] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1172worker1 # [ 0.454525] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1173worker2 # [ 0.447007] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1174client # [ 0.432398] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1175lb # [ 0.450075] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1176worker1 # [ 0.454547] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1177worker2 # [ 0.447279] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1178client # [ 0.432587] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1179worker1 # [ 0.454569] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1180lb # [ 0.450674] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1181client # [ 0.432604] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1182worker2 # [ 0.447311] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1183lb # [ 0.450869] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1184worker1 # [ 0.454591] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1185client # [ 0.432635] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1186worker2 # [ 0.447799] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1187worker1 # [ 0.454618] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1188lb # [ 0.450900] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1189worker2 # [ 0.448092] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1190client # [ 0.433254] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1191worker1 # [ 0.454641] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1192lb # [ 0.451377] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1193worker2 # [ 0.448110] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1194client # [ 0.433265] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1195lb # [ 0.451661] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1196worker1 # [ 0.454667] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1197worker2 # [ 0.448141] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1198client # [ 0.433272] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1199lb # [ 0.451678] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1200worker1 # [ 0.454690] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1201worker2 # [ 0.448615] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1202lb # [ 0.451709] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1203client # [ 0.433319] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1204worker1 # [ 0.454712] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1205worker2 # [ 0.448806] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1206worker1 # [ 0.454734] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1207lb # [ 0.452179] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1208client # [ 0.433368] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1209worker2 # [ 0.448822] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1210lb # [ 0.452369] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1211worker1 # [ 0.454755] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1212client # [ 0.433416] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1213worker2 # [ 0.448853] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1214worker1 # [ 0.454777] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1215lb # [ 0.452385] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1216worker2 # [ 0.449442] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1217client # [ 0.433464] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1218worker1 # [ 0.454798] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1219lb # [ 0.452416] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1220worker2 # [ 0.449454] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1221worker1 # [ 0.454819] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1222client # [ 0.433513] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1223lb # [ 0.453025] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1224worker1 # [ 0.454842] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1225worker2 # [ 0.449460] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1226lb # [ 0.453037] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1227worker1 # [ 0.454868] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1228worker2 # [ 0.449507] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1229lb # [ 0.453042] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1230worker1 # [ 0.454878] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1231worker2 # [ 0.449556] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1232worker1 # [ 0.454882] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1233lb # [ 0.453089] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1234worker1 # [ 0.455767] pci 0000:00:07.0: enabling device (0000 -> 0002)1235worker2 # [ 0.449605] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1236lb # [ 0.453138] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1237worker2 # [ 0.449654] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1238lb # [ 0.453187] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1239worker2 # [ 0.449704] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1240lb # [ 0.453238] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1241worker2 # [ 0.449753] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1242lb # [ 0.453287] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1243worker2 # [ 0.449803] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1244client # [ 0.473638] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1245lb # [ 0.453337] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1246worker2 # [ 0.449851] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1247client # [ 0.473705] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1248lb # [ 0.453385] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1249worker2 # [ 0.449899] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1250client # [ 0.473756] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1251lb # [ 0.453434] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1252worker2 # [ 0.449963] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1253client # [ 0.473805] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1254lb # [ 0.453481] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1255worker2 # [ 0.450011] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1256client # [ 0.473924] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1257lb # [ 0.453550] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1258worker2 # [ 0.450033] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1259client # [ 0.473976] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1260lb # [ 0.453597] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1261worker2 # [ 0.450055] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1262client # [ 0.474001] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1263lb # [ 0.453619] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1264worker2 # [ 0.450076] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1265client # [ 0.474025] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1266lb # [ 0.453640] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1267worker2 # [ 0.450098] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1268client # [ 0.474049] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1269lb # [ 0.453662] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1270worker2 # [ 0.450120] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1271client # [ 0.474073] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1272lb # [ 0.453684] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1273worker2 # [ 0.450142] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1274client # [ 0.474097] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1275lb # [ 0.453723] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1276worker2 # [ 0.450169] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1277client # [ 0.474119] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1278lb # [ 0.453745] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1279worker2 # [ 0.450191] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1280client # [ 0.474147] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1281lb # [ 0.453772] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1282worker2 # [ 0.450214] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1283client # [ 0.474170] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1284lb # [ 0.453794] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1285worker2 # [ 0.450237] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1286client # [ 0.474193] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1287lb # [ 0.453817] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1288worker2 # [ 0.450259] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1289client # [ 0.474217] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1290lb # [ 0.453840] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1291worker2 # [ 0.450281] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1292client # [ 0.474240] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1293lb # [ 0.453863] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1294worker2 # [ 0.450303] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1295client # [ 0.474263] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1296lb # [ 0.453885] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1297worker2 # [ 0.450346] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1298client # [ 0.474285] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1299lb # [ 0.453912] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1300worker2 # [ 0.450368] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1301client # [ 0.474308] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1302lb # [ 0.453935] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1303worker2 # [ 0.450390] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1304client # [ 0.474330] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1305lb # [ 0.453957] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1306worker2 # [ 0.450411] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1307client # [ 0.474352] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1308lb # [ 0.453979] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1309worker2 # [ 0.450437] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1310client # [ 0.474374] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1311lb # [ 0.454001] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1312worker2 # [ 0.450447] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1313client # [ 0.474406] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1314lb # [ 0.454029] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1315worker2 # [ 0.450452] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1316client # [ 0.474416] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1317lb # [ 0.454038] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1318worker2 # [ 0.451318] pci 0000:00:07.0: enabling device (0000 -> 0002)1319client # [ 0.474421] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1320worker1 # [ 0.522065] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1321lb # [ 0.454043] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1322client # [ 0.475273] pci 0000:00:07.0: enabling device (0000 -> 0002)1323lb # [ 0.454898] pci 0000:00:07.0: enabling device (0000 -> 0002)1324worker1 # [ 0.525462] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1325worker1 # [ 0.531218] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1326worker1 # [ 0.534031] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1327worker1 # [ 0.544449] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1328client # [ 0.524171] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1329lb # [ 0.524519] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1330client # [ 0.527585] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1331worker2 # [ 0.524987] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1332lb # [ 0.531081] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1333worker2 # [ 0.528163] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1334lb # [ 0.533088] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1335worker2 # [ 0.530195] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1336worker1 # [ 0.555219] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1337client # [ 0.537768] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1338worker1 # [ 0.557092] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1339client # [ 0.540126] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1340worker1 # [ 0.558903] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1341worker1 # [ 0.560974] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1342worker1 # [ 0.562904] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1343worker2 # [ 0.539757] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1344lb # [ 0.543170] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1345worker2 # [ 0.541856] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1346lb # [ 0.545256] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1347worker2 # [ 0.545663] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1348worker1 # [ 0.576775] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1349client # [ 0.550151] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1350client # [ 0.552366] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1351lb # [ 0.555463] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1352worker2 # [ 0.552087] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1353client # [ 0.554259] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1354lb # [ 0.557305] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1355worker2 # [ 0.554017] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1356client # [ 0.556373] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1357lb # [ 0.560906] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1358client # [ 0.566555] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1359client # [ 0.568489] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1360worker2 # [ 0.563798] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1361worker1 # [ 0.594125] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1362lb # [ 0.567343] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1363worker2 # [ 0.565623] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1364lb # [ 0.569160] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1365worker1 # [ 0.597046] msm_serial: driver initialized1366worker1 # [ 0.597234] SuperH (H)SCI(F) driver initialized1367worker1 # [ 0.597291] STM32 USART driver initialized1368client # [ 0.578988] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1369worker2 # [ 0.575905] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1370lb # [ 0.579518] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1371client # [ 0.590789] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1372worker2 # [ 0.587588] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1373client # [ 0.593383] msm_serial: driver initialized1374worker2 # [ 0.590315] msm_serial: driver initialized1375lb # [ 0.595070] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1376client # [ 0.594129] SuperH (H)SCI(F) driver initialized1377client # [ 0.594190] STM32 USART driver initialized1378worker2 # [ 0.591036] SuperH (H)SCI(F) driver initialized1379worker2 # [ 0.591093] STM32 USART driver initialized1380lb # [ 0.597753] msm_serial: driver initialized1381lb # [ 0.597931] SuperH (H)SCI(F) driver initialized1382lb # [ 0.597987] STM32 USART driver initialized1383worker1 # [ 0.630135] loop: module loaded1384worker1 # [ 0.630347] virtio_blk virtio2: 1/0/0 default/read/poll queues1385worker1 # [ 0.632528] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1386worker1 # [ 0.643830] megasas: 07.734.00.00-rc11387client # [ 0.626607] loop: module loaded1388worker1 # [ 0.644661] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1389client # [ 0.626811] virtio_blk virtio2: 1/0/0 default/read/poll queues1390worker2 # [ 0.623486] loop: module loaded1391worker1 # [ 0.646603] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001392worker2 # [ 0.623679] virtio_blk virtio2: 1/0/0 default/read/poll queues1393client # [ 0.627922] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1394worker1 # [ 0.646633] Intel/Sharp Extended Query Table at 0x00311395worker2 # [ 0.624485] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1396worker1 # [ 0.648357] Using buffer write method1397worker1 # [ 0.648432] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1398lb # [ 0.630340] loop: module loaded1399lb # [ 0.630732] virtio_blk virtio2: 1/0/0 default/read/poll queues1400worker1 # [ 0.650177] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001401worker1 # [ 0.650204] Intel/Sharp Extended Query Table at 0x00311402lb # [ 0.631549] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1403worker2 # [ 0.635043] megasas: 07.734.00.00-rc11404lb # [ 0.638411] megasas: 07.734.00.00-rc11405worker2 # [ 0.635910] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1406lb # [ 0.639146] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1407worker1 # [ 0.669108] Using buffer write method1408worker2 # [ 0.638030] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001409worker1 # [ 0.669155] Concatenating MTD devices:1410worker2 # [ 0.638059] Intel/Sharp Extended Query Table at 0x00311411worker1 # [ 0.669159] (0): "0.flash"1412worker1 # [ 0.669163] (1): "0.flash"1413worker1 # [ 0.669174] into device "0.flash"1414client # [ 0.642258] megasas: 07.734.00.00-rc11415client # [ 0.643000] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1416client # [ 0.645279] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417client # [ 0.645309] Intel/Sharp Extended Query Table at 0x00311418client # [ 0.647079] Using buffer write method1419worker2 # [ 0.647829] Using buffer write method1420client # [ 0.647155] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1421lb # [ 0.651019] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001422worker2 # [ 0.647920] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1423client # [ 0.649051] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001424lb # [ 0.651057] Intel/Sharp Extended Query Table at 0x00311425lb # [ 0.652751] Using buffer write method1426client # [ 0.649077] Intel/Sharp Extended Query Table at 0x00311427worker2 # [ 0.649542] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001428worker2 # [ 0.649568] Intel/Sharp Extended Query Table at 0x00311429lb # [ 0.652836] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1430worker2 # [ 0.663614] Using buffer write method1431client # [ 0.668346] Using buffer write method1432worker2 # [ 0.663655] Concatenating MTD devices:1433worker2 # [ 0.663659] (0): "0.flash"1434client # [ 0.668386] Concatenating MTD devices:1435worker2 # [ 0.663663] (1): "0.flash"1436client # [ 0.668391] (0): "0.flash"1437worker2 # [ 0.663672] into device "0.flash"1438client # [ 0.668395] (1): "0.flash"1439client # [ 0.668404] into device "0.flash"1440lb # [ 0.666502] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001441lb # [ 0.666609] Intel/Sharp Extended Query Table at 0x00311442lb # [ 0.668673] Using buffer write method1443lb # [ 0.668712] Concatenating MTD devices:1444lb # [ 0.668716] (0): "0.flash"1445lb # [ 0.668721] (1): "0.flash"1446lb # [ 0.668732] into device "0.flash"1447worker1 # [ 0.956185] Freeing initrd memory: 26976K1448worker1 # [ 0.962436] tun: Universal TUN/TAP device driver, 1.61449worker1 # [ 0.966335] thunder_xcv, ver 1.01450worker1 # [ 0.966383] thunder_bgx, ver 1.01451worker1 # [ 0.966406] nicpf, ver 1.01452worker1 # [ 0.966993] e1000: Intel(R) PRO/1000 Network Driver1453worker1 # [ 0.968919] e1000: Copyright (c) 1999-2006 Intel Corporation.1454worker1 # [ 0.968956] e1000e: Intel(R) PRO/1000 Network Driver1455worker1 # [ 0.968965] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1456worker1 # [ 0.968996] igb: Intel(R) Gigabit Ethernet Network Driver1457worker1 # [ 0.969002] igb: Copyright (c) 2007-2014 Intel Corporation.1458worker2 # [ 0.947507] Freeing initrd memory: 26944K1459worker1 # [ 0.969025] igbvf: Intel(R) Gigabit Virtual Function Network Driver1460worker1 # [ 0.969031] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1461worker1 # [ 0.969174] sky2: driver version 1.301462worker1 # [ 0.970805] usbcore: registered new interface driver usb-storage1463worker1 # [ 0.970858] usbcore: registered new interface driver usbserial_generic1464worker1 # [ 0.970873] usbserial: USB Serial support registered for generic1465worker1 # [ 0.971653] ehci-pci 0000:00:07.0: EHCI Host Controller1466client # [ 0.954514] Freeing initrd memory: 26972K1467worker1 # [ 0.971679] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11468worker1 # [ 0.971844] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001469worker2 # [ 0.953719] tun: Universal TUN/TAP device driver, 1.61470worker1 # [ 0.982725] hv_vmbus: registering driver hyperv_keyboard1471client # [ 0.960635] tun: Universal TUN/TAP device driver, 1.61472worker1 # [ 0.983052] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001473worker1 # [ 0.983435] hub 1-0:1.0: USB hub found1474worker1 # [ 0.983456] hub 1-0:1.0: 6 ports detected1475worker2 # [ 0.957427] thunder_xcv, ver 1.01476worker2 # [ 0.957472] thunder_bgx, ver 1.01477worker2 # [ 0.957503] nicpf, ver 1.01478worker1 # [ 0.986853] rtc-pl031 9010000.pl031: registered as rtc01479worker2 # [ 0.958050] e1000: Intel(R) PRO/1000 Network Driver1480worker1 # [ 0.986883] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T14:59:21 UTC (1790348361)1481worker2 # [ 0.958057] e1000: Copyright (c) 1999-2006 Intel Corporation.1482worker2 # [ 0.958082] e1000e: Intel(R) PRO/1000 Network Driver1483client # [ 0.964457] thunder_xcv, ver 1.01484client # [ 0.964504] thunder_bgx, ver 1.01485worker1 # [ 0.989052] i2c_dev: i2c /dev entries driver1486worker2 # [ 0.958090] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1487client # [ 0.964526] nicpf, ver 1.01488worker2 # [ 0.958122] igb: Intel(R) Gigabit Ethernet Network Driver1489client # [ 0.965084] e1000: Intel(R) PRO/1000 Network Driver1490lb # [ 0.966189] Freeing initrd memory: 26948K1491worker2 # [ 0.958128] igb: Copyright (c) 2007-2014 Intel Corporation.1492client # [ 0.965092] e1000: Copyright (c) 1999-2006 Intel Corporation.1493client # [ 0.965116] e1000e: Intel(R) PRO/1000 Network Driver1494worker2 # [ 0.958167] igbvf: Intel(R) Gigabit Virtual Function Network Driver1495client # [ 0.965124] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1496worker2 # [ 0.958174] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1497worker2 # [ 0.958309] sky2: driver version 1.301498client # [ 0.965155] igb: Intel(R) Gigabit Ethernet Network Driver1499client # [ 0.965160] igb: Copyright (c) 2007-2014 Intel Corporation.1500client # [ 0.965183] igbvf: Intel(R) Gigabit Virtual Function Network Driver1501worker1 # [ 0.992917] sdhci: Secure Digital Host Controller Interface driver1502worker1 # [ 0.992930] sdhci: Copyright(c) Pierre Ossman1503client # [ 0.965189] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1504client # [ 0.965322] sky2: driver version 1.301505worker1 # [ 0.993201] Synopsys Designware Multimedia Card Interface Driver1506worker1 # [ 0.993583] sdhci-pltfm: SDHCI platform and OF driver helper1507lb # [ 0.972440] tun: Universal TUN/TAP device driver, 1.61508worker2 # [ 0.967623] usbcore: registered new interface driver usb-storage1509worker2 # [ 0.967736] usbcore: registered new interface driver usbserial_generic1510worker1 # [ 0.997983] hid: raw HID events driver (C) Jiri Kosina1511worker2 # [ 0.967751] usbserial: USB Serial support registered for generic1512worker1 # [ 0.998246] usbcore: registered new interface driver usbhid1513worker1 # [ 0.998253] usbhid: USB HID core driver1514worker2 # [ 0.968349] hv_vmbus: registering driver hyperv_keyboard1515client # [ 0.974507] usbcore: registered new interface driver usb-storage1516worker2 # [ 0.969663] ehci-pci 0000:00:07.0: EHCI Host Controller1517client # [ 0.974561] usbcore: registered new interface driver usbserial_generic1518worker2 # [ 0.969690] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11519lb # [ 0.976325] thunder_xcv, ver 1.01520client # [ 0.974574] usbserial: USB Serial support registered for generic1521lb # [ 0.976380] thunder_bgx, ver 1.01522worker2 # [ 0.969859] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001523lb # [ 0.976404] nicpf, ver 1.01524client # [ 0.975159] hv_vmbus: registering driver hyperv_keyboard1525worker1 # [ 1.001338] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1526lb # [ 0.976990] e1000: Intel(R) PRO/1000 Network Driver1527client # [ 0.976522] ehci-pci 0000:00:07.0: EHCI Host Controller1528worker1 # [ 1.002920] drop_monitor: Initializing network drop monitor service1529lb # [ 0.976999] e1000: Copyright (c) 1999-2006 Intel Corporation.1530client # [ 0.976553] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11531worker2 # [ 0.975648] rtc-pl031 9010000.pl031: registered as rtc01532lb # [ 0.977023] e1000e: Intel(R) PRO/1000 Network Driver1533worker1 # [ 1.005024] NET: Registered PF_INET6 protocol family1534client # [ 0.976724] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001535lb # [ 0.977032] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1536worker2 # [ 0.975679] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T14:59:21 UTC (1790348361)1537worker1 # [ 1.006236] Segment Routing with IPv61538lb # [ 0.977062] igb: Intel(R) Gigabit Ethernet Network Driver1539worker2 # [ 0.976052] i2c_dev: i2c /dev entries driver1540worker1 # [ 1.006259] In-situ OAM (IOAM) with IPv61541lb # [ 0.977068] igb: Copyright (c) 2007-2014 Intel Corporation.1542worker1 # [ 1.006288] NET: Registered PF_PACKET protocol family1543client # [ 0.982534] rtc-pl031 9010000.pl031: registered as rtc01544worker2 # [ 0.979264] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001545lb # [ 0.977091] igbvf: Intel(R) Gigabit Virtual Function Network Driver1546worker2 # [ 0.979552] hub 1-0:1.0: USB hub found1547client # [ 0.982567] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T14:59:21 UTC (1790348361)1548lb # [ 0.977097] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1549worker2 # [ 0.979633] hub 1-0:1.0: 6 ports detected1550client # [ 0.982910] i2c_dev: i2c /dev entries driver1551lb # [ 0.977235] sky2: driver version 1.301552client # [ 0.986110] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001553worker1 # [ 1.008328] 9pnet: Installing 9P2000 support1554client # [ 0.986414] hub 1-0:1.0: USB hub found1555worker1 # [ 1.008375] Key type dns_resolver registered1556client # [ 0.986435] hub 1-0:1.0: 6 ports detected1557lb # [ 0.986512] usbcore: registered new interface driver usb-storage1558worker2 # [ 0.984173] sdhci: Secure Digital Host Controller Interface driver1559lb # [ 0.986726] usbcore: registered new interface driver usbserial_generic1560worker2 # [ 0.984186] sdhci: Copyright(c) Pierre Ossman1561lb # [ 0.986743] usbserial: USB Serial support registered for generic1562worker2 # [ 0.984449] Synopsys Designware Multimedia Card Interface Driver1563lb # [ 0.987343] hv_vmbus: registering driver hyperv_keyboard1564worker2 # [ 0.984827] sdhci-pltfm: SDHCI platform and OF driver helper1565lb # [ 0.988675] ehci-pci 0000:00:07.0: EHCI Host Controller1566client # [ 0.991008] sdhci: Secure Digital Host Controller Interface driver1567lb # [ 0.988705] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11568client # [ 0.991023] sdhci: Copyright(c) Pierre Ossman1569worker2 # [ 0.989321] hid: raw HID events driver (C) Jiri Kosina1570lb # [ 0.988873] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001571client # [ 0.991278] Synopsys Designware Multimedia Card Interface Driver1572worker2 # [ 0.989623] usbcore: registered new interface driver usbhid1573worker1 # [ 1.015176] registered taskstats version 11574client # [ 0.991664] sdhci-pltfm: SDHCI platform and OF driver helper1575worker2 # [ 0.989631] usbhid: USB HID core driver1576worker1 # [ 1.015392] Loading compiled-in X.509 certificates1577client # [ 0.993152] hid: raw HID events driver (C) Jiri Kosina1578lb # [ 0.994898] rtc-pl031 9010000.pl031: registered as rtc01579client # [ 0.993395] usbcore: registered new interface driver usbhid1580client # [ 0.993401] usbhid: USB HID core driver1581lb # [ 0.994940] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T14:59:21 UTC (1790348361)1582lb # [ 0.995303] i2c_dev: i2c /dev entries driver1583worker2 # [ 0.992779] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1584worker2 # [ 0.994317] drop_monitor: Initializing network drop monitor service1585lb # [ 0.998483] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001586lb # [ 0.998849] hub 1-0:1.0: USB hub found1587worker2 # [ 0.996431] NET: Registered PF_INET6 protocol family1588worker1 # [ 1.024241] Demotion targets for Node 0: null1589lb # [ 0.998870] hub 1-0:1.0: 6 ports detected1590worker1 # [ 1.024355] Key type .fscrypt registered1591worker2 # [ 0.997623] Segment Routing with IPv61592client # [ 0.999510] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1593worker1 # [ 1.024362] Key type fscrypt-provisioning registered1594worker2 # [ 0.997656] In-situ OAM (IOAM) with IPv61595worker1 # [ 1.024468] ima: No TPM chip found, activating TPM-bypass!1596client # [ 1.001013] drop_monitor: Initializing network drop monitor service1597worker2 # [ 0.997685] NET: Registered PF_PACKET protocol family1598worker1 # [ 1.024488] ima: Allocated hash algorithm: sha11599client # [ 1.001203] NET: Registered PF_INET6 protocol family1600worker1 # [ 1.024510] ima: No architecture policies found1601client # [ 1.004523] Segment Routing with IPv61602client # [ 1.004544] In-situ OAM (IOAM) with IPv61603worker1 # [ 1.028786] input: gpio-keys as /devices/platform/gpio-keys/input/input01604client # [ 1.004572] NET: Registered PF_PACKET protocol family1605worker2 # [ 0.999550] 9pnet: Installing 9P2000 support1606lb # [ 1.003529] sdhci: Secure Digital Host Controller Interface driver1607worker2 # [ 0.999600] Key type dns_resolver registered1608lb # [ 1.003543] sdhci: Copyright(c) Pierre Ossman1609lb # [ 1.003809] Synopsys Designware Multimedia Card Interface Driver1610lb # [ 1.004183] sdhci-pltfm: SDHCI platform and OF driver helper1611lb # [ 1.005705] hid: raw HID events driver (C) Jiri Kosina1612client # [ 1.006277] 9pnet: Installing 9P2000 support1613client # [ 1.006324] Key type dns_resolver registered1614lb # [ 1.009469] usbcore: registered new interface driver usbhid1615lb # [ 1.009476] usbhid: USB HID core driver1616client # [ 1.013255] registered taskstats version 11617client # [ 1.013416] Loading compiled-in X.509 certificates1618worker2 # [ 1.006495] registered taskstats version 11619lb # [ 1.011877] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1620worker2 # [ 1.006670] Loading compiled-in X.509 certificates1621lb # [ 1.013466] drop_monitor: Initializing network drop monitor service1622lb # [ 1.013662] NET: Registered PF_INET6 protocol family1623lb # [ 1.016926] Segment Routing with IPv61624lb # [ 1.016948] In-situ OAM (IOAM) with IPv61625lb # [ 1.016976] NET: Registered PF_PACKET protocol family1626worker2 # [ 1.015873] Demotion targets for Node 0: null1627worker2 # [ 1.016004] Key type .fscrypt registered1628worker2 # [ 1.016011] Key type fscrypt-provisioning registered1629worker2 # [ 1.016109] ima: No TPM chip found, activating TPM-bypass!1630client # [ 1.022285] Demotion targets for Node 0: null1631lb # [ 1.018753] 9pnet: Installing 9P2000 support1632worker2 # [ 1.016133] ima: Allocated hash algorithm: sha11633client # [ 1.022397] Key type .fscrypt registered1634lb # [ 1.018803] Key type dns_resolver registered1635worker2 # [ 1.016155] ima: No architecture policies found1636client # [ 1.022404] Key type fscrypt-provisioning registered1637client # [ 1.022504] ima: No TPM chip found, activating TPM-bypass!1638client # [ 1.022523] ima: Allocated hash algorithm: sha11639worker2 # [ 1.020498] input: gpio-keys as /devices/platform/gpio-keys/input/input01640worker1 # [ 1.047366] clk: Disabling unused clocks1641client # [ 1.022545] ima: No architecture policies found1642worker1 # [ 1.047397] PM: genpd: Disabling unused power domains1643client # [ 1.027041] input: gpio-keys as /devices/platform/gpio-keys/input/input01644worker1 # [ 1.051662] Freeing unused kernel memory: 4736K1645worker1 # [ 1.051878] Run /init as init process1646lb # [ 1.025891] registered taskstats version 11647lb # [ 1.026044] Loading compiled-in X.509 certificates1648lb # [ 1.035143] Demotion targets for Node 0: null1649lb # [ 1.035263] Key type .fscrypt registered1650lb # [ 1.035272] Key type fscrypt-provisioning registered1651lb # [ 1.035377] ima: No TPM chip found, activating TPM-bypass!1652lb # [ 1.035397] ima: Allocated hash algorithm: sha11653lb # [ 1.035419] ima: No architecture policies found1654lb # [ 1.039763] input: gpio-keys as /devices/platform/gpio-keys/input/input01655worker1 # [ 1.069401] systemd[1]: Successfully made /usr/ read-only.1656worker2 # [ 1.039629] clk: Disabling unused clocks1657worker2 # [ 1.039663] PM: genpd: Disabling unused power domains1658client # [ 1.045976] clk: Disabling unused clocks1659client # [ 1.046009] PM: genpd: Disabling unused power domains1660worker2 # [ 1.043988] Freeing unused kernel memory: 4736K1661worker2 # [ 1.044184] Run /init as init process1662client # [ 1.050306] Freeing unused kernel memory: 4736K1663client # [ 1.050502] Run /init as init process1664lb # [ 1.058959] clk: Disabling unused clocks1665lb # [ 1.058999] PM: genpd: Disabling unused power domains1666lb # [ 1.063350] Freeing unused kernel memory: 4736K1667worker2 # [ 1.060736] systemd[1]: Successfully made /usr/ read-only.1668lb # [ 1.063600] Run /init as init process1669client # [ 1.067960] systemd[1]: Successfully made /usr/ read-only.1670lb # [ 1.080115] systemd[1]: Successfully made /usr/ read-only.1671worker1 # [ 1.231039] usb 1-1: new high-speed USB device number 2 using ehci-pci1672worker2 # [ 1.226371] usb 1-1: new high-speed USB device number 2 using ehci-pci1673client # [ 1.233579] usb 1-1: new high-speed USB device number 2 using ehci-pci1674lb # [ 1.245764] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker1 # [ 1.380886] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676worker2 # [ 1.376175] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677worker1 # [ 1.404136] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678client # [ 1.383314] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679worker1 # [ 1.409908] systemd[1]: Detected virtualization qemu.1680worker1 # [ 1.410833] systemd[1]: Detected architecture arm64.1681worker1 # [ 1.411770] systemd[1]: Running in initrd.1682worker1 # [ 1.412999] systemd[1]: Initializing machine ID from random generator.1683worker1 # [ 1.414238] systemd[1]: Hostname set to <worker1>.1684lb # [ 1.395538] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11685worker2 # [ 1.395508] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1686worker2 # [ 1.402035] systemd[1]: Detected virtualization qemu.1687worker2 # [ 1.403198] systemd[1]: Detected architecture arm64.1688client # [ 1.402700] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1689worker2 # [ 1.404265] systemd[1]: Running in initrd.1690client # [ 1.409344] systemd[1]: Detected virtualization qemu.1691worker2 # [ 1.405702] systemd[1]: Initializing machine ID from random generator.1692client # [ 1.410590] systemd[1]: Detected architecture arm64.1693client # [ 1.411789] systemd[1]: Running in initrd.1694worker2 # [ 1.407514] systemd[1]: Hostname set to <worker2>.1695client # [ 1.413209] systemd[1]: Initializing machine ID from random generator.1696client # [ 1.414761] systemd[1]: Hostname set to <client>.1697lb # [ 1.414897] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1698lb # [ 1.421377] systemd[1]: Detected virtualization qemu.1699lb # [ 1.422520] systemd[1]: Detected architecture arm64.1700lb # [ 1.423551] systemd[1]: Running in initrd.1701lb # [ 1.424977] systemd[1]: Initializing machine ID from random generator.1702lb # [ 1.426631] systemd[1]: Hostname set to <lb>.1703worker1 # [ 1.467209] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01704client # [ 1.465744] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705worker2 # [ 1.462591] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706lb # [ 1.481927] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker1 # [ 1.587050] usb 1-2: new high-speed USB device number 3 using ehci-pci1708client # [ 1.585596] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker2 # [ 1.582376] usb 1-2: new high-speed USB device number 3 using ehci-pci1710lb # [ 1.601765] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker1 # [ 1.664271] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712worker2 # [ 1.672538] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713client # [ 1.687661] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714lb # [ 1.688770] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker1 # [ 1.737528] systemd[1]: Queued start job for default target Initrd Default Target.1716worker1 # [ 1.740652] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21717worker1 # [ 1.742914] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01718worker1 # [ 1.749355] systemd[1]: Created slice Slice /system/modprobe.1719worker1 # [ 1.750673] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1720worker1 # [ 1.752175] systemd[1]: Expecting device /dev/disk/by-label/nixos...1721worker1 # [ 1.753183] systemd[1]: Reached target Path Units.1722worker1 # [ 1.753995] systemd[1]: Reached target Slice Units.1723worker1 # [ 1.754760] systemd[1]: Reached target Swaps.1724worker1 # [ 1.755530] systemd[1]: Reached target Timer Units.1725worker1 # [ 1.756495] systemd[1]: Listening on D-Bus System Message Bus Socket.1726worker1 # [ 1.757700] systemd[1]: Listening on Journal Socket (/dev/log).1727worker1 # [ 1.758763] systemd[1]: Listening on Journal Sockets.1728worker1 # [ 1.759830] systemd[1]: Listening on udev Control Socket.1729worker1 # [ 1.760834] systemd[1]: Listening on udev Kernel Socket.1730worker1 # [ 1.761683] systemd[1]: Reached target Socket Units.1731worker1 # [ 1.764024] systemd[1]: Starting Create List of Static Device Nodes...1732client # [ 1.739316] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21733worker1 # [ 1.765130] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1734client # [ 1.741623] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01735worker1 # [ 1.772506] systemd[1]: Mounting Kernel Configuration File System...1736worker2 # [ 1.746252] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21737worker2 # [ 1.748866] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01738client # [ 1.759423] systemd[1]: Queued start job for default target Initrd Default Target.1739worker1 # [ 1.783969] systemd[1]: Starting Journal Service...1740worker2 # [ 1.756728] systemd[1]: Queued start job for default target Initrd Default Target.1741lb # [ 1.760498] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21742lb # [ 1.763084] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01743client # [ 1.767770] systemd[1]: Created slice Slice /system/modprobe.1744client # [ 1.768863] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1745lb # [ 1.767996] systemd[1]: Queued start job for default target Initrd Default Target.1746client # [ 1.770221] systemd[1]: Expecting device /dev/disk/by-label/nixos...1747client # [ 1.771173] systemd[1]: Reached target Path Units.1748client # [ 1.771923] systemd[1]: Reached target Slice Units.1749client # [ 1.772718] systemd[1]: Reached target Swaps.1750worker2 # [ 1.768844] systemd[1]: Created slice Slice /system/modprobe.1751client # [ 1.773401] systemd[1]: Reached target Timer Units.1752client # [ 1.774451] systemd[1]: Listening on D-Bus System Message Bus Socket.1753worker2 # [ 1.770065] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1754client # [ 1.775615] systemd[1]: Listening on Journal Socket (/dev/log).1755worker2 # [ 1.771568] systemd[1]: Expecting device /dev/disk/by-label/nixos...1756client # [ 1.776642] systemd[1]: Listening on Journal Sockets.1757worker2 # [ 1.772667] systemd[1]: Reached target Path Units.1758lb # [ 1.776279] systemd[1]: Created slice Slice /system/modprobe.1759client # [ 1.777593] systemd[1]: Listening on udev Control Socket.1760worker2 # [ 1.773525] systemd[1]: Reached target Slice Units.1761client # [ 1.778559] systemd[1]: Listening on udev Kernel Socket.1762worker2 # [ 1.774453] systemd[1]: Reached target Swaps.1763lb # [ 1.777586] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1764client # [ 1.779432] systemd[1]: Reached target Socket Units.1765worker2 # [ 1.775297] systemd[1]: Reached target Timer Units.1766lb # [ 1.779136] systemd[1]: Expecting device /dev/disk/by-label/nixos...1767worker2 # [ 1.776332] systemd[1]: Listening on D-Bus System Message Bus Socket.1768lb # [ 1.780216] systemd[1]: Reached target Path Units.1769lb # [ 1.781055] systemd[1]: Reached target Slice Units.1770client # [ 1.781841] systemd[1]: Starting Create List of Static Device Nodes...1771worker2 # [ 1.777628] systemd[1]: Listening on Journal Socket (/dev/log).1772lb # [ 1.781964] systemd[1]: Reached target Swaps.1773worker2 # [ 1.778820] systemd[1]: Listening on Journal Sockets.1774client # [ 1.782903] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1775lb # [ 1.782803] systemd[1]: Reached target Timer Units.1776worker2 # [ 1.779857] systemd[1]: Listening on udev Control Socket.1777lb # [ 1.783852] systemd[1]: Listening on D-Bus System Message Bus Socket.1778worker2 # [ 1.780933] systemd[1]: Listening on udev Kernel Socket.1779worker2 # [ 1.781849] systemd[1]: Reached target Socket Units.1780lb # [ 1.785089] systemd[1]: Listening on Journal Socket (/dev/log).1781lb # [ 1.786286] systemd[1]: Listening on Journal Sockets.1782lb # [ 1.787343] systemd[1]: Listening on udev Control Socket.1783worker1 # [ 1.811314] systemd[1]: Starting Load Kernel Modules...1784lb # [ 1.788422] systemd[1]: Listening on udev Kernel Socket.1785worker1 # [ 1.811428] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1786lb # [ 1.789321] systemd[1]: Reached target Socket Units.1787worker2 # [ 1.784468] systemd[1]: Starting Create List of Static Device Nodes...1788worker1 # [ 1.815437] systemd[1]: Starting Coldplug All udev Devices...1789worker2 # [ 1.784556] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1790client # [ 1.792097] systemd[1]: Mounting Kernel Configuration File System...1791lb # [ 1.791914] systemd[1]: Starting Create List of Static Device Nodes...1792lb # [ 1.791997] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1793worker2 # [ 1.794512] systemd[1]: Mounting Kernel Configuration File System...1794worker1 # [ 1.826496] systemd[1]: Finished Create List of Static Device Nodes.1795lb # [ 1.801943] systemd[1]: Mounting Kernel Configuration File System...1796worker2 # [ 1.802610] systemd[1]: Starting Journal Service...1797client # [ 1.809743] systemd[1]: Starting Journal Service...1798lb # [ 1.810098] systemd[1]: Starting Journal Service...1799worker1 # [ 1.834371] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1800worker1 # [ 1.848348] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1801client # [ 1.825779] systemd[1]: Starting Load Kernel Modules...1802client # [ 1.826598] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1803worker1 # [ 1.850784] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1804worker1 # [ 1.852533] systemd[1]: Mounted Kernel Configuration File System.1805worker2 # [ 1.825752] systemd[1]: Starting Load Kernel Modules...1806worker2 # [ 1.826918] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1807worker1 # [ 1.865589] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01808lb # [ 1.842166] systemd[1]: Starting Load Kernel Modules...1809worker1 # [ 1.865846] [drm] features: -virgl +edid -resource_blob -host_visible1810worker1 # [ 1.865854] [drm] features: -context_init1811lb # [ 1.843194] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1812worker1 # [ 1.866585] [drm] number of scanouts: 11813worker1 # [ 1.866603] [drm] number of cap sets: 01814client # [ 1.849708] systemd[1]: Starting Coldplug All udev Devices...1815worker2 # [ 1.846616] systemd[1]: Starting Coldplug All udev Devices...1816client # [ 1.853752] systemd[1]: Finished Create List of Static Device Nodes.1817worker2 # [ 1.849929] systemd-journald[72]: Collecting audit messages is disabled.1818client # [ 1.855335] systemd[1]: Mounted Kernel Configuration File System.1819lb # [ 1.859573] systemd[1]: Starting Coldplug All udev Devices...1820client # [ 1.864262] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1821lb # [ 1.871174] systemd-journald[72]: Collecting audit messages is disabled.1822client # [ 1.873826] systemd-journald[72]: Collecting audit messages is disabled.1823worker1 # [ 1.891397] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1824worker1 # [ 1.891421] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01825worker2 # [ 1.870517] systemd[1]: Finished Create List of Static Device Nodes.1826worker2 # [ 1.871384] systemd[1]: Mounted Kernel Configuration File System.1827client # [ 1.878268] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1828client # [ 1.879583] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1829worker1 # [ 1.906759] systemd-journald[72]: Collecting audit messages is disabled.1830worker1 # [ 1.906925] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1831worker1 # [ 1.911487] systemd[1]: Starting Create Static Device Nodes in /dev...1832lb # [ 1.890098] systemd[1]: Finished Create List of Static Device Nodes.1833lb # [ 1.891221] systemd[1]: Mounted Kernel Configuration File System.1834worker2 # [ 1.882777] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1835lb # [ 1.894693] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1836client # [ 1.899007] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01837client # [ 1.899265] [drm] features: -virgl +edid -resource_blob -host_visible1838client # [ 1.899276] [drm] features: -context_init1839client # [ 1.900053] [drm] number of scanouts: 11840client # [ 1.900072] [drm] number of cap sets: 01841worker1 # [ 1.923357] Console: switching to colour frame buffer device 160x501842lb # [ 1.906942] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1843worker2 # [ 1.907513] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1844worker1 # [ 1.939713] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1845client # [ 1.918231] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1846client # [ 1.920884] systemd[1]: Starting Create Static Device Nodes in /dev...1847worker1 # [ 1.956261] systemd[1]: Finished Load Kernel Modules.1848lb # [ 1.921803] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1849lb # [ 1.927992] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01850worker1 # [ 1.958514] systemd[1]: Starting Apply Kernel Variables...1851client # [ 1.929874] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1852lb # [ 1.928233] [drm] features: -virgl +edid -resource_blob -host_visible1853lb # [ 1.928243] [drm] features: -context_init1854client # [ 1.929894] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01855lb # [ 1.928990] [drm] number of scanouts: 11856lb # [ 1.929009] [drm] number of cap sets: 01857worker2 # [ 1.934425] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1858worker2 # [ 1.940787] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01859worker2 # [ 1.941031] [drm] features: -virgl +edid -resource_blob -host_visible1860worker2 # [ 1.941042] [drm] features: -context_init1861worker2 # [ 1.941763] [drm] number of scanouts: 11862worker2 # [ 1.941781] [drm] number of cap sets: 01863worker2 # [ 1.955053] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1864lb # [ 1.958429] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1865worker2 # [ 1.957783] systemd[1]: Starting Create Static Device Nodes in /dev...1866client # [ 1.958003] Console: switching to colour frame buffer device 160x501867client # [ 1.962222] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1868worker1 # [ 1.987505] systemd[1]: Finished Create Static Device Nodes in /dev.1869lb # [ 1.961205] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1870worker1 # [ 1.987737] systemd[1]: Reached target Preparation for Local File Systems.1871lb # [ 1.961230] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01872worker1 # [ 1.987774] systemd[1]: Reached target Local File Systems.1873worker1 # [ 1.993061] systemd[1]: Starting Rule-based Manager for Device Events and Files...1874lb # [ 1.968011] systemd[1]: Starting Create Static Device Nodes in /dev...1875worker2 # [ 1.966868] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1876worker2 # [ 1.966887] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01877client # [ 1.975385] systemd[1]: Finished Create Static Device Nodes in /dev.1878client # [ 1.975807] systemd[1]: Reached target Preparation for Local File Systems.1879client # [ 1.975847] systemd[1]: Reached target Local File Systems.1880client # [ 1.981981] systemd[1]: Starting Rule-based Manager for Device Events and Files...1881lb # [ 1.986012] Console: switching to colour frame buffer device 160x501882worker2 # [ 1.990159] Console: switching to colour frame buffer device 160x501883client # [ 1.997705] systemd[1]: Finished Load Kernel Modules.1884worker1 # [ 2.031392] systemd[1]: Finished Apply Kernel Variables.1885client # [ 2.005924] systemd[1]: Starting Apply Kernel Variables...1886lb # [ 2.010400] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1887worker2 # [ 2.010798] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1888lb # [ 2.013415] systemd[1]: Finished Create Static Device Nodes in /dev.1889lb # [ 2.013849] systemd[1]: Reached target Preparation for Local File Systems.1890lb # [ 2.013883] systemd[1]: Reached target Local File Systems.1891lb # [ 2.022223] systemd[1]: Starting Rule-based Manager for Device Events and Files...1892worker2 # [ 2.019759] systemd[1]: Finished Create Static Device Nodes in /dev.1893worker2 # [ 2.021793] systemd[1]: Reached target Preparation for Local File Systems.1894worker2 # [ 2.023682] systemd[1]: Reached target Local File Systems.1895worker2 # [ 2.031040] systemd[1]: Starting Rule-based Manager for Device Events and Files...1896worker1 # [ 2.048568] systemd-modules-load[73]: Inserted module 'dm_mod'1897worker1 # [ 2.068686] systemd[1]: Started Journal Service.1898worker2 # [ 2.024626] systemd-modules-load[74]: Inserted module 'dm_mod'1899worker2 # [ 2.025929] systemd-modules-load[74]: Module 'virtio_balloon' is built in1900worker2 # [ 2.027020] systemd-modules-load[74]: Module 'virtio_console' is built in1901client # [ 2.050191] systemd[1]: Finished Apply Kernel Variables.1902worker1 # [ 2.056515] systemd-modules-load[73]: Module 'virtio_balloon' is built in1903worker2 # [ 2.046455] systemd[1]: Finished Load Kernel Modules.1904worker1 # [ 2.057673] systemd-modules-load[73]: Module 'virtio_console' is built in1905worker2 # [ 2.046724] systemd[1]: Started Journal Service.1906lb # [ 2.050010] systemd[1]: Finished Load Kernel Modules.1907worker1 # [ 2.058766] systemd-modules-load[73]: Inserted module 'virtio_gpu'1908worker2 # [ 2.032193] systemd-modules-load[74]: Inserted module 'virtio_gpu'1909worker1 # [ 2.059756] systemd-modules-load[73]: Module 'virtio_rng' is built in1910lb # [ 2.053396] systemd[1]: Starting Apply Kernel Variables...1911worker2 # [ 2.036329] systemd-modules-load[74]: Module 'virtio_rng' is built in1912lb # [ 2.040496] systemd-modules-load[73]: Inserted module 'dm_mod'1913client # [ 2.057707] systemd[1]: Started Journal Service.1914lb # [ 2.041865] systemd-modules-load[73]: Module 'virtio_balloon' is built in1915client # [ 2.044431] systemd-modules-load[73]: Inserted module 'dm_mod'1916client # [ 2.045701] systemd-modules-load[73]: Module 'virtio_balloon' is built in1917worker1 # [ 2.069450] systemd[1]: Starting Create System Files and Directories...1918lb # [ 2.049221] systemd-modules-load[73]: Module 'virtio_console' is built in1919lb # [ 2.069911] systemd[1]: Started Journal Service.1920lb # [ 2.054882] systemd-modules-load[73]: Inserted module 'virtio_gpu'1921client # [ 2.056333] systemd-modules-load[73]: Module 'virtio_console' is built in1922worker2 # [ 2.054493] systemd[1]: Starting Apply Kernel Variables...1923client # [ 2.060273] systemd-modules-load[73]: Inserted module 'virtio_gpu'1924client # [ 2.061399] systemd-modules-load[73]: Module 'virtio_rng' is built in1925lb # [ 2.060296] systemd-modules-load[73]: Module 'virtio_rng' is built in1926worker2 # [ 2.056420] systemd[1]: Starting Create System Files and Directories...1927worker1 # [ 2.089213] systemd-udevd[80]: Using default interface naming scheme 'v261'.1928client # [ 2.069154] systemd[1]: Starting Create System Files and Directories...1929worker1 # [ 2.105379] systemd[1]: Finished Create System Files and Directories.1930client # [ 2.084214] systemd-udevd[79]: Using default interface naming scheme 'v261'.1931lb # [ 2.085127] systemd[1]: Starting Create System Files and Directories...1932worker1 # [ 2.118360] systemd[1]: Started Rule-based Manager for Device Events and Files.1933worker2 # [ 2.096743] systemd[1]: Finished Create System Files and Directories.1934lb # [ 2.100900] systemd[1]: Finished Apply Kernel Variables.1935client # [ 2.113908] systemd[1]: Finished Create System Files and Directories.1936worker2 # [ 2.109362] systemd[1]: Finished Apply Kernel Variables.1937lb # [ 2.116312] systemd-udevd[79]: Using default interface naming scheme 'v261'.1938lb # [ 2.127131] systemd[1]: Finished Create System Files and Directories.1939client # [ 2.129235] systemd[1]: Started Rule-based Manager for Device Events and Files.1940worker2 # [ 2.129311] systemd-udevd[78]: Using default interface naming scheme 'v261'.1941lb # [ 2.146711] systemd[1]: Started Rule-based Manager for Device Events and Files.1942worker1 # [ 2.176664] systemd[1]: Starting Virtual Console Setup...1943worker2 # [ 2.156484] systemd[1]: Started Rule-based Manager for Device Events and Files.1944client # [ 2.180630] systemd[1]: Starting Virtual Console Setup...1945worker1 # [ 2.228579] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1946worker1 # [ 2.232347] systemd[1]: Finished Virtual Console Setup.1947lb # [ 2.212185] systemd[1]: Starting Virtual Console Setup...1948worker2 # [ 2.217408] systemd[1]: Starting Virtual Console Setup...1949client # [ 2.242256] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1950client # [ 2.244101] systemd[1]: Finished Virtual Console Setup.1951lb # [ 2.260588] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1952lb # [ 2.264065] systemd[1]: Finished Virtual Console Setup.1953worker2 # [ 2.272549] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1954worker2 # [ 2.276370] systemd[1]: Finished Virtual Console Setup.1955worker1 # [ 2.854755] systemd[1]: Finished Coldplug All udev Devices.1956worker1 # [ 2.861175] systemd[1]: Reached target System Initialization.1957worker1 # [ 2.862509] systemd[1]: Reached target Basic System.1958worker2 # [ 2.870706] systemd[1]: Finished Coldplug All udev Devices.1959worker2 # [ 2.872143] systemd[1]: Reached target System Initialization.1960worker2 # [ 2.876556] systemd[1]: Reached target Basic System.1961lb # [ 2.896597] systemd[1]: Finished Coldplug All udev Devices.1962lb # [ 2.898397] systemd[1]: Reached target System Initialization.1963lb # [ 2.899293] systemd[1]: Reached target Basic System.1964client # [ 2.907025] systemd[1]: Finished Coldplug All udev Devices.1965client # [ 2.908075] systemd[1]: Reached target System Initialization.1966client # [ 2.908978] systemd[1]: Reached target Basic System.1967worker1 # [ 3.008138] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1968worker2 # [ 3.004182] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1969worker1 # [ 3.045540] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1970worker1 # [ 3.060458] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1971client # [ 3.056477] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972worker2 # [ 3.056760] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973worker2 # [ 3.064469] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1974lb # [ 3.071102] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1975lb # [ 3.076879] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976lb # [ 3.084217] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1977client # [ 3.087113] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1978client # [ 3.101983] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1979worker1 # [ 3.149483] systemd[1]: Found device /dev/disk/by-label/nixos.1980worker1 # [ 3.151393] systemd[1]: Reached target Initrd Root Device.1981worker1 # [ 3.156771] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982worker2 # [ 3.154280] systemd[1]: Found device /dev/disk/by-label/nixos.1983worker2 # [ 3.157934] systemd[1]: Reached target Initrd Root Device.1984worker2 # [ 3.161480] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985lb # [ 3.176129] systemd[1]: Found device /dev/disk/by-label/nixos.1986client # [ 3.177249] systemd[1]: Found device /dev/disk/by-label/nixos.1987lb # [ 3.177302] systemd[1]: Reached target Initrd Root Device.1988worker1 # [ 3.202298] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1989lb # [ 3.181365] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1990client # [ 3.183100] systemd[1]: Reached target Initrd Root Device.1991client # [ 3.187141] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992worker1 # [ 3.210988] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1993worker1 # [ 3.213951] systemd[1]: Mounting /sysroot...1994worker2 # [ 3.213492] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1995worker2 # [ 3.223055] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1996worker1 # [ 3.272465] EXT4-fs (vda): mounted filesystem d9b10c6c-1981-4594-a993-de61abb4c086 r/w with ordered data mode. Quota mode: none.1997worker2 # [ 3.229371] systemd[1]: Mounting /sysroot...1998worker1 # [ 3.259634] systemd[1]: Mounted /sysroot.1999lb # [ 3.235874] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks2000worker1 # [ 3.261531] systemd[1]: Reached target Initrd Root File System.
loading 2,148 hidden lines…
4149lb: must succeed: curl -sf localhost:9901/clusters4150lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4151worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4152worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4153(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.50 seconds)4154subtest: outputs read back through any worker carry the cache signature4155client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24156client # don't know how to build these paths:4157client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4158worker2 # [ 66.011201] nix-grpc-daemon[786]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502624159worker1 # [ 66.042556] nix-grpc-daemon[485]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904160worker1 # [ 66.057626] nix-grpc-daemon[485]: ts=2026-09-25T15:00:26Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04161worker1 # [ 66.061663] nix-grpc-daemon[485]: ts=2026-09-25T15:00:26Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=14162worker1 # [ 66.065730] nix-grpc-daemon[485]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904163worker1 # [ 66.075397] nix-grpc-daemon[485]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904164client # copying 1 paths to 'lb:50051'...4165client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4166worker1 # [ 66.085213] nix-grpc-daemon[485]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537904167worker1 # [ 66.094382] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4168worker1 # [ 66.112618] nix-grpc-daemon[485]: ts=2026-09-25T15:00:26Z level=info event=niks3_push_started pid=9054169worker1 # [ 66.261730] nix-grpc-daemon[905]: time=2026-09-25T15:00:26.793Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4170worker1 # [ 66.367442] nix-daemon[855]: accepted connection from pid 917, user nix-grpc-daemon (trusted)4171lb # [ 66.367374] niks3-server[1035]: 2026/09/25 15:00:26 INFO Received push request method=POST path=/api/pushes4172worker1 # [ 66.393918] nix-daemon[855]: reaped child process 920, status = succeeded4173worker1 # [ 66.586749] nix-grpc-daemon[905]: time=2026-09-25T15:00:27.116Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4174worker1 # [ 66.593340] nix-grpc-daemon[905]: time=2026-09-25T15:00:27.117Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4175lb # [ 66.583400] niks3-server[1035]: 2026/09/25 15:00:27 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4176worker1 # [ 66.613558] nix-grpc-daemon[905]: time=2026-09-25T15:00:27.144Z level=INFO msg="Uploading 1 narinfos"4177lb # [ 66.592419] niks3-server[1035]: 2026/09/25 15:00:27 INFO Signed narinfos id=9 count=14178lb # [ 66.594153] niks3-server[1035]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4179lb # [ 66.603949] niks3-server[1035]: 2026/09/25 15:00:27 INFO Received complete push request method=POST path=/api/pushes/9/complete4180lb # [ 66.609941] niks3-server[1035]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4181lb # [ 66.615219] niks3-server[1035]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4182worker1 # [ 66.649048] nix-grpc-daemon[905]: time=2026-09-25T15:00:27.180Z level=INFO msg="Upload complete. (383ms)"4183worker1 # [ 66.656100] nix-grpc-daemon[485]: ts=2026-09-25T15:00:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 paths=1 nar_bytes_in=5124184worker1 # [ 66.659334] nix-grpc-daemon[485]: ts=2026-09-25T15:00:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904185worker1 # [ 66.671946] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4186worker1 # [ 66.676582] nix-daemon[855]: reaped child process 903, status = succeeded4187client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4188client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4189worker1 # [ 66.688067] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4190client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4191client # farm-a> LOG-farm-a-sig4192worker1 # [ 68.127442] nix-daemon[855]: reaped child process 924, status = succeeded4193worker1 # [ 68.240414] nix-daemon[855]: accepted connection from pid 940, user nix-grpc-daemon (trusted)4194lb # [ 68.235331] niks3-server[1035]: 2026/09/25 15:00:28 INFO Received push request method=POST path=/api/pushes4195worker1 # [ 68.261493] nix-daemon[855]: reaped child process 943, status = succeeded4196lb # [ 68.262756] envoy[897]: conn peer=[2001:db8:1::1]:56692 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9519 tx=4367 ms=48654197lb # [ 68.265647] envoy[897]: conn peer=[2001:db8:1::1]:54074 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=604198lb # [ 68.268155] envoy[897]: conn peer=[2001:db8:1::1]:54080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4168 tx=1902 ms=19584199worker1 # [ 68.399720] nix-grpc-daemon[905]: time=2026-09-25T15:00:28.928Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4200worker1 # [ 68.404487] nix-grpc-daemon[905]: time=2026-09-25T15:00:28.928Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4201lb # [ 68.395895] niks3-server[1035]: 2026/09/25 15:00:28 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4202worker1 # [ 68.423933] nix-grpc-daemon[905]: time=2026-09-25T15:00:28.955Z level=INFO msg="Uploading 1 narinfos"4203lb # [ 68.402349] niks3-server[1035]: 2026/09/25 15:00:28 INFO Signed narinfos id=10 count=14204lb # [ 68.409390] niks3-server[1035]: 2026/09/25 15:00:28 INFO Received complete push request method=POST path=/api/pushes/10/complete4205lb # [ 68.414913] niks3-server[1035]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4206lb # [ 68.421311] niks3-server[1035]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4207lb # [ 68.430256] niks3-server[1035]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4208worker1 # [ 68.463380] nix-grpc-daemon[905]: time=2026-09-25T15:00:28.995Z level=INFO msg="Upload complete. (363ms)"4209lb # [ 68.442467] niks3-server[1035]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4210worker1 # [ 68.470755] nix-grpc-daemon[485]: ts=2026-09-25T15:00:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14211worker1 # [ 68.483315] nix-grpc-daemon[485]: ts=2026-09-25T15:00:29Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04212worker1 # [ 68.491068] nix-grpc-daemon[485]: ts=2026-09-25T15:00:29Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14213worker1 # [ 68.495539] nix-grpc-daemon[485]: ts=2026-09-25T15:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904214worker1 # [ 68.499938] nix-daemon[855]: reaped child process 922, status = succeeded4215worker1 # [ 68.502858] nix-grpc-daemon[485]: ts=2026-09-25T15:00:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904216worker1 # [ 68.508568] nix-grpc-daemon[905]: time=2026-09-25T15:00:29.040Z level=INFO msg="All 1 paths already cached"4217client # copying 1 paths to 'lb:50051'...4218client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4219worker1 # [ 68.514887] nix-grpc-daemon[485]: ts=2026-09-25T15:00:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537904220worker1 # [ 68.527927] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4221worker1 # [ 68.641735] nix-daemon[855]: accepted connection from pid 949, user nix-grpc-daemon (trusted)4222worker1 # [ 68.662243] nix-daemon[855]: reaped child process 952, status = succeeded4223lb # [ 68.641590] niks3-server[1035]: 2026/09/25 15:00:29 INFO Received push request method=POST path=/api/pushes4224worker1 # [ 68.671094] nix-grpc-daemon[905]: time=2026-09-25T15:00:29.202Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4225worker1 # [ 68.672862] nix-grpc-daemon[905]: time=2026-09-25T15:00:29.202Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4226worker1 # [ 68.683696] nix-grpc-daemon[905]: time=2026-09-25T15:00:29.215Z level=INFO msg="Uploading 1 narinfos"4227lb # [ 68.659845] niks3-server[1035]: 2026/09/25 15:00:29 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4228lb # [ 68.667255] niks3-server[1035]: 2026/09/25 15:00:29 INFO Signed narinfos id=11 count=14229lb # [ 68.671824] niks3-server[1035]: 2026/09/25 15:00:29 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4230lb # [ 68.680414] niks3-server[1035]: 2026/09/25 15:00:29 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4231lb # [ 68.682929] niks3-server[1035]: 2026/09/25 15:00:29 INFO Received complete push request method=POST path=/api/pushes/11/complete4232worker1 # [ 68.712784] nix-grpc-daemon[905]: time=2026-09-25T15:00:29.244Z level=INFO msg="Upload complete. (180ms)"4233lb # [ 68.691426] niks3-server[1035]: 2026/09/25 15:00:29 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4234worker1 # [ 68.720108] nix-grpc-daemon[485]: ts=2026-09-25T15:00:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 paths=1 nar_bytes_in=6244235worker1 # [ 68.728588] nix-grpc-daemon[485]: ts=2026-09-25T15:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904236worker1 # [ 68.733932] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4237worker1 # [ 68.738136] nix-daemon[855]: reaped child process 947, status = succeeded4238client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4239client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4240worker1 # [ 68.747286] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4241client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4242client # farm-b> LOG-farm-b-sig4243worker1 # [ 69.986878] nix-daemon[855]: reaped child process 956, status = succeeded4244worker1 # [ 70.086995] nix-daemon[855]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4245lb # [ 70.079766] niks3-server[1035]: 2026/09/25 15:00:30 INFO Received push request method=POST path=/api/pushes4246worker1 # [ 70.105697] nix-daemon[855]: reaped child process 975, status = succeeded4247worker1 # [ 70.109957] nix-grpc-daemon[905]: time=2026-09-25T15:00:30.640Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4248worker1 # [ 70.111412] nix-grpc-daemon[905]: time=2026-09-25T15:00:30.641Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4249lb # [ 70.099988] niks3-server[1035]: 2026/09/25 15:00:30 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4250worker1 # [ 70.133146] nix-grpc-daemon[905]: time=2026-09-25T15:00:30.665Z level=INFO msg="Uploading 1 narinfos"4251lb # [ 70.110052] niks3-server[1035]: 2026/09/25 15:00:30 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4252lb # [ 70.116487] niks3-server[1035]: 2026/09/25 15:00:30 INFO Signed narinfos id=12 count=14253lb # [ 70.120963] niks3-server[1035]: 2026/09/25 15:00:30 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4254lb # [ 70.126702] niks3-server[1035]: 2026/09/25 15:00:30 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4255lb # [ 70.135997] niks3-server[1035]: 2026/09/25 15:00:30 INFO Received complete push request method=POST path=/api/pushes/12/complete4256worker1 # [ 70.163147] nix-grpc-daemon[905]: time=2026-09-25T15:00:30.695Z level=INFO msg="Upload complete. (205ms)"4257lb # [ 70.140000] niks3-server[1035]: 2026/09/25 15:00:30 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4258worker1 # [ 70.172497] nix-grpc-daemon[485]: ts=2026-09-25T15:00:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14259worker1 # [ 70.183701] nix-grpc-daemon[485]: ts=2026-09-25T15:00:30Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04260worker1 # [ 70.189186] nix-grpc-daemon[485]: ts=2026-09-25T15:00:30Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14261worker1 # [ 70.193113] nix-grpc-daemon[485]: ts=2026-09-25T15:00:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904262worker1 # [ 70.195837] nix-daemon[855]: reaped child process 954, status = succeeded4263worker1 # [ 70.200679] nix-grpc-daemon[485]: ts=2026-09-25T15:00:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904264worker1 # [ 70.208171] nix-grpc-daemon[905]: time=2026-09-25T15:00:30.740Z level=INFO msg="All 2 paths already cached"4265client # copying 1 paths to 'lb:50051'...4266client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4267worker1 # [ 70.214712] nix-grpc-daemon[485]: ts=2026-09-25T15:00:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537904268worker1 # [ 70.223782] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4269worker1 # [ 70.332774] nix-daemon[855]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4270worker1 # [ 70.351805] nix-daemon[855]: reaped child process 984, status = succeeded4271lb # [ 70.332254] niks3-server[1035]: 2026/09/25 15:00:30 INFO Received push request method=POST path=/api/pushes4272worker1 # [ 70.360845] nix-grpc-daemon[905]: time=2026-09-25T15:00:30.892Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4273worker1 # [ 70.362355] nix-grpc-daemon[905]: time=2026-09-25T15:00:30.892Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4274lb # [ 70.350804] niks3-server[1035]: 2026/09/25 15:00:30 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4275worker1 # [ 70.380102] nix-grpc-daemon[905]: time=2026-09-25T15:00:30.911Z level=INFO msg="Uploading 1 narinfos"4276lb # [ 70.358974] niks3-server[1035]: 2026/09/25 15:00:30 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4277lb # [ 70.365200] niks3-server[1035]: 2026/09/25 15:00:30 INFO Signed narinfos id=13 count=14278lb # [ 70.372967] niks3-server[1035]: 2026/09/25 15:00:30 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4279lb # [ 70.375921] niks3-server[1035]: 2026/09/25 15:00:30 INFO Received complete push request method=POST path=/api/pushes/13/complete4280worker1 # [ 70.407822] nix-grpc-daemon[905]: time=2026-09-25T15:00:30.939Z level=INFO msg="Upload complete. (176ms)"4281lb # [ 70.385346] niks3-server[1035]: 2026/09/25 15:00:30 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4282worker1 # [ 70.414625] nix-grpc-daemon[485]: ts=2026-09-25T15:00:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 paths=1 nar_bytes_in=7524283worker1 # [ 70.420627] nix-grpc-daemon[485]: ts=2026-09-25T15:00:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904284worker1 # [ 70.429048] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4285worker1 # [ 70.433358] nix-daemon[855]: reaped child process 979, status = succeeded4286client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4287worker1 # [ 70.442739] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4288client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4289client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4290client # farm-top> LOG-farm-top-sig4291worker1 # [ 70.909992] nix-daemon[855]: reaped child process 988, status = succeeded4292worker1 # [ 70.972138] nix-daemon[855]: accepted connection from pid 1004, user nix-grpc-daemon (trusted)4293worker1 # [ 70.988494] nix-daemon[855]: reaped child process 1007, status = succeeded4294lb # [ 70.970265] niks3-server[1035]: 2026/09/25 15:00:31 INFO Received push request method=POST path=/api/pushes4295worker1 # [ 71.000118] nix-grpc-daemon[905]: time=2026-09-25T15:00:31.530Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4296worker1 # [ 71.001615] nix-grpc-daemon[905]: time=2026-09-25T15:00:31.530Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4297lb # [ 70.991538] niks3-server[1035]: 2026/09/25 15:00:31 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4298lb # [ 71.000430] niks3-server[1035]: 2026/09/25 15:00:31 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4299worker1 # [ 71.029207] nix-grpc-daemon[905]: time=2026-09-25T15:00:31.561Z level=INFO msg="Uploading 1 narinfos"4300lb # [ 71.008416] niks3-server[1035]: 2026/09/25 15:00:31 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4301lb # [ 71.014458] niks3-server[1035]: 2026/09/25 15:00:31 INFO Signed narinfos id=14 count=14302lb # [ 71.021165] niks3-server[1035]: 2026/09/25 15:00:31 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4303lb # [ 71.023625] niks3-server[1035]: 2026/09/25 15:00:31 INFO Received complete push request method=POST path=/api/pushes/14/complete4304worker1 # [ 71.052879] nix-grpc-daemon[905]: time=2026-09-25T15:00:31.584Z level=INFO msg="Upload complete. (200ms)"4305lb # [ 71.030814] niks3-server[1035]: 2026/09/25 15:00:31 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4306worker1 # [ 71.059960] nix-grpc-daemon[485]: ts=2026-09-25T15:00:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14307worker1 # [ 71.071317] nix-daemon[855]: reaped child process 986, status = succeeded4308client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.23 seconds)4309client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4310client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.14 seconds)4311client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4312worker1 # [ 71.344731] nix-grpc-daemon[485]: ts=2026-09-25T15:00:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:587144313worker1 # [ 71.349532] nix-grpc-daemon[485]: ts=2026-09-25T15:00:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:587144314client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.14 seconds)4315client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4316worker2 # [ 71.460772] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:577664317client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4318client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4319client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4320client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4321worker2 # [ 71.477640] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:577664322worker2 # [ 71.482595] nix-grpc-daemon[786]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4323worker2 # [ 71.491552] nix-grpc-daemon[786]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4324worker2 # [ 71.499291] nix-grpc-daemon[786]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4325client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.19 seconds)4326client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4327client # finding garbage collector roots...4328client # removing stale temporary roots file "/nix/var/nix/temproots/855"4329client # removing stale temporary roots file "/nix/var/nix/temproots/814"4330client # removing stale temporary roots file "/nix/var/nix/temproots/868"4331client # removing stale temporary roots file "/nix/var/nix/temproots/891"4332client # removing stale temporary roots file "/nix/var/nix/temproots/835"4333client # removing stale temporary roots file "/nix/var/nix/temproots/799"4334client # removing stale temporary roots file "/nix/var/nix/temproots/847"4335client # removing stale temporary roots file "/nix/var/nix/temproots/876"4336client # deleting specified paths...4337client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4338client # deleting unused links...4339client # note: hard linking is currently saving 0.0 KiB4340worker1 # [ 71.843182] nix-grpc-daemon[485]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537904341worker2 # [ 71.821181] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624342worker2 # [ 71.828770] nix-grpc-daemon[857]: time=2026-09-25T15:00:32.370Z level=INFO msg="All 1 paths already cached"4343worker2 # [ 71.833571] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:502624344worker2 # [ 71.838668] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:502624345client # copying 3 paths...4346worker2 # [ 71.847468] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:502624347client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4348worker2 # [ 71.863498] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:502624349worker2 # [ 71.867869] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_out=2324350worker1 # [ 71.899923] nix-grpc-daemon[485]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537904351worker1 # [ 71.904529] nix-grpc-daemon[485]: ts=2026-09-25T15:00:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 paths=1 nar_bytes_out=1764352worker2 # [ 71.880944] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:502624353worker2 # [ 71.884638] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_out=1284354client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4355client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4356client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.39 seconds)4357(finished: subtest: outputs read back through any worker carry the cache signature, in 6.08 seconds)4358subtest: two clients wanting the same drv build it once4359client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4360client # [ 71.951523] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4361client # Running as unit: dedup1.service; invocation ID: 9ce9d87ec96149ad880e84025925a71c4362client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.04 seconds)4363client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4364worker1 # [ 72.136220] nix-grpc-daemon[485]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537904365client # [ 72.117886] nix[944]: don't know how to build these paths:4366client # [ 72.119684] nix[944]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4367worker1 # [ 72.148122] nix-grpc-daemon[485]: ts=2026-09-25T15:00:32Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04368worker1 # [ 72.152228] nix-grpc-daemon[485]: ts=2026-09-25T15:00:32Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14369worker2 # [ 72.127624] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624370worker2 # [ 72.135944] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624371client # [ 72.148110] nix[944]: copying 1 paths to 'lb:50051'...4372client # [ 72.148976] nix[944]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4373worker2 # [ 72.143615] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502624374worker2 # [ 72.153144] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4375worker2 # [ 72.285439] nix-daemon[845]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4376worker2 # [ 72.303823] nix-daemon[845]: reaped child process 1009, status = succeeded4377lb # [ 72.313794] niks3-server[1035]: 2026/09/25 15:00:32 INFO Received push request method=POST path=/api/pushes4378worker2 # [ 72.320114] nix-grpc-daemon[857]: time=2026-09-25T15:00:32.862Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4379worker2 # [ 72.321806] nix-grpc-daemon[857]: time=2026-09-25T15:00:32.862Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4380lb # [ 72.339584] niks3-server[1035]: 2026/09/25 15:00:32 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4381worker2 # [ 72.343577] nix-grpc-daemon[857]: time=2026-09-25T15:00:32.885Z level=INFO msg="Uploading 1 narinfos"4382lb # [ 72.348673] niks3-server[1035]: 2026/09/25 15:00:32 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4383lb # [ 72.350982] niks3-server[1035]: 2026/09/25 15:00:32 INFO Signed narinfos id=15 count=14384lb # [ 72.358670] niks3-server[1035]: 2026/09/25 15:00:32 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4385lb # [ 72.366992] niks3-server[1035]: 2026/09/25 15:00:32 INFO Received complete push request method=POST path=/api/pushes/15/complete4386worker2 # [ 72.367590] nix-grpc-daemon[857]: time=2026-09-25T15:00:32.910Z level=INFO msg="Upload complete. (203ms)"4387lb # [ 72.372598] niks3-server[1035]: 2026/09/25 15:00:32 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4388worker2 # [ 72.375963] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_in=4324389worker2 # [ 72.386088] nix-grpc-daemon[786]: ts=2026-09-25T15:00:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624390worker2 # [ 72.391127] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4391worker2 # [ 72.395498] nix-daemon[845]: reaped child process 1004, status = succeeded4392client # [ 72.408739] nix[944]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4393worker2 # [ 72.406087] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4394client # [ 73.014200] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4395client # Running as unit: dedup2.service; invocation ID: 2f92b5f2d3f142f49d023d893e4620dd4396client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.07 seconds)4397worker1 # [ 73.240313] nix-grpc-daemon[485]: ts=2026-09-25T15:00:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537904398client # [ 73.222782] nix[960]: don't know how to build these paths:4399client # [ 73.224063] nix[960]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4400worker1 # [ 73.253820] nix-grpc-daemon[485]: ts=2026-09-25T15:00:33Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04401worker2 # [ 73.231589] nix-grpc-daemon[786]: ts=2026-09-25T15:00:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624402client # [ 73.240836] nix[960]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4403client: must succeed: sleep 24404client: (finished: must succeed: sleep 2, in 2.03 seconds)4405client: waiting for success: ! systemctl is-active dedup1 dedup24406worker2 # [ 76.448272] nix-daemon[845]: reaped child process 1013, status = succeeded4407worker2 # [ 76.550127] nix-daemon[845]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)4408worker2 # [ 76.566422] nix-daemon[845]: reaped child process 1046, status = succeeded4409lb # [ 76.576640] niks3-server[1035]: 2026/09/25 15:00:37 INFO Received push request method=POST path=/api/pushes4410worker2 # [ 76.580462] nix-grpc-daemon[857]: time=2026-09-25T15:00:37.121Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4411worker2 # [ 76.581992] nix-grpc-daemon[857]: time=2026-09-25T15:00:37.121Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4412lb # [ 76.603510] niks3-server[1035]: 2026/09/25 15:00:37 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4413worker2 # [ 76.607812] nix-grpc-daemon[857]: time=2026-09-25T15:00:37.150Z level=INFO msg="Uploading 1 narinfos"4414lb # [ 76.611944] niks3-server[1035]: 2026/09/25 15:00:37 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4415lb # [ 76.617495] niks3-server[1035]: 2026/09/25 15:00:37 INFO Signed narinfos id=16 count=14416lb # [ 76.622182] niks3-server[1035]: 2026/09/25 15:00:37 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4417lb # [ 76.629496] niks3-server[1035]: 2026/09/25 15:00:37 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4418lb # [ 76.638568] niks3-server[1035]: 2026/09/25 15:00:37 INFO Received complete push request method=POST path=/api/pushes/16/complete4419worker2 # [ 76.639257] nix-grpc-daemon[857]: time=2026-09-25T15:00:37.181Z level=INFO msg="Upload complete. (222ms)"4420lb # [ 76.644279] niks3-server[1035]: 2026/09/25 15:00:37 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4421worker2 # [ 76.648694] nix-grpc-daemon[786]: ts=2026-09-25T15:00:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14422worker2 # [ 76.657839] nix-grpc-daemon[786]: ts=2026-09-25T15:00:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14423worker2 # [ 76.662291] nix-daemon[845]: reaped child process 1011, status = succeeded4424client # [ 76.691668] systemd[1]: dedup1.service: Deactivated successfully.4425client # [ 76.694361] systemd[1]: dedup1.service: Consumed 149ms CPU time over 4.742s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.4426client # [ 76.701776] systemd[1]: dedup2.service: Deactivated successfully.4427client # [ 76.703191] systemd[1]: dedup2.service: Consumed 185ms CPU time over 3.686s wall clock time, 35.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4428client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.07 seconds)4429client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4430client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4431worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4432worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4433worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4434worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4435worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4436worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4437worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4438worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4439(finished: subtest: two clients wanting the same drv build it once, in 5.69 seconds)4440subtest: --timeout and --max-silent-time reach the builder4441worker2 # [ 77.785264] nix-grpc-daemon[786]: ts=2026-09-25T15:00:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502624442worker1 # [ 77.823995] nix-grpc-daemon[485]: ts=2026-09-25T15:00:38Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04443worker1 # [ 77.828624] nix-grpc-daemon[485]: ts=2026-09-25T15:00:38Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14444worker1 # [ 77.833257] nix-grpc-daemon[485]: ts=2026-09-25T15:00:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904445worker1 # [ 77.837273] nix-grpc-daemon[485]: ts=2026-09-25T15:00:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904446worker1 # [ 77.845068] nix-grpc-daemon[485]: ts=2026-09-25T15:00:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537904447worker1 # [ 77.853648] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4448worker1 # [ 77.980296] nix-daemon[855]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)4449worker1 # [ 77.995648] nix-daemon[855]: reaped child process 1041, status = succeeded4450lb # [ 77.979543] niks3-server[1035]: 2026/09/25 15:00:38 INFO Received push request method=POST path=/api/pushes4451worker1 # [ 78.008727] nix-grpc-daemon[905]: time=2026-09-25T15:00:38.540Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4452worker1 # [ 78.010347] nix-grpc-daemon[905]: time=2026-09-25T15:00:38.540Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4453lb # [ 77.993512] niks3-server[1035]: 2026/09/25 15:00:38 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4454worker1 # [ 78.022272] nix-grpc-daemon[905]: time=2026-09-25T15:00:38.554Z level=INFO msg="Uploading 1 narinfos"4455lb # [ 78.000538] niks3-server[1035]: 2026/09/25 15:00:38 INFO Signed narinfos id=17 count=14456lb # [ 78.005207] niks3-server[1035]: 2026/09/25 15:00:38 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4457lb # [ 78.011792] niks3-server[1035]: 2026/09/25 15:00:38 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4458lb # [ 78.020690] niks3-server[1035]: 2026/09/25 15:00:38 INFO Received complete push request method=POST path=/api/pushes/17/complete4459lb # [ 78.024455] niks3-server[1035]: 2026/09/25 15:00:38 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4460worker1 # [ 78.051779] nix-grpc-daemon[905]: time=2026-09-25T15:00:38.583Z level=INFO msg="Upload complete. (188ms)"4461worker1 # [ 78.057582] nix-grpc-daemon[485]: ts=2026-09-25T15:00:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 paths=1 nar_bytes_in=4244462worker1 # [ 78.064757] nix-grpc-daemon[485]: ts=2026-09-25T15:00:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904463worker1 # [ 78.073422] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4464worker1 # [ 78.077370] nix-daemon[855]: reaped child process 1036, status = succeeded4465worker1 # [ 78.086087] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4466lb # [ 78.261084] envoy[897]: conn peer=[2001:db8:1::1]:54084 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9511 tx=4329 ms=50324467lb # [ 78.263635] envoy[897]: conn peer=[2001:db8:1::1]:33380 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2868 ms=884468lb # [ 78.266364] envoy[897]: conn peer=[2001:db8:1::1]:33402 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=244469lb # [ 78.268980] envoy[897]: conn peer=[2001:db8:1::1]:33388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=414470lb # [ 78.271577] envoy[897]: conn peer=[2001:db8:1::1]:33396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=334471lb # [ 78.274354] envoy[897]: conn peer=[2001:db8:1::1]:33404 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=1521 ms=45514472lb # [ 78.277555] envoy[897]: conn peer=[2001:db8:1::1]:33418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1194 ms=34544473worker1 # [ 84.301593] nix-grpc-daemon[485]: ts=2026-09-25T15:00:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04474worker1 # [ 84.327770] nix-daemon[855]: reaped child process 1043, status = succeeded4475worker1 # [ 84.329989] nix-daemon[855]: reaped child process 1045, status = succeeded4476worker1 # [ 84.494222] nix-grpc-daemon[485]: ts=2026-09-25T15:00:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537904477worker1 # [ 84.503963] nix-grpc-daemon[485]: ts=2026-09-25T15:00:45Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04478worker2 # [ 84.479333] nix-grpc-daemon[786]: ts=2026-09-25T15:00:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624479worker1 # [ 84.507064] nix-grpc-daemon[485]: ts=2026-09-25T15:00:45Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14480worker2 # [ 84.489538] nix-grpc-daemon[786]: ts=2026-09-25T15:00:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624481worker2 # [ 84.497930] nix-grpc-daemon[786]: ts=2026-09-25T15:00:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502624482worker2 # [ 84.506736] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4483worker2 # [ 84.636565] nix-daemon[845]: accepted connection from pid 1059, user nix-grpc-daemon (trusted)4484worker2 # [ 84.653398] nix-daemon[845]: reaped child process 1062, status = succeeded4485lb # [ 84.663247] niks3-server[1035]: 2026/09/25 15:00:45 INFO Received push request method=POST path=/api/pushes4486worker2 # [ 84.666341] nix-grpc-daemon[857]: time=2026-09-25T15:00:45.208Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4487worker2 # [ 84.667875] nix-grpc-daemon[857]: time=2026-09-25T15:00:45.208Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4488lb # [ 84.684565] niks3-server[1035]: 2026/09/25 15:00:45 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4489worker2 # [ 84.690414] nix-grpc-daemon[857]: time=2026-09-25T15:00:45.232Z level=INFO msg="Uploading 1 narinfos"4490lb # [ 84.694630] niks3-server[1035]: 2026/09/25 15:00:45 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4491lb # [ 84.700827] niks3-server[1035]: 2026/09/25 15:00:45 INFO Signed narinfos id=18 count=14492lb # [ 84.704648] niks3-server[1035]: 2026/09/25 15:00:45 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4493lb # [ 84.716649] niks3-server[1035]: 2026/09/25 15:00:45 INFO Received complete push request method=POST path=/api/pushes/18/complete4494worker2 # [ 84.718819] nix-grpc-daemon[857]: time=2026-09-25T15:00:45.261Z level=INFO msg="Upload complete. (202ms)"4495lb # [ 84.722995] niks3-server[1035]: 2026/09/25 15:00:45 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4496worker2 # [ 84.725685] nix-grpc-daemon[786]: ts=2026-09-25T15:00:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_in=4244497worker2 # [ 84.732657] nix-grpc-daemon[786]: ts=2026-09-25T15:00:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624498worker2 # [ 84.743554] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4499worker2 # [ 84.745844] nix-daemon[845]: reaped child process 1057, status = succeeded4500worker2 # [ 84.757229] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4501lb # [ 88.263124] envoy[897]: conn peer=[2001:db8:1::1]:33420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2971 tx=956 ms=65044502worker2 # [ 90.973372] nix-grpc-daemon[786]: ts=2026-09-25T15:00:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04503worker2 # [ 90.990176] nix-daemon[845]: reaped child process 1064, status = succeeded4504worker2 # [ 90.991345] nix-daemon[845]: reaped child process 1066, status = succeeded4505(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.46 seconds)4506subtest: the first client leaving does not fail the second4507client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4508client # [ 91.115155] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4509client # Running as unit: share1.service; invocation ID: 59d8cb92c43c496d8db79f95ef426c064510client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.05 seconds)4511client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4512worker2 # [ 91.297344] nix-grpc-daemon[786]: ts=2026-09-25T15:00:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502624513client # [ 91.306729] nix[1032]: don't know how to build these paths:4514client # [ 91.308470] nix[1032]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4515worker1 # [ 91.337444] nix-grpc-daemon[485]: ts=2026-09-25T15:00:51Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04516worker1 # [ 91.341581] nix-grpc-daemon[485]: ts=2026-09-25T15:00:51Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14517worker1 # [ 91.345836] nix-grpc-daemon[485]: ts=2026-09-25T15:00:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904518worker1 # [ 91.351957] nix-grpc-daemon[485]: ts=2026-09-25T15:00:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904519client # [ 91.337795] nix[1032]: copying 1 paths to 'lb:50051'...4520worker1 # [ 91.360005] nix-grpc-daemon[485]: ts=2026-09-25T15:00:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537904521client # [ 91.338934] nix[1032]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4522worker1 # [ 91.368749] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4523worker1 # [ 91.497135] nix-daemon[855]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4524worker1 # [ 91.516089] nix-daemon[855]: reaped child process 1075, status = succeeded4525lb # [ 91.491050] niks3-server[1035]: 2026/09/25 15:00:52 INFO Received push request method=POST path=/api/pushes4526worker1 # [ 91.520373] nix-grpc-daemon[905]: time=2026-09-25T15:00:52.051Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4527worker1 # [ 91.522109] nix-grpc-daemon[905]: time=2026-09-25T15:00:52.051Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4528lb # [ 91.505533] niks3-server[1035]: 2026/09/25 15:00:52 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4529lb # [ 91.514047] niks3-server[1035]: 2026/09/25 15:00:52 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4530worker1 # [ 91.540967] nix-grpc-daemon[905]: time=2026-09-25T15:00:52.072Z level=INFO msg="Uploading 1 narinfos"4531lb # [ 91.521346] niks3-server[1035]: 2026/09/25 15:00:52 INFO Signed narinfos id=19 count=14532lb # [ 91.528676] niks3-server[1035]: 2026/09/25 15:00:52 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4533lb # [ 91.540068] niks3-server[1035]: 2026/09/25 15:00:52 INFO Received complete push request method=POST path=/api/pushes/19/complete4534worker1 # [ 91.568033] nix-grpc-daemon[905]: time=2026-09-25T15:00:52.100Z level=INFO msg="Upload complete. (190ms)"4535lb # [ 91.546597] niks3-server[1035]: 2026/09/25 15:00:52 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4536worker1 # [ 91.575753] nix-grpc-daemon[485]: ts=2026-09-25T15:00:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 paths=1 nar_bytes_in=4324537worker1 # [ 91.584994] nix-grpc-daemon[485]: ts=2026-09-25T15:00:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904538worker1 # [ 91.589555] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4539worker1 # [ 91.596271] nix-daemon[855]: reaped child process 1070, status = succeeded4540client # [ 91.579309] nix[1032]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4541worker1 # [ 91.604006] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4542client # [ 92.192836] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4543client # Running as unit: share2.service; invocation ID: 619300efc577488295a2f37193cf055f4544client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.08 seconds)4545worker2 # [ 92.374460] nix-grpc-daemon[786]: ts=2026-09-25T15:00:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502624546client # [ 92.384247] nix[1048]: don't know how to build these paths:4547client # [ 92.385998] nix[1048]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4548worker1 # [ 92.416008] nix-grpc-daemon[485]: ts=2026-09-25T15:00:52Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04549worker1 # [ 92.420624] nix-grpc-daemon[485]: ts=2026-09-25T15:00:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904550client # [ 92.402686] nix[1048]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4551client: must succeed: sleep 2; systemctl kill -s INT share14552client # [ 95.427549] systemd[1]: share1.service: Sent signal SIGINT to main process 1032 (nix) on client request.4553client # [ 95.431183] nix[1032]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4554client # [ 95.432567] nix[1032]: error: interrupted by the user4555client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.05 seconds)4556client: waiting for success: ! systemctl is-active share14557client # [ 95.485850] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4558client # [ 95.488439] systemd[1]: share1.service: Failed with result 'exit-code'.4559client # [ 95.491376] systemd[1]: share1.service: Consumed 161ms CPU time over 4.373s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.4560client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)4561client: waiting for success: ! systemctl is-active share24562worker1 # [ 96.838413] nix-daemon[855]: reaped child process 1079, status = succeeded4563worker1 # [ 96.949083] nix-daemon[855]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4564worker1 # [ 96.969328] nix-daemon[855]: reaped child process 1118, status = succeeded4565lb # [ 96.948847] niks3-server[1035]: 2026/09/25 15:00:57 INFO Received push request method=POST path=/api/pushes4566worker1 # [ 96.980464] nix-grpc-daemon[905]: time=2026-09-25T15:00:57.511Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4567worker1 # [ 96.981966] nix-grpc-daemon[905]: time=2026-09-25T15:00:57.511Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4568lb # [ 96.979383] niks3-server[1035]: 2026/09/25 15:00:57 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4569worker1 # [ 97.010169] nix-grpc-daemon[905]: time=2026-09-25T15:00:57.542Z level=INFO msg="Uploading 1 narinfos"4570lb # [ 96.986993] niks3-server[1035]: 2026/09/25 15:00:57 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign4571lb # [ 96.993511] niks3-server[1035]: 2026/09/25 15:00:57 INFO Signed narinfos id=20 count=14572lb # [ 96.995216] niks3-server[1035]: 2026/09/25 15:00:57 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4573lb # [ 97.007739] niks3-server[1035]: 2026/09/25 15:00:57 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4574worker1 # [ 97.038096] nix-grpc-daemon[905]: time=2026-09-25T15:00:57.570Z level=INFO msg="Upload complete. (222ms)"4575lb # [ 97.014706] niks3-server[1035]: 2026/09/25 15:00:57 INFO Received complete push request method=POST path=/api/pushes/20/complete4576lb # [ 97.017893] niks3-server[1035]: 2026/09/25 15:00:57 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4577worker1 # [ 97.044724] nix-grpc-daemon[485]: ts=2026-09-25T15:00:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14578worker1 # [ 97.057286] nix-grpc-daemon[485]: ts=2026-09-25T15:00:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14579worker1 # [ 97.061619] nix-daemon[855]: reaped child process 1077, status = succeeded4580client # [ 97.053318] systemd[1]: share2.service: Deactivated successfully.4581client # [ 97.056218] systemd[1]: share2.service: Consumed 167ms CPU time over 4.862s wall clock time, 33.3M memory peak, 6K incoming IP traffic, 5.6K outgoing IP traffic.4582client: (finished: waiting for success: ! systemctl is-active share2, in 1.07 seconds)4583client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4584client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds)4585(finished: subtest: the first client leaving does not fail the second, in 6.72 seconds)4586subtest: build hook: nix-daemon with builders = grpc://lb4587client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4588client # [ 97.922123] systemd[1]: Started Nix Daemon.4589client # [ 98.030088] nix-daemon[1100]: accepted connection from pid 1097, user root (trusted)4590worker1 # [ 98.191539] nix-grpc-daemon[485]: ts=2026-09-25T15:00:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:537904591worker1 # [ 98.195376] nix-grpc-daemon[485]: ts=2026-09-25T15:00:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904592worker1 # [ 98.204097] nix-grpc-daemon[485]: ts=2026-09-25T15:00:58Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04593worker1 # [ 98.206988] nix-grpc-daemon[485]: ts=2026-09-25T15:00:58Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14594worker2 # [ 98.186184] nix-grpc-daemon[786]: ts=2026-09-25T15:00:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624595worker2 # [ 98.195208] nix-grpc-daemon[786]: ts=2026-09-25T15:00:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624596worker2 # [ 98.203327] nix-grpc-daemon[786]: ts=2026-09-25T15:00:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502624597worker2 # [ 98.212414] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4598lb # [ 98.264361] envoy[897]: conn peer=[2001:db8:1::1]:44874 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=934 ms=65194599lb # [ 98.267014] envoy[897]: conn peer=[2001:db8:1::1]:60214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=769 ms=41434600lb # [ 98.270571] envoy[897]: conn peer=[2001:db8:1::1]:60228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=46624601worker2 # [ 98.341246] nix-daemon[845]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)4602worker2 # [ 98.358077] nix-daemon[845]: reaped child process 1108, status = succeeded4603lb # [ 98.366539] niks3-server[1035]: 2026/09/25 15:00:58 INFO Received push request method=POST path=/api/pushes4604worker2 # [ 98.368984] nix-grpc-daemon[857]: time=2026-09-25T15:00:58.910Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4605worker2 # [ 98.370522] nix-grpc-daemon[857]: time=2026-09-25T15:00:58.910Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4606lb # [ 98.384325] niks3-server[1035]: 2026/09/25 15:00:58 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4607worker2 # [ 98.391655] nix-grpc-daemon[857]: time=2026-09-25T15:00:58.934Z level=INFO msg="Uploading 1 narinfos"4608lb # [ 98.390869] niks3-server[1035]: 2026/09/25 15:00:58 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign4609lb # [ 98.399034] niks3-server[1035]: 2026/09/25 15:00:58 INFO Signed narinfos id=21 count=14610lb # [ 98.402357] niks3-server[1035]: 2026/09/25 15:00:58 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4611lb # [ 98.410013] niks3-server[1035]: 2026/09/25 15:00:58 INFO Received complete push request method=POST path=/api/pushes/21/complete4612worker2 # [ 98.414028] nix-grpc-daemon[857]: time=2026-09-25T15:00:58.956Z level=INFO msg="Upload complete. (192ms)"4613lb # [ 98.418512] niks3-server[1035]: 2026/09/25 15:00:58 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4614worker2 # [ 98.420191] nix-grpc-daemon[786]: ts=2026-09-25T15:00:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_in=5124615worker2 # [ 98.423547] nix-grpc-daemon[786]: ts=2026-09-25T15:00:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624616worker2 # [ 98.437147] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4617worker2 # [ 98.439575] nix-daemon[845]: reaped child process 1103, status = succeeded4618worker2 # [ 98.451034] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4619worker2 # [ 99.921975] nix-daemon[845]: reaped child process 1112, status = succeeded4620worker2 # [ 99.980929] nix-daemon[845]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)4621worker2 # [ 99.998884] nix-daemon[845]: reaped child process 1131, status = succeeded4622lb # [ 100.007419] niks3-server[1035]: 2026/09/25 15:01:00 INFO Received push request method=POST path=/api/pushes4623worker2 # [ 100.010421] nix-grpc-daemon[857]: time=2026-09-25T15:01:00.551Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4624worker2 # [ 100.012101] nix-grpc-daemon[857]: time=2026-09-25T15:01:00.551Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4625worker2 # [ 100.032521] nix-grpc-daemon[857]: time=2026-09-25T15:01:00.574Z level=INFO msg="Uploading 1 narinfos"4626lb # [ 100.030947] niks3-server[1035]: 2026/09/25 15:01:00 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign4627lb # [ 100.039721] niks3-server[1035]: 2026/09/25 15:01:00 INFO Signed narinfos id=22 count=14628lb # [ 100.044679] niks3-server[1035]: 2026/09/25 15:01:00 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4629lb # [ 100.047135] niks3-server[1035]: 2026/09/25 15:01:00 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4630lb # [ 100.059530] niks3-server[1035]: 2026/09/25 15:01:00 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4631lb # [ 100.064075] niks3-server[1035]: 2026/09/25 15:01:00 INFO Received complete push request method=POST path=/api/pushes/22/complete4632worker2 # [ 100.064113] nix-grpc-daemon[857]: time=2026-09-25T15:01:00.606Z level=INFO msg="Upload complete. (204ms)"4633lb # [ 100.069917] niks3-server[1035]: 2026/09/25 15:01:00 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4634worker2 # [ 100.072587] nix-grpc-daemon[786]: ts=2026-09-25T15:01:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14635worker2 # [ 100.082731] nix-grpc-daemon[786]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:502624636worker2 # [ 100.086345] nix-daemon[845]: reaped child process 1110, status = succeeded4637worker2 # [ 100.098751] nix-grpc-daemon[786]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:502624638worker2 # [ 100.102712] nix-grpc-daemon[786]: ts=2026-09-25T15:01:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_out=1284639worker2 # [ 100.301326] nix-grpc-daemon[786]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:502624640worker2 # [ 100.305619] nix-grpc-daemon[786]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624641worker2 # [ 100.309672] nix-grpc-daemon[857]: time=2026-09-25T15:01:00.851Z level=INFO msg="All 1 paths already cached"4642worker1 # [ 100.345792] nix-grpc-daemon[485]: ts=2026-09-25T15:01:00Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04643worker2 # [ 100.319707] nix-grpc-daemon[786]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624644worker1 # [ 100.346011] nix-grpc-daemon[485]: ts=2026-09-25T15:01:00Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14645worker2 # [ 100.328552] nix-grpc-daemon[786]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624646worker2 # [ 100.336190] nix-grpc-daemon[857]: time=2026-09-25T15:01:00.878Z level=INFO msg="All 1 paths already cached"4647worker2 # [ 100.341578] nix-grpc-daemon[786]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502624648worker2 # [ 100.349438] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4649worker2 # [ 100.474901] nix-daemon[845]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)4650worker2 # [ 100.492839] nix-daemon[845]: reaped child process 1138, status = succeeded4651lb # [ 100.502258] niks3-server[1035]: 2026/09/25 15:01:01 INFO Received push request method=POST path=/api/pushes4652worker2 # [ 100.503375] nix-grpc-daemon[857]: time=2026-09-25T15:01:01.045Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4653worker2 # [ 100.505063] nix-grpc-daemon[857]: time=2026-09-25T15:01:01.045Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4654lb # [ 100.517661] niks3-server[1035]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4655worker2 # [ 100.522647] nix-grpc-daemon[857]: time=2026-09-25T15:01:01.065Z level=INFO msg="Uploading 1 narinfos"4656lb # [ 100.527851] niks3-server[1035]: 2026/09/25 15:01:01 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign4657lb # [ 100.533673] niks3-server[1035]: 2026/09/25 15:01:01 INFO Signed narinfos id=23 count=14658lb # [ 100.535413] niks3-server[1035]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4659lb # [ 100.543530] niks3-server[1035]: 2026/09/25 15:01:01 INFO Received complete push request method=POST path=/api/pushes/23/complete4660lb # [ 100.550118] niks3-server[1035]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4661worker2 # [ 100.549359] nix-grpc-daemon[857]: time=2026-09-25T15:01:01.091Z level=INFO msg="Upload complete. (190ms)"4662worker2 # [ 100.555801] nix-grpc-daemon[786]: ts=2026-09-25T15:01:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_in=6244663worker2 # [ 100.567191] nix-grpc-daemon[786]: ts=2026-09-25T15:01:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624664worker2 # [ 100.571819] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4665worker2 # [ 100.575673] nix-daemon[845]: reaped child process 1133, status = succeeded4666worker2 # [ 100.584339] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4667worker2 # [ 101.261564] nix-daemon[845]: reaped child process 1142, status = succeeded4668worker2 # [ 101.318939] nix-daemon[845]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4669worker2 # [ 101.337444] nix-daemon[845]: reaped child process 1161, status = succeeded4670lb # [ 101.345502] niks3-server[1035]: 2026/09/25 15:01:01 INFO Received push request method=POST path=/api/pushes4671worker2 # [ 101.348379] nix-grpc-daemon[857]: time=2026-09-25T15:01:01.889Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4672worker2 # [ 101.349918] nix-grpc-daemon[857]: time=2026-09-25T15:01:01.889Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4673lb # [ 101.375872] niks3-server[1035]: 2026/09/25 15:01:01 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign4674worker2 # [ 101.378281] nix-grpc-daemon[857]: time=2026-09-25T15:01:01.920Z level=INFO msg="Uploading 1 narinfos"4675lb # [ 101.384221] niks3-server[1035]: 2026/09/25 15:01:01 INFO Signed narinfos id=24 count=14676lb # [ 101.388218] niks3-server[1035]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4677lb # [ 101.396675] niks3-server[1035]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4678worker2 # [ 101.398138] nix-grpc-daemon[857]: time=2026-09-25T15:01:01.940Z level=INFO msg="Upload complete. (204ms)"4679lb # [ 101.402952] niks3-server[1035]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4680lb # [ 101.407977] niks3-server[1035]: 2026/09/25 15:01:01 INFO Received complete push request method=POST path=/api/pushes/24/complete4681worker2 # [ 101.404668] nix-grpc-daemon[786]: ts=2026-09-25T15:01:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14682lb # [ 101.411155] niks3-server[1035]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4683worker2 # [ 101.417455] nix-grpc-daemon[786]: ts=2026-09-25T15:01:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:502624684worker2 # [ 101.420961] nix-daemon[845]: reaped child process 1140, status = succeeded4685worker2 # [ 101.433333] nix-grpc-daemon[786]: ts=2026-09-25T15:01:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:502624686worker2 # [ 101.437076] nix-grpc-daemon[786]: ts=2026-09-25T15:01:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_out=1764687worker2 # [ 101.630469] nix-grpc-daemon[786]: ts=2026-09-25T15:01:02Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:502624688worker2 # [ 101.634723] nix-grpc-daemon[786]: ts=2026-09-25T15:01:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624689worker2 # [ 101.638985] nix-grpc-daemon[857]: time=2026-09-25T15:01:02.180Z level=INFO msg="All 2 paths already cached"4690worker1 # [ 101.674709] nix-grpc-daemon[485]: ts=2026-09-25T15:01:02Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04691worker1 # [ 101.678272] nix-grpc-daemon[485]: ts=2026-09-25T15:01:02Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14692worker2 # [ 101.652812] nix-grpc-daemon[786]: ts=2026-09-25T15:01:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624693worker2 # [ 101.661667] nix-grpc-daemon[786]: ts=2026-09-25T15:01:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624694worker2 # [ 101.669558] nix-grpc-daemon[857]: time=2026-09-25T15:01:02.212Z level=INFO msg="All 2 paths already cached"4695worker2 # [ 101.675378] nix-grpc-daemon[786]: ts=2026-09-25T15:01:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502624696worker2 # [ 101.683538] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4697worker2 # [ 101.815824] nix-daemon[845]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)4698lb # [ 101.837299] niks3-server[1035]: 2026/09/25 15:01:02 INFO Received push request method=POST path=/api/pushes4699worker2 # [ 101.837254] nix-daemon[845]: reaped child process 1168, status = succeeded4700worker2 # [ 101.839072] nix-grpc-daemon[857]: time=2026-09-25T15:01:02.381Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4701worker2 # [ 101.840771] nix-grpc-daemon[857]: time=2026-09-25T15:01:02.381Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4702lb # [ 101.854769] niks3-server[1035]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4703worker2 # [ 101.858309] nix-grpc-daemon[857]: time=2026-09-25T15:01:02.400Z level=INFO msg="Uploading 1 narinfos"4704lb # [ 101.863988] niks3-server[1035]: 2026/09/25 15:01:02 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign4705lb # [ 101.866772] niks3-server[1035]: 2026/09/25 15:01:02 INFO Signed narinfos id=25 count=14706lb # [ 101.872640] niks3-server[1035]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4707worker2 # [ 101.877246] nix-grpc-daemon[857]: time=2026-09-25T15:01:02.419Z level=INFO msg="Upload complete. (183ms)"4708lb # [ 101.881253] niks3-server[1035]: 2026/09/25 15:01:02 INFO Received complete push request method=POST path=/api/pushes/25/complete4709lb # [ 101.885412] niks3-server[1035]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4710worker2 # [ 101.884160] nix-grpc-daemon[786]: ts=2026-09-25T15:01:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_in=7524711worker2 # [ 101.888640] nix-grpc-daemon[786]: ts=2026-09-25T15:01:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624712worker2 # [ 101.896185] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4713worker2 # [ 101.899435] nix-daemon[845]: reaped child process 1163, status = succeeded4714worker2 # [ 101.909364] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4715worker2 # [ 103.589374] nix-daemon[845]: reaped child process 1172, status = succeeded4716worker2 # [ 103.652568] nix-daemon[845]: accepted connection from pid 1188, user nix-grpc-daemon (trusted)4717worker2 # [ 103.668461] nix-daemon[845]: reaped child process 1191, status = succeeded4718lb # [ 103.677909] niks3-server[1035]: 2026/09/25 15:01:04 INFO Received push request method=POST path=/api/pushes4719worker2 # [ 103.681719] nix-grpc-daemon[857]: time=2026-09-25T15:01:04.223Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4720worker2 # [ 103.683316] nix-grpc-daemon[857]: time=2026-09-25T15:01:04.223Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4721lb # [ 103.700924] niks3-server[1035]: 2026/09/25 15:01:04 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign4722worker2 # [ 103.702755] nix-grpc-daemon[857]: time=2026-09-25T15:01:04.245Z level=INFO msg="Uploading 1 narinfos"4723lb # [ 103.709395] niks3-server[1035]: 2026/09/25 15:01:04 INFO Signed narinfos id=26 count=14724lb # [ 103.714506] niks3-server[1035]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4725lb # [ 103.720673] niks3-server[1035]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4726lb # [ 103.723558] niks3-server[1035]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4727lb # [ 103.734779] niks3-server[1035]: 2026/09/25 15:01:04 INFO Received complete push request method=POST path=/api/pushes/26/complete4728worker2 # [ 103.734000] nix-grpc-daemon[857]: time=2026-09-25T15:01:04.276Z level=INFO msg="Upload complete. (216ms)"4729lb # [ 103.738219] niks3-server[1035]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4730worker2 # [ 103.740677] nix-grpc-daemon[786]: ts=2026-09-25T15:01:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14731worker1 # [ 103.774467] nix-grpc-daemon[485]: ts=2026-09-25T15:01:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537904732worker2 # [ 103.751624] nix-daemon[845]: reaped child process 1170, status = succeeded4733worker1 # [ 103.789106] nix-grpc-daemon[485]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4734worker1 # [ 103.798712] nix-grpc-daemon[485]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4735worker1 # [ 103.807118] nix-grpc-daemon[485]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4736worker1 # [ 103.827839] nix-grpc-daemon[485]: ts=2026-09-25T15:01:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537904737worker1 # [ 103.831890] nix-grpc-daemon[485]: ts=2026-09-25T15:01:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 paths=1 nar_bytes_out=2324738client # [ 103.871681] nix-daemon[1100]: reaped child process 1102, status = succeeded4739client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.08 seconds)4740client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4741client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.02 seconds)4742client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4743client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4744client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4745client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4746(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.17 seconds)4747subtest: an input only one worker has still reaches the builder4748client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4749client # this derivation will be built:4750client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4751client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4752client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.27 seconds)4753client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4754client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.11 seconds)4755worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4756worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.15 seconds)4757worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4758worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.02 seconds)4759client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24760client # [ 104.657864] nix-daemon[1100]: accepted connection from pid 1183, user root (trusted)4761client # this derivation will be built:4762client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4763worker2 # [ 104.787195] nix-grpc-daemon[786]: ts=2026-09-25T15:01:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:502624764client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4765worker1 # [ 104.818539] nix-grpc-daemon[485]: ts=2026-09-25T15:01:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904766worker1 # [ 104.956776] nix-daemon[855]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)4767lb # [ 104.950208] niks3-server[1035]: 2026/09/25 15:01:05 INFO Received push request method=POST path=/api/pushes4768worker1 # [ 104.976396] nix-daemon[855]: reaped child process 1137, status = succeeded4769worker1 # [ 104.979748] nix-grpc-daemon[905]: time=2026-09-25T15:01:05.511Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4770worker1 # [ 104.981440] nix-grpc-daemon[905]: time=2026-09-25T15:01:05.511Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4771lb # [ 104.968365] niks3-server[1035]: 2026/09/25 15:01:05 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4772worker1 # [ 104.998950] nix-grpc-daemon[905]: time=2026-09-25T15:01:05.530Z level=INFO msg="Uploading 1 narinfos"4773lb # [ 104.977078] niks3-server[1035]: 2026/09/25 15:01:05 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign4774lb # [ 104.982255] niks3-server[1035]: 2026/09/25 15:01:05 INFO Signed narinfos id=27 count=14775lb # [ 104.988860] niks3-server[1035]: 2026/09/25 15:01:05 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4776lb # [ 104.991382] niks3-server[1035]: 2026/09/25 15:01:05 INFO Received complete push request method=POST path=/api/pushes/27/complete4777worker1 # [ 105.019141] nix-grpc-daemon[905]: time=2026-09-25T15:01:05.551Z level=INFO msg="Upload complete. (196ms)"4778lb # [ 105.000623] niks3-server[1035]: 2026/09/25 15:01:05 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4779worker1 # [ 105.033375] nix-grpc-daemon[485]: ts=2026-09-25T15:01:05Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04780worker1 # [ 105.038719] nix-grpc-daemon[485]: ts=2026-09-25T15:01:05Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14781worker1 # [ 105.042921] nix-grpc-daemon[485]: ts=2026-09-25T15:01:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904782worker1 # [ 105.048782] nix-grpc-daemon[485]: ts=2026-09-25T15:01:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904783client # copying 2 paths to 'lb:50051'...4784client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4785worker1 # [ 105.059131] nix-grpc-daemon[485]: ts=2026-09-25T15:01:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537904786client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4787worker1 # [ 105.067317] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4788worker1 # [ 105.182277] nix-daemon[855]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4789lb # [ 105.173778] niks3-server[1035]: 2026/09/25 15:01:05 INFO Received push request method=POST path=/api/pushes4790worker1 # [ 105.201056] nix-daemon[855]: reaped child process 1146, status = succeeded4791worker1 # [ 105.204257] nix-grpc-daemon[905]: time=2026-09-25T15:01:05.734Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4792worker1 # [ 105.205780] nix-grpc-daemon[905]: time=2026-09-25T15:01:05.734Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4793worker1 # [ 105.207600] nix-grpc-daemon[905]: time=2026-09-25T15:01:05.734Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4794lb # [ 105.199659] niks3-server[1035]: 2026/09/25 15:01:05 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4795lb # [ 105.205936] niks3-server[1035]: 2026/09/25 15:01:05 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4796worker1 # [ 105.237765] nix-grpc-daemon[905]: time=2026-09-25T15:01:05.769Z level=INFO msg="Uploading 2 narinfos"4797lb # [ 105.215701] niks3-server[1035]: 2026/09/25 15:01:05 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign4798lb # [ 105.221721] niks3-server[1035]: 2026/09/25 15:01:05 INFO Signed narinfos id=28 count=24799lb # [ 105.223469] niks3-server[1035]: 2026/09/25 15:01:05 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4800lb # [ 105.235104] niks3-server[1035]: 2026/09/25 15:01:05 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4801lb # [ 105.244468] niks3-server[1035]: 2026/09/25 15:01:05 INFO Received complete push request method=POST path=/api/pushes/28/complete4802lb # [ 105.246742] niks3-server[1035]: 2026/09/25 15:01:05 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4803worker1 # [ 105.277042] nix-grpc-daemon[905]: time=2026-09-25T15:01:05.809Z level=INFO msg="Upload complete. (199ms)"4804lb # [ 105.253138] niks3-server[1035]: 2026/09/25 15:01:05 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4805worker1 # [ 105.283821] nix-grpc-daemon[485]: ts=2026-09-25T15:01:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 paths=2 nar_bytes_in=9844806worker1 # [ 105.294900] nix-grpc-daemon[485]: ts=2026-09-25T15:01:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904807worker1 # [ 105.300241] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4808worker1 # [ 105.304634] nix-daemon[855]: reaped child process 1141, status = succeeded4809client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4810client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4811worker1 # [ 105.318376] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4812client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4813worker1 # [ 106.368386] nix-daemon[855]: reaped child process 1150, status = succeeded4814worker1 # [ 106.470041] nix-daemon[855]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)4815worker1 # [ 106.487799] nix-daemon[855]: reaped child process 1169, status = succeeded4816lb # [ 106.469811] niks3-server[1035]: 2026/09/25 15:01:06 INFO Received push request method=POST path=/api/pushes4817worker1 # [ 106.500370] nix-grpc-daemon[905]: time=2026-09-25T15:01:07.030Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4818lb # [ 106.487671] niks3-server[1035]: 2026/09/25 15:01:07 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4819worker1 # [ 106.518377] nix-grpc-daemon[905]: time=2026-09-25T15:01:07.050Z level=INFO msg="Uploading 1 narinfos"4820lb # [ 106.496300] niks3-server[1035]: 2026/09/25 15:01:07 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign4821lb # [ 106.501779] niks3-server[1035]: 2026/09/25 15:01:07 INFO Signed narinfos id=29 count=14822lb # [ 106.508313] niks3-server[1035]: 2026/09/25 15:01:07 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4823worker1 # [ 106.540053] nix-grpc-daemon[905]: time=2026-09-25T15:01:07.072Z level=INFO msg="Upload complete. (213ms)"4824lb # [ 106.516680] niks3-server[1035]: 2026/09/25 15:01:07 INFO Received complete push request method=POST path=/api/pushes/29/complete4825lb # [ 106.519139] niks3-server[1035]: 2026/09/25 15:01:07 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4826worker1 # [ 106.547493] nix-grpc-daemon[485]: ts=2026-09-25T15:01:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=14827worker1 # [ 106.558860] nix-daemon[855]: reaped child process 1148, status = succeeded4828client # copying 1 paths...4829worker2 # [ 106.535503] nix-grpc-daemon[786]: ts=2026-09-25T15:01:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:502624830worker2 # [ 106.543332] nix-grpc-daemon[786]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4831client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4832worker2 # [ 106.568440] nix-grpc-daemon[786]: ts=2026-09-25T15:01:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:502624833worker2 # [ 106.572390] nix-grpc-daemon[786]: ts=2026-09-25T15:01:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_out=1204834client # [ 106.639825] nix-daemon[1100]: reaped child process 1186, status = succeeded4835client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.13 seconds)4836client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24837client # [ 106.776818] nix-daemon[1100]: accepted connection from pid 1205, user root (trusted)4838client # this derivation will be built:4839client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4840worker2 # [ 106.904236] nix-grpc-daemon[786]: ts=2026-09-25T15:01:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:502624841client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4842worker1 # [ 106.934901] nix-grpc-daemon[485]: ts=2026-09-25T15:01:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904843worker1 # [ 106.940495] nix-grpc-daemon[905]: time=2026-09-25T15:01:07.472Z level=INFO msg="All 1 paths already cached"4844worker1 # [ 106.951183] nix-grpc-daemon[485]: ts=2026-09-25T15:01:07Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04845worker2 # [ 106.925756] nix-grpc-daemon[786]: ts=2026-09-25T15:01:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624846worker1 # [ 106.954660] nix-grpc-daemon[485]: ts=2026-09-25T15:01:07Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14847worker2 # [ 106.934227] nix-grpc-daemon[786]: ts=2026-09-25T15:01:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624848client # copying 1 paths to 'lb:50051'...4849client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4850worker2 # [ 106.944837] nix-grpc-daemon[786]: ts=2026-09-25T15:01:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502624851worker2 # [ 106.953314] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4852worker2 # [ 106.967190] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)4853worker2 # [ 107.026288] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 127 ms (attempt 2/5)4854worker2 # [ 107.154387] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 3/5)4855worker2 # [ 107.266371] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 4/5)4856worker2 # [ 107.298423] nix-grpc-daemon[786]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4857worker2 # [ 107.302777] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4858worker2 # [ 107.307074] nix-grpc-daemon[786]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4859worker2 # [ 107.440088] nix-daemon[845]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)4860lb # [ 107.460729] niks3-server[1035]: 2026/09/25 15:01:07 INFO Received push request method=POST path=/api/pushes4861worker2 # [ 107.460080] nix-daemon[845]: reaped child process 1207, status = succeeded4862worker2 # [ 107.463174] nix-grpc-daemon[857]: time=2026-09-25T15:01:08.005Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4863worker2 # [ 107.464878] nix-grpc-daemon[857]: time=2026-09-25T15:01:08.005Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4864worker2 # [ 107.476725] nix-grpc-daemon[857]: time=2026-09-25T15:01:08.019Z level=INFO msg="Uploading 1 narinfos"4865lb # [ 107.481243] niks3-server[1035]: 2026/09/25 15:01:08 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign4866lb # [ 107.483641] niks3-server[1035]: 2026/09/25 15:01:08 INFO Signed narinfos id=30 count=14867lb # [ 107.491909] niks3-server[1035]: 2026/09/25 15:01:08 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4868lb # [ 107.501238] niks3-server[1035]: 2026/09/25 15:01:08 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4869worker2 # [ 107.502874] nix-grpc-daemon[857]: time=2026-09-25T15:01:08.045Z level=INFO msg="Upload complete. (188ms)"4870lb # [ 107.506838] niks3-server[1035]: 2026/09/25 15:01:08 INFO Received complete push request method=POST path=/api/pushes/30/complete4871lb # [ 107.511486] niks3-server[1035]: 2026/09/25 15:01:08 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4872worker2 # [ 107.511961] nix-grpc-daemon[786]: ts=2026-09-25T15:01:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_in=5684873worker2 # [ 107.515321] nix-grpc-daemon[786]: ts=2026-09-25T15:01:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502624874worker2 # [ 107.524682] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4875worker2 # [ 107.528448] nix-daemon[845]: reaped child process 1197, status = succeeded4876worker2 # [ 107.540318] nix-grpc-daemon[786]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4877client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4878worker2 # [ 107.550571] nix-daemon[845]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4879client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4880client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4881worker2 # [ 108.187021] nix-daemon[845]: reaped child process 1212, status = succeeded4882lb # [ 108.265948] envoy[897]: conn peer=[2001:db8:1::1]:60246 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=184883lb # [ 108.266161] envoy[897]: conn peer=[2001:db8:1::1]:60234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3397 tx=1924 ms=19534884lb # [ 108.266226] envoy[897]: conn peer=[2001:db8:1::1]:60254 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3871 tx=2138 ms=11464885lb # [ 108.266286] envoy[897]: conn peer=[2001:db8:1::1]:55458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=154886lb # [ 108.266338] envoy[897]: conn peer=[2001:db8:1::1]:55470 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4335 tx=2372 ms=21844887lb # [ 108.266405] envoy[897]: conn peer=[2001:db8:1::1]:55474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=154888lb # [ 108.266467] envoy[897]: conn peer=[2001:db8:1::1]:55486 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3862 tx=2065 ms=17964889lb # [ 108.266527] envoy[897]: conn peer=[2001:db8:1::1]:55498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=164890worker2 # [ 108.293522] nix-daemon[845]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)4891lb # [ 108.312797] niks3-server[1035]: 2026/09/25 15:01:08 INFO Received push request method=POST path=/api/pushes4892worker2 # [ 108.311975] nix-daemon[845]: reaped child process 1231, status = succeeded4893worker2 # [ 108.315423] nix-grpc-daemon[857]: time=2026-09-25T15:01:08.856Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4894lb # [ 108.332690] niks3-server[1035]: 2026/09/25 15:01:08 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign4895worker2 # [ 108.333134] nix-grpc-daemon[857]: time=2026-09-25T15:01:08.875Z level=INFO msg="Uploading 1 narinfos"4896lb # [ 108.339592] niks3-server[1035]: 2026/09/25 15:01:08 INFO Signed narinfos id=31 count=14897lb # [ 108.344362] niks3-server[1035]: 2026/09/25 15:01:08 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4898lb # [ 108.352135] niks3-server[1035]: 2026/09/25 15:01:08 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4899lb # [ 108.354600] niks3-server[1035]: 2026/09/25 15:01:08 INFO Received complete push request method=POST path=/api/pushes/31/complete4900worker2 # [ 108.356003] nix-grpc-daemon[857]: time=2026-09-25T15:01:08.898Z level=INFO msg="Upload complete. (195ms)"4901lb # [ 108.361576] niks3-server[1035]: 2026/09/25 15:01:08 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4902client # copying 1 paths...4903worker2 # [ 108.364254] nix-grpc-daemon[786]: ts=2026-09-25T15:01:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=14904client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4905worker2 # [ 108.373991] nix-grpc-daemon[786]: ts=2026-09-25T15:01:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:502624906worker2 # [ 108.377080] nix-daemon[845]: reaped child process 1209, status = succeeded4907worker2 # [ 108.383157] nix-grpc-daemon[786]: ts=2026-09-25T15:01:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:502624908worker2 # [ 108.387613] nix-grpc-daemon[786]: ts=2026-09-25T15:01:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_out=1204909client # [ 108.453924] nix-daemon[1100]: reaped child process 1208, status = succeeded4910client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 1.81 seconds)4911client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24912client # [ 108.587276] nix-daemon[1100]: accepted connection from pid 1227, user root (trusted)4913client # this derivation will be built:4914client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4915worker1 # [ 108.750377] nix-grpc-daemon[485]: ts=2026-09-25T15:01:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:537904916client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4917worker2 # [ 108.726196] nix-grpc-daemon[786]: ts=2026-09-25T15:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502624918worker2 # [ 108.732774] nix-grpc-daemon[857]: time=2026-09-25T15:01:09.274Z level=INFO msg="All 1 paths already cached"4919worker1 # [ 108.768189] nix-grpc-daemon[485]: ts=2026-09-25T15:01:09Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04920worker1 # [ 108.773380] nix-grpc-daemon[485]: ts=2026-09-25T15:01:09Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14921worker1 # [ 108.778229] nix-grpc-daemon[485]: ts=2026-09-25T15:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904922worker1 # [ 108.782392] nix-grpc-daemon[485]: ts=2026-09-25T15:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537904923worker1 # [ 108.788617] nix-grpc-daemon[905]: time=2026-09-25T15:01:09.320Z level=INFO msg="All 1 paths already cached"4924client # copying 1 paths to 'lb:50051'...4925client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4926worker1 # [ 108.795176] nix-grpc-daemon[485]: ts=2026-09-25T15:01:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537904927worker1 # [ 108.804301] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4928worker1 # [ 108.927890] nix-daemon[855]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)4929worker1 # [ 108.947405] nix-daemon[855]: reaped child process 1180, status = succeeded4930lb # [ 108.928362] niks3-server[1035]: 2026/09/25 15:01:09 INFO Received push request method=POST path=/api/pushes4931worker1 # [ 108.957743] nix-grpc-daemon[905]: time=2026-09-25T15:01:09.489Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4932worker1 # [ 108.959305] nix-grpc-daemon[905]: time=2026-09-25T15:01:09.489Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4933lb # [ 108.949470] niks3-server[1035]: 2026/09/25 15:01:09 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign4934worker1 # [ 108.977036] nix-grpc-daemon[905]: time=2026-09-25T15:01:09.508Z level=INFO msg="Uploading 1 narinfos"4935lb # [ 108.956080] niks3-server[1035]: 2026/09/25 15:01:09 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4936lb # [ 108.959026] niks3-server[1035]: 2026/09/25 15:01:09 INFO Signed narinfos id=32 count=14937lb # [ 108.963936] niks3-server[1035]: 2026/09/25 15:01:09 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4938worker1 # [ 108.995512] nix-grpc-daemon[905]: time=2026-09-25T15:01:09.527Z level=INFO msg="Upload complete. (180ms)"4939lb # [ 108.972508] niks3-server[1035]: 2026/09/25 15:01:09 INFO Received complete push request method=POST path=/api/pushes/32/complete4940lb # [ 108.974937] niks3-server[1035]: 2026/09/25 15:01:09 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4941worker1 # [ 109.004304] nix-grpc-daemon[485]: ts=2026-09-25T15:01:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=0 paths=1 nar_bytes_in=5684942worker1 # [ 109.012210] nix-grpc-daemon[485]: ts=2026-09-25T15:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537904943worker1 # [ 109.017613] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4944worker1 # [ 109.022108] nix-daemon[855]: reaped child process 1175, status = succeeded4945client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4946worker1 # [ 109.031376] nix-daemon[855]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4947client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4948client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4949worker1 # [ 110.470013] nix-daemon[855]: reaped child process 1184, status = succeeded4950worker1 # [ 110.578401] nix-daemon[855]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)4951worker1 # [ 110.595264] nix-daemon[855]: reaped child process 1203, status = succeeded4952lb # [ 110.577812] niks3-server[1035]: 2026/09/25 15:01:11 INFO Received push request method=POST path=/api/pushes4953worker1 # [ 110.607475] nix-grpc-daemon[905]: time=2026-09-25T15:01:11.138Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4954lb # [ 110.593419] niks3-server[1035]: 2026/09/25 15:01:11 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign4955worker1 # [ 110.622089] nix-grpc-daemon[905]: time=2026-09-25T15:01:11.153Z level=INFO msg="Uploading 1 narinfos"4956lb # [ 110.600200] niks3-server[1035]: 2026/09/25 15:01:11 INFO Signed narinfos id=33 count=14957lb # [ 110.605488] niks3-server[1035]: 2026/09/25 15:01:11 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4958lb # [ 110.615924] niks3-server[1035]: 2026/09/25 15:01:11 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4959worker1 # [ 110.644406] nix-grpc-daemon[905]: time=2026-09-25T15:01:11.176Z level=INFO msg="Upload complete. (203ms)"4960lb # [ 110.621231] niks3-server[1035]: 2026/09/25 15:01:11 INFO Received complete push request method=POST path=/api/pushes/33/complete4961lb # [ 110.625706] niks3-server[1035]: 2026/09/25 15:01:11 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4962worker1 # [ 110.652570] nix-grpc-daemon[485]: ts=2026-09-25T15:01:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53790 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=14963client # copying 1 paths...4964worker1 # [ 110.663897] nix-grpc-daemon[485]: ts=2026-09-25T15:01:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537904965worker1 # [ 110.667610] nix-daemon[855]: reaped child process 1182, status = succeeded4966client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4967worker2 # [ 110.656156] nix-grpc-daemon[786]: ts=2026-09-25T15:01:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:502624968worker2 # [ 110.663349] nix-grpc-daemon[786]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4969worker2 # [ 110.672817] nix-grpc-daemon[786]: ts=2026-09-25T15:01:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50262 duration_s=0 paths=1 nar_bytes_out=1204970client # [ 110.740924] nix-daemon[1100]: reaped child process 1230, status = succeeded4971client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.29 seconds)4972(finished: subtest: an input only one worker has still reaches the builder, in 6.78 seconds)4973subtest: upload whose reference the worker lacks is completed from the cache4974client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4975client # this derivation will be built:4976client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4977client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4978client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.29 seconds)4979client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4980client # this derivation will be built:4981client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4982client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4983client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.27 seconds)4984client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4985client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds)4986worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4987worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.15 seconds)4988worker2 # [ 111.593988] systemd[1]: Stopping Nix worker-protocol over gRPC...4989worker2 # [ 111.969039] nix-grpc-daemon[786]: ts=2026-09-25T15:01:12Z level=info event=signal_exit4990worker2 # [ 113.950268] nix-grpc-daemon[786]: ts=2026-09-25T15:01:14Z level=info event=scheduler_restarting4991worker1 # [ 113.986598] nix-grpc-daemon[485]: ts=2026-09-25T15:01:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:601284992worker1 # [ 113.991300] nix-grpc-daemon[485]: ts=2026-09-25T15:01:14Z level=info event=worker_gone addr=192.168.1.4:500514993worker2 # [ 113.973345] nix-grpc-daemon[786]: ts=2026-09-25T15:01:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4994worker2 # [ 113.987129] nix-daemon[845]: reaped child process 847, status = succeeded4995worker2 # [ 115.665782] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4996worker2 # [ 115.668321] systemd[1]: Stopped Nix worker-protocol over gRPC.4997worker2 # [ 115.672662] systemd[1]: nix-grpc-daemon.service: Consumed 2.891s CPU time over 1min 21.885s wall clock time, 65.2M memory peak, 191.5K incoming IP traffic, 286.2K outgoing IP traffic.4998lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux4999lb: must succeed: curl -sf localhost:9901/clusters5000lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5001lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.06 seconds)5002client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25003worker1 # [ 115.944553] nix-grpc-daemon[485]: ts=2026-09-25T15:01:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537905004client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5005worker1 # [ 115.948962] nix-grpc-daemon[485]: ts=2026-09-25T15:01:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537905006client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.20 seconds)5007worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5008worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.02 seconds)5009worker2 # [ 115.993816] systemd[1]: Starting scheduler-token.service...5010worker2 # [ 116.047655] systemd[1]: scheduler-token.service: Deactivated successfully.5011worker2 # [ 116.052399] systemd[1]: Finished scheduler-token.service.5012worker2 # [ 116.053591] systemd[1]: scheduler-token.service: Consumed 32ms CPU time over 56ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5013worker2 # [ 116.059611] systemd[1]: Starting Nix worker-protocol over gRPC...5014worker2 # [ 116.172509] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5015worker2 # [ 116.174958] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5016worker2 # [ 116.179164] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15017worker2 # [ 116.184064] nix-grpc-daemon[1256]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5018worker2 # [ 116.185663] nix-grpc-daemon[1256]: E0000 00:00:1790348476.726265 1256 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5019worker2 # [ 116.190640] nix-grpc-daemon[1256]: E0000 00:00:1790348476.732898 1256 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5020worker2 # [ 116.235633] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:16Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5021worker2 # [ 116.238145] nix-grpc-daemon[1256]: E0000 00:00:1790348476.742291 1296 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5022worker2 # [ 116.243662] nix-grpc-daemon[1256]: E0000 00:00:1790348476.760099 1296 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5023worker2 # [ 116.248165] nix-grpc-daemon[1256]: E0000 00:00:1790348476.760260 1296 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5024worker1 # [ 116.277398] nix-grpc-daemon[485]: ts=2026-09-25T15:01:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:601285025worker2 # [ 116.251465] nix-grpc-daemon[1256]: E0000 00:00:1790348476.777125 1296 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5026worker2 # [ 116.254233] systemd[1]: Started Nix worker-protocol over gRPC.5027worker2 # [ 116.255095] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:16Z level=info event=scheduler_connected addr=lb:500515028worker1 # [ 116.282785] nix-grpc-daemon[485]: ts=2026-09-25T15:01:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05029worker1 # [ 116.321344] systemd[1]: Stopping Nix worker-protocol over gRPC...5030worker1 # [ 116.851441] nix-grpc-daemon[485]: ts=2026-09-25T15:01:17Z level=info event=signal_exit5031lb # [ 117.520938] niks3-server[1035]: 2026/09/25 15:01:18 INFO lead: released remote=192.168.1.3:453625032lb # [ 118.267003] envoy[897]: conn peer=[2001:db8:1::1]:55508 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3723 tx=2105 ms=14935033lb # [ 118.269914] envoy[897]: conn peer=[2001:db8:1::1]:55520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=145034lb # [ 118.272420] envoy[897]: conn peer=[2001:db8:1::1]:55536 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=329 ms=295035lb # [ 118.274863] envoy[897]: conn peer=[2001:db8:1::1]:55522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3724 tx=2103 ms=19615036lb # [ 118.277493] envoy[897]: conn peer=[2001:db8:1::4]:32886 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=34292 tx=2498 ms=799685037lb # [ 118.280134] envoy[897]: conn peer=[2001:db8:1::1]:58536 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=155038worker1 # [ 118.543760] nix-grpc-daemon[485]: ts=2026-09-25T15:01:19Z level=info event=scheduler_restarting5039worker2 # [ 118.518195] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:19Z level=info event=scheduler_restarting addr=lb:500515040lb # [ 118.519931] envoy[897]: [2026-09-25 15:01:19.042][897][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5041worker1 # [ 118.548062] nix-grpc-daemon[485]: ts=2026-09-25T15:01:19Z level=info event=scheduler_restarting addr=lb:500515042worker1 # [ 118.554551] nix-grpc-daemon[485]: ts=2026-09-25T15:01:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:601285043worker1 # [ 118.557927] nix-grpc-daemon[485]: ts=2026-09-25T15:01:19Z level=info event=worker_gone addr=192.168.1.4:500515044worker1 # [ 118.560436] nix-grpc-daemon[485]: ts=2026-09-25T15:01:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:601285045worker1 # [ 118.562727] nix-grpc-daemon[485]: ts=2026-09-25T15:01:19Z level=info event=worker_gone addr=192.168.1.3:500515046worker1 # [ 118.582659] nix-daemon[855]: reaped child process 857, status = succeeded5047worker2 # [ 118.718652] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:19Z level=info event=scheduler_connected addr=lb:500515048worker2 # [ 118.721997] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5049worker1 # [ 118.763872] nix-grpc-daemon[485]: ts=2026-09-25T15:01:19Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5050worker2 # [ 118.922797] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:19Z level=info event=scheduler_connected addr=lb:500515051worker2 # [ 118.926027] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5052worker2 # [ 119.126817] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:19Z level=info event=scheduler_connected addr=lb:500515053worker2 # [ 119.129993] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5054worker2 # [ 119.331241] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:19Z level=info event=scheduler_connected addr=lb:500515055worker2 # [ 119.334758] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5056worker2 # [ 119.536723] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_connected addr=lb:500515057worker2 # [ 119.539012] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5058worker1 # [ 119.662345] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5059worker1 # [ 119.666844] systemd[1]: Stopped Nix worker-protocol over gRPC.5060worker1 # [ 119.670195] systemd[1]: nix-grpc-daemon.service: Consumed 2.408s CPU time over 1min 49.147s wall clock time, 66.2M memory peak, 425K incoming IP traffic, 262.3K outgoing IP traffic.5061lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5062lb: must succeed: curl -sf localhost:9901/clusters5063lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5064lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.04 seconds)5065client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25066worker2 # [ 119.737379] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_connected addr=lb:500515067worker2 # [ 119.740415] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5068worker2 # [ 119.864606] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389445069worker2 # [ 119.873326] nix-daemon[845]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5070worker2 # [ 119.886737] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)5071worker2 # [ 119.892086] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 101 ms (attempt 2/5)5072worker2 # [ 119.941130] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_connected addr=lb:500515073worker2 # [ 119.943951] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5074worker2 # [ 119.994145] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 369 ms (attempt 3/5)5075worker2 # [ 120.144793] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_connected addr=lb:500515076worker2 # [ 120.147618] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5077worker2 # [ 120.348557] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_connected addr=lb:500515078worker2 # [ 120.351288] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5079worker2 # [ 120.364394] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 259 ms (attempt 4/5)5080worker2 # [ 120.553013] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_connected addr=lb:500515081worker2 # [ 120.555269] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5082worker2 # [ 120.629022] nix-grpc-daemon[1256]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5083worker2 # [ 120.632875] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5084client # copying 2 paths...5085client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5086worker2 # [ 120.646971] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:389445087client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5088worker2 # [ 120.656135] nix-daemon[845]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5089worker2 # [ 120.673338] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=niks3_push_started pid=13165090worker2 # [ 120.756624] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_connected addr=lb:500515091worker2 # [ 120.780192] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5092worker2 # [ 120.808735] nix-grpc-daemon[1316]: time=2026-09-25T15:01:21.349Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5093worker2 # [ 120.917786] nix-daemon[845]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5094lb # [ 120.940785] niks3-server[1035]: 2026/09/25 15:01:21 INFO Received push request method=POST path=/api/pushes5095worker2 # [ 120.939535] nix-daemon[845]: reaped child process 1332, status = succeeded5096worker2 # [ 120.944110] nix-grpc-daemon[1316]: time=2026-09-25T15:01:21.485Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5097worker2 # [ 120.945685] nix-grpc-daemon[1316]: time=2026-09-25T15:01:21.485Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5098worker2 # [ 120.947510] nix-grpc-daemon[1316]: time=2026-09-25T15:01:21.485Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5099worker2 # [ 120.960824] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_connected addr=lb:500515100worker2 # [ 120.963001] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5101lb # [ 120.985423] niks3-server[1035]: 2026/09/25 15:01:21 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5102worker2 # [ 120.987639] nix-grpc-daemon[1316]: time=2026-09-25T15:01:21.528Z level=INFO msg="Uploading 2 narinfos"5103lb # [ 120.993389] niks3-server[1035]: 2026/09/25 15:01:21 INFO Signed narinfos id=34 count=25104lb # [ 121.000047] niks3-server[1035]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5105lb # [ 121.009771] niks3-server[1035]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5106lb # [ 121.017950] niks3-server[1035]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5107lb # [ 121.031880] niks3-server[1035]: 2026/09/25 15:01:21 INFO Received complete push request method=POST path=/api/pushes/34/complete5108lb # [ 121.041358] niks3-server[1035]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5109lb # [ 121.050114] niks3-server[1035]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5110worker2 # [ 121.049535] nix-grpc-daemon[1316]: time=2026-09-25T15:01:21.592Z level=INFO msg="Upload complete. (241ms)"5111lb # [ 121.055603] niks3-server[1035]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5112worker2 # [ 121.060256] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38944 duration_s=0 paths=2 nar_bytes_in=3125113worker2 # [ 121.069088] nix-daemon[845]: reaped child process 1314, status = succeeded5114client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.38 seconds)5115worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5116worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.02 seconds)5117worker1 # [ 121.168836] systemd[1]: Starting Nix worker-protocol over gRPC...5118worker2 # [ 121.163968] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_connected addr=lb:500515119worker2 # [ 121.166856] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5120lb # [ 121.203096] niks3-server[1035]: 2026/09/25 15:01:21 INFO lead: acquired remote=192.168.1.4:368345121worker2 # [ 121.201825] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_take_over5122worker1 # [ 121.285035] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5123worker1 # [ 121.287302] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5124worker1 # [ 121.291200] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15125worker1 # [ 121.296072] nix-grpc-daemon[1226]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5126worker1 # [ 121.297578] nix-grpc-daemon[1226]: E0000 00:00:1790348481.827776 1226 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5127worker1 # [ 121.302331] nix-grpc-daemon[1226]: E0000 00:00:1790348481.834198 1226 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5128lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5129lb: must succeed: curl -sf localhost:9901/clusters5130worker1 # [ 121.347293] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:21Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5131worker1 # [ 121.349773] nix-grpc-daemon[1226]: E0000 00:00:1790348481.843832 1267 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5132worker1 # [ 121.352855] nix-grpc-daemon[1226]: E0000 00:00:1790348481.861525 1267 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5133worker1 # [ 121.358791] nix-grpc-daemon[1226]: E0000 00:00:1790348481.861688 1267 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5134worker1 # [ 121.363960] nix-grpc-daemon[1226]: E0000 00:00:1790348481.878349 1267 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5135worker1 # [ 121.366902] systemd[1]: Started Nix worker-protocol over gRPC.5136worker1 # [ 121.367891] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:21Z level=info event=scheduler_connected addr=lb:500515137worker1 # [ 121.369325] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5138lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5139worker2 # [ 121.368039] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_connected addr=lb:500515140worker2 # [ 121.371183] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5141worker1 # [ 121.567635] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:22Z level=info event=scheduler_connected addr=lb:500515142worker2 # [ 121.551092] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:480225143worker2 # [ 121.555126] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05144worker2 # [ 121.572053] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:22Z level=info event=scheduler_connected addr=lb:500515145worker2 # [ 121.580120] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:22Z level=info event=oidc_jwks provider=mock keys=15146worker2 # [ 121.582949] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:480225147worker2 # [ 121.586027] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05148lb: must succeed: curl -sf localhost:9901/clusters5149lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5150lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.09 seconds)5151(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.66 seconds)5152subtest: a niks3 restart does not cost the leader its role5153lb: must succeed: curl -sf localhost:9901/clusters5154lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5155lb: must succeed: curl -sf localhost:9901/clusters5156lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5157lb # [ 122.503111] systemd[1]: Stopping niks3 server...5158worker2 # [ 122.499157] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:23Z level=info event=scheduler_lock_lost status=2005159worker1 # [ 122.526813] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:23Z level=info event=scheduler_lock_lost status=2005160lb # [ 122.503798] niks3-server[1035]: 2026/09/25 15:01:23 INFO Shutdown signal received, draining in-flight requests timeout=10s5161lb # [ 122.503902] niks3-server[1035]: 2026/09/25 15:01:23 INFO lead: released remote=192.168.1.4:368345162lb # [ 122.528484] systemd[1]: niks3.service: Deactivated successfully.5163lb # [ 122.530939] systemd[1]: Stopped niks3 server.5164lb # [ 122.531887] systemd[1]: niks3.service: Consumed 424ms CPU time over 1min 31.280s wall clock time, 28M memory peak, 93.7K incoming IP traffic, 135.5K outgoing IP traffic.5165lb # [ 122.541482] systemd[1]: Starting niks3 server...5166lb # [ 122.599124] niks3-server[1213]: 2026/09/25 15:01:23 goose: no migrations to run. current version: 202609231200005167lb # [ 122.605373] niks3-server[1213]: 2026/09/25 15:01:23 OK 1_commit_pending_closure.sql (4.35ms)5168lb # [ 122.608555] niks3-server[1213]: 2026/09/25 15:01:23 OK 2_object_stats_trigger.sql (1.81ms)5169lb # [ 122.611570] niks3-server[1213]: 2026/09/25 15:01:23 OK 3_commit_push.sql (2.68ms)5170lb # [ 122.613907] niks3-server[1213]: 2026/09/25 15:01:23 goose: up to current file version: 35171lb # [ 122.615054] niks3-server[1213]: 2026/09/25 15:01:23 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5172lb # [ 122.625269] niks3-server[1213]: 2026/09/25 15:01:23 INFO Read proxy enabled — serving cache objects from S35173lb # [ 122.626649] niks3-server[1213]: 2026/09/25 15:01:23 INFO Using socket-activated listener address=0.0.0.0:57515174lb # [ 122.627966] niks3-server[1213]: 2026/09/25 15:01:23 INFO systemd watchdog enabled interval=15s5175lb # [ 122.633863] niks3-server[1213]: 2026/09/25 15:01:23 INFO Starting HTTP server address=0.0.0.0:57515176lb # [ 122.635325] systemd[1]: Started niks3 server.5177lb: waiting for unit niks3.service5178lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5179lb: must succeed: curl -sf localhost:9901/clusters5180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5181worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5182worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5183lb: must succeed: curl -sf localhost:9901/clusters5184lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5185lb: must succeed: curl -sf localhost:9901/clusters5186lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5187worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5188worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5189client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25190worker1 # [ 123.014740] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:337285191worker1 # [ 123.024375] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5192client # don't know how to build these paths:5193client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5194worker2 # [ 123.011660] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389445195worker2 # [ 123.031448] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:23Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05196worker1 # [ 123.061225] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285197worker2 # [ 123.035287] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:23Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=15198worker1 # [ 123.070173] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5)5199worker1 # [ 123.154180] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 2/5)5200worker1 # [ 123.171175] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 3/5)5201worker1 # [ 123.198941] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 570 ms (attempt 4/5)5202lb # [ 123.506515] niks3-server[1213]: 2026/09/25 15:01:24 INFO lead: acquired remote=192.168.1.4:366805203worker1 # [ 123.771019] nix-grpc-daemon[1226]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5204worker1 # [ 123.774763] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5205worker1 # [ 123.785850] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337285206client # copying 1 paths to 'lb:50051'...5207client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5208worker1 # [ 123.794570] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:337285209worker1 # [ 123.803103] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5210worker1 # [ 123.818648] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:24Z level=info event=niks3_push_started pid=12885211worker1 # [ 123.944873] nix-grpc-daemon[1288]: time=2026-09-25T15:01:24.476Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5212worker1 # [ 124.056483] nix-daemon[855]: accepted connection from pid 1300, user nix-grpc-daemon (trusted)5213worker1 # [ 124.076360] nix-daemon[855]: reaped child process 1303, status = succeeded5214lb # [ 124.053978] niks3-server[1213]: 2026/09/25 15:01:24 INFO Received push request method=POST path=/api/pushes5215worker1 # [ 124.087519] nix-grpc-daemon[1288]: time=2026-09-25T15:01:24.618Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5216worker1 # [ 124.091406] nix-grpc-daemon[1288]: time=2026-09-25T15:01:24.618Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5217lb # [ 124.090544] niks3-server[1213]: 2026/09/25 15:01:24 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5218lb # [ 124.114394] niks3-server[1213]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5219lb # [ 124.124125] niks3-server[1213]: 2026/09/25 15:01:24 INFO Signed narinfos id=35 count=15220worker1 # [ 124.151396] nix-grpc-daemon[1288]: time=2026-09-25T15:01:24.682Z level=INFO msg="Uploading 1 narinfos"5221lb # [ 124.126503] niks3-server[1213]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5222lb # [ 124.140116] niks3-server[1213]: 2026/09/25 15:01:24 INFO Received complete push request method=POST path=/api/pushes/35/complete5223worker1 # [ 124.172108] nix-grpc-daemon[1288]: time=2026-09-25T15:01:24.704Z level=INFO msg="Upload complete. (223ms)"5224lb # [ 124.152842] niks3-server[1213]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5225worker1 # [ 124.180441] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=0 paths=1 nar_bytes_in=5445226worker1 # [ 124.183753] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285227worker1 # [ 124.194616] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5228worker1 # [ 124.198743] nix-daemon[855]: reaped child process 1286, status = succeeded5229client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5230client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5231worker1 # [ 124.208352] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5232client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5233client # farm-a> LOG-farm-a-after-niks3-restart5234worker1 # [ 125.034792] nix-daemon[855]: reaped child process 1307, status = succeeded5235worker1 # [ 125.150594] nix-daemon[855]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5236worker1 # [ 125.166575] nix-daemon[855]: reaped child process 1326, status = succeeded5237lb # [ 125.149169] niks3-server[1213]: 2026/09/25 15:01:25 INFO Received push request method=POST path=/api/pushes5238worker1 # [ 125.180385] nix-grpc-daemon[1288]: time=2026-09-25T15:01:25.711Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5239worker1 # [ 125.181959] nix-grpc-daemon[1288]: time=2026-09-25T15:01:25.711Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5240lb # [ 125.178703] niks3-server[1213]: 2026/09/25 15:01:25 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5241worker1 # [ 125.208864] nix-grpc-daemon[1288]: time=2026-09-25T15:01:25.740Z level=INFO msg="Uploading 1 narinfos"5242lb # [ 125.188842] niks3-server[1213]: 2026/09/25 15:01:25 INFO Signed narinfos id=36 count=15243lb # [ 125.202670] niks3-server[1213]: 2026/09/25 15:01:25 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5244lb # [ 125.208433] niks3-server[1213]: 2026/09/25 15:01:25 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5245lb # [ 125.219575] niks3-server[1213]: 2026/09/25 15:01:25 INFO Received complete push request method=POST path=/api/pushes/36/complete5246lb # [ 125.225068] niks3-server[1213]: 2026/09/25 15:01:25 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5247lb # [ 125.229192] niks3-server[1213]: 2026/09/25 15:01:25 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5248worker1 # [ 125.255603] nix-grpc-daemon[1288]: time=2026-09-25T15:01:25.787Z level=INFO msg="Upload complete. (237ms)"5249worker1 # [ 125.264614] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15250worker2 # [ 125.239975] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:25Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05251worker2 # [ 125.243971] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:25Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=15252worker1 # [ 125.274444] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285253worker1 # [ 125.278523] nix-daemon[855]: reaped child process 1305, status = succeeded5254worker1 # [ 125.283706] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337285255worker1 # [ 125.291606] nix-grpc-daemon[1288]: time=2026-09-25T15:01:25.823Z level=INFO msg="All 1 paths already cached"5256client # copying 1 paths to 'lb:50051'...5257client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5258worker1 # [ 125.297646] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:337285259worker1 # [ 125.305052] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5260worker1 # [ 125.420744] nix-daemon[855]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)5261worker1 # [ 125.436457] nix-daemon[855]: reaped child process 1333, status = succeeded5262lb # [ 125.418644] niks3-server[1213]: 2026/09/25 15:01:25 INFO Received push request method=POST path=/api/pushes5263worker1 # [ 125.449781] nix-grpc-daemon[1288]: time=2026-09-25T15:01:25.981Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5264worker1 # [ 125.451355] nix-grpc-daemon[1288]: time=2026-09-25T15:01:25.981Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5265lb # [ 125.438742] niks3-server[1213]: 2026/09/25 15:01:25 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5266worker1 # [ 125.467722] nix-grpc-daemon[1288]: time=2026-09-25T15:01:25.999Z level=INFO msg="Uploading 1 narinfos"5267lb # [ 125.446591] niks3-server[1213]: 2026/09/25 15:01:25 INFO Signed narinfos id=37 count=15268lb # [ 125.452101] niks3-server[1213]: 2026/09/25 15:01:25 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5269lb # [ 125.465147] niks3-server[1213]: 2026/09/25 15:01:25 INFO Received complete push request method=POST path=/api/pushes/37/complete5270lb # [ 125.469090] niks3-server[1213]: 2026/09/25 15:01:25 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5271worker1 # [ 125.497577] nix-grpc-daemon[1288]: time=2026-09-25T15:01:26.029Z level=INFO msg="Upload complete. (183ms)"5272lb # [ 125.475327] niks3-server[1213]: 2026/09/25 15:01:25 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5273worker1 # [ 125.504281] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=0 paths=1 nar_bytes_in=6565274worker1 # [ 125.507534] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285275worker1 # [ 125.518037] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5276worker1 # [ 125.522145] nix-daemon[855]: reaped child process 1328, status = succeeded5277client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5278client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5279worker1 # [ 125.532290] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5280client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5281client # farm-b> LOG-farm-b-after-niks3-restart5282worker1 # [ 126.390351] nix-daemon[855]: reaped child process 1337, status = succeeded5283worker1 # [ 126.446942] nix-daemon[855]: accepted connection from pid 1353, user nix-grpc-daemon (trusted)5284worker1 # [ 126.463217] nix-daemon[855]: reaped child process 1356, status = succeeded5285lb # [ 126.446091] niks3-server[1213]: 2026/09/25 15:01:26 INFO Received push request method=POST path=/api/pushes5286worker1 # [ 126.476151] nix-grpc-daemon[1288]: time=2026-09-25T15:01:27.007Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5287worker1 # [ 126.477685] nix-grpc-daemon[1288]: time=2026-09-25T15:01:27.007Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5288lb # [ 126.472932] niks3-server[1213]: 2026/09/25 15:01:26 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5289worker1 # [ 126.502363] nix-grpc-daemon[1288]: time=2026-09-25T15:01:27.034Z level=INFO msg="Uploading 1 narinfos"5290lb # [ 126.480205] niks3-server[1213]: 2026/09/25 15:01:26 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5291lb # [ 126.488269] niks3-server[1213]: 2026/09/25 15:01:27 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5292lb # [ 126.496539] niks3-server[1213]: 2026/09/25 15:01:27 INFO Signed narinfos id=38 count=15293lb # [ 126.498332] niks3-server[1213]: 2026/09/25 15:01:27 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5294worker1 # [ 126.526638] nix-grpc-daemon[1288]: time=2026-09-25T15:01:27.058Z level=INFO msg="Upload complete. (186ms)"5295lb # [ 126.503827] niks3-server[1213]: 2026/09/25 15:01:27 INFO Received complete push request method=POST path=/api/pushes/38/complete5296lb # [ 126.508839] niks3-server[1213]: 2026/09/25 15:01:27 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5297worker1 # [ 126.533879] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15298worker2 # [ 126.511232] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:27Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05299worker2 # [ 126.514779] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:27Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=15300worker1 # [ 126.545542] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285301worker1 # [ 126.549496] nix-daemon[855]: reaped child process 1335, status = succeeded5302worker1 # [ 126.555384] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337285303worker1 # [ 126.563274] nix-grpc-daemon[1288]: time=2026-09-25T15:01:27.095Z level=INFO msg="All 2 paths already cached"5304client # copying 1 paths to 'lb:50051'...5305client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5306worker1 # [ 126.569300] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:337285307worker1 # [ 126.576955] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5308worker1 # [ 126.690482] nix-daemon[855]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5309lb # [ 126.683741] niks3-server[1213]: 2026/09/25 15:01:27 INFO Received push request method=POST path=/api/pushes5310worker1 # [ 126.709155] nix-daemon[855]: reaped child process 1363, status = succeeded5311worker1 # [ 126.712688] nix-grpc-daemon[1288]: time=2026-09-25T15:01:27.244Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5312worker1 # [ 126.714175] nix-grpc-daemon[1288]: time=2026-09-25T15:01:27.244Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5313lb # [ 126.702338] niks3-server[1213]: 2026/09/25 15:01:27 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5314worker1 # [ 126.731167] nix-grpc-daemon[1288]: time=2026-09-25T15:01:27.262Z level=INFO msg="Uploading 1 narinfos"5315lb # [ 126.709150] niks3-server[1213]: 2026/09/25 15:01:27 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5316lb # [ 126.711470] niks3-server[1213]: 2026/09/25 15:01:27 INFO Signed narinfos id=39 count=15317lb # [ 126.720573] niks3-server[1213]: 2026/09/25 15:01:27 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5318worker1 # [ 126.752793] nix-grpc-daemon[1288]: time=2026-09-25T15:01:27.284Z level=INFO msg="Upload complete. (167ms)"5319lb # [ 126.729803] niks3-server[1213]: 2026/09/25 15:01:27 INFO Received complete push request method=POST path=/api/pushes/39/complete5320lb # [ 126.733564] niks3-server[1213]: 2026/09/25 15:01:27 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5321worker1 # [ 126.760201] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=0 paths=1 nar_bytes_in=7845322worker1 # [ 126.768207] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285323worker1 # [ 126.774601] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5324worker1 # [ 126.776568] nix-daemon[855]: reaped child process 1358, status = succeeded5325client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5326client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5327worker1 # [ 126.786998] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5328client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5329client # farm-top> LOG-farm-top-after-niks3-restart5330worker1 # [ 127.621375] nix-daemon[855]: reaped child process 1367, status = succeeded5331worker1 # [ 127.726711] nix-daemon[855]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)5332worker1 # [ 127.746106] nix-daemon[855]: reaped child process 1386, status = succeeded5333lb # [ 127.725688] niks3-server[1213]: 2026/09/25 15:01:28 INFO Received push request method=POST path=/api/pushes5334worker1 # [ 127.756414] nix-grpc-daemon[1288]: time=2026-09-25T15:01:28.287Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5335worker1 # [ 127.758114] nix-grpc-daemon[1288]: time=2026-09-25T15:01:28.287Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5336lb # [ 127.755851] niks3-server[1213]: 2026/09/25 15:01:28 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5337worker1 # [ 127.782962] nix-grpc-daemon[1288]: time=2026-09-25T15:01:28.314Z level=INFO msg="Uploading 1 narinfos"5338lb # [ 127.766089] niks3-server[1213]: 2026/09/25 15:01:28 INFO Signed narinfos id=40 count=15339lb # [ 127.767856] niks3-server[1213]: 2026/09/25 15:01:28 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5340lb # [ 127.776525] niks3-server[1213]: 2026/09/25 15:01:28 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5341lb # [ 127.779433] niks3-server[1213]: 2026/09/25 15:01:28 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5342lb # [ 127.789055] niks3-server[1213]: 2026/09/25 15:01:28 INFO Received complete push request method=POST path=/api/pushes/40/complete5343worker1 # [ 127.815852] nix-grpc-daemon[1288]: time=2026-09-25T15:01:28.347Z level=INFO msg="Upload complete. (221ms)"5344lb # [ 127.792923] niks3-server[1213]: 2026/09/25 15:01:28 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5345worker1 # [ 127.824615] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15346worker1 # [ 127.833485] nix-daemon[855]: reaped child process 1365, status = succeeded5347client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.01 seconds)5348client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5349client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.14 seconds)5350(finished: subtest: a niks3 restart does not cost the leader its role, in 5.56 seconds)5351subtest: scheduler down: the other node takes the lock and keeps it5352lb: must succeed: curl -sf localhost:9901/clusters5353lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5354lb: must succeed: curl -sf localhost:9901/clusters5355lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5356lb: must succeed: curl -sf localhost:9901/clusters5357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5358lb: must succeed: curl -sf localhost:9901/clusters5359lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5360worker2 # [ 128.134076] systemd[1]: Stopping Nix worker-protocol over gRPC...5361worker2 # [ 128.199436] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:28Z level=info event=signal_exit5362lb # [ 128.267318] envoy[897]: conn peer=[2001:db8:1::3]:51224 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2359 tx=1186 ms=809415363lb # [ 128.270194] envoy[897]: conn peer=[2001:db8:1::1]:58548 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=305 ms=12045364lb # [ 128.272789] envoy[897]: conn peer=[2001:db8:1::1]:53302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9680 tx=4400 ms=48235365lb # [ 128.512140] niks3-server[1213]: 2026/09/25 15:01:29 INFO lead: released remote=192.168.1.4:366805366worker1 # [ 129.533429] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:30Z level=info event=scheduler_restarting addr=lb:500515367worker2 # [ 129.509814] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:30Z level=info event=scheduler_restarting5368worker2 # [ 129.512769] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:30Z level=info event=scheduler_restarting addr=lb:500515369worker2 # [ 129.515014] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:30Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:480225370worker2 # [ 129.517939] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:30Z level=info event=worker_gone addr=192.168.1.3:500515371worker2 # [ 129.520173] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:30Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:480225372worker2 # [ 129.522634] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:30Z level=info event=worker_gone addr=192.168.1.4:500515373worker2 # [ 129.538141] nix-daemon[845]: reaped child process 1308, status = succeeded5374worker1 # [ 129.733569] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:30Z level=info event=scheduler_connected addr=lb:500515375worker1 # [ 129.736695] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker2 # [ 129.717614] nix-grpc-daemon[1256]: ts=2026-09-25T15:01:30Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5377worker1 # [ 129.937064] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:30Z level=info event=scheduler_connected addr=lb:500515378worker1 # [ 129.939872] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5379worker1 # [ 130.140416] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:30Z level=info event=scheduler_connected addr=lb:500515380worker1 # [ 130.143232] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5381worker1 # [ 130.343843] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:30Z level=info event=scheduler_connected addr=lb:500515382worker1 # [ 130.346760] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5383worker1 # [ 130.548222] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_connected addr=lb:500515384worker1 # [ 130.550341] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5385worker1 # [ 130.748639] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_connected addr=lb:500515386worker1 # [ 130.751453] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5387worker2 # [ 130.746167] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5388worker2 # [ 130.750479] systemd[1]: Stopped Nix worker-protocol over gRPC.5389worker2 # [ 130.751962] systemd[1]: nix-grpc-daemon.service: Consumed 495ms CPU time over 14.687s wall clock time, 33.5M memory peak, 45.8K incoming IP traffic, 70.5K outgoing IP traffic.5390worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5391worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.04 seconds)5392worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5393worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5394worker1 # [ 130.952198] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_connected addr=lb:500515395worker1 # [ 130.954951] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5396worker1 # [ 131.155498] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_connected addr=lb:500515397worker1 # [ 131.158371] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5398worker1 # [ 131.358860] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_connected addr=lb:500515399worker1 # [ 131.361694] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5400worker1 # [ 131.562951] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_connected addr=lb:500515401worker1 # [ 131.565209] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5402worker1 # [ 131.763166] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_connected addr=lb:500515403worker1 # [ 131.766007] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5404worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5405worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5406worker1 # [ 131.966426] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_connected addr=lb:500515407worker1 # [ 131.969231] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5408worker1 # [ 132.169624] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_connected addr=lb:500515409worker1 # [ 132.172348] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5410worker1 # [ 132.372756] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_connected addr=lb:500515411worker1 # [ 132.375462] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5412worker1 # [ 132.576537] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_connected addr=lb:500515413worker1 # [ 132.579091] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5414worker1 # [ 132.779555] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_connected addr=lb:500515415worker1 # [ 132.782308] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5416worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5417worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5418worker1 # [ 132.982782] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_connected addr=lb:500515419worker1 # [ 132.985564] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5420worker1 # [ 133.186005] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_connected addr=lb:500515421worker1 # [ 133.188722] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5422worker1 # [ 133.389197] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_connected addr=lb:500515423worker1 # [ 133.391916] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5424worker1 # [ 133.530162] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_take_over5425lb # [ 133.507179] niks3-server[1213]: 2026/09/25 15:01:34 INFO lead: acquired remote=192.168.1.3:376365426worker1 # [ 133.593272] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_connected addr=lb:500515427worker1 # [ 133.595427] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5428worker1 # [ 133.793652] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_connected addr=lb:500515429worker1 # [ 133.796386] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5430worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5431worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5432worker1 # [ 133.996871] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_connected addr=lb:500515433worker1 # [ 133.999617] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5434worker1 # [ 134.200056] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_connected addr=lb:500515435worker1 # [ 134.202718] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5436worker1 # [ 134.403257] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_connected addr=lb:500515437worker1 # [ 134.406028] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5438worker1 # [ 134.607203] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=info event=scheduler_connected addr=lb:500515439worker1 # [ 134.617685] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:382305440worker1 # [ 134.620521] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05441worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5442worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5443client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25444client # don't know how to build these paths:5445client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5446worker1 # [ 135.194212] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:337285447worker1 # [ 135.199727] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337285448worker1 # [ 135.211082] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05449worker1 # [ 135.216259] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15450worker1 # [ 135.220685] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285451worker1 # [ 135.224626] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337285452client # copying 1 paths to 'lb:50051'...5453client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5454worker1 # [ 135.231377] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:337285455worker1 # [ 135.240143] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5456worker1 # [ 135.361646] nix-daemon[855]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)5457worker1 # [ 135.377825] nix-daemon[855]: reaped child process 1416, status = succeeded5458lb # [ 135.359958] niks3-server[1213]: 2026/09/25 15:01:35 INFO Received push request method=POST path=/api/pushes5459worker1 # [ 135.390627] nix-grpc-daemon[1288]: time=2026-09-25T15:01:35.921Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5460worker1 # [ 135.392318] nix-grpc-daemon[1288]: time=2026-09-25T15:01:35.922Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5461lb # [ 135.378797] niks3-server[1213]: 2026/09/25 15:01:35 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5462worker1 # [ 135.408774] nix-grpc-daemon[1288]: time=2026-09-25T15:01:35.940Z level=INFO msg="Uploading 1 narinfos"5463lb # [ 135.387350] niks3-server[1213]: 2026/09/25 15:01:35 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5464lb # [ 135.392521] niks3-server[1213]: 2026/09/25 15:01:35 INFO Signed narinfos id=41 count=15465lb # [ 135.400955] niks3-server[1213]: 2026/09/25 15:01:35 INFO Received complete push request method=POST path=/api/pushes/41/complete5466lb # [ 135.403374] niks3-server[1213]: 2026/09/25 15:01:35 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5467lb # [ 135.414777] niks3-server[1213]: 2026/09/25 15:01:35 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5468worker1 # [ 135.454230] nix-grpc-daemon[1288]: time=2026-09-25T15:01:35.986Z level=INFO msg="Upload complete. (205ms)"5469worker1 # [ 135.460269] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=0 paths=1 nar_bytes_in=5205470worker1 # [ 135.463582] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285471worker1 # [ 135.475100] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5472worker1 # [ 135.479406] nix-daemon[855]: reaped child process 1411, status = succeeded5473client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5474client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5475worker1 # [ 135.488597] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5476client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5477client # farm-a> LOG-farm-a-on-standby5478worker1 # [ 136.366485] nix-daemon[855]: reaped child process 1420, status = succeeded5479worker1 # [ 136.428048] nix-daemon[855]: accepted connection from pid 1436, user nix-grpc-daemon (trusted)5480worker1 # [ 136.450373] nix-daemon[855]: reaped child process 1439, status = succeeded5481lb # [ 136.426470] niks3-server[1213]: 2026/09/25 15:01:36 INFO Received push request method=POST path=/api/pushes5482worker1 # [ 136.459982] nix-grpc-daemon[1288]: time=2026-09-25T15:01:36.989Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5483worker1 # [ 136.461719] nix-grpc-daemon[1288]: time=2026-09-25T15:01:36.989Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5484lb # [ 136.457977] niks3-server[1213]: 2026/09/25 15:01:36 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5485worker1 # [ 136.489816] nix-grpc-daemon[1288]: time=2026-09-25T15:01:37.021Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 136.467792] niks3-server[1213]: 2026/09/25 15:01:36 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5487lb # [ 136.475465] niks3-server[1213]: 2026/09/25 15:01:36 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5488lb # [ 136.482352] niks3-server[1213]: 2026/09/25 15:01:36 INFO Signed narinfos id=42 count=15489lb # [ 136.489685] niks3-server[1213]: 2026/09/25 15:01:36 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5490worker1 # [ 136.520289] nix-grpc-daemon[1288]: time=2026-09-25T15:01:37.052Z level=INFO msg="Upload complete. (222ms)"5491lb # [ 136.497000] niks3-server[1213]: 2026/09/25 15:01:37 INFO Received complete push request method=POST path=/api/pushes/42/complete5492lb # [ 136.500742] niks3-server[1213]: 2026/09/25 15:01:37 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5493worker1 # [ 136.535135] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15494worker1 # [ 136.551125] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:37Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05495worker1 # [ 136.554426] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:37Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15496worker1 # [ 136.560132] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285497worker1 # [ 136.564229] nix-daemon[855]: reaped child process 1418, status = succeeded5498worker1 # [ 136.566441] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337285499worker1 # [ 136.574720] nix-grpc-daemon[1288]: time=2026-09-25T15:01:37.106Z level=INFO msg="All 1 paths already cached"5500client # copying 1 paths to 'lb:50051'...5501client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5502worker1 # [ 136.581398] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:337285503worker1 # [ 136.589578] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5504worker1 # [ 136.707236] nix-daemon[855]: accepted connection from pid 1445, user nix-grpc-daemon (trusted)5505worker1 # [ 136.723263] nix-daemon[855]: reaped child process 1448, status = succeeded5506lb # [ 136.705764] niks3-server[1213]: 2026/09/25 15:01:37 INFO Received push request method=POST path=/api/pushes5507worker1 # [ 136.735303] nix-grpc-daemon[1288]: time=2026-09-25T15:01:37.266Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5508worker1 # [ 136.737000] nix-grpc-daemon[1288]: time=2026-09-25T15:01:37.266Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5509worker1 # [ 136.752849] nix-grpc-daemon[1288]: time=2026-09-25T15:01:37.284Z level=INFO msg="Uploading 1 narinfos"5510lb # [ 136.725566] niks3-server[1213]: 2026/09/25 15:01:37 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5511lb # [ 136.733654] niks3-server[1213]: 2026/09/25 15:01:37 INFO Signed narinfos id=43 count=15512lb # [ 136.735341] niks3-server[1213]: 2026/09/25 15:01:37 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5513lb # [ 136.746624] niks3-server[1213]: 2026/09/25 15:01:37 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5514worker1 # [ 136.774231] nix-grpc-daemon[1288]: time=2026-09-25T15:01:37.306Z level=INFO msg="Upload complete. (175ms)"5515lb # [ 136.751600] niks3-server[1213]: 2026/09/25 15:01:37 INFO Received complete push request method=POST path=/api/pushes/43/complete5516worker1 # [ 136.780181] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=0 paths=1 nar_bytes_in=6405517lb # [ 136.757646] niks3-server[1213]: 2026/09/25 15:01:37 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5518worker1 # [ 136.790203] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285519worker1 # [ 136.794705] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5520worker1 # [ 136.798434] nix-daemon[855]: reaped child process 1443, status = succeeded5521client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5522worker1 # [ 136.807295] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5523client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5524client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5525client # farm-b> LOG-farm-b-on-standby5526worker1 # [ 138.047417] nix-daemon[855]: reaped child process 1452, status = succeeded5527worker1 # [ 138.147833] nix-daemon[855]: accepted connection from pid 1468, user nix-grpc-daemon (trusted)5528worker1 # [ 138.167603] nix-daemon[855]: reaped child process 1471, status = succeeded5529lb # [ 138.148678] niks3-server[1213]: 2026/09/25 15:01:38 INFO Received push request method=POST path=/api/pushes5530worker1 # [ 138.180421] nix-grpc-daemon[1288]: time=2026-09-25T15:01:38.710Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5531worker1 # [ 138.181968] nix-grpc-daemon[1288]: time=2026-09-25T15:01:38.710Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5532lb # [ 138.182466] niks3-server[1213]: 2026/09/25 15:01:38 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5533worker1 # [ 138.212113] nix-grpc-daemon[1288]: time=2026-09-25T15:01:38.743Z level=INFO msg="Uploading 1 narinfos"5534lb # [ 138.190505] niks3-server[1213]: 2026/09/25 15:01:38 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5535lb # [ 138.197050] niks3-server[1213]: 2026/09/25 15:01:38 INFO Signed narinfos id=44 count=15536lb # [ 138.198715] niks3-server[1213]: 2026/09/25 15:01:38 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5537lb # [ 138.209475] niks3-server[1213]: 2026/09/25 15:01:38 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5538worker1 # [ 138.236403] nix-grpc-daemon[1288]: time=2026-09-25T15:01:38.768Z level=INFO msg="Upload complete. (219ms)"5539lb # [ 138.212435] niks3-server[1213]: 2026/09/25 15:01:38 INFO Received complete push request method=POST path=/api/pushes/44/complete5540lb # [ 138.216385] niks3-server[1213]: 2026/09/25 15:01:38 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5541worker1 # [ 138.245021] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15542worker1 # [ 138.258124] nix-daemon[855]: reaped child process 1450, status = succeeded5543worker1 # [ 138.260847] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:38Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05544worker1 # [ 138.266072] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:38Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15545worker1 # [ 138.270909] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285546worker1 # [ 138.275143] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337285547worker1 # [ 138.281661] nix-grpc-daemon[1288]: time=2026-09-25T15:01:38.813Z level=INFO msg="All 2 paths already cached"5548client # copying 1 paths to 'lb:50051'...5549client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5550worker1 # [ 138.288853] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:337285551lb # [ 138.269897] envoy[897]: conn peer=[2001:db8:1::4]:52164 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=12628 tx=764 ms=134695552worker1 # [ 138.297620] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5553worker1 # [ 138.418656] nix-daemon[855]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)5554worker1 # [ 138.440274] nix-daemon[855]: reaped child process 1480, status = succeeded5555lb # [ 138.420453] niks3-server[1213]: 2026/09/25 15:01:38 INFO Received push request method=POST path=/api/pushes5556worker1 # [ 138.450176] nix-grpc-daemon[1288]: time=2026-09-25T15:01:38.981Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5557worker1 # [ 138.451759] nix-grpc-daemon[1288]: time=2026-09-25T15:01:38.981Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5558lb # [ 138.441444] niks3-server[1213]: 2026/09/25 15:01:38 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign5559worker1 # [ 138.468293] nix-grpc-daemon[1288]: time=2026-09-25T15:01:39.000Z level=INFO msg="Uploading 1 narinfos"5560lb # [ 138.448122] niks3-server[1213]: 2026/09/25 15:01:38 INFO Signed narinfos id=45 count=15561lb # [ 138.450307] niks3-server[1213]: 2026/09/25 15:01:38 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5562lb # [ 138.461109] niks3-server[1213]: 2026/09/25 15:01:38 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5563lb # [ 138.463725] niks3-server[1213]: 2026/09/25 15:01:38 INFO Received complete push request method=POST path=/api/pushes/45/complete5564worker1 # [ 138.493668] nix-grpc-daemon[1288]: time=2026-09-25T15:01:39.025Z level=INFO msg="Upload complete. (185ms)"5565lb # [ 138.471366] niks3-server[1213]: 2026/09/25 15:01:38 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5566worker1 # [ 138.500173] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=0 paths=1 nar_bytes_in=7685567worker1 # [ 138.509560] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285568worker1 # [ 138.514219] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5569worker1 # [ 138.517891] nix-daemon[855]: reaped child process 1475, status = succeeded5570client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5571client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5572worker1 # [ 138.528045] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5573worker1 # [ 139.064108] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:39Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:382445574client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5575client # farm-top> LOG-farm-top-on-standby5576worker1 # [ 139.351510] nix-daemon[855]: reaped child process 1484, status = succeeded5577worker1 # [ 139.438297] nix-daemon[855]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)5578worker1 # [ 139.455888] nix-daemon[855]: reaped child process 1503, status = succeeded5579lb # [ 139.436759] niks3-server[1213]: 2026/09/25 15:01:39 INFO Received push request method=POST path=/api/pushes5580worker1 # [ 139.474890] nix-grpc-daemon[1288]: time=2026-09-25T15:01:40.001Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5581worker1 # [ 139.478410] nix-grpc-daemon[1288]: time=2026-09-25T15:01:40.001Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5582lb # [ 139.473206] niks3-server[1213]: 2026/09/25 15:01:39 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign5583worker1 # [ 139.505833] nix-grpc-daemon[1288]: time=2026-09-25T15:01:40.037Z level=INFO msg="Uploading 1 narinfos"5584lb # [ 139.482040] niks3-server[1213]: 2026/09/25 15:01:39 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5585lb # [ 139.485004] niks3-server[1213]: 2026/09/25 15:01:40 INFO Signed narinfos id=46 count=15586lb # [ 139.494375] niks3-server[1213]: 2026/09/25 15:01:40 INFO Received complete push request method=POST path=/api/pushes/46/complete5587lb # [ 139.505147] niks3-server[1213]: 2026/09/25 15:01:40 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5588lb # [ 139.512259] niks3-server[1213]: 2026/09/25 15:01:40 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5589worker1 # [ 139.540384] nix-grpc-daemon[1288]: time=2026-09-25T15:01:40.072Z level=INFO msg="Upload complete. (204ms)"5590lb # [ 139.519118] niks3-server[1213]: 2026/09/25 15:01:40 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5591worker1 # [ 139.548554] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15592worker1 # [ 139.557679] nix-daemon[855]: reaped child process 1482, status = succeeded5593client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 4.56 seconds)5594client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5595client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.16 seconds)5596worker2 # [ 139.757427] systemd[1]: Starting scheduler-token.service...5597worker2 # [ 139.808092] systemd[1]: scheduler-token.service: Deactivated successfully.5598worker2 # [ 139.812123] systemd[1]: Finished scheduler-token.service.5599worker2 # [ 139.813055] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 52ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5600worker2 # [ 139.817804] systemd[1]: Starting Nix worker-protocol over gRPC...5601worker2 # [ 139.941249] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5602worker2 # [ 139.944845] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5603worker2 # [ 139.949295] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15604worker2 # [ 139.980576] nix-grpc-daemon[1372]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5605worker2 # [ 139.982899] nix-grpc-daemon[1372]: E0000 00:00:1790348500.497272 1372 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5606lb: must succeed: curl -sf localhost:9901/clusters5607worker2 # [ 140.008385] nix-grpc-daemon[1372]: E0000 00:00:1790348500.499531 1372 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5608worker2 # [ 140.012724] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:40Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5609worker2 # [ 140.015496] nix-grpc-daemon[1372]: E0000 00:00:1790348500.507064 1412 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5610worker2 # [ 140.019265] systemd[1]: Started Nix worker-protocol over gRPC.5611worker2 # [ 140.024420] nix-grpc-daemon[1372]: E0000 00:00:1790348500.533352 1412 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5612worker2 # [ 140.028317] nix-grpc-daemon[1372]: E0000 00:00:1790348500.533521 1412 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5613worker2 # [ 140.034512] nix-grpc-daemon[1372]: E0000 00:00:1790348500.549545 1412 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5614worker2 # [ 140.038302] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:40Z level=info event=scheduler_connected addr=lb:500515615lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5616worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5617worker1 # [ 140.088400] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:40Z level=info event=oidc_jwks provider=mock keys=15618worker1 # [ 140.096371] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:382305619worker1 # [ 140.101506] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05620worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5621lb: must succeed: curl -sf localhost:9901/clusters5622lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.18 seconds)5623lb: must succeed: curl -sf localhost:9901/clusters5624lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5625worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5626worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5627(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.42 seconds)5628subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5629client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5630client # [ 140.417502] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5631client # Running as unit: rs.service; invocation ID: 62639a92d1a74bc8bd5610aff662ded75632client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.05 seconds)5633worker1 # [ 140.646464] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:337285634client # [ 140.630335] nix[1354]: don't know how to build these paths:5635client # [ 140.631285] nix[1354]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5636worker1 # [ 140.662469] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:41Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05637worker1 # [ 140.664824] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:41Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15638worker1 # [ 140.677084] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285639worker1 # [ 140.690658] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337285640client # [ 140.676223] nix[1354]: copying 1 paths to 'lb:50051'...5641worker1 # [ 140.699550] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:337285642client # [ 140.679127] nix[1354]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5643worker1 # [ 140.709584] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5644worker1 # [ 140.834672] nix-daemon[855]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)5645worker1 # [ 140.852475] nix-daemon[855]: reaped child process 1519, status = succeeded5646lb # [ 140.834723] niks3-server[1213]: 2026/09/25 15:01:41 INFO Received push request method=POST path=/api/pushes5647worker1 # [ 140.863194] nix-grpc-daemon[1288]: time=2026-09-25T15:01:41.394Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5648worker1 # [ 140.865723] nix-grpc-daemon[1288]: time=2026-09-25T15:01:41.394Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5649lb # [ 140.854779] niks3-server[1213]: 2026/09/25 15:01:41 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign5650worker1 # [ 140.885114] nix-grpc-daemon[1288]: time=2026-09-25T15:01:41.417Z level=INFO msg="Uploading 1 narinfos"5651lb # [ 140.863462] niks3-server[1213]: 2026/09/25 15:01:41 INFO Signed narinfos id=47 count=15652lb # [ 140.867917] niks3-server[1213]: 2026/09/25 15:01:41 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5653lb # [ 140.876651] niks3-server[1213]: 2026/09/25 15:01:41 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5654lb # [ 140.881949] niks3-server[1213]: 2026/09/25 15:01:41 INFO Received complete push request method=POST path=/api/pushes/47/complete5655worker1 # [ 140.910652] nix-grpc-daemon[1288]: time=2026-09-25T15:01:41.442Z level=INFO msg="Upload complete. (191ms)"5656lb # [ 140.887583] niks3-server[1213]: 2026/09/25 15:01:41 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5657worker1 # [ 140.916191] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=0 paths=1 nar_bytes_in=4165658worker1 # [ 140.923292] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337285659worker1 # [ 140.932152] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5660worker1 # [ 140.936072] nix-daemon[855]: reaped child process 1514, status = succeeded5661client # [ 140.922020] nix[1354]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5662worker1 # [ 140.947511] nix-daemon[855]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5663lb: must succeed: curl -sf localhost:9901/clusters5664lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5665lb: must succeed: curl -sf localhost:9901/clusters5666lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5667lb: must succeed: curl -sf localhost:9901/clusters5668lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5669lb: must succeed: curl -sf localhost:9901/clusters5670lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5671worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5672worker2 # [ 142.974930] systemd-journald[272]: Received client request to rotate journal, rotating.5673worker2 # [ 142.986270] systemd-journald[272]: Vacuuming done, freed 0B of archived journals from /var/log/journal/0530605f59a340c7bb6a90097d89d276.5674worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.04 seconds)5675worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5676worker1 # [ 143.035702] systemd[1]: Stopping Nix worker-protocol over gRPC...5677worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5678client: must succeed: sleep 5.05679worker1 # [ 143.313890] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:43Z level=info event=draining5680client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5681worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5682lb # [ 148.268134] envoy[897]: conn peer=[2001:db8:1::1]:55064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9602 tx=4346 ms=43795683worker1 # [ 148.431423] nix-daemon[855]: reaped child process 1523, status = succeeded5684worker1 # [ 148.494379] nix-daemon[855]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5685worker1 # [ 148.510162] nix-daemon[855]: reaped child process 1567, status = succeeded5686lb # [ 148.492217] niks3-server[1213]: 2026/09/25 15:01:49 INFO Received push request method=POST path=/api/pushes5687worker1 # [ 148.524260] nix-grpc-daemon[1288]: time=2026-09-25T15:01:49.053Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5688worker1 # [ 148.525754] nix-grpc-daemon[1288]: time=2026-09-25T15:01:49.053Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5689worker1 # [ 148.545580] nix-grpc-daemon[1288]: time=2026-09-25T15:01:49.077Z level=INFO msg="Uploading 1 narinfos"5690lb # [ 148.522805] niks3-server[1213]: 2026/09/25 15:01:49 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign5691lb # [ 148.528932] niks3-server[1213]: 2026/09/25 15:01:49 INFO Signed narinfos id=48 count=15692lb # [ 148.530692] niks3-server[1213]: 2026/09/25 15:01:49 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5693lb # [ 148.540691] niks3-server[1213]: 2026/09/25 15:01:49 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5694worker1 # [ 148.574676] nix-grpc-daemon[1288]: time=2026-09-25T15:01:49.106Z level=INFO msg="Upload complete. (214ms)"5695lb # [ 148.551134] niks3-server[1213]: 2026/09/25 15:01:49 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5696lb # [ 148.557423] niks3-server[1213]: 2026/09/25 15:01:49 INFO Received complete push request method=POST path=/api/pushes/48/complete5697lb # [ 148.560423] niks3-server[1213]: 2026/09/25 15:01:49 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5698worker1 # [ 148.584687] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33728 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15699worker1 # [ 148.594360] nix-daemon[855]: reaped child process 1521, status = succeeded5700client # [ 148.585718] systemd[1]: rs.service: Deactivated successfully.5701client # [ 148.588782] systemd[1]: rs.service: Consumed 173ms CPU time over 8.170s wall clock time, 41.4M memory peak, 6.9K incoming IP traffic, 8.3K outgoing IP traffic.5702worker1 # [ 149.315579] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:49Z level=info event=signal_exit5703lb # [ 149.513283] niks3-server[1213]: 2026/09/25 15:01:50 INFO lead: released remote=192.168.1.3:376365704lb # [ 149.976992] niks3-server[1213]: 2026/09/25 15:01:50 INFO lead: acquired remote=192.168.1.4:501885705worker2 # [ 149.976218] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:50Z level=info event=scheduler_take_over5706worker1 # [ 150.533894] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:51Z level=info event=scheduler_restarting5707lb # [ 150.509679] envoy[897]: [2026-09-25 15:01:51.031][897][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5708worker2 # [ 150.511529] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:51Z level=info event=scheduler_restarting addr=lb:500515709worker1 # [ 150.539743] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:382305710worker1 # [ 150.545957] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:51Z level=info event=worker_gone addr=192.168.1.4:500515711worker1 # [ 150.548940] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:51Z level=info event=scheduler_restarting addr=lb:500515712worker1 # [ 150.551226] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:382305713worker1 # [ 150.554732] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:51Z level=info event=worker_gone addr=192.168.1.3:500515714worker1 # [ 150.569397] nix-daemon[855]: reaped child process 1278, status = succeeded5715worker2 # [ 150.712496] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:51Z level=info event=scheduler_connected addr=lb:500515716worker2 # [ 150.721637] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:51Z level=info event=oidc_jwks provider=mock keys=15717worker2 # [ 150.723845] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:332705718worker2 # [ 150.727067] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05719worker1 # [ 150.758433] nix-grpc-daemon[1226]: ts=2026-09-25T15:01:51Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5720worker2 # [ 151.005080] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:51Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:589545721worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.14 seconds)5722client: waiting for success: ! systemctl is-active rs5723client: (finished: waiting for success: ! systemctl is-active rs, in 0.03 seconds)5724client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5725client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5726client: must fail: journalctl -u rs | grep -q 'reconnecting'5727client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5728lb: must succeed: curl -sf localhost:9901/clusters5729lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5730worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5731worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5732worker1 # [ 152.170790] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5733worker1 # [ 152.174720] systemd[1]: Stopped Nix worker-protocol over gRPC.5734worker1 # [ 152.177849] systemd[1]: nix-grpc-daemon.service: Consumed 2.000s CPU time over 31.007s wall clock time, 46.8M memory peak, 365.4K incoming IP traffic, 244.7K outgoing IP traffic.5735worker1 # [ 152.188078] systemd[1]: Starting Nix worker-protocol over gRPC...5736worker1 # [ 152.299263] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5737worker1 # [ 152.301589] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5738worker1 # [ 152.305414] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15739worker1 # [ 152.311820] nix-grpc-daemon[1575]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5740worker1 # [ 152.328105] nix-grpc-daemon[1575]: E0000 00:00:1790348512.841734 1575 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5741worker1 # [ 152.353001] nix-grpc-daemon[1575]: E0000 00:00:1790348512.843732 1575 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5742worker1 # [ 152.355821] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:52Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5743worker1 # [ 152.357756] nix-grpc-daemon[1575]: E0000 00:00:1790348512.851620 1616 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5744worker1 # [ 152.363166] nix-grpc-daemon[1575]: E0000 00:00:1790348512.869567 1616 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5745worker1 # [ 152.367570] nix-grpc-daemon[1575]: E0000 00:00:1790348512.869709 1616 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5746worker2 # [ 152.341594] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:332705747worker1 # [ 152.370713] nix-grpc-daemon[1575]: E0000 00:00:1790348512.884115 1616 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5748worker2 # [ 152.341799] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05749worker1 # [ 152.375615] systemd[1]: Started Nix worker-protocol over gRPC.5750worker1 # [ 152.377582] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:52Z level=info event=scheduler_connected addr=lb:500515751lb: must succeed: curl -sf localhost:9901/clusters5752lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5753worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5754worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5755(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.24 seconds)5756subtest: scheduler restart mid-build: build finishes, no double build5757client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5758client # [ 152.664876] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5759client # Running as unit: mid.service; invocation ID: df07780cf0d740db81f7751a50c931755760client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.05 seconds)5761worker1 # [ 152.859663] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524665762worker1 # [ 152.868295] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)5763client # [ 152.857355] nix[1390]: don't know how to build these paths:5764client # [ 152.858449] nix[1390]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5765worker2 # [ 152.857501] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:53Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05766worker2 # [ 152.864357] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:53Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15767worker2 # [ 152.872628] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589585768worker2 # [ 152.879385] nix-daemon[845]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5769worker2 # [ 152.892300] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 1/5)5770worker2 # [ 152.967525] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 114 ms (attempt 2/5)5771worker2 # [ 153.082402] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 3/5)5772worker2 # [ 153.143323] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 662 ms (attempt 4/5)5773worker2 # [ 153.808816] nix-grpc-daemon[1372]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5774worker2 # [ 153.811104] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5775worker2 # [ 153.821975] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589585776client # [ 153.835999] nix[1390]: copying 1 paths to 'lb:50051'...5777worker2 # [ 153.830660] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589585778client # [ 153.836285] nix[1390]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5779worker2 # [ 153.840865] nix-daemon[845]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5780worker2 # [ 153.875223] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:54Z level=info event=niks3_push_started pid=15045781worker2 # [ 153.995057] nix-grpc-daemon[1504]: time=2026-09-25T15:01:54.537Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5782worker2 # [ 154.100061] nix-daemon[845]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)5783worker2 # [ 154.118483] nix-daemon[845]: reaped child process 1519, status = succeeded5784lb # [ 154.128672] niks3-server[1213]: 2026/09/25 15:01:54 INFO Received push request method=POST path=/api/pushes5785worker2 # [ 154.131005] nix-grpc-daemon[1504]: time=2026-09-25T15:01:54.673Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5786worker2 # [ 154.136656] nix-grpc-daemon[1504]: time=2026-09-25T15:01:54.673Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5787lb # [ 154.155315] niks3-server[1213]: 2026/09/25 15:01:54 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign5788worker2 # [ 154.159016] nix-grpc-daemon[1504]: time=2026-09-25T15:01:54.700Z level=INFO msg="Uploading 1 narinfos"5789lb # [ 154.164807] niks3-server[1213]: 2026/09/25 15:01:54 INFO Signed narinfos id=49 count=15790lb # [ 154.170650] niks3-server[1213]: 2026/09/25 15:01:54 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5791lb # [ 154.180407] niks3-server[1213]: 2026/09/25 15:01:54 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5792lb # [ 154.182860] niks3-server[1213]: 2026/09/25 15:01:54 INFO Received complete push request method=POST path=/api/pushes/49/complete5793worker2 # [ 154.184642] nix-grpc-daemon[1504]: time=2026-09-25T15:01:54.727Z level=INFO msg="Upload complete. (185ms)"5794lb # [ 154.189572] niks3-server[1213]: 2026/09/25 15:01:54 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5795worker2 # [ 154.191314] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58958 duration_s=0 paths=1 nar_bytes_in=4245796worker2 # [ 154.196645] nix-grpc-daemon[1372]: ts=2026-09-25T15:01:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589585797worker2 # [ 154.206235] nix-daemon[845]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5798worker2 # [ 154.208274] nix-daemon[845]: reaped child process 1502, status = succeeded5799client # [ 154.221115] nix[1390]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5800worker2 # [ 154.218031] nix-daemon[845]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5801lb: must succeed: curl -sf localhost:9901/clusters5802lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5803lb: must succeed: curl -sf localhost:9901/clusters5804lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5805worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5806worker2 # [ 156.306387] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1372 (nix-grpc-daemon) on client request.5807worker2 # [ 156.308463] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5808lb # [ 156.326593] niks3-server[1213]: 2026/09/25 15:01:56 INFO lead: released remote=192.168.1.4:501885809worker2 # [ 156.328876] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5810client # [ 156.332910] nix[1390]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5811lb # [ 156.331045] envoy[897]: [2026-09-25 15:01:56.851][897][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5812worker2 # [ 156.330533] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5813worker1 # [ 156.355150] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:56Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5814client # [ 156.336304] nix[1390]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5815worker2 # [ 156.331917] systemd[1]: nix-grpc-daemon.service: Consumed 428ms CPU time over 16.508s wall clock time, 34.8M memory peak, 216.7K incoming IP traffic, 52.7K outgoing IP traffic.5816worker2 # [ 156.355864] systemd[1]: Starting scheduler-token.service...5817worker2 # [ 156.390088] nix-daemon[845]: reaped child process 1522, status = succeeded5818worker2 # [ 156.391752] nix-daemon[845]: reaped child process 1492, status = succeeded5819worker2 # [ 156.404336] nix-daemon[1524]: unexpected Nix daemon error: error: interrupted by the user5820worker2 # [ 156.413008] nix-daemon[845]: reaped child process 1524, status = failed with exit code 15821worker2 # [ 156.422303] systemd[1]: scheduler-token.service: Deactivated successfully.5822worker2 # [ 156.424881] systemd[1]: Finished scheduler-token.service.5823worker2 # [ 156.425728] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 76ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5824worker2 # [ 156.431894] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5825worker2 # [ 156.437269] systemd[1]: Starting scheduler-token.service...5826worker2 # [ 156.472903] systemd[1]: scheduler-token.service: Deactivated successfully.5827worker2 # [ 156.474689] systemd[1]: Finished scheduler-token.service.5828worker2 # [ 156.478171] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 43ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5829worker2 # [ 156.486101] systemd[1]: Starting Nix worker-protocol over gRPC...5830worker1 # [ 156.555243] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:57Z level=info event=scheduler_connected addr=lb:500515831worker1 # [ 156.589359] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:57Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. retried and the latest reset reason: remote connection failure"5832worker2 # [ 156.581457] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5833worker2 # [ 156.584158] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5834worker2 # [ 156.588079] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15835worker2 # [ 156.594877] nix-grpc-daemon[1567]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5836lb # [ 156.609063] niks3-server[1213]: 2026/09/25 15:01:57 INFO lead: acquired remote=192.168.1.4:521305837worker2 # [ 156.623441] nix-grpc-daemon[1567]: E0000 00:00:1790348517.135189 1567 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5838worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.37 seconds)5839client: must succeed: sleep 5.05840worker2 # [ 156.644959] nix-grpc-daemon[1567]: E0000 00:00:1790348517.137283 1567 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5841worker2 # [ 156.648696] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5842worker2 # [ 156.650465] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=scheduler_take_over5843worker2 # [ 156.654989] nix-grpc-daemon[1567]: E0000 00:00:1790348517.148443 1607 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5844worker2 # [ 156.662559] nix-grpc-daemon[1567]: E0000 00:00:1790348517.169268 1607 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5845worker2 # [ 156.666203] nix-grpc-daemon[1567]: E0000 00:00:1790348517.169845 1607 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5846worker2 # [ 156.670450] nix-grpc-daemon[1567]: E0000 00:00:1790348517.186342 1607 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5847worker2 # [ 156.673724] systemd[1]: Started Nix worker-protocol over gRPC.5848worker2 # [ 156.676638] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=scheduler_connected addr=lb:500515849worker2 # [ 156.678024] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=oidc_jwks provider=mock keys=15850worker2 # [ 156.679443] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:590305851worker2 # [ 156.681310] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05852worker1 # [ 156.790437] nix-grpc-daemon[1575]: ts=2026-09-25T15:01:57Z level=info event=scheduler_connected addr=lb:500515853worker2 # [ 156.765022] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:590305854worker2 # [ 156.767876] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05855worker2 # [ 156.836172] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05856worker2 # [ 156.843555] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15857worker2 # [ 156.850079] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590385858worker2 # [ 156.858115] nix-daemon[845]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)5859worker2 # [ 156.871673] nix-daemon[845]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)5860client # [ 156.885715] nix[1390]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5861worker2 # [ 156.881920] nix-daemon[845]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)5862worker2 # [ 157.963516] nix-grpc-daemon[1567]: ts=2026-09-25T15:01:58Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:590745863lb # [ 158.272872] envoy[897]: conn peer=[2001:db8:1::1]:55076 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2962 tx=1497 ms=79545864lb # [ 158.275548] envoy[897]: conn peer=[2001:db8:1::3]:39150 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5454 tx=1760 ms=293945865lb # [ 158.278276] envoy[897]: conn peer=[2001:db8:1::4]:60622 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1961 tx=328 ms=163105866client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5867client: waiting for success: ! systemctl is-active mid5868worker2 # [ 161.902419] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:02Z level=info event=niks3_push_started pid=16505869worker2 # [ 161.922887] nix-daemon[845]: reaped child process 1625, status = succeeded5870worker2 # [ 162.021672] nix-grpc-daemon[1650]: time=2026-09-25T15:02:02.563Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5871worker2 # [ 162.123414] nix-daemon[845]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)5872worker2 # [ 162.144364] nix-daemon[845]: reaped child process 1665, status = succeeded5873lb # [ 162.146904] niks3-server[1213]: 2026/09/25 15:02:02 INFO Received push request method=POST path=/api/pushes5874worker2 # [ 162.151948] nix-grpc-daemon[1650]: time=2026-09-25T15:02:02.692Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5875worker2 # [ 162.155756] nix-grpc-daemon[1650]: time=2026-09-25T15:02:02.692Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5876lb # [ 162.183104] niks3-server[1213]: 2026/09/25 15:02:02 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign5877lb # [ 162.194873] niks3-server[1213]: 2026/09/25 15:02:02 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5878lb # [ 162.206841] niks3-server[1213]: 2026/09/25 15:02:02 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5879worker2 # [ 162.212117] nix-grpc-daemon[1650]: time=2026-09-25T15:02:02.753Z level=INFO msg="Uploading 1 narinfos"5880lb # [ 162.217892] niks3-server[1213]: 2026/09/25 15:02:02 INFO Signed narinfos id=50 count=15881lb # [ 162.227821] niks3-server[1213]: 2026/09/25 15:02:02 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5882lb # [ 162.236734] niks3-server[1213]: 2026/09/25 15:02:02 INFO Received complete push request method=POST path=/api/pushes/50/complete5883lb # [ 162.248479] niks3-server[1213]: 2026/09/25 15:02:02 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5884worker2 # [ 162.248772] nix-grpc-daemon[1650]: time=2026-09-25T15:02:02.791Z level=INFO msg="Upload complete. (223ms)"5885worker2 # [ 162.260123] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59038 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15886worker2 # [ 162.269277] nix-daemon[845]: reaped child process 1623, status = succeeded5887client # [ 162.284626] systemd[1]: mid.service: Deactivated successfully.5888client # [ 162.287382] systemd[1]: mid.service: Consumed 173ms CPU time over 9.621s wall clock time, 41.2M memory peak, 8.6K incoming IP traffic, 10.4K outgoing IP traffic.5889client: (finished: waiting for success: ! systemctl is-active mid, in 1.07 seconds)5890client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5891client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5892(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.34 seconds)5893subtest: interrupting the client stops the build on the worker5894client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5895client # [ 163.001823] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5896client # Running as unit: intr.service; invocation ID: 8d9c47fa70ca44148e392e3055ed805f5897client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.04 seconds)5898worker1 # [ 163.179888] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524665899client # [ 163.162298] nix[1426]: don't know how to build these paths:5900client # [ 163.163268] nix[1426]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5901worker2 # [ 163.166041] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:03Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05902worker2 # [ 163.169493] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:03Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15903worker1 # [ 163.199021] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524665904worker1 # [ 163.207569] nix-grpc-daemon[1575]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)5905worker1 # [ 163.278583] nix-grpc-daemon[1575]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 2/5)5906worker1 # [ 163.361622] nix-grpc-daemon[1575]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 302 ms (attempt 3/5)5907worker1 # [ 163.664705] nix-grpc-daemon[1575]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 444 ms (attempt 4/5)5908worker1 # [ 164.110482] nix-grpc-daemon[1575]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5909worker1 # [ 164.113774] nix-grpc-daemon[1575]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5910worker1 # [ 164.126476] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524665911client # [ 164.112135] nix[1426]: copying 1 paths to 'lb:50051'...5912worker1 # [ 164.135619] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524665913client # [ 164.115128] nix[1426]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5914worker1 # [ 164.147590] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)5915worker1 # [ 164.176462] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:04Z level=info event=niks3_push_started pid=16715916worker1 # [ 164.305491] nix-grpc-daemon[1671]: time=2026-09-25T15:02:04.836Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5917worker1 # [ 164.403545] nix-daemon[855]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)5918worker1 # [ 164.421339] nix-daemon[855]: reaped child process 1686, status = succeeded5919lb # [ 164.401506] niks3-server[1213]: 2026/09/25 15:02:04 INFO Received push request method=POST path=/api/pushes5920worker1 # [ 164.433715] nix-grpc-daemon[1671]: time=2026-09-25T15:02:04.964Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5921worker1 # [ 164.435270] nix-grpc-daemon[1671]: time=2026-09-25T15:02:04.964Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5922lb # [ 164.428886] niks3-server[1213]: 2026/09/25 15:02:04 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign5923worker1 # [ 164.456878] nix-grpc-daemon[1671]: time=2026-09-25T15:02:04.988Z level=INFO msg="Uploading 1 narinfos"5924lb # [ 164.437046] niks3-server[1213]: 2026/09/25 15:02:04 INFO Signed narinfos id=51 count=15925lb # [ 164.441462] niks3-server[1213]: 2026/09/25 15:02:04 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5926lb # [ 164.448969] niks3-server[1213]: 2026/09/25 15:02:04 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5927lb # [ 164.460131] niks3-server[1213]: 2026/09/25 15:02:04 INFO Received complete push request method=POST path=/api/pushes/51/complete5928lb # [ 164.467783] niks3-server[1213]: 2026/09/25 15:02:04 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5929worker1 # [ 164.492669] nix-grpc-daemon[1671]: time=2026-09-25T15:02:05.023Z level=INFO msg="Upload complete. (182ms)"5930worker1 # [ 164.495247] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=4245931worker1 # [ 164.509798] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524665932worker1 # [ 164.514503] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)5933worker1 # [ 164.518600] nix-daemon[855]: reaped child process 1668, status = succeeded5934worker1 # [ 164.527519] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)5935client # [ 164.504963] nix[1426]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5936client: must succeed: systemctl kill -s INT intr5937client # [ 166.505015] systemd[1]: intr.service: Sent signal SIGINT to main process 1426 (nix) on client request.5938worker1 # [ 166.531287] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:07Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5939client # [ 166.509731] nix[1426]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5940client # [ 166.517767] nix[1426]: error: interrupted by the user5941client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)5942client # [ 166.536109] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5943client # [ 166.538209] systemd[1]: intr.service: Failed with result 'exit-code'.5944worker1 # [ 166.565937] nix-daemon[1690]: unexpected Nix daemon error: error: interrupted by the user5945client # [ 166.541905] systemd[1]: intr.service: Consumed 150ms CPU time over 3.538s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.5946worker1 # [ 166.576325] nix-daemon[855]: reaped child process 1688, status = succeeded5947worker1 # [ 166.585311] nix-daemon[855]: reaped child process 1690, status = failed with exit code 15948(finished: subtest: interrupting the client stops the build on the worker, in 3.64 seconds)5949subtest: a deploy mid-build drains: the build finishes, the new generation takes over5950client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5951client # [ 166.648109] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5952client # Running as unit: sw.service; invocation ID: ea7ddf7a3c9d453aa87ee4897d8429185953client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.05 seconds)5954worker2 # [ 166.817687] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590385955client # [ 166.827926] nix[1447]: don't know how to build these paths:5956client # [ 166.829017] nix[1447]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5957worker2 # [ 166.830511] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:07Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05958worker2 # [ 166.834458] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:07Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15959worker2 # [ 166.841798] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590385960worker2 # [ 166.846711] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)5961worker2 # [ 166.881642] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 2/5)5962worker2 # [ 167.051748] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 3/5)5963worker2 # [ 167.104712] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 563 ms (attempt 4/5)5964worker2 # [ 167.669076] nix-grpc-daemon[1567]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5965worker2 # [ 167.671093] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5966worker2 # [ 167.683069] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590385967client # [ 167.696093] nix[1447]: copying 1 paths to 'lb:50051'...5968worker2 # [ 167.691844] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590385969client # [ 167.699059] nix[1447]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5970worker2 # [ 167.700609] nix-daemon[845]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)5971worker2 # [ 167.838273] nix-daemon[845]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)5972lb # [ 167.865880] niks3-server[1213]: 2026/09/25 15:02:08 INFO Received push request method=POST path=/api/pushes5973worker2 # [ 167.872419] nix-grpc-daemon[1650]: time=2026-09-25T15:02:08.411Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5974worker2 # [ 167.873936] nix-grpc-daemon[1650]: time=2026-09-25T15:02:08.411Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5975worker2 # [ 167.880117] nix-daemon[845]: reaped child process 1707, status = succeeded5976lb # [ 167.886616] niks3-server[1213]: 2026/09/25 15:02:08 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5977worker2 # [ 167.888158] nix-grpc-daemon[1650]: time=2026-09-25T15:02:08.430Z level=INFO msg="Uploading 1 narinfos"5978lb # [ 167.894879] niks3-server[1213]: 2026/09/25 15:02:08 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign5979lb # [ 167.900896] niks3-server[1213]: 2026/09/25 15:02:08 INFO Signed narinfos id=52 count=15980lb # [ 167.902640] niks3-server[1213]: 2026/09/25 15:02:08 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5981lb # [ 167.913069] niks3-server[1213]: 2026/09/25 15:02:08 INFO Received complete push request method=POST path=/api/pushes/52/complete5982worker2 # [ 167.914168] nix-grpc-daemon[1650]: time=2026-09-25T15:02:08.456Z level=INFO msg="Upload complete. (204ms)"5983lb # [ 167.918649] niks3-server[1213]: 2026/09/25 15:02:08 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5984worker2 # [ 167.924915] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59038 duration_s=0 paths=1 nar_bytes_in=4165985worker2 # [ 167.927499] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590385986worker2 # [ 167.931003] nix-daemon[845]: reaped child process 1699, status = succeeded5987worker2 # [ 167.934839] nix-daemon[845]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)5988client # [ 167.947704] nix[1447]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5989worker2 # [ 167.944231] nix-daemon[845]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)5990lb # [ 168.272858] envoy[897]: conn peer=[2001:db8:1::1]:50044 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3927 tx=1941 ms=94375991lb # [ 168.275534] envoy[897]: conn peer=[2001:db8:1::1]:37262 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2974 tx=825 ms=33665992worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25993worker2 # Checking switch inhibitors... done5994worker2 # [ 169.455154] nixos[1740]: switching to system configuration /nix/store/psv6z7q5q9yl9akmyzsis8bxmmshihgf-nixos-system-worker2-test5995worker2 # activating the configuration...5996worker2 # [ 169.462427] systemd[1]: Stopped target Local File Systems.5997worker2 # [ 169.463285] systemd[1]: Stopped target Remote File Systems.5998worker2 # setting up /etc...5999worker2 # [ 169.976696] systemd[1]: Reload requested from client PID 1740 ('.switch-to-conf') (unit backdoor.service)...6000worker2 # [ 169.978234] systemd[1]: Reloading...6001worker2 # [ 174.636623] systemd[1]: Reloading finished in 4646 ms.6002worker2 # restarting sysinit-reactivation.target6003worker2 # [ 174.763083] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6004worker2 # [ 174.886745] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6005worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service[ 174.890020] systemd[1]: Reached target Reactivate sysinit units.6006worker2 # 6007worker2 # [ 174.892670] systemd[1]: Reloading D-Bus System Message Bus...6008worker2 # [ 174.893837] dbus-broker-launch[492]: Caught SIGHUP, trigger reload.6009worker2 # [ 174.905759] systemd[1]: Reloading Nix worker-protocol over gRPC...6010worker2 # [ 174.918106] dbus-broker-launch[492]: Looking up NSS user entry for 'systemd-timesync'...6011worker2 # [ 174.928701] dbus-broker-launch[492]: NSS returned no entry for 'systemd-timesync'6012worker2 # [ 174.929744] dbus-broker-launch[492]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6013worker2 # [ 174.942590] systemd[1]: Reloaded Nix worker-protocol over gRPC.6014worker2 # [ 174.954811] systemd[1]: Reloaded D-Bus System Message Bus.6015worker2 # [ 174.961627] systemd[1]: Reached target Local File Systems.6016worker2 # [ 174.964758] systemd[1]: Reached target Remote File Systems.6017worker2 # [ 174.974042] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6018worker2 # [ 174.975565] systemd[1]: Starting Create SUID/SGID Wrappers...6019worker2 # [ 174.985166] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6020worker2 # [ 174.986477] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676021worker2 # [ 174.996105] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6022worker2 # [ 174.997714] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6023worker2 # [ 174.998983] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6024worker2 # [ 175.003423] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6025worker2 # [ 175.005647] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6026worker2 # [ 175.300544] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6027worker2 # [ 175.301661] systemd[1]: Finished Create SUID/SGID Wrappers.6028worker2 # [ 175.308468] systemd[1]: Starting linger-users.service...6029worker2 # [ 175.350364] systemd[1]: linger-users.service: Deactivated successfully.6030worker2 # [ 175.351381] systemd[1]: Finished linger-users.service.6031worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6032worker2 # [ 175.604266] nixos[1740]: finished switching to system configuration /nix/store/psv6z7q5q9yl9akmyzsis8bxmmshihgf-nixos-system-worker2-test6033worker2 # [ 175.607457] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:16Z level=info event=draining6034worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.55 seconds)6035worker2: must succeed: pgrep -f 'read -t [9]0 x'6036worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6037client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26038worker1 # [ 175.878127] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524666039client # don't know how to build these paths:6040client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6041worker1 # [ 175.885451] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666042worker2 # [ 175.873122] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:16Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06043worker2 # [ 175.875263] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:16Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16044worker1 # [ 175.904709] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666045worker1 # [ 175.912924] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666046client # copying 1 paths to 'lb:50051'...6047worker1 # [ 175.921492] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666048client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6049worker1 # [ 175.932586] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6050worker1 # [ 176.062389] nix-daemon[855]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6051worker1 # [ 176.078028] nix-daemon[855]: reaped child process 1740, status = succeeded6052lb # [ 176.061050] niks3-server[1213]: 2026/09/25 15:02:16 INFO Received push request method=POST path=/api/pushes6053worker1 # [ 176.091853] nix-grpc-daemon[1671]: time=2026-09-25T15:02:16.623Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6054worker1 # [ 176.093537] nix-grpc-daemon[1671]: time=2026-09-25T15:02:16.623Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6055lb # [ 176.078573] niks3-server[1213]: 2026/09/25 15:02:16 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6056worker1 # [ 176.109767] nix-grpc-daemon[1671]: time=2026-09-25T15:02:16.641Z level=INFO msg="Uploading 1 narinfos"6057lb # [ 176.086687] niks3-server[1213]: 2026/09/25 15:02:16 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6058lb # [ 176.093026] niks3-server[1213]: 2026/09/25 15:02:16 INFO Signed narinfos id=53 count=16059lb # [ 176.097316] niks3-server[1213]: 2026/09/25 15:02:16 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6060lb # [ 176.106921] niks3-server[1213]: 2026/09/25 15:02:16 INFO Received complete push request method=POST path=/api/pushes/53/complete6061worker1 # [ 176.135319] nix-grpc-daemon[1671]: time=2026-09-25T15:02:16.667Z level=INFO msg="Upload complete. (191ms)"6062lb # [ 176.112740] niks3-server[1213]: 2026/09/25 15:02:16 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6063worker1 # [ 176.141522] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=5286064worker1 # [ 176.151511] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666065worker1 # [ 176.156494] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6066worker1 # [ 176.160649] nix-daemon[855]: reaped child process 1735, status = succeeded6067client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6068client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6069worker1 # [ 176.170879] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6070client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6071client # farm-a> LOG-farm-a-during-drain6072worker1 # [ 177.431736] nix-daemon[855]: reaped child process 1744, status = succeeded6073worker1 # [ 177.489210] nix-daemon[855]: accepted connection from pid 1760, user nix-grpc-daemon (trusted)6074worker1 # [ 177.510256] nix-daemon[855]: reaped child process 1763, status = succeeded6075lb # [ 177.484947] niks3-server[1213]: 2026/09/25 15:02:18 INFO Received push request method=POST path=/api/pushes6076worker1 # [ 177.516378] nix-grpc-daemon[1671]: time=2026-09-25T15:02:18.047Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6077worker1 # [ 177.519670] nix-grpc-daemon[1671]: time=2026-09-25T15:02:18.047Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6078lb # [ 177.507706] niks3-server[1213]: 2026/09/25 15:02:18 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6079lb # [ 177.517584] niks3-server[1213]: 2026/09/25 15:02:18 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6080lb # [ 177.524576] niks3-server[1213]: 2026/09/25 15:02:18 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6081worker1 # [ 177.552293] nix-grpc-daemon[1671]: time=2026-09-25T15:02:18.084Z level=INFO msg="Uploading 1 narinfos"6082lb # [ 177.530977] niks3-server[1213]: 2026/09/25 15:02:18 INFO Signed narinfos id=54 count=16083lb # [ 177.534589] niks3-server[1213]: 2026/09/25 15:02:18 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6084lb # [ 177.543317] niks3-server[1213]: 2026/09/25 15:02:18 INFO Received complete push request method=POST path=/api/pushes/54/complete6085worker1 # [ 177.571910] nix-grpc-daemon[1671]: time=2026-09-25T15:02:18.103Z level=INFO msg="Upload complete. (193ms)"6086lb # [ 177.549221] niks3-server[1213]: 2026/09/25 15:02:18 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6087worker2 # [ 177.553944] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:18Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06088worker1 # [ 177.580616] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16089worker2 # [ 177.556038] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:18Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16090worker1 # [ 177.590854] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666091worker1 # [ 177.594805] nix-daemon[855]: reaped child process 1742, status = succeeded6092worker1 # [ 177.599927] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666093worker1 # [ 177.607982] nix-grpc-daemon[1671]: time=2026-09-25T15:02:18.140Z level=INFO msg="All 1 paths already cached"6094client # copying 1 paths to 'lb:50051'...6095worker1 # [ 177.614117] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666096client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6097worker1 # [ 177.627529] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6098worker1 # [ 177.745934] nix-daemon[855]: accepted connection from pid 1767, user nix-grpc-daemon (trusted)6099worker1 # [ 177.764448] nix-daemon[855]: reaped child process 1770, status = succeeded6100lb # [ 177.742322] niks3-server[1213]: 2026/09/25 15:02:18 INFO Received push request method=POST path=/api/pushes6101worker1 # [ 177.771016] nix-grpc-daemon[1671]: time=2026-09-25T15:02:18.302Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6102worker1 # [ 177.772717] nix-grpc-daemon[1671]: time=2026-09-25T15:02:18.302Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6103lb # [ 177.758772] niks3-server[1213]: 2026/09/25 15:02:18 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6104worker1 # [ 177.785455] nix-grpc-daemon[1671]: time=2026-09-25T15:02:18.317Z level=INFO msg="Uploading 1 narinfos"6105lb # [ 177.765589] niks3-server[1213]: 2026/09/25 15:02:18 INFO Signed narinfos id=55 count=16106lb # [ 177.770386] niks3-server[1213]: 2026/09/25 15:02:18 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6107lb # [ 177.779657] niks3-server[1213]: 2026/09/25 15:02:18 INFO Received complete push request method=POST path=/api/pushes/55/complete6108worker1 # [ 177.808425] nix-grpc-daemon[1671]: time=2026-09-25T15:02:18.340Z level=INFO msg="Upload complete. (171ms)"6109lb # [ 177.786492] niks3-server[1213]: 2026/09/25 15:02:18 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6110worker1 # [ 177.815067] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=6406111lb # [ 177.791831] niks3-server[1213]: 2026/09/25 15:02:18 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6112worker1 # [ 177.823697] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666113worker1 # [ 177.830736] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6114worker1 # [ 177.832481] nix-daemon[855]: reaped child process 1765, status = succeeded6115client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6116client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6117worker1 # [ 177.842189] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6118client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6119client # farm-b> LOG-farm-b-during-drain6120worker1 # [ 179.080448] nix-daemon[855]: reaped child process 1774, status = succeeded6121worker1 # [ 179.170781] nix-daemon[855]: accepted connection from pid 1790, user nix-grpc-daemon (trusted)6122worker1 # [ 179.188497] nix-daemon[855]: reaped child process 1793, status = succeeded6123lb # [ 179.169715] niks3-server[1213]: 2026/09/25 15:02:19 INFO Received push request method=POST path=/api/pushes6124worker1 # [ 179.199337] nix-grpc-daemon[1671]: time=2026-09-25T15:02:19.730Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6125worker1 # [ 179.200976] nix-grpc-daemon[1671]: time=2026-09-25T15:02:19.730Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6126lb # [ 179.192727] niks3-server[1213]: 2026/09/25 15:02:19 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6127worker1 # [ 179.224802] nix-grpc-daemon[1671]: time=2026-09-25T15:02:19.756Z level=INFO msg="Uploading 1 narinfos"6128lb # [ 179.202595] niks3-server[1213]: 2026/09/25 15:02:19 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6129lb # [ 179.208295] niks3-server[1213]: 2026/09/25 15:02:19 INFO Signed narinfos id=56 count=16130lb # [ 179.209958] niks3-server[1213]: 2026/09/25 15:02:19 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6131lb # [ 179.220514] niks3-server[1213]: 2026/09/25 15:02:19 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6132worker1 # [ 179.247752] nix-grpc-daemon[1671]: time=2026-09-25T15:02:19.779Z level=INFO msg="Upload complete. (198ms)"6133lb # [ 179.223713] niks3-server[1213]: 2026/09/25 15:02:19 INFO Received complete push request method=POST path=/api/pushes/56/complete6134lb # [ 179.228674] niks3-server[1213]: 2026/09/25 15:02:19 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6135worker1 # [ 179.256610] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16136worker2 # [ 179.232288] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:19Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06137worker2 # [ 179.234343] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:19Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16138worker1 # [ 179.267286] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666139worker1 # [ 179.271393] nix-daemon[855]: reaped child process 1772, status = succeeded6140worker1 # [ 179.276698] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666141worker1 # [ 179.284882] nix-grpc-daemon[1671]: time=2026-09-25T15:02:19.816Z level=INFO msg="All 2 paths already cached"6142client # copying 1 paths to 'lb:50051'...6143client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6144worker1 # [ 179.291377] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666145worker1 # [ 179.300956] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6146worker1 # [ 179.416140] nix-daemon[855]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6147worker1 # [ 179.432534] nix-daemon[855]: reaped child process 1800, status = succeeded6148lb # [ 179.413346] niks3-server[1213]: 2026/09/25 15:02:19 INFO Received push request method=POST path=/api/pushes6149worker1 # [ 179.441623] nix-grpc-daemon[1671]: time=2026-09-25T15:02:19.973Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6150worker1 # [ 179.443183] nix-grpc-daemon[1671]: time=2026-09-25T15:02:19.973Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6151lb # [ 179.429559] niks3-server[1213]: 2026/09/25 15:02:19 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6152worker1 # [ 179.459278] nix-grpc-daemon[1671]: time=2026-09-25T15:02:19.991Z level=INFO msg="Uploading 1 narinfos"6153lb # [ 179.438429] niks3-server[1213]: 2026/09/25 15:02:19 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6154lb # [ 179.443966] niks3-server[1213]: 2026/09/25 15:02:19 INFO Signed narinfos id=57 count=16155lb # [ 179.447499] niks3-server[1213]: 2026/09/25 15:02:19 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6156lb # [ 179.456206] niks3-server[1213]: 2026/09/25 15:02:19 INFO Received complete push request method=POST path=/api/pushes/57/complete6157worker1 # [ 179.486101] nix-grpc-daemon[1671]: time=2026-09-25T15:02:20.018Z level=INFO msg="Upload complete. (175ms)"6158lb # [ 179.462210] niks3-server[1213]: 2026/09/25 15:02:19 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6159worker1 # [ 179.491297] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=7686160worker1 # [ 179.500140] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666161worker1 # [ 179.505841] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6162worker1 # [ 179.507818] nix-daemon[855]: reaped child process 1795, status = succeeded6163client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6164worker1 # [ 179.516324] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6165client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6166client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6167client # farm-top> LOG-farm-top-during-drain6168worker1 # [ 180.752509] nix-daemon[855]: reaped child process 1804, status = succeeded6169worker1 # [ 180.850610] nix-daemon[855]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6170worker1 # [ 180.866753] nix-daemon[855]: reaped child process 1823, status = succeeded6171lb # [ 180.848562] niks3-server[1213]: 2026/09/25 15:02:21 INFO Received push request method=POST path=/api/pushes6172worker1 # [ 180.878632] nix-grpc-daemon[1671]: time=2026-09-25T15:02:21.408Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6173worker1 # [ 180.880222] nix-grpc-daemon[1671]: time=2026-09-25T15:02:21.409Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6174lb # [ 180.869391] niks3-server[1213]: 2026/09/25 15:02:21 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6175worker1 # [ 180.898468] nix-grpc-daemon[1671]: time=2026-09-25T15:02:21.430Z level=INFO msg="Uploading 1 narinfos"6176lb # [ 180.876970] niks3-server[1213]: 2026/09/25 15:02:21 INFO Signed narinfos id=58 count=16177lb # [ 180.882779] niks3-server[1213]: 2026/09/25 15:02:21 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6178lb # [ 180.888795] niks3-server[1213]: 2026/09/25 15:02:21 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6179worker1 # [ 180.921895] nix-grpc-daemon[1671]: time=2026-09-25T15:02:21.453Z level=INFO msg="Upload complete. (196ms)"6180lb # [ 180.899789] niks3-server[1213]: 2026/09/25 15:02:21 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6181lb # [ 180.906043] niks3-server[1213]: 2026/09/25 15:02:21 INFO Received complete push request method=POST path=/api/pushes/58/complete6182worker1 # [ 180.928616] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16183lb # [ 180.909348] niks3-server[1213]: 2026/09/25 15:02:21 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6184worker1 # [ 180.939163] nix-daemon[855]: reaped child process 1802, status = succeeded6185client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.26 seconds)6186client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6187client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.14 seconds)6188worker2: must succeed: pkill -f 'read -t [9]0 x'6189worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6190client: waiting for success: ! systemctl is-active sw6191worker2 # [ 181.194039] nix-daemon[845]: reaped child process 1712, status = succeeded6192worker2 # [ 181.284550] nix-daemon[845]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)6193lb # [ 181.307052] niks3-server[1213]: 2026/09/25 15:02:21 INFO Received push request method=POST path=/api/pushes6194worker2 # [ 181.306245] nix-daemon[845]: reaped child process 1889, status = succeeded6195worker2 # [ 181.309392] nix-grpc-daemon[1650]: time=2026-09-25T15:02:21.850Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6196worker2 # [ 181.310949] nix-grpc-daemon[1650]: time=2026-09-25T15:02:21.851Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6197lb # [ 181.329023] niks3-server[1213]: 2026/09/25 15:02:21 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6198worker2 # [ 181.328290] nix-grpc-daemon[1650]: time=2026-09-25T15:02:21.870Z level=INFO msg="Uploading 1 narinfos"6199lb # [ 181.337196] niks3-server[1213]: 2026/09/25 15:02:21 INFO Signed narinfos id=59 count=16200lb # [ 181.341409] niks3-server[1213]: 2026/09/25 15:02:21 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6201lb # [ 181.352204] niks3-server[1213]: 2026/09/25 15:02:21 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6202lb # [ 181.354830] niks3-server[1213]: 2026/09/25 15:02:21 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6203worker2 # [ 181.356188] nix-grpc-daemon[1650]: time=2026-09-25T15:02:21.898Z level=INFO msg="Upload complete. (188ms)"6204lb # [ 181.361835] niks3-server[1213]: 2026/09/25 15:02:21 INFO Received complete push request method=POST path=/api/pushes/59/complete6205lb # [ 181.365713] niks3-server[1213]: 2026/09/25 15:02:21 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6206worker2 # [ 181.364516] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59038 duration_s=13 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16207worker2 # [ 181.374278] nix-daemon[845]: reaped child process 1710, status = succeeded6208client # [ 181.397475] systemd[1]: sw.service: Deactivated successfully.6209client # [ 181.400694] systemd[1]: sw.service: Consumed 168ms CPU time over 14.750s wall clock time, 41.4M memory peak, 7.6K incoming IP traffic, 8.5K outgoing IP traffic.6210worker2 # [ 181.607319] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:22Z level=info event=signal_exit6211client: (finished: waiting for success: ! systemctl is-active sw, in 1.07 seconds)6212client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6213client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6214worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6215worker2 # [ 182.271453] systemd[1]: Starting scheduler-token.service...6216worker2 # [ 182.327945] systemd[1]: scheduler-token.service: Deactivated successfully.6217worker2 # [ 182.331799] systemd[1]: Finished scheduler-token.service.6218worker2 # [ 182.333226] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 59ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6219lb # [ 182.612807] niks3-server[1213]: 2026/09/25 15:02:23 INFO lead: released remote=192.168.1.4:521306220worker2 # [ 183.608102] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:24Z level=info event=scheduler_restarting6221worker1 # [ 183.637258] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:24Z level=info event=scheduler_restarting addr=lb:500516222lb # [ 183.612125] envoy[897]: [2026-09-25 15:02:24.134][897][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6223worker2 # [ 183.612083] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:24Z level=info event=scheduler_restarting addr=lb:500516224worker2 # [ 183.617798] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590306225worker2 # [ 183.621039] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:24Z level=info event=worker_gone addr=192.168.1.3:500516226worker2 # [ 183.623432] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590306227worker2 # [ 183.626261] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:24Z level=info event=worker_gone addr=192.168.1.4:500516228worker2 # [ 183.641981] nix-daemon[845]: reaped child process 1621, status = succeeded6229worker1 # [ 183.837259] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:24Z level=info event=scheduler_connected addr=lb:500516230worker1 # [ 183.840179] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6231worker2 # [ 183.823213] nix-grpc-daemon[1567]: ts=2026-09-25T15:02:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6232worker1 # [ 184.040574] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:24Z level=info event=scheduler_connected addr=lb:500516233worker1 # [ 184.043348] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6234worker1 # [ 184.243749] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:24Z level=info event=scheduler_connected addr=lb:500516235worker1 # [ 184.246674] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6236worker1 # [ 184.447083] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:24Z level=info event=scheduler_connected addr=lb:500516237worker1 # [ 184.449964] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6238worker1 # [ 184.651496] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_connected addr=lb:500516239worker1 # [ 184.654079] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6240worker2 # [ 184.630970] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6241worker2 # [ 184.635489] systemd[1]: nix-grpc-daemon.service: Consumed 713ms CPU time over 28.305s wall clock time, 37.8M memory peak, 235.2K incoming IP traffic, 69.1K outgoing IP traffic.6242worker2 # [ 184.644807] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6243worker2 # [ 184.652570] systemd[1]: Starting scheduler-token.service...6244worker2 # [ 184.707557] systemd[1]: scheduler-token.service: Deactivated successfully.6245worker2 # [ 184.712380] systemd[1]: Finished scheduler-token.service.6246worker2 # [ 184.713530] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 58ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6247worker2 # [ 184.726679] systemd[1]: Starting Nix worker-protocol over gRPC...6248worker1 # [ 184.851894] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_connected addr=lb:500516249worker1 # [ 184.854837] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6250worker2 # [ 184.828582] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6251worker2 # [ 184.830380] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6252worker2 # [ 184.834472] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16253worker2 # [ 184.839625] nix-grpc-daemon[1910]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6254worker2 # [ 184.842735] nix-grpc-daemon[1910]: E0000 00:00:1790348545.381388 1910 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6255lb # [ 184.856532] niks3-server[1213]: 2026/09/25 15:02:25 INFO lead: acquired remote=192.168.1.4:423446256worker2 # [ 184.864199] nix-grpc-daemon[1910]: E0000 00:00:1790348545.385121 1910 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6257worker2 # [ 184.866585] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6258worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.67 seconds)6259lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6260lb: must succeed: curl -sf localhost:9901/clusters6261worker2 # [ 184.891496] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=scheduler_take_over6262worker2 # [ 184.893258] nix-grpc-daemon[1910]: E0000 00:00:1790348545.395462 1950 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6263worker2 # [ 184.898813] systemd[1]: Started Nix worker-protocol over gRPC.6264worker2 # [ 184.901485] nix-grpc-daemon[1910]: E0000 00:00:1790348545.416403 1950 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6265worker2 # [ 184.904319] nix-grpc-daemon[1910]: E0000 00:00:1790348545.416586 1950 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6266worker2 # [ 184.906780] nix-grpc-daemon[1910]: E0000 00:00:1790348545.433109 1950 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6267worker2 # [ 184.909351] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=scheduler_connected addr=lb:500516268worker2 # [ 184.910701] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6270worker1 # [ 185.055261] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_connected addr=lb:500516271worker1 # [ 185.058040] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6272worker2 # [ 185.105029] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=scheduler_connected addr=lb:500516273worker2 # [ 185.107858] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6274worker1 # [ 185.258644] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_connected addr=lb:500516275worker1 # [ 185.261393] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6276worker2 # [ 185.308529] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=scheduler_connected addr=lb:500516277worker2 # [ 185.311418] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6278worker1 # [ 185.461969] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_connected addr=lb:500516279worker1 # [ 185.464768] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6280worker2 # [ 185.511979] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:26Z level=info event=scheduler_connected addr=lb:500516281worker2 # [ 185.514730] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6282worker1 # [ 185.665729] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:26Z level=info event=scheduler_connected addr=lb:500516283worker2 # [ 185.640862] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:481606284worker2 # [ 185.643564] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06285worker2 # [ 185.715380] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:26Z level=info event=scheduler_connected addr=lb:500516286worker2 # [ 185.722733] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:26Z level=info event=oidc_jwks provider=mock keys=16287worker2 # [ 185.725307] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:26Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:481606288worker2 # [ 185.728186] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:26Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06289lb: must succeed: curl -sf localhost:9901/clusters6290lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6291lb: must succeed: curl -sf localhost:9901/clusters6292lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6293lb: must succeed: curl -sf localhost:9901/clusters6294lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6295lb # [ 188.273513] envoy[897]: conn peer=[2001:db8:1::1]:41526 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9578 tx=4357 ms=50666296lb # [ 188.276479] envoy[897]: conn peer=[2001:db8:1::1]:37268 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2982 tx=1559 ms=145616297lb # [ 188.278977] envoy[897]: conn peer=[2001:db8:1::4]:41892 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1528 tx=378 ms=271636298lb: must succeed: curl -sf localhost:9901/clusters6299lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6300worker2 # [ 189.362593] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:29Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:481646301lb: must succeed: curl -sf localhost:9901/clusters6302lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6303lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 5.23 seconds)6304lb: must succeed: curl -sf localhost:9901/clusters6305lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6306worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6307worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6308(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 23.58 seconds)6309subtest: queue gauges show a burst while every build blocks6310lb: must succeed: curl -sf localhost:9901/clusters6311lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6312lb: must succeed: curl -sf localhost:9901/clusters6313lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6314worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6315worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6316client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6317client # [ 190.338059] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6318client # Running as unit: burst.service; invocation ID: 0d88e9d808d14c6589a45d2da8429adf6319client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6320lb: must succeed: curl -sf localhost:9901/clusters6321lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6322lb: must succeed: curl -sf localhost:9901/clusters6323lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6324worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6325worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6326worker2 # [ 190.511720] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:481726327worker2 # [ 190.520651] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6328client # [ 190.536556] nix[1502]: don't know how to build these paths:6329client # [ 190.537598] nix[1502]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6330client # [ 190.538685] nix[1502]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6331client # [ 190.539760] nix[1502]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6332client # [ 190.541774] nix[1502]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6333worker2 # [ 190.542383] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:31Z level=debug event=want client=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06334worker1 # [ 190.573426] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666335worker2 # [ 190.550513] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:31Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16336worker1 # [ 190.583195] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666337worker2 # [ 190.556371] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:31Z level=debug event=want client=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06338worker2 # [ 190.559235] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:31Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16339client # [ 190.569565] nix[1502]: copying 1 paths to 'lb:50051'...6340worker2 # [ 190.562688] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:31Z level=debug event=want client=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06341worker1 # [ 190.591676] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666342client # [ 190.570903] nix[1502]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6343worker2 # [ 190.565830] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:31Z level=debug event=want client=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06344worker2 # [ 190.568869] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481726345worker1 # [ 190.600580] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6346worker2 # [ 190.571613] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5)6347worker2 # [ 190.601465] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 144 ms (attempt 2/5)6348worker1 # [ 190.729387] nix-daemon[855]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)6349worker1 # [ 190.747913] nix-daemon[855]: reaped child process 1830, status = succeeded6350lb # [ 190.728962] niks3-server[1213]: 2026/09/25 15:02:31 INFO Received push request method=POST path=/api/pushes6351worker1 # [ 190.758617] nix-grpc-daemon[1671]: time=2026-09-25T15:02:31.289Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6352worker1 # [ 190.760260] nix-grpc-daemon[1671]: time=2026-09-25T15:02:31.290Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6353lb # [ 190.748301] niks3-server[1213]: 2026/09/25 15:02:31 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6354worker2 # [ 190.746678] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 386 ms (attempt 3/5)6355lb # [ 190.757031] niks3-server[1213]: 2026/09/25 15:02:31 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6356lb # [ 190.766193] niks3-server[1213]: 2026/09/25 15:02:31 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6357lb # [ 190.776921] niks3-server[1213]: 2026/09/25 15:02:31 INFO Signed narinfos id=60 count=16358worker1 # [ 190.800752] nix-grpc-daemon[1671]: time=2026-09-25T15:02:31.332Z level=INFO msg="Uploading 1 narinfos"6359lb # [ 190.786791] niks3-server[1213]: 2026/09/25 15:02:31 INFO Received complete push request method=POST path=/api/pushes/60/complete6360lb # [ 190.794659] niks3-server[1213]: 2026/09/25 15:02:31 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6361worker1 # [ 190.815531] nix-grpc-daemon[1671]: time=2026-09-25T15:02:31.347Z level=INFO msg="Upload complete. (204ms)"6362worker1 # [ 190.824585] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=4166363worker1 # [ 190.827802] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666364worker1 # [ 190.838419] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6365worker1 # [ 190.842757] nix-daemon[855]: reaped child process 1825, status = succeeded6366client # [ 190.828843] nix[1502]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6367worker1 # [ 190.852198] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6368worker2 # [ 191.148469] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 362 ms (attempt 4/5)6369lb: must succeed: curl -sf localhost:9901/clusters6370lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6371lb: must succeed: curl -sf localhost:9901/clusters6372worker2 # [ 191.517260] nix-grpc-daemon[1910]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6373worker2 # [ 191.519369] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6374worker2 # [ 191.532603] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:481726375client # [ 191.548898] nix[1502]: copying 1 paths to 'lb:50051'...6376worker2 # [ 191.543918] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:481726377client # [ 191.550690] nix[1502]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6378lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6379worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6380worker2 # [ 191.556592] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6381worker2 # [ 191.591621] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:32Z level=info event=niks3_push_started pid=19846382worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.17 seconds)6383worker2 # [ 191.725136] nix-grpc-daemon[1984]: time=2026-09-25T15:02:32.267Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6384worker2 # [ 191.838334] nix-daemon[845]: accepted connection from pid 1996, user nix-grpc-daemon (trusted)6385worker1 # [ 191.885438] nix-daemon[855]: reaped child process 1834, status = succeeded6386lb # [ 191.873909] niks3-server[1213]: 2026/09/25 15:02:32 INFO Received push request method=POST path=/api/pushes6387worker2 # [ 191.872108] nix-daemon[845]: reaped child process 2003, status = succeeded6388worker2 # [ 191.878969] nix-grpc-daemon[1984]: time=2026-09-25T15:02:32.418Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6389worker2 # [ 191.881567] nix-grpc-daemon[1984]: time=2026-09-25T15:02:32.418Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6390lb # [ 191.899750] niks3-server[1213]: 2026/09/25 15:02:32 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6391worker2 # [ 191.901189] nix-grpc-daemon[1984]: time=2026-09-25T15:02:32.443Z level=INFO msg="Uploading 1 narinfos"6392lb # [ 191.908638] niks3-server[1213]: 2026/09/25 15:02:32 INFO Signed narinfos id=61 count=16393lb # [ 191.914554] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6394lb # [ 191.924714] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6395lb # [ 191.932339] niks3-server[1213]: 2026/09/25 15:02:32 INFO Received complete push request method=POST path=/api/pushes/61/complete6396worker2 # [ 191.934168] nix-grpc-daemon[1984]: time=2026-09-25T15:02:32.476Z level=INFO msg="Upload complete. (205ms)"6397lb # [ 191.939306] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6398worker2 # [ 191.940083] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=0 paths=1 nar_bytes_in=4166399worker2 # [ 191.949479] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481726400worker2 # [ 191.953159] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6401worker1 # [ 191.982195] nix-daemon[855]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)6402worker2 # [ 191.956650] nix-daemon[845]: reaped child process 1981, status = succeeded6403client # [ 191.968497] nix[1502]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6404worker2 # [ 191.965694] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6405lb # [ 191.977505] niks3-server[1213]: 2026/09/25 15:02:32 INFO Received push request method=POST path=/api/pushes6406worker1 # [ 192.002888] nix-daemon[855]: reaped child process 1858, status = succeeded6407worker1 # [ 192.008309] nix-grpc-daemon[1671]: time=2026-09-25T15:02:32.537Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6408worker1 # [ 192.009817] nix-grpc-daemon[1671]: time=2026-09-25T15:02:32.537Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6409lb # [ 191.998847] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6410worker1 # [ 192.031624] nix-grpc-daemon[1671]: time=2026-09-25T15:02:32.563Z level=INFO msg="Uploading 1 narinfos"6411lb # [ 192.007904] niks3-server[1213]: 2026/09/25 15:02:32 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6412lb # [ 192.014450] niks3-server[1213]: 2026/09/25 15:02:32 INFO Signed narinfos id=62 count=16413lb # [ 192.019013] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6414lb # [ 192.026366] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6415lb # [ 192.034975] niks3-server[1213]: 2026/09/25 15:02:32 INFO Received complete push request method=POST path=/api/pushes/62/complete6416worker1 # [ 192.061645] nix-grpc-daemon[1671]: time=2026-09-25T15:02:32.593Z level=INFO msg="Upload complete. (199ms)"6417lb # [ 192.040366] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6418worker2 # [ 192.042103] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:32Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.3:50051 clients=16419worker1 # [ 192.069015] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=1 outputs=16420worker1 # [ 192.081581] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666421worker1 # [ 192.085335] nix-daemon[855]: reaped child process 1832, status = succeeded6422worker1 # [ 192.089909] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666423client # [ 192.075432] nix[1502]: copying 1 paths to 'lb:50051'...6424worker1 # [ 192.097365] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666425client # [ 192.076940] nix[1502]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6426worker1 # [ 192.104132] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6427worker1 # [ 192.218343] nix-daemon[855]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6428lb # [ 192.212632] niks3-server[1213]: 2026/09/25 15:02:32 INFO Received push request method=POST path=/api/pushes6429worker1 # [ 192.238943] nix-daemon[855]: reaped child process 1865, status = succeeded6430worker1 # [ 192.241302] nix-grpc-daemon[1671]: time=2026-09-25T15:02:32.772Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6431worker1 # [ 192.242849] nix-grpc-daemon[1671]: time=2026-09-25T15:02:32.773Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6432lb # [ 192.227886] niks3-server[1213]: 2026/09/25 15:02:32 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6433worker1 # [ 192.256319] nix-grpc-daemon[1671]: time=2026-09-25T15:02:32.788Z level=INFO msg="Uploading 1 narinfos"6434lb # [ 192.235828] niks3-server[1213]: 2026/09/25 15:02:32 INFO Signed narinfos id=63 count=16435lb # [ 192.240827] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6436lb # [ 192.250426] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6437worker1 # [ 192.277085] nix-grpc-daemon[1671]: time=2026-09-25T15:02:32.809Z level=INFO msg="Upload complete. (164ms)"6438lb # [ 192.254591] niks3-server[1213]: 2026/09/25 15:02:32 INFO Received complete push request method=POST path=/api/pushes/63/complete6439worker1 # [ 192.281867] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=4166440lb # [ 192.260284] niks3-server[1213]: 2026/09/25 15:02:32 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6441worker1 # [ 192.288452] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666442worker1 # [ 192.296090] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6443worker1 # [ 192.299387] nix-daemon[855]: reaped child process 1860, status = succeeded6444client # [ 192.284269] nix[1502]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6445worker1 # [ 192.308774] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6446worker2 # [ 194.191872] nix-daemon[845]: reaped child process 2007, status = succeeded6447worker2 # [ 194.278845] nix-daemon[845]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)6448worker2 # [ 194.297613] nix-daemon[845]: reaped child process 2035, status = succeeded6449lb # [ 194.307208] niks3-server[1213]: 2026/09/25 15:02:34 INFO Received push request method=POST path=/api/pushes6450worker2 # [ 194.312384] nix-grpc-daemon[1984]: time=2026-09-25T15:02:34.852Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6451worker2 # [ 194.313936] nix-grpc-daemon[1984]: time=2026-09-25T15:02:34.852Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6452lb # [ 194.329574] niks3-server[1213]: 2026/09/25 15:02:34 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6453worker1 # [ 194.364892] nix-daemon[855]: reaped child process 1869, status = succeeded6454lb # [ 194.340897] niks3-server[1213]: 2026/09/25 15:02:34 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6455worker2 # [ 194.341424] nix-grpc-daemon[1984]: time=2026-09-25T15:02:34.883Z level=INFO msg="Uploading 1 narinfos"6456lb # [ 194.347757] niks3-server[1213]: 2026/09/25 15:02:34 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6457lb # [ 194.353547] niks3-server[1213]: 2026/09/25 15:02:34 INFO Signed narinfos id=64 count=16458lb # [ 194.355273] niks3-server[1213]: 2026/09/25 15:02:34 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6459lb # [ 194.364145] niks3-server[1213]: 2026/09/25 15:02:34 INFO Received complete push request method=POST path=/api/pushes/64/complete6460lb # [ 194.371572] niks3-server[1213]: 2026/09/25 15:02:34 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6461worker2 # [ 194.372596] nix-grpc-daemon[1984]: time=2026-09-25T15:02:34.915Z level=INFO msg="Upload complete. (197ms)"6462worker2 # [ 194.384599] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=2 outputs=16463worker2 # [ 194.389728] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:34Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.4:50051 clients=16464worker2 # [ 194.398812] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481726465worker2 # [ 194.402850] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:481726466worker2 # [ 194.406677] nix-daemon[845]: reaped child process 2005, status = succeeded6467client # [ 194.414921] nix[1502]: copying 1 paths to 'lb:50051'...6468worker2 # [ 194.409248] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:481726469client # [ 194.415076] nix[1502]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6470worker2 # [ 194.418536] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6471worker1 # [ 194.462168] nix-daemon[855]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6472lb # [ 194.455678] niks3-server[1213]: 2026/09/25 15:02:34 INFO Received push request method=POST path=/api/pushes6473worker1 # [ 194.481750] nix-daemon[855]: reaped child process 1897, status = succeeded6474worker1 # [ 194.484406] nix-grpc-daemon[1671]: time=2026-09-25T15:02:35.015Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6475worker1 # [ 194.485895] nix-grpc-daemon[1671]: time=2026-09-25T15:02:35.016Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6476lb # [ 194.483669] niks3-server[1213]: 2026/09/25 15:02:35 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6477worker1 # [ 194.511595] nix-grpc-daemon[1671]: time=2026-09-25T15:02:35.043Z level=INFO msg="Uploading 1 narinfos"6478lb # [ 194.492560] niks3-server[1213]: 2026/09/25 15:02:35 INFO Signed narinfos id=65 count=16479lb # [ 194.496734] niks3-server[1213]: 2026/09/25 15:02:35 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6480lb # [ 194.502897] niks3-server[1213]: 2026/09/25 15:02:35 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6481lb # [ 194.510156] niks3-server[1213]: 2026/09/25 15:02:35 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6482lb # [ 194.515675] niks3-server[1213]: 2026/09/25 15:02:35 INFO Received complete push request method=POST path=/api/pushes/65/complete6483worker1 # [ 194.543364] nix-grpc-daemon[1671]: time=2026-09-25T15:02:35.075Z level=INFO msg="Upload complete. (226ms)"6484lb # [ 194.520656] niks3-server[1213]: 2026/09/25 15:02:35 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6485worker2 # [ 194.527491] nix-daemon[845]: accepted connection from pid 2039, user nix-grpc-daemon (trusted)6486worker1 # [ 194.552642] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=3 outputs=16487worker1 # [ 194.560941] nix-daemon[855]: reaped child process 1867, status = succeeded6488lb # [ 194.549602] niks3-server[1213]: 2026/09/25 15:02:35 INFO Received push request method=POST path=/api/pushes6489worker2 # [ 194.548859] nix-daemon[845]: reaped child process 2042, status = succeeded6490worker2 # [ 194.550538] nix-grpc-daemon[1984]: time=2026-09-25T15:02:35.092Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6491worker2 # [ 194.552256] nix-grpc-daemon[1984]: time=2026-09-25T15:02:35.092Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6492lb # [ 194.564848] niks3-server[1213]: 2026/09/25 15:02:35 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6493worker2 # [ 194.568390] nix-grpc-daemon[1984]: time=2026-09-25T15:02:35.110Z level=INFO msg="Uploading 1 narinfos"6494lb # [ 194.572195] niks3-server[1213]: 2026/09/25 15:02:35 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6495lb # [ 194.575707] niks3-server[1213]: 2026/09/25 15:02:35 INFO Signed narinfos id=66 count=16496lb # [ 194.579442] niks3-server[1213]: 2026/09/25 15:02:35 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6497lb # [ 194.589910] niks3-server[1213]: 2026/09/25 15:02:35 INFO Received complete push request method=POST path=/api/pushes/66/complete6498worker2 # [ 194.590829] nix-grpc-daemon[1984]: time=2026-09-25T15:02:35.133Z level=INFO msg="Upload complete. (165ms)"6499lb # [ 194.595571] niks3-server[1213]: 2026/09/25 15:02:35 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6500worker2 # [ 194.596176] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=0 paths=1 nar_bytes_in=4166501worker2 # [ 194.605374] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481726502worker2 # [ 194.609649] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6503worker2 # [ 194.613575] nix-daemon[845]: reaped child process 2037, status = succeeded6504client # [ 194.625787] nix[1502]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6505worker2 # [ 194.623104] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6506worker2 # [ 196.470073] nix-daemon[845]: reaped child process 2046, status = succeeded6507worker2 # [ 196.535867] nix-daemon[845]: accepted connection from pid 2071, user nix-grpc-daemon (trusted)6508worker2 # [ 196.551249] nix-daemon[845]: reaped child process 2074, status = succeeded6509lb # [ 196.561103] niks3-server[1213]: 2026/09/25 15:02:37 INFO Received push request method=POST path=/api/pushes6510worker2 # [ 196.563905] nix-grpc-daemon[1984]: time=2026-09-25T15:02:37.105Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6511worker2 # [ 196.565631] nix-grpc-daemon[1984]: time=2026-09-25T15:02:37.105Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6512lb # [ 196.587159] niks3-server[1213]: 2026/09/25 15:02:37 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6513worker2 # [ 196.590805] nix-grpc-daemon[1984]: time=2026-09-25T15:02:37.133Z level=INFO msg="Uploading 1 narinfos"6514lb # [ 196.596315] niks3-server[1213]: 2026/09/25 15:02:37 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6515lb # [ 196.598590] niks3-server[1213]: 2026/09/25 15:02:37 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6516lb # [ 196.607462] niks3-server[1213]: 2026/09/25 15:02:37 INFO Signed narinfos id=67 count=16517lb # [ 196.614927] niks3-server[1213]: 2026/09/25 15:02:37 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6518worker2 # [ 196.614317] nix-grpc-daemon[1984]: time=2026-09-25T15:02:37.156Z level=INFO msg="Upload complete. (183ms)"6519lb # [ 196.619190] niks3-server[1213]: 2026/09/25 15:02:37 INFO Received complete push request method=POST path=/api/pushes/67/complete6520lb # [ 196.623865] niks3-server[1213]: 2026/09/25 15:02:37 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6521worker2 # [ 196.624626] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=4 outputs=16522worker2 # [ 196.632584] nix-daemon[845]: reaped child process 2044, status = succeeded6523client # [ 196.652257] systemd[1]: burst.service: Deactivated successfully.6524client # [ 196.654544] systemd[1]: burst.service: Consumed 191ms CPU time over 6.316s wall clock time, 53.6M memory peak, 14K incoming IP traffic, 19.2K outgoing IP traffic.6525client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6526client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)6527lb: must succeed: curl -sf localhost:9901/clusters6528lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6529lb: must succeed: curl -sf localhost:9901/clusters6530lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6531worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6532worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6533(finished: subtest: queue gauges show a burst while every build blocks, in 7.51 seconds)6534subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6535client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16536worker2 # [ 197.873293] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:481726537worker1 # [ 197.904006] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666538worker2 # [ 197.901034] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:38Z level=debug event=want client=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06539worker2 # [ 197.903903] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:38Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16540worker2 # [ 197.907494] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:38Z level=debug event=want client=2 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06541worker1 # [ 197.937150] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666542worker2 # [ 197.910629] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:38Z level=debug event=want client=2 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06543worker1 # [ 197.945768] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666544worker1 # [ 197.953917] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666545worker1 # [ 197.963035] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6546worker1 # [ 198.094928] nix-daemon[855]: accepted connection from pid 1913, user nix-grpc-daemon (trusted)6547worker1 # [ 198.113061] nix-daemon[855]: reaped child process 1916, status = succeeded6548lb # [ 198.092753] niks3-server[1213]: 2026/09/25 15:02:38 INFO Received push request method=POST path=/api/pushes6549worker1 # [ 198.121968] nix-grpc-daemon[1671]: time=2026-09-25T15:02:38.653Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6550worker1 # [ 198.123700] nix-grpc-daemon[1671]: time=2026-09-25T15:02:38.653Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6551lb # [ 198.108108] niks3-server[1213]: 2026/09/25 15:02:38 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6552worker1 # [ 198.141199] nix-grpc-daemon[1671]: time=2026-09-25T15:02:38.673Z level=INFO msg="Uploading 1 narinfos"6553lb # [ 198.114299] niks3-server[1213]: 2026/09/25 15:02:38 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6554lb # [ 198.120514] niks3-server[1213]: 2026/09/25 15:02:38 INFO Signed narinfos id=68 count=16555lb # [ 198.124692] niks3-server[1213]: 2026/09/25 15:02:38 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6556lb # [ 198.132464] niks3-server[1213]: 2026/09/25 15:02:38 INFO Received complete push request method=POST path=/api/pushes/68/complete6557worker1 # [ 198.163570] nix-grpc-daemon[1671]: time=2026-09-25T15:02:38.695Z level=INFO msg="Upload complete. (190ms)"6558lb # [ 198.140095] niks3-server[1213]: 2026/09/25 15:02:38 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6559worker1 # [ 198.169598] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=5126560worker1 # [ 198.176719] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666561worker1 # [ 198.185061] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6562worker1 # [ 198.189020] nix-daemon[855]: reaped child process 1911, status = succeeded6563worker1 # [ 198.199254] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6564lb # [ 198.274092] envoy[897]: conn peer=[2001:db8:1::1]:44104 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9714 tx=5240 ms=61256565worker1 # [ 199.652630] nix-daemon[855]: reaped child process 1920, status = succeeded6566worker1 # [ 199.742095] nix-daemon[855]: accepted connection from pid 1936, user nix-grpc-daemon (trusted)6567worker1 # [ 199.758007] nix-daemon[855]: reaped child process 1939, status = succeeded6568lb # [ 199.741279] niks3-server[1213]: 2026/09/25 15:02:40 INFO Received push request method=POST path=/api/pushes6569worker1 # [ 199.772379] nix-grpc-daemon[1671]: time=2026-09-25T15:02:40.303Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6570worker1 # [ 199.773930] nix-grpc-daemon[1671]: time=2026-09-25T15:02:40.303Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6571lb # [ 199.770551] niks3-server[1213]: 2026/09/25 15:02:40 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6572worker1 # [ 199.800654] nix-grpc-daemon[1671]: time=2026-09-25T15:02:40.332Z level=INFO msg="Uploading 1 narinfos"6573lb # [ 199.779549] niks3-server[1213]: 2026/09/25 15:02:40 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6574lb # [ 199.785448] niks3-server[1213]: 2026/09/25 15:02:40 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6575lb # [ 199.787805] niks3-server[1213]: 2026/09/25 15:02:40 INFO Signed narinfos id=69 count=16576lb # [ 199.797282] niks3-server[1213]: 2026/09/25 15:02:40 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6577worker1 # [ 199.826218] nix-grpc-daemon[1671]: time=2026-09-25T15:02:40.358Z level=INFO msg="Upload complete. (218ms)"6578lb # [ 199.803991] niks3-server[1213]: 2026/09/25 15:02:40 INFO Received complete push request method=POST path=/api/pushes/69/complete6579lb # [ 199.807474] niks3-server[1213]: 2026/09/25 15:02:40 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6580worker2 # [ 199.804775] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:40Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16581worker1 # [ 199.832647] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=5 outputs=16582worker2 # [ 199.812081] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:40Z level=debug event=want client=2 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06583worker1 # [ 199.844948] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666584worker1 # [ 199.848831] nix-daemon[855]: reaped child process 1918, status = succeeded6585worker1 # [ 199.853801] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666586worker1 # [ 199.861742] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666587worker1 # [ 199.869559] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6588worker1 # [ 199.989453] nix-daemon[855]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)6589lb # [ 199.984666] niks3-server[1213]: 2026/09/25 15:02:40 INFO Received push request method=POST path=/api/pushes6590worker1 # [ 200.010645] nix-daemon[855]: reaped child process 1946, status = succeeded6591worker1 # [ 200.013305] nix-grpc-daemon[1671]: time=2026-09-25T15:02:40.545Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6592worker1 # [ 200.014977] nix-grpc-daemon[1671]: time=2026-09-25T15:02:40.545Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6593lb # [ 200.000633] niks3-server[1213]: 2026/09/25 15:02:40 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6594worker1 # [ 200.029274] nix-grpc-daemon[1671]: time=2026-09-25T15:02:40.561Z level=INFO msg="Uploading 1 narinfos"6595lb # [ 200.008603] niks3-server[1213]: 2026/09/25 15:02:40 INFO Signed narinfos id=70 count=16596lb # [ 200.010363] niks3-server[1213]: 2026/09/25 15:02:40 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6597lb # [ 200.020381] niks3-server[1213]: 2026/09/25 15:02:40 INFO Received complete push request method=POST path=/api/pushes/70/complete6598worker1 # [ 200.050027] nix-grpc-daemon[1671]: time=2026-09-25T15:02:40.582Z level=INFO msg="Upload complete. (171ms)"6599lb # [ 200.026981] niks3-server[1213]: 2026/09/25 15:02:40 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6600worker1 # [ 200.054940] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=5126601lb # [ 200.031898] niks3-server[1213]: 2026/09/25 15:02:40 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6602worker1 # [ 200.063673] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666603worker1 # [ 200.067432] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6604worker1 # [ 200.070811] nix-daemon[855]: reaped child process 1941, status = succeeded6605worker1 # [ 200.078626] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6606worker1 # [ 201.153658] nix-daemon[855]: reaped child process 1950, status = succeeded6607worker1 # [ 201.218095] nix-daemon[855]: accepted connection from pid 1966, user nix-grpc-daemon (trusted)6608worker1 # [ 201.234231] nix-daemon[855]: reaped child process 1969, status = succeeded6609lb # [ 201.217068] niks3-server[1213]: 2026/09/25 15:02:41 INFO Received push request method=POST path=/api/pushes6610worker1 # [ 201.248383] nix-grpc-daemon[1671]: time=2026-09-25T15:02:41.777Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6611worker1 # [ 201.249912] nix-grpc-daemon[1671]: time=2026-09-25T15:02:41.777Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6612lb # [ 201.242124] niks3-server[1213]: 2026/09/25 15:02:41 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6613worker1 # [ 201.270197] nix-grpc-daemon[1671]: time=2026-09-25T15:02:41.802Z level=INFO msg="Uploading 1 narinfos"6614lb # [ 201.248288] niks3-server[1213]: 2026/09/25 15:02:41 INFO Signed narinfos id=71 count=16615lb # [ 201.254406] niks3-server[1213]: 2026/09/25 15:02:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6616lb # [ 201.260976] niks3-server[1213]: 2026/09/25 15:02:41 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6617lb # [ 201.271247] niks3-server[1213]: 2026/09/25 15:02:41 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6618worker1 # [ 201.301270] nix-grpc-daemon[1671]: time=2026-09-25T15:02:41.833Z level=INFO msg="Upload complete. (214ms)"6619lb # [ 201.278004] niks3-server[1213]: 2026/09/25 15:02:41 INFO Received complete push request method=POST path=/api/pushes/71/complete6620lb # [ 201.280846] niks3-server[1213]: 2026/09/25 15:02:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6621worker1 # [ 201.308555] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=6 outputs=16622worker2 # [ 201.282712] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:41Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16623worker2 # [ 201.289852] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:41Z level=debug event=want client=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06624worker1 # [ 201.320250] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666625worker1 # [ 201.323939] nix-daemon[855]: reaped child process 1948, status = succeeded6626worker1 # [ 201.331254] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666627worker1 # [ 201.339999] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666628worker1 # [ 201.347618] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6629worker1 # [ 201.468824] nix-daemon[855]: accepted connection from pid 1973, user nix-grpc-daemon (trusted)6630worker1 # [ 201.485456] nix-daemon[855]: reaped child process 1976, status = succeeded6631lb # [ 201.464945] niks3-server[1213]: 2026/09/25 15:02:41 INFO Received push request method=POST path=/api/pushes6632worker1 # [ 201.493510] nix-grpc-daemon[1671]: time=2026-09-25T15:02:42.025Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6633worker1 # [ 201.495075] nix-grpc-daemon[1671]: time=2026-09-25T15:02:42.025Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6634worker1 # [ 201.510635] nix-grpc-daemon[1671]: time=2026-09-25T15:02:42.042Z level=INFO msg="Uploading 1 narinfos"6635lb # [ 201.482225] niks3-server[1213]: 2026/09/25 15:02:42 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6636lb # [ 201.489156] niks3-server[1213]: 2026/09/25 15:02:42 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6637lb # [ 201.491481] niks3-server[1213]: 2026/09/25 15:02:42 INFO Signed narinfos id=72 count=16638lb # [ 201.498977] niks3-server[1213]: 2026/09/25 15:02:42 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6639lb # [ 201.509713] niks3-server[1213]: 2026/09/25 15:02:42 INFO Received complete push request method=POST path=/api/pushes/72/complete6640worker1 # [ 201.536764] nix-grpc-daemon[1671]: time=2026-09-25T15:02:42.068Z level=INFO msg="Upload complete. (179ms)"6641lb # [ 201.514099] niks3-server[1213]: 2026/09/25 15:02:42 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6642worker1 # [ 201.542736] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=5126643worker1 # [ 201.551779] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666644worker1 # [ 201.557832] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6645worker1 # [ 201.560227] nix-daemon[855]: reaped child process 1971, status = succeeded6646worker1 # [ 201.570136] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6647worker1 # [ 202.212287] nix-daemon[855]: reaped child process 1980, status = succeeded6648worker1 # [ 202.297589] nix-daemon[855]: accepted connection from pid 1996, user nix-grpc-daemon (trusted)6649worker1 # [ 202.317451] nix-daemon[855]: reaped child process 1999, status = succeeded6650lb # [ 202.293087] niks3-server[1213]: 2026/09/25 15:02:42 INFO Received push request method=POST path=/api/pushes6651worker1 # [ 202.323152] nix-grpc-daemon[1671]: time=2026-09-25T15:02:42.854Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6652worker1 # [ 202.324884] nix-grpc-daemon[1671]: time=2026-09-25T15:02:42.854Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6653lb # [ 202.315524] niks3-server[1213]: 2026/09/25 15:02:42 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6654worker1 # [ 202.345437] nix-grpc-daemon[1671]: time=2026-09-25T15:02:42.877Z level=INFO msg="Uploading 1 narinfos"6655lb # [ 202.322322] niks3-server[1213]: 2026/09/25 15:02:42 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign6656lb # [ 202.328498] niks3-server[1213]: 2026/09/25 15:02:42 INFO Signed narinfos id=73 count=16657lb # [ 202.332980] niks3-server[1213]: 2026/09/25 15:02:42 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6658lb # [ 202.340264] niks3-server[1213]: 2026/09/25 15:02:42 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6659worker1 # [ 202.372803] nix-grpc-daemon[1671]: time=2026-09-25T15:02:42.904Z level=INFO msg="Upload complete. (195ms)"6660lb # [ 202.349741] niks3-server[1213]: 2026/09/25 15:02:42 INFO Received complete push request method=POST path=/api/pushes/73/complete6661lb # [ 202.356605] niks3-server[1213]: 2026/09/25 15:02:42 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6662worker2 # [ 202.352557] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:42Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16663worker1 # [ 202.379443] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=7 outputs=16664worker2 # [ 202.358032] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:42Z level=debug event=want client=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06665worker1 # [ 202.391737] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666666worker1 # [ 202.395932] nix-daemon[855]: reaped child process 1978, status = succeeded6667worker1 # [ 202.401143] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666668worker1 # [ 202.409419] nix-grpc-daemon[1671]: time=2026-09-25T15:02:42.941Z level=INFO msg="All 1 paths already cached"6669worker1 # [ 202.414752] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666670worker1 # [ 202.422307] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6671worker1 # [ 202.533554] nix-daemon[855]: accepted connection from pid 2003, user nix-grpc-daemon (trusted)6672worker1 # [ 202.551731] nix-daemon[855]: reaped child process 2006, status = succeeded6673lb # [ 202.532512] niks3-server[1213]: 2026/09/25 15:02:43 INFO Received push request method=POST path=/api/pushes6674worker1 # [ 202.561002] nix-grpc-daemon[1671]: time=2026-09-25T15:02:43.092Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6675worker1 # [ 202.562564] nix-grpc-daemon[1671]: time=2026-09-25T15:02:43.092Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6676lb # [ 202.547523] niks3-server[1213]: 2026/09/25 15:02:43 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6677worker1 # [ 202.578250] nix-grpc-daemon[1671]: time=2026-09-25T15:02:43.110Z level=INFO msg="Uploading 1 narinfos"6678lb # [ 202.554603] niks3-server[1213]: 2026/09/25 15:02:43 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign6679lb # [ 202.559669] niks3-server[1213]: 2026/09/25 15:02:43 INFO Signed narinfos id=74 count=16680lb # [ 202.568646] niks3-server[1213]: 2026/09/25 15:02:43 INFO Received complete push request method=POST path=/api/pushes/74/complete6681lb # [ 202.577561] niks3-server[1213]: 2026/09/25 15:02:43 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6682worker1 # [ 202.605561] nix-grpc-daemon[1671]: time=2026-09-25T15:02:43.137Z level=INFO msg="Upload complete. (174ms)"6683lb # [ 202.582383] niks3-server[1213]: 2026/09/25 15:02:43 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6684worker1 # [ 202.611331] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=6246685worker1 # [ 202.621131] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666686worker1 # [ 202.626119] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6687worker1 # [ 202.630605] nix-daemon[855]: reaped child process 2001, status = succeeded6688worker1 # [ 202.639960] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6689worker1 # [ 203.497042] nix-daemon[855]: reaped child process 2010, status = succeeded6690worker1 # [ 203.566190] nix-daemon[855]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)6691worker1 # [ 203.581661] nix-daemon[855]: reaped child process 2029, status = succeeded6692lb # [ 203.563704] niks3-server[1213]: 2026/09/25 15:02:44 INFO Received push request method=POST path=/api/pushes6693worker1 # [ 203.593944] nix-grpc-daemon[1671]: time=2026-09-25T15:02:44.124Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6694worker1 # [ 203.595531] nix-grpc-daemon[1671]: time=2026-09-25T15:02:44.124Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6695lb # [ 203.587957] niks3-server[1213]: 2026/09/25 15:02:44 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign6696worker1 # [ 203.617241] nix-grpc-daemon[1671]: time=2026-09-25T15:02:44.149Z level=INFO msg="Uploading 1 narinfos"6697lb # [ 203.596185] niks3-server[1213]: 2026/09/25 15:02:44 INFO Signed narinfos id=75 count=16698lb # [ 203.597904] niks3-server[1213]: 2026/09/25 15:02:44 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6699lb # [ 203.607230] niks3-server[1213]: 2026/09/25 15:02:44 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6700lb # [ 203.612954] niks3-server[1213]: 2026/09/25 15:02:44 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6701worker1 # [ 203.644965] nix-grpc-daemon[1671]: time=2026-09-25T15:02:44.176Z level=INFO msg="Upload complete. (197ms)"6702lb # [ 203.621738] niks3-server[1213]: 2026/09/25 15:02:44 INFO Received complete push request method=POST path=/api/pushes/75/complete6703worker2 # [ 203.623296] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:44Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16704lb # [ 203.629384] niks3-server[1213]: 2026/09/25 15:02:44 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6705worker1 # [ 203.653311] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=8 outputs=16706worker2 # [ 203.630825] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:44Z level=debug event=want client=2 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06707worker1 # [ 203.663549] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666708worker1 # [ 203.667615] nix-daemon[855]: reaped child process 2008, status = succeeded6709worker1 # [ 203.672943] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666710worker1 # [ 203.681290] nix-grpc-daemon[1671]: time=2026-09-25T15:02:44.213Z level=INFO msg="All 1 paths already cached"6711worker1 # [ 203.687011] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666712worker1 # [ 203.694721] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6713worker1 # [ 203.809718] nix-daemon[855]: accepted connection from pid 2033, user nix-grpc-daemon (trusted)6714worker1 # [ 203.828541] nix-daemon[855]: reaped child process 2036, status = succeeded6715lb # [ 203.808442] niks3-server[1213]: 2026/09/25 15:02:44 INFO Received push request method=POST path=/api/pushes6716worker1 # [ 203.836899] nix-grpc-daemon[1671]: time=2026-09-25T15:02:44.368Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6717worker1 # [ 203.838477] nix-grpc-daemon[1671]: time=2026-09-25T15:02:44.368Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6718lb # [ 203.823071] niks3-server[1213]: 2026/09/25 15:02:44 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign6719worker1 # [ 203.849441] nix-grpc-daemon[1671]: time=2026-09-25T15:02:44.381Z level=INFO msg="Uploading 1 narinfos"6720lb # [ 203.828641] niks3-server[1213]: 2026/09/25 15:02:44 INFO Signed narinfos id=76 count=16721lb # [ 203.834258] niks3-server[1213]: 2026/09/25 15:02:44 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6722lb # [ 203.840708] niks3-server[1213]: 2026/09/25 15:02:44 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6723lb # [ 203.848863] niks3-server[1213]: 2026/09/25 15:02:44 INFO Received complete push request method=POST path=/api/pushes/76/complete6724worker1 # [ 203.877189] nix-grpc-daemon[1671]: time=2026-09-25T15:02:44.409Z level=INFO msg="Upload complete. (173ms)"6725lb # [ 203.854615] niks3-server[1213]: 2026/09/25 15:02:44 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6726worker1 # [ 203.883479] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=6246727worker1 # [ 203.892866] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666728worker1 # [ 203.897214] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6729worker1 # [ 203.901127] nix-daemon[855]: reaped child process 2031, status = succeeded6730worker1 # [ 203.910669] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6731worker1 # [ 204.943619] nix-daemon[855]: reaped child process 2040, status = succeeded6732worker1 # [ 205.053019] nix-daemon[855]: accepted connection from pid 2056, user nix-grpc-daemon (trusted)6733worker1 # [ 205.067884] nix-daemon[855]: reaped child process 2059, status = succeeded6734lb # [ 205.050243] niks3-server[1213]: 2026/09/25 15:02:45 INFO Received push request method=POST path=/api/pushes6735worker1 # [ 205.080436] nix-grpc-daemon[1671]: time=2026-09-25T15:02:45.611Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6736worker1 # [ 205.082427] nix-grpc-daemon[1671]: time=2026-09-25T15:02:45.611Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6737lb # [ 205.073536] niks3-server[1213]: 2026/09/25 15:02:45 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign6738worker1 # [ 205.101434] nix-grpc-daemon[1671]: time=2026-09-25T15:02:45.633Z level=INFO msg="Uploading 1 narinfos"6739lb # [ 205.080505] niks3-server[1213]: 2026/09/25 15:02:45 INFO Signed narinfos id=77 count=16740lb # [ 205.085590] niks3-server[1213]: 2026/09/25 15:02:45 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6741lb # [ 205.092200] niks3-server[1213]: 2026/09/25 15:02:45 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6742lb # [ 205.102108] niks3-server[1213]: 2026/09/25 15:02:45 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6743worker1 # [ 205.130551] nix-grpc-daemon[1671]: time=2026-09-25T15:02:45.662Z level=INFO msg="Upload complete. (213ms)"6744lb # [ 205.107753] niks3-server[1213]: 2026/09/25 15:02:45 INFO Received complete push request method=POST path=/api/pushes/77/complete6745lb # [ 205.111438] niks3-server[1213]: 2026/09/25 15:02:45 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6746worker2 # [ 205.108729] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:45Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16747worker1 # [ 205.140263] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=9 outputs=16748worker2 # [ 205.117029] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:45Z level=debug event=want client=2 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06749worker1 # [ 205.149612] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666750worker1 # [ 205.153598] nix-daemon[855]: reaped child process 2038, status = succeeded6751worker1 # [ 205.159052] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666752worker1 # [ 205.166880] nix-grpc-daemon[1671]: time=2026-09-25T15:02:45.698Z level=INFO msg="All 1 paths already cached"6753worker1 # [ 205.172589] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666754worker1 # [ 205.179923] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6755worker1 # [ 205.297919] nix-daemon[855]: accepted connection from pid 2063, user nix-grpc-daemon (trusted)6756lb # [ 205.290389] niks3-server[1213]: 2026/09/25 15:02:45 INFO Received push request method=POST path=/api/pushes6757worker1 # [ 205.317643] nix-daemon[855]: reaped child process 2066, status = succeeded6758worker1 # [ 205.319537] nix-grpc-daemon[1671]: time=2026-09-25T15:02:45.851Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6759worker1 # [ 205.321121] nix-grpc-daemon[1671]: time=2026-09-25T15:02:45.851Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6760lb # [ 205.305867] niks3-server[1213]: 2026/09/25 15:02:45 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign6761worker1 # [ 205.333177] nix-grpc-daemon[1671]: time=2026-09-25T15:02:45.865Z level=INFO msg="Uploading 1 narinfos"6762lb # [ 205.312946] niks3-server[1213]: 2026/09/25 15:02:45 INFO Signed narinfos id=78 count=16763lb # [ 205.314670] niks3-server[1213]: 2026/09/25 15:02:45 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6764lb # [ 205.324461] niks3-server[1213]: 2026/09/25 15:02:45 INFO Received complete push request method=POST path=/api/pushes/78/complete6765lb # [ 205.329179] niks3-server[1213]: 2026/09/25 15:02:45 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6766worker1 # [ 205.356955] nix-grpc-daemon[1671]: time=2026-09-25T15:02:45.888Z level=INFO msg="Upload complete. (168ms)"6767lb # [ 205.334151] niks3-server[1213]: 2026/09/25 15:02:45 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6768worker1 # [ 205.362997] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=6246769worker1 # [ 205.370410] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666770worker1 # [ 205.375058] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6771worker1 # [ 205.378534] nix-daemon[855]: reaped child process 2061, status = succeeded6772worker1 # [ 205.387838] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6773worker1 # [ 205.845856] nix-daemon[855]: reaped child process 2070, status = succeeded6774worker1 # [ 205.904983] nix-daemon[855]: accepted connection from pid 2086, user nix-grpc-daemon (trusted)6775worker1 # [ 205.920907] nix-daemon[855]: reaped child process 2089, status = succeeded6776lb # [ 205.901430] niks3-server[1213]: 2026/09/25 15:02:46 INFO Received push request method=POST path=/api/pushes6777worker1 # [ 205.932147] nix-grpc-daemon[1671]: time=2026-09-25T15:02:46.461Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6778worker1 # [ 205.933678] nix-grpc-daemon[1671]: time=2026-09-25T15:02:46.461Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6779lb # [ 205.928839] niks3-server[1213]: 2026/09/25 15:02:46 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign6780worker1 # [ 205.957605] nix-grpc-daemon[1671]: time=2026-09-25T15:02:46.489Z level=INFO msg="Uploading 1 narinfos"6781lb # [ 205.937934] niks3-server[1213]: 2026/09/25 15:02:46 INFO Signed narinfos id=79 count=16782lb # [ 205.942132] niks3-server[1213]: 2026/09/25 15:02:46 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6783lb # [ 205.951167] niks3-server[1213]: 2026/09/25 15:02:46 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6784worker1 # [ 205.978980] nix-grpc-daemon[1671]: time=2026-09-25T15:02:46.511Z level=INFO msg="Upload complete. (185ms)"6785lb # [ 205.957581] niks3-server[1213]: 2026/09/25 15:02:46 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6786lb # [ 205.961890] niks3-server[1213]: 2026/09/25 15:02:46 INFO Received complete push request method=POST path=/api/pushes/79/complete6787worker2 # [ 205.958212] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:46Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16788lb # [ 205.966440] niks3-server[1213]: 2026/09/25 15:02:46 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6789worker1 # [ 205.984303] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=10 outputs=16790worker2 # [ 205.967559] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:46Z level=debug event=want client=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06791worker1 # [ 205.996250] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666792worker1 # [ 206.001065] nix-daemon[855]: reaped child process 2068, status = succeeded6793worker1 # [ 206.006170] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666794worker1 # [ 206.013728] nix-grpc-daemon[1671]: time=2026-09-25T15:02:46.545Z level=INFO msg="All 2 paths already cached"6795worker1 # [ 206.019429] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666796worker1 # [ 206.026962] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6797worker1 # [ 206.135962] nix-daemon[855]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)6798worker1 # [ 206.150574] nix-daemon[855]: reaped child process 2096, status = succeeded6799lb # [ 206.133196] niks3-server[1213]: 2026/09/25 15:02:46 INFO Received push request method=POST path=/api/pushes6800worker1 # [ 206.161730] nix-grpc-daemon[1671]: time=2026-09-25T15:02:46.693Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6801worker1 # [ 206.163272] nix-grpc-daemon[1671]: time=2026-09-25T15:02:46.693Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6802lb # [ 206.149052] niks3-server[1213]: 2026/09/25 15:02:46 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign6803worker1 # [ 206.177346] nix-grpc-daemon[1671]: time=2026-09-25T15:02:46.709Z level=INFO msg="Uploading 1 narinfos"6804lb # [ 206.156946] niks3-server[1213]: 2026/09/25 15:02:46 INFO Signed narinfos id=80 count=16805lb # [ 206.161322] niks3-server[1213]: 2026/09/25 15:02:46 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6806lb # [ 206.163754] niks3-server[1213]: 2026/09/25 15:02:46 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6807lb # [ 206.174591] niks3-server[1213]: 2026/09/25 15:02:46 INFO Received complete push request method=POST path=/api/pushes/80/complete6808worker1 # [ 206.202662] nix-grpc-daemon[1671]: time=2026-09-25T15:02:46.734Z level=INFO msg="Upload complete. (167ms)"6809lb # [ 206.179115] niks3-server[1213]: 2026/09/25 15:02:46 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6810worker1 # [ 206.208943] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=7526811worker1 # [ 206.218203] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666812worker1 # [ 206.224225] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6813worker1 # [ 206.226709] nix-daemon[855]: reaped child process 2091, status = succeeded6814worker1 # [ 206.236401] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6815worker1 # [ 207.465673] nix-daemon[855]: reaped child process 2100, status = succeeded6816worker1 # [ 207.579812] nix-daemon[855]: accepted connection from pid 2116, user nix-grpc-daemon (trusted)6817worker1 # [ 207.600687] nix-daemon[855]: reaped child process 2119, status = succeeded6818lb # [ 207.581452] niks3-server[1213]: 2026/09/25 15:02:48 INFO Received push request method=POST path=/api/pushes6819worker1 # [ 207.612005] nix-grpc-daemon[1671]: time=2026-09-25T15:02:48.142Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6820worker1 # [ 207.613935] nix-grpc-daemon[1671]: time=2026-09-25T15:02:48.142Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6821lb # [ 207.606919] niks3-server[1213]: 2026/09/25 15:02:48 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6822worker1 # [ 207.638178] nix-grpc-daemon[1671]: time=2026-09-25T15:02:48.170Z level=INFO msg="Uploading 1 narinfos"6823lb # [ 207.615609] niks3-server[1213]: 2026/09/25 15:02:48 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6824lb # [ 207.622398] niks3-server[1213]: 2026/09/25 15:02:48 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign6825lb # [ 207.630960] niks3-server[1213]: 2026/09/25 15:02:48 INFO Signed narinfos id=81 count=16826worker1 # [ 207.658422] nix-grpc-daemon[1671]: time=2026-09-25T15:02:48.190Z level=INFO msg="Upload complete. (214ms)"6827lb # [ 207.635062] niks3-server[1213]: 2026/09/25 15:02:48 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6828worker2 # [ 207.637125] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:48Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16829lb # [ 207.642287] niks3-server[1213]: 2026/09/25 15:02:48 INFO Received complete push request method=POST path=/api/pushes/81/complete6830worker1 # [ 207.665320] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=11 outputs=16831lb # [ 207.645136] niks3-server[1213]: 2026/09/25 15:02:48 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6832worker1 # [ 207.678208] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666833worker1 # [ 207.682251] nix-daemon[855]: reaped child process 2098, status = succeeded6834worker1 # [ 207.687430] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666835worker1 # [ 207.695161] nix-grpc-daemon[1671]: time=2026-09-25T15:02:48.227Z level=INFO msg="All 2 paths already cached"6836worker1 # [ 207.700792] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666837worker1 # [ 207.709102] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6838worker1 # [ 207.822113] nix-daemon[855]: accepted connection from pid 2123, user nix-grpc-daemon (trusted)6839lb # [ 207.818819] niks3-server[1213]: 2026/09/25 15:02:48 INFO Received push request method=POST path=/api/pushes6840worker1 # [ 207.844609] nix-daemon[855]: reaped child process 2126, status = succeeded6841worker1 # [ 207.848247] nix-grpc-daemon[1671]: time=2026-09-25T15:02:48.380Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6842worker1 # [ 207.849773] nix-grpc-daemon[1671]: time=2026-09-25T15:02:48.380Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6843lb # [ 207.839006] niks3-server[1213]: 2026/09/25 15:02:48 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign6844worker1 # [ 207.866622] nix-grpc-daemon[1671]: time=2026-09-25T15:02:48.398Z level=INFO msg="Uploading 1 narinfos"6845lb # [ 207.847006] niks3-server[1213]: 2026/09/25 15:02:48 INFO Signed narinfos id=82 count=16846lb # [ 207.851406] niks3-server[1213]: 2026/09/25 15:02:48 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6847lb # [ 207.857017] niks3-server[1213]: 2026/09/25 15:02:48 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6848lb # [ 207.865022] niks3-server[1213]: 2026/09/25 15:02:48 INFO Received complete push request method=POST path=/api/pushes/82/complete6849worker1 # [ 207.892098] nix-grpc-daemon[1671]: time=2026-09-25T15:02:48.424Z level=INFO msg="Upload complete. (174ms)"6850lb # [ 207.869018] niks3-server[1213]: 2026/09/25 15:02:48 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6851worker1 # [ 207.898080] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=7526852worker1 # [ 207.907067] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666853worker1 # [ 207.912007] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6854worker1 # [ 207.915895] nix-daemon[855]: reaped child process 2121, status = succeeded6855worker1 # [ 207.926064] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6856worker1 # [ 208.784457] nix-daemon[855]: reaped child process 2130, status = succeeded6857worker1 # [ 208.864358] nix-daemon[855]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)6858lb # [ 208.860224] niks3-server[1213]: 2026/09/25 15:02:49 INFO Received push request method=POST path=/api/pushes6859worker1 # [ 208.887190] nix-daemon[855]: reaped child process 2149, status = succeeded6860worker1 # [ 208.892444] nix-grpc-daemon[1671]: time=2026-09-25T15:02:49.422Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6861worker1 # [ 208.893957] nix-grpc-daemon[1671]: time=2026-09-25T15:02:49.422Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6862lb # [ 208.890664] niks3-server[1213]: 2026/09/25 15:02:49 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6863lb # [ 208.899439] niks3-server[1213]: 2026/09/25 15:02:49 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6864worker1 # [ 208.926054] nix-grpc-daemon[1671]: time=2026-09-25T15:02:49.457Z level=INFO msg="Uploading 1 narinfos"6865lb # [ 208.905770] niks3-server[1213]: 2026/09/25 15:02:49 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign6866lb # [ 208.911916] niks3-server[1213]: 2026/09/25 15:02:49 INFO Signed narinfos id=83 count=16867lb # [ 208.915891] niks3-server[1213]: 2026/09/25 15:02:49 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6868lb # [ 208.922668] niks3-server[1213]: 2026/09/25 15:02:49 INFO Received complete push request method=POST path=/api/pushes/83/complete6869worker1 # [ 208.950750] nix-grpc-daemon[1671]: time=2026-09-25T15:02:49.482Z level=INFO msg="Upload complete. (217ms)"6870lb # [ 208.928152] niks3-server[1213]: 2026/09/25 15:02:49 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6871worker2 # [ 208.929579] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:49Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16872worker1 # [ 208.958354] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=12 outputs=16873worker1 # [ 208.970621] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666874worker1 # [ 208.974716] nix-daemon[855]: reaped child process 2128, status = succeeded6875worker1 # [ 208.980512] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666876worker1 # [ 208.989172] nix-grpc-daemon[1671]: time=2026-09-25T15:02:49.521Z level=INFO msg="All 2 paths already cached"6877worker1 # [ 208.995409] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524666878worker1 # [ 209.004313] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6879worker1 # [ 209.122357] nix-daemon[855]: accepted connection from pid 2153, user nix-grpc-daemon (trusted)6880worker1 # [ 209.140419] nix-daemon[855]: reaped child process 2156, status = succeeded6881lb # [ 209.122831] niks3-server[1213]: 2026/09/25 15:02:49 INFO Received push request method=POST path=/api/pushes6882worker1 # [ 209.151620] nix-grpc-daemon[1671]: time=2026-09-25T15:02:49.683Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6883worker1 # [ 209.153323] nix-grpc-daemon[1671]: time=2026-09-25T15:02:49.683Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6884lb # [ 209.142527] niks3-server[1213]: 2026/09/25 15:02:49 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign6885worker1 # [ 209.169377] nix-grpc-daemon[1671]: time=2026-09-25T15:02:49.701Z level=INFO msg="Uploading 1 narinfos"6886lb # [ 209.149242] niks3-server[1213]: 2026/09/25 15:02:49 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6887lb # [ 209.151791] niks3-server[1213]: 2026/09/25 15:02:49 INFO Signed narinfos id=84 count=16888lb # [ 209.161276] niks3-server[1213]: 2026/09/25 15:02:49 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6889worker1 # [ 209.189559] nix-grpc-daemon[1671]: time=2026-09-25T15:02:49.721Z level=INFO msg="Upload complete. (177ms)"6890lb # [ 209.166860] niks3-server[1213]: 2026/09/25 15:02:49 INFO Received complete push request method=POST path=/api/pushes/84/complete6891lb # [ 209.170579] niks3-server[1213]: 2026/09/25 15:02:49 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6892worker1 # [ 209.196181] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=0 paths=1 nar_bytes_in=7526893worker1 # [ 209.200557] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524666894worker1 # [ 209.208394] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6895worker1 # [ 209.211774] nix-daemon[855]: reaped child process 2151, status = succeeded6896worker1 # [ 209.221701] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6897worker1 # [ 210.501236] nix-daemon[855]: reaped child process 2160, status = succeeded6898worker1 # [ 210.564260] nix-daemon[855]: accepted connection from pid 2176, user nix-grpc-daemon (trusted)6899worker1 # [ 210.581539] nix-daemon[855]: reaped child process 2179, status = succeeded6900lb # [ 210.562697] niks3-server[1213]: 2026/09/25 15:02:51 INFO Received push request method=POST path=/api/pushes6901worker1 # [ 210.592376] nix-grpc-daemon[1671]: time=2026-09-25T15:02:51.123Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6902worker1 # [ 210.593924] nix-grpc-daemon[1671]: time=2026-09-25T15:02:51.123Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6903lb # [ 210.590579] niks3-server[1213]: 2026/09/25 15:02:51 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign6904worker1 # [ 210.621167] nix-grpc-daemon[1671]: time=2026-09-25T15:02:51.153Z level=INFO msg="Uploading 1 narinfos"6905lb # [ 210.599308] niks3-server[1213]: 2026/09/25 15:02:51 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6906lb # [ 210.605335] niks3-server[1213]: 2026/09/25 15:02:51 INFO Signed narinfos id=85 count=16907lb # [ 210.609886] niks3-server[1213]: 2026/09/25 15:02:51 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6908lb # [ 210.619219] niks3-server[1213]: 2026/09/25 15:02:51 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6909worker1 # [ 210.646687] nix-grpc-daemon[1671]: time=2026-09-25T15:02:51.178Z level=INFO msg="Upload complete. (217ms)"6910lb # [ 210.622637] niks3-server[1213]: 2026/09/25 15:02:51 INFO Received complete push request method=POST path=/api/pushes/85/complete6911lb # [ 210.627712] niks3-server[1213]: 2026/09/25 15:02:51 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6912worker1 # [ 210.659593] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52466 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=13 outputs=16913worker1 # [ 210.664296] nix-daemon[855]: reaped child process 2158, status = succeeded6914client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 12.97 seconds)6915worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6916worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6917client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6918client # [ 210.745192] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6919client # Running as unit: gpu.service; invocation ID: 87edbe81bdf94c4c9ca5211b1aec5c6f6920client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.05 seconds)6921lb: must succeed: curl -sf localhost:9901/clusters6922lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6923lb: must succeed: curl -sf localhost:9901/clusters6924lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6925worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6926worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6927worker2 # [ 210.918514] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:481726928client # [ 210.928695] nix[1576]: don't know how to build these paths:6929client # [ 210.929631] nix[1576]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6930worker1 # [ 210.951904] nix-grpc-daemon[1575]: ts=2026-09-25T15:02:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524666931worker2 # [ 210.938475] nix-grpc-daemon[1910]: ts=2026-09-25T15:02:51Z level=debug event=want client=3 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06932client # [ 210.944175] nix[1576]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6933lb: must succeed: curl -sf localhost:9901/clusters6934lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6935lb: must succeed: curl -sf localhost:9901/clusters6936lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6937worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6938worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6939client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6940lb # [ 218.277256] envoy[897]: conn peer=[2001:db8:1::1]:51206 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27093 tx=12255 ms=127686941client # [ 240.945131] nix[1576]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6942client # [ 240.955068] nix[1576]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6943client # [ 240.973713] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6944client # [ 240.977687] systemd[1]: gpu.service: Failed with result 'exit-code'.6945client # [ 240.979602] systemd[1]: gpu.service: Consumed 174ms CPU time over 30.231s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.6946client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.24 seconds)6947client: must succeed: journalctl -u gpu -o cat6948client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)6949lb: must succeed: curl -sf localhost:9901/clusters6950lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6951lb: must succeed: curl -sf localhost:9901/clusters6952lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6953worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6954worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6955(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 43.65 seconds)6956subtest: low disk drains a worker and builds go to the other6957worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6958worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.19 seconds)6959worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6960worker1 # [ 243.349733] nix-grpc-daemon[1575]: ts=2026-09-25T15:03:23Z level=info event=unhealthy reason=min_free6961worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.07 seconds)6962client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26963client # don't know how to build these paths:6964client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6965worker2 # [ 243.788279] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:481726966worker2 # [ 243.793319] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:481726967worker2 # [ 243.808235] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:24Z level=debug event=want client=4 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06968worker2 # [ 243.811217] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:24Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16969worker2 # [ 243.816874] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481726970worker2 # [ 243.827979] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:481726971client # copying 1 paths to 'lb:50051'...6972client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6973worker2 # [ 243.837669] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:481726974worker2 # [ 243.846290] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6975worker2 # [ 243.972374] nix-daemon[845]: accepted connection from pid 2112, user nix-grpc-daemon (trusted)6976worker2 # [ 243.994848] nix-daemon[845]: reaped child process 2115, status = succeeded6977lb # [ 243.997541] niks3-server[1213]: 2026/09/25 15:03:24 INFO Received push request method=POST path=/api/pushes6978worker2 # [ 244.000879] nix-grpc-daemon[1984]: time=2026-09-25T15:03:24.543Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6979worker2 # [ 244.003130] nix-grpc-daemon[1984]: time=2026-09-25T15:03:24.545Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6980lb # [ 244.016222] niks3-server[1213]: 2026/09/25 15:03:24 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign6981worker2 # [ 244.018839] nix-grpc-daemon[1984]: time=2026-09-25T15:03:24.561Z level=INFO msg="Uploading 1 narinfos"6982lb # [ 244.025316] niks3-server[1213]: 2026/09/25 15:03:24 INFO Signed narinfos id=86 count=16983lb # [ 244.029276] niks3-server[1213]: 2026/09/25 15:03:24 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6984lb # [ 244.036676] niks3-server[1213]: 2026/09/25 15:03:24 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6985lb # [ 244.045229] niks3-server[1213]: 2026/09/25 15:03:24 INFO Received complete push request method=POST path=/api/pushes/86/complete6986lb # [ 244.051582] niks3-server[1213]: 2026/09/25 15:03:24 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6987worker2 # [ 244.052073] nix-grpc-daemon[1984]: time=2026-09-25T15:03:24.594Z level=INFO msg="Upload complete. (196ms)"6988worker2 # [ 244.058057] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=0 paths=1 nar_bytes_in=5126989worker2 # [ 244.064826] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481726990worker2 # [ 244.076086] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6991worker2 # [ 244.078515] nix-daemon[845]: reaped child process 2110, status = succeeded6992client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6993client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6994worker2 # [ 244.089359] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6995client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6996client # farm-a> LOG-farm-a-drain6997worker2 # [ 245.152799] nix-daemon[845]: reaped child process 2119, status = succeeded6998worker2 # [ 245.221057] nix-daemon[845]: accepted connection from pid 2135, user nix-grpc-daemon (trusted)6999worker2 # [ 245.242868] nix-daemon[845]: reaped child process 2138, status = succeeded7000lb # [ 245.245008] niks3-server[1213]: 2026/09/25 15:03:25 INFO Received push request method=POST path=/api/pushes7001worker2 # [ 245.248302] nix-grpc-daemon[1984]: time=2026-09-25T15:03:25.788Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7002worker2 # [ 245.249858] nix-grpc-daemon[1984]: time=2026-09-25T15:03:25.789Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7003lb # [ 245.265002] niks3-server[1213]: 2026/09/25 15:03:25 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7004lb # [ 245.273250] niks3-server[1213]: 2026/09/25 15:03:25 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7005worker2 # [ 245.277498] nix-grpc-daemon[1984]: time=2026-09-25T15:03:25.819Z level=INFO msg="Uploading 1 narinfos"7006lb # [ 245.278513] niks3-server[1213]: 2026/09/25 15:03:25 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign7007lb # [ 245.284826] niks3-server[1213]: 2026/09/25 15:03:25 INFO Signed narinfos id=87 count=17008lb # [ 245.288872] niks3-server[1213]: 2026/09/25 15:03:25 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7009lb # [ 245.296672] niks3-server[1213]: 2026/09/25 15:03:25 INFO Received complete push request method=POST path=/api/pushes/87/complete7010worker2 # [ 245.301202] nix-grpc-daemon[1984]: time=2026-09-25T15:03:25.843Z level=INFO msg="Upload complete. (203ms)"7011lb # [ 245.305380] niks3-server[1213]: 2026/09/25 15:03:25 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7012worker2 # [ 245.309639] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=14 outputs=17013worker2 # [ 245.325334] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:25Z level=debug event=want client=4 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07014worker2 # [ 245.328144] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:25Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17015worker2 # [ 245.331105] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481727016worker2 # [ 245.334252] nix-daemon[845]: reaped child process 2117, status = succeeded7017worker2 # [ 245.338914] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:481727018worker2 # [ 245.346889] nix-grpc-daemon[1984]: time=2026-09-25T15:03:25.889Z level=INFO msg="All 1 paths already cached"7019client # copying 1 paths to 'lb:50051'...7020client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7021worker2 # [ 245.353782] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:481727022worker2 # [ 245.361238] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7023worker2 # [ 245.475032] nix-daemon[845]: accepted connection from pid 2144, user nix-grpc-daemon (trusted)7024lb # [ 245.495591] niks3-server[1213]: 2026/09/25 15:03:26 INFO Received push request method=POST path=/api/pushes7025worker2 # [ 245.495046] nix-daemon[845]: reaped child process 2147, status = succeeded7026worker2 # [ 245.497083] nix-grpc-daemon[1984]: time=2026-09-25T15:03:26.039Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7027worker2 # [ 245.498722] nix-grpc-daemon[1984]: time=2026-09-25T15:03:26.039Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7028lb # [ 245.510987] niks3-server[1213]: 2026/09/25 15:03:26 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7029worker2 # [ 245.516476] nix-grpc-daemon[1984]: time=2026-09-25T15:03:26.058Z level=INFO msg="Uploading 1 narinfos"7030lb # [ 245.520563] niks3-server[1213]: 2026/09/25 15:03:26 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign7031lb # [ 245.523034] niks3-server[1213]: 2026/09/25 15:03:26 INFO Signed narinfos id=88 count=17032lb # [ 245.530369] niks3-server[1213]: 2026/09/25 15:03:26 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7033lb # [ 245.537156] niks3-server[1213]: 2026/09/25 15:03:26 INFO Received complete push request method=POST path=/api/pushes/88/complete7034worker2 # [ 245.540279] nix-grpc-daemon[1984]: time=2026-09-25T15:03:26.082Z level=INFO msg="Upload complete. (170ms)"7035lb # [ 245.545223] niks3-server[1213]: 2026/09/25 15:03:26 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7036worker2 # [ 245.547152] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=0 paths=1 nar_bytes_in=6327037worker2 # [ 245.552211] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481727038worker2 # [ 245.563358] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7039worker2 # [ 245.565532] nix-daemon[845]: reaped child process 2142, status = succeeded7040client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7041worker2 # [ 245.574022] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7042client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7043client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7044client # farm-b> LOG-farm-b-drain7045worker2 # [ 246.224850] nix-daemon[845]: reaped child process 2151, status = succeeded7046worker2 # [ 246.312729] nix-daemon[845]: accepted connection from pid 2167, user nix-grpc-daemon (trusted)7047worker2 # [ 246.329894] nix-daemon[845]: reaped child process 2170, status = succeeded7048lb # [ 246.339311] niks3-server[1213]: 2026/09/25 15:03:26 INFO Received push request method=POST path=/api/pushes7049worker2 # [ 246.343850] nix-grpc-daemon[1984]: time=2026-09-25T15:03:26.883Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7050worker2 # [ 246.345500] nix-grpc-daemon[1984]: time=2026-09-25T15:03:26.883Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7051lb # [ 246.364542] niks3-server[1213]: 2026/09/25 15:03:26 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7052worker2 # [ 246.366087] nix-grpc-daemon[1984]: time=2026-09-25T15:03:26.908Z level=INFO msg="Uploading 1 narinfos"7053lb # [ 246.372460] niks3-server[1213]: 2026/09/25 15:03:26 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7054lb # [ 246.378441] niks3-server[1213]: 2026/09/25 15:03:26 INFO Signed narinfos id=89 count=17055lb # [ 246.386331] niks3-server[1213]: 2026/09/25 15:03:26 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7056worker2 # [ 246.386525] nix-grpc-daemon[1984]: time=2026-09-25T15:03:26.929Z level=INFO msg="Upload complete. (203ms)"7057lb # [ 246.392429] niks3-server[1213]: 2026/09/25 15:03:26 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7058lb # [ 246.398187] niks3-server[1213]: 2026/09/25 15:03:26 INFO Received complete push request method=POST path=/api/pushes/89/complete7059lb # [ 246.401385] niks3-server[1213]: 2026/09/25 15:03:26 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7060worker2 # [ 246.395626] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=0 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=15 outputs=17061worker2 # [ 246.408335] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:26Z level=debug event=want client=4 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07062worker2 # [ 246.413137] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:26Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17063worker2 # [ 246.417480] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481727064worker2 # [ 246.421002] nix-daemon[845]: reaped child process 2149, status = succeeded7065worker2 # [ 246.423556] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:481727066worker2 # [ 246.432434] nix-grpc-daemon[1984]: time=2026-09-25T15:03:26.974Z level=INFO msg="All 2 paths already cached"7067client # copying 1 paths to 'lb:50051'...7068client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7069worker2 # [ 246.438887] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:481727070worker2 # [ 246.446155] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7071worker2 # [ 246.557258] nix-daemon[845]: accepted connection from pid 2176, user nix-grpc-daemon (trusted)7072worker2 # [ 246.575288] nix-daemon[845]: reaped child process 2179, status = succeeded7073lb # [ 246.584302] niks3-server[1213]: 2026/09/25 15:03:27 INFO Received push request method=POST path=/api/pushes7074worker2 # [ 246.585786] nix-grpc-daemon[1984]: time=2026-09-25T15:03:27.127Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7075worker2 # [ 246.587338] nix-grpc-daemon[1984]: time=2026-09-25T15:03:27.128Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7076lb # [ 246.601661] niks3-server[1213]: 2026/09/25 15:03:27 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7077worker2 # [ 246.601391] nix-grpc-daemon[1984]: time=2026-09-25T15:03:27.143Z level=INFO msg="Uploading 1 narinfos"7078lb # [ 246.609279] niks3-server[1213]: 2026/09/25 15:03:27 INFO Signed narinfos id=90 count=17079lb # [ 246.611005] niks3-server[1213]: 2026/09/25 15:03:27 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7080lb # [ 246.621943] niks3-server[1213]: 2026/09/25 15:03:27 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7081worker2 # [ 246.620849] nix-grpc-daemon[1984]: time=2026-09-25T15:03:27.163Z level=INFO msg="Upload complete. (165ms)"7082lb # [ 246.628057] niks3-server[1213]: 2026/09/25 15:03:27 INFO Received complete push request method=POST path=/api/pushes/90/complete7083lb # [ 246.630485] niks3-server[1213]: 2026/09/25 15:03:27 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7084worker2 # [ 246.628108] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=0 paths=1 nar_bytes_in=7607085worker2 # [ 246.632080] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:481727086worker2 # [ 246.639640] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7087worker2 # [ 246.643220] nix-daemon[845]: reaped child process 2174, status = succeeded7088client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7089worker2 # [ 246.651731] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7090client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7091client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7092client # farm-top> LOG-farm-top-drain7093worker2 # [ 247.891029] nix-daemon[845]: reaped child process 2183, status = succeeded7094worker2 # [ 247.987515] nix-daemon[845]: accepted connection from pid 2199, user nix-grpc-daemon (trusted)7095worker2 # [ 248.002207] nix-daemon[845]: reaped child process 2202, status = succeeded7096lb # [ 248.012960] niks3-server[1213]: 2026/09/25 15:03:28 INFO Received push request method=POST path=/api/pushes7097worker2 # [ 248.016399] nix-grpc-daemon[1984]: time=2026-09-25T15:03:28.556Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7098worker2 # [ 248.017910] nix-grpc-daemon[1984]: time=2026-09-25T15:03:28.556Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7099lb # [ 248.038902] niks3-server[1213]: 2026/09/25 15:03:28 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7100worker2 # [ 248.040298] nix-grpc-daemon[1984]: time=2026-09-25T15:03:28.582Z level=INFO msg="Uploading 1 narinfos"7101lb # [ 248.046572] niks3-server[1213]: 2026/09/25 15:03:28 INFO Signed narinfos id=91 count=17102lb # [ 248.051518] niks3-server[1213]: 2026/09/25 15:03:28 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7103lb # [ 248.058940] niks3-server[1213]: 2026/09/25 15:03:28 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7104lb # [ 248.065949] niks3-server[1213]: 2026/09/25 15:03:28 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7105worker2 # [ 248.068607] nix-grpc-daemon[1984]: time=2026-09-25T15:03:28.611Z level=INFO msg="Upload complete. (207ms)"7106lb # [ 248.073576] niks3-server[1213]: 2026/09/25 15:03:28 INFO Received complete push request method=POST path=/api/pushes/91/complete7107lb # [ 248.077592] niks3-server[1213]: 2026/09/25 15:03:28 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7108worker2 # [ 248.077250] nix-grpc-daemon[1910]: ts=2026-09-25T15:03:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48172 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=16 outputs=17109worker2 # [ 248.087125] nix-daemon[845]: reaped child process 2181, status = succeeded7110client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 4.49 seconds)7111client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7112client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.14 seconds)7113lb # [ 248.282384] envoy[897]: conn peer=[2001:db8:1::1]:55598 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=612 ms=300387114lb # [ 248.285155] envoy[897]: conn peer=[2001:db8:1::1]:33056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9485 tx=4349 ms=43037115worker1: must succeed: rm /nix/.rw-store/fill7116worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7117worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7118worker1 # [ 249.358227] nix-grpc-daemon[1575]: ts=2026-09-25T15:03:29Z level=info event=healthy reason=min_free7119worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7120(finished: subtest: low disk drains a worker and builds go to the other, in 8.13 seconds)7121(finished: run the VM test script, in 250.45 seconds)7122test script finished in 250.59s7123cleanup7124kill QemuMachine (pid 49)7125client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7126client # [2026-09-25T15:03:30Z INFO virtiofsd] Client disconnected, shutting down7127client # [2026-09-25T15:03:30Z INFO virtiofsd] Client disconnected, shutting down7128client # [2026-09-25T15:03:30Z INFO virtiofsd] Client disconnected, shutting down7129kill QemuMachine (pid 50)7130lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7131lb # [2026-09-25T15:03:30Z INFO virtiofsd] Client disconnected, shutting down7132lb # [2026-09-25T15:03:30Z INFO virtiofsd] Client disconnected, shutting down7133lb # [2026-09-25T15:03:30Z INFO virtiofsd] Client disconnected, shutting down7134kill QemuMachine (pid 51)7135worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7136worker1 # [2026-09-25T15:03:30Z INFO virtiofsd] Client disconnected, shutting down7137worker1 # [2026-09-25T15:03:30Z INFO virtiofsd] Client disconnected, shutting down7138worker1 # [2026-09-25T15:03:30Z INFO virtiofsd] Client disconnected, shutting down7139kill QemuMachine (pid 52)7140worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7141worker2 # [2026-09-25T15:03:31Z INFO virtiofsd] Client disconnected, shutting down7142worker2 # [2026-09-25T15:03:31Z INFO virtiofsd] Client disconnected, shutting down7143worker2 # [2026-09-25T15:03:31Z INFO virtiofsd] Client disconnected, shutting down7144(finished: cleanup, in 1.04 seconds)7145additionally exposed symbols:7146 client, lb, worker1, worker2,7147 vlan1,7148 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh