vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #555
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13worker1 # Disk image does not exist, creating the virtualisation disk image...14worker1: QEMU running (pid 51)15lb # Disk image does not exist, creating the virtualisation disk image...16client: QEMU running (pid 50)17worker1 # Formatting '/build/vm-state-worker1/tmp.uDhutsKmsN', fmt=raw size=107374182418client # Formatting '/build/vm-state-client/tmp.8s2Y6277X8', fmt=raw size=107374182419lb # Formatting '/build/vm-state-lb/tmp.1irkYEKo7j', fmt=raw size=107374182420worker2: QEMU running (pid 52)21worker2 # Disk image does not exist, creating the virtualisation disk image...22lb: QEMU running (pid 49)23worker1 # mke2fs 1.47.4 (6-Mar-2025)24worker2 # Formatting '/build/vm-state-worker2/tmp.Hq7xoa5Qbs', fmt=raw size=107374182425client # mke2fs 1.47.4 (6-Mar-2025)26lb # mke2fs 1.47.4 (6-Mar-2025)27worker1 # Discarding device blocks: 0/262144 done28client # Discarding device blocks: 0/262144 done29lb # Discarding device blocks: 0/262144 done30worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes31client # Creating filesystem with 262144 4k blocks and 65536 inodes32lb # Creating filesystem with 262144 4k blocks and 65536 inodes33worker1 # Filesystem UUID: 952bd526-8fa1-4ca8-a31e-45e129a69cbb34client # Filesystem UUID: 8f5e06fa-443c-4d89-947a-de9f8eb450d135lb # Filesystem UUID: 78d9a501-aae4-470a-9e1b-a614268ec99d36worker1 # Superblock backups stored on blocks:37client # Superblock backups stored on blocks:38lb # Superblock backups stored on blocks:39worker1 # 32768, 98304, 163840, 22937640client # 32768, 98304, 163840, 22937641lb # 32768, 98304, 163840, 22937642worker1 # 43client # 44lb # 45worker1 # Allocating group tables: 0/8 done46client # Allocating group tables: 0/8 done47lb # Allocating group tables: 0/8 done48worker1 # Writing inode tables: 0/8 done49client # Writing inode tables: 0/8 done50lb # Writing inode tables: 0/8 done51worker1 # Creating journal (8192 blocks): done52client # Creating journal (8192 blocks): done53lb # Creating journal (8192 blocks): done54worker1 # Writing superblocks and filesystem accounting information: 0/8 done55client # Writing superblocks and filesystem accounting information: 0/8 done56lb # Writing superblocks and filesystem accounting information: 0/8 done57worker1 # 58client # 59lb # 60worker1 # Virtualisation disk image created.61client # Virtualisation disk image created.62lb # Virtualisation disk image created.63worker1 # Starting virtiofs daemons...64client # Starting virtiofs daemons...65lb # Starting virtiofs daemons...66worker1 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67client # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)68lb # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69worker1 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70client # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether71lb # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72worker1 # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...73client # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74lb # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75worker1 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76client # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77lb # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78worker1 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79client # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...80lb # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...81worker1 # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...82client # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...83lb # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...84worker1 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85client # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86lb # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)87worker1 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether88client # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89lb # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90worker1 # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...91client # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...92lb # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker1 # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests94client # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests95lb # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests96worker1 # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests97client # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests98lb # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests99worker1 # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests100client # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests101lb # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests102worker2 # mke2fs 1.47.4 (6-Mar-2025)103(finished: start all VMs, in 0.43 seconds)104worker2 # Discarding device blocks: 0/262144 done105client: waiting for unit network-addresses-eth1.service106worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes107client: waiting for the VM to finish booting108worker2 # Filesystem UUID: 5b3f459a-3a60-4ab1-a06c-8cccd02afe4b109worker2 # Superblock backups stored on blocks:110worker2 # 32768, 98304, 163840, 229376111worker2 # 112worker2 # Allocating group tables: 0/8 done113worker2 # Writing inode tables: 0/8 done114worker2 # Creating journal (8192 blocks): done115worker2 # Writing superblocks and filesystem accounting information: 0/8 done116worker2 # 117worker2 # Virtualisation disk image created.118worker2 # Starting virtiofs daemons...119worker2 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker2 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker2 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)122worker2 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether123worker2 # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...124worker2 # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker2 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker2 # [2026-09-25T15:12:43Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker2 # [2026-09-25T15:12:43Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker2 # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests129worker2 # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests130worker2 # [2026-09-25T15:12:43Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]133client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026134lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]135client # [ 0.000000] KASLR enabled136client # [ 0.000000] random: crng init done137worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026138worker2 # [ 0.000000] KASLR enabled139client # [ 0.000000] Machine model: linux,dummy-virt140worker2 # [ 0.000000] random: crng init done141client # [ 0.000000] efi: UEFI not found.142lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026143worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]144worker2 # [ 0.000000] Machine model: linux,dummy-virt145lb # [ 0.000000] KASLR enabled146worker2 # [ 0.000000] efi: UEFI not found.147lb # [ 0.000000] random: crng init done148client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT149lb # [ 0.000000] Machine model: linux,dummy-virt150worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT151worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026152client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]153lb # [ 0.000000] efi: UEFI not found.154worker1 # [ 0.000000] KASLR enabled155client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]156worker1 # [ 0.000000] random: crng init done157worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]158lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT159client # [ 0.000000] Zone ranges:160worker1 # [ 0.000000] Machine model: linux,dummy-virt161worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]162worker1 # [ 0.000000] efi: UEFI not found.163lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]164worker2 # [ 0.000000] Zone ranges:165client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]166client # [ 0.000000] DMA32 empty167lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]168worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT169worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]170client # [ 0.000000] Normal empty171lb # [ 0.000000] Zone ranges:172worker2 # [ 0.000000] DMA32 empty173client # [ 0.000000] Device empty174worker2 # [ 0.000000] Normal empty175worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]176client # [ 0.000000] Movable zone start for each node177lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]178worker2 # [ 0.000000] Device empty179client # [ 0.000000] Early memory node ranges180lb # [ 0.000000] DMA32 empty181worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]182worker2 # [ 0.000000] Movable zone start for each node183lb # [ 0.000000] Normal empty184worker1 # [ 0.000000] Zone ranges:185worker2 # [ 0.000000] Early memory node ranges186lb # [ 0.000000] Device empty187client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]188worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]189lb # [ 0.000000] Movable zone start for each node190worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]191client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]192worker1 # [ 0.000000] DMA32 empty193lb # [ 0.000000] Early memory node ranges194worker1 # [ 0.000000] Normal empty195client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000196worker1 # [ 0.000000] Device empty197worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]198lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]199client # [ 0.000000] psci: probing for conduit method from DT.200worker1 # [ 0.000000] Movable zone start for each node201worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000202worker1 # [ 0.000000] Early memory node ranges203lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]204client # [ 0.000000] psci: PSCIv1.3 detected in firmware.205worker2 # [ 0.000000] psci: probing for conduit method from DT.206lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000207worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]208client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs209worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.210lb # [ 0.000000] psci: probing for conduit method from DT.211client # [ 0.000000] psci: Trusted OS migration not required212worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs213worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]214lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.215client # [ 0.000000] psci: SMC Calling Convention v1.1216worker2 # [ 0.000000] psci: Trusted OS migration not required217worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000218lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs219worker2 # [ 0.000000] psci: SMC Calling Convention v1.1220worker1 # [ 0.000000] psci: probing for conduit method from DT.221client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)222lb # [ 0.000000] psci: Trusted OS migration not required223worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.224lb # [ 0.000000] psci: SMC Calling Convention v1.1225worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)226client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296227worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs228client # [ 0.000000] Detected PIPT I-cache on CPU0229worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296230worker1 # [ 0.000000] psci: Trusted OS migration not required231lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)232worker2 # [ 0.000000] Detected PIPT I-cache on CPU0233worker1 # [ 0.000000] psci: SMC Calling Convention v1.1234client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)235lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296236worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)237client # [ 0.000000] CPU features: detected: GICv3 CPU interface238lb # [ 0.000000] Detected PIPT I-cache on CPU0239worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)240client # [ 0.000000] CPU features: detected: Spectre-v4241worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface242worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296243lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)244client # [ 0.000000] CPU features: detected: Spectre-BHB245worker2 # [ 0.000000] CPU features: detected: Spectre-v4246worker1 # [ 0.000000] Detected PIPT I-cache on CPU0247worker2 # [ 0.000000] CPU features: detected: Spectre-BHB248lb # [ 0.000000] CPU features: detected: GICv3 CPU interface249client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38250lb # [ 0.000000] CPU features: detected: Spectre-v4251worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38252worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)253client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23254lb # [ 0.000000] CPU features: detected: Spectre-BHB255worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface256client # [ 0.000000] alternatives: applying boot alternatives257worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23258lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38259worker1 # [ 0.000000] CPU features: detected: Spectre-v4260worker2 # [ 0.000000] alternatives: applying boot alternatives261worker1 # [ 0.000000] CPU features: detected: Spectre-BHB262lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23263lb # [ 0.000000] alternatives: applying boot alternatives264worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38265worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23266worker1 # [ 0.000000] alternatives: applying boot alternatives267client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/asn96skv4pz1dy7kpxiknx6acaxaqim2-nixos-system-client-test/init regInfo=/nix/store/jmsaixk5vg7a4wk5ycc9m5i6y1rhv2gz-closure-info/registration console=ttyAMA0,115200n8 console=tty0268worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8hfm5g55mm1a78kycaqy2y8ivyppbyxv-nixos-system-worker2-test/init regInfo=/nix/store/arwyc78zk5iwrdx09lg5hc94m22fbarv-closure-info/registration console=ttyAMA0,115200n8 console=tty0269lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hmvppg1fs8xkiccvl0m50n4dxjz92fr-nixos-system-lb-test/init regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration console=ttyAMA0,115200n8 console=tty0270client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/jmsaixk5vg7a4wk5ycc9m5i6y1rhv2gz-closure-info/registration", will be passed to user space.271worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/arwyc78zk5iwrdx09lg5hc94m22fbarv-closure-info/registration", will be passed to user space.272worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/cf7m9898wwk3l26b0ymyd3x7fy5a8z8w-nixos-system-worker1-test/init regInfo=/nix/store/gj2j4n8z8rs4i1976lidcazqxiyx1r97-closure-info/registration console=ttyAMA0,115200n8 console=tty0273client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes274worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes275lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration", will be passed to user space.276worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)277client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)278lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes279worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/gj2j4n8z8rs4i1976lidcazqxiyx1r97-closure-info/registration", will be passed to user space.280client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)281worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)282lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)283worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes284client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB285worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB286lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)287client # [ 0.000000] software IO TLB: area num 1.288worker2 # [ 0.000000] software IO TLB: area num 1.289worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)290lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB291worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB292client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB293lb # [ 0.000000] software IO TLB: area num 1.294worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)295worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)296client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)297lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB298worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB299client # [ 0.000000] Fallback order for Node 0: 0300worker2 # [ 0.000000] Fallback order for Node 0: 0301worker1 # [ 0.000000] software IO TLB: area num 1.302lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)303client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216304worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216305worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB306lb # [ 0.000000] Fallback order for Node 0: 0307client # [ 0.000000] Policy zone: DMA308worker2 # [ 0.000000] Policy zone: DMA309lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216310client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off311worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off312worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)313lb # [ 0.000000] Policy zone: DMA314worker1 # [ 0.000000] Fallback order for Node 0: 0315worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1316client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1317lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off318worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216319client # [ 0.000000] allocated 3145728 bytes of page_ext320worker2 # [ 0.000000] allocated 3145728 bytes of page_ext321worker1 # [ 0.000000] Policy zone: DMA322lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1323client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages324worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages325lb # [ 0.000000] allocated 3145728 bytes of page_ext326worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off327worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups328client # [ 0.000000] ftrace: allocated 294 pages with 4 groups329lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages330worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.331client # [ 0.000000] rcu: Hierarchical RCU implementation.332worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1333lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups334worker2 # [ 0.000000] rcu: RCU event tracing is enabled.335client # [ 0.000000] rcu: RCU event tracing is enabled.336worker1 # [ 0.000000] allocated 3145728 bytes of page_ext337lb # [ 0.000000] rcu: Hierarchical RCU implementation.338worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages339worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.340client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.341lb # [ 0.000000] rcu: RCU event tracing is enabled.342worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups343worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.344client # [ 0.000000] Trampoline variant of Tasks RCU enabled.345lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.346worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.347client # [ 0.000000] Rude variant of Tasks RCU enabled.348worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.349worker1 # [ 0.000000] rcu: RCU event tracing is enabled.350lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.351worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.352client # [ 0.000000] Tracing variant of Tasks RCU enabled.353lb # [ 0.000000] Rude variant of Tasks RCU enabled.354worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.355worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.356client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.357lb # [ 0.000000] Tracing variant of Tasks RCU enabled.358worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.359worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1360client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1361worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.362lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.363worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.364worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.365client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.366lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1367worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.368worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.369lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.370client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.371worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1372worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.373lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.374client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.375worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.376worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0377client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0378lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.379worker2 # [ 0.000000] GICv3: 256 SPIs implemented380client # [ 0.000000] GICv3: 256 SPIs implemented381worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.382worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented383client # [ 0.000000] GICv3: 0 Extended SPIs implemented384lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0385lb # [ 0.000000] GICv3: 256 SPIs implemented386client # [ 0.000000] Root IRQ handler: gic_handle_irq387worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq388worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.389lb # [ 0.000000] GICv3: 0 Extended SPIs implemented390worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI391client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI392worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0393lb # [ 0.000000] Root IRQ handler: gic_handle_irq394worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0395client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0396worker1 # [ 0.000000] GICv3: 256 SPIs implemented397lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI398worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented399worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000400client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000401lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0402worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq403worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]404client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]405worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI406lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000407worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0408worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)409lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]410client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)411worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000412lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)413client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)414worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)415worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]416client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000417worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000418worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)419lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)420client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000421worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000422lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000423worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)424worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.425client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.426lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000427worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000428worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns429client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns430lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.431worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000432worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).433client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).434lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns435worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.436lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).437worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns438client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns439worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns440client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns441worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns442worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).443lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns444client # [ 0.000028] arm-pv: using stolen time PV445worker2 # [ 0.000029] arm-pv: using stolen time PV446lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns447worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns448client # [ 0.000379] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)449lb # [ 0.000030] arm-pv: using stolen time PV450worker2 # [ 0.000427] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)451worker2 # [ 0.000570] Console: colour dummy device 80x25452client # [ 0.000560] Console: colour dummy device 80x25453worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns454worker2 # [ 0.000577] printk: legacy console [tty0] enabled455client # [ 0.000568] printk: legacy console [tty0] enabled456lb # [ 0.000422] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)457worker1 # [ 0.000028] arm-pv: using stolen time PV458lb # [ 0.000559] Console: colour dummy device 80x25459lb # [ 0.000567] printk: legacy console [tty0] enabled460worker1 # [ 0.000387] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)461worker2 # [ 0.000791] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)462client # [ 0.000776] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)463worker1 # [ 0.000563] Console: colour dummy device 80x25464client # [ 0.000783] pid_max: default: 32768 minimum: 301465worker2 # [ 0.000798] pid_max: default: 32768 minimum: 301466worker1 # [ 0.000571] printk: legacy console [tty0] enabled467lb # [ 0.000764] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)468client # [ 0.000856] LSM: initializing lsm=capability,landlock,yama,bpf,ima469worker2 # [ 0.000873] LSM: initializing lsm=capability,landlock,yama,bpf,ima470lb # [ 0.000771] pid_max: default: 32768 minimum: 301471client # [ 0.000990] landlock: Up and running.472worker2 # [ 0.001009] landlock: Up and running.473client # [ 0.000993] Yama: becoming mindful.474worker2 # [ 0.001012] Yama: becoming mindful.475worker1 # [ 0.000764] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)476lb # [ 0.000846] LSM: initializing lsm=capability,landlock,yama,bpf,ima477worker2 # [ 0.001470] LSM support for eBPF active478client # [ 0.001438] LSM support for eBPF active479worker1 # [ 0.000772] pid_max: default: 32768 minimum: 301480lb # [ 0.000974] landlock: Up and running.481lb # [ 0.000978] Yama: becoming mindful.482client # [ 0.001580] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)483worker2 # [ 0.001593] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)484worker1 # [ 0.000853] LSM: initializing lsm=capability,landlock,yama,bpf,ima485lb # [ 0.001418] LSM support for eBPF active486worker1 # [ 0.001130] landlock: Up and running.487client # [ 0.001619] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)488worker1 # [ 0.001133] Yama: becoming mindful.489worker2 # [ 0.001628] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)490lb # [ 0.001590] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)491worker1 # [ 0.001580] LSM support for eBPF active492client # [ 0.003507] rcu: Hierarchical SRCU implementation.493worker2 # [ 0.003550] rcu: Hierarchical SRCU implementation.494lb # [ 0.001624] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)495client # [ 0.003512] rcu: Max phase no-delay instances is 1000.496worker2 # [ 0.003555] rcu: Max phase no-delay instances is 1000.497worker1 # [ 0.001733] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)498lb # [ 0.003614] rcu: Hierarchical SRCU implementation.499client # [ 0.004897] fsl-mc MSI: its@8080000 domain created500worker2 # [ 0.005010] fsl-mc MSI: its@8080000 domain created501worker1 # [ 0.001770] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)502client # [ 0.004987] EFI services will not be available.503worker2 # [ 0.005099] EFI services will not be available.504lb # [ 0.003620] rcu: Max phase no-delay instances is 1000.505client # [ 0.005060] smp: Bringing up secondary CPUs ...506worker1 # [ 0.003595] rcu: Hierarchical SRCU implementation.507lb # [ 0.004929] fsl-mc MSI: its@8080000 domain created508worker2 # [ 0.005219] smp: Bringing up secondary CPUs ...509worker2 # [ 0.005227] smp: Brought up 1 node, 1 CPU510client # [ 0.005069] smp: Brought up 1 node, 1 CPU511worker1 # [ 0.003600] rcu: Max phase no-delay instances is 1000.512lb # [ 0.005025] EFI services will not be available.513client # [ 0.005072] SMP: Total of 1 processors activated.514worker2 # [ 0.005230] SMP: Total of 1 processors activated.515lb # [ 0.005105] smp: Bringing up secondary CPUs ...516worker1 # [ 0.004931] fsl-mc MSI: its@8080000 domain created517client # [ 0.005078] CPU: All CPU(s) started at EL1518worker2 # [ 0.005235] CPU: All CPU(s) started at EL1519lb # [ 0.005114] smp: Brought up 1 node, 1 CPU520worker1 # [ 0.005021] EFI services will not be available.521lb # [ 0.005117] SMP: Total of 1 processors activated.522client # [ 0.005088] CPU features: detected: Branch Target Identification523worker2 # [ 0.005246] CPU features: detected: Branch Target Identification524worker1 # [ 0.005091] smp: Bringing up secondary CPUs ...525lb # [ 0.005122] CPU: All CPU(s) started at EL1526worker1 # [ 0.005099] smp: Brought up 1 node, 1 CPU527client # [ 0.005093] CPU features: detected: ARMv8.4 Translation Table Level528worker2 # [ 0.005250] CPU features: detected: ARMv8.4 Translation Table Level529worker1 # [ 0.005102] SMP: Total of 1 processors activated.530lb # [ 0.005131] CPU features: detected: Branch Target Identification531worker1 # [ 0.005108] CPU: All CPU(s) started at EL1532client # [ 0.005096] CPU features: detected: Instruction cache invalidation not required for I/D coherence533worker2 # [ 0.005253] CPU features: detected: Instruction cache invalidation not required for I/D coherence534lb # [ 0.005136] CPU features: detected: ARMv8.4 Translation Table Level535worker1 # [ 0.005118] CPU features: detected: Branch Target Identification536client # [ 0.005100] CPU features: detected: Data cache clean to the PoU not required for I/D coherence537worker2 # [ 0.005257] CPU features: detected: Data cache clean to the PoU not required for I/D coherence538lb # [ 0.005139] CPU features: detected: Instruction cache invalidation not required for I/D coherence539worker1 # [ 0.005123] CPU features: detected: ARMv8.4 Translation Table Level540client # [ 0.005103] CPU features: detected: Common not Private translations541worker2 # [ 0.005261] CPU features: detected: Common not Private translations542lb # [ 0.005143] CPU features: detected: Data cache clean to the PoU not required for I/D coherence543worker1 # [ 0.005127] CPU features: detected: Instruction cache invalidation not required for I/D coherence544worker2 # [ 0.005264] CPU features: detected: CRC32 instructions545client # [ 0.005106] CPU features: detected: CRC32 instructions546lb # [ 0.005147] CPU features: detected: Common not Private translations547worker1 # [ 0.005130] CPU features: detected: Data cache clean to the PoU not required for I/D coherence548worker2 # [ 0.005267] CPU features: detected: Data cache clean to Point of Deep Persistence549client # [ 0.005109] CPU features: detected: Data cache clean to Point of Deep Persistence550lb # [ 0.005150] CPU features: detected: CRC32 instructions551worker1 # [ 0.005134] CPU features: detected: Common not Private translations552worker2 # [ 0.005270] CPU features: detected: Data cache clean to Point of Persistence553client # [ 0.005113] CPU features: detected: Data cache clean to Point of Persistence554lb # [ 0.005153] CPU features: detected: Data cache clean to Point of Deep Persistence555worker1 # [ 0.005137] CPU features: detected: CRC32 instructions556worker2 # [ 0.005273] CPU features: detected: Data independent timing control (DIT)557client # [ 0.005116] CPU features: detected: Data independent timing control (DIT)558lb # [ 0.005156] CPU features: detected: Data cache clean to Point of Persistence559worker2 # [ 0.005276] CPU features: detected: E0PD560worker1 # [ 0.005140] CPU features: detected: Data cache clean to Point of Deep Persistence561client # [ 0.005119] CPU features: detected: E0PD562lb # [ 0.005159] CPU features: detected: Data independent timing control (DIT)563worker2 # [ 0.005279] CPU features: detected: Enhanced Counter Virtualization564client # [ 0.005121] CPU features: detected: Enhanced Counter Virtualization565worker1 # [ 0.005144] CPU features: detected: Data cache clean to Point of Persistence566lb # [ 0.005163] CPU features: detected: E0PD567worker2 # [ 0.005282] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)568worker1 # [ 0.005147] CPU features: detected: Data independent timing control (DIT)569client # [ 0.005124] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)570lb # [ 0.005165] CPU features: detected: Enhanced Counter Virtualization571worker1 # [ 0.005150] CPU features: detected: E0PD572worker2 # [ 0.005285] CPU features: detected: Enhanced Virtualization Traps573client # [ 0.005128] CPU features: detected: Enhanced Virtualization Traps574lb # [ 0.005168] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)575worker2 # [ 0.005288] CPU features: detected: Fine Grained Traps576worker1 # [ 0.005153] CPU features: detected: Enhanced Counter Virtualization577client # [ 0.005131] CPU features: detected: Fine Grained Traps578lb # [ 0.005172] CPU features: detected: Enhanced Virtualization Traps579worker1 # [ 0.005156] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)580worker2 # [ 0.005292] CPU features: detected: Generic authentication (architected QARMA5 algorithm)581lb # [ 0.005175] CPU features: detected: Fine Grained Traps582client # [ 0.005134] CPU features: detected: Generic authentication (architected QARMA5 algorithm)583worker1 # [ 0.005159] CPU features: detected: Enhanced Virtualization Traps584worker2 # [ 0.005297] CPU features: detected: RCpc load-acquire (LDAPR)585client # [ 0.005138] CPU features: detected: RCpc load-acquire (LDAPR)586lb # [ 0.005178] CPU features: detected: Generic authentication (architected QARMA5 algorithm)587worker1 # [ 0.005162] CPU features: detected: Fine Grained Traps588worker2 # [ 0.005300] CPU features: detected: LSE atomic instructions589client # [ 0.005141] CPU features: detected: LSE atomic instructions590lb # [ 0.005183] CPU features: detected: RCpc load-acquire (LDAPR)591worker2 # [ 0.005303] CPU features: detected: Privileged Access Never592worker1 # [ 0.005166] CPU features: detected: Generic authentication (architected QARMA5 algorithm)593client # [ 0.005144] CPU features: detected: Privileged Access Never594worker2 # [ 0.005306] CPU features: detected: PMUv3595lb # [ 0.005186] CPU features: detected: LSE atomic instructions596client # [ 0.005147] CPU features: detected: PMUv3597worker1 # [ 0.005172] CPU features: detected: RCpc load-acquire (LDAPR)598worker2 # [ 0.005309] CPU features: detected: RAS Extension Support599lb # [ 0.005189] CPU features: detected: Privileged Access Never600client # [ 0.005149] CPU features: detected: RAS Extension Support601worker1 # [ 0.005175] CPU features: detected: LSE atomic instructions602lb # [ 0.005192] CPU features: detected: PMUv3603worker2 # [ 0.005312] CPU features: detected: RASv1p1 Extension Support604client # [ 0.005152] CPU features: detected: RASv1p1 Extension Support605worker1 # [ 0.005178] CPU features: detected: Privileged Access Never606lb # [ 0.005194] CPU features: detected: RAS Extension Support607worker2 # [ 0.005314] CPU features: detected: Random Number Generator608worker1 # [ 0.005180] CPU features: detected: PMUv3609client # [ 0.005155] CPU features: detected: Random Number Generator610lb # [ 0.005197] CPU features: detected: RASv1p1 Extension Support611worker2 # [ 0.005317] CPU features: detected: Speculation barrier (SB)612worker1 # [ 0.005183] CPU features: detected: RAS Extension Support613client # [ 0.005157] CPU features: detected: Speculation barrier (SB)614lb # [ 0.005200] CPU features: detected: Random Number Generator615worker2 # [ 0.005320] CPU features: detected: Stage-2 Force Write-Back616worker1 # [ 0.005186] CPU features: detected: RASv1p1 Extension Support617client # [ 0.005160] CPU features: detected: Stage-2 Force Write-Back618lb # [ 0.005203] CPU features: detected: Speculation barrier (SB)619worker2 # [ 0.005323] CPU features: detected: TLB range maintenance instructions620worker1 # [ 0.005189] CPU features: detected: Random Number Generator621client # [ 0.005162] CPU features: detected: TLB range maintenance instructions622lb # [ 0.005205] CPU features: detected: Stage-2 Force Write-Back623worker1 # [ 0.005192] CPU features: detected: Speculation barrier (SB)624worker2 # [ 0.005327] CPU features: detected: Speculative Store Bypassing Safe (SSBS)625client # [ 0.005167] CPU features: detected: Speculative Store Bypassing Safe (SSBS)626lb # [ 0.005208] CPU features: detected: TLB range maintenance instructions627worker1 # [ 0.005194] CPU features: detected: Stage-2 Force Write-Back628worker2 # [ 0.005364] alternatives: applying system-wide alternatives629client # [ 0.005205] alternatives: applying system-wide alternatives630lb # [ 0.005213] CPU features: detected: Speculative Store Bypassing Safe (SSBS)631worker1 # [ 0.005197] CPU features: detected: TLB range maintenance instructions632worker2 # [ 0.008296] CPU features: detected: BBM Level 2 without TLB conflict abort633client # [ 0.008296] CPU features: detected: BBM Level 2 without TLB conflict abort634lb # [ 0.005249] alternatives: applying system-wide alternatives635worker1 # [ 0.005202] CPU features: detected: Speculative Store Bypassing Safe (SSBS)636lb # [ 0.008219] CPU features: detected: BBM Level 2 without TLB conflict abort637worker1 # [ 0.005239] alternatives: applying system-wide alternatives638worker2 # [ 0.008439] Memory: 1405688K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)639client # [ 0.008463] Memory: 1405604K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)640worker1 # [ 0.008168] CPU features: detected: BBM Level 2 without TLB conflict abort641worker2 # [ 0.008831] devtmpfs: initialized642lb # [ 0.008446] Memory: 1405700K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)643worker2 # [ 0.010708] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)644worker1 # [ 0.008321] Memory: 1405628K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)645lb # [ 0.008824] devtmpfs: initialized646worker2 # [ 0.010731] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).647lb # [ 0.010585] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)648worker1 # [ 0.008691] devtmpfs: initialized649worker2 # [ 0.010929] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL650worker1 # [ 0.010569] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)651lb # [ 0.010607] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).652worker2 # [ 0.010933] 0 pages in range for non-PLT usage653worker2 # [ 0.010934] 508272 pages in range for PLT usage654lb # [ 0.010803] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL655worker1 # [ 0.010593] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).656worker2 # [ 0.011048] pinctrl core: initialized pinctrl subsystem657lb # [ 0.010807] 0 pages in range for non-PLT usage658worker1 # [ 0.010795] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL659worker2 # [ 0.011784] DMI not present or invalid.660lb # [ 0.010809] 508272 pages in range for PLT usage661worker1 # [ 0.010799] 0 pages in range for non-PLT usage662worker2 # [ 0.015168] NET: Registered PF_NETLINK/PF_ROUTE protocol family663lb # [ 0.011006] pinctrl core: initialized pinctrl subsystem664worker1 # [ 0.010800] 508272 pages in range for PLT usage665lb # [ 0.012080] DMI not present or invalid.666worker2 # [ 0.017598] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations667worker1 # [ 0.010928] pinctrl core: initialized pinctrl subsystem668lb # [ 0.015196] NET: Registered PF_NETLINK/PF_ROUTE protocol family669worker1 # [ 0.011697] DMI not present or invalid.670client # [ 0.008885] devtmpfs: initialized671worker2 # [ 0.017792] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations672lb # [ 0.017494] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations673worker1 # [ 0.015107] NET: Registered PF_NETLINK/PF_ROUTE protocol family674client # [ 0.010692] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)675worker2 # [ 0.017989] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations676lb # [ 0.017673] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations677worker1 # [ 0.017448] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations678worker2 # [ 0.018010] audit: initializing netlink subsys (disabled)679client # [ 0.010715] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).680lb # [ 0.017869] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations681worker1 # [ 0.017642] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations682worker2 # [ 0.018614] thermal_sys: Registered thermal governor 'fair_share'683client # [ 0.010895] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL684lb # [ 0.017888] audit: initializing netlink subsys (disabled)685worker2 # [ 0.018616] thermal_sys: Registered thermal governor 'bang_bang'686client # [ 0.010899] 0 pages in range for non-PLT usage687worker1 # [ 0.017840] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations688lb # [ 0.018473] thermal_sys: Registered thermal governor 'fair_share'689client # [ 0.010900] 508272 pages in range for PLT usage690worker1 # [ 0.017862] audit: initializing netlink subsys (disabled)691worker2 # [ 0.018620] thermal_sys: Registered thermal governor 'step_wise'692client # [ 0.011010] pinctrl core: initialized pinctrl subsystem693lb # [ 0.018475] thermal_sys: Registered thermal governor 'bang_bang'694worker1 # [ 0.018465] thermal_sys: Registered thermal governor 'fair_share'695worker2 # [ 0.018622] thermal_sys: Registered thermal governor 'user_space'696client # [ 0.011744] DMI not present or invalid.697lb # [ 0.018478] thermal_sys: Registered thermal governor 'step_wise'698worker1 # [ 0.018467] thermal_sys: Registered thermal governor 'bang_bang'699worker2 # [ 0.018628] thermal_sys: Registered thermal governor 'power_allocator'700client # [ 0.015133] NET: Registered PF_NETLINK/PF_ROUTE protocol family701lb # [ 0.018481] thermal_sys: Registered thermal governor 'user_space'702worker1 # [ 0.018471] thermal_sys: Registered thermal governor 'step_wise'703worker2 # [ 0.018654] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1704client # [ 0.017480] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations705lb # [ 0.018486] thermal_sys: Registered thermal governor 'power_allocator'706worker1 # [ 0.018474] thermal_sys: Registered thermal governor 'user_space'707worker2 # [ 0.018663] cpuidle: using governor ladder708client # [ 0.017677] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations709worker2 # [ 0.018668] cpuidle: using governor menu710worker1 # [ 0.018478] thermal_sys: Registered thermal governor 'power_allocator'711lb # [ 0.018511] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1712lb # [ 0.018519] cpuidle: using governor ladder713worker2 # [ 0.018870] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.714client # [ 0.017874] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations715worker1 # [ 0.018503] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1716lb # [ 0.018525] cpuidle: using governor menu717worker2 # [ 0.018886] ASID allocator initialised with 65536 entries718client # [ 0.017899] audit: initializing netlink subsys (disabled)719worker1 # [ 0.018511] cpuidle: using governor ladder720worker2 # [ 0.020207] Serial: AMBA PL011 UART driver721lb # [ 0.018703] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.722worker1 # [ 0.018517] cpuidle: using governor menu723client # [ 0.018525] thermal_sys: Registered thermal governor 'fair_share'724lb # [ 0.018718] ASID allocator initialised with 65536 entries725worker2 # [ 0.025399] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1726worker1 # [ 0.018711] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.727client # [ 0.018527] thermal_sys: Registered thermal governor 'bang_bang'728lb # [ 0.020046] Serial: AMBA PL011 UART driver729worker2 # [ 0.025525] printk: console [ttyAMA0] enabled730worker1 # [ 0.018725] ASID allocator initialised with 65536 entries731client # [ 0.018531] thermal_sys: Registered thermal governor 'step_wise'732worker1 # [ 0.019939] Serial: AMBA PL011 UART driver733lb # [ 0.025374] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1734client # [ 0.018533] thermal_sys: Registered thermal governor 'user_space'735lb # [ 0.025509] printk: console [ttyAMA0] enabled736worker1 # [ 0.025145] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1737client # [ 0.018539] thermal_sys: Registered thermal governor 'power_allocator'738worker1 # [ 0.025270] printk: console [ttyAMA0] enabled739client # [ 0.018564] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1740client # [ 0.018573] cpuidle: using governor ladder741client # [ 0.018579] cpuidle: using governor menu742client # [ 0.018765] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.743client # [ 0.018780] ASID allocator initialised with 65536 entries744client # [ 0.019979] Serial: AMBA PL011 UART driver745client # [ 0.025269] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1746worker2 # [ 0.149050] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages747client # [ 0.025417] printk: console [ttyAMA0] enabled748worker2 # [ 0.149067] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page749lb # [ 0.149336] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages750worker2 # [ 0.149072] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages751client # [ 0.151557] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages752worker1 # [ 0.148222] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages753lb # [ 0.149354] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page754worker2 # [ 0.149077] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page755client # [ 0.151576] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page756worker1 # [ 0.148238] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page757lb # [ 0.149359] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages758worker2 # [ 0.149081] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages759client # [ 0.151582] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages760worker1 # [ 0.148244] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages761lb # [ 0.149363] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page762worker2 # [ 0.149086] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page763client # [ 0.151587] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page764worker1 # [ 0.148248] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page765lb # [ 0.149368] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages766worker2 # [ 0.149090] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages767client # [ 0.151591] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages768worker1 # [ 0.148253] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages769lb # [ 0.149373] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page770worker2 # [ 0.149095] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page771client # [ 0.151596] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page772worker1 # [ 0.148258] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page773lb # [ 0.149377] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages774client # [ 0.151601] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages775worker1 # [ 0.148262] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages776lb # [ 0.149382] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page777client # [ 0.151606] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page778worker2 # [ 0.156853] fbcon: Taking over console779worker1 # [ 0.148267] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page780worker2 # [ 0.156868] ACPI: Interpreter disabled.781lb # [ 0.157228] fbcon: Taking over console782lb # [ 0.157243] ACPI: Interpreter disabled.783client # [ 0.159330] fbcon: Taking over console784worker1 # [ 0.155948] fbcon: Taking over console785client # [ 0.159345] ACPI: Interpreter disabled.786worker1 # [ 0.155962] ACPI: Interpreter disabled.787client # [ 0.161246] iommu: Default domain type: Translated788client # [ 0.161256] iommu: DMA domain TLB invalidation policy: strict mode789client # [ 0.162990] SCSI subsystem initialized790worker2 # [ 0.165447] iommu: Default domain type: Translated791worker2 # [ 0.165458] iommu: DMA domain TLB invalidation policy: strict mode792worker2 # [ 0.165863] SCSI subsystem initialized793lb # [ 0.165836] iommu: Default domain type: Translated794worker1 # [ 0.164504] iommu: Default domain type: Translated795lb # [ 0.165847] iommu: DMA domain TLB invalidation policy: strict mode796worker2 # [ 0.167922] usbcore: registered new interface driver usbfs797lb # [ 0.166222] SCSI subsystem initialized798worker1 # [ 0.164514] iommu: DMA domain TLB invalidation policy: strict mode799worker2 # [ 0.167957] usbcore: registered new interface driver hub800worker1 # [ 0.164896] SCSI subsystem initialized801worker2 # [ 0.167974] usbcore: registered new device driver usb802lb # [ 0.168333] usbcore: registered new interface driver usbfs803client # [ 0.170674] usbcore: registered new interface driver usbfs804worker2 # [ 0.168256] pps_core: LinuxPPS API ver. 1 registered805worker1 # [ 0.166958] usbcore: registered new interface driver usbfs806lb # [ 0.168365] usbcore: registered new interface driver hub807client # [ 0.170709] usbcore: registered new interface driver hub808worker1 # [ 0.166990] usbcore: registered new interface driver hub809lb # [ 0.168387] usbcore: registered new device driver usb810worker2 # [ 0.168262] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>811client # [ 0.170741] usbcore: registered new device driver usb812lb # [ 0.168667] pps_core: LinuxPPS API ver. 1 registered813worker1 # [ 0.167006] usbcore: registered new device driver usb814worker2 # [ 0.168272] PTP clock support registered815client # [ 0.171022] pps_core: LinuxPPS API ver. 1 registered816worker2 # [ 0.168327] EDAC MC: Ver: 3.0.0817worker1 # [ 0.167280] pps_core: LinuxPPS API ver. 1 registered818lb # [ 0.168673] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>819worker2 # [ 0.173136] scmi_core: SCMI protocol bus registered820client # [ 0.171028] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>821lb # [ 0.168691] PTP clock support registered822worker1 # [ 0.167286] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>823lb # [ 0.168740] EDAC MC: Ver: 3.0.0824client # [ 0.171038] PTP clock support registered825worker1 # [ 0.167296] PTP clock support registered826client # [ 0.171078] EDAC MC: Ver: 3.0.0827worker2 # [ 0.174126] FPGA manager framework828worker1 # [ 0.167361] EDAC MC: Ver: 3.0.0829lb # [ 0.173515] scmi_core: SCMI protocol bus registered830worker2 # [ 0.175090] vgaarb: loaded831client # [ 0.175859] scmi_core: SCMI protocol bus registered832worker1 # [ 0.172123] scmi_core: SCMI protocol bus registered833lb # [ 0.174529] FPGA manager framework834worker2 # [ 0.175731] clocksource: Switched to clocksource arch_sys_counter835client # [ 0.176851] FPGA manager framework836lb # [ 0.175473] vgaarb: loaded837worker1 # [ 0.173147] FPGA manager framework838client # [ 0.177789] vgaarb: loaded839worker1 # [ 0.174134] vgaarb: loaded840lb # [ 0.176122] clocksource: Switched to clocksource arch_sys_counter841worker1 # [ 0.174766] clocksource: Switched to clocksource arch_sys_counter842client # [ 0.178458] clocksource: Switched to clocksource arch_sys_counter843client # [ 0.179853] VFS: Disk quotas dquot_6.6.0844client # [ 0.179884] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)845client # [ 0.181557] netfs: FS-Cache loaded846client # [ 0.181653] pnp: PnP ACPI: disabled847worker2 # [ 0.182056] VFS: Disk quotas dquot_6.6.0848worker2 # [ 0.182100] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)849lb # [ 0.182376] VFS: Disk quotas dquot_6.6.0850worker2 # [ 0.183790] netfs: FS-Cache loaded851worker1 # [ 0.181153] VFS: Disk quotas dquot_6.6.0852worker2 # [ 0.183921] pnp: PnP ACPI: disabled853lb # [ 0.182420] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)854lb # [ 0.184119] netfs: FS-Cache loaded855worker1 # [ 0.181199] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)856lb # [ 0.184718] pnp: PnP ACPI: disabled857worker1 # [ 0.182913] netfs: FS-Cache loaded858worker1 # [ 0.183029] pnp: PnP ACPI: disabled859client # [ 0.187669] NET: Registered PF_INET protocol family860client # [ 0.187942] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)861worker2 # [ 0.187827] NET: Registered PF_INET protocol family862worker2 # [ 0.188135] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)863lb # [ 0.188289] NET: Registered PF_INET protocol family864worker1 # [ 0.186906] NET: Registered PF_INET protocol family865lb # [ 0.188556] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)866worker1 # [ 0.187175] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)867client # [ 0.217876] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)868client # [ 0.217928] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)869client # [ 0.217956] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)870client # [ 0.218027] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)871client # [ 0.218171] TCP: Hash tables configured (established 16384 bind 16384)872client # [ 0.218261] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)873client # [ 0.218297] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)874worker2 # [ 0.217374] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)875client # [ 0.218333] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)876worker2 # [ 0.217422] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)877client # [ 0.218446] NET: Registered PF_UNIX/PF_LOCAL protocol family878lb # [ 0.218254] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)879client # [ 0.218465] NET: Registered PF_XDP protocol family880worker2 # [ 0.217453] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)881worker1 # [ 0.216983] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)882lb # [ 0.218302] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)883worker2 # [ 0.217527] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)884worker1 # [ 0.217035] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)885lb # [ 0.218334] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)886worker2 # [ 0.217670] TCP: Hash tables configured (established 16384 bind 16384)887worker1 # [ 0.217067] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)888lb # [ 0.218413] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)889worker2 # [ 0.217762] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)890worker1 # [ 0.217142] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)891lb # [ 0.218558] TCP: Hash tables configured (established 16384 bind 16384)892worker2 # [ 0.217798] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)893worker1 # [ 0.217285] TCP: Hash tables configured (established 16384 bind 16384)894lb # [ 0.218642] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)895worker2 # [ 0.217832] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)896worker1 # [ 0.217371] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)897lb # [ 0.218677] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)898worker2 # [ 0.217924] NET: Registered PF_UNIX/PF_LOCAL protocol family899worker1 # [ 0.217406] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)900lb # [ 0.218714] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)901worker2 # [ 0.217952] NET: Registered PF_XDP protocol family902worker2 # [ 0.217969] PCI: CLS 0 bytes, default 64903worker1 # [ 0.217441] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)904lb # [ 0.218835] NET: Registered PF_UNIX/PF_LOCAL protocol family905worker2 # [ 0.218201] Trying to unpack rootfs image as initramfs...906worker1 # [ 0.217539] NET: Registered PF_UNIX/PF_LOCAL protocol family907lb # [ 0.218973] NET: Registered PF_XDP protocol family908client # [ 0.227728] PCI: CLS 0 bytes, default 64909lb # [ 0.218992] PCI: CLS 0 bytes, default 64910worker1 # [ 0.217594] NET: Registered PF_XDP protocol family911client # [ 0.227993] Trying to unpack rootfs image as initramfs...912worker1 # [ 0.217612] PCI: CLS 0 bytes, default 64913lb # [ 0.219228] Trying to unpack rootfs image as initramfs...914worker1 # [ 0.217865] Trying to unpack rootfs image as initramfs...915worker2 # [ 0.233968] kvm [1]: HYP mode not available916lb # [ 0.234253] kvm [1]: HYP mode not available917client # [ 0.235730] kvm [1]: HYP mode not available918worker1 # [ 0.232870] kvm [1]: HYP mode not available919client # [ 0.327025] Initialise system trusted keyrings920client # [ 0.327791] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker2 # [ 0.327364] Initialise system trusted keyrings922client # [ 0.329099] squashfs: version 4.0 (2009/01/31) Phillip Lougher923worker2 # [ 0.328207] workingset: timestamp_bits=42 max_order=19 bucket_order=0924lb # [ 0.328423] Initialise system trusted keyrings925worker2 # [ 0.329495] squashfs: version 4.0 (2009/01/31) Phillip Lougher926client # [ 0.329916] 9p: Installing v9fs 9p2000 file system support927lb # [ 0.329211] workingset: timestamp_bits=42 max_order=19 bucket_order=0928worker1 # [ 0.327492] Initialise system trusted keyrings929worker1 # [ 0.328279] workingset: timestamp_bits=42 max_order=19 bucket_order=0930lb # [ 0.330557] squashfs: version 4.0 (2009/01/31) Phillip Lougher931worker2 # [ 0.330320] 9p: Installing v9fs 9p2000 file system support932worker1 # [ 0.329735] squashfs: version 4.0 (2009/01/31) Phillip Lougher933lb # [ 0.331451] 9p: Installing v9fs 9p2000 file system support934worker1 # [ 0.330535] 9p: Installing v9fs 9p2000 file system support935worker2 # [ 0.354399] Key type asymmetric registered936worker2 # [ 0.354423] Asymmetric key parser 'x509' registered937worker2 # [ 0.354491] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)938client # [ 0.358916] Key type asymmetric registered939client # [ 0.358941] Asymmetric key parser 'x509' registered940client # [ 0.359018] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)941worker2 # [ 0.359786] io scheduler mq-deadline registered942worker2 # [ 0.359801] io scheduler kyber registered943client # [ 0.361224] io scheduler mq-deadline registered944client # [ 0.361235] io scheduler kyber registered945lb # [ 0.360271] Key type asymmetric registered946lb # [ 0.360296] Asymmetric key parser 'x509' registered947worker1 # [ 0.359337] Key type asymmetric registered948lb # [ 0.360373] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)949worker1 # [ 0.359362] Asymmetric key parser 'x509' registered950lb # [ 0.362565] io scheduler mq-deadline registered951worker1 # [ 0.359439] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)952lb # [ 0.362576] io scheduler kyber registered953worker1 # [ 0.361699] io scheduler mq-deadline registered954worker1 # [ 0.361710] io scheduler kyber registered955client # [ 0.370601] pl061_gpio 9030000.pl061: PL061 GPIO chip registered956client # [ 0.372005] ledtrig-cpu: registered to indicate activity on CPUs957lb # [ 0.372266] pl061_gpio 9030000.pl061: PL061 GPIO chip registered958worker1 # [ 0.370895] pl061_gpio 9030000.pl061: PL061 GPIO chip registered959client # [ 0.372389] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:960client # [ 0.372407] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000961client # [ 0.372428] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000962lb # [ 0.373735] ledtrig-cpu: registered to indicate activity on CPUs963client # [ 0.372437] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000964lb # [ 0.374147] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:965client # [ 0.372459] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits966lb # [ 0.374165] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000967worker2 # [ 0.371884] pl061_gpio 9030000.pl061: PL061 GPIO chip registered968client # [ 0.372497] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]969lb # [ 0.374178] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000970worker2 # [ 0.372558] ledtrig-cpu: registered to indicate activity on CPUs971client # [ 0.372584] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00972worker2 # [ 0.372932] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:973lb # [ 0.374187] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000974worker1 # [ 0.372341] ledtrig-cpu: registered to indicate activity on CPUs975client # [ 0.372593] pci_bus 0000:00: root bus resource [bus 00-ff]976worker2 # [ 0.372963] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000977lb # [ 0.374208] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits978client # [ 0.372600] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]979worker1 # [ 0.372725] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:980client # [ 0.372605] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]981lb # [ 0.374232] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]982worker2 # [ 0.372976] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000983worker1 # [ 0.372743] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000984client # [ 0.372611] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]985lb # [ 0.374301] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00986worker2 # [ 0.372985] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000987worker1 # [ 0.372755] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000988lb # [ 0.374311] pci_bus 0000:00: root bus resource [bus 00-ff]989client # [ 0.372666] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint990worker2 # [ 0.373018] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits991worker1 # [ 0.372765] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000992lb # [ 0.374317] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]993client # [ 0.373171] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint994lb # [ 0.374322] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]995worker2 # [ 0.373042] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]996worker1 # [ 0.372793] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits997client # [ 0.373360] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]998lb # [ 0.374328] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]999client # [ 0.373378] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1000worker2 # [ 0.373112] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001001worker1 # [ 0.372817] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1002worker2 # [ 0.373130] pci_bus 0000:00: root bus resource [bus 00-ff]1003client # [ 0.373408] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1004lb # [ 0.374386] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1005worker1 # [ 0.372928] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001006worker2 # [ 0.373136] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1007client # [ 0.373425] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1008worker1 # [ 0.372937] pci_bus 0000:00: root bus resource [bus 00-ff]1009lb # [ 0.374839] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1010worker2 # [ 0.373142] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1011lb # [ 0.375066] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1012worker1 # [ 0.372944] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1013client # [ 0.373894] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1014worker2 # [ 0.373147] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1015lb # [ 0.375085] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1016client # [ 0.374079] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1017worker1 # [ 0.372950] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1018client # [ 0.374096] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1019lb # [ 0.375117] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1020worker2 # [ 0.373249] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1021worker1 # [ 0.372955] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1022lb # [ 0.375134] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1023client # [ 0.374127] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1024worker2 # [ 0.373700] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1025worker1 # [ 0.373009] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1026lb # [ 0.375627] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1027worker2 # [ 0.373892] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1028worker1 # [ 0.373460] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1029lb # [ 0.375819] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1030worker2 # [ 0.373909] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1031worker1 # [ 0.373652] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1032lb # [ 0.375835] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1033worker2 # [ 0.373940] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1034worker1 # [ 0.373669] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1035lb # [ 0.375866] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1036worker2 # [ 0.373958] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1037worker1 # [ 0.373700] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1038worker2 # [ 0.374428] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1039worker1 # [ 0.373716] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1040lb # [ 0.395930] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1041worker2 # [ 0.374618] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1042worker1 # [ 0.374183] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1043worker2 # [ 0.374634] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1044worker1 # [ 0.374372] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1045worker2 # [ 0.374665] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046worker1 # [ 0.374388] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1047worker2 # [ 0.375139] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1048worker1 # [ 0.374418] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049worker2 # [ 0.375329] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1050worker1 # [ 0.374909] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1051worker2 # [ 0.375345] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1052worker1 # [ 0.375098] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1053worker2 # [ 0.375376] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054worker1 # [ 0.375114] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1055worker2 # [ 0.375870] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1056worker1 # [ 0.375144] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057worker2 # [ 0.376072] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1058worker1 # [ 0.375606] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1059worker2 # [ 0.376088] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1060worker1 # [ 0.375796] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1061worker2 # [ 0.376118] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1062worker1 # [ 0.375812] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1063worker2 # [ 0.376138] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1064worker1 # [ 0.375842] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065worker2 # [ 0.376613] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1066worker1 # [ 0.375861] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1067worker2 # [ 0.376814] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1068worker1 # [ 0.376328] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1069worker2 # [ 0.376845] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070worker1 # [ 0.376522] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1071client # [ 0.402677] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1072client # [ 0.402886] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1073worker2 # [ 0.377311] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1074worker1 # [ 0.376552] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1075client # [ 0.402905] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1076worker2 # [ 0.377504] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1077worker1 # [ 0.377080] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1078client # [ 0.402936] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1079worker2 # [ 0.377534] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1080worker1 # [ 0.377274] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1081lb # [ 0.404165] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1082client # [ 0.403428] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1083worker2 # [ 0.377933] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1084worker1 # [ 0.377305] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085lb # [ 0.404205] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1086client # [ 0.403619] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1087worker2 # [ 0.378120] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1088worker1 # [ 0.377724] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1089lb # [ 0.404238] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1090client # [ 0.403636] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1091worker2 # [ 0.378407] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1092worker1 # [ 0.377911] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1093client # [ 0.403666] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094lb # [ 0.404745] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1095worker2 # [ 0.378601] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1096lb # [ 0.404941] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1097client # [ 0.403685] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1098worker1 # [ 0.378155] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1099worker2 # [ 0.378631] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100lb # [ 0.404957] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1101worker1 # [ 0.378348] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1102client # [ 0.404152] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1103worker2 # [ 0.379093] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1104worker1 # [ 0.378378] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1105lb # [ 0.404988] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1106client # [ 0.404344] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1107worker2 # [ 0.379286] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1108lb # [ 0.405007] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1109client # [ 0.404375] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1110worker2 # [ 0.379317] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111lb # [ 0.405478] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1112client # [ 0.404900] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1113lb # [ 0.405673] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1114client # [ 0.405095] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1115lb # [ 0.405704] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1116client # [ 0.405126] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1117lb # [ 0.406208] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1118client # [ 0.405531] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1119lb # [ 0.406401] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1120client # [ 0.405718] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1121lb # [ 0.406432] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122client # [ 0.406010] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1123lb # [ 0.406831] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1124client # [ 0.406201] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1125lb # [ 0.407030] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1126client # [ 0.406232] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127lb # [ 0.407290] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1128client # [ 0.406711] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1129lb # [ 0.407485] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1130client # [ 0.406904] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1131lb # [ 0.407516] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1132client # [ 0.406935] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133lb # [ 0.407983] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1134client # [ 0.407395] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1135lb # [ 0.408193] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1136client # [ 0.407595] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1137lb # [ 0.408224] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1138client # [ 0.407627] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1139worker2 # [ 0.427873] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1140lb # [ 0.408687] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1141client # [ 0.408094] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1142worker2 # [ 0.428192] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1143lb # [ 0.408881] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1144client # [ 0.408378] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1145worker2 # [ 0.428225] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1146lb # [ 0.408912] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147client # [ 0.408396] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1148worker2 # [ 0.428739] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1149client # [ 0.408427] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1150lb # [ 0.409394] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1151worker2 # [ 0.429032] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1152lb # [ 0.409683] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1153client # [ 0.408905] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1154worker2 # [ 0.429049] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1155lb # [ 0.409701] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1156client # [ 0.409100] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1157worker2 # [ 0.429080] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158worker1 # [ 0.426924] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1159lb # [ 0.409732] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1160client # [ 0.409117] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1161worker1 # [ 0.427141] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1162worker2 # [ 0.429551] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1163lb # [ 0.410215] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1164client # [ 0.409147] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1165worker2 # [ 0.429741] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1166worker1 # [ 0.427174] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167lb # [ 0.410406] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1168client # [ 0.409737] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1169worker2 # [ 0.429757] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1170lb # [ 0.410422] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1171worker1 # [ 0.427687] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1172client # [ 0.409749] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1173worker2 # [ 0.429788] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1174worker1 # [ 0.427881] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1175lb # [ 0.410453] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1176client # [ 0.409754] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1177worker2 # [ 0.430372] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1178lb # [ 0.411110] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1179worker1 # [ 0.427912] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180client # [ 0.409800] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1181worker2 # [ 0.430384] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1182lb # [ 0.411122] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1183worker1 # [ 0.428390] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1184worker1 # [ 0.428648] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1185client # [ 0.409849] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1186worker2 # [ 0.430389] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1187lb # [ 0.411128] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1188worker1 # [ 0.428671] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1189client # [ 0.409896] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1190worker2 # [ 0.430440] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1191lb # [ 0.411174] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1192worker1 # [ 0.428701] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1193client # [ 0.409944] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1194worker2 # [ 0.430488] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1195lb # [ 0.411222] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1196worker1 # [ 0.429184] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1197client # [ 0.409993] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1198worker2 # [ 0.430537] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1199worker1 # [ 0.429374] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1200lb # [ 0.411270] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1201worker1 # [ 0.429390] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1202client # [ 0.410041] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1203worker2 # [ 0.430587] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1204lb # [ 0.411317] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1205worker1 # [ 0.429420] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1206client # [ 0.410088] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1207worker2 # [ 0.430635] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1208lb # [ 0.411366] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1209worker1 # [ 0.430003] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1210client # [ 0.410135] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1211worker2 # [ 0.430684] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1212lb # [ 0.411413] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1213worker1 # [ 0.430015] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1214client # [ 0.410183] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1215worker2 # [ 0.430734] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1216lb # [ 0.411461] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1217worker1 # [ 0.430021] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1218client # [ 0.410258] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1219worker2 # [ 0.430782] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1220lb # [ 0.411508] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1221worker1 # [ 0.430068] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1222client # [ 0.410305] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1223worker2 # [ 0.430830] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1224lb # [ 0.411556] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1225worker1 # [ 0.430117] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1226client # [ 0.410327] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1227worker2 # [ 0.430903] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1228lb # [ 0.411625] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1229worker1 # [ 0.430167] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1230client # [ 0.410349] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1231worker2 # [ 0.430951] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1232lb # [ 0.411673] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1233client # [ 0.410371] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1234worker1 # [ 0.430217] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1235worker2 # [ 0.430973] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1236lb # [ 0.411699] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1237client # [ 0.410392] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1238worker1 # [ 0.430266] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1239worker2 # [ 0.430995] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1240lb # [ 0.411723] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1241client # [ 0.410414] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1242worker2 # [ 0.431017] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1243worker1 # [ 0.430315] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1244lb # [ 0.411745] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1245client # [ 0.410436] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1246worker2 # [ 0.431039] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1247lb # [ 0.411767] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1248worker1 # [ 0.430365] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1249client # [ 0.458294] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1250worker2 # [ 0.431061] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1251lb # [ 0.411790] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1252worker1 # [ 0.430416] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1253client # [ 0.458324] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1254worker2 # [ 0.431083] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1255lb # [ 0.411812] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1256worker1 # [ 0.430465] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1257client # [ 0.458347] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1258worker2 # [ 0.431111] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1259lb # [ 0.411841] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1260client # [ 0.458369] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1261worker1 # [ 0.430537] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1262worker2 # [ 0.431133] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1263lb # [ 0.411864] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1264client # [ 0.458392] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1265worker1 # [ 0.430586] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1266worker2 # [ 0.431159] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1267lb # [ 0.411890] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1268client # [ 0.458414] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1269worker1 # [ 0.430609] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1270worker2 # [ 0.431182] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1271lb # [ 0.411916] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1272client # [ 0.458436] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1273worker1 # [ 0.430631] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1274worker2 # [ 0.431204] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1275client # [ 0.458457] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1276lb # [ 0.411944] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1277worker1 # [ 0.430654] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1278worker2 # [ 0.431226] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1279lb # [ 0.411970] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1280worker1 # [ 0.430676] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1281worker2 # [ 0.431248] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1282lb # [ 0.411995] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1283worker1 # [ 0.430699] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1284worker2 # [ 0.431270] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1285lb # [ 0.412021] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1286worker2 # [ 0.431292] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1287worker1 # [ 0.430721] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1288lb # [ 0.412044] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1289worker2 # [ 0.431313] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1290worker1 # [ 0.430750] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1291lb # [ 0.412066] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1292worker2 # [ 0.431335] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1293worker1 # [ 0.430794] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1294lb # [ 0.412088] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1295worker2 # [ 0.431361] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1296lb # [ 0.412114] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1297worker1 # [ 0.430817] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1298worker2 # [ 0.431371] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1299lb # [ 0.412123] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1300worker1 # [ 0.430840] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1301worker2 # [ 0.431376] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1302worker1 # [ 0.430863] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1303worker1 # [ 0.430886] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1304worker2 # [ 0.471288] pci 0000:00:07.0: enabling device (0000 -> 0002)1305worker1 # [ 0.430908] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1306worker1 # [ 0.430930] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1307client # [ 0.470544] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1308worker1 # [ 0.430952] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1309client # [ 0.470580] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1310worker1 # [ 0.430973] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1311client # [ 0.470603] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1312worker1 # [ 0.430996] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1313client # [ 0.470636] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1314worker1 # [ 0.431022] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1315client # [ 0.470646] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1316worker1 # [ 0.431032] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1317client # [ 0.470651] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1318worker1 # [ 0.431037] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1319client # [ 0.471500] pci 0000:00:07.0: enabling device (0000 -> 0002)1320worker1 # [ 0.431896] pci 0000:00:07.0: enabling device (0000 -> 0002)1321lb # [ 0.476178] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1322lb # [ 0.477071] pci 0000:00:07.0: enabling device (0000 -> 0002)1323worker2 # [ 0.495687] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1324client # [ 0.497552] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1325worker1 # [ 0.494207] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1326lb # [ 0.499367] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1327client # [ 0.504031] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1328client # [ 0.506024] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1329worker2 # [ 0.505119] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1330worker1 # [ 0.504201] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1331worker2 # [ 0.509068] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1332worker2 # [ 0.511198] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1333lb # [ 0.509580] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1334lb # [ 0.513337] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1335client # [ 0.515908] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1336lb # [ 0.515361] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1337client # [ 0.517958] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1338worker1 # [ 0.515009] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1339worker1 # [ 0.517370] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1340worker1 # [ 0.519627] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1341worker2 # [ 0.525319] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1342worker1 # [ 0.521682] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1343worker2 # [ 0.527385] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1344client # [ 0.528534] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1345lb # [ 0.525818] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1346lb # [ 0.527854] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1347worker2 # [ 0.530851] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1348lb # [ 0.531664] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1349client # [ 0.531420] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1350client # [ 0.533412] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1351worker2 # [ 0.533790] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1352worker1 # [ 0.531890] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1353worker1 # [ 0.533821] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1354lb # [ 0.538078] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1355lb # [ 0.540052] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1356client # [ 0.543626] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1357client # [ 0.545533] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1358worker1 # [ 0.544326] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1359worker1 # [ 0.546281] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1360worker2 # [ 0.543938] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1361worker2 # [ 0.545832] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1362lb # [ 0.550335] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1363worker2 # [ 0.548153] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1364worker1 # [ 0.550190] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1365client # [ 0.556497] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1366lb # [ 0.556812] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1367worker2 # [ 0.561187] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1368worker1 # [ 0.564015] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1369worker1 # [ 0.566572] msm_serial: driver initialized1370worker1 # [ 0.566709] SuperH (H)SCI(F) driver initialized1371worker1 # [ 0.566763] STM32 USART driver initialized1372worker2 # [ 0.569006] msm_serial: driver initialized1373lb # [ 0.569928] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1374worker2 # [ 0.569172] SuperH (H)SCI(F) driver initialized1375worker2 # [ 0.569227] STM32 USART driver initialized1376client # [ 0.567720] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1377client # [ 0.569531] msm_serial: driver initialized1378client # [ 0.569679] SuperH (H)SCI(F) driver initialized1379client # [ 0.569732] STM32 USART driver initialized1380lb # [ 0.572527] msm_serial: driver initialized1381lb # [ 0.572664] SuperH (H)SCI(F) driver initialized1382lb # [ 0.572721] STM32 USART driver initialized1383worker2 # [ 0.598819] loop: module loaded1384worker2 # [ 0.598997] virtio_blk virtio2: 1/0/0 default/read/poll queues1385worker2 # [ 0.601009] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1386client # [ 0.604710] loop: module loaded1387client # [ 0.604966] virtio_blk virtio2: 1/0/0 default/read/poll queues1388client # [ 0.605804] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1389worker1 # [ 0.603384] loop: module loaded1390lb # [ 0.605662] loop: module loaded1391worker1 # [ 0.603578] virtio_blk virtio2: 1/0/0 default/read/poll queues1392lb # [ 0.605855] virtio_blk virtio2: 1/0/0 default/read/poll queues1393worker1 # [ 0.604391] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1394lb # [ 0.606663] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1395client # [ 0.611064] megasas: 07.734.00.00-rc11396client # [ 0.611773] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1397worker2 # [ 0.607839] megasas: 07.734.00.00-rc11398client # [ 0.614248] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001399worker2 # [ 0.608680] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1400client # [ 0.614273] Intel/Sharp Extended Query Table at 0x00311401lb # [ 0.612730] megasas: 07.734.00.00-rc11402worker2 # [ 0.610685] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001403lb # [ 0.613448] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1404worker2 # [ 0.610710] Intel/Sharp Extended Query Table at 0x00311405worker2 # [ 0.612446] Using buffer write method1406worker2 # [ 0.612514] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1407worker2 # [ 0.614260] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408worker2 # [ 0.614316] Intel/Sharp Extended Query Table at 0x00311409client # [ 0.623470] Using buffer write method1410worker1 # [ 0.615383] megasas: 07.734.00.00-rc11411client # [ 0.623540] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1412worker1 # [ 0.616099] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1413client # [ 0.625253] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001414worker1 # [ 0.618068] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001415client # [ 0.625302] Intel/Sharp Extended Query Table at 0x00311416worker1 # [ 0.618094] Intel/Sharp Extended Query Table at 0x00311417worker1 # [ 0.619768] Using buffer write method1418worker1 # [ 0.619839] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1419lb # [ 0.624174] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001420worker1 # [ 0.621709] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001421worker1 # [ 0.621732] Intel/Sharp Extended Query Table at 0x00311422lb # [ 0.624198] Intel/Sharp Extended Query Table at 0x00311423worker2 # [ 0.633417] Using buffer write method1424lb # [ 0.626697] Using buffer write method1425worker2 # [ 0.633454] Concatenating MTD devices:1426worker2 # [ 0.633458] (0): "0.flash"1427worker2 # [ 0.633462] (1): "0.flash"1428lb # [ 0.626758] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1429worker2 # [ 0.633474] into device "0.flash"1430lb # [ 0.628663] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001431lb # [ 0.628722] Intel/Sharp Extended Query Table at 0x00311432lb # [ 0.630622] Using buffer write method1433lb # [ 0.630646] Concatenating MTD devices:1434client # [ 0.639108] Using buffer write method1435lb # [ 0.630650] (0): "0.flash"1436client # [ 0.639141] Concatenating MTD devices:1437lb # [ 0.630654] (1): "0.flash"1438client # [ 0.639146] (0): "0.flash"1439lb # [ 0.630663] into device "0.flash"1440client # [ 0.639150] (1): "0.flash"1441client # [ 0.639158] into device "0.flash"1442worker1 # [ 0.640793] Using buffer write method1443worker1 # [ 0.640828] Concatenating MTD devices:1444worker1 # [ 0.640833] (0): "0.flash"1445worker1 # [ 0.640837] (1): "0.flash"1446worker1 # [ 0.640845] into device "0.flash"1447worker2 # [ 0.891258] Freeing initrd memory: 26944K1448client # [ 0.892476] Freeing initrd memory: 26972K1449worker1 # [ 0.894143] Freeing initrd memory: 26976K1450worker2 # [ 0.897551] tun: Universal TUN/TAP device driver, 1.61451client # [ 0.898750] tun: Universal TUN/TAP device driver, 1.61452lb # [ 0.899140] Freeing initrd memory: 26948K1453worker1 # [ 0.900359] tun: Universal TUN/TAP device driver, 1.61454worker2 # [ 0.901523] thunder_xcv, ver 1.01455worker2 # [ 0.901562] thunder_bgx, ver 1.01456worker2 # [ 0.901589] nicpf, ver 1.01457worker2 # [ 0.902155] e1000: Intel(R) PRO/1000 Network Driver1458client # [ 0.902593] thunder_xcv, ver 1.01459lb # [ 0.905191] tun: Universal TUN/TAP device driver, 1.61460worker2 # [ 0.902164] e1000: Copyright (c) 1999-2006 Intel Corporation.1461client # [ 0.902633] thunder_bgx, ver 1.01462client # [ 0.902676] nicpf, ver 1.01463worker2 # [ 0.902190] e1000e: Intel(R) PRO/1000 Network Driver1464client # [ 0.903239] e1000: Intel(R) PRO/1000 Network Driver1465worker2 # [ 0.902199] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1466client # [ 0.903247] e1000: Copyright (c) 1999-2006 Intel Corporation.1467worker2 # [ 0.902234] igb: Intel(R) Gigabit Ethernet Network Driver1468client # [ 0.903270] e1000e: Intel(R) PRO/1000 Network Driver1469worker2 # [ 0.902239] igb: Copyright (c) 2007-2014 Intel Corporation.1470client # [ 0.903278] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1471worker2 # [ 0.902281] igbvf: Intel(R) Gigabit Virtual Function Network Driver1472worker1 # [ 0.904060] thunder_xcv, ver 1.01473client # [ 0.903308] igb: Intel(R) Gigabit Ethernet Network Driver1474worker1 # [ 0.904103] thunder_bgx, ver 1.01475worker2 # [ 0.902287] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1476client # [ 0.903314] igb: Copyright (c) 2007-2014 Intel Corporation.1477worker1 # [ 0.904124] nicpf, ver 1.01478worker2 # [ 0.902428] sky2: driver version 1.301479worker1 # [ 0.904686] e1000: Intel(R) PRO/1000 Network Driver1480client # [ 0.903337] igbvf: Intel(R) Gigabit Virtual Function Network Driver1481worker2 # [ 0.904428] ehci-pci 0000:00:07.0: EHCI Host Controller1482worker1 # [ 0.904695] e1000: Copyright (c) 1999-2006 Intel Corporation.1483client # [ 0.903343] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1484worker2 # [ 0.904453] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11485client # [ 0.903475] sky2: driver version 1.301486worker1 # [ 0.904718] e1000e: Intel(R) PRO/1000 Network Driver1487lb # [ 0.908974] thunder_xcv, ver 1.01488worker2 # [ 0.904681] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001489worker1 # [ 0.904727] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1490client # [ 0.905110] usbcore: registered new interface driver usb-storage1491lb # [ 0.909014] thunder_bgx, ver 1.01492lb # [ 0.909036] nicpf, ver 1.01493worker1 # [ 0.904757] igb: Intel(R) Gigabit Ethernet Network Driver1494client # [ 0.905160] usbcore: registered new interface driver usbserial_generic1495lb # [ 0.909606] e1000: Intel(R) PRO/1000 Network Driver1496worker1 # [ 0.904763] igb: Copyright (c) 2007-2014 Intel Corporation.1497client # [ 0.905175] usbserial: USB Serial support registered for generic1498lb # [ 0.909614] e1000: Copyright (c) 1999-2006 Intel Corporation.1499worker1 # [ 0.904785] igbvf: Intel(R) Gigabit Virtual Function Network Driver1500client # [ 0.905764] hv_vmbus: registering driver hyperv_keyboard1501lb # [ 0.909638] e1000e: Intel(R) PRO/1000 Network Driver1502worker2 # [ 0.915056] usbcore: registered new interface driver usb-storage1503worker1 # [ 0.904791] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1504client # [ 0.907060] ehci-pci 0000:00:07.0: EHCI Host Controller1505lb # [ 0.909646] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1506worker1 # [ 0.904994] sky2: driver version 1.301507worker2 # [ 0.915110] usbcore: registered new interface driver usbserial_generic1508lb # [ 0.909675] igb: Intel(R) Gigabit Ethernet Network Driver1509worker1 # [ 0.906531] usbcore: registered new interface driver usb-storage1510worker2 # [ 0.915125] usbserial: USB Serial support registered for generic1511client # [ 0.907086] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11512lb # [ 0.909681] igb: Copyright (c) 2007-2014 Intel Corporation.1513worker2 # [ 0.915778] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001514worker1 # [ 0.906585] usbcore: registered new interface driver usbserial_generic1515client # [ 0.907291] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001516worker2 # [ 0.916148] hub 1-0:1.0: USB hub found1517lb # [ 0.909704] igbvf: Intel(R) Gigabit Virtual Function Network Driver1518worker1 # [ 0.906600] usbserial: USB Serial support registered for generic1519worker2 # [ 0.916170] hub 1-0:1.0: 6 ports detected1520client # [ 0.918608] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001521lb # [ 0.909710] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1522client # [ 0.918942] hub 1-0:1.0: USB hub found1523worker1 # [ 0.907437] ehci-pci 0000:00:07.0: EHCI Host Controller1524lb # [ 0.909847] sky2: driver version 1.301525client # [ 0.918957] hub 1-0:1.0: 6 ports detected1526worker2 # [ 0.920597] hv_vmbus: registering driver hyperv_keyboard1527worker1 # [ 0.907460] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11528lb # [ 0.911458] usbcore: registered new interface driver usb-storage1529worker1 # [ 0.907661] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001530lb # [ 0.911538] usbcore: registered new interface driver usbserial_generic1531lb # [ 0.911551] usbserial: USB Serial support registered for generic1532client # [ 0.922789] rtc-pl031 9010000.pl031: registered as rtc01533worker2 # [ 0.922184] rtc-pl031 9010000.pl031: registered as rtc01534worker1 # [ 0.919754] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001535lb # [ 0.912793] ehci-pci 0000:00:07.0: EHCI Host Controller1536worker1 # [ 0.920057] hub 1-0:1.0: USB hub found1537client # [ 0.922822] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T15:12:45 UTC (1790349165)1538worker1 # [ 0.920078] hub 1-0:1.0: 6 ports detected1539worker2 # [ 0.922211] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T15:12:45 UTC (1790349165)1540lb # [ 0.912818] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11541client # [ 0.923186] i2c_dev: i2c /dev entries driver1542worker2 # [ 0.922524] i2c_dev: i2c /dev entries driver1543lb # [ 0.912985] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001544worker1 # [ 0.922645] hv_vmbus: registering driver hyperv_keyboard1545lb # [ 0.925105] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001546lb # [ 0.925419] hub 1-0:1.0: USB hub found1547lb # [ 0.925452] hub 1-0:1.0: 6 ports detected1548worker1 # [ 0.924268] rtc-pl031 9010000.pl031: registered as rtc01549lb # [ 0.927594] hv_vmbus: registering driver hyperv_keyboard1550worker1 # [ 0.924296] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T15:12:45 UTC (1790349165)1551client # [ 0.928626] sdhci: Secure Digital Host Controller Interface driver1552worker1 # [ 0.924600] i2c_dev: i2c /dev entries driver1553client # [ 0.928636] sdhci: Copyright(c) Pierre Ossman1554client # [ 0.928904] Synopsys Designware Multimedia Card Interface Driver1555lb # [ 0.929197] rtc-pl031 9010000.pl031: registered as rtc01556client # [ 0.929273] sdhci-pltfm: SDHCI platform and OF driver helper1557worker2 # [ 0.928086] sdhci: Secure Digital Host Controller Interface driver1558worker2 # [ 0.928097] sdhci: Copyright(c) Pierre Ossman1559lb # [ 0.929225] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T15:12:45 UTC (1790349165)1560lb # [ 0.929532] i2c_dev: i2c /dev entries driver1561worker2 # [ 0.928373] Synopsys Designware Multimedia Card Interface Driver1562client # [ 0.933702] hid: raw HID events driver (C) Jiri Kosina1563worker1 # [ 0.930251] sdhci: Secure Digital Host Controller Interface driver1564worker2 # [ 0.928757] sdhci-pltfm: SDHCI platform and OF driver helper1565client # [ 0.933960] usbcore: registered new interface driver usbhid1566worker1 # [ 0.930267] sdhci: Copyright(c) Pierre Ossman1567worker2 # [ 0.930310] hid: raw HID events driver (C) Jiri Kosina1568client # [ 0.933967] usbhid: USB HID core driver1569worker1 # [ 0.930530] Synopsys Designware Multimedia Card Interface Driver1570worker2 # [ 0.930599] usbcore: registered new interface driver usbhid1571worker2 # [ 0.930606] usbhid: USB HID core driver1572lb # [ 0.935157] sdhci: Secure Digital Host Controller Interface driver1573lb # [ 0.935167] sdhci: Copyright(c) Pierre Ossman1574worker1 # [ 0.933035] sdhci-pltfm: SDHCI platform and OF driver helper1575client # [ 0.937009] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1576worker1 # [ 0.934556] hid: raw HID events driver (C) Jiri Kosina1577lb # [ 0.935438] Synopsys Designware Multimedia Card Interface Driver1578worker2 # [ 0.936693] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1579client # [ 0.939682] drop_monitor: Initializing network drop monitor service1580lb # [ 0.935817] sdhci-pltfm: SDHCI platform and OF driver helper1581client # [ 0.939891] NET: Registered PF_INET6 protocol family1582worker2 # [ 0.938322] drop_monitor: Initializing network drop monitor service1583worker1 # [ 0.936233] usbcore: registered new interface driver usbhid1584worker2 # [ 0.938479] NET: Registered PF_INET6 protocol family1585worker1 # [ 0.936243] usbhid: USB HID core driver1586client # [ 0.941932] Segment Routing with IPv61587client # [ 0.941965] In-situ OAM (IOAM) with IPv61588worker2 # [ 0.941877] Segment Routing with IPv61589client # [ 0.941994] NET: Registered PF_PACKET protocol family1590worker1 # [ 0.938611] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1591worker2 # [ 0.941894] In-situ OAM (IOAM) with IPv61592lb # [ 0.940276] hid: raw HID events driver (C) Jiri Kosina1593worker2 # [ 0.941953] NET: Registered PF_PACKET protocol family1594lb # [ 0.940525] usbcore: registered new interface driver usbhid1595worker2 # [ 0.943652] 9pnet: Installing 9P2000 support1596lb # [ 0.940531] usbhid: USB HID core driver1597worker2 # [ 0.943698] Key type dns_resolver registered1598worker1 # [ 0.941295] drop_monitor: Initializing network drop monitor service1599worker1 # [ 0.941474] NET: Registered PF_INET6 protocol family1600lb # [ 0.943519] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1601client # [ 0.943719] 9pnet: Installing 9P2000 support1602client # [ 0.943808] Key type dns_resolver registered1603worker1 # [ 0.943684] Segment Routing with IPv61604worker1 # [ 0.943704] In-situ OAM (IOAM) with IPv61605lb # [ 0.946271] drop_monitor: Initializing network drop monitor service1606worker1 # [ 0.943762] NET: Registered PF_PACKET protocol family1607lb # [ 0.946405] NET: Registered PF_INET6 protocol family1608lb # [ 0.948660] Segment Routing with IPv61609worker1 # [ 0.945519] 9pnet: Installing 9P2000 support1610lb # [ 0.948693] In-situ OAM (IOAM) with IPv61611worker1 # [ 0.945568] Key type dns_resolver registered1612lb # [ 0.948726] NET: Registered PF_PACKET protocol family1613worker2 # [ 0.950604] registered taskstats version 11614worker2 # [ 0.950753] Loading compiled-in X.509 certificates1615lb # [ 0.950432] 9pnet: Installing 9P2000 support1616lb # [ 0.950480] Key type dns_resolver registered1617client # [ 0.950750] registered taskstats version 11618client # [ 0.950914] Loading compiled-in X.509 certificates1619worker1 # [ 0.952319] registered taskstats version 11620worker1 # [ 0.952474] Loading compiled-in X.509 certificates1621client # [ 0.959792] Demotion targets for Node 0: null1622client # [ 0.959904] Key type .fscrypt registered1623worker2 # [ 0.959780] Demotion targets for Node 0: null1624client # [ 0.959911] Key type fscrypt-provisioning registered1625worker2 # [ 0.959886] Key type .fscrypt registered1626client # [ 0.960001] ima: No TPM chip found, activating TPM-bypass!1627lb # [ 0.957269] registered taskstats version 11628worker2 # [ 0.959903] Key type fscrypt-provisioning registered1629client # [ 0.960020] ima: Allocated hash algorithm: sha11630lb # [ 0.957406] Loading compiled-in X.509 certificates1631worker2 # [ 0.959993] ima: No TPM chip found, activating TPM-bypass!1632client # [ 0.960044] ima: No architecture policies found1633worker2 # [ 0.960012] ima: Allocated hash algorithm: sha11634worker2 # [ 0.960034] ima: No architecture policies found1635client # [ 0.964188] input: gpio-keys as /devices/platform/gpio-keys/input/input01636worker1 # [ 0.961129] Demotion targets for Node 0: null1637worker2 # [ 0.964500] input: gpio-keys as /devices/platform/gpio-keys/input/input01638worker1 # [ 0.961235] Key type .fscrypt registered1639worker1 # [ 0.961242] Key type fscrypt-provisioning registered1640worker1 # [ 0.961345] ima: No TPM chip found, activating TPM-bypass!1641worker1 # [ 0.961364] ima: Allocated hash algorithm: sha11642worker1 # [ 0.961384] ima: No architecture policies found1643lb # [ 0.966141] Demotion targets for Node 0: null1644lb # [ 0.966246] Key type .fscrypt registered1645lb # [ 0.966252] Key type fscrypt-provisioning registered1646worker1 # [ 0.965504] input: gpio-keys as /devices/platform/gpio-keys/input/input01647lb # [ 0.966346] ima: No TPM chip found, activating TPM-bypass!1648lb # [ 0.966365] ima: Allocated hash algorithm: sha11649lb # [ 0.966386] ima: No architecture policies found1650lb # [ 0.970493] input: gpio-keys as /devices/platform/gpio-keys/input/input01651worker2 # [ 0.981821] clk: Disabling unused clocks1652client # [ 0.982586] clk: Disabling unused clocks1653worker2 # [ 0.981843] PM: genpd: Disabling unused power domains1654client # [ 0.982608] PM: genpd: Disabling unused power domains1655client # [ 0.986803] Freeing unused kernel memory: 4736K1656worker2 # [ 0.986056] Freeing unused kernel memory: 4736K1657client # [ 0.986998] Run /init as init process1658worker2 # [ 0.986268] Run /init as init process1659worker1 # [ 0.983647] clk: Disabling unused clocks1660worker1 # [ 0.983671] PM: genpd: Disabling unused power domains1661worker1 # [ 0.987910] Freeing unused kernel memory: 4736K1662lb # [ 0.988544] clk: Disabling unused clocks1663worker1 # [ 0.988129] Run /init as init process1664lb # [ 0.988572] PM: genpd: Disabling unused power domains1665lb # [ 0.992808] Freeing unused kernel memory: 4736K1666lb # [ 0.993013] Run /init as init process1667worker2 # [ 1.002020] systemd[1]: Successfully made /usr/ read-only.1668client # [ 1.004673] systemd[1]: Successfully made /usr/ read-only.1669worker1 # [ 1.005047] systemd[1]: Successfully made /usr/ read-only.1670lb # [ 1.010311] systemd[1]: Successfully made /usr/ read-only.1671worker2 # [ 1.163778] usb 1-1: new high-speed USB device number 2 using ehci-pci1672client # [ 1.170499] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker1 # [ 1.166813] usb 1-1: new high-speed USB device number 2 using ehci-pci1674lb # [ 1.172172] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker2 # [ 1.313516] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676client # [ 1.320249] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677worker1 # [ 1.316586] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11678lb # [ 1.321913] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679worker2 # [ 1.336687] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1680worker2 # [ 1.343274] systemd[1]: Detected virtualization qemu.1681client # [ 1.339426] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1682worker2 # [ 1.344401] systemd[1]: Detected architecture arm64.1683client # [ 1.345929] systemd[1]: Detected virtualization qemu.1684worker2 # [ 1.345459] systemd[1]: Running in initrd.1685client # [ 1.347046] systemd[1]: Detected architecture arm64.1686client # [ 1.348053] systemd[1]: Running in initrd.1687worker2 # [ 1.346858] systemd[1]: Initializing machine ID from random generator.1688worker1 # [ 1.339728] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1689worker2 # [ 1.348477] systemd[1]: Hostname set to <worker2>.1690client # [ 1.349508] systemd[1]: Initializing machine ID from random generator.1691worker1 # [ 1.346218] systemd[1]: Detected virtualization qemu.1692client # [ 1.351037] systemd[1]: Hostname set to <client>.1693worker1 # [ 1.347342] systemd[1]: Detected architecture arm64.1694worker1 # [ 1.348398] systemd[1]: Running in initrd.1695lb # [ 1.345087] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1696lb # [ 1.351697] systemd[1]: Detected virtualization qemu.1697worker1 # [ 1.349827] systemd[1]: Initializing machine ID from random generator.1698lb # [ 1.352813] systemd[1]: Detected architecture arm64.1699worker1 # [ 1.351377] systemd[1]: Hostname set to <worker1>.1700lb # [ 1.353864] systemd[1]: Running in initrd.1701lb # [ 1.355344] systemd[1]: Initializing machine ID from random generator.1702lb # [ 1.356946] systemd[1]: Hostname set to <lb>.1703worker2 # [ 1.392736] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01704client # [ 1.396704] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705worker1 # [ 1.399021] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706lb # [ 1.404350] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker2 # [ 1.511770] usb 1-2: new high-speed USB device number 3 using ehci-pci1708client # [ 1.518506] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker1 # [ 1.518824] usb 1-2: new high-speed USB device number 3 using ehci-pci1710lb # [ 1.524195] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker2 # [ 1.587012] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712client # [ 1.594432] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713lb # [ 1.624193] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714worker1 # [ 1.622557] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker2 # [ 1.649904] systemd[1]: Queued start job for default target Initrd Default Target.1716worker2 # [ 1.658158] systemd[1]: Created slice Slice /system/modprobe.1717worker2 # [ 1.659405] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1718worker2 # [ 1.661032] systemd[1]: Expecting device /dev/disk/by-label/nixos...1719worker2 # [ 1.662262] systemd[1]: Reached target Path Units.1720client # [ 1.662748] systemd[1]: Queued start job for default target Initrd Default Target.1721worker2 # [ 1.663090] systemd[1]: Reached target Slice Units.1722worker2 # [ 1.664130] systemd[1]: Reached target Swaps.1723worker2 # [ 1.664156] systemd[1]: Reached target Timer Units.1724worker2 # [ 1.664331] systemd[1]: Listening on D-Bus System Message Bus Socket.1725worker2 # [ 1.664518] systemd[1]: Listening on Journal Socket (/dev/log).1726worker2 # [ 1.664668] systemd[1]: Listening on Journal Sockets.1727worker2 # [ 1.664804] systemd[1]: Listening on udev Control Socket.1728client # [ 1.670127] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21729worker2 # [ 1.664920] systemd[1]: Listening on udev Kernel Socket.1730worker2 # [ 1.664944] systemd[1]: Reached target Socket Units.1731lb # [ 1.671727] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21732worker2 # [ 1.674309] systemd[1]: Starting Create List of Static Device Nodes...1733worker2 # [ 1.675478] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1734client # [ 1.675100] systemd[1]: Created slice Slice /system/modprobe.1735lb # [ 1.676275] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01736client # [ 1.675308] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1737client # [ 1.675347] systemd[1]: Expecting device /dev/disk/by-label/nixos...1738client # [ 1.675374] systemd[1]: Reached target Path Units.1739client # [ 1.675393] systemd[1]: Reached target Slice Units.1740worker2 # [ 1.679685] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21741worker1 # [ 1.676518] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21742client # [ 1.675412] systemd[1]: Reached target Swaps.1743client # [ 1.675430] systemd[1]: Reached target Timer Units.1744worker2 # [ 1.679866] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01745client # [ 1.675580] systemd[1]: Listening on D-Bus System Message Bus Socket.1746worker1 # [ 1.676708] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01747client # [ 1.675757] systemd[1]: Listening on Journal Socket (/dev/log).1748client # [ 1.675904] systemd[1]: Listening on Journal Sockets.1749client # [ 1.676036] systemd[1]: Listening on udev Control Socket.1750client # [ 1.676146] systemd[1]: Listening on udev Kernel Socket.1751lb # [ 1.686146] systemd[1]: Queued start job for default target Initrd Default Target.1752client # [ 1.676168] systemd[1]: Reached target Socket Units.1753worker2 # [ 1.688601] systemd[1]: Mounting Kernel Configuration File System...1754client # [ 1.689880] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01755client # [ 1.692350] systemd[1]: Starting Create List of Static Device Nodes...1756client # [ 1.692429] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1757lb # [ 1.694003] systemd[1]: Created slice Slice /system/modprobe.1758lb # [ 1.695050] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1759lb # [ 1.696099] systemd[1]: Expecting device /dev/disk/by-label/nixos...1760lb # [ 1.697041] systemd[1]: Reached target Path Units.1761lb # [ 1.697683] systemd[1]: Reached target Slice Units.1762lb # [ 1.698341] systemd[1]: Reached target Swaps.1763lb # [ 1.698943] systemd[1]: Reached target Timer Units.1764worker2 # [ 1.700163] systemd[1]: Starting Journal Service...1765lb # [ 1.699788] systemd[1]: Listening on D-Bus System Message Bus Socket.1766lb # [ 1.700842] systemd[1]: Listening on Journal Socket (/dev/log).1767lb # [ 1.701744] systemd[1]: Listening on Journal Sockets.1768lb # [ 1.702521] systemd[1]: Listening on udev Control Socket.1769client # [ 1.702637] systemd[1]: Mounting Kernel Configuration File System...1770lb # [ 1.703349] systemd[1]: Listening on udev Kernel Socket.1771lb # [ 1.704066] systemd[1]: Reached target Socket Units.1772lb # [ 1.706090] systemd[1]: Starting Create List of Static Device Nodes...1773lb # [ 1.707087] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1774client # [ 1.710723] systemd[1]: Starting Journal Service...1775worker2 # [ 1.709400] systemd[1]: Starting Load Kernel Modules...1776worker2 # [ 1.709501] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1777worker1 # [ 1.708268] systemd[1]: Queued start job for default target Initrd Default Target.1778lb # [ 1.715704] systemd[1]: Mounting Kernel Configuration File System...1779worker1 # [ 1.717158] systemd[1]: Created slice Slice /system/modprobe.1780worker1 # [ 1.718370] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1781worker1 # [ 1.719796] systemd[1]: Expecting device /dev/disk/by-label/nixos...1782worker1 # [ 1.720856] systemd[1]: Reached target Path Units.1783lb # [ 1.724550] systemd[1]: Starting Journal Service...1784worker1 # [ 1.720897] systemd[1]: Reached target Slice Units.1785worker1 # [ 1.720919] systemd[1]: Reached target Swaps.1786worker1 # [ 1.720940] systemd[1]: Reached target Timer Units.1787worker1 # [ 1.721126] systemd[1]: Listening on D-Bus System Message Bus Socket.1788worker1 # [ 1.721276] systemd[1]: Listening on Journal Socket (/dev/log).1789worker1 # [ 1.721422] systemd[1]: Listening on Journal Sockets.1790worker1 # [ 1.721554] systemd[1]: Listening on udev Control Socket.1791worker1 # [ 1.721681] systemd[1]: Listening on udev Kernel Socket.1792worker2 # [ 1.731960] systemd[1]: Starting Coldplug All udev Devices...1793worker1 # [ 1.721703] systemd[1]: Reached target Socket Units.1794client # [ 1.734905] systemd[1]: Starting Load Kernel Modules...1795worker1 # [ 1.731720] systemd[1]: Starting Create List of Static Device Nodes...1796client # [ 1.735772] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1797worker1 # [ 1.732895] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1798lb # [ 1.739757] systemd[1]: Starting Load Kernel Modules...1799lb # [ 1.740752] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1800worker1 # [ 1.740335] systemd[1]: Mounting Kernel Configuration File System...1801worker2 # [ 1.748027] systemd[1]: Finished Create List of Static Device Nodes.1802client # [ 1.750614] systemd[1]: Starting Coldplug All udev Devices...1803lb # [ 1.753872] systemd[1]: Starting Coldplug All udev Devices...1804worker1 # [ 1.755212] systemd[1]: Starting Journal Service...1805client # [ 1.763222] systemd-journald[72]: Collecting audit messages is disabled.1806worker2 # [ 1.764125] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1807worker2 # [ 1.765475] systemd[1]: Mounted Kernel Configuration File System.1808worker1 # [ 1.762956] systemd[1]: Starting Load Kernel Modules...1809worker2 # [ 1.766677] systemd-journald[72]: Collecting audit messages is disabled.1810worker1 # [ 1.763898] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1811worker2 # [ 1.768682] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1812client # [ 1.770758] systemd[1]: Finished Create List of Static Device Nodes.1813client # [ 1.778869] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1814lb # [ 1.776287] systemd[1]: Finished Create List of Static Device Nodes.1815lb # [ 1.777196] systemd[1]: Mounted Kernel Configuration File System.1816client # [ 1.780160] systemd[1]: Mounted Kernel Configuration File System.1817lb # [ 1.785834] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1818worker2 # [ 1.780114] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1819worker2 # [ 1.786322] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01820worker2 # [ 1.786574] [drm] features: -virgl +edid -resource_blob -host_visible1821worker2 # [ 1.786584] [drm] features: -context_init1822worker2 # [ 1.787298] [drm] number of scanouts: 11823worker2 # [ 1.787315] [drm] number of cap sets: 01824worker1 # [ 1.790235] systemd[1]: Starting Coldplug All udev Devices...1825lb # [ 1.801669] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1826worker1 # [ 1.801307] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1827lb # [ 1.806637] systemd-journald[72]: Collecting audit messages is disabled.1828worker2 # [ 1.808403] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1829worker1 # [ 1.803936] systemd[1]: Finished Create List of Static Device Nodes.1830worker2 # [ 1.810912] systemd[1]: Starting Create Static Device Nodes in /dev...1831client # [ 1.819412] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1832worker2 # [ 1.815492] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1833worker2 # [ 1.815512] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01834worker1 # [ 1.823019] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1835lb # [ 1.820478] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1836lb # [ 1.826579] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01837lb # [ 1.826836] [drm] features: -virgl +edid -resource_blob -host_visible1838lb # [ 1.826846] [drm] features: -context_init1839lb # [ 1.827640] [drm] number of scanouts: 11840lb # [ 1.827659] [drm] number of cap sets: 01841worker1 # [ 1.825331] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1842client # [ 1.834529] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1843client # [ 1.840758] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01844client # [ 1.840999] [drm] features: -virgl +edid -resource_blob -host_visible1845client # [ 1.841009] [drm] features: -context_init1846client # [ 1.841727] [drm] number of scanouts: 11847worker1 # [ 1.841087] systemd[1]: Mounted Kernel Configuration File System.1848client # [ 1.841745] [drm] number of cap sets: 01849worker1 # [ 1.845204] systemd-journald[72]: Collecting audit messages is disabled.1850worker2 # [ 1.844153] Console: switching to colour frame buffer device 160x501851worker2 # [ 1.848418] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1852client # [ 1.850613] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1853lb # [ 1.848753] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1854lb # [ 1.851403] systemd[1]: Starting Create Static Device Nodes in /dev...1855client # [ 1.853252] systemd[1]: Starting Create Static Device Nodes in /dev...1856worker1 # [ 1.854226] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01857worker1 # [ 1.854480] [drm] features: -virgl +edid -resource_blob -host_visible1858worker1 # [ 1.854491] [drm] features: -context_init1859worker1 # [ 1.857469] [drm] number of scanouts: 11860worker1 # [ 1.857490] [drm] number of cap sets: 01861worker2 # [ 1.864418] systemd[1]: Finished Create Static Device Nodes in /dev.1862worker2 # [ 1.864664] systemd[1]: Reached target Preparation for Local File Systems.1863worker1 # [ 1.863625] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1864lb # [ 1.860525] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1865worker2 # [ 1.864699] systemd[1]: Reached target Local File Systems.1866lb # [ 1.860547] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01867worker2 # [ 1.870204] systemd[1]: Starting Rule-based Manager for Device Events and Files...1868worker1 # [ 1.865941] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1869worker1 # [ 1.865958] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01870worker1 # [ 1.873002] systemd[1]: Starting Create Static Device Nodes in /dev...1871client # [ 1.870802] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1872client # [ 1.870820] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01873worker2 # [ 1.883994] systemd[1]: Finished Load Kernel Modules.1874worker2 # [ 1.886938] systemd[1]: Starting Apply Kernel Variables...1875client # [ 1.895126] systemd[1]: Finished Create Static Device Nodes in /dev.1876client # [ 1.895317] systemd[1]: Reached target Preparation for Local File Systems.1877client # [ 1.895343] systemd[1]: Reached target Local File Systems.1878worker1 # [ 1.891035] Console: switching to colour frame buffer device 160x501879lb # [ 1.896786] systemd[1]: Finished Create Static Device Nodes in /dev.1880worker1 # [ 1.895202] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1881lb # [ 1.896981] systemd[1]: Reached target Preparation for Local File Systems.1882lb # [ 1.897004] systemd[1]: Reached target Local File Systems.1883client # [ 1.899441] systemd[1]: Starting Rule-based Manager for Device Events and Files...1884lb # [ 1.901042] systemd[1]: Starting Rule-based Manager for Device Events and Files...1885lb # [ 1.908335] Console: switching to colour frame buffer device 160x501886client # [ 1.906882] Console: switching to colour frame buffer device 160x501887lb # [ 1.909040] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1888client # [ 1.911100] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1889worker1 # [ 1.912134] systemd[1]: Finished Load Kernel Modules.1890worker2 # [ 1.927932] systemd[1]: Finished Apply Kernel Variables.1891worker1 # [ 1.927190] systemd[1]: Starting Apply Kernel Variables...1892worker1 # [ 1.930985] systemd[1]: Finished Create Static Device Nodes in /dev.1893lb # [ 1.932429] systemd[1]: Finished Load Kernel Modules.1894worker1 # [ 1.932936] systemd[1]: Reached target Preparation for Local File Systems.1895lb # [ 1.935532] systemd[1]: Starting Apply Kernel Variables...1896client # [ 1.938717] systemd[1]: Finished Load Kernel Modules.1897client # [ 1.941976] systemd[1]: Starting Apply Kernel Variables...1898worker1 # [ 1.938978] systemd[1]: Reached target Local File Systems.1899worker2 # [ 1.946356] systemd[1]: Started Journal Service.1900client # [ 1.951641] systemd[1]: Started Journal Service.1901worker2 # [ 1.944077] systemd-modules-load[73]: Inserted module 'dm_mod'1902worker2 # [ 1.945407] systemd-modules-load[73]: Module 'virtio_balloon' is built in1903worker2 # [ 1.946515] systemd-modules-load[73]: Module 'virtio_console' is built in1904client # [ 1.945242] systemd-modules-load[74]: Inserted module 'dm_mod'1905worker2 # [ 1.947597] systemd-modules-load[73]: Inserted module 'virtio_gpu'1906client # [ 1.948437] systemd-modules-load[74]: Module 'virtio_balloon' is built in1907client # [ 1.949572] systemd-modules-load[74]: Module 'virtio_console' is built in1908client # [ 1.950674] systemd-modules-load[74]: Inserted module 'virtio_gpu'1909client # [ 1.951725] systemd-modules-load[74]: Module 'virtio_rng' is built in1910worker1 # [ 1.962934] systemd[1]: Starting Rule-based Manager for Device Events and Files...1911worker2 # [ 1.960701] systemd-modules-load[73]: Module 'virtio_rng' is built in1912worker2 # [ 1.961795] systemd[1]: Starting Create System Files and Directories...1913lb # [ 1.980802] systemd[1]: Finished Apply Kernel Variables.1914client # [ 1.968393] systemd[1]: Starting Create System Files and Directories...1915worker1 # [ 1.984809] systemd[1]: Finished Apply Kernel Variables.1916worker2 # [ 1.973112] systemd-udevd[79]: Using default interface naming scheme 'v261'.1917lb # [ 1.986822] systemd[1]: Started Journal Service.1918lb # [ 1.980104] systemd-modules-load[73]: Inserted module 'dm_mod'1919lb # [ 1.981381] systemd-modules-load[73]: Module 'virtio_balloon' is built in1920worker2 # [ 1.985571] systemd[1]: Finished Create System Files and Directories.1921lb # [ 1.988378] systemd-modules-load[73]: Module 'virtio_console' is built in1922client # [ 1.988869] systemd[1]: Finished Apply Kernel Variables.1923lb # [ 1.991740] systemd-modules-load[73]: Inserted module 'virtio_gpu'1924lb # [ 2.001515] systemd-modules-load[73]: Module 'virtio_rng' is built in1925client # [ 2.001487] systemd[1]: Finished Create System Files and Directories.1926lb # [ 2.002614] systemd[1]: Starting Create System Files and Directories...1927client # [ 2.005070] systemd-udevd[78]: Using default interface naming scheme 'v261'.1928worker2 # [ 2.007202] systemd[1]: Started Rule-based Manager for Device Events and Files.1929worker1 # [ 2.023009] systemd[1]: Started Journal Service.1930worker1 # [ 2.011451] systemd-modules-load[73]: Inserted module 'dm_mod'1931worker1 # [ 2.012778] systemd-modules-load[73]: Module 'virtio_balloon' is built in1932worker1 # [ 2.013870] systemd-modules-load[73]: Module 'virtio_console' is built in1933worker1 # [ 2.014950] systemd-modules-load[73]: Inserted module 'virtio_gpu'1934lb # [ 2.016125] systemd-udevd[78]: Using default interface naming scheme 'v261'.1935lb # [ 2.023653] systemd[1]: Finished Create System Files and Directories.1936worker1 # [ 2.030680] systemd-modules-load[73]: Module 'virtio_rng' is built in1937client # [ 2.031858] systemd[1]: Started Rule-based Manager for Device Events and Files.1938worker1 # [ 2.033399] systemd[1]: Starting Create System Files and Directories...1939lb # [ 2.041387] systemd[1]: Started Rule-based Manager for Device Events and Files.1940worker1 # [ 2.041274] systemd-udevd[80]: Using default interface naming scheme 'v261'.1941worker1 # [ 2.062830] systemd[1]: Finished Create System Files and Directories.1942worker2 # [ 2.072875] systemd[1]: Starting Virtual Console Setup...1943worker1 # [ 2.073884] systemd[1]: Started Rule-based Manager for Device Events and Files.1944lb # [ 2.102336] systemd[1]: Starting Virtual Console Setup...1945client # [ 2.103665] systemd[1]: Starting Virtual Console Setup...1946worker1 # [ 2.132325] systemd[1]: Starting Virtual Console Setup...1947worker2 # [ 2.140625] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1948worker2 # [ 2.145230] systemd[1]: Finished Virtual Console Setup.1949lb # [ 2.152509] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1950lb # [ 2.155981] systemd[1]: Finished Virtual Console Setup.1951client # [ 2.161245] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1952client # [ 2.163981] systemd[1]: Finished Virtual Console Setup.1953worker1 # [ 2.180421] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1954worker1 # [ 2.183722] systemd[1]: Finished Virtual Console Setup.1955lb # [ 2.771225] systemd[1]: Finished Coldplug All udev Devices.1956lb # [ 2.772266] systemd[1]: Reached target System Initialization.1957lb # [ 2.773289] systemd[1]: Reached target Basic System.1958client # [ 2.793501] systemd[1]: Finished Coldplug All udev Devices.1959client # [ 2.794557] systemd[1]: Reached target System Initialization.1960client # [ 2.795382] systemd[1]: Reached target Basic System.1961worker2 # [ 2.797255] systemd[1]: Finished Coldplug All udev Devices.1962worker2 # [ 2.798247] systemd[1]: Reached target System Initialization.1963worker2 # [ 2.799070] systemd[1]: Reached target Basic System.1964worker1 # [ 2.805827] systemd[1]: Finished Coldplug All udev Devices.1965worker1 # [ 2.806832] systemd[1]: Reached target System Initialization.1966worker1 # [ 2.808232] systemd[1]: Reached target Basic System.1967client # [ 2.924097] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1968lb # [ 2.939956] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969lb # [ 2.958421] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1970worker2 # [ 2.964836] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1971client # [ 2.964118] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1972client # [ 2.970030] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1973worker1 # [ 2.974193] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1974lb # [ 2.976430] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line.1975worker1 # [ 2.979425] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976worker1 # [ 2.984472] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1977worker2 # [ 2.986732] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1978worker2 # [ 2.992247] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1979client # [ 3.025574] systemd[1]: Found device /dev/disk/by-label/nixos.1980client # [ 3.028428] systemd[1]: Reached target Initrd Root Device.1981client # [ 3.032305] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982lb # [ 3.060175] systemd[1]: Found device /dev/disk/by-label/nixos.1983lb # [ 3.064093] systemd[1]: Reached target Initrd Root Device.1984worker2 # [ 3.068979] systemd[1]: Found device /dev/disk/by-label/nixos.1985lb # [ 3.067525] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986worker2 # [ 3.071257] systemd[1]: Reached target Initrd Root Device.1987worker2 # [ 3.075072] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1988worker1 # [ 3.078167] systemd[1]: Found device /dev/disk/by-label/nixos.1989worker1 # [ 3.080628] systemd[1]: Reached target Initrd Root Device.1990worker1 # [ 3.084473] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1991client # [ 3.086417] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1992client # [ 3.094131] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1993client # [ 3.103322] systemd[1]: Mounting /sysroot...1994lb # [ 3.114279] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1995worker2 # [ 3.117424] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1996worker2 # [ 3.124129] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1997lb # [ 3.123376] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1998worker2 # [ 3.128586] systemd[1]: Mounting /sysroot...1999lb # [ 3.128283] systemd[1]: Mounting /sysroot...2000worker1 # [ 3.128721] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks
loading 2,131 hidden lines…
4132worker1 # [ 65.400605] nix-grpc-daemon[485]: ts=2026-09-25T15:13:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14133worker2 # [ 65.408891] nix-grpc-daemon[770]: ts=2026-09-25T15:13:50Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04134worker2 # [ 65.412486] nix-grpc-daemon[770]: ts=2026-09-25T15:13:50Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14135worker1 # [ 65.412881] nix-grpc-daemon[485]: ts=2026-09-25T15:13:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364136worker1 # [ 65.418227] nix-daemon[837]: reaped child process 879, status = succeeded4137worker1 # [ 65.423446] nix-grpc-daemon[485]: ts=2026-09-25T15:13:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364138worker1 # [ 65.432079] nix-grpc-daemon[862]: time=2026-09-25T15:13:50.022Z level=INFO msg="All 1 paths already cached"4139client # copying 1 paths to 'lb:50051'...4140client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4141worker1 # [ 65.439098] nix-grpc-daemon[485]: ts=2026-09-25T15:13:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455364142worker1 # [ 65.447590] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4143worker1 # [ 65.572930] nix-daemon[837]: accepted connection from pid 904, user nix-grpc-daemon (trusted)4144worker1 # [ 65.593691] nix-daemon[837]: reaped child process 907, status = succeeded4145lb # [ 65.597471] niks3-server[984]: 2026/09/25 15:13:50 INFO Received push request method=POST path=/api/pushes4146worker1 # [ 65.602365] nix-grpc-daemon[862]: time=2026-09-25T15:13:50.192Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4147worker1 # [ 65.603950] nix-grpc-daemon[862]: time=2026-09-25T15:13:50.192Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4148worker1 # [ 65.614505] nix-grpc-daemon[862]: time=2026-09-25T15:13:50.204Z level=INFO msg="Uploading 1 narinfos"4149lb # [ 65.614874] niks3-server[984]: 2026/09/25 15:13:50 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4150lb # [ 65.622232] niks3-server[984]: 2026/09/25 15:13:50 INFO Signed narinfos id=11 count=14151lb # [ 65.626986] niks3-server[984]: 2026/09/25 15:13:50 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4152lb # [ 65.636556] niks3-server[984]: 2026/09/25 15:13:50 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4153lb # [ 65.639061] niks3-server[984]: 2026/09/25 15:13:50 INFO Received complete push request method=POST path=/api/pushes/11/complete4154worker1 # [ 65.643507] nix-grpc-daemon[862]: time=2026-09-25T15:13:50.233Z level=INFO msg="Upload complete. (185ms)"4155lb # [ 65.645682] niks3-server[984]: 2026/09/25 15:13:50 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4156worker1 # [ 65.650166] nix-grpc-daemon[485]: ts=2026-09-25T15:13:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_in=6244157worker1 # [ 65.656252] nix-grpc-daemon[485]: ts=2026-09-25T15:13:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364158worker1 # [ 65.667302] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4159worker1 # [ 65.669207] nix-daemon[837]: reaped child process 902, status = succeeded4160client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4161client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4162worker1 # [ 65.680156] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4163client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4164client # farm-b> LOG-farm-b-sig4165worker1 # [ 66.710054] nix-daemon[837]: reaped child process 911, status = succeeded4166worker1 # [ 66.828108] nix-daemon[837]: accepted connection from pid 927, user nix-grpc-daemon (trusted)4167lb # [ 66.847470] niks3-server[984]: 2026/09/25 15:13:51 INFO Received push request method=POST path=/api/pushes4168worker1 # [ 66.850339] nix-daemon[837]: reaped child process 930, status = succeeded4169worker1 # [ 66.853887] nix-grpc-daemon[862]: time=2026-09-25T15:13:51.443Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4170worker1 # [ 66.855400] nix-grpc-daemon[862]: time=2026-09-25T15:13:51.443Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4171lb # [ 66.871800] niks3-server[984]: 2026/09/25 15:13:51 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4172lb # [ 66.877983] niks3-server[984]: 2026/09/25 15:13:51 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4173worker1 # [ 66.884195] nix-grpc-daemon[862]: time=2026-09-25T15:13:51.474Z level=INFO msg="Uploading 1 narinfos"4174lb # [ 66.886894] niks3-server[984]: 2026/09/25 15:13:51 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4175lb # [ 66.892282] niks3-server[984]: 2026/09/25 15:13:51 INFO Signed narinfos id=12 count=14176lb # [ 66.895821] niks3-server[984]: 2026/09/25 15:13:51 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4177lb # [ 66.903348] niks3-server[984]: 2026/09/25 15:13:51 INFO Received complete push request method=POST path=/api/pushes/12/complete4178worker1 # [ 66.907977] nix-grpc-daemon[862]: time=2026-09-25T15:13:51.498Z level=INFO msg="Upload complete. (218ms)"4179lb # [ 66.909585] niks3-server[984]: 2026/09/25 15:13:51 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4180worker1 # [ 66.914411] nix-grpc-daemon[485]: ts=2026-09-25T15:13:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14181worker2 # [ 66.921289] nix-grpc-daemon[770]: ts=2026-09-25T15:13:51Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04182worker2 # [ 66.924453] nix-grpc-daemon[770]: ts=2026-09-25T15:13:51Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14183worker1 # [ 66.927629] nix-grpc-daemon[485]: ts=2026-09-25T15:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364184worker1 # [ 66.930914] nix-daemon[837]: reaped child process 909, status = succeeded4185worker1 # [ 66.938744] nix-grpc-daemon[485]: ts=2026-09-25T15:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364186worker1 # [ 66.946772] nix-grpc-daemon[862]: time=2026-09-25T15:13:51.537Z level=INFO msg="All 2 paths already cached"4187client # copying 1 paths to 'lb:50051'...4188client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4189worker1 # [ 66.953693] nix-grpc-daemon[485]: ts=2026-09-25T15:13:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455364190worker1 # [ 66.962126] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4191worker1 # [ 67.081002] nix-daemon[837]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4192lb # [ 67.093206] envoy[893]: conn peer=[2001:db8:1::1]:52078 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9502 tx=4376 ms=49684193lb # [ 67.096145] envoy[893]: conn peer=[2001:db8:1::1]:43760 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=484194lb # [ 67.098609] envoy[893]: conn peer=[2001:db8:1::1]:43770 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4181 tx=1875 ms=15564195worker1 # [ 67.102241] nix-daemon[837]: reaped child process 937, status = succeeded4196lb # [ 67.103138] niks3-server[984]: 2026/09/25 15:13:51 INFO Received push request method=POST path=/api/pushes4197worker1 # [ 67.107030] nix-grpc-daemon[862]: time=2026-09-25T15:13:51.697Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4198worker1 # [ 67.108787] nix-grpc-daemon[862]: time=2026-09-25T15:13:51.697Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4199lb # [ 67.118707] niks3-server[984]: 2026/09/25 15:13:51 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4200worker1 # [ 67.123222] nix-grpc-daemon[862]: time=2026-09-25T15:13:51.713Z level=INFO msg="Uploading 1 narinfos"4201lb # [ 67.126612] niks3-server[984]: 2026/09/25 15:13:51 INFO Signed narinfos id=13 count=14202lb # [ 67.131233] niks3-server[984]: 2026/09/25 15:13:51 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4203lb # [ 67.141866] niks3-server[984]: 2026/09/25 15:13:51 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4204lb # [ 67.148899] niks3-server[984]: 2026/09/25 15:13:51 INFO Received complete push request method=POST path=/api/pushes/13/complete4205worker1 # [ 67.152833] nix-grpc-daemon[862]: time=2026-09-25T15:13:51.743Z level=INFO msg="Upload complete. (181ms)"4206lb # [ 67.152699] niks3-server[984]: 2026/09/25 15:13:51 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4207worker1 # [ 67.159631] nix-grpc-daemon[485]: ts=2026-09-25T15:13:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_in=7524208worker1 # [ 67.170829] nix-grpc-daemon[485]: ts=2026-09-25T15:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364209worker1 # [ 67.175309] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4210worker1 # [ 67.179285] nix-daemon[837]: reaped child process 932, status = succeeded4211client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4212worker1 # [ 67.188203] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4213client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4214client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4215client # farm-top> LOG-farm-top-sig4216worker1 # [ 68.032641] nix-daemon[837]: reaped child process 941, status = succeeded4217worker1 # [ 68.136869] nix-daemon[837]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4218worker1 # [ 68.157550] nix-daemon[837]: reaped child process 960, status = succeeded4219lb # [ 68.163726] niks3-server[984]: 2026/09/25 15:13:52 INFO Received push request method=POST path=/api/pushes4220worker1 # [ 68.172292] nix-grpc-daemon[862]: time=2026-09-25T15:13:52.760Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4221worker1 # [ 68.174173] nix-grpc-daemon[862]: time=2026-09-25T15:13:52.760Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4222lb # [ 68.193728] niks3-server[984]: 2026/09/25 15:13:52 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4223worker1 # [ 68.198577] nix-grpc-daemon[862]: time=2026-09-25T15:13:52.788Z level=INFO msg="Uploading 1 narinfos"4224lb # [ 68.201937] niks3-server[984]: 2026/09/25 15:13:52 INFO Signed narinfos id=14 count=14225lb # [ 68.209238] niks3-server[984]: 2026/09/25 15:13:52 INFO Received complete push request method=POST path=/api/pushes/14/complete4226lb # [ 68.219474] niks3-server[984]: 2026/09/25 15:13:52 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4227lb # [ 68.229923] niks3-server[984]: 2026/09/25 15:13:52 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4228worker1 # [ 68.234076] nix-grpc-daemon[862]: time=2026-09-25T15:13:52.824Z level=INFO msg="Upload complete. (236ms)"4229lb # [ 68.236082] niks3-server[984]: 2026/09/25 15:13:52 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4230worker1 # [ 68.240694] nix-grpc-daemon[485]: ts=2026-09-25T15:13:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14231lb # [ 68.241877] niks3-server[984]: 2026/09/25 15:13:52 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4232worker1 # [ 68.253198] nix-daemon[837]: reaped child process 939, status = succeeded4233client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 4.33 seconds)4234client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4235client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.15 seconds)4236client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4237worker1 # [ 68.555723] nix-grpc-daemon[485]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:609064238worker1 # [ 68.561782] nix-grpc-daemon[485]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:609064239client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.16 seconds)4240client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4241worker2 # [ 68.730769] nix-grpc-daemon[770]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:463104242client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4243client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4244client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4245client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4246worker2 # [ 68.745499] nix-grpc-daemon[770]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:463104247worker2 # [ 68.750675] nix-grpc-daemon[770]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4248worker2 # [ 68.764093] nix-grpc-daemon[770]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4249worker2 # [ 68.773937] nix-grpc-daemon[770]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4250client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.22 seconds)4251client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4252client # finding garbage collector roots...4253client # removing stale temporary roots file "/nix/var/nix/temproots/887"4254client # removing stale temporary roots file "/nix/var/nix/temproots/810"4255client # removing stale temporary roots file "/nix/var/nix/temproots/831"4256client # removing stale temporary roots file "/nix/var/nix/temproots/843"4257client # removing stale temporary roots file "/nix/var/nix/temproots/795"4258client # removing stale temporary roots file "/nix/var/nix/temproots/872"4259client # removing stale temporary roots file "/nix/var/nix/temproots/864"4260client # removing stale temporary roots file "/nix/var/nix/temproots/851"4261client # deleting specified paths...4262client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4263client # deleting unused links...4264client # note: hard linking is currently saving 0.0 KiB4265worker2 # [ 69.109994] nix-grpc-daemon[770]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505204266worker2 # [ 69.116117] nix-grpc-daemon[770]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505204267worker2 # [ 69.123802] nix-grpc-daemon[873]: time=2026-09-25T15:13:53.716Z level=INFO msg="All 1 paths already cached"4268worker1 # [ 69.128257] nix-grpc-daemon[485]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455364269worker1 # [ 69.134108] nix-grpc-daemon[485]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455364270client # copying 3 paths...4271worker1 # [ 69.144634] nix-grpc-daemon[485]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455364272client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4273worker2 # [ 69.165402] nix-grpc-daemon[770]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:505204274worker2 # [ 69.170339] nix-grpc-daemon[770]: ts=2026-09-25T15:13:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=0 paths=1 nar_bytes_out=2324275worker1 # [ 69.172791] nix-grpc-daemon[485]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:455364276worker1 # [ 69.177247] nix-grpc-daemon[485]: ts=2026-09-25T15:13:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_out=1764277worker1 # [ 69.181738] nix-grpc-daemon[485]: ts=2026-09-25T15:13:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:455364278worker1 # [ 69.185923] nix-grpc-daemon[485]: ts=2026-09-25T15:13:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_out=1284279client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4280client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4281client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.42 seconds)4282(finished: subtest: outputs read back through any worker carry the cache signature, in 5.28 seconds)4283subtest: two clients wanting the same drv build it once4284client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4285client # [ 69.255808] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4286client # Running as unit: dedup1.service; invocation ID: bbb5271fef0a41b189a18bcd470c23ff4287client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.05 seconds)4288client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4289worker2 # [ 69.459150] nix-grpc-daemon[770]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505204290client # [ 69.462781] nix[940]: don't know how to build these paths:4291client # [ 69.463825] nix[940]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4292worker2 # [ 69.473940] nix-grpc-daemon[770]: ts=2026-09-25T15:13:54Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04293worker2 # [ 69.479873] nix-grpc-daemon[770]: ts=2026-09-25T15:13:54Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14294worker2 # [ 69.485389] nix-grpc-daemon[770]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204295worker2 # [ 69.490167] nix-grpc-daemon[770]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505204296client # [ 69.495133] nix[940]: copying 1 paths to 'lb:50051'...4297client # [ 69.496275] nix[940]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4298worker2 # [ 69.497320] nix-grpc-daemon[770]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505204299worker2 # [ 69.507337] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4300worker2 # [ 69.638230] nix-daemon[859]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)4301worker2 # [ 69.658496] nix-daemon[859]: reaped child process 1054, status = succeeded4302lb # [ 69.662794] niks3-server[984]: 2026/09/25 15:13:54 INFO Received push request method=POST path=/api/pushes4303worker2 # [ 69.671156] nix-grpc-daemon[873]: time=2026-09-25T15:13:54.264Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4304worker2 # [ 69.673782] nix-grpc-daemon[873]: time=2026-09-25T15:13:54.264Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4305lb # [ 69.684417] niks3-server[984]: 2026/09/25 15:13:54 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4306worker2 # [ 69.691625] nix-grpc-daemon[873]: time=2026-09-25T15:13:54.284Z level=INFO msg="Uploading 1 narinfos"4307lb # [ 69.693029] niks3-server[984]: 2026/09/25 15:13:54 INFO Signed narinfos id=15 count=14308lb # [ 69.698783] niks3-server[984]: 2026/09/25 15:13:54 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4309lb # [ 69.707037] niks3-server[984]: 2026/09/25 15:13:54 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4310lb # [ 69.717395] niks3-server[984]: 2026/09/25 15:13:54 INFO Received complete push request method=POST path=/api/pushes/15/complete4311worker2 # [ 69.722261] nix-grpc-daemon[873]: time=2026-09-25T15:13:54.315Z level=INFO msg="Upload complete. (204ms)"4312lb # [ 69.723373] niks3-server[984]: 2026/09/25 15:13:54 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4313worker2 # [ 69.728250] nix-grpc-daemon[770]: ts=2026-09-25T15:13:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=0 paths=1 nar_bytes_in=4324314worker2 # [ 69.739979] nix-grpc-daemon[770]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204315worker2 # [ 69.747414] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4316worker2 # [ 69.750181] nix-daemon[859]: reaped child process 1049, status = succeeded4317client # [ 69.757375] nix[940]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4318worker2 # [ 69.759874] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4319client # [ 70.326024] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4320client # Running as unit: dedup2.service; invocation ID: c18e293071914bd7894b250bb49f5e1f4321client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.08 seconds)4322worker1 # [ 70.515037] nix-grpc-daemon[485]: ts=2026-09-25T15:13:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455364323client # [ 70.520467] nix[956]: don't know how to build these paths:4324client # [ 70.521430] nix[956]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4325worker2 # [ 70.527905] nix-grpc-daemon[770]: ts=2026-09-25T15:13:55Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04326worker2 # [ 70.532368] nix-grpc-daemon[770]: ts=2026-09-25T15:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204327client # [ 70.535614] nix[956]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4328client: must succeed: sleep 24329client: (finished: must succeed: sleep 2, in 2.03 seconds)4330client: waiting for success: ! systemctl is-active dedup1 dedup24331worker2 # [ 73.837949] nix-daemon[859]: reaped child process 1058, status = succeeded4332worker2 # [ 73.896310] nix-daemon[859]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4333worker2 # [ 73.919726] nix-daemon[859]: reaped child process 1093, status = succeeded4334lb # [ 73.916926] niks3-server[984]: 2026/09/25 15:13:58 INFO Received push request method=POST path=/api/pushes4335worker2 # [ 73.928528] nix-grpc-daemon[873]: time=2026-09-25T15:13:58.519Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4336worker2 # [ 73.930111] nix-grpc-daemon[873]: time=2026-09-25T15:13:58.519Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4337lb # [ 73.953549] niks3-server[984]: 2026/09/25 15:13:58 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4338worker2 # [ 73.961574] nix-grpc-daemon[873]: time=2026-09-25T15:13:58.554Z level=INFO msg="Uploading 1 narinfos"4339lb # [ 73.961232] niks3-server[984]: 2026/09/25 15:13:58 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4340lb # [ 73.963852] niks3-server[984]: 2026/09/25 15:13:58 INFO Signed narinfos id=16 count=14341lb # [ 73.971633] niks3-server[984]: 2026/09/25 15:13:58 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4342lb # [ 73.981129] niks3-server[984]: 2026/09/25 15:13:58 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4343lb # [ 73.983055] niks3-server[984]: 2026/09/25 15:13:58 INFO Received complete push request method=POST path=/api/pushes/16/complete4344worker2 # [ 73.991333] nix-grpc-daemon[873]: time=2026-09-25T15:13:58.584Z level=INFO msg="Upload complete. (218ms)"4345lb # [ 73.993388] niks3-server[984]: 2026/09/25 15:13:58 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4346worker2 # [ 74.000803] nix-grpc-daemon[770]: ts=2026-09-25T15:13:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14347worker2 # [ 74.013116] nix-grpc-daemon[770]: ts=2026-09-25T15:13:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14348worker2 # [ 74.017858] nix-daemon[859]: reaped child process 1056, status = succeeded4349client # [ 74.035916] systemd[1]: dedup2.service: Deactivated successfully.4350client # [ 74.038275] systemd[1]: dedup2.service: Consumed 175ms CPU time over 3.709s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4351client # [ 74.044418] systemd[1]: dedup1.service: Deactivated successfully.4352client # [ 74.047278] systemd[1]: dedup1.service: Consumed 177ms CPU time over 4.788s wall clock time, 39.5M memory peak, 7K incoming IP traffic, 8.3K outgoing IP traffic.4353client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.07 seconds)4354client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4355client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4356worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4357worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4358worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4359worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4360worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4361worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4362worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4363worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4364(finished: subtest: two clients wanting the same drv build it once, in 5.74 seconds)4365subtest: --timeout and --max-silent-time reach the builder4366worker1 # [ 75.150951] nix-grpc-daemon[485]: ts=2026-09-25T15:13:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455364367worker2 # [ 75.165918] nix-grpc-daemon[770]: ts=2026-09-25T15:13:59Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04368worker1 # [ 75.166097] nix-grpc-daemon[485]: ts=2026-09-25T15:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364369worker2 # [ 75.169603] nix-grpc-daemon[770]: ts=2026-09-25T15:13:59Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14370worker1 # [ 75.177724] nix-grpc-daemon[485]: ts=2026-09-25T15:13:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364371worker1 # [ 75.189967] nix-grpc-daemon[485]: ts=2026-09-25T15:13:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455364372worker1 # [ 75.200164] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4373worker1 # [ 75.344637] nix-daemon[837]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4374lb # [ 75.367114] niks3-server[984]: 2026/09/25 15:13:59 INFO Received push request method=POST path=/api/pushes4375worker1 # [ 75.370016] nix-daemon[837]: reaped child process 988, status = succeeded4376worker1 # [ 75.423895] nix-grpc-daemon[862]: time=2026-09-25T15:14:00.012Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4377worker1 # [ 75.426968] nix-grpc-daemon[862]: time=2026-09-25T15:14:00.012Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4378lb # [ 75.434896] niks3-server[984]: 2026/09/25 15:14:00 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4379worker1 # [ 75.442172] nix-grpc-daemon[862]: time=2026-09-25T15:14:00.032Z level=INFO msg="Uploading 1 narinfos"4380lb # [ 75.444644] niks3-server[984]: 2026/09/25 15:14:00 INFO Signed narinfos id=17 count=14381lb # [ 75.449943] niks3-server[984]: 2026/09/25 15:14:00 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4382lb # [ 75.463493] niks3-server[984]: 2026/09/25 15:14:00 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4383lb # [ 75.477384] niks3-server[984]: 2026/09/25 15:14:00 INFO Received complete push request method=POST path=/api/pushes/17/complete4384worker1 # [ 75.484350] nix-grpc-daemon[862]: time=2026-09-25T15:14:00.074Z level=INFO msg="Upload complete. (273ms)"4385lb # [ 75.485690] niks3-server[984]: 2026/09/25 15:14:00 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4386worker1 # [ 75.492184] nix-grpc-daemon[485]: ts=2026-09-25T15:14:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_in=4244387worker1 # [ 75.500898] nix-grpc-daemon[485]: ts=2026-09-25T15:14:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364388worker1 # [ 75.515071] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4389worker1 # [ 75.517652] nix-daemon[837]: reaped child process 983, status = succeeded4390worker1 # [ 75.530443] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4391lb # [ 77.096944] envoy[893]: conn peer=[2001:db8:1::1]:43780 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9459 tx=4346 ms=41434392lb # [ 77.099998] envoy[893]: conn peer=[2001:db8:1::1]:43794 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2870 ms=1024393lb # [ 77.102686] envoy[893]: conn peer=[2001:db8:1::1]:43836 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=284394lb # [ 77.105395] envoy[893]: conn peer=[2001:db8:1::1]:43822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=374395lb # [ 77.108140] envoy[893]: conn peer=[2001:db8:1::1]:43810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=474396lb # [ 77.110693] envoy[893]: conn peer=[2001:db8:1::1]:43840 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3001 tx=1545 ms=45534397lb # [ 77.113391] envoy[893]: conn peer=[2001:db8:1::1]:43850 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1194 ms=34974398worker1 # [ 81.151661] nix-grpc-daemon[485]: ts=2026-09-25T15:14:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04399worker1 # [ 81.203804] nix-daemon[837]: reaped child process 990, status = succeeded4400worker1 # [ 81.205800] nix-daemon[837]: reaped child process 992, status = succeeded4401worker1 # [ 81.380303] nix-grpc-daemon[485]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455364402worker2 # [ 81.395742] nix-grpc-daemon[770]: ts=2026-09-25T15:14:05Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04403worker2 # [ 81.400724] nix-grpc-daemon[770]: ts=2026-09-25T15:14:05Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14404worker2 # [ 81.405592] nix-grpc-daemon[770]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204405worker2 # [ 81.413677] nix-grpc-daemon[770]: ts=2026-09-25T15:14:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505204406worker2 # [ 81.424325] nix-grpc-daemon[770]: ts=2026-09-25T15:14:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505204407worker2 # [ 81.433314] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4408worker2 # [ 81.563315] nix-daemon[859]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)4409lb # [ 81.582531] niks3-server[984]: 2026/09/25 15:14:06 INFO Received push request method=POST path=/api/pushes4410worker2 # [ 81.587330] nix-daemon[859]: reaped child process 1113, status = succeeded4411worker2 # [ 81.590514] nix-grpc-daemon[873]: time=2026-09-25T15:14:06.183Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4412worker2 # [ 81.592206] nix-grpc-daemon[873]: time=2026-09-25T15:14:06.183Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4413lb # [ 81.604783] niks3-server[984]: 2026/09/25 15:14:06 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4414worker2 # [ 81.610220] nix-grpc-daemon[873]: time=2026-09-25T15:14:06.203Z level=INFO msg="Uploading 1 narinfos"4415lb # [ 81.611489] niks3-server[984]: 2026/09/25 15:14:06 INFO Signed narinfos id=18 count=14416lb # [ 81.617347] niks3-server[984]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4417lb # [ 81.629826] niks3-server[984]: 2026/09/25 15:14:06 INFO Received complete push request method=POST path=/api/pushes/18/complete4418worker2 # [ 81.638967] nix-grpc-daemon[873]: time=2026-09-25T15:14:06.232Z level=INFO msg="Upload complete. (195ms)"4419lb # [ 81.637783] niks3-server[984]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4420worker2 # [ 81.647197] nix-grpc-daemon[770]: ts=2026-09-25T15:14:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=0 paths=1 nar_bytes_in=4244421lb # [ 81.646876] niks3-server[984]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4422worker2 # [ 81.658166] nix-grpc-daemon[770]: ts=2026-09-25T15:14:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204423worker2 # [ 81.663136] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4424worker2 # [ 81.672162] nix-daemon[859]: reaped child process 1108, status = succeeded4425worker2 # [ 81.678855] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4426lb # [ 87.100811] envoy[893]: conn peer=[2001:db8:1::1]:37410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3001 tx=950 ms=60454427worker2 # [ 87.491741] nix-grpc-daemon[770]: ts=2026-09-25T15:14:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04428worker2 # [ 87.510812] nix-daemon[859]: reaped child process 1115, status = succeeded4429worker2 # [ 87.512092] nix-daemon[859]: reaped child process 1117, status = succeeded4430worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4431worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4432worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4433worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4434(finished: subtest: --timeout and --max-silent-time reach the builder, in 12.71 seconds)4435subtest: a failing build is counted with its own reason4436worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4437worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4438worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4439worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4440worker2 # [ 87.922129] nix-grpc-daemon[770]: ts=2026-09-25T15:14:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505204441worker2 # [ 87.932802] nix-grpc-daemon[770]: ts=2026-09-25T15:14:12Z level=debug event=want client=9 drv=j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv cached=04442worker1 # [ 87.933620] nix-grpc-daemon[485]: ts=2026-09-25T15:14:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364443worker2 # [ 87.937172] nix-grpc-daemon[770]: ts=2026-09-25T15:14:12Z level=debug event=assigned drv=j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv worker=192.168.1.3:50051 clients=14444worker1 # [ 87.944061] nix-grpc-daemon[485]: ts=2026-09-25T15:14:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364445worker1 # [ 87.952712] nix-grpc-daemon[485]: ts=2026-09-25T15:14:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455364446worker1 # [ 87.961545] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4447worker1 # [ 88.090612] nix-daemon[837]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)4448lb # [ 88.108345] niks3-server[984]: 2026/09/25 15:14:12 INFO Received push request method=POST path=/api/pushes4449worker1 # [ 88.111665] nix-daemon[837]: reaped child process 1024, status = succeeded4450worker1 # [ 88.115378] nix-grpc-daemon[862]: time=2026-09-25T15:14:12.705Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4451worker1 # [ 88.117057] nix-grpc-daemon[862]: time=2026-09-25T15:14:12.705Z level=INFO msg="Uploading j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv (400B)"4452lb # [ 88.130530] niks3-server[984]: 2026/09/25 15:14:12 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4453worker1 # [ 88.135664] nix-grpc-daemon[862]: time=2026-09-25T15:14:12.725Z level=INFO msg="Uploading 1 narinfos"4454lb # [ 88.137232] niks3-server[984]: 2026/09/25 15:14:12 INFO Signed narinfos id=19 count=14455lb # [ 88.143633] niks3-server[984]: 2026/09/25 15:14:12 INFO Registered completed upload object_key=nar/0d79vqzwc4s1agk431w3csq2p4brahnzmanwj14fjf383ld797i6.nar.zst4456lb # [ 88.150929] niks3-server[984]: 2026/09/25 15:14:12 INFO Registered completed upload object_key=j4rjl229w5xzbr9614xcipiv7652fdka.ls4457lb # [ 88.160553] niks3-server[984]: 2026/09/25 15:14:12 INFO Received complete push request method=POST path=/api/pushes/19/complete4458worker1 # [ 88.164703] nix-grpc-daemon[862]: time=2026-09-25T15:14:12.755Z level=INFO msg="Upload complete. (193ms)"4459lb # [ 88.167064] niks3-server[984]: 2026/09/25 15:14:12 INFO Registered completed upload object_key=j4rjl229w5xzbr9614xcipiv7652fdka.narinfo4460worker1 # [ 88.172473] nix-grpc-daemon[485]: ts=2026-09-25T15:14:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_in=4004461worker1 # [ 88.175800] nix-grpc-daemon[485]: ts=2026-09-25T15:14:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364462worker1 # [ 88.187758] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4463worker1 # [ 88.191681] nix-daemon[837]: reaped child process 1019, status = succeeded4464worker1 # [ 88.200432] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4465worker1 # [ 89.610777] nix-grpc-daemon[485]: ts=2026-09-25T15:14:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=1 drv=j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv assign_id=11 outputs=04466worker1 # [ 89.634257] nix-daemon[837]: reaped child process 1028, status = succeeded4467worker1 # [ 89.635974] nix-daemon[837]: reaped child process 1026, status = succeeded4468worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4469worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4470worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4471worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4472worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4473worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4474worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4475worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4476worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4477worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4478worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4479worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4480worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4481worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4482worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4483worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4484(finished: subtest: a failing build is counted with its own reason, in 2.25 seconds)4485subtest: the first client leaving does not fail the second4486client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4487client # [ 89.960873] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4488client # Running as unit: share1.service; invocation ID: d935de3389704c58a0ce2fb8653eeab64489client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.06 seconds)4490client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4491worker1 # [ 90.152705] nix-grpc-daemon[485]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455364492client # [ 90.158761] nix[1044]: don't know how to build these paths:4493client # [ 90.159724] nix[1044]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4494worker2 # [ 90.167037] nix-grpc-daemon[770]: ts=2026-09-25T15:14:14Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04495worker2 # [ 90.172137] nix-grpc-daemon[770]: ts=2026-09-25T15:14:14Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14496worker2 # [ 90.177263] nix-grpc-daemon[770]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204497worker2 # [ 90.181682] nix-grpc-daemon[770]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505204498client # [ 90.186386] nix[1044]: copying 1 paths to 'lb:50051'...4499client # [ 90.187463] nix[1044]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4500worker2 # [ 90.188341] nix-grpc-daemon[770]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505204501worker2 # [ 90.197380] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4502worker2 # [ 90.344632] nix-daemon[859]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)4503worker2 # [ 90.366254] nix-daemon[859]: reaped child process 1165, status = succeeded4504lb # [ 90.369320] niks3-server[984]: 2026/09/25 15:14:14 INFO Received push request method=POST path=/api/pushes4505worker2 # [ 90.379286] nix-grpc-daemon[873]: time=2026-09-25T15:14:14.971Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4506worker2 # [ 90.382256] nix-grpc-daemon[873]: time=2026-09-25T15:14:14.972Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4507worker2 # [ 90.397892] nix-grpc-daemon[873]: time=2026-09-25T15:14:14.991Z level=INFO msg="Uploading 1 narinfos"4508lb # [ 90.389875] niks3-server[984]: 2026/09/25 15:14:14 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign4509lb # [ 90.399380] niks3-server[984]: 2026/09/25 15:14:14 INFO Signed narinfos id=20 count=14510lb # [ 90.406407] niks3-server[984]: 2026/09/25 15:14:14 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4511lb # [ 90.412297] niks3-server[984]: 2026/09/25 15:14:14 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4512lb # [ 90.424565] niks3-server[984]: 2026/09/25 15:14:15 INFO Received complete push request method=POST path=/api/pushes/20/complete4513worker2 # [ 90.432863] nix-grpc-daemon[873]: time=2026-09-25T15:14:15.026Z level=INFO msg="Upload complete. (223ms)"4514lb # [ 90.431180] niks3-server[984]: 2026/09/25 15:14:15 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4515worker2 # [ 90.440350] nix-grpc-daemon[770]: ts=2026-09-25T15:14:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=0 paths=1 nar_bytes_in=4324516worker2 # [ 90.443840] nix-grpc-daemon[770]: ts=2026-09-25T15:14:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204517worker2 # [ 90.457176] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4518worker2 # [ 90.461794] nix-daemon[859]: reaped child process 1160, status = succeeded4519client # [ 90.469030] nix[1044]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4520worker2 # [ 90.471947] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4521client # [ 91.045383] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4522client # Running as unit: share2.service; invocation ID: 231fc68d7372477cb27fef67fe703f194523client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.09 seconds)4524worker1 # [ 91.231915] nix-grpc-daemon[485]: ts=2026-09-25T15:14:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455364525client # [ 91.236568] nix[1060]: don't know how to build these paths:4526client # [ 91.238501] nix[1060]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4527worker2 # [ 91.246483] nix-grpc-daemon[770]: ts=2026-09-25T15:14:15Z level=debug event=want client=11 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04528client # [ 91.249579] nix[1060]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4529worker2 # [ 91.250842] nix-grpc-daemon[770]: ts=2026-09-25T15:14:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204530client: must succeed: sleep 2; systemctl kill -s INT share14531client # [ 94.286161] systemd[1]: share1.service: Sent signal SIGINT to main process 1044 (nix) on client request.4532client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.05 seconds)4533client: waiting for success: ! systemctl is-active share14534client # [ 94.291908] nix[1044]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4535client # [ 94.292991] nix[1044]: error: interrupted by the user4536client # [ 94.342831] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4537client # [ 94.350819] systemd[1]: share1.service: Failed with result 'exit-code'.4538client # [ 94.352782] systemd[1]: share1.service: Consumed 166ms CPU time over 4.389s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.4539client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)4540client: waiting for success: ! systemctl is-active share24541worker2 # [ 95.703880] nix-daemon[859]: reaped child process 1169, status = succeeded4542worker2 # [ 95.813149] nix-daemon[859]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)4543worker2 # [ 95.834725] nix-daemon[859]: reaped child process 1208, status = succeeded4544lb # [ 95.839188] niks3-server[984]: 2026/09/25 15:14:20 INFO Received push request method=POST path=/api/pushes4545worker2 # [ 95.848429] nix-grpc-daemon[873]: time=2026-09-25T15:14:20.440Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4546worker2 # [ 95.850034] nix-grpc-daemon[873]: time=2026-09-25T15:14:20.440Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4547lb # [ 95.872943] niks3-server[984]: 2026/09/25 15:14:20 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign4548worker2 # [ 95.880749] nix-grpc-daemon[873]: time=2026-09-25T15:14:20.473Z level=INFO msg="Uploading 1 narinfos"4549lb # [ 95.880468] niks3-server[984]: 2026/09/25 15:14:20 INFO Signed narinfos id=21 count=14550lb # [ 95.887593] niks3-server[984]: 2026/09/25 15:14:20 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4551lb # [ 95.898196] niks3-server[984]: 2026/09/25 15:14:20 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4552lb # [ 95.904527] niks3-server[984]: 2026/09/25 15:14:20 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4553lb # [ 95.914888] niks3-server[984]: 2026/09/25 15:14:20 INFO Received complete push request method=POST path=/api/pushes/21/complete4554worker2 # [ 95.921890] nix-grpc-daemon[873]: time=2026-09-25T15:14:20.515Z level=INFO msg="Upload complete. (233ms)"4555lb # [ 95.922697] niks3-server[984]: 2026/09/25 15:14:20 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4556worker2 # [ 95.930062] nix-grpc-daemon[770]: ts=2026-09-25T15:14:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=12 outputs=14557worker2 # [ 95.943118] nix-grpc-daemon[770]: ts=2026-09-25T15:14:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=12 outputs=14558worker2 # [ 95.947841] nix-daemon[859]: reaped child process 1167, status = succeeded4559client # [ 95.952620] systemd[1]: share2.service: Deactivated successfully.4560client # [ 95.955094] systemd[1]: share2.service: Consumed 168ms CPU time over 4.909s wall clock time, 33.3M memory peak, 6K incoming IP traffic, 5.6K outgoing IP traffic.4561client: (finished: waiting for success: ! systemctl is-active share2, in 1.07 seconds)4562client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4563client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds)4564(finished: subtest: the first client leaving does not fail the second, in 6.75 seconds)4565subtest: build hook: nix-daemon with builders = grpc://lb4566client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4567client # [ 96.794071] systemd[1]: Started Nix Daemon.4568client # [ 96.911276] nix-daemon[1111]: accepted connection from pid 1108, user root (trusted)4569worker2 # [ 97.055439] nix-grpc-daemon[770]: ts=2026-09-25T15:14:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:505204570worker1 # [ 97.057090] nix-grpc-daemon[485]: ts=2026-09-25T15:14:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364571worker2 # [ 97.069013] nix-grpc-daemon[770]: ts=2026-09-25T15:14:21Z level=debug event=want client=12 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04572worker2 # [ 97.072995] nix-grpc-daemon[770]: ts=2026-09-25T15:14:21Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14573worker1 # [ 97.073747] nix-grpc-daemon[485]: ts=2026-09-25T15:14:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364574worker1 # [ 97.083881] nix-grpc-daemon[485]: ts=2026-09-25T15:14:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364575worker1 # [ 97.093512] nix-grpc-daemon[485]: ts=2026-09-25T15:14:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455364576lb # [ 97.097757] envoy[893]: conn peer=[2001:db8:1::1]:55096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2985 tx=935 ms=61274577lb # [ 97.100639] envoy[893]: conn peer=[2001:db8:1::1]:55104 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2881 tx=1245 ms=17074578worker1 # [ 97.102481] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4579lb # [ 97.103484] envoy[893]: conn peer=[2001:db8:1::1]:55118 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=772 ms=41494580lb # [ 97.106216] envoy[893]: conn peer=[2001:db8:1::1]:56682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=47084581worker1 # [ 97.230996] nix-daemon[837]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4582lb # [ 97.249723] niks3-server[984]: 2026/09/25 15:14:21 INFO Received push request method=POST path=/api/pushes4583worker1 # [ 97.250732] nix-daemon[837]: reaped child process 1077, status = succeeded4584worker1 # [ 97.258856] nix-grpc-daemon[862]: time=2026-09-25T15:14:21.848Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4585worker1 # [ 97.260730] nix-grpc-daemon[862]: time=2026-09-25T15:14:21.848Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4586lb # [ 97.272974] niks3-server[984]: 2026/09/25 15:14:21 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign4587worker1 # [ 97.276899] nix-grpc-daemon[862]: time=2026-09-25T15:14:21.867Z level=INFO msg="Uploading 1 narinfos"4588lb # [ 97.282544] niks3-server[984]: 2026/09/25 15:14:21 INFO Signed narinfos id=22 count=14589lb # [ 97.286805] niks3-server[984]: 2026/09/25 15:14:21 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4590lb # [ 97.295847] niks3-server[984]: 2026/09/25 15:14:21 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4591lb # [ 97.298813] niks3-server[984]: 2026/09/25 15:14:21 INFO Received complete push request method=POST path=/api/pushes/22/complete4592worker1 # [ 97.301152] nix-grpc-daemon[862]: time=2026-09-25T15:14:21.891Z level=INFO msg="Upload complete. (188ms)"4593lb # [ 97.303081] niks3-server[984]: 2026/09/25 15:14:21 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4594worker1 # [ 97.308833] nix-grpc-daemon[485]: ts=2026-09-25T15:14:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_in=5124595worker1 # [ 97.318325] nix-grpc-daemon[485]: ts=2026-09-25T15:14:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364596worker1 # [ 97.323347] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4597worker1 # [ 97.327661] nix-daemon[837]: reaped child process 1072, status = succeeded4598worker1 # [ 97.336202] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4599worker1 # [ 98.571983] nix-daemon[837]: reaped child process 1081, status = succeeded4600worker1 # [ 98.688545] nix-daemon[837]: accepted connection from pid 1097, user nix-grpc-daemon (trusted)4601worker1 # [ 98.710578] nix-daemon[837]: reaped child process 1100, status = succeeded4602lb # [ 98.722609] niks3-server[984]: 2026/09/25 15:14:23 INFO Received push request method=POST path=/api/pushes4603worker1 # [ 98.736492] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.324Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4604worker1 # [ 98.738324] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.324Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4605worker1 # [ 98.768530] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.358Z level=INFO msg="Uploading 1 narinfos"4606lb # [ 98.761407] niks3-server[984]: 2026/09/25 15:14:23 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign4607lb # [ 98.773184] niks3-server[984]: 2026/09/25 15:14:23 INFO Signed narinfos id=23 count=14608lb # [ 98.778120] niks3-server[984]: 2026/09/25 15:14:23 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4609lb # [ 98.785995] niks3-server[984]: 2026/09/25 15:14:23 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4610lb # [ 98.790447] niks3-server[984]: 2026/09/25 15:14:23 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4611lb # [ 98.798511] niks3-server[984]: 2026/09/25 15:14:23 INFO Received complete push request method=POST path=/api/pushes/23/complete4612worker1 # [ 98.801890] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.391Z level=INFO msg="Upload complete. (254ms)"4613lb # [ 98.803717] niks3-server[984]: 2026/09/25 15:14:23 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4614worker1 # [ 98.808570] nix-grpc-daemon[485]: ts=2026-09-25T15:14:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=13 outputs=14615worker1 # [ 98.819134] nix-daemon[837]: reaped child process 1079, status = succeeded4616worker2 # [ 98.820388] nix-grpc-daemon[770]: ts=2026-09-25T15:14:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:505204617worker2 # [ 98.828963] nix-grpc-daemon[770]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4618worker1 # [ 98.858435] nix-grpc-daemon[485]: ts=2026-09-25T15:14:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:455364619worker1 # [ 98.862723] nix-grpc-daemon[485]: ts=2026-09-25T15:14:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_out=1284620worker2 # [ 99.070199] nix-grpc-daemon[770]: ts=2026-09-25T15:14:23Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:505204621worker1 # [ 99.072746] nix-grpc-daemon[485]: ts=2026-09-25T15:14:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364622worker1 # [ 99.079173] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.669Z level=INFO msg="All 1 paths already cached"4623worker2 # [ 99.093565] nix-grpc-daemon[770]: ts=2026-09-25T15:14:23Z level=debug event=want client=13 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04624worker1 # [ 99.093958] nix-grpc-daemon[485]: ts=2026-09-25T15:14:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364625worker2 # [ 99.097007] nix-grpc-daemon[770]: ts=2026-09-25T15:14:23Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14626worker1 # [ 99.104208] nix-grpc-daemon[485]: ts=2026-09-25T15:14:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364627worker1 # [ 99.112934] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.703Z level=INFO msg="All 1 paths already cached"4628worker1 # [ 99.119380] nix-grpc-daemon[485]: ts=2026-09-25T15:14:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455364629worker1 # [ 99.128246] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4630worker1 # [ 99.253452] nix-daemon[837]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4631lb # [ 99.270951] niks3-server[984]: 2026/09/25 15:14:23 INFO Received push request method=POST path=/api/pushes4632worker1 # [ 99.274337] nix-daemon[837]: reaped child process 1107, status = succeeded4633worker1 # [ 99.276985] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.866Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4634worker1 # [ 99.278545] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.867Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4635lb # [ 99.293827] niks3-server[984]: 2026/09/25 15:14:23 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign4636worker1 # [ 99.297651] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.887Z level=INFO msg="Uploading 1 narinfos"4637lb # [ 99.301323] niks3-server[984]: 2026/09/25 15:14:23 INFO Signed narinfos id=24 count=14638lb # [ 99.307316] niks3-server[984]: 2026/09/25 15:14:23 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4639lb # [ 99.311228] niks3-server[984]: 2026/09/25 15:14:23 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4640lb # [ 99.321505] niks3-server[984]: 2026/09/25 15:14:23 INFO Received complete push request method=POST path=/api/pushes/24/complete4641worker1 # [ 99.328687] nix-grpc-daemon[862]: time=2026-09-25T15:14:23.919Z level=INFO msg="Upload complete. (189ms)"4642lb # [ 99.331580] niks3-server[984]: 2026/09/25 15:14:23 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4643worker1 # [ 99.336169] nix-grpc-daemon[485]: ts=2026-09-25T15:14:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_in=6244644worker1 # [ 99.346506] nix-grpc-daemon[485]: ts=2026-09-25T15:14:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364645worker1 # [ 99.351268] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4646worker1 # [ 99.355780] nix-daemon[837]: reaped child process 1102, status = succeeded4647worker1 # [ 99.367010] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4648worker1 # [ 100.202907] nix-daemon[837]: reaped child process 1111, status = succeeded4649worker1 # [ 100.303170] nix-daemon[837]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)4650worker1 # [ 100.320342] nix-daemon[837]: reaped child process 1130, status = succeeded4651lb # [ 100.327059] niks3-server[984]: 2026/09/25 15:14:24 INFO Received push request method=POST path=/api/pushes4652worker1 # [ 100.334144] nix-grpc-daemon[862]: time=2026-09-25T15:14:24.922Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4653worker1 # [ 100.335685] nix-grpc-daemon[862]: time=2026-09-25T15:14:24.922Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4654lb # [ 100.365747] niks3-server[984]: 2026/09/25 15:14:24 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign4655worker1 # [ 100.370542] nix-grpc-daemon[862]: time=2026-09-25T15:14:24.960Z level=INFO msg="Uploading 1 narinfos"4656lb # [ 100.373902] niks3-server[984]: 2026/09/25 15:14:24 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4657lb # [ 100.380768] niks3-server[984]: 2026/09/25 15:14:24 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4658lb # [ 100.389618] niks3-server[984]: 2026/09/25 15:14:24 INFO Signed narinfos id=25 count=14659lb # [ 100.391345] niks3-server[984]: 2026/09/25 15:14:24 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4660worker1 # [ 100.394985] nix-grpc-daemon[862]: time=2026-09-25T15:14:24.985Z level=INFO msg="Upload complete. (220ms)"4661lb # [ 100.396514] niks3-server[984]: 2026/09/25 15:14:24 INFO Received complete push request method=POST path=/api/pushes/25/complete4662lb # [ 100.398959] niks3-server[984]: 2026/09/25 15:14:24 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4663worker1 # [ 100.404683] nix-grpc-daemon[485]: ts=2026-09-25T15:14:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=14 outputs=14664worker2 # [ 100.409118] nix-grpc-daemon[770]: ts=2026-09-25T15:14:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:505204665worker1 # [ 100.412232] nix-daemon[837]: reaped child process 1109, status = succeeded4666worker2 # [ 100.417664] nix-grpc-daemon[770]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4667worker1 # [ 100.439970] nix-grpc-daemon[485]: ts=2026-09-25T15:14:25Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:455364668worker1 # [ 100.444171] nix-grpc-daemon[485]: ts=2026-09-25T15:14:25Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_out=1764669worker2 # [ 100.637690] nix-grpc-daemon[770]: ts=2026-09-25T15:14:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:505204670worker1 # [ 100.638685] nix-grpc-daemon[485]: ts=2026-09-25T15:14:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364671worker1 # [ 100.644729] nix-grpc-daemon[862]: time=2026-09-25T15:14:25.235Z level=INFO msg="All 2 paths already cached"4672worker2 # [ 100.660442] nix-grpc-daemon[770]: ts=2026-09-25T15:14:25Z level=debug event=want client=14 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04673worker1 # [ 100.660611] nix-grpc-daemon[485]: ts=2026-09-25T15:14:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364674worker2 # [ 100.663824] nix-grpc-daemon[770]: ts=2026-09-25T15:14:25Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14675worker1 # [ 100.670266] nix-grpc-daemon[485]: ts=2026-09-25T15:14:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364676worker1 # [ 100.678463] nix-grpc-daemon[862]: time=2026-09-25T15:14:25.268Z level=INFO msg="All 2 paths already cached"4677worker1 # [ 100.684635] nix-grpc-daemon[485]: ts=2026-09-25T15:14:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455364678worker1 # [ 100.692819] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4679worker1 # [ 100.813656] nix-daemon[837]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)4680lb # [ 100.833752] niks3-server[984]: 2026/09/25 15:14:25 INFO Received push request method=POST path=/api/pushes4681worker1 # [ 100.836422] nix-daemon[837]: reaped child process 1137, status = succeeded4682worker1 # [ 100.838697] nix-grpc-daemon[862]: time=2026-09-25T15:14:25.428Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4683worker1 # [ 100.840360] nix-grpc-daemon[862]: time=2026-09-25T15:14:25.428Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4684lb # [ 100.852381] niks3-server[984]: 2026/09/25 15:14:25 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign4685worker1 # [ 100.857133] nix-grpc-daemon[862]: time=2026-09-25T15:14:25.447Z level=INFO msg="Uploading 1 narinfos"4686lb # [ 100.860599] niks3-server[984]: 2026/09/25 15:14:25 INFO Signed narinfos id=26 count=14687lb # [ 100.865570] niks3-server[984]: 2026/09/25 15:14:25 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4688lb # [ 100.871482] niks3-server[984]: 2026/09/25 15:14:25 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4689lb # [ 100.882732] niks3-server[984]: 2026/09/25 15:14:25 INFO Received complete push request method=POST path=/api/pushes/26/complete4690worker1 # [ 100.888684] nix-grpc-daemon[862]: time=2026-09-25T15:14:25.479Z level=INFO msg="Upload complete. (186ms)"4691lb # [ 100.889206] niks3-server[984]: 2026/09/25 15:14:25 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4692worker1 # [ 100.895821] nix-grpc-daemon[485]: ts=2026-09-25T15:14:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_in=7524693worker1 # [ 100.904381] nix-grpc-daemon[485]: ts=2026-09-25T15:14:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364694worker1 # [ 100.912109] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4695worker1 # [ 100.915206] nix-daemon[837]: reaped child process 1132, status = succeeded4696worker1 # [ 100.924139] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4697worker1 # [ 102.167440] nix-daemon[837]: reaped child process 1141, status = succeeded4698worker1 # [ 102.272453] nix-daemon[837]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)4699worker1 # [ 102.292792] nix-daemon[837]: reaped child process 1160, status = succeeded4700lb # [ 102.297298] niks3-server[984]: 2026/09/25 15:14:26 INFO Received push request method=POST path=/api/pushes4701worker1 # [ 102.304400] nix-grpc-daemon[862]: time=2026-09-25T15:14:26.892Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4702worker1 # [ 102.306044] nix-grpc-daemon[862]: time=2026-09-25T15:14:26.892Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4703lb # [ 102.324646] niks3-server[984]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4704worker1 # [ 102.335576] nix-grpc-daemon[862]: time=2026-09-25T15:14:26.925Z level=INFO msg="Uploading 1 narinfos"4705lb # [ 102.336487] niks3-server[984]: 2026/09/25 15:14:26 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign4706lb # [ 102.343920] niks3-server[984]: 2026/09/25 15:14:26 INFO Signed narinfos id=27 count=14707lb # [ 102.347250] niks3-server[984]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4708lb # [ 102.354587] niks3-server[984]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4709lb # [ 102.363235] niks3-server[984]: 2026/09/25 15:14:26 INFO Received complete push request method=POST path=/api/pushes/27/complete4710worker1 # [ 102.367420] nix-grpc-daemon[862]: time=2026-09-25T15:14:26.957Z level=INFO msg="Upload complete. (230ms)"4711lb # [ 102.369107] niks3-server[984]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4712worker1 # [ 102.376705] nix-grpc-daemon[485]: ts=2026-09-25T15:14:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=15 outputs=14713worker2 # [ 102.381039] nix-grpc-daemon[770]: ts=2026-09-25T15:14:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:505204714worker1 # [ 102.386401] nix-daemon[837]: reaped child process 1139, status = succeeded4715worker2 # [ 102.389778] nix-grpc-daemon[770]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4716worker2 # [ 102.416891] nix-grpc-daemon[770]: ts=2026-09-25T15:14:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:505204717worker2 # [ 102.422274] nix-grpc-daemon[770]: ts=2026-09-25T15:14:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=0 paths=1 nar_bytes_out=2324718client # [ 102.489272] nix-daemon[1111]: reaped child process 1113, status = succeeded4719client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.83 seconds)4720client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4721client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.02 seconds)4722client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4723client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4724client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4725client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4726(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.92 seconds)4727subtest: an input only one worker has still reaches the builder4728client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4729client # this derivation will be built:4730client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4731client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4732client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds)4733client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4734client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4735worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4736worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.15 seconds)4737worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4738worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.02 seconds)4739client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24740client # [ 103.309166] nix-daemon[1111]: accepted connection from pid 1193, user root (trusted)4741client # this derivation will be built:4742client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4743worker1 # [ 103.453335] nix-grpc-daemon[485]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:455364744client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4745worker1 # [ 103.457153] nix-grpc-daemon[485]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364746worker1 # [ 103.587758] nix-daemon[837]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)4747worker1 # [ 103.607248] nix-daemon[837]: reaped child process 1170, status = succeeded4748lb # [ 103.612160] niks3-server[984]: 2026/09/25 15:14:28 INFO Received push request method=POST path=/api/pushes4749worker1 # [ 103.617527] nix-grpc-daemon[862]: time=2026-09-25T15:14:28.207Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4750worker1 # [ 103.619070] nix-grpc-daemon[862]: time=2026-09-25T15:14:28.207Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4751lb # [ 103.630815] niks3-server[984]: 2026/09/25 15:14:28 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4752worker1 # [ 103.635836] nix-grpc-daemon[862]: time=2026-09-25T15:14:28.226Z level=INFO msg="Uploading 1 narinfos"4753lb # [ 103.637406] niks3-server[984]: 2026/09/25 15:14:28 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign4754lb # [ 103.639826] niks3-server[984]: 2026/09/25 15:14:28 INFO Signed narinfos id=28 count=14755lb # [ 103.647710] niks3-server[984]: 2026/09/25 15:14:28 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4756lb # [ 103.658678] niks3-server[984]: 2026/09/25 15:14:28 INFO Received complete push request method=POST path=/api/pushes/28/complete4757worker1 # [ 103.664644] nix-grpc-daemon[862]: time=2026-09-25T15:14:28.255Z level=INFO msg="Upload complete. (207ms)"4758lb # [ 103.665126] niks3-server[984]: 2026/09/25 15:14:28 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4759worker2 # [ 103.683311] nix-grpc-daemon[770]: ts=2026-09-25T15:14:28Z level=debug event=want client=15 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04760worker2 # [ 103.683546] nix-grpc-daemon[770]: ts=2026-09-25T15:14:28Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14761worker2 # [ 103.690863] nix-grpc-daemon[770]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204762worker2 # [ 103.700483] nix-grpc-daemon[770]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505204763client # copying 2 paths to 'lb:50051'...4764client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4765worker2 # [ 103.711967] nix-grpc-daemon[770]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505204766client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4767worker2 # [ 103.723652] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4768worker2 # [ 103.857321] nix-daemon[859]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)4769worker2 # [ 103.877399] nix-daemon[859]: reaped child process 1229, status = succeeded4770lb # [ 103.882245] niks3-server[984]: 2026/09/25 15:14:28 INFO Received push request method=POST path=/api/pushes4771worker2 # [ 103.890219] nix-grpc-daemon[873]: time=2026-09-25T15:14:28.482Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4772worker2 # [ 103.893114] nix-grpc-daemon[873]: time=2026-09-25T15:14:28.482Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4773worker2 # [ 103.895400] nix-grpc-daemon[873]: time=2026-09-25T15:14:28.482Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4774lb # [ 103.907068] niks3-server[984]: 2026/09/25 15:14:28 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4775worker2 # [ 103.923806] nix-grpc-daemon[873]: time=2026-09-25T15:14:28.516Z level=INFO msg="Uploading 2 narinfos"4776lb # [ 103.914308] niks3-server[984]: 2026/09/25 15:14:28 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4777lb # [ 103.924536] niks3-server[984]: 2026/09/25 15:14:28 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign4778lb # [ 103.926889] niks3-server[984]: 2026/09/25 15:14:28 INFO Signed narinfos id=29 count=24779lb # [ 103.934691] niks3-server[984]: 2026/09/25 15:14:28 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4780lb # [ 103.940171] niks3-server[984]: 2026/09/25 15:14:28 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4781worker2 # [ 103.953370] nix-grpc-daemon[873]: time=2026-09-25T15:14:28.546Z level=INFO msg="Upload complete. (218ms)"4782lb # [ 103.951742] niks3-server[984]: 2026/09/25 15:14:28 INFO Received complete push request method=POST path=/api/pushes/29/complete4783lb # [ 103.958791] niks3-server[984]: 2026/09/25 15:14:28 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4784worker2 # [ 103.959909] nix-grpc-daemon[770]: ts=2026-09-25T15:14:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=0 paths=2 nar_bytes_in=9844785lb # [ 103.962079] niks3-server[984]: 2026/09/25 15:14:28 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4786worker2 # [ 103.968628] nix-grpc-daemon[770]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204787worker2 # [ 103.978240] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4788worker2 # [ 103.982555] nix-daemon[859]: reaped child process 1224, status = succeeded4789worker2 # [ 103.994764] nix-grpc-daemon[770]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4790client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4791worker2 # [ 104.005621] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4792client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4793client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4794worker2 # [ 104.897881] nix-daemon[859]: reaped child process 1234, status = succeeded4795worker2 # [ 104.958853] nix-daemon[859]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)4796worker2 # [ 104.977589] nix-daemon[859]: reaped child process 1253, status = succeeded4797lb # [ 104.982181] niks3-server[984]: 2026/09/25 15:14:29 INFO Received push request method=POST path=/api/pushes4798worker2 # [ 104.992086] nix-grpc-daemon[873]: time=2026-09-25T15:14:29.583Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4799lb # [ 105.003717] niks3-server[984]: 2026/09/25 15:14:29 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4800worker2 # [ 105.011215] nix-grpc-daemon[873]: time=2026-09-25T15:14:29.604Z level=INFO msg="Uploading 1 narinfos"4801lb # [ 105.011264] niks3-server[984]: 2026/09/25 15:14:29 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign4802lb # [ 105.018550] niks3-server[984]: 2026/09/25 15:14:29 INFO Signed narinfos id=30 count=14803lb # [ 105.023180] niks3-server[984]: 2026/09/25 15:14:29 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4804lb # [ 105.034865] niks3-server[984]: 2026/09/25 15:14:29 INFO Received complete push request method=POST path=/api/pushes/30/complete4805worker2 # [ 105.040729] nix-grpc-daemon[873]: time=2026-09-25T15:14:29.634Z level=INFO msg="Upload complete. (224ms)"4806lb # [ 105.042037] niks3-server[984]: 2026/09/25 15:14:29 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4807worker2 # [ 105.048302] nix-grpc-daemon[770]: ts=2026-09-25T15:14:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=16 outputs=14808client # copying 1 paths...4809worker2 # [ 105.062357] nix-daemon[859]: reaped child process 1231, status = succeeded4810worker1 # [ 105.059519] nix-grpc-daemon[485]: ts=2026-09-25T15:14:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455364811worker1 # [ 105.068280] nix-grpc-daemon[485]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4812client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4813worker1 # [ 105.095691] nix-grpc-daemon[485]: ts=2026-09-25T15:14:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:455364814worker1 # [ 105.099849] nix-grpc-daemon[485]: ts=2026-09-25T15:14:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_out=1204815client # [ 105.164249] nix-daemon[1111]: reaped child process 1196, status = succeeded4816client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.00 seconds)4817client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24818client # [ 105.298345] nix-daemon[1111]: accepted connection from pid 1215, user root (trusted)4819client # this derivation will be built:4820client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4821worker2 # [ 105.437414] nix-grpc-daemon[770]: ts=2026-09-25T15:14:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:505204822client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4823worker2 # [ 105.442992] nix-grpc-daemon[770]: ts=2026-09-25T15:14:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505204824worker2 # [ 105.447640] nix-grpc-daemon[873]: time=2026-09-25T15:14:30.040Z level=INFO msg="All 1 paths already cached"4825worker2 # [ 105.457176] nix-grpc-daemon[770]: ts=2026-09-25T15:14:30Z level=debug event=want client=16 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04826worker1 # [ 105.457481] nix-grpc-daemon[485]: ts=2026-09-25T15:14:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364827worker2 # [ 105.460904] nix-grpc-daemon[770]: ts=2026-09-25T15:14:30Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14828worker1 # [ 105.466902] nix-grpc-daemon[485]: ts=2026-09-25T15:14:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364829client # copying 1 paths to 'lb:50051'...4830client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4831worker1 # [ 105.479073] nix-grpc-daemon[485]: ts=2026-09-25T15:14:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455364832worker1 # [ 105.487910] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4833worker1 # [ 105.500589] nix-grpc-daemon[485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)4834worker1 # [ 105.540836] nix-grpc-daemon[485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 107 ms (attempt 2/5)4835worker1 # [ 105.652782] nix-grpc-daemon[485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 159 ms (attempt 3/5)4836worker1 # [ 105.813100] nix-grpc-daemon[485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 338 ms (attempt 4/5)4837worker1 # [ 106.153280] nix-grpc-daemon[485]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4838worker1 # [ 106.155218] nix-grpc-daemon[485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4839worker1 # [ 106.160904] nix-grpc-daemon[485]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4840worker1 # [ 106.302642] nix-daemon[837]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)4841worker1 # [ 106.324794] nix-daemon[837]: reaped child process 1183, status = succeeded4842lb # [ 106.331322] niks3-server[984]: 2026/09/25 15:14:30 INFO Received push request method=POST path=/api/pushes4843worker1 # [ 106.337539] nix-grpc-daemon[862]: time=2026-09-25T15:14:30.927Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4844worker1 # [ 106.339069] nix-grpc-daemon[862]: time=2026-09-25T15:14:30.927Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4845lb # [ 106.352660] niks3-server[984]: 2026/09/25 15:14:30 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign4846worker1 # [ 106.357773] nix-grpc-daemon[862]: time=2026-09-25T15:14:30.948Z level=INFO msg="Uploading 1 narinfos"4847lb # [ 106.360097] niks3-server[984]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4848lb # [ 106.365826] niks3-server[984]: 2026/09/25 15:14:30 INFO Signed narinfos id=31 count=14849lb # [ 106.367569] niks3-server[984]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4850lb # [ 106.377442] niks3-server[984]: 2026/09/25 15:14:30 INFO Received complete push request method=POST path=/api/pushes/31/complete4851worker1 # [ 106.382354] nix-grpc-daemon[862]: time=2026-09-25T15:14:30.971Z level=INFO msg="Upload complete. (208ms)"4852lb # [ 106.382750] niks3-server[984]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4853worker1 # [ 106.387265] nix-grpc-daemon[485]: ts=2026-09-25T15:14:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 paths=1 nar_bytes_in=5684854worker1 # [ 106.395483] nix-grpc-daemon[485]: ts=2026-09-25T15:14:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455364855worker1 # [ 106.404514] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4856worker1 # [ 106.409146] nix-daemon[837]: reaped child process 1173, status = succeeded4857client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4858worker1 # [ 106.417719] nix-daemon[837]: accepted connection from pid 485, user nix-grpc-daemon (trusted)4859client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4860client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4861worker1 # [ 107.077822] nix-daemon[837]: reaped child process 1187, status = succeeded4862lb # [ 107.097448] envoy[893]: conn peer=[2001:db8:1::1]:56690 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=214863lb # [ 107.100139] envoy[893]: conn peer=[2001:db8:1::1]:56688 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3399 tx=1915 ms=18264864lb # [ 107.102986] envoy[893]: conn peer=[2001:db8:1::1]:56702 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=164865lb # [ 107.105563] envoy[893]: conn peer=[2001:db8:1::1]:56700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3871 tx=2137 ms=13874866lb # [ 107.108190] envoy[893]: conn peer=[2001:db8:1::1]:56718 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4322 tx=2395 ms=17974867lb # [ 107.110701] envoy[893]: conn peer=[2001:db8:1::1]:60292 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=204868lb # [ 107.113188] envoy[893]: conn peer=[2001:db8:1::1]:60304 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3857 tx=2051 ms=16594869lb # [ 107.115726] envoy[893]: conn peer=[2001:db8:1::1]:60308 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=174870worker1 # [ 107.134861] nix-daemon[837]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)4871worker1 # [ 107.154558] nix-daemon[837]: reaped child process 1206, status = succeeded4872lb # [ 107.157765] niks3-server[984]: 2026/09/25 15:14:31 INFO Received push request method=POST path=/api/pushes4873worker1 # [ 107.164167] nix-grpc-daemon[862]: time=2026-09-25T15:14:31.753Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4874lb # [ 107.174550] niks3-server[984]: 2026/09/25 15:14:31 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4875lb # [ 107.182729] niks3-server[984]: 2026/09/25 15:14:31 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign4876worker1 # [ 107.185840] nix-grpc-daemon[862]: time=2026-09-25T15:14:31.776Z level=INFO msg="Uploading 1 narinfos"4877lb # [ 107.189421] niks3-server[984]: 2026/09/25 15:14:31 INFO Signed narinfos id=32 count=14878lb # [ 107.191132] niks3-server[984]: 2026/09/25 15:14:31 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4879lb # [ 107.198525] niks3-server[984]: 2026/09/25 15:14:31 INFO Received complete push request method=POST path=/api/pushes/32/complete4880worker1 # [ 107.206813] nix-grpc-daemon[862]: time=2026-09-25T15:14:31.797Z level=INFO msg="Upload complete. (179ms)"4881lb # [ 107.208553] niks3-server[984]: 2026/09/25 15:14:31 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4882client # copying 1 paths...4883worker1 # [ 107.212600] nix-grpc-daemon[485]: ts=2026-09-25T15:14:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45536 duration_s=0 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=17 outputs=14884client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4885worker1 # [ 107.223711] nix-grpc-daemon[485]: ts=2026-09-25T15:14:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455364886worker1 # [ 107.226439] nix-daemon[837]: reaped child process 1185, status = succeeded4887worker2 # [ 107.237781] nix-grpc-daemon[770]: ts=2026-09-25T15:14:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:505204888worker2 # [ 107.245701] nix-grpc-daemon[770]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4889worker2 # [ 107.256607] nix-grpc-daemon[770]: ts=2026-09-25T15:14:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=0 paths=1 nar_bytes_out=1204890client # [ 107.316198] nix-daemon[1111]: reaped child process 1218, status = succeeded4891client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.15 seconds)4892client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24893client # [ 107.453808] nix-daemon[1111]: accepted connection from pid 1237, user root (trusted)4894client # this derivation will be built:4895client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4896worker1 # [ 107.589114] nix-grpc-daemon[485]: ts=2026-09-25T15:14:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:455364897client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4898worker1 # [ 107.593368] nix-grpc-daemon[485]: ts=2026-09-25T15:14:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455364899worker1 # [ 107.597260] nix-grpc-daemon[862]: time=2026-09-25T15:14:32.187Z level=INFO msg="All 1 paths already cached"4900worker2 # [ 107.611311] nix-grpc-daemon[770]: ts=2026-09-25T15:14:32Z level=debug event=want client=17 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04901worker2 # [ 107.617178] nix-grpc-daemon[770]: ts=2026-09-25T15:14:32Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14902worker2 # [ 107.622224] nix-grpc-daemon[770]: ts=2026-09-25T15:14:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204903worker2 # [ 107.626370] nix-grpc-daemon[770]: ts=2026-09-25T15:14:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505204904worker2 # [ 107.633120] nix-grpc-daemon[873]: time=2026-09-25T15:14:32.226Z level=INFO msg="All 1 paths already cached"4905client # copying 1 paths to 'lb:50051'...4906client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4907worker2 # [ 107.639524] nix-grpc-daemon[770]: ts=2026-09-25T15:14:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505204908worker2 # [ 107.648389] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4909worker2 # [ 107.774405] nix-daemon[859]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)4910lb # [ 107.791871] niks3-server[984]: 2026/09/25 15:14:32 INFO Received push request method=POST path=/api/pushes4911worker2 # [ 107.795420] nix-daemon[859]: reaped child process 1265, status = succeeded4912worker2 # [ 107.800006] nix-grpc-daemon[873]: time=2026-09-25T15:14:32.393Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4913worker2 # [ 107.801789] nix-grpc-daemon[873]: time=2026-09-25T15:14:32.393Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4914lb # [ 107.812062] niks3-server[984]: 2026/09/25 15:14:32 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign4915worker2 # [ 107.820002] nix-grpc-daemon[873]: time=2026-09-25T15:14:32.413Z level=INFO msg="Uploading 1 narinfos"4916lb # [ 107.820238] niks3-server[984]: 2026/09/25 15:14:32 INFO Signed narinfos id=33 count=14917lb # [ 107.822150] niks3-server[984]: 2026/09/25 15:14:32 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4918lb # [ 107.830842] niks3-server[984]: 2026/09/25 15:14:32 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4919lb # [ 107.841482] niks3-server[984]: 2026/09/25 15:14:32 INFO Received complete push request method=POST path=/api/pushes/33/complete4920worker2 # [ 107.848508] nix-grpc-daemon[873]: time=2026-09-25T15:14:32.441Z level=INFO msg="Upload complete. (190ms)"4921lb # [ 107.848699] niks3-server[984]: 2026/09/25 15:14:32 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4922worker2 # [ 107.856854] nix-grpc-daemon[770]: ts=2026-09-25T15:14:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=0 paths=1 nar_bytes_in=5684923worker2 # [ 107.866413] nix-grpc-daemon[770]: ts=2026-09-25T15:14:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505204924worker2 # [ 107.871268] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4925worker2 # [ 107.876197] nix-daemon[859]: reaped child process 1260, status = succeeded4926client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4927client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4928worker2 # [ 107.886445] nix-daemon[859]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4929client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4930worker2 # [ 108.968918] nix-daemon[859]: reaped child process 1269, status = succeeded4931worker2 # [ 109.031178] nix-daemon[859]: accepted connection from pid 1285, user nix-grpc-daemon (trusted)4932worker2 # [ 109.051483] nix-daemon[859]: reaped child process 1288, status = succeeded4933lb # [ 109.053226] niks3-server[984]: 2026/09/25 15:14:33 INFO Received push request method=POST path=/api/pushes4934worker2 # [ 109.061895] nix-grpc-daemon[873]: time=2026-09-25T15:14:33.654Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4935lb # [ 109.072713] niks3-server[984]: 2026/09/25 15:14:33 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign4936worker2 # [ 109.080216] nix-grpc-daemon[873]: time=2026-09-25T15:14:33.673Z level=INFO msg="Uploading 1 narinfos"4937lb # [ 109.081195] niks3-server[984]: 2026/09/25 15:14:33 INFO Signed narinfos id=34 count=14938lb # [ 109.086844] niks3-server[984]: 2026/09/25 15:14:33 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4939lb # [ 109.096112] niks3-server[984]: 2026/09/25 15:14:33 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4940lb # [ 109.102008] niks3-server[984]: 2026/09/25 15:14:33 INFO Received complete push request method=POST path=/api/pushes/34/complete4941worker2 # [ 109.107065] nix-grpc-daemon[873]: time=2026-09-25T15:14:33.700Z level=INFO msg="Upload complete. (212ms)"4942lb # [ 109.107498] niks3-server[984]: 2026/09/25 15:14:33 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4943client # copying 1 paths...4944worker2 # [ 109.116943] nix-grpc-daemon[770]: ts=2026-09-25T15:14:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=18 outputs=14945client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4946worker2 # [ 109.129426] nix-grpc-daemon[770]: ts=2026-09-25T15:14:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:505204947worker2 # [ 109.132730] nix-daemon[859]: reaped child process 1267, status = succeeded4948worker2 # [ 109.140937] nix-grpc-daemon[770]: ts=2026-09-25T15:14:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:505204949worker2 # [ 109.144961] nix-grpc-daemon[770]: ts=2026-09-25T15:14:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50520 duration_s=0 paths=1 nar_bytes_out=1204950client # [ 109.206004] nix-daemon[1111]: reaped child process 1240, status = succeeded4951client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.89 seconds)4952(finished: subtest: an input only one worker has still reaches the builder, in 6.62 seconds)4953subtest: upload whose reference the worker lacks is completed from the cache4954client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4955client # this derivation will be built:4956client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4957client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4958client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.32 seconds)4959client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4960client # this derivation will be built:4961client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4962client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4963client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.29 seconds)4964client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4965client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)4966worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4967worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.16 seconds)4968worker2 # [ 110.139323] systemd[1]: Stopping Nix worker-protocol over gRPC...4969worker2 # [ 110.589018] nix-grpc-daemon[770]: ts=2026-09-25T15:14:35Z level=info event=signal_exit4970lb # [ 110.817566] niks3-server[984]: 2026/09/25 15:14:35 INFO lead: released remote=192.168.1.4:578304971worker2 # [ 111.821546] nix-grpc-daemon[770]: ts=2026-09-25T15:14:36Z level=info event=scheduler_restarting4972worker1 # [ 111.819750] nix-grpc-daemon[485]: ts=2026-09-25T15:14:36Z level=info event=scheduler_restarting addr=lb:500514973lb # [ 111.818128] envoy[893]: [2026-09-25 15:14:36.403][893][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}4974worker2 # [ 111.821887] nix-grpc-daemon[770]: ts=2026-09-25T15:14:36Z level=info event=scheduler_restarting addr=lb:500514975worker2 # [ 111.828475] nix-grpc-daemon[770]: ts=2026-09-25T15:14:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:609064976worker2 # [ 111.831278] nix-grpc-daemon[770]: ts=2026-09-25T15:14:36Z level=info event=worker_gone addr=192.168.1.3:500514977worker2 # [ 111.838615] nix-grpc-daemon[770]: ts=2026-09-25T15:14:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:609064978worker2 # [ 111.841660] nix-grpc-daemon[770]: ts=2026-09-25T15:14:36Z level=info event=worker_gone addr=192.168.1.4:500514979worker2 # [ 111.862146] nix-daemon[859]: reaped child process 861, status = succeeded4980worker1 # [ 112.020435] nix-grpc-daemon[485]: ts=2026-09-25T15:14:36Z level=info event=scheduler_connected addr=lb:500514981worker1 # [ 112.023824] nix-grpc-daemon[485]: ts=2026-09-25T15:14:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4982worker2 # [ 112.040618] nix-grpc-daemon[770]: ts=2026-09-25T15:14:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4983worker1 # [ 112.224440] nix-grpc-daemon[485]: ts=2026-09-25T15:14:36Z level=info event=scheduler_connected addr=lb:500514984worker1 # [ 112.227498] nix-grpc-daemon[485]: ts=2026-09-25T15:14:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4985worker1 # [ 112.428087] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_connected addr=lb:500514986worker1 # [ 112.431185] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4987worker1 # [ 112.631815] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_connected addr=lb:500514988worker1 # [ 112.634988] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4989worker1 # [ 112.836501] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_connected addr=lb:500514990worker1 # [ 112.839528] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4991worker1 # [ 113.040280] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_connected addr=lb:500514992worker1 # [ 113.043388] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4993worker1 # [ 113.244092] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_connected addr=lb:500514994worker1 # [ 113.247345] nix-grpc-daemon[485]: ts=2026-09-25T15:14:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4995worker1 # [ 113.447868] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_connected addr=lb:500514996worker1 # [ 113.451179] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4997worker1 # [ 113.651655] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_connected addr=lb:500514998worker1 # [ 113.654811] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4999worker1 # [ 113.856149] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_connected addr=lb:500515000worker1 # [ 113.859056] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5001worker2 # [ 113.960834] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5002worker2 # [ 113.964305] systemd[1]: Stopped Nix worker-protocol over gRPC.5003worker2 # [ 113.965290] systemd[1]: nix-grpc-daemon.service: Consumed 2.954s CPU time over 1min 36.589s wall clock time, 71.8M memory peak, 328.1K incoming IP traffic, 378.8K outgoing IP traffic.5004lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux5005lb: must succeed: curl -sf localhost:9901/clusters5006lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5007lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.04 seconds)5008client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25009worker1 # [ 114.059571] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_connected addr=lb:500515010worker1 # [ 114.062861] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5011worker1 # [ 114.200538] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455365012worker1 # [ 114.206578] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455365013client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5014client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.20 seconds)5015worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5016worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.02 seconds)5017worker1 # [ 114.263410] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_connected addr=lb:500515018worker1 # [ 114.266477] nix-grpc-daemon[485]: ts=2026-09-25T15:14:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5019worker2 # [ 114.300738] systemd[1]: Starting scheduler-token.service...5020worker2 # [ 114.356501] systemd[1]: scheduler-token.service: Deactivated successfully.5021worker2 # [ 114.359862] systemd[1]: Finished scheduler-token.service.5022worker2 # [ 114.363593] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 58ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5023worker2 # [ 114.367997] systemd[1]: Starting Nix worker-protocol over gRPC...5024worker1 # [ 114.467113] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515025worker1 # [ 114.470205] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5026worker2 # [ 114.477952] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5027worker2 # [ 114.480509] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5028worker2 # [ 114.484657] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15029worker2 # [ 114.491963] nix-grpc-daemon[1312]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5030worker2 # [ 114.493938] nix-grpc-daemon[1312]: E0000 00:00:1790349279.082922 1312 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5031lb # [ 114.502677] niks3-server[984]: 2026/09/25 15:14:39 INFO lead: acquired remote=192.168.1.4:341905032worker2 # [ 114.528153] nix-grpc-daemon[1312]: E0000 00:00:1790349279.085244 1312 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5033worker2 # [ 114.530649] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5034worker2 # [ 114.544790] nix-grpc-daemon[1312]: E0000 00:00:1790349279.097619 1355 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5035worker2 # [ 114.550535] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_take_over5036worker2 # [ 114.554381] nix-grpc-daemon[1312]: E0000 00:00:1790349279.114902 1355 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5037worker2 # [ 114.557128] nix-grpc-daemon[1312]: E0000 00:00:1790349279.115165 1355 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5038worker2 # [ 114.559689] systemd[1]: Started Nix worker-protocol over gRPC.5039worker2 # [ 114.560662] nix-grpc-daemon[1312]: E0000 00:00:1790349279.136628 1355 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5040worker2 # [ 114.563099] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515041worker2 # [ 114.564617] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5042worker1 # [ 114.574275] systemd[1]: Stopping Nix worker-protocol over gRPC...5043worker1 # [ 114.670918] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515044worker1 # [ 114.674301] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5045worker2 # [ 114.755660] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515046worker2 # [ 114.758825] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5047worker1 # [ 114.875664] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515048worker1 # [ 114.878821] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5049worker1 # [ 114.890965] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=signal_exit5050worker2 # [ 114.959758] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515051worker2 # [ 114.962894] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5052worker1 # [ 115.079419] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515053worker1 # [ 115.082585] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5054worker2 # [ 115.163878] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515055worker2 # [ 115.167015] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5056worker1 # [ 115.283203] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515057worker1 # [ 115.286170] nix-grpc-daemon[485]: ts=2026-09-25T15:14:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5058worker2 # [ 115.367997] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=scheduler_connected addr=lb:500515059worker2 # [ 115.378746] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=oidc_jwks provider=mock keys=15060worker2 # [ 115.381690] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:442985061worker2 # [ 115.384778] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05062worker1 # [ 115.486777] nix-grpc-daemon[485]: ts=2026-09-25T15:14:40Z level=info event=scheduler_connected addr=lb:500515063worker2 # [ 115.492199] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:442985064worker2 # [ 115.495002] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05065worker2 # [ 116.820848] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:442985066worker1 # [ 116.820456] nix-grpc-daemon[485]: ts=2026-09-25T15:14:41Z level=info event=scheduler_restarting5067worker2 # [ 116.823636] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:41Z level=info event=worker_gone addr=192.168.1.3:500515068worker1 # [ 116.835708] nix-grpc-daemon[485]: ts=2026-09-25T15:14:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5069worker1 # [ 116.854879] nix-daemon[837]: reaped child process 839, status = succeeded5070lb # [ 117.096631] envoy[893]: conn peer=[2001:db8:1::1]:60320 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3738 tx=2101 ms=18305071lb # [ 117.099388] envoy[893]: conn peer=[2001:db8:1::1]:60330 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=329 ms=315072lb # [ 117.102225] envoy[893]: conn peer=[2001:db8:1::1]:60334 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=2103 ms=15655073lb # [ 117.104878] envoy[893]: conn peer=[2001:db8:1::1]:60336 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=175074lb # [ 117.107391] envoy[893]: conn peer=192.168.1.4:34638 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8323 tx=1352 ms=797325075lb # [ 117.110068] envoy[893]: conn peer=[2001:db8:1::1]:45136 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=205076lb # [ 117.112552] envoy[893]: conn peer=[2001:db8:1::3]:50294 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4723 tx=1661 ms=824485077worker1 # [ 117.935372] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5078worker1 # [ 117.936391] systemd[1]: Stopped Nix worker-protocol over gRPC.5079worker1 # [ 117.937775] systemd[1]: nix-grpc-daemon.service: Consumed 2.758s CPU time over 1min 48.357s wall clock time, 66.4M memory peak, 181.8K incoming IP traffic, 305.3K outgoing IP traffic.5080lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5081lb: must succeed: curl -sf localhost:9901/clusters5082lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5083lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.05 seconds)5084client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25085worker2 # [ 118.197496] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443125086worker2 # [ 118.206194] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5087worker2 # [ 118.222596] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 1/5)5088worker2 # [ 118.290460] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)5089worker2 # [ 118.300753] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 253 ms (attempt 3/5)5090worker2 # [ 118.555630] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 253 ms (attempt 4/5)5091worker2 # [ 118.810744] nix-grpc-daemon[1312]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5092worker2 # [ 118.815703] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5093client # copying 2 paths...5094client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5095worker2 # [ 118.832659] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443125096client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5097worker2 # [ 118.843953] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5098worker2 # [ 118.864587] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:43Z level=info event=niks3_push_started pid=13725099worker2 # [ 119.009691] nix-grpc-daemon[1372]: time=2026-09-25T15:14:43.601Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5100worker2 # [ 119.122433] nix-daemon[859]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)5101worker2 # [ 119.145593] nix-daemon[859]: reaped child process 1387, status = succeeded5102lb # [ 119.147461] niks3-server[984]: 2026/09/25 15:14:43 INFO Received push request method=POST path=/api/pushes5103worker2 # [ 119.156129] nix-grpc-daemon[1372]: time=2026-09-25T15:14:43.748Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5104worker2 # [ 119.157768] nix-grpc-daemon[1372]: time=2026-09-25T15:14:43.749Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5105worker2 # [ 119.159603] nix-grpc-daemon[1372]: time=2026-09-25T15:14:43.749Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5106lb # [ 119.191893] niks3-server[984]: 2026/09/25 15:14:43 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5107worker2 # [ 119.200148] nix-grpc-daemon[1372]: time=2026-09-25T15:14:43.793Z level=INFO msg="Uploading 2 narinfos"5108lb # [ 119.201417] niks3-server[984]: 2026/09/25 15:14:43 INFO Signed narinfos id=35 count=25109lb # [ 119.210914] niks3-server[984]: 2026/09/25 15:14:43 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5110lb # [ 119.224813] niks3-server[984]: 2026/09/25 15:14:43 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5111lb # [ 119.248084] niks3-server[984]: 2026/09/25 15:14:43 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5112lb # [ 119.259183] niks3-server[984]: 2026/09/25 15:14:43 INFO Received complete push request method=POST path=/api/pushes/35/complete5113lb # [ 119.264149] niks3-server[984]: 2026/09/25 15:14:43 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5114lb # [ 119.266435] niks3-server[984]: 2026/09/25 15:14:43 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5115lb # [ 119.270812] niks3-server[984]: 2026/09/25 15:14:43 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5116worker2 # [ 119.276955] nix-grpc-daemon[1372]: time=2026-09-25T15:14:43.870Z level=INFO msg="Upload complete. (268ms)"5117worker2 # [ 119.284687] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=2 nar_bytes_in=3125118worker2 # [ 119.298610] nix-daemon[859]: reaped child process 1370, status = succeeded5119client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.29 seconds)5120worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5121worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.02 seconds)5122worker1 # [ 119.371013] systemd[1]: Starting Nix worker-protocol over gRPC...5123worker1 # [ 119.495196] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5124worker1 # [ 119.497939] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5125worker1 # [ 119.502043] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15126worker1 # [ 119.508098] nix-grpc-daemon[1229]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5127worker1 # [ 119.509717] nix-grpc-daemon[1229]: E0000 00:00:1790349284.097254 1229 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5128worker1 # [ 119.514228] nix-grpc-daemon[1229]: E0000 00:00:1790349284.104271 1229 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5129worker1 # [ 119.561640] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:44Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5130lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5131lb: must succeed: curl -sf localhost:9901/clusters5132worker1 # [ 119.564694] nix-grpc-daemon[1229]: E0000 00:00:1790349284.114299 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5133worker1 # [ 119.570931] nix-grpc-daemon[1229]: E0000 00:00:1790349284.132592 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5134worker1 # [ 119.575981] nix-grpc-daemon[1229]: E0000 00:00:1790349284.132901 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5135worker2 # [ 119.578390] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:442985136worker1 # [ 119.578732] nix-grpc-daemon[1229]: E0000 00:00:1790349284.150849 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5137worker1 # [ 119.581298] systemd[1]: Started Nix worker-protocol over gRPC.5138worker1 # [ 119.582239] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:44Z level=info event=scheduler_connected addr=lb:500515139worker2 # [ 119.582799] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05140lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5141worker2 # [ 120.613493] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:45Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:443285142lb: must succeed: curl -sf localhost:9901/clusters5143lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5144lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.11 seconds)5145(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.46 seconds)5146subtest: a niks3 restart does not cost the leader its role5147lb: must succeed: curl -sf localhost:9901/clusters5148lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5149lb: must succeed: curl -sf localhost:9901/clusters5150lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5151lb # [ 120.774248] systemd[1]: Stopping niks3 server...5152lb # [ 120.774821] niks3-server[984]: 2026/09/25 15:14:45 INFO Shutdown signal received, draining in-flight requests timeout=10s5153lb # [ 120.781005] niks3-server[984]: 2026/09/25 15:14:45 INFO lead: released remote=192.168.1.4:341905154worker2 # [ 120.781763] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:45Z level=info event=scheduler_lock_lost status=2005155worker1 # [ 120.779913] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:45Z level=info event=scheduler_lock_lost status=2005156lb # [ 120.808219] systemd[1]: niks3.service: Deactivated successfully.5157lb # [ 120.811607] systemd[1]: Stopped niks3 server.5158lb # [ 120.812612] systemd[1]: niks3.service: Consumed 484ms CPU time over 1min 36.237s wall clock time, 27.9M memory peak, 99.1K incoming IP traffic, 142.5K outgoing IP traffic.5159lb # [ 120.825154] systemd[1]: Starting niks3 server...5160lb # [ 120.899606] niks3-server[1182]: 2026/09/25 15:14:45 goose: no migrations to run. current version: 202609231200005161lb # [ 120.906288] niks3-server[1182]: 2026/09/25 15:14:45 OK 1_commit_pending_closure.sql (8.82ms)5162lb # [ 120.910512] niks3-server[1182]: 2026/09/25 15:14:45 OK 2_object_stats_trigger.sql (2.57ms)5163lb # [ 120.914400] niks3-server[1182]: 2026/09/25 15:14:45 OK 3_commit_push.sql (3.37ms)5164lb # [ 120.915666] niks3-server[1182]: 2026/09/25 15:14:45 goose: up to current file version: 35165lb # [ 120.918652] niks3-server[1182]: 2026/09/25 15:14:45 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5166lb # [ 120.933768] niks3-server[1182]: 2026/09/25 15:14:45 INFO Read proxy enabled — serving cache objects from S35167lb # [ 120.935449] niks3-server[1182]: 2026/09/25 15:14:45 INFO Using socket-activated listener address=0.0.0.0:57515168lb # [ 120.941874] niks3-server[1182]: 2026/09/25 15:14:45 INFO systemd watchdog enabled interval=15s5169lb # [ 120.944540] niks3-server[1182]: 2026/09/25 15:14:45 INFO Starting HTTP server address=0.0.0.0:57515170lb # [ 120.946955] systemd[1]: Started niks3 server.5171lb: waiting for unit niks3.service5172lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5173lb: must succeed: curl -sf localhost:9901/clusters5174lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5175worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5176worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5177lb: must succeed: curl -sf localhost:9901/clusters5178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5179lb: must succeed: curl -sf localhost:9901/clusters5180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5181worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5182worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5183client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25184worker2 # [ 121.371349] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443125185client # don't know how to build these paths:5186client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5187worker1 # [ 121.383614] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415165188worker1 # [ 121.396266] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5189worker1 # [ 121.413688] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 1/5)5190worker1 # [ 121.428644] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 2/5)5191worker1 # [ 121.434904] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 330 ms (attempt 3/5)5192worker1 # [ 121.766535] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 551 ms (attempt 4/5)5193lb # [ 121.785172] niks3-server[1182]: 2026/09/25 15:14:46 INFO lead: acquired remote=192.168.1.4:584585194worker1 # [ 122.321148] nix-grpc-daemon[1229]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5195worker1 # [ 122.325404] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5196worker2 # [ 122.364007] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:46Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05197worker2 # [ 122.371458] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:46Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15198worker2 # [ 122.375736] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443125199worker2 # [ 122.384238] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443125200client # copying 1 paths to 'lb:50051'...5201client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5202worker2 # [ 122.395395] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443125203worker2 # [ 122.406440] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5204worker2 # [ 122.559264] nix-daemon[859]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)5205worker2 # [ 122.587013] nix-daemon[859]: reaped child process 1409, status = succeeded5206lb # [ 122.593853] niks3-server[1182]: 2026/09/25 15:14:47 INFO Received push request method=POST path=/api/pushes5207worker2 # [ 122.606928] nix-grpc-daemon[1372]: time=2026-09-25T15:14:47.198Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5208worker2 # [ 122.609353] nix-grpc-daemon[1372]: time=2026-09-25T15:14:47.198Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5209lb # [ 122.627522] niks3-server[1182]: 2026/09/25 15:14:47 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5210lb # [ 122.656387] niks3-server[1182]: 2026/09/25 15:14:47 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5211lb # [ 122.664577] niks3-server[1182]: 2026/09/25 15:14:47 INFO Signed narinfos id=36 count=15212worker2 # [ 122.669755] nix-grpc-daemon[1372]: time=2026-09-25T15:14:47.261Z level=INFO msg="Uploading 1 narinfos"5213lb # [ 122.678286] niks3-server[1182]: 2026/09/25 15:14:47 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5214lb # [ 122.689307] niks3-server[1182]: 2026/09/25 15:14:47 INFO Received complete push request method=POST path=/api/pushes/36/complete5215lb # [ 122.697331] niks3-server[1182]: 2026/09/25 15:14:47 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5216worker2 # [ 122.706272] nix-grpc-daemon[1372]: time=2026-09-25T15:14:47.299Z level=INFO msg="Upload complete. (286ms)"5217worker2 # [ 122.712411] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=5445218worker2 # [ 122.728449] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443125219worker2 # [ 122.735186] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5220worker2 # [ 122.741604] nix-daemon[859]: reaped child process 1404, status = succeeded5221client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5222client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5223worker2 # [ 122.754481] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5224client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5225client # farm-a> LOG-farm-a-after-niks3-restart5226worker2 # [ 124.006396] nix-daemon[859]: reaped child process 1413, status = succeeded5227worker2 # [ 124.121663] nix-daemon[859]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5228lb # [ 124.146771] niks3-server[1182]: 2026/09/25 15:14:48 INFO Received push request method=POST path=/api/pushes5229worker2 # [ 124.150245] nix-daemon[859]: reaped child process 1432, status = succeeded5230worker2 # [ 124.159980] nix-grpc-daemon[1372]: time=2026-09-25T15:14:48.750Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5231worker2 # [ 124.161987] nix-grpc-daemon[1372]: time=2026-09-25T15:14:48.750Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5232lb # [ 124.182056] niks3-server[1182]: 2026/09/25 15:14:48 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5233lb # [ 124.190677] niks3-server[1182]: 2026/09/25 15:14:48 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5234worker2 # [ 124.200065] nix-grpc-daemon[1372]: time=2026-09-25T15:14:48.793Z level=INFO msg="Uploading 1 narinfos"5235lb # [ 124.198310] niks3-server[1182]: 2026/09/25 15:14:48 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5236lb # [ 124.204333] niks3-server[1182]: 2026/09/25 15:14:48 INFO Signed narinfos id=37 count=15237lb # [ 124.210698] niks3-server[1182]: 2026/09/25 15:14:48 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5238lb # [ 124.224228] niks3-server[1182]: 2026/09/25 15:14:48 INFO Received complete push request method=POST path=/api/pushes/37/complete5239lb # [ 124.231812] niks3-server[1182]: 2026/09/25 15:14:48 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5240worker2 # [ 124.240257] nix-grpc-daemon[1372]: time=2026-09-25T15:14:48.833Z level=INFO msg="Upload complete. (270ms)"5241worker2 # [ 124.258304] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15242worker2 # [ 124.268573] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:48Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05243worker2 # [ 124.271504] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:48Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15244worker2 # [ 124.282430] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443125245worker2 # [ 124.287265] nix-daemon[859]: reaped child process 1411, status = succeeded5246worker2 # [ 124.290166] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443125247worker2 # [ 124.301109] nix-grpc-daemon[1372]: time=2026-09-25T15:14:48.894Z level=INFO msg="All 1 paths already cached"5248client # copying 1 paths to 'lb:50051'...5249client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5250worker2 # [ 124.309434] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443125251worker2 # [ 124.317596] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5252worker2 # [ 124.464487] nix-daemon[859]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)5253worker2 # [ 124.494480] nix-daemon[859]: reaped child process 1441, status = succeeded5254lb # [ 124.498180] niks3-server[1182]: 2026/09/25 15:14:49 INFO Received push request method=POST path=/api/pushes5255worker2 # [ 124.506823] nix-grpc-daemon[1372]: time=2026-09-25T15:14:49.099Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5256worker2 # [ 124.509113] nix-grpc-daemon[1372]: time=2026-09-25T15:14:49.099Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5257lb # [ 124.522355] niks3-server[1182]: 2026/09/25 15:14:49 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5258lb # [ 124.530312] niks3-server[1182]: 2026/09/25 15:14:49 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5259worker2 # [ 124.536990] nix-grpc-daemon[1372]: time=2026-09-25T15:14:49.129Z level=INFO msg="Uploading 1 narinfos"5260lb # [ 124.538879] niks3-server[1182]: 2026/09/25 15:14:49 INFO Signed narinfos id=38 count=15261lb # [ 124.543032] niks3-server[1182]: 2026/09/25 15:14:49 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5262lb # [ 124.555067] niks3-server[1182]: 2026/09/25 15:14:49 INFO Received complete push request method=POST path=/api/pushes/38/complete5263worker2 # [ 124.565096] nix-grpc-daemon[1372]: time=2026-09-25T15:14:49.158Z level=INFO msg="Upload complete. (235ms)"5264lb # [ 124.564187] niks3-server[1182]: 2026/09/25 15:14:49 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5265worker2 # [ 124.573384] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=6565266worker2 # [ 124.588734] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443125267worker2 # [ 124.594282] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5268worker2 # [ 124.599714] nix-daemon[859]: reaped child process 1436, status = succeeded5269client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5270client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5271worker2 # [ 124.612487] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5272client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5273client # farm-b> LOG-farm-b-after-niks3-restart5274worker2 # [ 125.669162] nix-daemon[859]: reaped child process 1445, status = succeeded5275worker2 # [ 125.787632] nix-daemon[859]: accepted connection from pid 1461, user nix-grpc-daemon (trusted)5276lb # [ 125.813402] niks3-server[1182]: 2026/09/25 15:14:50 INFO Received push request method=POST path=/api/pushes5277worker2 # [ 125.822693] nix-grpc-daemon[1372]: time=2026-09-25T15:14:50.415Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5278worker2 # [ 125.826854] nix-grpc-daemon[1372]: time=2026-09-25T15:14:50.418Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5279worker2 # [ 125.829283] nix-daemon[859]: reaped child process 1464, status = succeeded5280lb # [ 125.841517] niks3-server[1182]: 2026/09/25 15:14:50 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5281lb # [ 125.853157] niks3-server[1182]: 2026/09/25 15:14:50 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5282lb # [ 125.855823] niks3-server[1182]: 2026/09/25 15:14:50 INFO Signed narinfos id=39 count=15283worker2 # [ 125.860816] nix-grpc-daemon[1372]: time=2026-09-25T15:14:50.453Z level=INFO msg="Uploading 1 narinfos"5284lb # [ 125.866598] niks3-server[1182]: 2026/09/25 15:14:50 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5285lb # [ 125.873439] niks3-server[1182]: 2026/09/25 15:14:50 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5286lb # [ 125.883156] niks3-server[1182]: 2026/09/25 15:14:50 INFO Received complete push request method=POST path=/api/pushes/39/complete5287worker2 # [ 125.890429] nix-grpc-daemon[1372]: time=2026-09-25T15:14:50.483Z level=INFO msg="Upload complete. (264ms)"5288lb # [ 125.892303] niks3-server[1182]: 2026/09/25 15:14:50 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5289worker2 # [ 125.902696] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15290worker2 # [ 125.922604] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:50Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05291worker2 # [ 125.926158] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:50Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15292worker2 # [ 125.931280] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443125293worker2 # [ 125.935354] nix-daemon[859]: reaped child process 1443, status = succeeded5294worker2 # [ 125.942156] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443125295worker2 # [ 125.953813] nix-grpc-daemon[1372]: time=2026-09-25T15:14:50.547Z level=INFO msg="All 2 paths already cached"5296client # copying 1 paths to 'lb:50051'...5297client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5298worker2 # [ 125.962463] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443125299worker2 # [ 125.973317] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5300worker2 # [ 126.119929] nix-daemon[859]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)5301lb # [ 126.148468] niks3-server[1182]: 2026/09/25 15:14:50 INFO Received push request method=POST path=/api/pushes5302worker2 # [ 126.154193] nix-daemon[859]: reaped child process 1473, status = succeeded5303worker2 # [ 126.157450] nix-grpc-daemon[1372]: time=2026-09-25T15:14:50.750Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5304worker2 # [ 126.159372] nix-grpc-daemon[1372]: time=2026-09-25T15:14:50.750Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5305lb # [ 126.171980] niks3-server[1182]: 2026/09/25 15:14:50 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5306worker2 # [ 126.178611] nix-grpc-daemon[1372]: time=2026-09-25T15:14:50.771Z level=INFO msg="Uploading 1 narinfos"5307lb # [ 126.180238] niks3-server[1182]: 2026/09/25 15:14:50 INFO Signed narinfos id=40 count=15308lb # [ 126.186175] niks3-server[1182]: 2026/09/25 15:14:50 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5309lb # [ 126.192976] niks3-server[1182]: 2026/09/25 15:14:50 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5310lb # [ 126.203748] niks3-server[1182]: 2026/09/25 15:14:50 INFO Received complete push request method=POST path=/api/pushes/40/complete5311worker2 # [ 126.213141] nix-grpc-daemon[1372]: time=2026-09-25T15:14:50.806Z level=INFO msg="Upload complete. (227ms)"5312lb # [ 126.211580] niks3-server[1182]: 2026/09/25 15:14:50 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5313worker2 # [ 126.220208] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=7845314worker2 # [ 126.228848] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443125315worker2 # [ 126.240217] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5316worker2 # [ 126.245569] nix-daemon[859]: reaped child process 1468, status = succeeded5317client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5318client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5319worker2 # [ 126.257136] nix-daemon[859]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5320client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5321client # farm-top> LOG-farm-top-after-niks3-restart5322lb # [ 127.100088] envoy[893]: conn peer=[2001:db8:1::1]:45148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1008 tx=282 ms=11015323worker2 # [ 127.120602] nix-daemon[859]: reaped child process 1477, status = succeeded5324worker2 # [ 127.240141] nix-daemon[859]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)5325worker2 # [ 127.262657] nix-daemon[859]: reaped child process 1496, status = succeeded5326lb # [ 127.269028] niks3-server[1182]: 2026/09/25 15:14:51 INFO Received push request method=POST path=/api/pushes5327worker2 # [ 127.278420] nix-grpc-daemon[1372]: time=2026-09-25T15:14:51.870Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5328worker2 # [ 127.280395] nix-grpc-daemon[1372]: time=2026-09-25T15:14:51.870Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5329lb # [ 127.296309] niks3-server[1182]: 2026/09/25 15:14:51 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5330worker2 # [ 127.306119] nix-grpc-daemon[1372]: time=2026-09-25T15:14:51.899Z level=INFO msg="Uploading 1 narinfos"5331lb # [ 127.310749] niks3-server[1182]: 2026/09/25 15:14:51 INFO Signed narinfos id=41 count=15332lb # [ 127.325176] niks3-server[1182]: 2026/09/25 15:14:51 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5333lb # [ 127.335859] niks3-server[1182]: 2026/09/25 15:14:51 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5334lb # [ 127.344938] niks3-server[1182]: 2026/09/25 15:14:51 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5335worker2 # [ 127.351351] nix-grpc-daemon[1372]: time=2026-09-25T15:14:51.944Z level=INFO msg="Upload complete. (279ms)"5336lb # [ 127.351076] niks3-server[1182]: 2026/09/25 15:14:51 INFO Received complete push request method=POST path=/api/pushes/41/complete5337lb # [ 127.356399] niks3-server[1182]: 2026/09/25 15:14:51 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5338worker2 # [ 127.360846] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15339worker2 # [ 127.375013] nix-daemon[859]: reaped child process 1475, status = succeeded5340client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.22 seconds)5341client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5342client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.16 seconds)5343(finished: subtest: a niks3 restart does not cost the leader its role, in 6.88 seconds)5344subtest: scheduler down: the other node takes the lock and keeps it5345lb: must succeed: curl -sf localhost:9901/clusters5346lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5347lb: must succeed: curl -sf localhost:9901/clusters5348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5349lb: must succeed: curl -sf localhost:9901/clusters5350lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5351lb: must succeed: curl -sf localhost:9901/clusters5352lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5353worker2 # [ 127.738220] systemd[1]: Stopping Nix worker-protocol over gRPC...5354worker2 # [ 128.510170] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:53Z level=info event=signal_exit5355lb # [ 128.792925] niks3-server[1182]: 2026/09/25 15:14:53 INFO lead: released remote=192.168.1.4:584585356worker1 # [ 129.792778] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:54Z level=info event=scheduler_restarting addr=lb:500515357worker2 # [ 129.795332] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:54Z level=info event=scheduler_restarting5358lb # [ 129.790858] envoy[893]: [2026-09-25 15:14:54.376][893][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5359worker2 # [ 129.795665] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:54Z level=info event=scheduler_restarting addr=lb:500515360worker2 # [ 129.801733] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:54Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:442985361worker2 # [ 129.807996] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:54Z level=info event=worker_gone addr=192.168.1.4:500515362worker2 # [ 129.811946] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:54Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:442985363worker2 # [ 129.815333] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:54Z level=info event=worker_gone addr=192.168.1.3:500515364worker2 # [ 129.836133] nix-daemon[859]: reaped child process 1364, status = succeeded5365worker1 # [ 129.993227] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:54Z level=info event=scheduler_connected addr=lb:500515366worker1 # [ 129.997805] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5367worker2 # [ 130.007615] nix-grpc-daemon[1312]: ts=2026-09-25T15:14:54Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5368worker1 # [ 130.198009] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:54Z level=info event=scheduler_connected addr=lb:500515369worker1 # [ 130.201830] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5370worker1 # [ 130.402513] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:54Z level=info event=scheduler_connected addr=lb:500515371worker1 # [ 130.406089] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5372worker1 # [ 130.606810] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:55Z level=info event=scheduler_connected addr=lb:500515373worker1 # [ 130.610563] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5374worker1 # [ 130.813000] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:55Z level=info event=scheduler_connected addr=lb:500515375worker1 # [ 130.815892] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker1 # [ 131.014012] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:55Z level=info event=scheduler_connected addr=lb:500515377worker1 # [ 131.017779] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5378worker2 # [ 131.104650] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5379worker2 # [ 131.106957] systemd[1]: Stopped Nix worker-protocol over gRPC.5380worker2 # [ 131.113016] systemd[1]: nix-grpc-daemon.service: Consumed 1.388s CPU time over 16.736s wall clock time, 39.4M memory peak, 196.7K incoming IP traffic, 113.2K outgoing IP traffic.5381worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5382worker1 # [ 131.218576] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:55Z level=info event=scheduler_connected addr=lb:500515383worker1 # [ 131.222362] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker1 # [ 131.422998] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_connected addr=lb:500515385worker1 # [ 131.429721] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5386worker1 # [ 131.630728] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_connected addr=lb:500515387worker1 # [ 131.637390] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5388lb # [ 131.786035] niks3-server[1182]: 2026/09/25 15:14:56 INFO lead: acquired remote=192.168.1.3:416685389worker1 # [ 131.786683] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_take_over5390worker1 # [ 131.839472] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_connected addr=lb:500515391worker1 # [ 131.843821] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5392worker1 # [ 132.041531] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_connected addr=lb:500515393worker1 # [ 132.045220] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5394worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.11 seconds)5395worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5396worker1 # [ 132.245860] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_connected addr=lb:500515397worker1 # [ 132.254021] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5398worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5399worker1 # [ 132.452190] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:57Z level=info event=scheduler_connected addr=lb:500515400worker1 # [ 132.466767] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:387365401worker1 # [ 132.469192] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05402worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5403worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5404client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25405client # don't know how to build these paths:5406client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5407worker1 # [ 133.549707] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:415165408worker1 # [ 133.555311] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415165409worker1 # [ 133.578605] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05410worker1 # [ 133.584209] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15411worker1 # [ 133.591832] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415165412worker1 # [ 133.603110] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415165413client # copying 1 paths to 'lb:50051'...5414client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5415worker1 # [ 133.618673] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415165416worker1 # [ 133.631726] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5417worker1 # [ 133.656152] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=info event=niks3_push_started pid=13115418worker1 # [ 133.801480] nix-grpc-daemon[1311]: time=2026-09-25T15:14:58.390Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5419worker1 # [ 133.930742] nix-daemon[837]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5420worker1 # [ 133.955358] nix-daemon[837]: reaped child process 1327, status = succeeded5421lb # [ 133.964488] niks3-server[1182]: 2026/09/25 15:14:58 INFO Received push request method=POST path=/api/pushes5422worker1 # [ 133.976153] nix-grpc-daemon[1311]: time=2026-09-25T15:14:58.565Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5423worker1 # [ 133.978068] nix-grpc-daemon[1311]: time=2026-09-25T15:14:58.565Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5424lb # [ 134.004484] niks3-server[1182]: 2026/09/25 15:14:58 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5425worker1 # [ 134.015895] nix-grpc-daemon[1311]: time=2026-09-25T15:14:58.605Z level=INFO msg="Uploading 1 narinfos"5426lb # [ 134.010525] niks3-server[1182]: 2026/09/25 15:14:58 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5427lb # [ 134.020948] niks3-server[1182]: 2026/09/25 15:14:58 INFO Signed narinfos id=42 count=15428lb # [ 134.027007] niks3-server[1182]: 2026/09/25 15:14:58 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5429lb # [ 134.039990] niks3-server[1182]: 2026/09/25 15:14:58 INFO Received complete push request method=POST path=/api/pushes/42/complete5430lb # [ 134.051739] niks3-server[1182]: 2026/09/25 15:14:58 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5431worker1 # [ 134.064512] nix-grpc-daemon[1311]: time=2026-09-25T15:14:58.654Z level=INFO msg="Upload complete. (258ms)"5432worker1 # [ 134.072244] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41516 duration_s=0 paths=1 nar_bytes_in=5205433worker1 # [ 134.084764] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415165434worker1 # [ 134.093353] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5435worker1 # [ 134.106917] nix-daemon[837]: reaped child process 1309, status = succeeded5436client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5437client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5438worker1 # [ 134.116703] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5439client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5440client # farm-a> LOG-farm-a-on-standby5441worker1 # [ 135.012668] nix-daemon[837]: reaped child process 1331, status = succeeded5442worker1 # [ 135.031703] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:59Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:387385443worker1 # [ 135.094532] nix-daemon[837]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5444lb # [ 135.121238] niks3-server[1182]: 2026/09/25 15:14:59 INFO Received push request method=POST path=/api/pushes5445worker1 # [ 135.126955] nix-daemon[837]: reaped child process 1350, status = succeeded5446worker1 # [ 135.138440] nix-grpc-daemon[1311]: time=2026-09-25T15:14:59.725Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5447worker1 # [ 135.141955] nix-grpc-daemon[1311]: time=2026-09-25T15:14:59.725Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5448lb # [ 135.160146] niks3-server[1182]: 2026/09/25 15:14:59 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5449lb # [ 135.170604] niks3-server[1182]: 2026/09/25 15:14:59 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5450lb # [ 135.179581] niks3-server[1182]: 2026/09/25 15:14:59 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5451worker1 # [ 135.183665] nix-grpc-daemon[1311]: time=2026-09-25T15:14:59.773Z level=INFO msg="Uploading 1 narinfos"5452lb # [ 135.187945] niks3-server[1182]: 2026/09/25 15:14:59 INFO Signed narinfos id=43 count=15453lb # [ 135.193044] niks3-server[1182]: 2026/09/25 15:14:59 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5454lb # [ 135.208123] niks3-server[1182]: 2026/09/25 15:14:59 INFO Received complete push request method=POST path=/api/pushes/43/complete5455worker1 # [ 135.213975] nix-grpc-daemon[1311]: time=2026-09-25T15:14:59.804Z level=INFO msg="Upload complete. (287ms)"5456lb # [ 135.216146] niks3-server[1182]: 2026/09/25 15:14:59 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5457worker1 # [ 135.227245] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41516 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15458worker1 # [ 135.240549] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:59Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05459worker1 # [ 135.243624] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:59Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15460worker1 # [ 135.254590] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415165461worker1 # [ 135.259524] nix-daemon[837]: reaped child process 1329, status = succeeded5462worker1 # [ 135.267430] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415165463worker1 # [ 135.282518] nix-grpc-daemon[1311]: time=2026-09-25T15:14:59.872Z level=INFO msg="All 1 paths already cached"5464client # copying 1 paths to 'lb:50051'...5465client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5466worker1 # [ 135.295809] nix-grpc-daemon[1229]: ts=2026-09-25T15:14:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415165467worker1 # [ 135.306206] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5468worker1 # [ 135.460246] nix-daemon[837]: accepted connection from pid 1356, user nix-grpc-daemon (trusted)5469lb # [ 135.483748] niks3-server[1182]: 2026/09/25 15:15:00 INFO Received push request method=POST path=/api/pushes5470worker1 # [ 135.489569] nix-daemon[837]: reaped child process 1359, status = succeeded5471worker1 # [ 135.493292] nix-grpc-daemon[1311]: time=2026-09-25T15:15:00.082Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5472worker1 # [ 135.495037] nix-grpc-daemon[1311]: time=2026-09-25T15:15:00.082Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5473lb # [ 135.511837] niks3-server[1182]: 2026/09/25 15:15:00 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5474lb # [ 135.520242] niks3-server[1182]: 2026/09/25 15:15:00 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5475worker1 # [ 135.525893] nix-grpc-daemon[1311]: time=2026-09-25T15:15:00.115Z level=INFO msg="Uploading 1 narinfos"5476lb # [ 135.529033] niks3-server[1182]: 2026/09/25 15:15:00 INFO Signed narinfos id=44 count=15477lb # [ 135.534225] niks3-server[1182]: 2026/09/25 15:15:00 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5478lb # [ 135.544155] niks3-server[1182]: 2026/09/25 15:15:00 INFO Received complete push request method=POST path=/api/pushes/44/complete5479worker1 # [ 135.554288] nix-grpc-daemon[1311]: time=2026-09-25T15:15:00.144Z level=INFO msg="Upload complete. (235ms)"5480lb # [ 135.557172] niks3-server[1182]: 2026/09/25 15:15:00 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5481worker1 # [ 135.560418] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41516 duration_s=0 paths=1 nar_bytes_in=6405482worker1 # [ 135.569137] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415165483worker1 # [ 135.583063] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5484worker1 # [ 135.585138] nix-daemon[837]: reaped child process 1354, status = succeeded5485client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5486client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5487worker1 # [ 135.599126] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5488client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5489client # farm-b> LOG-farm-b-on-standby5490worker1 # [ 136.856488] nix-daemon[837]: reaped child process 1363, status = succeeded5491worker1 # [ 136.955801] nix-daemon[837]: accepted connection from pid 1379, user nix-grpc-daemon (trusted)5492worker1 # [ 136.983685] nix-daemon[837]: reaped child process 1382, status = succeeded5493lb # [ 136.988701] niks3-server[1182]: 2026/09/25 15:15:01 INFO Received push request method=POST path=/api/pushes5494worker1 # [ 136.996588] nix-grpc-daemon[1311]: time=2026-09-25T15:15:01.585Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5495worker1 # [ 136.998402] nix-grpc-daemon[1311]: time=2026-09-25T15:15:01.585Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5496lb # [ 137.015072] niks3-server[1182]: 2026/09/25 15:15:01 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5497lb # [ 137.035202] niks3-server[1182]: 2026/09/25 15:15:01 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5498lb # [ 137.043718] niks3-server[1182]: 2026/09/25 15:15:01 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign5499worker1 # [ 137.046528] nix-grpc-daemon[1311]: time=2026-09-25T15:15:01.636Z level=INFO msg="Uploading 1 narinfos"5500lb # [ 137.050661] niks3-server[1182]: 2026/09/25 15:15:01 INFO Signed narinfos id=45 count=15501lb # [ 137.056074] niks3-server[1182]: 2026/09/25 15:15:01 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5502lb # [ 137.070726] niks3-server[1182]: 2026/09/25 15:15:01 INFO Received complete push request method=POST path=/api/pushes/45/complete5503worker1 # [ 137.075397] nix-grpc-daemon[1311]: time=2026-09-25T15:15:01.665Z level=INFO msg="Upload complete. (265ms)"5504lb # [ 137.078664] niks3-server[1182]: 2026/09/25 15:15:01 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5505worker1 # [ 137.088472] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41516 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15506lb # [ 137.101169] envoy[893]: conn peer=[2001:db8:1::1]:49390 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9749 tx=4451 ms=60085507worker1 # [ 137.100758] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:01Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05508lb # [ 137.104429] envoy[893]: conn peer=[2001:db8:1::4]:44576 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4752 tx=637 ms=154525509worker1 # [ 137.110937] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:01Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15510worker1 # [ 137.116298] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415165511worker1 # [ 137.120573] nix-daemon[837]: reaped child process 1361, status = succeeded5512worker1 # [ 137.127640] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415165513worker1 # [ 137.138705] nix-grpc-daemon[1311]: time=2026-09-25T15:15:01.728Z level=INFO msg="All 2 paths already cached"5514client # copying 1 paths to 'lb:50051'...5515client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5516worker1 # [ 137.147752] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415165517worker1 # [ 137.158000] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5518worker1 # [ 137.295112] nix-daemon[837]: accepted connection from pid 1388, user nix-grpc-daemon (trusted)5519worker1 # [ 137.323746] nix-daemon[837]: reaped child process 1391, status = succeeded5520lb # [ 137.328881] niks3-server[1182]: 2026/09/25 15:15:01 INFO Received push request method=POST path=/api/pushes5521worker1 # [ 137.336607] nix-grpc-daemon[1311]: time=2026-09-25T15:15:01.926Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5522worker1 # [ 137.339840] nix-grpc-daemon[1311]: time=2026-09-25T15:15:01.929Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5523worker1 # [ 137.370051] nix-grpc-daemon[1311]: time=2026-09-25T15:15:01.959Z level=INFO msg="Uploading 1 narinfos"5524lb # [ 137.362006] niks3-server[1182]: 2026/09/25 15:15:01 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5525lb # [ 137.372892] niks3-server[1182]: 2026/09/25 15:15:01 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign5526lb # [ 137.380980] niks3-server[1182]: 2026/09/25 15:15:01 INFO Signed narinfos id=46 count=15527lb # [ 137.386359] niks3-server[1182]: 2026/09/25 15:15:01 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5528lb # [ 137.399211] niks3-server[1182]: 2026/09/25 15:15:01 INFO Received complete push request method=POST path=/api/pushes/46/complete5529worker1 # [ 137.406918] nix-grpc-daemon[1311]: time=2026-09-25T15:15:01.997Z level=INFO msg="Upload complete. (236ms)"5530lb # [ 137.408475] niks3-server[1182]: 2026/09/25 15:15:01 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5531worker1 # [ 137.416468] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41516 duration_s=0 paths=1 nar_bytes_in=7685532worker1 # [ 137.430617] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415165533worker1 # [ 137.436981] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5534worker1 # [ 137.450238] nix-daemon[837]: reaped child process 1386, status = succeeded5535client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5536worker1 # [ 137.456260] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5537client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5538client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5539client # farm-top> LOG-farm-top-on-standby5540worker1 # [ 138.890988] nix-daemon[837]: reaped child process 1395, status = succeeded5541worker1 # [ 139.041353] nix-daemon[837]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)5542worker1 # [ 139.073853] nix-daemon[837]: reaped child process 1414, status = succeeded5543lb # [ 139.078785] niks3-server[1182]: 2026/09/25 15:15:03 INFO Received push request method=POST path=/api/pushes5544worker1 # [ 139.095480] nix-grpc-daemon[1311]: time=2026-09-25T15:15:03.682Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5545worker1 # [ 139.097901] nix-grpc-daemon[1311]: time=2026-09-25T15:15:03.682Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5546lb # [ 139.119850] niks3-server[1182]: 2026/09/25 15:15:03 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5547lb # [ 139.128503] niks3-server[1182]: 2026/09/25 15:15:03 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign5548worker1 # [ 139.129309] nix-grpc-daemon[1311]: time=2026-09-25T15:15:03.719Z level=INFO msg="Uploading 1 narinfos"5549lb # [ 139.137583] niks3-server[1182]: 2026/09/25 15:15:03 INFO Signed narinfos id=47 count=15550lb # [ 139.139753] niks3-server[1182]: 2026/09/25 15:15:03 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5551lb # [ 139.152470] niks3-server[1182]: 2026/09/25 15:15:03 INFO Received complete push request method=POST path=/api/pushes/47/complete5552worker1 # [ 139.167144] nix-grpc-daemon[1311]: time=2026-09-25T15:15:03.757Z level=INFO msg="Upload complete. (296ms)"5553lb # [ 139.171214] niks3-server[1182]: 2026/09/25 15:15:03 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5554lb # [ 139.184782] niks3-server[1182]: 2026/09/25 15:15:03 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5555worker1 # [ 139.176777] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41516 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15556worker1 # [ 139.189159] nix-daemon[837]: reaped child process 1393, status = succeeded5557client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.87 seconds)5558client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5559client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.17 seconds)5560worker2 # [ 139.427558] systemd[1]: Starting scheduler-token.service...5561worker2 # [ 139.500542] systemd[1]: scheduler-token.service: Deactivated successfully.5562worker2 # [ 139.504725] systemd[1]: Finished scheduler-token.service.5563worker2 # [ 139.506098] systemd[1]: scheduler-token.service: Consumed 40ms CPU time over 76ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5564worker2 # [ 139.526607] systemd[1]: Starting Nix worker-protocol over gRPC...5565worker2 # [ 139.655130] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5566worker2 # [ 139.657890] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5567worker2 # [ 139.662148] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15568worker2 # [ 139.692792] nix-grpc-daemon[1517]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5569worker2 # [ 139.714538] nix-grpc-daemon[1517]: E0000 00:00:1790349304.260410 1517 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5570lb: must succeed: curl -sf localhost:9901/clusters5571worker2 # [ 139.722159] nix-grpc-daemon[1517]: E0000 00:00:1790349304.263207 1517 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5572worker2 # [ 139.724973] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:04Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5573worker2 # [ 139.730299] nix-grpc-daemon[1517]: E0000 00:00:1790349304.271613 1560 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5574worker2 # [ 139.735870] nix-grpc-daemon[1517]: E0000 00:00:1790349304.292564 1560 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5575worker2 # [ 139.738866] nix-grpc-daemon[1517]: E0000 00:00:1790349304.292829 1560 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5576worker2 # [ 139.742019] nix-grpc-daemon[1517]: E0000 00:00:1790349304.313508 1560 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5577worker2 # [ 139.744459] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:04Z level=info event=scheduler_connected addr=lb:500515578worker2 # [ 139.745811] systemd[1]: Started Nix worker-protocol over gRPC.5579worker1 # [ 139.742164] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:04Z level=info event=oidc_jwks provider=mock keys=15580worker1 # [ 139.746066] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:04Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:387365581worker1 # [ 139.752430] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05582lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5583worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5584worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5585lb: must succeed: curl -sf localhost:9901/clusters5586lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5587lb: must succeed: curl -sf localhost:9901/clusters5588lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5589worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5590worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5591(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.41 seconds)5592subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5593client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5594client # [ 139.994860] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5595client # Running as unit: rs.service; invocation ID: 439efc25262c49df9417deba5693d8cb5596client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.06 seconds)5597client # [ 140.200523] nix[1363]: don't know how to build these paths:5598worker1 # [ 140.199013] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:415165599client # [ 140.201719] nix[1363]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5600worker1 # [ 140.210966] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:04Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05601worker1 # [ 140.215500] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:04Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15602worker1 # [ 140.221521] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415165603worker1 # [ 140.229310] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415165604client # [ 140.240126] nix[1363]: copying 1 paths to 'lb:50051'...5605client # [ 140.241686] nix[1363]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5606worker1 # [ 140.240798] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415165607worker1 # [ 140.250633] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5608worker1 # [ 140.397089] nix-daemon[837]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)5609worker1 # [ 140.423022] nix-daemon[837]: reaped child process 1430, status = succeeded5610lb # [ 140.431462] niks3-server[1182]: 2026/09/25 15:15:05 INFO Received push request method=POST path=/api/pushes5611worker1 # [ 140.447917] nix-grpc-daemon[1311]: time=2026-09-25T15:15:05.037Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5612worker1 # [ 140.450058] nix-grpc-daemon[1311]: time=2026-09-25T15:15:05.037Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5613lb # [ 140.463098] niks3-server[1182]: 2026/09/25 15:15:05 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign5614worker1 # [ 140.472195] nix-grpc-daemon[1311]: time=2026-09-25T15:15:05.062Z level=INFO msg="Uploading 1 narinfos"5615lb # [ 140.473644] niks3-server[1182]: 2026/09/25 15:15:05 INFO Signed narinfos id=48 count=15616lb # [ 140.479636] niks3-server[1182]: 2026/09/25 15:15:05 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5617lb # [ 140.488250] niks3-server[1182]: 2026/09/25 15:15:05 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5618lb # [ 140.500366] niks3-server[1182]: 2026/09/25 15:15:05 INFO Received complete push request method=POST path=/api/pushes/48/complete5619lb # [ 140.511855] niks3-server[1182]: 2026/09/25 15:15:05 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5620worker1 # [ 140.515367] nix-grpc-daemon[1311]: time=2026-09-25T15:15:05.105Z level=INFO msg="Upload complete. (251ms)"5621worker1 # [ 140.522451] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41516 duration_s=0 paths=1 nar_bytes_in=4165622worker1 # [ 140.532436] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415165623worker1 # [ 140.543903] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5624worker1 # [ 140.550040] nix-daemon[837]: reaped child process 1425, status = succeeded5625client # [ 140.560626] nix[1363]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5626worker1 # [ 140.565277] nix-daemon[837]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5627lb: must succeed: curl -sf localhost:9901/clusters5628lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5629lb: must succeed: curl -sf localhost:9901/clusters5630lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5631lb: must succeed: curl -sf localhost:9901/clusters5632lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5633lb: must succeed: curl -sf localhost:9901/clusters5634lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5635worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5636worker2 # [ 142.654900] systemd-journald[273]: Received client request to rotate journal, rotating.5637worker2 # [ 142.670365] systemd-journald[273]: Vacuuming done, freed 0B of archived journals from /var/log/journal/4a671b3b6423413bb76b7ee36fad8662.5638worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)5639worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5640worker1 # [ 142.716095] systemd[1]: Stopping Nix worker-protocol over gRPC...5641worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5642client: must succeed: sleep 5.05643worker1 # [ 143.531212] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:08Z level=info event=draining5644lb # [ 147.102141] envoy[893]: conn peer=[2001:db8:1::1]:34572 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9606 tx=4388 ms=56575645client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5646worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5647worker1 # [ 148.006814] nix-daemon[837]: reaped child process 1434, status = succeeded5648worker1 # [ 148.105995] nix-daemon[837]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5649worker1 # [ 148.128704] nix-daemon[837]: reaped child process 1478, status = succeeded5650lb # [ 148.135732] niks3-server[1182]: 2026/09/25 15:15:12 INFO Received push request method=POST path=/api/pushes5651worker1 # [ 148.144468] nix-grpc-daemon[1311]: time=2026-09-25T15:15:12.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5652worker1 # [ 148.146147] nix-grpc-daemon[1311]: time=2026-09-25T15:15:12.733Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5653lb # [ 148.178182] niks3-server[1182]: 2026/09/25 15:15:12 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign5654worker1 # [ 148.184412] nix-grpc-daemon[1311]: time=2026-09-25T15:15:12.774Z level=INFO msg="Uploading 1 narinfos"5655lb # [ 148.187220] niks3-server[1182]: 2026/09/25 15:15:12 INFO Signed narinfos id=49 count=15656lb # [ 148.194652] niks3-server[1182]: 2026/09/25 15:15:12 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5657lb # [ 148.199230] niks3-server[1182]: 2026/09/25 15:15:12 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5658lb # [ 148.203595] niks3-server[1182]: 2026/09/25 15:15:12 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5659lb # [ 148.212089] niks3-server[1182]: 2026/09/25 15:15:12 INFO Received complete push request method=POST path=/api/pushes/49/complete5660worker1 # [ 148.220907] nix-grpc-daemon[1311]: time=2026-09-25T15:15:12.811Z level=INFO msg="Upload complete. (238ms)"5661lb # [ 148.221809] niks3-server[1182]: 2026/09/25 15:15:12 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5662worker1 # [ 148.233096] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41516 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15663worker1 # [ 148.243750] nix-daemon[837]: reaped child process 1432, status = succeeded5664client # [ 148.252692] systemd[1]: rs.service: Deactivated successfully.5665client # [ 148.255540] systemd[1]: rs.service: Consumed 191ms CPU time over 8.260s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.5666worker1 # [ 148.533251] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:13Z level=info event=signal_exit5667lb # [ 148.792202] niks3-server[1182]: 2026/09/25 15:15:13 INFO lead: released remote=192.168.1.3:416685668lb # [ 149.678153] niks3-server[1182]: 2026/09/25 15:15:14 INFO lead: acquired remote=192.168.1.4:421625669worker2 # [ 149.681334] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:14Z level=info event=scheduler_take_over5670worker1 # [ 149.789536] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:14Z level=info event=scheduler_restarting5671lb # [ 149.790299] envoy[893]: [2026-09-25 15:15:14.376][893][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5672worker2 # [ 149.798009] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:14Z level=info event=scheduler_restarting addr=lb:500515673worker1 # [ 149.796977] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:387365674worker1 # [ 149.799561] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:14Z level=info event=worker_gone addr=192.168.1.4:500515675worker1 # [ 149.802560] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:14Z level=info event=scheduler_restarting addr=lb:500515676worker1 # [ 149.808459] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:387365677worker1 # [ 149.811076] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:14Z level=info event=worker_gone addr=192.168.1.3:500515678worker1 # [ 149.829364] nix-daemon[837]: reaped child process 1281, status = succeeded5679worker2 # [ 149.999548] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:14Z level=info event=scheduler_connected addr=lb:500515680worker2 # [ 150.003022] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5681worker1 # [ 150.017388] nix-grpc-daemon[1229]: ts=2026-09-25T15:15:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5682worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.13 seconds)5683client: waiting for success: ! systemctl is-active rs5684client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)5685client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5686client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5687client: must fail: journalctl -u rs | grep -q 'reconnecting'5688worker2 # [ 150.204000] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:14Z level=info event=scheduler_connected addr=lb:500515689client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5690lb: must succeed: curl -sf localhost:9901/clusters5691worker2 # [ 150.207315] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5692lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5693worker2 # [ 150.408197] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_connected addr=lb:500515694worker2 # [ 150.411508] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5695worker2 # [ 150.612546] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_connected addr=lb:500515696worker2 # [ 150.615738] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5697worker2 # [ 150.818029] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_connected addr=lb:500515698worker2 # [ 150.820595] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5699worker2 # [ 151.018733] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_connected addr=lb:500515700worker2 # [ 151.022132] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5701worker2 # [ 151.223140] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_connected addr=lb:500515702worker2 # [ 151.226522] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5703lb: must succeed: curl -sf localhost:9901/clusters5704lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5705worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5706worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5707worker2 # [ 151.427411] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:16Z level=info event=scheduler_connected addr=lb:500515708worker2 # [ 151.438501] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:16Z level=info event=oidc_jwks provider=mock keys=15709worker2 # [ 151.441343] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:441745710worker2 # [ 151.444415] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05711worker2 # [ 151.465940] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:16Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:441885712worker1 # [ 151.855565] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5713worker1 # [ 151.856979] systemd[1]: Stopped Nix worker-protocol over gRPC.5714worker1 # [ 151.857316] systemd[1]: nix-grpc-daemon.service: Consumed 1.585s CPU time over 32.486s wall clock time, 39.8M memory peak, 331.1K incoming IP traffic, 144.1K outgoing IP traffic.5715worker1 # [ 151.872839] systemd[1]: Starting Nix worker-protocol over gRPC...5716worker1 # [ 151.994498] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5717worker1 # [ 151.997089] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5718worker1 # [ 152.001162] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15719worker1 # [ 152.007813] nix-grpc-daemon[1486]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5720worker1 # [ 152.028324] nix-grpc-daemon[1486]: E0000 00:00:1790349316.595887 1486 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5721worker1 # [ 152.053043] nix-grpc-daemon[1486]: E0000 00:00:1790349316.598045 1486 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5722worker1 # [ 152.056153] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:16Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5723worker1 # [ 152.058008] nix-grpc-daemon[1486]: E0000 00:00:1790349316.606795 1526 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5724worker1 # [ 152.063735] nix-grpc-daemon[1486]: E0000 00:00:1790349316.627006 1526 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5725worker1 # [ 152.068732] nix-grpc-daemon[1486]: E0000 00:00:1790349316.627241 1526 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5726worker2 # [ 152.072561] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:441745727worker1 # [ 152.072051] nix-grpc-daemon[1486]: E0000 00:00:1790349316.642226 1526 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5728worker2 # [ 152.072808] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05729worker1 # [ 152.077247] systemd[1]: Started Nix worker-protocol over gRPC.5730worker1 # [ 152.078777] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:16Z level=info event=scheduler_connected addr=lb:500515731lb: must succeed: curl -sf localhost:9901/clusters5732lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5733worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5734worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5735(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.47 seconds)5736subtest: scheduler restart mid-build: build finishes, no double build5737client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5738client # [ 152.458531] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5739client # Running as unit: mid.service; invocation ID: 7bece89d4b7642a88a533344c48ecc225740client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.06 seconds)5741worker1 # [ 152.645637] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:562245742worker1 # [ 152.654925] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)5743client # [ 152.666967] nix[1399]: don't know how to build these paths:5744client # [ 152.669712] nix[1399]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5745worker2 # [ 152.678832] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:17Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05746worker2 # [ 152.683535] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:17Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15747worker2 # [ 152.696129] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442025748worker2 # [ 152.706035] nix-daemon[859]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5749worker2 # [ 152.722944] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 1/5)5750worker2 # [ 152.729984] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 2/5)5751worker2 # [ 152.776761] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 3/5)5752worker2 # [ 152.938620] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 716 ms (attempt 4/5)5753worker2 # [ 153.659061] nix-grpc-daemon[1517]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5754worker2 # [ 153.661696] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5755worker2 # [ 153.678744] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442025756client # [ 153.686512] nix[1399]: copying 1 paths to 'lb:50051'...5757worker2 # [ 153.690537] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442025758worker2 # [ 153.701891] nix-daemon[859]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5759worker2 # [ 153.721574] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:18Z level=info event=niks3_push_started pid=16435760worker2 # [ 153.871492] nix-grpc-daemon[1643]: time=2026-09-25T15:15:18.464Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5761worker2 # [ 153.994298] nix-daemon[859]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5762lb # [ 154.029193] niks3-server[1182]: 2026/09/25 15:15:18 INFO Received push request method=POST path=/api/pushes5763worker2 # [ 154.032431] nix-daemon[859]: reaped child process 1659, status = succeeded5764worker2 # [ 154.041356] nix-grpc-daemon[1643]: time=2026-09-25T15:15:18.633Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5765worker2 # [ 154.043077] nix-grpc-daemon[1643]: time=2026-09-25T15:15:18.633Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5766lb # [ 154.063125] niks3-server[1182]: 2026/09/25 15:15:18 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign5767lb # [ 154.070787] niks3-server[1182]: 2026/09/25 15:15:18 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5768lb # [ 154.079853] niks3-server[1182]: 2026/09/25 15:15:18 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5769worker2 # [ 154.090208] nix-grpc-daemon[1643]: time=2026-09-25T15:15:18.682Z level=INFO msg="Uploading 1 narinfos"5770lb # [ 154.087002] niks3-server[1182]: 2026/09/25 15:15:18 INFO Signed narinfos id=50 count=15771lb # [ 154.100743] niks3-server[1182]: 2026/09/25 15:15:18 INFO Received complete push request method=POST path=/api/pushes/50/complete5772worker2 # [ 154.108637] nix-grpc-daemon[1643]: time=2026-09-25T15:15:18.701Z level=INFO msg="Upload complete. (232ms)"5773worker2 # [ 154.116249] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44202 duration_s=0 paths=1 nar_bytes_in=4245774lb # [ 154.117467] niks3-server[1182]: 2026/09/25 15:15:18 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5775worker2 # [ 154.131747] nix-grpc-daemon[1517]: ts=2026-09-25T15:15:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442025776worker2 # [ 154.138301] nix-daemon[859]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5777worker2 # [ 154.143449] nix-daemon[859]: reaped child process 1641, status = succeeded5778client # [ 153.690870] nix[1399]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5779client # [ 154.152636] nix[1399]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5780worker2 # [ 154.159256] nix-daemon[859]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5781lb: must succeed: curl -sf localhost:9901/clusters5782lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5783lb: must succeed: curl -sf localhost:9901/clusters5784lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5785worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5786worker2 # [ 155.037057] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1517 (nix-grpc-daemon) on client request.5787worker2 # [ 155.038819] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5788lb # [ 155.047232] niks3-server[1182]: 2026/09/25 15:15:19 INFO lead: released remote=192.168.1.4:421625789client # [ 155.052221] nix[1399]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5790worker2 # [ 155.056213] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5791worker1 # [ 155.051837] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:19Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5792lb # [ 155.052184] envoy[893]: [2026-09-25 15:15:19.637][893][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5793worker2 # [ 155.057921] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5794client # [ 155.055803] nix[1399]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5795worker2 # [ 155.059287] systemd[1]: nix-grpc-daemon.service: Consumed 536ms CPU time over 15.538s wall clock time, 35.1M memory peak, 215K incoming IP traffic, 56.5K outgoing IP traffic.5796worker2 # [ 155.088349] systemd[1]: Starting scheduler-token.service...5797worker2 # [ 155.123802] nix-daemon[859]: reaped child process 1661, status = succeeded5798worker2 # [ 155.127422] nix-daemon[859]: reaped child process 1631, status = succeeded5799worker2 # [ 155.136544] nix-daemon[1663]: unexpected Nix daemon error: error: interrupted by the user5800worker2 # [ 155.152097] nix-daemon[859]: reaped child process 1663, status = failed with exit code 15801worker2 # [ 155.161154] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5802worker2 # [ 155.163683] systemd[1]: scheduler-token.service: Deactivated successfully.5803worker2 # [ 155.166434] systemd[1]: Finished scheduler-token.service.5804worker2 # [ 155.167298] systemd[1]: scheduler-token.service: Consumed 27ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5805worker2 # [ 155.172838] systemd[1]: Starting Nix worker-protocol over gRPC...5806worker1 # [ 155.252181] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:19Z level=info event=scheduler_connected addr=lb:500515807worker2 # [ 155.278114] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5808worker2 # [ 155.280593] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5809worker2 # [ 155.284646] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15810worker1 # [ 155.286296] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:19Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. retried and the latest reset reason: remote connection failure"5811worker2 # [ 155.291827] nix-grpc-daemon[1699]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5812lb # [ 155.299558] niks3-server[1182]: 2026/09/25 15:15:19 INFO lead: acquired remote=192.168.1.4:511325813worker2 # [ 155.316193] nix-grpc-daemon[1699]: E0000 00:00:1790349319.882879 1699 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5814worker2 # [ 155.318723] nix-grpc-daemon[1699]: E0000 00:00:1790349319.885060 1699 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5815worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.34 seconds)5816client: must succeed: sleep 5.05817worker2 # [ 155.342065] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:19Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5818worker2 # [ 155.344535] nix-grpc-daemon[1699]: E0000 00:00:1790349319.893928 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5819worker2 # [ 155.350290] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:19Z level=info event=scheduler_take_over5820worker2 # [ 155.357830] nix-grpc-daemon[1699]: E0000 00:00:1790349319.917905 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5821worker2 # [ 155.361469] nix-grpc-daemon[1699]: E0000 00:00:1790349319.918127 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5822worker2 # [ 155.365966] nix-grpc-daemon[1699]: E0000 00:00:1790349319.934258 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5823worker2 # [ 155.369689] systemd[1]: Started Nix worker-protocol over gRPC.5824worker2 # [ 155.371047] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:19Z level=info event=scheduler_connected addr=lb:500515825worker2 # [ 155.374565] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:19Z level=info event=oidc_jwks provider=mock keys=15826worker2 # [ 155.375954] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:19Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:442385827worker2 # [ 155.378104] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05828worker1 # [ 155.486995] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:20Z level=info event=scheduler_connected addr=lb:500515829worker2 # [ 155.491172] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:20Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:442385830worker2 # [ 155.494187] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:20Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05831worker2 # [ 155.562638] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:20Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05832worker2 # [ 155.570039] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:20Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15833worker2 # [ 155.577413] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442485834worker2 # [ 155.585312] nix-daemon[859]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)5835worker2 # [ 155.600552] nix-daemon[859]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)5836client # [ 155.609810] nix[1399]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5837worker2 # [ 155.613131] nix-daemon[859]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)5838worker2 # [ 156.151676] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:20Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:442665839lb # [ 157.106607] envoy[893]: conn peer=[2001:db8:1::1]:34582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1514 ms=80425840lb # [ 157.109609] envoy[893]: conn peer=[2001:db8:1::3]:36146 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3109 tx=1033 ms=304405841lb # [ 157.112379] envoy[893]: conn peer=[2001:db8:1::4]:36922 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7249 tx=584 ms=153295842client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5843client: waiting for success: ! systemctl is-active mid5844worker2 # [ 160.633732] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:25Z level=info event=niks3_push_started pid=17825845worker2 # [ 160.702813] nix-daemon[859]: reaped child process 1757, status = succeeded5846worker2 # [ 160.772535] nix-grpc-daemon[1782]: time=2026-09-25T15:15:25.365Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5847worker2 # [ 160.885802] nix-daemon[859]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)5848worker2 # [ 160.906976] nix-daemon[859]: reaped child process 1797, status = succeeded5849lb # [ 160.913701] niks3-server[1182]: 2026/09/25 15:15:25 INFO Received push request method=POST path=/api/pushes5850worker2 # [ 160.924120] nix-grpc-daemon[1782]: time=2026-09-25T15:15:25.517Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5851worker2 # [ 160.925742] nix-grpc-daemon[1782]: time=2026-09-25T15:15:25.517Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5852lb # [ 160.956968] niks3-server[1182]: 2026/09/25 15:15:25 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign5853worker2 # [ 160.966557] nix-grpc-daemon[1782]: time=2026-09-25T15:15:25.558Z level=INFO msg="Uploading 1 narinfos"5854lb # [ 160.967251] niks3-server[1182]: 2026/09/25 15:15:25 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5855lb # [ 160.977121] niks3-server[1182]: 2026/09/25 15:15:25 INFO Signed narinfos id=51 count=15856lb # [ 160.983438] niks3-server[1182]: 2026/09/25 15:15:25 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5857lb # [ 160.995986] niks3-server[1182]: 2026/09/25 15:15:25 INFO Received complete push request method=POST path=/api/pushes/51/complete5858lb # [ 161.008004] niks3-server[1182]: 2026/09/25 15:15:25 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5859lb # [ 161.015578] niks3-server[1182]: 2026/09/25 15:15:25 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5860worker2 # [ 161.020331] nix-grpc-daemon[1782]: time=2026-09-25T15:15:25.613Z level=INFO msg="Upload complete. (243ms)"5861worker2 # [ 161.032142] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44248 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15862worker2 # [ 161.042696] nix-daemon[859]: reaped child process 1755, status = succeeded5863client # [ 161.049697] systemd[1]: mid.service: Deactivated successfully.5864client # [ 161.052263] systemd[1]: mid.service: Consumed 185ms CPU time over 8.592s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.4K outgoing IP traffic.5865client: (finished: waiting for success: ! systemctl is-active mid, in 1.07 seconds)5866client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5867client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5868(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.26 seconds)5869subtest: interrupting the client stops the build on the worker5870client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5871client # [ 161.711754] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5872client # Running as unit: intr.service; invocation ID: dbb128d7c5ce4b978523a205627adcc25873client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.04 seconds)5874worker1 # [ 161.874032] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:562245875client # [ 161.879398] nix[1435]: don't know how to build these paths:5876client # [ 161.880453] nix[1435]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5877worker2 # [ 161.891967] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:26Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05878worker1 # [ 161.892878] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562245879worker2 # [ 161.895483] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:26Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15880worker1 # [ 161.901307] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5)5881worker1 # [ 161.996814] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 2/5)5882worker1 # [ 162.072209] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 242 ms (attempt 3/5)5883worker1 # [ 162.319879] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 770 ms (attempt 4/5)5884worker1 # [ 163.097502] nix-grpc-daemon[1486]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5885worker1 # [ 163.097910] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5886worker1 # [ 163.112895] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562245887client # [ 163.124637] nix[1435]: copying 1 paths to 'lb:50051'...5888worker1 # [ 163.123267] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562245889client # [ 163.124806] nix[1435]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5890worker1 # [ 163.132822] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)5891worker1 # [ 163.158423] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:27Z level=info event=niks3_push_started pid=15785892worker1 # [ 163.282318] nix-grpc-daemon[1578]: time=2026-09-25T15:15:27.870Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5893worker1 # [ 163.386747] nix-daemon[837]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)5894lb # [ 163.406763] niks3-server[1182]: 2026/09/25 15:15:27 INFO Received push request method=POST path=/api/pushes5895worker1 # [ 163.408781] nix-daemon[837]: reaped child process 1594, status = succeeded5896worker1 # [ 163.415610] nix-grpc-daemon[1578]: time=2026-09-25T15:15:28.004Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5897worker1 # [ 163.420945] nix-grpc-daemon[1578]: time=2026-09-25T15:15:28.004Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5898lb # [ 163.437718] niks3-server[1182]: 2026/09/25 15:15:28 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign5899worker1 # [ 163.444959] nix-grpc-daemon[1578]: time=2026-09-25T15:15:28.034Z level=INFO msg="Uploading 1 narinfos"5900lb # [ 163.445766] niks3-server[1182]: 2026/09/25 15:15:28 INFO Signed narinfos id=52 count=15901lb # [ 163.452242] niks3-server[1182]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5902lb # [ 163.460187] niks3-server[1182]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5903lb # [ 163.469881] niks3-server[1182]: 2026/09/25 15:15:28 INFO Received complete push request method=POST path=/api/pushes/52/complete5904lb # [ 163.477687] niks3-server[1182]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5905worker1 # [ 163.480718] nix-grpc-daemon[1578]: time=2026-09-25T15:15:28.070Z level=INFO msg="Upload complete. (200ms)"5906worker1 # [ 163.488298] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56224 duration_s=0 paths=1 nar_bytes_in=4245907worker1 # [ 163.491770] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562245908worker1 # [ 163.506994] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)5909worker1 # [ 163.508867] nix-daemon[837]: reaped child process 1576, status = succeeded5910client # [ 163.517207] nix[1435]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5911worker1 # [ 163.517808] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)5912client: must succeed: systemctl kill -s INT intr5913client # [ 165.156295] systemd[1]: intr.service: Sent signal SIGINT to main process 1435 (nix) on client request.5914worker1 # [ 165.157672] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:29Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5915client # [ 165.160570] nix[1435]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5916client # [ 165.168300] nix[1435]: error: interrupted by the user5917client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)5918client # [ 165.188241] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5919client # [ 165.192596] systemd[1]: intr.service: Failed with result 'exit-code'.5920client # [ 165.194681] systemd[1]: intr.service: Consumed 158ms CPU time over 3.479s wall clock time, 41.5M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.5921worker1 # [ 165.345253] nix-daemon[837]: reaped child process 1596, status = succeeded5922worker1 # [ 165.349724] nix-daemon[1598]: unexpected Nix daemon error: error: interrupted by the user5923worker1 # [ 165.357504] nix-daemon[837]: reaped child process 1598, status = failed with exit code 15924(finished: subtest: interrupting the client stops the build on the worker, in 4.69 seconds)5925subtest: a deploy mid-build drains: the build finishes, the new generation takes over5926client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5927client # [ 166.404670] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5928client # Running as unit: sw.service; invocation ID: 1041712933c54c3b875fc7398bed437b5929client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.06 seconds)5930client # [ 166.597019] nix[1455]: don't know how to build these paths:5931worker1 # [ 166.595483] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:562245932client # [ 166.598102] nix[1455]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5933worker2 # [ 166.606372] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:31Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05934worker2 # [ 166.610454] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:31Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15935worker2 # [ 166.614507] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442485936worker2 # [ 166.618798] nix-grpc-daemon[1699]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5)5937worker2 # [ 166.644880] nix-grpc-daemon[1699]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 2/5)5938worker2 # [ 166.815934] nix-grpc-daemon[1699]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 3/5)5939worker2 # [ 166.919973] nix-grpc-daemon[1699]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 420 ms (attempt 4/5)5940lb # [ 167.106555] envoy[893]: conn peer=[2001:db8:1::1]:55026 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3927 tx=1964 ms=83935941worker2 # [ 167.342141] nix-grpc-daemon[1699]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5942worker2 # [ 167.346635] nix-grpc-daemon[1699]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5943worker2 # [ 167.361021] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442485944client # [ 167.370195] nix[1455]: copying 1 paths to 'lb:50051'...5945client # [ 167.373164] nix[1455]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5946worker2 # [ 167.376668] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442485947worker2 # [ 167.388243] nix-daemon[859]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)5948worker2 # [ 167.524223] nix-daemon[859]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)5949lb # [ 167.109484] envoy[893]: conn peer=[2001:db8:1::1]:34588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=825 ms=33005950worker2 # [ 167.546964] nix-daemon[859]: reaped child process 1840, status = succeeded5951lb # [ 167.543033] niks3-server[1182]: 2026/09/25 15:15:32 INFO Received push request method=POST path=/api/pushes5952worker2 # [ 167.556424] nix-grpc-daemon[1782]: time=2026-09-25T15:15:32.148Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5953worker2 # [ 167.558224] nix-grpc-daemon[1782]: time=2026-09-25T15:15:32.148Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5954lb # [ 167.567487] niks3-server[1182]: 2026/09/25 15:15:32 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign5955worker2 # [ 167.575418] nix-grpc-daemon[1782]: time=2026-09-25T15:15:32.168Z level=INFO msg="Uploading 1 narinfos"5956lb # [ 167.576672] niks3-server[1182]: 2026/09/25 15:15:32 INFO Signed narinfos id=53 count=15957lb # [ 167.578529] niks3-server[1182]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5958lb # [ 167.586803] niks3-server[1182]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5959lb # [ 167.595302] niks3-server[1182]: 2026/09/25 15:15:32 INFO Received complete push request method=POST path=/api/pushes/53/complete5960lb # [ 167.599827] niks3-server[1182]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5961worker2 # [ 167.601178] nix-grpc-daemon[1782]: time=2026-09-25T15:15:32.194Z level=INFO msg="Upload complete. (201ms)"5962worker2 # [ 167.609095] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44248 duration_s=0 paths=1 nar_bytes_in=4165963worker2 # [ 167.617470] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442485964worker2 # [ 167.622608] nix-daemon[859]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)5965worker2 # [ 167.624684] nix-daemon[859]: reaped child process 1835, status = succeeded5966worker2 # [ 167.632939] nix-daemon[859]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)5967client # [ 167.630322] nix[1455]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5968worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25969worker2 # Checking switch inhibitors... done5970worker2 # activating the configuration...5971worker2 # [ 169.228714] nixos[1876]: switching to system configuration /nix/store/9l3gbwk715f7y7pzflv5jzj6n33r5brk-nixos-system-worker2-test5972worker2 # [ 169.230661] systemd[1]: Stopped target Remote File Systems.5973worker2 # [ 169.231485] systemd[1]: Stopped target Local File Systems.5974worker2 # setting up /etc...5975worker2 # [ 169.865409] systemd[1]: Reload requested from client PID 1876 ('.switch-to-conf') (unit backdoor.service)...5976worker2 # [ 169.867131] systemd[1]: Reloading...5977worker2 # [ 172.692554] systemd[1]: Reloading finished in 2815 ms.5978worker2 # restarting sysinit-reactivation.target5979worker2 # [ 172.801597] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5980worker2 # [ 173.211127] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5981worker2 # [ 173.216642] systemd[1]: Reached target Reactivate sysinit units.5982worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5983worker2 # [ 173.222242] systemd[1]: Reloading D-Bus System Message Bus...5984worker2 # [ 173.228675] dbus-broker-launch[493]: Caught SIGHUP, trigger reload.5985worker2 # [ 173.233023] systemd[1]: Reloading Nix worker-protocol over gRPC...5986worker2 # [ 173.301146] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'...5987worker2 # [ 173.307842] systemd[1]: Reloaded Nix worker-protocol over gRPC.5988worker2 # [ 173.315556] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:37Z level=info event=draining5989worker2 # [ 173.323018] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync'5990worker2 # [ 173.330197] dbus-broker-launch[493]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5991worker2 # [ 173.349046] systemd[1]: Reloaded D-Bus System Message Bus.5992worker2 # [ 173.353846] systemd[1]: Reached target Remote File Systems.5993worker2 # [ 173.361727] systemd[1]: Reached target Local File Systems.5994worker2 # [ 173.363102] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5995worker2 # [ 173.379293] systemd[1]: Starting Create SUID/SGID Wrappers...5996worker2 # [ 173.392232] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5997worker2 # [ 173.394269] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675998worker2 # [ 173.410078] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5999worker2 # [ 173.418645] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6000worker2 # [ 173.426098] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6001worker2 # [ 173.435025] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6002worker2 # [ 173.441199] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6003worker2 # [ 173.970808] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6004worker2 # [ 173.980126] systemd[1]: Finished Create SUID/SGID Wrappers.6005worker2 # [ 173.981112] systemd[1]: Starting linger-users.service...6006worker2 # [ 174.023360] systemd[1]: linger-users.service: Deactivated successfully.6007worker2 # [ 174.028230] systemd[1]: Finished linger-users.service.6008worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6009worker2 # [ 174.277549] nixos[1876]: finished switching to system configuration /nix/store/9l3gbwk715f7y7pzflv5jzj6n33r5brk-nixos-system-worker2-test6010worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 5.49 seconds)6011worker2: must succeed: pgrep -f 'read -t [9]0 x'6012worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)6013client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26014worker1 # [ 174.531207] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:562246015client # don't know how to build these paths:6016client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6017worker1 # [ 174.539381] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562246018worker2 # [ 174.559728] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:39Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06019worker2 # [ 174.564199] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:39Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16020worker1 # [ 174.562722] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562246021worker1 # [ 174.573645] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562246022client # copying 1 paths to 'lb:50051'...6023worker1 # [ 174.584755] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562246024client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6025worker1 # [ 174.596771] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6026worker1 # [ 174.734343] nix-daemon[837]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6027worker1 # [ 174.754885] nix-daemon[837]: reaped child process 1652, status = succeeded6028lb # [ 174.761501] niks3-server[1182]: 2026/09/25 15:15:39 INFO Received push request method=POST path=/api/pushes6029worker1 # [ 174.769069] nix-grpc-daemon[1578]: time=2026-09-25T15:15:39.358Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6030worker1 # [ 174.770699] nix-grpc-daemon[1578]: time=2026-09-25T15:15:39.358Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6031lb # [ 174.783732] niks3-server[1182]: 2026/09/25 15:15:39 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6032worker1 # [ 174.787835] nix-grpc-daemon[1578]: time=2026-09-25T15:15:39.377Z level=INFO msg="Uploading 1 narinfos"6033lb # [ 174.792309] niks3-server[1182]: 2026/09/25 15:15:39 INFO Signed narinfos id=54 count=16034lb # [ 174.794362] niks3-server[1182]: 2026/09/25 15:15:39 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6035lb # [ 174.804834] niks3-server[1182]: 2026/09/25 15:15:39 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6036lb # [ 174.818036] niks3-server[1182]: 2026/09/25 15:15:39 INFO Received complete push request method=POST path=/api/pushes/54/complete6037worker1 # [ 174.823308] nix-grpc-daemon[1578]: time=2026-09-25T15:15:39.413Z level=INFO msg="Upload complete. (212ms)"6038lb # [ 174.825356] niks3-server[1182]: 2026/09/25 15:15:39 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6039worker1 # [ 174.831515] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56224 duration_s=0 paths=1 nar_bytes_in=5286040worker1 # [ 174.844403] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562246041worker1 # [ 174.850581] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6042worker1 # [ 174.855535] nix-daemon[837]: reaped child process 1647, status = succeeded6043client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6044worker1 # [ 174.866207] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6045client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6046client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6047client # farm-a> LOG-farm-a-during-drain6048worker1 # [ 176.120872] nix-daemon[837]: reaped child process 1656, status = succeeded6049worker1 # [ 176.227264] nix-daemon[837]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6050worker1 # [ 176.248782] nix-daemon[837]: reaped child process 1675, status = succeeded6051lb # [ 176.253579] niks3-server[1182]: 2026/09/25 15:15:40 INFO Received push request method=POST path=/api/pushes6052worker1 # [ 176.263347] nix-grpc-daemon[1578]: time=2026-09-25T15:15:40.851Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6053worker1 # [ 176.266515] nix-grpc-daemon[1578]: time=2026-09-25T15:15:40.851Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6054lb # [ 176.286632] niks3-server[1182]: 2026/09/25 15:15:40 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6055worker1 # [ 176.292226] nix-grpc-daemon[1578]: time=2026-09-25T15:15:40.882Z level=INFO msg="Uploading 1 narinfos"6056lb # [ 176.298664] niks3-server[1182]: 2026/09/25 15:15:40 INFO Signed narinfos id=55 count=16057lb # [ 176.307519] niks3-server[1182]: 2026/09/25 15:15:40 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6058lb # [ 176.316911] niks3-server[1182]: 2026/09/25 15:15:40 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6059lb # [ 176.319775] niks3-server[1182]: 2026/09/25 15:15:40 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6060lb # [ 176.330330] niks3-server[1182]: 2026/09/25 15:15:40 INFO Received complete push request method=POST path=/api/pushes/55/complete6061worker1 # [ 176.337430] nix-grpc-daemon[1578]: time=2026-09-25T15:15:40.927Z level=INFO msg="Upload complete. (259ms)"6062lb # [ 176.338039] niks3-server[1182]: 2026/09/25 15:15:40 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6063worker1 # [ 176.344717] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56224 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16064worker2 # [ 176.350262] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:40Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06065worker2 # [ 176.352751] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:40Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16066worker1 # [ 176.360664] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562246067worker1 # [ 176.364735] nix-daemon[837]: reaped child process 1654, status = succeeded6068worker1 # [ 176.374596] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562246069worker1 # [ 176.385860] nix-grpc-daemon[1578]: time=2026-09-25T15:15:40.976Z level=INFO msg="All 1 paths already cached"6070client # copying 1 paths to 'lb:50051'...6071worker1 # [ 176.394348] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562246072client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6073worker1 # [ 176.408252] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6074worker1 # [ 176.537575] nix-daemon[837]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6075worker1 # [ 176.560799] nix-daemon[837]: reaped child process 1682, status = succeeded6076lb # [ 176.563135] niks3-server[1182]: 2026/09/25 15:15:41 INFO Received push request method=POST path=/api/pushes6077worker1 # [ 176.569021] nix-grpc-daemon[1578]: time=2026-09-25T15:15:41.158Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6078worker1 # [ 176.570646] nix-grpc-daemon[1578]: time=2026-09-25T15:15:41.158Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6079lb # [ 176.584876] niks3-server[1182]: 2026/09/25 15:15:41 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6080lb # [ 176.592701] niks3-server[1182]: 2026/09/25 15:15:41 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6081worker1 # [ 176.596096] nix-grpc-daemon[1578]: time=2026-09-25T15:15:41.186Z level=INFO msg="Uploading 1 narinfos"6082lb # [ 176.601721] niks3-server[1182]: 2026/09/25 15:15:41 INFO Signed narinfos id=56 count=16083lb # [ 176.603674] niks3-server[1182]: 2026/09/25 15:15:41 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6084lb # [ 176.617698] niks3-server[1182]: 2026/09/25 15:15:41 INFO Received complete push request method=POST path=/api/pushes/56/complete6085worker1 # [ 176.622160] nix-grpc-daemon[1578]: time=2026-09-25T15:15:41.212Z level=INFO msg="Upload complete. (203ms)"6086lb # [ 176.624351] niks3-server[1182]: 2026/09/25 15:15:41 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6087worker1 # [ 176.630823] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56224 duration_s=0 paths=1 nar_bytes_in=6406088worker1 # [ 176.641967] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562246089worker1 # [ 176.647062] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6090worker1 # [ 176.651634] nix-daemon[837]: reaped child process 1677, status = succeeded6091client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6092worker1 # [ 176.661682] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6093client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6094client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6095client # farm-b> LOG-farm-b-during-drain6096worker1 # [ 178.128334] nix-daemon[837]: reaped child process 1686, status = succeeded6097worker1 # [ 178.209573] nix-daemon[837]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6098lb # [ 178.229464] niks3-server[1182]: 2026/09/25 15:15:42 INFO Received push request method=POST path=/api/pushes6099worker1 # [ 178.232140] nix-daemon[837]: reaped child process 1705, status = succeeded6100worker1 # [ 178.237738] nix-grpc-daemon[1578]: time=2026-09-25T15:15:42.827Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6101worker1 # [ 178.239286] nix-grpc-daemon[1578]: time=2026-09-25T15:15:42.827Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6102lb # [ 178.260384] niks3-server[1182]: 2026/09/25 15:15:42 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6103worker1 # [ 178.264076] nix-grpc-daemon[1578]: time=2026-09-25T15:15:42.854Z level=INFO msg="Uploading 1 narinfos"6104lb # [ 178.269824] niks3-server[1182]: 2026/09/25 15:15:42 INFO Signed narinfos id=57 count=16105lb # [ 178.275262] niks3-server[1182]: 2026/09/25 15:15:42 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6106lb # [ 178.282375] niks3-server[1182]: 2026/09/25 15:15:42 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6107lb # [ 178.289096] niks3-server[1182]: 2026/09/25 15:15:42 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6108lb # [ 178.298073] niks3-server[1182]: 2026/09/25 15:15:42 INFO Received complete push request method=POST path=/api/pushes/57/complete6109worker1 # [ 178.301032] nix-grpc-daemon[1578]: time=2026-09-25T15:15:42.891Z level=INFO msg="Upload complete. (227ms)"6110lb # [ 178.303920] niks3-server[1182]: 2026/09/25 15:15:42 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6111worker1 # [ 178.308633] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56224 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16112worker2 # [ 178.315337] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:42Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06113worker2 # [ 178.317612] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:42Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16114worker1 # [ 178.322796] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562246115worker1 # [ 178.327082] nix-daemon[837]: reaped child process 1684, status = succeeded6116worker1 # [ 178.336594] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562246117worker1 # [ 178.345954] nix-grpc-daemon[1578]: time=2026-09-25T15:15:42.936Z level=INFO msg="All 2 paths already cached"6118client # copying 1 paths to 'lb:50051'...6119worker1 # [ 178.353675] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562246120client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6121worker1 # [ 178.366650] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6122worker1 # [ 178.487853] nix-daemon[837]: accepted connection from pid 1709, user nix-grpc-daemon (trusted)6123lb # [ 178.508552] niks3-server[1182]: 2026/09/25 15:15:43 INFO Received push request method=POST path=/api/pushes6124worker1 # [ 178.511531] nix-daemon[837]: reaped child process 1712, status = succeeded6125worker1 # [ 178.514139] nix-grpc-daemon[1578]: time=2026-09-25T15:15:43.104Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6126worker1 # [ 178.515655] nix-grpc-daemon[1578]: time=2026-09-25T15:15:43.104Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6127worker1 # [ 178.531304] nix-grpc-daemon[1578]: time=2026-09-25T15:15:43.121Z level=INFO msg="Uploading 1 narinfos"6128lb # [ 178.525945] niks3-server[1182]: 2026/09/25 15:15:43 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6129lb # [ 178.534721] niks3-server[1182]: 2026/09/25 15:15:43 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6130lb # [ 178.540139] niks3-server[1182]: 2026/09/25 15:15:43 INFO Signed narinfos id=58 count=16131lb # [ 178.541924] niks3-server[1182]: 2026/09/25 15:15:43 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6132lb # [ 178.553378] niks3-server[1182]: 2026/09/25 15:15:43 INFO Received complete push request method=POST path=/api/pushes/58/complete6133lb # [ 178.559538] niks3-server[1182]: 2026/09/25 15:15:43 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6134worker1 # [ 178.563630] nix-grpc-daemon[1578]: time=2026-09-25T15:15:43.154Z level=INFO msg="Upload complete. (186ms)"6135worker1 # [ 178.570973] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56224 duration_s=0 paths=1 nar_bytes_in=7686136worker1 # [ 178.582107] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562246137worker1 # [ 178.587129] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6138worker1 # [ 178.591507] nix-daemon[837]: reaped child process 1707, status = succeeded6139client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6140worker1 # [ 178.601537] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6141client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6142client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6143client # farm-top> LOG-farm-top-during-drain6144worker1 # [ 179.685525] nix-daemon[837]: reaped child process 1716, status = succeeded6145worker1 # [ 179.741945] nix-daemon[837]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)6146worker1 # [ 179.761879] nix-daemon[837]: reaped child process 1735, status = succeeded6147lb # [ 179.769398] niks3-server[1182]: 2026/09/25 15:15:44 INFO Received push request method=POST path=/api/pushes6148worker1 # [ 179.774995] nix-grpc-daemon[1578]: time=2026-09-25T15:15:44.365Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6149worker1 # [ 179.778988] nix-grpc-daemon[1578]: time=2026-09-25T15:15:44.367Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6150lb # [ 179.800918] niks3-server[1182]: 2026/09/25 15:15:44 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6151worker1 # [ 179.809438] nix-grpc-daemon[1578]: time=2026-09-25T15:15:44.399Z level=INFO msg="Uploading 1 narinfos"6152lb # [ 179.810001] niks3-server[1182]: 2026/09/25 15:15:44 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6153lb # [ 179.816892] niks3-server[1182]: 2026/09/25 15:15:44 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6154lb # [ 179.823448] niks3-server[1182]: 2026/09/25 15:15:44 INFO Signed narinfos id=59 count=16155lb # [ 179.830731] niks3-server[1182]: 2026/09/25 15:15:44 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6156lb # [ 179.834970] niks3-server[1182]: 2026/09/25 15:15:44 INFO Received complete push request method=POST path=/api/pushes/59/complete6157worker1 # [ 179.837178] nix-grpc-daemon[1578]: time=2026-09-25T15:15:44.427Z level=INFO msg="Upload complete. (225ms)"6158lb # [ 179.838566] niks3-server[1182]: 2026/09/25 15:15:44 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6159worker1 # [ 179.844756] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56224 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16160worker1 # [ 179.856647] nix-daemon[837]: reaped child process 1714, status = succeeded6161client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.53 seconds)6162client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6163client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.14 seconds)6164worker2: must succeed: pkill -f 'read -t [9]0 x'6165worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6166client: waiting for success: ! systemctl is-active sw6167worker2 # [ 180.296889] nix-daemon[859]: reaped child process 1848, status = succeeded6168worker2 # [ 180.407495] nix-daemon[859]: accepted connection from pid 2022, user nix-grpc-daemon (trusted)6169lb # [ 180.426303] niks3-server[1182]: 2026/09/25 15:15:45 INFO Received push request method=POST path=/api/pushes6170worker2 # [ 180.431348] nix-daemon[859]: reaped child process 2025, status = succeeded6171worker2 # [ 180.434438] nix-grpc-daemon[1782]: time=2026-09-25T15:15:45.026Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6172worker2 # [ 180.436151] nix-grpc-daemon[1782]: time=2026-09-25T15:15:45.026Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6173lb # [ 180.449441] niks3-server[1182]: 2026/09/25 15:15:45 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6174worker2 # [ 180.454913] nix-grpc-daemon[1782]: time=2026-09-25T15:15:45.048Z level=INFO msg="Uploading 1 narinfos"6175lb # [ 180.457823] niks3-server[1182]: 2026/09/25 15:15:45 INFO Signed narinfos id=60 count=16176lb # [ 180.459574] niks3-server[1182]: 2026/09/25 15:15:45 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6177lb # [ 180.472068] niks3-server[1182]: 2026/09/25 15:15:45 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6178worker2 # [ 180.478300] nix-grpc-daemon[1782]: time=2026-09-25T15:15:45.071Z level=INFO msg="Upload complete. (215ms)"6179lb # [ 180.477554] niks3-server[1182]: 2026/09/25 15:15:45 INFO Received complete push request method=POST path=/api/pushes/60/complete6180lb # [ 180.483899] niks3-server[1182]: 2026/09/25 15:15:45 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6181lb # [ 180.488801] niks3-server[1182]: 2026/09/25 15:15:45 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6182worker2 # [ 180.488686] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44248 duration_s=12 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16183worker2 # [ 180.498626] nix-daemon[859]: reaped child process 1846, status = succeeded6184client # [ 180.506964] systemd[1]: sw.service: Deactivated successfully.6185client # [ 180.508297] systemd[1]: sw.service: Consumed 191ms CPU time over 14.103s wall clock time, 41.2M memory peak, 7.6K incoming IP traffic, 8.7K outgoing IP traffic.6186client: (finished: waiting for success: ! systemctl is-active sw, in 1.07 seconds)6187client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6188client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6189worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6190worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.03 seconds)6191lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6192lb: must succeed: curl -sf localhost:9901/clusters6193lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6194worker2 # [ 181.308681] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:45Z level=info event=signal_exit6195lb: must succeed: curl -sf localhost:9901/clusters6196lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6197lb # [ 182.304786] niks3-server[1182]: 2026/09/25 15:15:46 INFO lead: released remote=192.168.1.4:511326198lb: must succeed: curl -sf localhost:9901/clusters6199worker1 # [ 183.306205] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:47Z level=info event=scheduler_restarting addr=lb:500516200worker2 # [ 183.308660] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:47Z level=info event=scheduler_restarting6201lb # [ 183.307152] envoy[893]: [2026-09-25 15:15:47.893][893][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6202worker2 # [ 183.308956] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:47Z level=info event=scheduler_restarting addr=lb:500516203worker2 # [ 183.315028] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:442386204worker2 # [ 183.321523] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:47Z level=info event=worker_gone addr=192.168.1.4:500516205worker2 # [ 183.324358] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:442386206worker2 # [ 183.327204] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:47Z level=info event=worker_gone addr=192.168.1.3:500516207lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6208worker2 # [ 183.346086] nix-daemon[859]: reaped child process 1753, status = succeeded6209worker1 # [ 183.506255] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_connected addr=lb:500516210worker1 # [ 183.509678] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6211worker2 # [ 183.524945] nix-grpc-daemon[1699]: ts=2026-09-25T15:15:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6212worker1 # [ 183.710019] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_connected addr=lb:500516213worker1 # [ 183.713312] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6214worker1 # [ 183.913830] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_connected addr=lb:500516215worker1 # [ 183.917052] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6216worker1 # [ 184.117740] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_connected addr=lb:500516217worker1 # [ 184.120896] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6218worker1 # [ 184.322628] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_connected addr=lb:500516219worker1 # [ 184.325049] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6220lb: must succeed: curl -sf localhost:9901/clusters6221lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6222worker1 # [ 184.523323] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_connected addr=lb:500516223worker1 # [ 184.526615] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6224worker1 # [ 184.727227] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_connected addr=lb:500516225worker1 # [ 184.730548] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6226worker1 # [ 184.931203] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_connected addr=lb:500516227worker1 # [ 184.934449] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6228worker1 # [ 185.135098] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_connected addr=lb:500516229worker1 # [ 185.137494] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6230worker2 # [ 185.339338] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6231worker1 # [ 185.339066] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_connected addr=lb:500516232worker1 # [ 185.342286] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6233worker2 # [ 185.343972] systemd[1]: nix-grpc-daemon.service: Consumed 821ms CPU time over 30.286s wall clock time, 38.3M memory peak, 235.2K incoming IP traffic, 71K outgoing IP traffic.6234lb: must succeed: curl -sf localhost:9901/clusters6235lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6236worker1 # [ 185.542919] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_connected addr=lb:500516237worker1 # [ 185.546102] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6238worker2 # [ 185.575592] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6239worker2 # [ 185.583036] systemd[1]: Starting scheduler-token.service...6240worker2 # [ 185.638491] systemd[1]: scheduler-token.service: Deactivated successfully.6241worker2 # [ 185.641364] systemd[1]: Finished scheduler-token.service.6242worker2 # [ 185.646588] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 57ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6243worker2 # [ 185.661073] systemd[1]: Starting Nix worker-protocol over gRPC...6244worker1 # [ 185.746951] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_connected addr=lb:500516245worker1 # [ 185.750358] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6246worker2 # [ 185.770753] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6247worker2 # [ 185.773213] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6248worker2 # [ 185.777148] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16249worker2 # [ 185.783782] nix-grpc-daemon[2041]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6250lb # [ 185.791555] niks3-server[1182]: 2026/09/25 15:15:50 INFO lead: acquired remote=192.168.1.4:476106251worker2 # [ 185.800703] nix-grpc-daemon[2041]: E0000 00:00:1790349350.374996 2041 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6252worker2 # [ 185.803245] nix-grpc-daemon[2041]: E0000 00:00:1790349350.377099 2041 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6253worker2 # [ 185.833264] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6254worker2 # [ 185.835627] nix-grpc-daemon[2041]: E0000 00:00:1790349350.385792 2081 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6255worker2 # [ 185.841140] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=scheduler_take_over6256worker2 # [ 185.844509] nix-grpc-daemon[2041]: E0000 00:00:1790349350.406348 2081 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6257worker2 # [ 185.847025] nix-grpc-daemon[2041]: E0000 00:00:1790349350.406519 2081 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6258worker2 # [ 185.849624] nix-grpc-daemon[2041]: E0000 00:00:1790349350.425291 2081 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6259worker2 # [ 185.852194] systemd[1]: Started Nix worker-protocol over gRPC.6260worker2 # [ 185.854042] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6261worker1 # [ 185.951021] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_connected addr=lb:500516262worker1 # [ 185.954257] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6263worker2 # [ 186.045842] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=scheduler_connected addr=lb:500516264worker2 # [ 186.049437] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6265worker1 # [ 186.154769] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_connected addr=lb:500516266worker1 # [ 186.157872] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6267worker2 # [ 186.257612] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=scheduler_connected addr=lb:500516268worker2 # [ 186.261420] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6269worker1 # [ 186.359355] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_connected addr=lb:500516270worker1 # [ 186.362314] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6271lb: must succeed: curl -sf localhost:9901/clusters6272worker2 # [ 186.451733] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:51Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:355526273lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6274worker2 # [ 186.462191] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:51Z level=info event=scheduler_connected addr=lb:500516275worker2 # [ 186.473047] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:51Z level=info event=oidc_jwks provider=mock keys=16276worker2 # [ 186.475814] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:355666277worker2 # [ 186.479660] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06278worker1 # [ 186.563100] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:51Z level=info event=scheduler_connected addr=lb:500516279worker2 # [ 186.566421] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:355666280worker2 # [ 186.572163] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06281lb # [ 187.105187] envoy[893]: conn peer=[2001:db8:1::1]:40010 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9591 tx=4379 ms=53296282lb # [ 187.108050] envoy[893]: conn peer=[2001:db8:1::1]:34598 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3038 tx=1577 ms=139016283lb # [ 187.110663] envoy[893]: conn peer=[2001:db8:1::4]:60922 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1554 tx=378 ms=281676284lb: must succeed: curl -sf localhost:9901/clusters6285lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6286lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 6.30 seconds)6287lb: must succeed: curl -sf localhost:9901/clusters6288lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6289worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6290worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6291(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 21.21 seconds)6292subtest: queue gauges show a burst while every build blocks6293lb: must succeed: curl -sf localhost:9901/clusters6294lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6295lb: must succeed: curl -sf localhost:9901/clusters6296lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6297worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6298worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6299client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6300client # [ 187.738512] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6301client # Running as unit: burst.service; invocation ID: 0e2d4426d70e46a5be2e531ddc3348ca6302client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6303lb: must succeed: curl -sf localhost:9901/clusters6304lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6305lb: must succeed: curl -sf localhost:9901/clusters6306lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6307worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6308worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6309worker1 # [ 187.922904] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:609586310client # [ 187.929217] nix[1511]: don't know how to build these paths:6311client # [ 187.930193] nix[1511]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6312client # [ 187.931234] nix[1511]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6313client # [ 187.932965] nix[1511]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6314client # [ 187.934178] nix[1511]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6315worker2 # [ 187.943832] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:52Z level=debug event=want client=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06316worker1 # [ 187.942661] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586317worker2 # [ 187.950870] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:52Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.4:50051 clients=16318worker1 # [ 187.954188] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586319worker2 # [ 187.958394] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:52Z level=debug event=want client=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06320client # [ 187.963608] nix[1511]: copying 1 paths to 'lb:50051'...6321worker2 # [ 187.963840] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:52Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.3:50051 clients=16322worker1 # [ 187.963320] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586323client # [ 187.965122] nix[1511]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6324worker2 # [ 187.972223] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:52Z level=debug event=want client=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06325worker1 # [ 187.972618] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6326worker2 # [ 187.979910] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:52Z level=debug event=want client=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06327worker2 # [ 187.985080] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355806328worker2 # [ 187.991575] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6329worker2 # [ 187.995047] nix-grpc-daemon[2041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)6330worker2 # [ 188.041233] nix-grpc-daemon[2041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 2/5)6331worker1 # [ 188.102012] nix-daemon[837]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6332lb # [ 188.121886] niks3-server[1182]: 2026/09/25 15:15:52 INFO Received push request method=POST path=/api/pushes6333worker1 # [ 188.124143] nix-daemon[837]: reaped child process 1742, status = succeeded6334worker1 # [ 188.128728] nix-grpc-daemon[1578]: time=2026-09-25T15:15:52.718Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6335worker1 # [ 188.130291] nix-grpc-daemon[1578]: time=2026-09-25T15:15:52.718Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6336lb # [ 188.148217] niks3-server[1182]: 2026/09/25 15:15:52 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6337lb # [ 188.155266] niks3-server[1182]: 2026/09/25 15:15:52 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6338worker1 # [ 188.158574] nix-grpc-daemon[1578]: time=2026-09-25T15:15:52.748Z level=INFO msg="Uploading 1 narinfos"6339lb # [ 188.162609] niks3-server[1182]: 2026/09/25 15:15:52 INFO Signed narinfos id=61 count=16340lb # [ 188.166869] niks3-server[1182]: 2026/09/25 15:15:52 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6341lb # [ 188.178308] niks3-server[1182]: 2026/09/25 15:15:52 INFO Received complete push request method=POST path=/api/pushes/61/complete6342worker1 # [ 188.183405] nix-grpc-daemon[1578]: time=2026-09-25T15:15:52.773Z level=INFO msg="Upload complete. (201ms)"6343lb # [ 188.184646] niks3-server[1182]: 2026/09/25 15:15:52 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6344worker1 # [ 188.189949] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=4166345worker1 # [ 188.199758] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586346worker1 # [ 188.204171] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6347worker1 # [ 188.207570] nix-daemon[837]: reaped child process 1737, status = succeeded6348worker2 # [ 188.212038] nix-grpc-daemon[2041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 3/5)6349client # [ 188.215227] nix[1511]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6350worker1 # [ 188.216988] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6351worker2 # [ 188.376614] nix-grpc-daemon[2041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 338 ms (attempt 4/5)6352worker2 # [ 188.716769] nix-grpc-daemon[2041]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6353worker2 # [ 188.718988] nix-grpc-daemon[2041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6354worker2 # [ 188.732713] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355806355client # [ 188.740346] nix[1511]: copying 1 paths to 'lb:50051'...6356worker2 # [ 188.743505] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355806357client # [ 188.743322] nix[1511]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6358worker2 # [ 188.752234] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6359worker2 # [ 188.778419] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:53Z level=info event=niks3_push_started pid=21126360lb: must succeed: curl -sf localhost:9901/clusters6361worker2 # [ 188.902549] nix-grpc-daemon[2112]: time=2026-09-25T15:15:53.495Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6362lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6363lb: must succeed: curl -sf localhost:9901/clusters6364lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6365worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6366worker2 # [ 189.028139] nix-daemon[859]: accepted connection from pid 2124, user nix-grpc-daemon (trusted)6367lb # [ 189.056413] niks3-server[1182]: 2026/09/25 15:15:53 INFO Received push request method=POST path=/api/pushes6368worker2 # [ 189.067535] nix-grpc-daemon[2112]: time=2026-09-25T15:15:53.657Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6369worker2 # [ 189.074644] nix-grpc-daemon[2112]: time=2026-09-25T15:15:53.657Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6370worker2 # [ 189.077647] nix-daemon[859]: reaped child process 2130, status = succeeded6371worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.12 seconds)6372lb # [ 189.084983] niks3-server[1182]: 2026/09/25 15:15:53 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6373worker2 # [ 189.091721] nix-grpc-daemon[2112]: time=2026-09-25T15:15:53.684Z level=INFO msg="Uploading 1 narinfos"6374lb # [ 189.093217] niks3-server[1182]: 2026/09/25 15:15:53 INFO Signed narinfos id=62 count=16375lb # [ 189.095017] niks3-server[1182]: 2026/09/25 15:15:53 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6376lb # [ 189.108275] niks3-server[1182]: 2026/09/25 15:15:53 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6377lb # [ 189.110822] niks3-server[1182]: 2026/09/25 15:15:53 INFO Received complete push request method=POST path=/api/pushes/62/complete6378worker2 # [ 189.117912] nix-grpc-daemon[2112]: time=2026-09-25T15:15:53.711Z level=INFO msg="Upload complete. (211ms)"6379lb # [ 189.117210] niks3-server[1182]: 2026/09/25 15:15:53 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6380worker2 # [ 189.125251] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=0 paths=1 nar_bytes_in=4166381worker2 # [ 189.127710] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355806382worker2 # [ 189.136758] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6383worker2 # [ 189.142833] nix-daemon[859]: reaped child process 2110, status = succeeded6384client # [ 189.152956] nix[1511]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6385worker2 # [ 189.158148] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6386worker1 # [ 189.239011] nix-daemon[837]: reaped child process 1746, status = succeeded6387worker1 # [ 189.336106] nix-daemon[837]: accepted connection from pid 1767, user nix-grpc-daemon (trusted)6388lb # [ 189.358589] niks3-server[1182]: 2026/09/25 15:15:53 INFO Received push request method=POST path=/api/pushes6389worker1 # [ 189.359792] nix-daemon[837]: reaped child process 1770, status = succeeded6390worker1 # [ 189.364447] nix-grpc-daemon[1578]: time=2026-09-25T15:15:53.953Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6391worker1 # [ 189.365982] nix-grpc-daemon[1578]: time=2026-09-25T15:15:53.953Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6392lb # [ 189.386162] niks3-server[1182]: 2026/09/25 15:15:53 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6393lb # [ 189.395193] niks3-server[1182]: 2026/09/25 15:15:53 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6394lb # [ 189.401509] niks3-server[1182]: 2026/09/25 15:15:53 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6395worker1 # [ 189.405281] nix-grpc-daemon[1578]: time=2026-09-25T15:15:53.995Z level=INFO msg="Uploading 1 narinfos"6396lb # [ 189.408442] niks3-server[1182]: 2026/09/25 15:15:53 INFO Signed narinfos id=63 count=16397lb # [ 189.410317] niks3-server[1182]: 2026/09/25 15:15:53 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6398lb # [ 189.422181] niks3-server[1182]: 2026/09/25 15:15:54 INFO Received complete push request method=POST path=/api/pushes/63/complete6399worker1 # [ 189.426778] nix-grpc-daemon[1578]: time=2026-09-25T15:15:54.017Z level=INFO msg="Upload complete. (202ms)"6400lb # [ 189.429093] niks3-server[1182]: 2026/09/25 15:15:54 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6401worker2 # [ 189.438138] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:54Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.3:50051 clients=16402worker1 # [ 189.436848] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=2 outputs=16403worker1 # [ 189.443359] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586404worker1 # [ 189.451865] nix-daemon[837]: reaped child process 1744, status = succeeded6405worker1 # [ 189.457607] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586406client # [ 189.467038] nix[1511]: copying 1 paths to 'lb:50051'...6407worker1 # [ 189.466383] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586408client # [ 189.468483] nix[1511]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6409worker1 # [ 189.475447] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6410worker1 # [ 189.595533] nix-daemon[837]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6411lb # [ 189.618012] niks3-server[1182]: 2026/09/25 15:15:54 INFO Received push request method=POST path=/api/pushes6412worker1 # [ 189.619560] nix-daemon[837]: reaped child process 1777, status = succeeded6413worker1 # [ 189.623175] nix-grpc-daemon[1578]: time=2026-09-25T15:15:54.213Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6414worker1 # [ 189.624933] nix-grpc-daemon[1578]: time=2026-09-25T15:15:54.213Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6415lb # [ 189.637232] niks3-server[1182]: 2026/09/25 15:15:54 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6416lb # [ 189.646779] niks3-server[1182]: 2026/09/25 15:15:54 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6417lb # [ 189.656144] niks3-server[1182]: 2026/09/25 15:15:54 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6418lb # [ 189.665172] niks3-server[1182]: 2026/09/25 15:15:54 INFO Signed narinfos id=64 count=16419worker1 # [ 189.665029] nix-grpc-daemon[1578]: time=2026-09-25T15:15:54.255Z level=INFO msg="Uploading 1 narinfos"6420lb # [ 189.676211] niks3-server[1182]: 2026/09/25 15:15:54 INFO Received complete push request method=POST path=/api/pushes/64/complete6421lb # [ 189.682767] niks3-server[1182]: 2026/09/25 15:15:54 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6422worker1 # [ 189.695700] nix-grpc-daemon[1578]: time=2026-09-25T15:15:54.286Z level=INFO msg="Upload complete. (210ms)"6423worker1 # [ 189.702206] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=4166424worker1 # [ 189.714361] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586425worker1 # [ 189.719423] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6426worker1 # [ 189.723861] nix-daemon[837]: reaped child process 1772, status = succeeded6427client # [ 189.733074] nix[1511]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6428worker1 # [ 189.734044] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6429worker2 # [ 190.397902] nix-daemon[859]: reaped child process 2137, status = succeeded6430worker2 # [ 190.497379] nix-daemon[859]: accepted connection from pid 2159, user nix-grpc-daemon (trusted)6431worker2 # [ 190.520201] nix-daemon[859]: reaped child process 2162, status = succeeded6432lb # [ 190.516819] niks3-server[1182]: 2026/09/25 15:15:55 INFO Received push request method=POST path=/api/pushes6433worker2 # [ 190.527554] nix-grpc-daemon[2112]: time=2026-09-25T15:15:55.120Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6434worker2 # [ 190.532469] nix-grpc-daemon[2112]: time=2026-09-25T15:15:55.120Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6435lb # [ 190.550594] niks3-server[1182]: 2026/09/25 15:15:55 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6436worker2 # [ 190.555425] nix-grpc-daemon[2112]: time=2026-09-25T15:15:55.148Z level=INFO msg="Uploading 1 narinfos"6437lb # [ 190.556173] niks3-server[1182]: 2026/09/25 15:15:55 INFO Signed narinfos id=65 count=16438lb # [ 190.557556] niks3-server[1182]: 2026/09/25 15:15:55 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6439lb # [ 190.567348] niks3-server[1182]: 2026/09/25 15:15:55 INFO Received complete push request method=POST path=/api/pushes/65/complete6440worker2 # [ 190.572768] nix-grpc-daemon[2112]: time=2026-09-25T15:15:55.166Z level=INFO msg="Upload complete. (201ms)"6441lb # [ 190.573321] niks3-server[1182]: 2026/09/25 15:15:55 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6442lb # [ 190.579109] niks3-server[1182]: 2026/09/25 15:15:55 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6443worker2 # [ 190.581229] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=1 outputs=16444lb # [ 190.583579] niks3-server[1182]: 2026/09/25 15:15:55 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6445worker2 # [ 190.592111] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:55Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.4:50051 clients=16446worker2 # [ 190.595123] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355806447worker2 # [ 190.604757] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355806448worker2 # [ 190.608801] nix-daemon[859]: reaped child process 2132, status = succeeded6449client # [ 190.611345] nix[1511]: copying 1 paths to 'lb:50051'...6450worker2 # [ 190.615623] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355806451client # [ 190.617404] nix[1511]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6452worker2 # [ 190.627789] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6453worker2 # [ 190.745534] nix-daemon[859]: accepted connection from pid 2166, user nix-grpc-daemon (trusted)6454lb # [ 190.764953] niks3-server[1182]: 2026/09/25 15:15:55 INFO Received push request method=POST path=/api/pushes6455worker2 # [ 190.769719] nix-daemon[859]: reaped child process 2169, status = succeeded6456worker2 # [ 190.773160] nix-grpc-daemon[2112]: time=2026-09-25T15:15:55.366Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6457worker2 # [ 190.774846] nix-grpc-daemon[2112]: time=2026-09-25T15:15:55.366Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6458lb # [ 190.788775] niks3-server[1182]: 2026/09/25 15:15:55 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6459lb # [ 190.799341] niks3-server[1182]: 2026/09/25 15:15:55 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6460worker2 # [ 190.804450] nix-grpc-daemon[2112]: time=2026-09-25T15:15:55.397Z level=INFO msg="Uploading 1 narinfos"6461lb # [ 190.807080] niks3-server[1182]: 2026/09/25 15:15:55 INFO Signed narinfos id=66 count=16462lb # [ 190.811272] niks3-server[1182]: 2026/09/25 15:15:55 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6463lb # [ 190.821475] niks3-server[1182]: 2026/09/25 15:15:55 INFO Received complete push request method=POST path=/api/pushes/66/complete6464worker2 # [ 190.829141] nix-grpc-daemon[2112]: time=2026-09-25T15:15:55.422Z level=INFO msg="Upload complete. (191ms)"6465worker2 # [ 190.833978] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=0 paths=1 nar_bytes_in=4166466lb # [ 190.832976] niks3-server[1182]: 2026/09/25 15:15:55 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6467worker2 # [ 190.840607] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355806468worker2 # [ 190.847626] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6469worker2 # [ 190.851162] nix-daemon[859]: reaped child process 2164, status = succeeded6470client # [ 190.856869] nix[1511]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6471worker2 # [ 190.860993] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6472worker1 # [ 191.621950] nix-daemon[837]: reaped child process 1781, status = succeeded6473worker1 # [ 191.686653] nix-daemon[837]: accepted connection from pid 1806, user nix-grpc-daemon (trusted)6474worker1 # [ 191.708312] nix-daemon[837]: reaped child process 1809, status = succeeded6475lb # [ 191.706377] niks3-server[1182]: 2026/09/25 15:15:56 INFO Received push request method=POST path=/api/pushes6476worker1 # [ 191.712515] nix-grpc-daemon[1578]: time=2026-09-25T15:15:56.301Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6477worker1 # [ 191.717234] nix-grpc-daemon[1578]: time=2026-09-25T15:15:56.301Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6478lb # [ 191.734810] niks3-server[1182]: 2026/09/25 15:15:56 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6479worker1 # [ 191.739398] nix-grpc-daemon[1578]: time=2026-09-25T15:15:56.329Z level=INFO msg="Uploading 1 narinfos"6480lb # [ 191.744189] niks3-server[1182]: 2026/09/25 15:15:56 INFO Signed narinfos id=67 count=16481lb # [ 191.746074] niks3-server[1182]: 2026/09/25 15:15:56 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6482lb # [ 191.754755] niks3-server[1182]: 2026/09/25 15:15:56 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6483lb # [ 191.763447] niks3-server[1182]: 2026/09/25 15:15:56 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6484worker1 # [ 191.767363] nix-grpc-daemon[1578]: time=2026-09-25T15:15:56.357Z level=INFO msg="Upload complete. (221ms)"6485lb # [ 191.769436] niks3-server[1182]: 2026/09/25 15:15:56 INFO Received complete push request method=POST path=/api/pushes/67/complete6486lb # [ 191.773213] niks3-server[1182]: 2026/09/25 15:15:56 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6487worker1 # [ 191.776801] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=3 outputs=16488worker1 # [ 191.786916] nix-daemon[837]: reaped child process 1779, status = succeeded6489worker2 # [ 192.692681] nix-daemon[859]: reaped child process 2173, status = succeeded6490worker2 # [ 192.788310] nix-daemon[859]: accepted connection from pid 2198, user nix-grpc-daemon (trusted)6491lb # [ 192.810382] niks3-server[1182]: 2026/09/25 15:15:57 INFO Received push request method=POST path=/api/pushes6492worker2 # [ 192.814506] nix-daemon[859]: reaped child process 2201, status = succeeded6493worker2 # [ 192.819395] nix-grpc-daemon[2112]: time=2026-09-25T15:15:57.411Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6494worker2 # [ 192.821274] nix-grpc-daemon[2112]: time=2026-09-25T15:15:57.411Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6495worker2 # [ 192.838128] nix-grpc-daemon[2112]: time=2026-09-25T15:15:57.431Z level=INFO msg="Uploading 1 narinfos"6496lb # [ 192.837049] niks3-server[1182]: 2026/09/25 15:15:57 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6497lb # [ 192.840269] niks3-server[1182]: 2026/09/25 15:15:57 INFO Signed narinfos id=68 count=16498lb # [ 192.846676] niks3-server[1182]: 2026/09/25 15:15:57 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6499lb # [ 192.852670] niks3-server[1182]: 2026/09/25 15:15:57 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6500lb # [ 192.862275] niks3-server[1182]: 2026/09/25 15:15:57 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6501worker2 # [ 192.870691] nix-grpc-daemon[2112]: time=2026-09-25T15:15:57.464Z level=INFO msg="Upload complete. (201ms)"6502lb # [ 192.870069] niks3-server[1182]: 2026/09/25 15:15:57 INFO Received complete push request method=POST path=/api/pushes/68/complete6503worker2 # [ 192.876083] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=4 outputs=16504lb # [ 192.874268] niks3-server[1182]: 2026/09/25 15:15:57 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6505worker2 # [ 192.886363] nix-daemon[859]: reaped child process 2171, status = succeeded6506client # [ 192.904306] systemd[1]: burst.service: Deactivated successfully.6507client # [ 192.907849] systemd[1]: burst.service: Consumed 201ms CPU time over 5.168s wall clock time, 55.6M memory peak, 13.6K incoming IP traffic, 18.4K outgoing IP traffic.6508client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6509client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.05 seconds)6510lb: must succeed: curl -sf localhost:9901/clusters6511lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6512lb: must succeed: curl -sf localhost:9901/clusters6513lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6514worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6515worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6516(finished: subtest: queue gauges show a burst while every build blocks, in 6.38 seconds)6517subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6518client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16519worker2 # [ 194.144401] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355806520worker2 # [ 194.151797] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355806521worker2 # [ 194.180651] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:58Z level=debug event=want client=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06522worker2 # [ 194.183814] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:58Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16523worker2 # [ 194.187676] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:58Z level=debug event=want client=2 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06524worker2 # [ 194.190990] nix-grpc-daemon[2041]: ts=2026-09-25T15:15:58Z level=debug event=want client=2 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06525worker1 # [ 194.188829] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586526worker1 # [ 194.199149] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586527worker1 # [ 194.208145] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586528worker1 # [ 194.217203] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6529worker1 # [ 194.347183] nix-daemon[837]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6530worker1 # [ 194.365031] nix-daemon[837]: reaped child process 1824, status = succeeded6531lb # [ 194.372393] niks3-server[1182]: 2026/09/25 15:15:58 INFO Received push request method=POST path=/api/pushes6532worker1 # [ 194.378219] nix-grpc-daemon[1578]: time=2026-09-25T15:15:58.967Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6533worker1 # [ 194.380474] nix-grpc-daemon[1578]: time=2026-09-25T15:15:58.967Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6534lb # [ 194.389419] niks3-server[1182]: 2026/09/25 15:15:58 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6535worker1 # [ 194.393707] nix-grpc-daemon[1578]: time=2026-09-25T15:15:58.983Z level=INFO msg="Uploading 1 narinfos"6536lb # [ 194.396878] niks3-server[1182]: 2026/09/25 15:15:58 INFO Signed narinfos id=69 count=16537lb # [ 194.400846] niks3-server[1182]: 2026/09/25 15:15:58 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6538lb # [ 194.407755] niks3-server[1182]: 2026/09/25 15:15:58 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6539lb # [ 194.416679] niks3-server[1182]: 2026/09/25 15:15:58 INFO Received complete push request method=POST path=/api/pushes/69/complete6540worker1 # [ 194.422199] nix-grpc-daemon[1578]: time=2026-09-25T15:15:59.012Z level=INFO msg="Upload complete. (195ms)"6541lb # [ 194.422635] niks3-server[1182]: 2026/09/25 15:15:59 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6542worker1 # [ 194.428417] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=5126543worker1 # [ 194.436935] nix-grpc-daemon[1486]: ts=2026-09-25T15:15:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586544worker1 # [ 194.450729] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6545worker1 # [ 194.452774] nix-daemon[837]: reaped child process 1819, status = succeeded6546worker1 # [ 194.465789] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6547worker1 # [ 195.519459] nix-daemon[837]: reaped child process 1828, status = succeeded6548worker1 # [ 195.622387] nix-daemon[837]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6549lb # [ 195.643479] niks3-server[1182]: 2026/09/25 15:16:00 INFO Received push request method=POST path=/api/pushes6550worker1 # [ 195.645320] nix-daemon[837]: reaped child process 1847, status = succeeded6551worker1 # [ 195.650866] nix-grpc-daemon[1578]: time=2026-09-25T15:16:00.239Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6552worker1 # [ 195.652885] nix-grpc-daemon[1578]: time=2026-09-25T15:16:00.239Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6553lb # [ 195.673650] niks3-server[1182]: 2026/09/25 15:16:00 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6554worker1 # [ 195.681097] nix-grpc-daemon[1578]: time=2026-09-25T15:16:00.271Z level=INFO msg="Uploading 1 narinfos"6555lb # [ 195.683108] niks3-server[1182]: 2026/09/25 15:16:00 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6556lb # [ 195.690988] niks3-server[1182]: 2026/09/25 15:16:00 INFO Signed narinfos id=70 count=16557lb # [ 195.697128] niks3-server[1182]: 2026/09/25 15:16:00 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6558lb # [ 195.699993] niks3-server[1182]: 2026/09/25 15:16:00 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6559worker1 # [ 195.709966] nix-grpc-daemon[1578]: time=2026-09-25T15:16:00.300Z level=INFO msg="Upload complete. (234ms)"6560lb # [ 195.711407] niks3-server[1182]: 2026/09/25 15:16:00 INFO Received complete push request method=POST path=/api/pushes/70/complete6561lb # [ 195.713887] niks3-server[1182]: 2026/09/25 15:16:00 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6562worker2 # [ 195.720643] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:00Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16563worker1 # [ 195.719536] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=5 outputs=16564worker2 # [ 195.728591] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:00Z level=debug event=want client=2 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06565worker1 # [ 195.732740] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586566worker1 # [ 195.737088] nix-daemon[837]: reaped child process 1826, status = succeeded6567worker1 # [ 195.742911] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586568worker1 # [ 195.752152] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586569worker1 # [ 195.761002] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6570worker1 # [ 195.888857] nix-daemon[837]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6571lb # [ 195.908646] niks3-server[1182]: 2026/09/25 15:16:00 INFO Received push request method=POST path=/api/pushes6572worker1 # [ 195.911854] nix-daemon[837]: reaped child process 1854, status = succeeded6573worker1 # [ 195.914951] nix-grpc-daemon[1578]: time=2026-09-25T15:16:00.504Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6574worker1 # [ 195.916749] nix-grpc-daemon[1578]: time=2026-09-25T15:16:00.504Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6575lb # [ 195.929158] niks3-server[1182]: 2026/09/25 15:16:00 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6576worker1 # [ 195.932576] nix-grpc-daemon[1578]: time=2026-09-25T15:16:00.522Z level=INFO msg="Uploading 1 narinfos"6577lb # [ 195.938826] niks3-server[1182]: 2026/09/25 15:16:00 INFO Signed narinfos id=71 count=16578lb # [ 195.943830] niks3-server[1182]: 2026/09/25 15:16:00 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6579lb # [ 195.955431] niks3-server[1182]: 2026/09/25 15:16:00 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6580lb # [ 195.959413] niks3-server[1182]: 2026/09/25 15:16:00 INFO Received complete push request method=POST path=/api/pushes/71/complete6581worker1 # [ 195.964212] nix-grpc-daemon[1578]: time=2026-09-25T15:16:00.554Z level=INFO msg="Upload complete. (192ms)"6582lb # [ 195.966590] niks3-server[1182]: 2026/09/25 15:16:00 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6583worker1 # [ 195.970408] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=5126584worker1 # [ 195.982088] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586585worker1 # [ 195.987757] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6586worker1 # [ 195.992762] nix-daemon[837]: reaped child process 1849, status = succeeded6587worker1 # [ 196.001901] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6588lb # [ 197.105308] envoy[893]: conn peer=[2001:db8:1::1]:45334 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9628 tx=5189 ms=49706589worker1 # [ 197.245615] nix-daemon[837]: reaped child process 1858, status = succeeded6590worker1 # [ 197.345378] nix-daemon[837]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6591worker1 # [ 197.365830] nix-daemon[837]: reaped child process 1877, status = succeeded6592lb # [ 197.372598] niks3-server[1182]: 2026/09/25 15:16:01 INFO Received push request method=POST path=/api/pushes6593worker1 # [ 197.380520] nix-grpc-daemon[1578]: time=2026-09-25T15:16:01.968Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6594worker1 # [ 197.382138] nix-grpc-daemon[1578]: time=2026-09-25T15:16:01.968Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6595lb # [ 197.403579] niks3-server[1182]: 2026/09/25 15:16:01 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6596worker1 # [ 197.406528] nix-grpc-daemon[1578]: time=2026-09-25T15:16:01.996Z level=INFO msg="Uploading 1 narinfos"6597lb # [ 197.413801] niks3-server[1182]: 2026/09/25 15:16:01 INFO Signed narinfos id=72 count=16598lb # [ 197.415614] niks3-server[1182]: 2026/09/25 15:16:02 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6599lb # [ 197.427849] niks3-server[1182]: 2026/09/25 15:16:02 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6600lb # [ 197.433573] niks3-server[1182]: 2026/09/25 15:16:02 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6601worker1 # [ 197.436422] nix-grpc-daemon[1578]: time=2026-09-25T15:16:02.026Z level=INFO msg="Upload complete. (223ms)"6602lb # [ 197.437996] niks3-server[1182]: 2026/09/25 15:16:02 INFO Received complete push request method=POST path=/api/pushes/72/complete6603lb # [ 197.443147] niks3-server[1182]: 2026/09/25 15:16:02 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6604worker2 # [ 197.444910] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:02Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16605worker1 # [ 197.444686] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=6 outputs=16606worker2 # [ 197.454958] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:02Z level=debug event=want client=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06607worker1 # [ 197.454454] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586608worker1 # [ 197.464497] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586609worker1 # [ 197.468501] nix-daemon[837]: reaped child process 1856, status = succeeded6610worker1 # [ 197.474175] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586611worker1 # [ 197.482217] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6612worker1 # [ 197.607658] nix-daemon[837]: accepted connection from pid 1881, user nix-grpc-daemon (trusted)6613lb # [ 197.627977] niks3-server[1182]: 2026/09/25 15:16:02 INFO Received push request method=POST path=/api/pushes6614worker1 # [ 197.629438] nix-daemon[837]: reaped child process 1884, status = succeeded6615worker1 # [ 197.633341] nix-grpc-daemon[1578]: time=2026-09-25T15:16:02.223Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6616worker1 # [ 197.634933] nix-grpc-daemon[1578]: time=2026-09-25T15:16:02.223Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6617lb # [ 197.649307] niks3-server[1182]: 2026/09/25 15:16:02 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6618worker1 # [ 197.654160] nix-grpc-daemon[1578]: time=2026-09-25T15:16:02.244Z level=INFO msg="Uploading 1 narinfos"6619lb # [ 197.658085] niks3-server[1182]: 2026/09/25 15:16:02 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign6620lb # [ 197.664455] niks3-server[1182]: 2026/09/25 15:16:02 INFO Signed narinfos id=73 count=16621lb # [ 197.666382] niks3-server[1182]: 2026/09/25 15:16:02 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6622lb # [ 197.676848] niks3-server[1182]: 2026/09/25 15:16:02 INFO Received complete push request method=POST path=/api/pushes/73/complete6623worker1 # [ 197.681192] nix-grpc-daemon[1578]: time=2026-09-25T15:16:02.271Z level=INFO msg="Upload complete. (189ms)"6624lb # [ 197.682524] niks3-server[1182]: 2026/09/25 15:16:02 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6625worker1 # [ 197.686727] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=5126626worker1 # [ 197.700228] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586627worker1 # [ 197.705038] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6628worker1 # [ 197.709505] nix-daemon[837]: reaped child process 1879, status = succeeded6629worker1 # [ 197.718273] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6630worker1 # [ 198.560439] nix-daemon[837]: reaped child process 1888, status = succeeded6631worker1 # [ 198.665423] nix-daemon[837]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)6632lb # [ 198.686662] niks3-server[1182]: 2026/09/25 15:16:03 INFO Received push request method=POST path=/api/pushes6633worker1 # [ 198.687749] nix-daemon[837]: reaped child process 1907, status = succeeded6634worker1 # [ 198.692469] nix-grpc-daemon[1578]: time=2026-09-25T15:16:03.282Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6635worker1 # [ 198.694017] nix-grpc-daemon[1578]: time=2026-09-25T15:16:03.282Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6636lb # [ 198.712884] niks3-server[1182]: 2026/09/25 15:16:03 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6637lb # [ 198.722535] niks3-server[1182]: 2026/09/25 15:16:03 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6638worker1 # [ 198.728748] nix-grpc-daemon[1578]: time=2026-09-25T15:16:03.318Z level=INFO msg="Uploading 1 narinfos"6639lb # [ 198.729512] niks3-server[1182]: 2026/09/25 15:16:03 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign6640lb # [ 198.736162] niks3-server[1182]: 2026/09/25 15:16:03 INFO Signed narinfos id=74 count=16641lb # [ 198.741944] niks3-server[1182]: 2026/09/25 15:16:03 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6642lb # [ 198.750892] niks3-server[1182]: 2026/09/25 15:16:03 INFO Received complete push request method=POST path=/api/pushes/74/complete6643worker1 # [ 198.758787] nix-grpc-daemon[1578]: time=2026-09-25T15:16:03.349Z level=INFO msg="Upload complete. (229ms)"6644lb # [ 198.758867] niks3-server[1182]: 2026/09/25 15:16:03 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6645worker2 # [ 198.766996] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:03Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16646worker1 # [ 198.765633] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=7 outputs=16647worker2 # [ 198.774853] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:03Z level=debug event=want client=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06648worker1 # [ 198.777555] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586649worker1 # [ 198.781003] nix-daemon[837]: reaped child process 1886, status = succeeded6650worker1 # [ 198.786949] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586651worker1 # [ 198.834559] nix-grpc-daemon[1578]: time=2026-09-25T15:16:03.424Z level=INFO msg="All 1 paths already cached"6652worker1 # [ 198.842213] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586653worker1 # [ 198.850925] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6654worker1 # [ 198.980571] nix-daemon[837]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6655lb # [ 199.000763] niks3-server[1182]: 2026/09/25 15:16:03 INFO Received push request method=POST path=/api/pushes6656worker1 # [ 199.003165] nix-daemon[837]: reaped child process 1914, status = succeeded6657worker1 # [ 199.006988] nix-grpc-daemon[1578]: time=2026-09-25T15:16:03.596Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6658worker1 # [ 199.008944] nix-grpc-daemon[1578]: time=2026-09-25T15:16:03.597Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6659lb # [ 199.022088] niks3-server[1182]: 2026/09/25 15:16:03 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign6660worker1 # [ 199.024720] nix-grpc-daemon[1578]: time=2026-09-25T15:16:03.614Z level=INFO msg="Uploading 1 narinfos"6661lb # [ 199.028385] niks3-server[1182]: 2026/09/25 15:16:03 INFO Signed narinfos id=75 count=16662lb # [ 199.033176] niks3-server[1182]: 2026/09/25 15:16:03 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6663lb # [ 199.035264] niks3-server[1182]: 2026/09/25 15:16:03 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6664lb # [ 199.043110] niks3-server[1182]: 2026/09/25 15:16:03 INFO Received complete push request method=POST path=/api/pushes/75/complete6665worker1 # [ 199.047681] nix-grpc-daemon[1578]: time=2026-09-25T15:16:03.638Z level=INFO msg="Upload complete. (186ms)"6666lb # [ 199.048509] niks3-server[1182]: 2026/09/25 15:16:03 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6667worker1 # [ 199.054718] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=6246668worker1 # [ 199.060288] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586669worker1 # [ 199.069561] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6670worker1 # [ 199.073705] nix-daemon[837]: reaped child process 1909, status = succeeded6671worker1 # [ 199.082034] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6672worker1 # [ 200.321946] nix-daemon[837]: reaped child process 1918, status = succeeded6673worker1 # [ 200.432917] nix-daemon[837]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)6674worker1 # [ 200.452917] nix-daemon[837]: reaped child process 1937, status = succeeded6675lb # [ 200.460231] niks3-server[1182]: 2026/09/25 15:16:05 INFO Received push request method=POST path=/api/pushes6676worker1 # [ 200.470159] nix-grpc-daemon[1578]: time=2026-09-25T15:16:05.058Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6677worker1 # [ 200.471706] nix-grpc-daemon[1578]: time=2026-09-25T15:16:05.058Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6678lb # [ 200.495736] niks3-server[1182]: 2026/09/25 15:16:05 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6679lb # [ 200.502599] niks3-server[1182]: 2026/09/25 15:16:05 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6680worker1 # [ 200.508511] nix-grpc-daemon[1578]: time=2026-09-25T15:16:05.098Z level=INFO msg="Uploading 1 narinfos"6681lb # [ 200.512210] niks3-server[1182]: 2026/09/25 15:16:05 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign6682lb # [ 200.514530] niks3-server[1182]: 2026/09/25 15:16:05 INFO Signed narinfos id=76 count=16683lb # [ 200.521685] niks3-server[1182]: 2026/09/25 15:16:05 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6684lb # [ 200.531499] niks3-server[1182]: 2026/09/25 15:16:05 INFO Received complete push request method=POST path=/api/pushes/76/complete6685worker1 # [ 200.534258] nix-grpc-daemon[1578]: time=2026-09-25T15:16:05.124Z level=INFO msg="Upload complete. (242ms)"6686lb # [ 200.537618] niks3-server[1182]: 2026/09/25 15:16:05 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6687worker1 # [ 200.540752] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=8 outputs=16688worker2 # [ 200.546180] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:05Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16689worker2 # [ 200.552410] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:05Z level=debug event=want client=2 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06690worker1 # [ 200.552397] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586691worker1 # [ 200.561657] nix-daemon[837]: reaped child process 1916, status = succeeded6692worker1 # [ 200.564507] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586693worker1 # [ 200.574253] nix-grpc-daemon[1578]: time=2026-09-25T15:16:05.164Z level=INFO msg="All 1 paths already cached"6694worker1 # [ 200.580234] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586695worker1 # [ 200.590379] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6696worker1 # [ 200.712145] nix-daemon[837]: accepted connection from pid 1941, user nix-grpc-daemon (trusted)6697lb # [ 200.733908] niks3-server[1182]: 2026/09/25 15:16:05 INFO Received push request method=POST path=/api/pushes6698worker1 # [ 200.736274] nix-daemon[837]: reaped child process 1944, status = succeeded6699worker1 # [ 200.738794] nix-grpc-daemon[1578]: time=2026-09-25T15:16:05.328Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6700worker1 # [ 200.740522] nix-grpc-daemon[1578]: time=2026-09-25T15:16:05.328Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6701lb # [ 200.752789] niks3-server[1182]: 2026/09/25 15:16:05 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6702worker1 # [ 200.757007] nix-grpc-daemon[1578]: time=2026-09-25T15:16:05.347Z level=INFO msg="Uploading 1 narinfos"6703lb # [ 200.761704] niks3-server[1182]: 2026/09/25 15:16:05 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign6704lb # [ 200.766909] niks3-server[1182]: 2026/09/25 15:16:05 INFO Signed narinfos id=77 count=16705worker1 # [ 200.777586] nix-grpc-daemon[1578]: time=2026-09-25T15:16:05.367Z level=INFO msg="Upload complete. (177ms)"6706lb # [ 200.773629] niks3-server[1182]: 2026/09/25 15:16:05 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6707lb # [ 200.780624] niks3-server[1182]: 2026/09/25 15:16:05 INFO Received complete push request method=POST path=/api/pushes/77/complete6708lb # [ 200.783951] niks3-server[1182]: 2026/09/25 15:16:05 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6709worker1 # [ 200.783650] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=6246710worker1 # [ 200.793386] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586711worker1 # [ 200.798365] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6712worker1 # [ 200.802320] nix-daemon[837]: reaped child process 1939, status = succeeded6713worker1 # [ 200.810335] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6714worker1 # [ 201.664573] nix-daemon[837]: reaped child process 1948, status = succeeded6715worker1 # [ 201.758740] nix-daemon[837]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6716lb # [ 201.777100] niks3-server[1182]: 2026/09/25 15:16:06 INFO Received push request method=POST path=/api/pushes6717worker1 # [ 201.780452] nix-daemon[837]: reaped child process 1967, status = succeeded6718worker1 # [ 201.783857] nix-grpc-daemon[1578]: time=2026-09-25T15:16:06.372Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6719worker1 # [ 201.785691] nix-grpc-daemon[1578]: time=2026-09-25T15:16:06.372Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6720lb # [ 201.797782] niks3-server[1182]: 2026/09/25 15:16:06 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6721lb # [ 201.810169] niks3-server[1182]: 2026/09/25 15:16:06 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6722worker1 # [ 201.816636] nix-grpc-daemon[1578]: time=2026-09-25T15:16:06.406Z level=INFO msg="Uploading 1 narinfos"6723lb # [ 201.816825] niks3-server[1182]: 2026/09/25 15:16:06 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign6724lb # [ 201.823517] niks3-server[1182]: 2026/09/25 15:16:06 INFO Signed narinfos id=78 count=16725lb # [ 201.828117] niks3-server[1182]: 2026/09/25 15:16:06 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6726lb # [ 201.837872] niks3-server[1182]: 2026/09/25 15:16:06 INFO Received complete push request method=POST path=/api/pushes/78/complete6727worker1 # [ 201.841605] nix-grpc-daemon[1578]: time=2026-09-25T15:16:06.431Z level=INFO msg="Upload complete. (222ms)"6728lb # [ 201.843184] niks3-server[1182]: 2026/09/25 15:16:06 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6729worker2 # [ 201.852272] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:06Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16730worker2 # [ 201.856532] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:06Z level=debug event=want client=2 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06731worker1 # [ 201.848812] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=9 outputs=16732worker1 # [ 201.863557] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586733worker1 # [ 201.867818] nix-daemon[837]: reaped child process 1946, status = succeeded6734worker1 # [ 201.873459] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586735worker1 # [ 201.881652] nix-grpc-daemon[1578]: time=2026-09-25T15:16:06.472Z level=INFO msg="All 1 paths already cached"6736worker1 # [ 201.887542] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586737worker1 # [ 201.895759] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6738worker1 # [ 202.016966] nix-daemon[837]: accepted connection from pid 1971, user nix-grpc-daemon (trusted)6739lb # [ 202.039354] niks3-server[1182]: 2026/09/25 15:16:06 INFO Received push request method=POST path=/api/pushes6740worker1 # [ 202.041631] nix-daemon[837]: reaped child process 1974, status = succeeded6741worker1 # [ 202.043857] nix-grpc-daemon[1578]: time=2026-09-25T15:16:06.634Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6742worker1 # [ 202.046151] nix-grpc-daemon[1578]: time=2026-09-25T15:16:06.636Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6743lb # [ 202.057128] niks3-server[1182]: 2026/09/25 15:16:06 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign6744worker1 # [ 202.059639] nix-grpc-daemon[1578]: time=2026-09-25T15:16:06.649Z level=INFO msg="Uploading 1 narinfos"6745lb # [ 202.063126] niks3-server[1182]: 2026/09/25 15:16:06 INFO Signed narinfos id=79 count=16746lb # [ 202.069740] niks3-server[1182]: 2026/09/25 15:16:06 INFO Received complete push request method=POST path=/api/pushes/79/complete6747worker1 # [ 202.076255] nix-grpc-daemon[1578]: time=2026-09-25T15:16:06.666Z level=INFO msg="Upload complete. (171ms)"6748lb # [ 202.077376] niks3-server[1182]: 2026/09/25 15:16:06 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6749worker1 # [ 202.083067] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=6246750lb # [ 202.083888] niks3-server[1182]: 2026/09/25 15:16:06 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6751lb # [ 202.087348] niks3-server[1182]: 2026/09/25 15:16:06 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6752worker1 # [ 202.088204] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586753worker1 # [ 202.095541] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6754worker1 # [ 202.099200] nix-daemon[837]: reaped child process 1969, status = succeeded6755worker1 # [ 202.108156] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6756worker1 # [ 203.388518] nix-daemon[837]: reaped child process 1978, status = succeeded6757worker1 # [ 203.456669] nix-daemon[837]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6758worker1 # [ 203.473679] nix-daemon[837]: reaped child process 1997, status = succeeded6759lb # [ 203.480745] niks3-server[1182]: 2026/09/25 15:16:08 INFO Received push request method=POST path=/api/pushes6760worker1 # [ 203.488561] nix-grpc-daemon[1578]: time=2026-09-25T15:16:08.075Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6761worker1 # [ 203.490080] nix-grpc-daemon[1578]: time=2026-09-25T15:16:08.076Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6762lb # [ 203.504558] niks3-server[1182]: 2026/09/25 15:16:08 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6763worker1 # [ 203.516913] nix-grpc-daemon[1578]: time=2026-09-25T15:16:08.107Z level=INFO msg="Uploading 1 narinfos"6764lb # [ 203.517689] niks3-server[1182]: 2026/09/25 15:16:08 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign6765lb # [ 203.522524] niks3-server[1182]: 2026/09/25 15:16:08 INFO Signed narinfos id=80 count=16766lb # [ 203.527858] niks3-server[1182]: 2026/09/25 15:16:08 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6767lb # [ 203.535230] niks3-server[1182]: 2026/09/25 15:16:08 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6768lb # [ 203.543504] niks3-server[1182]: 2026/09/25 15:16:08 INFO Received complete push request method=POST path=/api/pushes/80/complete6769worker1 # [ 203.547871] nix-grpc-daemon[1578]: time=2026-09-25T15:16:08.138Z level=INFO msg="Upload complete. (232ms)"6770lb # [ 203.550212] niks3-server[1182]: 2026/09/25 15:16:08 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6771worker2 # [ 203.557056] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:08Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16772worker1 # [ 203.555797] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=10 outputs=16773worker2 # [ 203.564525] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:08Z level=debug event=want client=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06774worker1 # [ 203.569112] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586775worker1 # [ 203.573228] nix-daemon[837]: reaped child process 1976, status = succeeded6776worker1 # [ 203.578704] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586777worker1 # [ 203.586658] nix-grpc-daemon[1578]: time=2026-09-25T15:16:08.177Z level=INFO msg="All 2 paths already cached"6778worker1 # [ 203.592708] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586779worker1 # [ 203.601216] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6780worker1 # [ 203.720428] nix-daemon[837]: accepted connection from pid 2001, user nix-grpc-daemon (trusted)6781lb # [ 203.741324] niks3-server[1182]: 2026/09/25 15:16:08 INFO Received push request method=POST path=/api/pushes6782worker1 # [ 203.743648] nix-daemon[837]: reaped child process 2004, status = succeeded6783worker1 # [ 203.746166] nix-grpc-daemon[1578]: time=2026-09-25T15:16:08.336Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6784worker1 # [ 203.747720] nix-grpc-daemon[1578]: time=2026-09-25T15:16:08.336Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6785lb # [ 203.758763] niks3-server[1182]: 2026/09/25 15:16:08 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6786worker1 # [ 203.765505] nix-grpc-daemon[1578]: time=2026-09-25T15:16:08.355Z level=INFO msg="Uploading 1 narinfos"6787lb # [ 203.767684] niks3-server[1182]: 2026/09/25 15:16:08 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign6788lb # [ 203.773777] niks3-server[1182]: 2026/09/25 15:16:08 INFO Signed narinfos id=81 count=16789lb # [ 203.775548] niks3-server[1182]: 2026/09/25 15:16:08 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6790worker1 # [ 203.785081] nix-grpc-daemon[1578]: time=2026-09-25T15:16:08.375Z level=INFO msg="Upload complete. (174ms)"6791lb # [ 203.786076] niks3-server[1182]: 2026/09/25 15:16:08 INFO Received complete push request method=POST path=/api/pushes/81/complete6792lb # [ 203.789544] niks3-server[1182]: 2026/09/25 15:16:08 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6793worker1 # [ 203.792181] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=7526794worker1 # [ 203.801709] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586795worker1 # [ 203.805612] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6796worker1 # [ 203.809414] nix-daemon[837]: reaped child process 1999, status = succeeded6797worker1 # [ 203.817024] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6798worker1 # [ 205.452407] nix-daemon[837]: reaped child process 2008, status = succeeded6799worker1 # [ 205.566144] nix-daemon[837]: accepted connection from pid 2024, user nix-grpc-daemon (trusted)6800worker1 # [ 205.586602] nix-daemon[837]: reaped child process 2027, status = succeeded6801lb # [ 205.593163] niks3-server[1182]: 2026/09/25 15:16:10 INFO Received push request method=POST path=/api/pushes6802worker1 # [ 205.601309] nix-grpc-daemon[1578]: time=2026-09-25T15:16:10.190Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6803worker1 # [ 205.602908] nix-grpc-daemon[1578]: time=2026-09-25T15:16:10.190Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6804lb # [ 205.621067] niks3-server[1182]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6805worker1 # [ 205.625692] nix-grpc-daemon[1578]: time=2026-09-25T15:16:10.215Z level=INFO msg="Uploading 1 narinfos"6806lb # [ 205.629110] niks3-server[1182]: 2026/09/25 15:16:10 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign6807lb # [ 205.631570] niks3-server[1182]: 2026/09/25 15:16:10 INFO Signed narinfos id=82 count=16808worker1 # [ 205.646120] nix-grpc-daemon[1578]: time=2026-09-25T15:16:10.236Z level=INFO msg="Upload complete. (219ms)"6809lb # [ 205.642632] niks3-server[1182]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6810lb # [ 205.649621] niks3-server[1182]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6811lb # [ 205.654125] niks3-server[1182]: 2026/09/25 15:16:10 INFO Received complete push request method=POST path=/api/pushes/82/complete6812worker1 # [ 205.652422] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=11 outputs=16813worker2 # [ 205.657068] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:10Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16814lb # [ 205.657539] niks3-server[1182]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6815worker1 # [ 205.667133] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586816worker1 # [ 205.671497] nix-daemon[837]: reaped child process 2006, status = succeeded6817worker1 # [ 205.677248] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586818worker1 # [ 205.685930] nix-grpc-daemon[1578]: time=2026-09-25T15:16:10.276Z level=INFO msg="All 2 paths already cached"6819worker1 # [ 205.692006] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586820worker1 # [ 205.700794] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6821worker1 # [ 205.820121] nix-daemon[837]: accepted connection from pid 2031, user nix-grpc-daemon (trusted)6822worker1 # [ 205.837092] nix-daemon[837]: reaped child process 2034, status = succeeded6823lb # [ 205.843556] niks3-server[1182]: 2026/09/25 15:16:10 INFO Received push request method=POST path=/api/pushes6824worker1 # [ 205.848200] nix-grpc-daemon[1578]: time=2026-09-25T15:16:10.438Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6825worker1 # [ 205.849797] nix-grpc-daemon[1578]: time=2026-09-25T15:16:10.438Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6826lb # [ 205.865023] niks3-server[1182]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6827lb # [ 205.869067] niks3-server[1182]: 2026/09/25 15:16:10 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign6828worker1 # [ 205.872156] nix-grpc-daemon[1578]: time=2026-09-25T15:16:10.462Z level=INFO msg="Uploading 1 narinfos"6829lb # [ 205.876650] niks3-server[1182]: 2026/09/25 15:16:10 INFO Signed narinfos id=83 count=16830lb # [ 205.878406] niks3-server[1182]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6831lb # [ 205.889418] niks3-server[1182]: 2026/09/25 15:16:10 INFO Received complete push request method=POST path=/api/pushes/83/complete6832worker1 # [ 205.893335] nix-grpc-daemon[1578]: time=2026-09-25T15:16:10.483Z level=INFO msg="Upload complete. (183ms)"6833lb # [ 205.895113] niks3-server[1182]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6834worker1 # [ 205.900219] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=7526835worker1 # [ 205.910732] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586836worker1 # [ 205.915603] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6837worker1 # [ 205.919923] nix-daemon[837]: reaped child process 2029, status = succeeded6838worker1 # [ 205.928600] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6839worker1 # [ 207.157642] nix-daemon[837]: reaped child process 2038, status = succeeded6840worker1 # [ 207.255202] nix-daemon[837]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)6841worker1 # [ 207.276167] nix-daemon[837]: reaped child process 2057, status = succeeded6842lb # [ 207.281648] niks3-server[1182]: 2026/09/25 15:16:11 INFO Received push request method=POST path=/api/pushes6843worker1 # [ 207.288508] nix-grpc-daemon[1578]: time=2026-09-25T15:16:11.877Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6844worker1 # [ 207.292093] nix-grpc-daemon[1578]: time=2026-09-25T15:16:11.877Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6845lb # [ 207.310713] niks3-server[1182]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6846worker1 # [ 207.319038] nix-grpc-daemon[1578]: time=2026-09-25T15:16:11.909Z level=INFO msg="Uploading 1 narinfos"6847lb # [ 207.319779] niks3-server[1182]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6848lb # [ 207.326596] niks3-server[1182]: 2026/09/25 15:16:11 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign6849lb # [ 207.332180] niks3-server[1182]: 2026/09/25 15:16:11 INFO Signed narinfos id=84 count=16850lb # [ 207.334116] niks3-server[1182]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6851worker1 # [ 207.344063] nix-grpc-daemon[1578]: time=2026-09-25T15:16:11.934Z level=INFO msg="Upload complete. (206ms)"6852lb # [ 207.343534] niks3-server[1182]: 2026/09/25 15:16:11 INFO Received complete push request method=POST path=/api/pushes/84/complete6853lb # [ 207.347914] niks3-server[1182]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6854worker2 # [ 207.355027] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:11Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16855worker1 # [ 207.353450] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=12 outputs=16856worker1 # [ 207.364234] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586857worker1 # [ 207.370872] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609586858worker1 # [ 207.374807] nix-daemon[837]: reaped child process 2036, status = succeeded6859worker1 # [ 207.379502] nix-grpc-daemon[1578]: time=2026-09-25T15:16:11.969Z level=INFO msg="All 2 paths already cached"6860worker1 # [ 207.385223] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609586861worker1 # [ 207.393036] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6862worker1 # [ 207.508202] nix-daemon[837]: accepted connection from pid 2061, user nix-grpc-daemon (trusted)6863lb # [ 207.525892] niks3-server[1182]: 2026/09/25 15:16:12 INFO Received push request method=POST path=/api/pushes6864worker1 # [ 207.529480] nix-daemon[837]: reaped child process 2064, status = succeeded6865worker1 # [ 207.531437] nix-grpc-daemon[1578]: time=2026-09-25T15:16:12.121Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6866worker1 # [ 207.533095] nix-grpc-daemon[1578]: time=2026-09-25T15:16:12.121Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6867lb # [ 207.544494] niks3-server[1182]: 2026/09/25 15:16:12 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6868worker1 # [ 207.550548] nix-grpc-daemon[1578]: time=2026-09-25T15:16:12.140Z level=INFO msg="Uploading 1 narinfos"6869lb # [ 207.553721] niks3-server[1182]: 2026/09/25 15:16:12 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign6870lb # [ 207.562089] niks3-server[1182]: 2026/09/25 15:16:12 INFO Signed narinfos id=85 count=16871lb # [ 207.563901] niks3-server[1182]: 2026/09/25 15:16:12 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6872lb # [ 207.568550] niks3-server[1182]: 2026/09/25 15:16:12 INFO Received complete push request method=POST path=/api/pushes/85/complete6873worker1 # [ 207.571183] nix-grpc-daemon[1578]: time=2026-09-25T15:16:12.161Z level=INFO msg="Upload complete. (169ms)"6874lb # [ 207.573116] niks3-server[1182]: 2026/09/25 15:16:12 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6875worker1 # [ 207.576178] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=0 paths=1 nar_bytes_in=7526876worker1 # [ 207.586325] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609586877worker1 # [ 207.590594] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6878worker1 # [ 207.594264] nix-daemon[837]: reaped child process 2059, status = succeeded6879worker1 # [ 207.602140] nix-daemon[837]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6880worker1 # [ 208.675052] nix-daemon[837]: reaped child process 2068, status = succeeded6881worker1 # [ 208.737118] nix-daemon[837]: accepted connection from pid 2084, user nix-grpc-daemon (trusted)6882worker1 # [ 208.757508] nix-daemon[837]: reaped child process 2087, status = succeeded6883lb # [ 208.762411] niks3-server[1182]: 2026/09/25 15:16:13 INFO Received push request method=POST path=/api/pushes6884worker1 # [ 208.768567] nix-grpc-daemon[1578]: time=2026-09-25T15:16:13.357Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6885worker1 # [ 208.770128] nix-grpc-daemon[1578]: time=2026-09-25T15:16:13.357Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6886lb # [ 208.789933] niks3-server[1182]: 2026/09/25 15:16:13 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6887lb # [ 208.796667] niks3-server[1182]: 2026/09/25 15:16:13 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign6888lb # [ 208.799193] niks3-server[1182]: 2026/09/25 15:16:13 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6889worker1 # [ 208.803569] nix-grpc-daemon[1578]: time=2026-09-25T15:16:13.393Z level=INFO msg="Uploading 1 narinfos"6890lb # [ 208.809623] niks3-server[1182]: 2026/09/25 15:16:13 INFO Signed narinfos id=86 count=16891lb # [ 208.815521] niks3-server[1182]: 2026/09/25 15:16:13 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6892worker1 # [ 208.820505] nix-grpc-daemon[1578]: time=2026-09-25T15:16:13.410Z level=INFO msg="Upload complete. (209ms)"6893lb # [ 208.821699] niks3-server[1182]: 2026/09/25 15:16:13 INFO Received complete push request method=POST path=/api/pushes/86/complete6894lb # [ 208.824627] niks3-server[1182]: 2026/09/25 15:16:13 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6895worker1 # [ 208.827627] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60958 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=13 outputs=16896worker1 # [ 208.839356] nix-daemon[837]: reaped child process 2066, status = succeeded6897client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.90 seconds)6898worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6899worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)6900client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6901client # [ 208.939413] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6902client # Running as unit: gpu.service; invocation ID: ceef9113b2384aaf9f2f9a4dee870c496903client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.05 seconds)6904lb: must succeed: curl -sf localhost:9901/clusters6905lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6906lb: must succeed: curl -sf localhost:9901/clusters6907lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6908worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6909worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6910worker2 # [ 209.110980] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355806911client # [ 209.113900] nix[1581]: don't know how to build these paths:6912client # [ 209.115503] nix[1581]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6913worker2 # [ 209.118412] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355806914worker2 # [ 209.133465] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:13Z level=debug event=want client=3 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06915client # [ 209.132207] nix[1581]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6916lb: must succeed: curl -sf localhost:9901/clusters6917lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6918lb: must succeed: curl -sf localhost:9901/clusters6919lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6920worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6921worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6922client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6923lb # [ 217.108108] envoy[893]: conn peer=[2001:db8:1::1]:55998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27114 tx=12336 ms=147046924client # [ 239.133375] nix[1581]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6925client # [ 239.143007] nix[1581]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6926client # [ 239.161567] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6927client # [ 239.165537] systemd[1]: gpu.service: Failed with result 'exit-code'.6928client # [ 239.167524] systemd[1]: gpu.service: Consumed 166ms CPU time over 30.225s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.6929client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.24 seconds)6930client: must succeed: journalctl -u gpu -o cat6931client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)6932lb: must succeed: curl -sf localhost:9901/clusters6933lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6934lb: must succeed: curl -sf localhost:9901/clusters6935lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6936worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6937worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6938(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.60 seconds)6939subtest: low disk drains a worker and builds go to the other6940worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6941worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.17 seconds)6942worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6943worker1 # [ 241.049060] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:45Z level=info event=unhealthy reason=min_free6944worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.07 seconds)6945client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26946client # don't know how to build these paths:6947client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6948worker2 # [ 241.983545] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355806949worker2 # [ 241.988085] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355806950worker2 # [ 242.003445] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:46Z level=debug event=want client=4 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06951worker2 # [ 242.008834] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:46Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16952worker2 # [ 242.013551] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355806953worker2 # [ 242.017606] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355806954client # copying 1 paths to 'lb:50051'...6955client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6956worker2 # [ 242.024838] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355806957worker2 # [ 242.033436] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6958worker2 # [ 242.159034] nix-daemon[859]: accepted connection from pid 2239, user nix-grpc-daemon (trusted)6959worker2 # [ 242.179739] nix-daemon[859]: reaped child process 2242, status = succeeded6960lb # [ 242.176286] niks3-server[1182]: 2026/09/25 15:16:46 INFO Received push request method=POST path=/api/pushes6961worker2 # [ 242.185912] nix-grpc-daemon[2112]: time=2026-09-25T15:16:46.778Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6962worker2 # [ 242.187528] nix-grpc-daemon[2112]: time=2026-09-25T15:16:46.778Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6963lb # [ 242.195455] niks3-server[1182]: 2026/09/25 15:16:46 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign6964worker2 # [ 242.203139] nix-grpc-daemon[2112]: time=2026-09-25T15:16:46.796Z level=INFO msg="Uploading 1 narinfos"6965lb # [ 242.203904] niks3-server[1182]: 2026/09/25 15:16:46 INFO Signed narinfos id=87 count=16966lb # [ 242.210112] niks3-server[1182]: 2026/09/25 15:16:46 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6967lb # [ 242.216179] niks3-server[1182]: 2026/09/25 15:16:46 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6968lb # [ 242.225879] niks3-server[1182]: 2026/09/25 15:16:46 INFO Received complete push request method=POST path=/api/pushes/87/complete6969lb # [ 242.231544] niks3-server[1182]: 2026/09/25 15:16:46 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6970worker2 # [ 242.232949] nix-grpc-daemon[2112]: time=2026-09-25T15:16:46.826Z level=INFO msg="Upload complete. (189ms)"6971worker2 # [ 242.240503] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=0 paths=1 nar_bytes_in=5126972worker2 # [ 242.243734] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355806973worker2 # [ 242.254311] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6974worker2 # [ 242.263533] nix-daemon[859]: reaped child process 2237, status = succeeded6975client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6976worker2 # [ 242.267386] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6977client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6978client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6979client # farm-a> LOG-farm-a-drain6980worker2 # [ 242.901060] nix-daemon[859]: reaped child process 2246, status = succeeded6981worker2 # [ 242.990988] nix-daemon[859]: accepted connection from pid 2262, user nix-grpc-daemon (trusted)6982worker2 # [ 243.009284] nix-daemon[859]: reaped child process 2265, status = succeeded6983lb # [ 243.013741] niks3-server[1182]: 2026/09/25 15:16:47 INFO Received push request method=POST path=/api/pushes6984worker2 # [ 243.022334] nix-grpc-daemon[2112]: time=2026-09-25T15:16:47.614Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6985worker2 # [ 243.023942] nix-grpc-daemon[2112]: time=2026-09-25T15:16:47.614Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6986lb # [ 243.031977] niks3-server[1182]: 2026/09/25 15:16:47 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6987lb # [ 243.042753] niks3-server[1182]: 2026/09/25 15:16:47 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6988worker2 # [ 243.051523] nix-grpc-daemon[2112]: time=2026-09-25T15:16:47.644Z level=INFO msg="Uploading 1 narinfos"6989lb # [ 243.050308] niks3-server[1182]: 2026/09/25 15:16:47 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign6990lb # [ 243.056535] niks3-server[1182]: 2026/09/25 15:16:47 INFO Signed narinfos id=88 count=16991lb # [ 243.058382] niks3-server[1182]: 2026/09/25 15:16:47 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6992lb # [ 243.068891] niks3-server[1182]: 2026/09/25 15:16:47 INFO Received complete push request method=POST path=/api/pushes/88/complete6993worker2 # [ 243.073615] nix-grpc-daemon[2112]: time=2026-09-25T15:16:47.667Z level=INFO msg="Upload complete. (196ms)"6994lb # [ 243.073124] niks3-server[1182]: 2026/09/25 15:16:47 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6995worker2 # [ 243.085346] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=0 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=14 outputs=16996worker2 # [ 243.095692] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:47Z level=debug event=want client=4 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06997worker2 # [ 243.098477] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:47Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16998worker2 # [ 243.102386] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355806999worker2 # [ 243.106608] nix-daemon[859]: reaped child process 2244, status = succeeded7000worker2 # [ 243.109096] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355807001worker2 # [ 243.114583] nix-grpc-daemon[2112]: time=2026-09-25T15:16:47.708Z level=INFO msg="All 1 paths already cached"7002client # copying 1 paths to 'lb:50051'...7003client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7004worker2 # [ 243.120525] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355807005worker2 # [ 243.128585] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)7006worker2 # [ 243.246156] nix-daemon[859]: accepted connection from pid 2271, user nix-grpc-daemon (trusted)7007lb # [ 243.264681] niks3-server[1182]: 2026/09/25 15:16:47 INFO Received push request method=POST path=/api/pushes7008worker2 # [ 243.267971] nix-daemon[859]: reaped child process 2274, status = succeeded7009worker2 # [ 243.272488] nix-grpc-daemon[2112]: time=2026-09-25T15:16:47.865Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7010worker2 # [ 243.274634] nix-grpc-daemon[2112]: time=2026-09-25T15:16:47.867Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7011lb # [ 243.285775] niks3-server[1182]: 2026/09/25 15:16:47 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7012worker2 # [ 243.292884] nix-grpc-daemon[2112]: time=2026-09-25T15:16:47.886Z level=INFO msg="Uploading 1 narinfos"7013lb # [ 243.294125] niks3-server[1182]: 2026/09/25 15:16:47 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7014lb # [ 243.300842] niks3-server[1182]: 2026/09/25 15:16:47 INFO Signed narinfos id=89 count=17015lb # [ 243.302640] niks3-server[1182]: 2026/09/25 15:16:47 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7016lb # [ 243.314197] niks3-server[1182]: 2026/09/25 15:16:47 INFO Received complete push request method=POST path=/api/pushes/89/complete7017worker2 # [ 243.320165] nix-grpc-daemon[2112]: time=2026-09-25T15:16:47.913Z level=INFO msg="Upload complete. (183ms)"7018lb # [ 243.318539] niks3-server[1182]: 2026/09/25 15:16:47 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7019worker2 # [ 243.326449] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=0 paths=1 nar_bytes_in=6327020worker2 # [ 243.332619] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355807021worker2 # [ 243.342008] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)7022worker2 # [ 243.345930] nix-daemon[859]: reaped child process 2269, status = succeeded7023client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7024client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7025worker2 # [ 243.356690] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)7026client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7027client # farm-b> LOG-farm-b-drain7028worker2 # [ 244.441255] nix-daemon[859]: reaped child process 2278, status = succeeded7029worker2 # [ 244.505367] nix-daemon[859]: accepted connection from pid 2294, user nix-grpc-daemon (trusted)7030lb # [ 244.522783] niks3-server[1182]: 2026/09/25 15:16:49 INFO Received push request method=POST path=/api/pushes7031worker2 # [ 244.526763] nix-daemon[859]: reaped child process 2297, status = succeeded7032worker2 # [ 244.532430] nix-grpc-daemon[2112]: time=2026-09-25T15:16:49.123Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7033worker2 # [ 244.534955] nix-grpc-daemon[2112]: time=2026-09-25T15:16:49.123Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7034lb # [ 244.549508] niks3-server[1182]: 2026/09/25 15:16:49 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7035worker2 # [ 244.558400] nix-grpc-daemon[2112]: time=2026-09-25T15:16:49.151Z level=INFO msg="Uploading 1 narinfos"7036lb # [ 244.558871] niks3-server[1182]: 2026/09/25 15:16:49 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7037lb # [ 244.564911] niks3-server[1182]: 2026/09/25 15:16:49 INFO Signed narinfos id=90 count=17038lb # [ 244.566674] niks3-server[1182]: 2026/09/25 15:16:49 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7039lb # [ 244.577956] niks3-server[1182]: 2026/09/25 15:16:49 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7040lb # [ 244.581348] niks3-server[1182]: 2026/09/25 15:16:49 INFO Received complete push request method=POST path=/api/pushes/90/complete7041lb # [ 244.603019] niks3-server[1182]: 2026/09/25 15:16:49 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7042worker2 # [ 244.607859] nix-grpc-daemon[2112]: time=2026-09-25T15:16:49.201Z level=INFO msg="Upload complete. (242ms)"7043worker2 # [ 244.618752] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=15 outputs=17044worker2 # [ 244.631806] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:49Z level=debug event=want client=4 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07045worker2 # [ 244.635195] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:49Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17046worker2 # [ 244.639618] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355807047worker2 # [ 244.643992] nix-daemon[859]: reaped child process 2276, status = succeeded7048worker2 # [ 244.646918] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355807049worker2 # [ 244.654155] nix-grpc-daemon[2112]: time=2026-09-25T15:16:49.247Z level=INFO msg="All 2 paths already cached"7050client # copying 1 paths to 'lb:50051'...7051client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7052worker2 # [ 244.660520] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355807053worker2 # [ 244.668411] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)7054worker2 # [ 244.784888] nix-daemon[859]: accepted connection from pid 2303, user nix-grpc-daemon (trusted)7055lb # [ 244.803453] niks3-server[1182]: 2026/09/25 15:16:49 INFO Received push request method=POST path=/api/pushes7056worker2 # [ 244.808946] nix-daemon[859]: reaped child process 2306, status = succeeded7057worker2 # [ 244.811322] nix-grpc-daemon[2112]: time=2026-09-25T15:16:49.404Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7058worker2 # [ 244.813023] nix-grpc-daemon[2112]: time=2026-09-25T15:16:49.404Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7059lb # [ 244.824162] niks3-server[1182]: 2026/09/25 15:16:49 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7060worker2 # [ 244.831022] nix-grpc-daemon[2112]: time=2026-09-25T15:16:49.424Z level=INFO msg="Uploading 1 narinfos"7061lb # [ 244.833550] niks3-server[1182]: 2026/09/25 15:16:49 INFO Signed narinfos id=91 count=17062lb # [ 244.835503] niks3-server[1182]: 2026/09/25 15:16:49 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7063lb # [ 244.847039] niks3-server[1182]: 2026/09/25 15:16:49 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7064worker2 # [ 244.854963] nix-grpc-daemon[2112]: time=2026-09-25T15:16:49.448Z level=INFO msg="Upload complete. (176ms)"7065lb # [ 244.855472] niks3-server[1182]: 2026/09/25 15:16:49 INFO Received complete push request method=POST path=/api/pushes/91/complete7066lb # [ 244.859036] niks3-server[1182]: 2026/09/25 15:16:49 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7067worker2 # [ 244.860235] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=0 paths=1 nar_bytes_in=7607068worker2 # [ 244.868126] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355807069worker2 # [ 244.877828] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)7070worker2 # [ 244.881806] nix-daemon[859]: reaped child process 2301, status = succeeded7071client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7072worker2 # [ 244.890719] nix-daemon[859]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)7073client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7074client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7075client # farm-top> LOG-farm-top-drain7076worker2 # [ 246.579321] nix-daemon[859]: reaped child process 2310, status = succeeded7077worker2 # [ 246.643351] nix-daemon[859]: accepted connection from pid 2326, user nix-grpc-daemon (trusted)7078worker2 # [ 246.662805] nix-daemon[859]: reaped child process 2329, status = succeeded7079lb # [ 246.666789] niks3-server[1182]: 2026/09/25 15:16:51 INFO Received push request method=POST path=/api/pushes7080worker2 # [ 246.676485] nix-grpc-daemon[2112]: time=2026-09-25T15:16:51.268Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7081worker2 # [ 246.678304] nix-grpc-daemon[2112]: time=2026-09-25T15:16:51.268Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7082lb # [ 246.697206] niks3-server[1182]: 2026/09/25 15:16:51 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7083lb # [ 246.705345] niks3-server[1182]: 2026/09/25 15:16:51 INFO Received sign narinfos request method=POST path=/api/pushes/92/sign7084worker2 # [ 246.711746] nix-grpc-daemon[2112]: time=2026-09-25T15:16:51.304Z level=INFO msg="Uploading 1 narinfos"7085lb # [ 246.713016] niks3-server[1182]: 2026/09/25 15:16:51 INFO Signed narinfos id=92 count=17086lb # [ 246.718454] niks3-server[1182]: 2026/09/25 15:16:51 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7087lb # [ 246.729889] niks3-server[1182]: 2026/09/25 15:16:51 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7088worker2 # [ 246.735894] nix-grpc-daemon[2112]: time=2026-09-25T15:16:51.329Z level=INFO msg="Upload complete. (234ms)"7089lb # [ 246.736294] niks3-server[1182]: 2026/09/25 15:16:51 INFO Received complete push request method=POST path=/api/pushes/92/complete7090lb # [ 246.740757] niks3-server[1182]: 2026/09/25 15:16:51 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7091worker2 # [ 246.744713] nix-grpc-daemon[2041]: ts=2026-09-25T15:16:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35580 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=16 outputs=17092worker2 # [ 246.756421] nix-daemon[859]: reaped child process 2308, status = succeeded7093client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 4.97 seconds)7094client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7095client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.15 seconds)7096worker1: must succeed: rm /nix/.rw-store/fill7097worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7098worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7099lb # [ 247.116258] envoy[893]: conn peer=[2001:db8:1::1]:57586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=612 ms=300417100lb # [ 247.119010] envoy[893]: conn peer=[2001:db8:1::1]:41362 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9515 tx=4370 ms=47787101worker1 # [ 248.057926] nix-grpc-daemon[1486]: ts=2026-09-25T15:16:52Z level=info event=healthy reason=min_free7102worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7103(finished: subtest: low disk drains a worker and builds go to the other, in 8.59 seconds)7104(finished: run the VM test script, in 249.02 seconds)7105test script finished in 249.15s7106cleanup7107kill QemuMachine (pid 50)7108client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7109client # [2026-09-25T15:16:52Z INFO virtiofsd] Client disconnected, shutting down7110client # [2026-09-25T15:16:52Z INFO virtiofsd] Client disconnected, shutting down7111client # [2026-09-25T15:16:52Z INFO virtiofsd] Client disconnected, shutting down7112kill QemuMachine (pid 49)7113lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7114lb # [2026-09-25T15:16:52Z INFO virtiofsd] Client disconnected, shutting down7115lb # [2026-09-25T15:16:52Z INFO virtiofsd] Client disconnected, shutting down7116lb # [2026-09-25T15:16:52Z INFO virtiofsd] Client disconnected, shutting down7117kill QemuMachine (pid 51)7118worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7119worker1 # [2026-09-25T15:16:53Z INFO virtiofsd] Client disconnected, shutting down7120worker1 # [2026-09-25T15:16:53Z INFO virtiofsd] Client disconnected, shutting down7121worker1 # [2026-09-25T15:16:53Z INFO virtiofsd] Client disconnected, shutting down7122kill QemuMachine (pid 52)7123worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7124worker2 # [2026-09-25T15:16:53Z INFO virtiofsd] Client disconnected, shutting down7125worker2 # [2026-09-25T15:16:53Z INFO virtiofsd] Client disconnected, shutting down7126worker2 # [2026-09-25T15:16:53Z INFO virtiofsd] Client disconnected, shutting down7127(finished: cleanup, in 1.02 seconds)7128additionally exposed symbols:7129 client, lb, worker1, worker2,7130 vlan1,7131 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh