vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #422
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14client # Formatting '/build/vm-state-client/tmp.XFUQ8476ZM', fmt=raw size=107374182415client: QEMU running (pid 50)16worker2 # Disk image does not exist, creating the virtualisation disk image...17lb: QEMU running (pid 49)18client # mke2fs 1.47.4 (6-Mar-2025)19lb # Formatting '/build/vm-state-lb/tmp.T53YupQsVH', fmt=raw size=107374182420worker2 # Formatting '/build/vm-state-worker2/tmp.KWtr1qWyZJ', fmt=raw size=107374182421worker1: QEMU running (pid 51)22worker1 # Disk image does not exist, creating the virtualisation disk image...23worker2: QEMU running (pid 52)24client # Discarding device blocks: 0/262144 done25worker1 # Formatting '/build/vm-state-worker1/tmp.syXu35wcHq', fmt=raw size=107374182426lb # mke2fs 1.47.4 (6-Mar-2025)27worker2 # mke2fs 1.47.4 (6-Mar-2025)28client # Creating filesystem with 262144 4k blocks and 65536 inodes29lb # Discarding device blocks: 0/262144 done30worker2 # Discarding device blocks: 0/262144 done31client # Filesystem UUID: 1ee65b1f-83cb-4ae9-a5b4-a341fcdd71bc32lb # Creating filesystem with 262144 4k blocks and 65536 inodes33worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes34client # Superblock backups stored on blocks:35lb # Filesystem UUID: 8f61555e-e117-4c0c-8ac4-974b6e3e3b5d36worker2 # Filesystem UUID: 8b839fd3-a006-40a0-91e8-0df79ffa623037client # 32768, 98304, 163840, 22937638lb # Superblock backups stored on blocks:39worker2 # Superblock backups stored on blocks:40client # 41lb # 32768, 98304, 163840, 22937642worker2 # 32768, 98304, 163840, 22937643client # Allocating group tables: 0/8 done44lb # 45worker2 # 46client # Writing inode tables: 0/8 done47lb # Allocating group tables: 0/8 done48worker2 # Allocating group tables: 0/8 done49client # Creating journal (8192 blocks): done50lb # Writing inode tables: 0/8 done51worker2 # Writing inode tables: 0/8 done52client # Writing superblocks and filesystem accounting information: 0/8 done53lb # Creating journal (8192 blocks): done54worker2 # Creating journal (8192 blocks): done55client # 56lb # Writing superblocks and filesystem accounting information: 0/8 done57worker2 # Writing superblocks and filesystem accounting information: 0/8 done58client # Virtualisation disk image created.59lb # 60worker2 # 61worker1 # mke2fs 1.47.4 (6-Mar-2025)62(finished: start all VMs, in 0.47 seconds)63lb # Virtualisation disk image created.64worker2 # Virtualisation disk image created.65worker1 # Discarding device blocks: 0/262144 done66client: waiting for unit network-addresses-eth1.service67worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes68client: waiting for the VM to finish booting69worker1 # Filesystem UUID: b0f1d07d-b782-457f-88d9-1c9db3ece5ab70worker1 # Superblock backups stored on blocks:71worker1 # 32768, 98304, 163840, 22937672worker1 # 73worker1 # Allocating group tables: 0/8 done74worker1 # Writing inode tables: 0/8 done75worker1 # Creating journal (8192 blocks): done76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77worker1 # 78worker1 # Virtualisation disk image created.79worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker1 # [ 0.000000] KASLR enabled82worker1 # [ 0.000000] random: crng init done83worker1 # [ 0.000000] Machine model: linux,dummy-virt84worker1 # [ 0.000000] efi: UEFI not found.85worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88worker1 # [ 0.000000] Zone ranges:89worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90worker1 # [ 0.000000] DMA32 empty91worker1 # [ 0.000000] Normal empty92worker1 # [ 0.000000] Device empty93worker1 # [ 0.000000] Movable zone start for each node94worker1 # [ 0.000000] Early memory node ranges95worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098worker1 # [ 0.000000] psci: probing for conduit method from DT.99worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.100worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101worker1 # [ 0.000000] psci: Trusted OS migration not required102worker1 # [ 0.000000] psci: SMC Calling Convention v1.1103worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105worker1 # [ 0.000000] Detected PIPT I-cache on CPU0106worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface108worker1 # [ 0.000000] CPU features: detected: Spectre-v4109worker1 # [ 0.000000] CPU features: detected: Spectre-BHB110lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]111worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38112worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23113lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026114worker1 # [ 0.000000] alternatives: applying boot alternatives115lb # [ 0.000000] KASLR enabled116lb # [ 0.000000] random: crng init done117lb # [ 0.000000] Machine model: linux,dummy-virt118lb # [ 0.000000] efi: UEFI not found.119lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT120lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]121worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yqvifpngzkgk2gjqxsx0f875b9f5v22f-nixos-system-worker1-test/init regInfo=/nix/store/2yfimnn0qcadfg5qfp1pfkpyl17gb4wp-closure-info/registration console=ttyAMA0,115200n8 console=tty0122lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]123lb # [ 0.000000] Zone ranges:124lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]125lb # [ 0.000000] DMA32 empty126worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/2yfimnn0qcadfg5qfp1pfkpyl17gb4wp-closure-info/registration", will be passed to user space.127lb # [ 0.000000] Normal empty128lb # [ 0.000000] Device empty129worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]130worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes131lb # [ 0.000000] Movable zone start for each node132lb # [ 0.000000] Early memory node ranges133worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)134worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026135client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]136lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]137worker2 # [ 0.000000] KASLR enabled138worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)139worker2 # [ 0.000000] random: crng init done140lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]141worker2 # [ 0.000000] Machine model: linux,dummy-virt142client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026143worker2 # [ 0.000000] efi: UEFI not found.144worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB145lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000146client # [ 0.000000] KASLR enabled147worker1 # [ 0.000000] software IO TLB: area num 1.148lb # [ 0.000000] psci: probing for conduit method from DT.149client # [ 0.000000] random: crng init done150worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT151lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.152worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB153client # [ 0.000000] Machine model: linux,dummy-virt154client # [ 0.000000] efi: UEFI not found.155worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]156lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs157worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)158lb # [ 0.000000] psci: Trusted OS migration not required159client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT160worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]161worker1 # [ 0.000000] Fallback order for Node 0: 0162worker2 # [ 0.000000] Zone ranges:163lb # [ 0.000000] psci: SMC Calling Convention v1.1164client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]165worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216166worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]167worker1 # [ 0.000000] Policy zone: DMA168worker2 # [ 0.000000] DMA32 empty169lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)170client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]171worker2 # [ 0.000000] Normal empty172client # [ 0.000000] Zone ranges:173worker2 # [ 0.000000] Device empty174worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off175lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296176worker2 # [ 0.000000] Movable zone start for each node177client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]178lb # [ 0.000000] Detected PIPT I-cache on CPU0179worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1180worker2 # [ 0.000000] Early memory node ranges181client # [ 0.000000] DMA32 empty182worker1 # [ 0.000000] allocated 3145728 bytes of page_ext183client # [ 0.000000] Normal empty184client # [ 0.000000] Device empty185lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)186worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]187worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages188client # [ 0.000000] Movable zone start for each node189lb # [ 0.000000] CPU features: detected: GICv3 CPU interface190worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]191worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups192client # [ 0.000000] Early memory node ranges193lb # [ 0.000000] CPU features: detected: Spectre-v4194worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000195worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.196lb # [ 0.000000] CPU features: detected: Spectre-BHB197client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]198worker1 # [ 0.000000] rcu: RCU event tracing is enabled.199worker2 # [ 0.000000] psci: probing for conduit method from DT.200lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38201client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]202worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.203worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.204client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000205worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs206lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23207worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.208client # [ 0.000000] psci: probing for conduit method from DT.209worker2 # [ 0.000000] psci: Trusted OS migration not required210lb # [ 0.000000] alternatives: applying boot alternatives211worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.212worker2 # [ 0.000000] psci: SMC Calling Convention v1.1213client # [ 0.000000] psci: PSCIv1.3 detected in firmware.214worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.215client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs216worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)217client # [ 0.000000] psci: Trusted OS migration not required218worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.219client # [ 0.000000] psci: SMC Calling Convention v1.1220worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296221worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1222worker2 # [ 0.000000] Detected PIPT I-cache on CPU0223client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)224worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.225lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lai4n7chb0cnvnygfis27sipihspygia-nixos-system-lb-test/init regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration console=ttyAMA0,115200n8 console=tty0226worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)227client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296228worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface229client # [ 0.000000] Detected PIPT I-cache on CPU0230worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.231worker2 # [ 0.000000] CPU features: detected: Spectre-v4232lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration", will be passed to user space.233client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)234worker2 # [ 0.000000] CPU features: detected: Spectre-BHB235worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.236client # [ 0.000000] CPU features: detected: GICv3 CPU interface237worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38238lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes239worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0240client # [ 0.000000] CPU features: detected: Spectre-v4241worker1 # [ 0.000000] GICv3: 256 SPIs implemented242worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23243client # [ 0.000000] CPU features: detected: Spectre-BHB244lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)245worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented246worker2 # [ 0.000000] alternatives: applying boot alternatives247client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38248worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq249lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)250worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI251client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23252lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB253worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0254client # [ 0.000000] alternatives: applying boot alternatives255lb # [ 0.000000] software IO TLB: area num 1.256worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000257lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB258worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]259lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)260worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lzfphg14bbx0ymklkkdb2j4hxl686vk6-nixos-system-worker2-test/init regInfo=/nix/store/16qvqza31p65f30nhkwp09km5mgws4k5-closure-info/registration console=ttyAMA0,115200n8 console=tty0261worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)262lb # [ 0.000000] Fallback order for Node 0: 0263lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216264worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)265lb # [ 0.000000] Policy zone: DMA266client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pf18as2xhcjh3l28kia27rgak8kjgrxc-nixos-system-client-test/init regInfo=/nix/store/gp9rvyljlib3fy4ql6mjkp040sjlfdva-closure-info/registration console=ttyAMA0,115200n8 console=tty0267worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/16qvqza31p65f30nhkwp09km5mgws4k5-closure-info/registration", will be passed to user space.268worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000269lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off270worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes271lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1272worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000273client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/gp9rvyljlib3fy4ql6mjkp040sjlfdva-closure-info/registration", will be passed to user space.274worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)275lb # [ 0.000000] allocated 3145728 bytes of page_ext276worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.277client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes278lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages279worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)280lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups281worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns282worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB283client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)284lb # [ 0.000000] rcu: Hierarchical RCU implementation.285worker2 # [ 0.000000] software IO TLB: area num 1.286worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).287client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)288lb # [ 0.000000] rcu: RCU event tracing is enabled.289worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB290client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB291lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.292worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns293client # [ 0.000000] software IO TLB: area num 1.294worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)295lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.296worker2 # [ 0.000000] Fallback order for Node 0: 0297lb # [ 0.000000] Rude variant of Tasks RCU enabled.298client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB299worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns300worker1 # [ 0.000037] arm-pv: using stolen time PV301lb # [ 0.000000] Tracing variant of Tasks RCU enabled.302worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216303client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)304worker2 # [ 0.000000] Policy zone: DMA305client # [ 0.000000] Fallback order for Node 0: 0306lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.307worker1 # [ 0.000838] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)308worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off309client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216310worker1 # [ 0.001027] Console: colour dummy device 80x25311lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1312worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1313client # [ 0.000000] Policy zone: DMA314worker1 # [ 0.001035] printk: legacy console [tty0] enabled315worker2 # [ 0.000000] allocated 3145728 bytes of page_ext316lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.317client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off318worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages319client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1320worker1 # [ 0.001239] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)321worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups322lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.323client # [ 0.000000] allocated 3145728 bytes of page_ext324worker1 # [ 0.001246] pid_max: default: 32768 minimum: 301325worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.326client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages327worker1 # [ 0.001327] LSM: initializing lsm=capability,landlock,yama,bpf,ima328worker2 # [ 0.000000] rcu: RCU event tracing is enabled.329lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.330worker1 # [ 0.001540] landlock: Up and running.331client # [ 0.000000] ftrace: allocated 294 pages with 4 groups332worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.333lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0334worker1 # [ 0.001543] Yama: becoming mindful.335client # [ 0.000000] rcu: Hierarchical RCU implementation.336lb # [ 0.000000] GICv3: 256 SPIs implemented337worker1 # [ 0.002272] LSM support for eBPF active338worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.339client # [ 0.000000] rcu: RCU event tracing is enabled.340lb # [ 0.000000] GICv3: 0 Extended SPIs implemented341worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.342worker1 # [ 0.002471] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)343lb # [ 0.000000] Root IRQ handler: gic_handle_irq344client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.345worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.346lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI347client # [ 0.000000] Trampoline variant of Tasks RCU enabled.348worker1 # [ 0.002505] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)349lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0350worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.351client # [ 0.000000] Rude variant of Tasks RCU enabled.352worker1 # [ 0.003652] cacheinfo: Unable to detect cache hierarchy for CPU 0353client # [ 0.000000] Tracing variant of Tasks RCU enabled.354worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1355lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000356worker1 # [ 0.004455] rcu: Hierarchical SRCU implementation.357lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]358client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.359worker1 # [ 0.004459] rcu: Max phase no-delay instances is 1000.360worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.361worker1 # [ 0.005640] fsl-mc MSI: its@8080000 domain created362client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1363lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)364worker1 # [ 0.005730] EFI services will not be available.365worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.366worker1 # [ 0.005840] smp: Bringing up secondary CPUs ...367client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.368lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)369worker1 # [ 0.005848] smp: Brought up 1 node, 1 CPU370worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.371lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000372worker1 # [ 0.005852] SMP: Total of 1 processors activated.373client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.374worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0375worker1 # [ 0.005857] CPU: All CPU(s) started at EL1376worker2 # [ 0.000000] GICv3: 256 SPIs implemented377lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000378worker1 # [ 0.005868] CPU features: detected: Branch Target Identification379client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.380worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented381lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.382worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq383client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0384worker1 # [ 0.005872] CPU features: detected: ARMv8.4 Translation Table Level385client # [ 0.000000] GICv3: 256 SPIs implemented386worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI387lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns388client # [ 0.000000] GICv3: 0 Extended SPIs implemented389worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0390worker1 # [ 0.005875] CPU features: detected: Instruction cache invalidation not required for I/D coherence391client # [ 0.000000] Root IRQ handler: gic_handle_irq392lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).393worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000394client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI395worker1 # [ 0.005879] CPU features: detected: Data cache clean to the PoU not required for I/D coherence396worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]397client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0398worker1 # [ 0.005883] CPU features: detected: Common not Private translations399lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns400worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)401client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000402worker1 # [ 0.005886] CPU features: detected: CRC32 instructions403client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]404lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns405worker1 # [ 0.005889] CPU features: detected: Data cache clean to Point of Deep Persistence406lb # [ 0.000031] arm-pv: using stolen time PV407worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)408client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)409worker1 # [ 0.005892] CPU features: detected: Data cache clean to Point of Persistence410worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000411lb # [ 0.000485] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)412worker1 # [ 0.005895] CPU features: detected: Data independent timing control (DIT)413worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000414client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)415lb # [ 0.000657] Console: colour dummy device 80x25416worker1 # [ 0.005898] CPU features: detected: E0PD417lb # [ 0.000665] printk: legacy console [tty0] enabled418client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000419worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.420worker1 # [ 0.005901] CPU features: detected: Enhanced Counter Virtualization421client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000422lb # [ 0.000855] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)423worker1 # [ 0.005904] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)424worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns425lb # [ 0.000862] pid_max: default: 32768 minimum: 301426client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.427worker1 # [ 0.005907] CPU features: detected: Enhanced Virtualization Traps428worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).429lb # [ 0.000933] LSM: initializing lsm=capability,landlock,yama,bpf,ima430worker1 # [ 0.005910] CPU features: detected: Fine Grained Traps431client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns432lb # [ 0.001089] landlock: Up and running.433worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns434worker1 # [ 0.005914] CPU features: detected: Generic authentication (architected QARMA5 algorithm)435client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).436lb # [ 0.001093] Yama: becoming mindful.437worker1 # [ 0.005919] CPU features: detected: RCpc load-acquire (LDAPR)438worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns439lb # [ 0.001633] LSM support for eBPF active440worker2 # [ 0.000031] arm-pv: using stolen time PV441worker1 # [ 0.005922] CPU features: detected: LSE atomic instructions442client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns443worker1 # [ 0.005925] CPU features: detected: Privileged Access Never444lb # [ 0.001817] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)445client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns446worker2 # [ 0.000441] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)447worker1 # [ 0.005927] CPU features: detected: PMUv3448client # [ 0.000031] arm-pv: using stolen time PV449lb # [ 0.001860] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)450worker2 # [ 0.000614] Console: colour dummy device 80x25451worker1 # [ 0.005930] CPU features: detected: RAS Extension Support452worker2 # [ 0.000622] printk: legacy console [tty0] enabled453lb # [ 0.003011] cacheinfo: Unable to detect cache hierarchy for CPU 0454worker1 # [ 0.005933] CPU features: detected: RASv1p1 Extension Support455client # [ 0.000423] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)456lb # [ 0.003800] rcu: Hierarchical SRCU implementation.457worker1 # [ 0.005935] CPU features: detected: Random Number Generator458client # [ 0.000572] Console: colour dummy device 80x25459worker2 # [ 0.000808] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)460lb # [ 0.003806] rcu: Max phase no-delay instances is 1000.461worker1 # [ 0.005938] CPU features: detected: Speculation barrier (SB)462client # [ 0.000580] printk: legacy console [tty0] enabled463worker2 # [ 0.000815] pid_max: default: 32768 minimum: 301464lb # [ 0.004999] fsl-mc MSI: its@8080000 domain created465worker1 # [ 0.005941] CPU features: detected: Stage-2 Force Write-Back466lb # [ 0.005098] EFI services will not be available.467worker2 # [ 0.000886] LSM: initializing lsm=capability,landlock,yama,bpf,ima468worker2 # [ 0.001001] landlock: Up and running.469client # [ 0.000778] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)470lb # [ 0.005195] smp: Bringing up secondary CPUs ...471worker1 # [ 0.005944] CPU features: detected: TLB range maintenance instructions472worker2 # [ 0.001004] Yama: becoming mindful.473lb # [ 0.005205] smp: Brought up 1 node, 1 CPU474client # [ 0.000786] pid_max: default: 32768 minimum: 301475worker2 # [ 0.001465] LSM support for eBPF active476worker1 # [ 0.005949] CPU features: detected: Speculative Store Bypassing Safe (SSBS)477lb # [ 0.005208] SMP: Total of 1 processors activated.478client # [ 0.000868] LSM: initializing lsm=capability,landlock,yama,bpf,ima479lb # [ 0.005213] CPU: All CPU(s) started at EL1480client # [ 0.001051] landlock: Up and running.481worker2 # [ 0.001605] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)482worker1 # [ 0.005989] alternatives: applying system-wide alternatives483client # [ 0.001054] Yama: becoming mindful.484lb # [ 0.005224] CPU features: detected: Branch Target Identification485worker1 # [ 0.009103] CPU features: detected: BBM Level 2 without TLB conflict abort486client # [ 0.001568] LSM support for eBPF active487worker2 # [ 0.001639] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)488lb # [ 0.005229] CPU features: detected: ARMv8.4 Translation Table Level489worker2 # [ 0.002890] cacheinfo: Unable to detect cache hierarchy for CPU 0490client # [ 0.001713] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)491worker2 # [ 0.003626] rcu: Hierarchical SRCU implementation.492worker1 # [ 0.009286] Memory: 1406544K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)493lb # [ 0.005232] CPU features: detected: Instruction cache invalidation not required for I/D coherence494client # [ 0.001747] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)495worker1 # [ 0.009708] devtmpfs: initialized496worker2 # [ 0.003630] rcu: Max phase no-delay instances is 1000.497client # [ 0.002933] cacheinfo: Unable to detect cache hierarchy for CPU 0498worker2 # [ 0.004857] fsl-mc MSI: its@8080000 domain created499lb # [ 0.005236] CPU features: detected: Data cache clean to the PoU not required for I/D coherence500worker1 # [ 0.011528] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)501client # [ 0.003681] rcu: Hierarchical SRCU implementation.502worker2 # [ 0.004949] EFI services will not be available.503lb # [ 0.005240] CPU features: detected: Common not Private translations504client # [ 0.003687] rcu: Max phase no-delay instances is 1000.505worker2 # [ 0.005053] smp: Bringing up secondary CPUs ...506worker1 # [ 0.011552] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).507lb # [ 0.005243] CPU features: detected: CRC32 instructions508worker2 # [ 0.005063] smp: Brought up 1 node, 1 CPU509client # [ 0.004905] fsl-mc MSI: its@8080000 domain created510worker1 # [ 0.011740] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL511worker2 # [ 0.005066] SMP: Total of 1 processors activated.512client # [ 0.004997] EFI services will not be available.513lb # [ 0.005246] CPU features: detected: Data cache clean to Point of Deep Persistence514worker2 # [ 0.005072] CPU: All CPU(s) started at EL1515client # [ 0.005064] smp: Bringing up secondary CPUs ...516worker1 # [ 0.011745] 0 pages in range for non-PLT usage517worker1 # [ 0.011746] 508288 pages in range for PLT usage518client # [ 0.005073] smp: Brought up 1 node, 1 CPU519lb # [ 0.005250] CPU features: detected: Data cache clean to Point of Persistence520worker2 # [ 0.005084] CPU features: detected: Branch Target Identification521worker1 # [ 0.011861] pinctrl core: initialized pinctrl subsystem522client # [ 0.005076] SMP: Total of 1 processors activated.523lb # [ 0.005253] CPU features: detected: Data independent timing control (DIT)524worker1 # [ 0.012662] DMI not present or invalid.525client # [ 0.005082] CPU: All CPU(s) started at EL1526lb # [ 0.005256] CPU features: detected: E0PD527worker2 # [ 0.005089] CPU features: detected: ARMv8.4 Translation Table Level528worker1 # [ 0.016600] NET: Registered PF_NETLINK/PF_ROUTE protocol family529client # [ 0.005092] CPU features: detected: Branch Target Identification530lb # [ 0.005258] CPU features: detected: Enhanced Counter Virtualization531worker2 # [ 0.005092] CPU features: detected: Instruction cache invalidation not required for I/D coherence532worker1 # [ 0.018917] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations533client # [ 0.005097] CPU features: detected: ARMv8.4 Translation Table Level534lb # [ 0.005261] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)535worker1 # [ 0.019106] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations536worker2 # [ 0.005096] CPU features: detected: Data cache clean to the PoU not required for I/D coherence537client # [ 0.005100] CPU features: detected: Instruction cache invalidation not required for I/D coherence538lb # [ 0.005265] CPU features: detected: Enhanced Virtualization Traps539lb # [ 0.005268] CPU features: detected: Fine Grained Traps540worker2 # [ 0.005099] CPU features: detected: Common not Private translations541worker1 # [ 0.019307] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations542client # [ 0.005104] CPU features: detected: Data cache clean to the PoU not required for I/D coherence543worker2 # [ 0.005103] CPU features: detected: CRC32 instructions544worker1 # [ 0.019333] audit: initializing netlink subsys (disabled)545lb # [ 0.005271] CPU features: detected: Generic authentication (architected QARMA5 algorithm)546client # [ 0.005108] CPU features: detected: Common not Private translations547worker2 # [ 0.005106] CPU features: detected: Data cache clean to Point of Deep Persistence548worker1 # [ 0.019991] thermal_sys: Registered thermal governor 'fair_share'549lb # [ 0.005276] CPU features: detected: RCpc load-acquire (LDAPR)550client # [ 0.005111] CPU features: detected: CRC32 instructions551worker1 # [ 0.019993] thermal_sys: Registered thermal governor 'bang_bang'552worker2 # [ 0.005109] CPU features: detected: Data cache clean to Point of Persistence553lb # [ 0.005279] CPU features: detected: LSE atomic instructions554client # [ 0.005114] CPU features: detected: Data cache clean to Point of Deep Persistence555lb # [ 0.005282] CPU features: detected: Privileged Access Never556worker2 # [ 0.005113] CPU features: detected: Data independent timing control (DIT)557lb # [ 0.005285] CPU features: detected: PMUv3558client # [ 0.005117] CPU features: detected: Data cache clean to Point of Persistence559worker2 # [ 0.005116] CPU features: detected: E0PD560worker1 # [ 0.019997] thermal_sys: Registered thermal governor 'step_wise'561lb # [ 0.005288] CPU features: detected: RAS Extension Support562client # [ 0.005121] CPU features: detected: Data independent timing control (DIT)563worker2 # [ 0.005118] CPU features: detected: Enhanced Counter Virtualization564worker1 # [ 0.020002] thermal_sys: Registered thermal governor 'user_space'565lb # [ 0.005291] CPU features: detected: RASv1p1 Extension Support566worker2 # [ 0.005121] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)567worker1 # [ 0.020005] thermal_sys: Registered thermal governor 'power_allocator'568lb # [ 0.005293] CPU features: detected: Random Number Generator569client # [ 0.005124] CPU features: detected: E0PD570worker2 # [ 0.005124] CPU features: detected: Enhanced Virtualization Traps571worker1 # [ 0.020033] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1572client # [ 0.005126] CPU features: detected: Enhanced Counter Virtualization573lb # [ 0.005296] CPU features: detected: Speculation barrier (SB)574worker2 # [ 0.005127] CPU features: detected: Fine Grained Traps575worker1 # [ 0.020041] cpuidle: using governor ladder576lb # [ 0.005299] CPU features: detected: Stage-2 Force Write-Back577client # [ 0.005129] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)578worker1 # [ 0.020047] cpuidle: using governor menu579worker2 # [ 0.005131] CPU features: detected: Generic authentication (architected QARMA5 algorithm)580lb # [ 0.005302] CPU features: detected: TLB range maintenance instructions581client # [ 0.005133] CPU features: detected: Enhanced Virtualization Traps582worker1 # [ 0.020278] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.583worker2 # [ 0.005136] CPU features: detected: RCpc load-acquire (LDAPR)584client # [ 0.005136] CPU features: detected: Fine Grained Traps585lb # [ 0.005306] CPU features: detected: Speculative Store Bypassing Safe (SSBS)586worker1 # [ 0.020294] ASID allocator initialised with 65536 entries587worker2 # [ 0.005139] CPU features: detected: LSE atomic instructions588lb # [ 0.005356] alternatives: applying system-wide alternatives589worker1 # [ 0.022230] Serial: AMBA PL011 UART driver590client # [ 0.005140] CPU features: detected: Generic authentication (architected QARMA5 algorithm)591worker2 # [ 0.005142] CPU features: detected: Privileged Access Never592lb # [ 0.008393] CPU features: detected: BBM Level 2 without TLB conflict abort593worker2 # [ 0.005144] CPU features: detected: PMUv3594worker1 # [ 0.027488] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1595client # [ 0.005144] CPU features: detected: RCpc load-acquire (LDAPR)596worker2 # [ 0.005147] CPU features: detected: RAS Extension Support597worker1 # [ 0.027615] printk: console [ttyAMA0] enabled598client # [ 0.005147] CPU features: detected: LSE atomic instructions599client # [ 0.005151] CPU features: detected: Privileged Access Never600lb # [ 0.008551] Memory: 1406772K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)601client # [ 0.005153] CPU features: detected: PMUv3602lb # [ 0.009004] devtmpfs: initialized603worker2 # [ 0.005150] CPU features: detected: RASv1p1 Extension Support604client # [ 0.005156] CPU features: detected: RAS Extension Support605worker2 # [ 0.005153] CPU features: detected: Random Number Generator606lb # [ 0.011019] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)607client # [ 0.005159] CPU features: detected: RASv1p1 Extension Support608worker2 # [ 0.005155] CPU features: detected: Speculation barrier (SB)609client # [ 0.005162] CPU features: detected: Random Number Generator610worker2 # [ 0.005158] CPU features: detected: Stage-2 Force Write-Back611client # [ 0.005164] CPU features: detected: Speculation barrier (SB)612lb # [ 0.011044] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).613worker2 # [ 0.005161] CPU features: detected: TLB range maintenance instructions614client # [ 0.005167] CPU features: detected: Stage-2 Force Write-Back615lb # [ 0.011264] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL616worker2 # [ 0.005166] CPU features: detected: Speculative Store Bypassing Safe (SSBS)617lb # [ 0.011273] 0 pages in range for non-PLT usage618client # [ 0.005170] CPU features: detected: TLB range maintenance instructions619worker2 # [ 0.005203] alternatives: applying system-wide alternatives620lb # [ 0.011274] 508288 pages in range for PLT usage621client # [ 0.005175] CPU features: detected: Speculative Store Bypassing Safe (SSBS)622worker2 # [ 0.008234] CPU features: detected: BBM Level 2 without TLB conflict abort623lb # [ 0.011391] pinctrl core: initialized pinctrl subsystem624client # [ 0.005216] alternatives: applying system-wide alternatives625lb # [ 0.012186] DMI not present or invalid.626client # [ 0.008270] CPU features: detected: BBM Level 2 without TLB conflict abort627lb # [ 0.015722] NET: Registered PF_NETLINK/PF_ROUTE protocol family628worker2 # [ 0.008384] Memory: 1406792K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)629worker2 # [ 0.008834] devtmpfs: initialized630lb # [ 0.017981] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations631client # [ 0.008441] Memory: 1406520K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)632worker2 # [ 0.010739] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)633lb # [ 0.018164] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations634client # [ 0.008876] devtmpfs: initialized635worker1 # [ 0.149152] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages636worker2 # [ 0.010762] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).637lb # [ 0.018368] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations638worker1 # [ 0.149173] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page639client # [ 0.010808] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)640lb # [ 0.018390] audit: initializing netlink subsys (disabled)641worker2 # [ 0.010945] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL642worker1 # [ 0.149178] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages643client # [ 0.010832] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).644worker2 # [ 0.010950] 0 pages in range for non-PLT usage645lb # [ 0.019010] thermal_sys: Registered thermal governor 'fair_share'646worker1 # [ 0.149182] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page647worker2 # [ 0.010951] 508288 pages in range for PLT usage648client # [ 0.011024] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL649lb # [ 0.019012] thermal_sys: Registered thermal governor 'bang_bang'650worker2 # [ 0.011076] pinctrl core: initialized pinctrl subsystem651client # [ 0.011029] 0 pages in range for non-PLT usage652worker1 # [ 0.149187] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages653worker2 # [ 0.011844] DMI not present or invalid.654lb # [ 0.019015] thermal_sys: Registered thermal governor 'step_wise'655client # [ 0.011030] 508288 pages in range for PLT usage656worker1 # [ 0.149191] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page657worker2 # [ 0.015484] NET: Registered PF_NETLINK/PF_ROUTE protocol family658lb # [ 0.019021] thermal_sys: Registered thermal governor 'user_space'659client # [ 0.011126] pinctrl core: initialized pinctrl subsystem660worker1 # [ 0.149196] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages661client # [ 0.011872] DMI not present or invalid.662lb # [ 0.019024] thermal_sys: Registered thermal governor 'power_allocator'663worker2 # [ 0.017738] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations664worker1 # [ 0.149200] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page665client # [ 0.015143] NET: Registered PF_NETLINK/PF_ROUTE protocol family666worker2 # [ 0.017945] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations667lb # [ 0.019049] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1668client # [ 0.017841] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations669lb # [ 0.019057] cpuidle: using governor ladder670worker2 # [ 0.018161] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations671lb # [ 0.019063] cpuidle: using governor menu672client # [ 0.018074] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations673worker2 # [ 0.018187] audit: initializing netlink subsys (disabled)674lb # [ 0.019262] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.675worker1 # [ 0.156512] fbcon: Taking over console676client # [ 0.018285] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations677worker1 # [ 0.156534] ACPI: Interpreter disabled.678worker2 # [ 0.018769] thermal_sys: Registered thermal governor 'fair_share'679lb # [ 0.019279] ASID allocator initialised with 65536 entries680client # [ 0.018311] audit: initializing netlink subsys (disabled)681lb # [ 0.020512] Serial: AMBA PL011 UART driver682worker2 # [ 0.018771] thermal_sys: Registered thermal governor 'bang_bang'683client # [ 0.018927] thermal_sys: Registered thermal governor 'fair_share'684worker2 # [ 0.018774] thermal_sys: Registered thermal governor 'step_wise'685lb # [ 0.025914] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1686client # [ 0.018929] thermal_sys: Registered thermal governor 'bang_bang'687worker2 # [ 0.018780] thermal_sys: Registered thermal governor 'user_space'688lb # [ 0.026087] printk: console [ttyAMA0] enabled689client # [ 0.018933] thermal_sys: Registered thermal governor 'step_wise'690worker2 # [ 0.018783] thermal_sys: Registered thermal governor 'power_allocator'691client # [ 0.018939] thermal_sys: Registered thermal governor 'user_space'692worker2 # [ 0.018808] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1693client # [ 0.018941] thermal_sys: Registered thermal governor 'power_allocator'694worker2 # [ 0.018816] cpuidle: using governor ladder695worker2 # [ 0.018822] cpuidle: using governor menu696client # [ 0.018973] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1697client # [ 0.018981] cpuidle: using governor ladder698worker2 # [ 0.019010] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.699client # [ 0.018987] cpuidle: using governor menu700worker2 # [ 0.019025] ASID allocator initialised with 65536 entries701worker2 # [ 0.020266] Serial: AMBA PL011 UART driver702client # [ 0.019182] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.703client # [ 0.019200] ASID allocator initialised with 65536 entries704worker2 # [ 0.025502] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1705client # [ 0.020419] Serial: AMBA PL011 UART driver706worker1 # [ 0.165248] iommu: Default domain type: Translated707worker2 # [ 0.025633] printk: console [ttyAMA0] enabled708worker1 # [ 0.165261] iommu: DMA domain TLB invalidation policy: strict mode709client # [ 0.025764] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1710worker1 # [ 0.165630] SCSI subsystem initialized711client # [ 0.025893] printk: console [ttyAMA0] enabled712worker1 # [ 0.167652] usbcore: registered new interface driver usbfs713worker1 # [ 0.167687] usbcore: registered new interface driver hub714lb # [ 0.147701] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages715worker1 # [ 0.167713] usbcore: registered new device driver usb716lb # [ 0.147722] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page717worker1 # [ 0.167979] pps_core: LinuxPPS API ver. 1 registered718lb # [ 0.147727] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages719worker1 # [ 0.167985] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>720lb # [ 0.147731] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page721worker1 # [ 0.167995] PTP clock support registered722worker1 # [ 0.168054] EDAC MC: Ver: 3.0.0723lb # [ 0.147736] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages724worker1 # [ 0.172854] scmi_core: SCMI protocol bus registered725lb # [ 0.147740] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page726lb # [ 0.147744] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages727worker1 # [ 0.173880] FPGA manager framework728worker1 # [ 0.174868] vgaarb: loaded729lb # [ 0.147748] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page730client # [ 0.143610] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages731worker2 # [ 0.145402] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages732client # [ 0.143629] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page733worker1 # [ 0.175528] clocksource: Switched to clocksource arch_sys_counter734worker2 # [ 0.145422] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page735client # [ 0.143635] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages736lb # [ 0.155385] fbcon: Taking over console737worker2 # [ 0.145428] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages738worker1 # [ 0.177059] VFS: Disk quotas dquot_6.6.0739client # [ 0.143639] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page740lb # [ 0.155405] ACPI: Interpreter disabled.741worker2 # [ 0.145433] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page742worker1 # [ 0.177094] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)743client # [ 0.143644] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages744worker2 # [ 0.145438] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages745lb # [ 0.157350] iommu: Default domain type: Translated746client # [ 0.143648] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page747lb # [ 0.157361] iommu: DMA domain TLB invalidation policy: strict mode748worker2 # [ 0.145443] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page749client # [ 0.143652] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages750worker2 # [ 0.145447] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages751client # [ 0.143656] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page752worker2 # [ 0.145451] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page753worker1 # [ 0.180807] netfs: FS-Cache loaded754worker1 # [ 0.180939] pnp: PnP ACPI: disabled755client # [ 0.151067] fbcon: Taking over console756client # [ 0.151083] ACPI: Interpreter disabled.757worker2 # [ 0.153257] fbcon: Taking over console758worker2 # [ 0.153275] ACPI: Interpreter disabled.759lb # [ 0.164418] SCSI subsystem initialized760lb # [ 0.165116] usbcore: registered new interface driver usbfs761lb # [ 0.165150] usbcore: registered new interface driver hub762worker1 # [ 0.184965] NET: Registered PF_INET protocol family763lb # [ 0.165168] usbcore: registered new device driver usb764lb # [ 0.165474] pps_core: LinuxPPS API ver. 1 registered765worker1 # [ 0.185238] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)766lb # [ 0.165479] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>767lb # [ 0.165490] PTP clock support registered768lb # [ 0.165542] EDAC MC: Ver: 3.0.0769client # [ 0.160122] iommu: Default domain type: Translated770worker2 # [ 0.161624] iommu: Default domain type: Translated771lb # [ 0.170263] scmi_core: SCMI protocol bus registered772client # [ 0.160134] iommu: DMA domain TLB invalidation policy: strict mode773worker2 # [ 0.161636] iommu: DMA domain TLB invalidation policy: strict mode774client # [ 0.160466] SCSI subsystem initialized775lb # [ 0.171229] FPGA manager framework776worker2 # [ 0.161982] SCSI subsystem initialized777lb # [ 0.172170] vgaarb: loaded778client # [ 0.162462] usbcore: registered new interface driver usbfs779worker2 # [ 0.163995] usbcore: registered new interface driver usbfs780lb # [ 0.172838] clocksource: Switched to clocksource arch_sys_counter781client # [ 0.162494] usbcore: registered new interface driver hub782worker2 # [ 0.164030] usbcore: registered new interface driver hub783client # [ 0.162512] usbcore: registered new device driver usb784worker2 # [ 0.164048] usbcore: registered new device driver usb785client # [ 0.162767] pps_core: LinuxPPS API ver. 1 registered786worker2 # [ 0.164348] pps_core: LinuxPPS API ver. 1 registered787client # [ 0.162773] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>788worker2 # [ 0.164357] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>789lb # [ 0.175995] VFS: Disk quotas dquot_6.6.0790client # [ 0.162783] PTP clock support registered791worker2 # [ 0.164367] PTP clock support registered792client # [ 0.162833] EDAC MC: Ver: 3.0.0793worker2 # [ 0.164422] EDAC MC: Ver: 3.0.0794lb # [ 0.176039] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)795client # [ 0.167448] scmi_core: SCMI protocol bus registered796worker2 # [ 0.169068] scmi_core: SCMI protocol bus registered797client # [ 0.168463] FPGA manager framework798worker2 # [ 0.170118] FPGA manager framework799client # [ 0.169431] vgaarb: loaded800worker2 # [ 0.171149] vgaarb: loaded801lb # [ 0.179929] netfs: FS-Cache loaded802client # [ 0.170106] clocksource: Switched to clocksource arch_sys_counter803lb # [ 0.180066] pnp: PnP ACPI: disabled804worker2 # [ 0.171822] clocksource: Switched to clocksource arch_sys_counter805worker2 # [ 0.173917] VFS: Disk quotas dquot_6.6.0806worker2 # [ 0.173959] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)807client # [ 0.173987] VFS: Disk quotas dquot_6.6.0808client # [ 0.174022] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)809lb # [ 0.184128] NET: Registered PF_INET protocol family810lb # [ 0.184409] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)811worker2 # [ 0.177742] netfs: FS-Cache loaded812worker2 # [ 0.177880] pnp: PnP ACPI: disabled813client # [ 0.177664] netfs: FS-Cache loaded814client # [ 0.177775] pnp: PnP ACPI: disabled815client # [ 0.181759] NET: Registered PF_INET protocol family816worker2 # [ 0.181908] NET: Registered PF_INET protocol family817client # [ 0.182043] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)818worker2 # [ 0.182173] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)819worker1 # [ 0.215743] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)820worker1 # [ 0.215796] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)821worker1 # [ 0.215825] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)822worker1 # [ 0.215901] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)823worker1 # [ 0.216046] TCP: Hash tables configured (established 16384 bind 16384)824worker1 # [ 0.216157] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)825worker1 # [ 0.216209] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)826worker1 # [ 0.216247] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)827worker1 # [ 0.216353] NET: Registered PF_UNIX/PF_LOCAL protocol family828worker1 # [ 0.216398] NET: Registered PF_XDP protocol family829worker1 # [ 0.216418] PCI: CLS 0 bytes, default 64830worker1 # [ 0.216658] Trying to unpack rootfs image as initramfs...831lb # [ 0.213922] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)832lb # [ 0.213972] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)833lb # [ 0.214005] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)834lb # [ 0.214084] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)835lb # [ 0.214228] TCP: Hash tables configured (established 16384 bind 16384)836lb # [ 0.214319] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)837lb # [ 0.214356] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)838lb # [ 0.214394] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)839lb # [ 0.214532] NET: Registered PF_UNIX/PF_LOCAL protocol family840worker1 # [ 0.239568] kvm [1]: HYP mode not available841lb # [ 0.214611] NET: Registered PF_XDP protocol family842lb # [ 0.214631] PCI: CLS 0 bytes, default 64843client # [ 0.212010] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)844worker2 # [ 0.214121] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)845lb # [ 0.224094] Trying to unpack rootfs image as initramfs...846client # [ 0.212062] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)847worker2 # [ 0.214179] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)848client # [ 0.212091] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)849worker2 # [ 0.214222] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)850client # [ 0.212169] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)851worker2 # [ 0.214312] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)852client # [ 0.212312] TCP: Hash tables configured (established 16384 bind 16384)853worker2 # [ 0.214462] TCP: Hash tables configured (established 16384 bind 16384)854lb # [ 0.229558] kvm [1]: HYP mode not available855client # [ 0.212402] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)856worker2 # [ 0.214561] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)857client # [ 0.212449] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)858worker2 # [ 0.214599] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)859client # [ 0.212486] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)860worker2 # [ 0.214655] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)861client # [ 0.212582] NET: Registered PF_UNIX/PF_LOCAL protocol family862worker2 # [ 0.214758] NET: Registered PF_UNIX/PF_LOCAL protocol family863client # [ 0.212655] NET: Registered PF_XDP protocol family864client # [ 0.212674] PCI: CLS 0 bytes, default 64865worker2 # [ 0.214787] NET: Registered PF_XDP protocol family866worker2 # [ 0.214807] PCI: CLS 0 bytes, default 64867client # [ 0.212931] Trying to unpack rootfs image as initramfs...868worker2 # [ 0.215084] Trying to unpack rootfs image as initramfs...869client # [ 0.227986] kvm [1]: HYP mode not available870worker2 # [ 0.229798] kvm [1]: HYP mode not available871worker1 # [ 0.336991] Initialise system trusted keyrings872worker1 # [ 0.337749] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker1 # [ 0.339020] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker1 # [ 0.339820] 9p: Installing v9fs 9p2000 file system support875client # [ 0.322725] Initialise system trusted keyrings876client # [ 0.323476] workingset: timestamp_bits=42 max_order=19 bucket_order=0877lb # [ 0.333510] Initialise system trusted keyrings878lb # [ 0.334306] workingset: timestamp_bits=42 max_order=19 bucket_order=0879client # [ 0.324916] squashfs: version 4.0 (2009/01/31) Phillip Lougher880lb # [ 0.335681] squashfs: version 4.0 (2009/01/31) Phillip Lougher881client # [ 0.325712] 9p: Installing v9fs 9p2000 file system support882lb # [ 0.336479] 9p: Installing v9fs 9p2000 file system support883worker2 # [ 0.332396] Initialise system trusted keyrings884worker2 # [ 0.333153] workingset: timestamp_bits=42 max_order=19 bucket_order=0885worker2 # [ 0.334468] squashfs: version 4.0 (2009/01/31) Phillip Lougher886worker2 # [ 0.335274] 9p: Installing v9fs 9p2000 file system support887worker1 # [ 0.367788] Key type asymmetric registered888worker1 # [ 0.367818] Asymmetric key parser 'x509' registered889worker1 # [ 0.367899] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)890worker1 # [ 0.370088] io scheduler mq-deadline registered891worker1 # [ 0.370100] io scheduler kyber registered892worker1 # [ 0.379700] pl061_gpio 9030000.pl061: PL061 GPIO chip registered893client # [ 0.349738] Key type asymmetric registered894client # [ 0.349771] Asymmetric key parser 'x509' registered895client # [ 0.349857] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)896worker1 # [ 0.381138] ledtrig-cpu: registered to indicate activity on CPUs897worker1 # [ 0.381553] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:898worker1 # [ 0.381573] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000899lb # [ 0.365200] Key type asymmetric registered900worker1 # [ 0.381584] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000901lb # [ 0.365231] Asymmetric key parser 'x509' registered902lb # [ 0.365307] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)903worker1 # [ 0.381592] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000904lb # [ 0.367491] io scheduler mq-deadline registered905worker1 # [ 0.381611] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits906lb # [ 0.367502] io scheduler kyber registered907worker1 # [ 0.381634] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]908worker1 # [ 0.381716] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00909worker1 # [ 0.381726] pci_bus 0000:00: root bus resource [bus 00-ff]910worker1 # [ 0.381732] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]911client # [ 0.358180] io scheduler mq-deadline registered912worker1 # [ 0.381737] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]913client # [ 0.358204] io scheduler kyber registered914worker2 # [ 0.364080] Key type asymmetric registered915worker1 # [ 0.381742] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]916worker2 # [ 0.364112] Asymmetric key parser 'x509' registered917client # [ 0.363467] pl061_gpio 9030000.pl061: PL061 GPIO chip registered918worker1 # [ 0.381799] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint919worker2 # [ 0.364182] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)920worker2 # [ 0.366288] io scheduler mq-deadline registered921worker1 # [ 0.382234] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint922worker2 # [ 0.366299] io scheduler kyber registered923worker1 # [ 0.382423] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]924client # [ 0.364717] ledtrig-cpu: registered to indicate activity on CPUs925worker1 # [ 0.382438] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]926client # [ 0.365086] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:927worker1 # [ 0.382467] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]928worker1 # [ 0.382483] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]929client # [ 0.365103] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000930worker1 # [ 0.382944] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint931client # [ 0.365120] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000932worker1 # [ 0.383128] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]933client # [ 0.365133] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000934worker1 # [ 0.383143] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]935client # [ 0.365165] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits936worker1 # [ 0.383173] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]937client # [ 0.365186] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]938client # [ 0.365261] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00939client # [ 0.365270] pci_bus 0000:00: root bus resource [bus 00-ff]940client # [ 0.365277] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]941worker2 # [ 0.375983] pl061_gpio 9030000.pl061: PL061 GPIO chip registered942client # [ 0.365283] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]943client # [ 0.365288] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]944client # [ 0.365364] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint945client # [ 0.365813] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint946worker2 # [ 0.377421] ledtrig-cpu: registered to indicate activity on CPUs947client # [ 0.366004] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]948lb # [ 0.381272] pl061_gpio 9030000.pl061: PL061 GPIO chip registered949worker2 # [ 0.377798] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:950client # [ 0.366021] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]951lb # [ 0.381967] ledtrig-cpu: registered to indicate activity on CPUs952worker2 # [ 0.377819] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000953client # [ 0.366052] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]954lb # [ 0.382356] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:955client # [ 0.366069] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]956worker2 # [ 0.377831] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000957lb # [ 0.382391] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000958worker2 # [ 0.377840] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000959lb # [ 0.382405] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000960worker2 # [ 0.377870] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits961lb # [ 0.382413] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000962worker2 # [ 0.377896] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]963worker1 # [ 0.407699] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint964lb # [ 0.382435] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits965worker2 # [ 0.377975] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00966worker1 # [ 0.407906] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]967lb # [ 0.382458] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]968worker2 # [ 0.377984] pci_bus 0000:00: root bus resource [bus 00-ff]969worker1 # [ 0.407923] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]970worker2 # [ 0.377990] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]971lb # [ 0.382535] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00972worker1 # [ 0.407952] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]973lb # [ 0.382545] pci_bus 0000:00: root bus resource [bus 00-ff]974worker2 # [ 0.377996] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]975worker1 # [ 0.408444] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint976lb # [ 0.382551] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]977worker2 # [ 0.378001] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]978worker1 # [ 0.408629] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]979lb # [ 0.382556] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]980worker1 # [ 0.408647] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]981worker2 # [ 0.378058] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint982lb # [ 0.382561] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]983worker1 # [ 0.408680] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]984worker2 # [ 0.378512] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint985lb # [ 0.382668] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint986worker2 # [ 0.378714] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]987worker1 # [ 0.409144] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint988worker2 # [ 0.378731] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]989worker1 # [ 0.409326] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]990lb # [ 0.383135] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint991worker1 # [ 0.409341] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]992worker2 # [ 0.378761] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]993lb # [ 0.383324] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]994lb # [ 0.383340] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]995worker2 # [ 0.378777] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]996worker1 # [ 0.409370] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]997lb # [ 0.383370] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]998worker2 # [ 0.379251] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint999worker1 # [ 0.409825] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1000client # [ 0.386620] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1001lb # [ 0.383386] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1002worker2 # [ 0.379438] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1003worker1 # [ 0.410010] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1004client # [ 0.386828] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1005worker2 # [ 0.379454] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1006worker1 # [ 0.410026] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1007client # [ 0.386845] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1008lb # [ 0.383848] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1009worker2 # [ 0.379485] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1010lb # [ 0.384033] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1011worker1 # [ 0.410056] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1012client # [ 0.386875] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1013lb # [ 0.384048] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1014client # [ 0.387350] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1015worker1 # [ 0.410514] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1016lb # [ 0.384078] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1017client # [ 0.387535] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1018worker1 # [ 0.410699] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1019worker1 # [ 0.410715] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1020client # [ 0.387552] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1021lb # [ 0.384539] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1022client # [ 0.387582] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1023worker1 # [ 0.410744] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1024lb # [ 0.384723] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1025worker1 # [ 0.410760] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1026lb # [ 0.384740] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1027client # [ 0.388047] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1028lb # [ 0.384769] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1029client # [ 0.388233] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1030worker1 # [ 0.411262] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1031client # [ 0.388249] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1032worker1 # [ 0.411451] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1033lb # [ 0.385253] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1034client # [ 0.388283] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035lb # [ 0.385439] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1036worker1 # [ 0.411480] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1037lb # [ 0.385455] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1038client # [ 0.388764] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1039worker1 # [ 0.411949] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1040client # [ 0.388947] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1041lb # [ 0.385487] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042worker1 # [ 0.412142] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1043client # [ 0.388963] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1044lb # [ 0.385961] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1045worker1 # [ 0.412172] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046client # [ 0.388993] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1047lb # [ 0.386142] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1048worker1 # [ 0.412574] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1049lb # [ 0.386158] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1050client # [ 0.389457] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1051worker1 # [ 0.412758] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1052client # [ 0.389641] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1053lb # [ 0.386187] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054worker1 # [ 0.413005] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1055client # [ 0.389658] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1056lb # [ 0.386641] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1057worker1 # [ 0.413278] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1058client # [ 0.389688] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059lb # [ 0.386823] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1060worker1 # [ 0.413294] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1061lb # [ 0.386838] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1062client # [ 0.390195] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1063worker1 # [ 0.413323] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064client # [ 0.390380] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1065lb # [ 0.386868] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1066worker1 # [ 0.413789] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1067worker2 # [ 0.404016] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1068client # [ 0.390396] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1069worker1 # [ 0.413976] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1070worker2 # [ 0.404221] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1071lb # [ 0.387344] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1072client # [ 0.390426] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1073lb # [ 0.387532] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1074worker1 # [ 0.413992] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1075worker2 # [ 0.404238] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1076client # [ 0.390443] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1077lb # [ 0.387547] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1078worker2 # [ 0.404267] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1079worker1 # [ 0.414022] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1080client # [ 0.390904] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1081lb # [ 0.387578] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1082worker1 # [ 0.414600] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1083worker2 # [ 0.404785] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1084client # [ 0.391090] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1085lb # [ 0.387594] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1086worker1 # [ 0.414612] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1087worker2 # [ 0.404970] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1088client # [ 0.391120] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089lb # [ 0.388093] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1090worker2 # [ 0.404986] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1091worker1 # [ 0.414618] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1092client # [ 0.391586] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1093lb # [ 0.388281] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1094worker2 # [ 0.405019] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1095worker1 # [ 0.414664] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1096client # [ 0.391775] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1097lb # [ 0.388310] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098worker2 # [ 0.405483] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1099client # [ 0.391805] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100worker1 # [ 0.414712] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1101worker2 # [ 0.405665] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1102lb # [ 0.388767] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1103worker2 # [ 0.405680] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1104client # [ 0.392200] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1105worker1 # [ 0.414757] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1106client # [ 0.392386] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1107worker2 # [ 0.405709] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108worker1 # [ 0.414802] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1109client # [ 0.392640] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1110worker2 # [ 0.406160] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1111worker1 # [ 0.414848] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1112client # [ 0.392916] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1113worker2 # [ 0.406344] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1114worker1 # [ 0.414895] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1115client # [ 0.392934] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1116worker2 # [ 0.406359] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1117client # [ 0.392963] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118worker2 # [ 0.406388] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1119worker1 # [ 0.414941] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1120client # [ 0.393427] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1121worker2 # [ 0.406855] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1122worker1 # [ 0.414988] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1123client # [ 0.393612] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1124worker2 # [ 0.407042] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1125worker1 # [ 0.415035] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1126client # [ 0.393628] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1127worker2 # [ 0.407058] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1128worker1 # [ 0.415095] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1129client # [ 0.393658] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1130worker2 # [ 0.407088] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1131worker2 # [ 0.407104] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1132worker1 # [ 0.415142] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1133worker1 # [ 0.415164] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1134worker2 # [ 0.407568] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1135lb # [ 0.432991] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1136worker1 # [ 0.415185] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1137worker2 # [ 0.407757] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1138lb # [ 0.433034] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1139worker1 # [ 0.415207] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1140worker2 # [ 0.407788] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1141lb # [ 0.433484] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1142worker1 # [ 0.415230] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1143worker2 # [ 0.408273] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1144lb # [ 0.433668] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1145worker1 # [ 0.415252] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1146worker2 # [ 0.408463] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1147lb # [ 0.433918] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1148worker1 # [ 0.415273] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1149worker2 # [ 0.408493] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1150lb # [ 0.434182] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1151worker1 # [ 0.415295] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1152worker2 # [ 0.408906] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1153lb # [ 0.434199] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1154worker1 # [ 0.415317] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1155worker2 # [ 0.409091] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1156lb # [ 0.434228] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1157worker1 # [ 0.415339] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1158worker2 # [ 0.409337] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1159lb # [ 0.434697] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1160worker1 # [ 0.415361] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1161worker2 # [ 0.409617] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1162lb # [ 0.434883] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1163worker1 # [ 0.415383] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1164worker2 # [ 0.409634] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1165lb # [ 0.434899] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1166worker1 # [ 0.415404] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1167worker2 # [ 0.409663] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1168lb # [ 0.434929] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1169worker1 # [ 0.415425] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1170worker2 # [ 0.410166] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1171worker1 # [ 0.415445] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1172lb # [ 0.435518] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1173worker2 # [ 0.410355] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1174worker1 # [ 0.415466] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1175lb # [ 0.435529] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1176worker2 # [ 0.410371] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1177worker1 # [ 0.415486] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1178client # [ 0.434352] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1179lb # [ 0.435535] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1180worker2 # [ 0.410401] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1181client # [ 0.434378] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1182worker2 # [ 0.410989] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1183lb # [ 0.435581] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1184worker1 # [ 0.415508] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1185client # [ 0.434384] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1186worker2 # [ 0.411000] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1187lb # [ 0.435628] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1188client # [ 0.434440] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1189worker2 # [ 0.411006] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1190lb # [ 0.435676] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1191client # [ 0.434491] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1192worker2 # [ 0.411050] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1193lb # [ 0.435723] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1194client # [ 0.434540] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1195worker2 # [ 0.411097] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1196lb # [ 0.435770] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1197client # [ 0.434590] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1198worker2 # [ 0.411142] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1199lb # [ 0.435818] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1200client # [ 0.434638] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1201worker2 # [ 0.411187] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1202lb # [ 0.435866] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1203worker2 # [ 0.411234] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1204client # [ 0.434688] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1205lb # [ 0.435914] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1206client # [ 0.434738] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1207worker2 # [ 0.411285] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1208lb # [ 0.435960] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1209client # [ 0.434788] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1210worker2 # [ 0.411332] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1211worker1 # [ 0.475551] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1212lb # [ 0.436032] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1213worker1 # [ 0.475592] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1214client # [ 0.434837] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1215worker2 # [ 0.411379] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1216lb # [ 0.436078] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1217worker1 # [ 0.475614] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1218client # [ 0.434960] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1219worker2 # [ 0.411426] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1220lb # [ 0.436099] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1221worker1 # [ 0.475644] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1222client # [ 0.435009] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1223worker2 # [ 0.411497] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1224lb # [ 0.436121] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1225worker1 # [ 0.475654] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1226client # [ 0.435031] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1227worker2 # [ 0.411544] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1228lb # [ 0.436142] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1229worker1 # [ 0.475659] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1230client # [ 0.435053] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1231worker1 # [ 0.476852] pci 0000:00:0a.0: enabling device (0000 -> 0002)1232worker2 # [ 0.411566] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1233lb # [ 0.436167] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1234client # [ 0.435075] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1235lb # [ 0.436188] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1236worker2 # [ 0.411588] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1237client # [ 0.435101] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1238worker2 # [ 0.411610] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1239lb # [ 0.436210] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1240client # [ 0.435123] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1241lb # [ 0.436232] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1242worker2 # [ 0.411634] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1243client # [ 0.435145] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1244worker2 # [ 0.411656] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1245lb # [ 0.436253] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1246client # [ 0.435167] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1247worker2 # [ 0.411678] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1248lb # [ 0.436275] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1249client # [ 0.435189] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1250worker2 # [ 0.411700] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1251lb # [ 0.436297] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1252client # [ 0.435211] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1253worker2 # [ 0.411722] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1254lb # [ 0.436320] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1255client # [ 0.435235] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1256lb # [ 0.436341] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1257worker2 # [ 0.411744] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1258client # [ 0.435257] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1259lb # [ 0.436363] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1260worker2 # [ 0.411767] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1261client # [ 0.435279] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1262lb # [ 0.436385] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1263worker2 # [ 0.411789] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1264client # [ 0.435300] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1265lb # [ 0.436406] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1266worker2 # [ 0.411811] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1267client # [ 0.435321] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1268lb # [ 0.436428] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1269client # [ 0.435343] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1270lb # [ 0.436450] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1271client # [ 0.435364] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1272lb # [ 0.436471] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1273client # [ 0.435386] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1274lb # [ 0.436493] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1275client # [ 0.435407] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1276client # [ 0.435429] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1277lb # [ 0.436515] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1278client # [ 0.435456] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1279lb # [ 0.436540] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1280client # [ 0.435484] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1281lb # [ 0.436549] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1282client # [ 0.435494] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1283lb # [ 0.436554] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1284client # [ 0.435499] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1285client # [ 0.436677] pci 0000:00:0a.0: enabling device (0000 -> 0002)1286worker2 # [ 0.467882] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1287worker2 # [ 0.467920] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1288worker2 # [ 0.467942] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1289worker2 # [ 0.467965] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1290worker2 # [ 0.467987] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1291worker2 # [ 0.468009] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1292worker2 # [ 0.468032] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1293worker2 # [ 0.468055] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1294worker2 # [ 0.468085] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1295worker2 # [ 0.468095] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1296worker1 # [ 0.506426] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1297worker2 # [ 0.468100] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1298lb # [ 0.485807] pci 0000:00:0a.0: enabling device (0000 -> 0002)1299worker2 # [ 0.469310] pci 0000:00:0a.0: enabling device (0000 -> 0002)1300worker1 # [ 0.517379] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1301client # [ 0.488047] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1302worker1 # [ 0.520566] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1303worker1 # [ 0.522930] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1304lb # [ 0.507455] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1305client # [ 0.498317] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1306worker2 # [ 0.502609] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1307worker1 # [ 0.532984] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1308worker1 # [ 0.535312] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1309client # [ 0.506327] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1310worker2 # [ 0.509041] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1311client # [ 0.508433] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1312lb # [ 0.517512] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1313worker2 # [ 0.511177] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1314worker1 # [ 0.547268] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1315client # [ 0.518728] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1316worker2 # [ 0.521361] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1317client # [ 0.521268] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1318worker2 # [ 0.523343] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1319worker1 # [ 0.550502] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1320lb # [ 0.529439] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1321lb # [ 0.531583] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1322worker1 # [ 0.557679] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1323lb # [ 0.533643] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1324lb # [ 0.535896] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1325client # [ 0.532841] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1326worker2 # [ 0.535387] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1327worker1 # [ 0.564752] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1328client # [ 0.535922] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1329worker2 # [ 0.538559] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1330lb # [ 0.546329] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1331lb # [ 0.548559] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1332worker1 # [ 0.571682] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1333worker2 # [ 0.545120] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1334lb # [ 0.560406] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1335client # [ 0.546290] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1336worker1 # [ 0.584876] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1337client # [ 0.548245] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1338client # [ 0.550592] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1339worker2 # [ 0.556712] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1340worker2 # [ 0.558581] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1341worker1 # [ 0.587658] msm_serial: driver initialized1342worker1 # [ 0.587845] SuperH (H)SCI(F) driver initialized1343worker1 # [ 0.587900] STM32 USART driver initialized1344lb # [ 0.570193] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1345client # [ 0.563928] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1346lb # [ 0.575178] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1347client # [ 0.566501] msm_serial: driver initialized1348client # [ 0.566672] SuperH (H)SCI(F) driver initialized1349worker2 # [ 0.570501] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1350client # [ 0.566734] STM32 USART driver initialized1351lb # [ 0.586110] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1352worker2 # [ 0.585164] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1353lb # [ 0.594120] msm_serial: driver initialized1354lb # [ 0.594289] SuperH (H)SCI(F) driver initialized1355lb # [ 0.594345] STM32 USART driver initialized1356worker2 # [ 0.587964] msm_serial: driver initialized1357worker2 # [ 0.588101] SuperH (H)SCI(F) driver initialized1358worker2 # [ 0.588155] STM32 USART driver initialized1359worker1 # [ 0.622510] loop: module loaded1360worker1 # [ 0.622699] virtio_blk virtio5: 1/0/0 default/read/poll queues1361worker1 # [ 0.623457] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1362worker1 # [ 0.629079] megasas: 07.734.00.00-rc11363worker1 # [ 0.629793] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1364client # [ 0.602780] loop: module loaded1365client # [ 0.602958] virtio_blk virtio5: 1/0/0 default/read/poll queues1366client # [ 0.603712] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1367worker1 # [ 0.639576] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001368worker1 # [ 0.639602] Intel/Sharp Extended Query Table at 0x00311369worker1 # [ 0.642393] Using buffer write method1370lb # [ 0.623949] loop: module loaded1371worker1 # [ 0.642465] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1372lb # [ 0.624128] virtio_blk virtio5: 1/0/0 default/read/poll queues1373lb # [ 0.626078] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1374worker2 # [ 0.621107] loop: module loaded1375worker2 # [ 0.621308] virtio_blk virtio5: 1/0/0 default/read/poll queues1376client # [ 0.614735] megasas: 07.734.00.00-rc11377worker2 # [ 0.622049] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1378client # [ 0.615444] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1379worker1 # [ 0.652206] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001380worker1 # [ 0.652261] Intel/Sharp Extended Query Table at 0x00311381client # [ 0.617408] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001382worker1 # [ 0.653965] Using buffer write method1383worker1 # [ 0.653993] Concatenating MTD devices:1384worker1 # [ 0.653997] (0): "0.flash"1385worker1 # [ 0.654001] (1): "0.flash"1386client # [ 0.617433] Intel/Sharp Extended Query Table at 0x00311387worker1 # [ 0.654004] into device "0.flash"1388client # [ 0.619262] Using buffer write method1389client # [ 0.619328] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1390client # [ 0.621105] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001391client # [ 0.621137] Intel/Sharp Extended Query Table at 0x00311392worker2 # [ 0.628451] megasas: 07.734.00.00-rc11393worker2 # [ 0.629174] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1394worker2 # [ 0.631451] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001395worker2 # [ 0.631477] Intel/Sharp Extended Query Table at 0x00311396lb # [ 0.637521] megasas: 07.734.00.00-rc11397lb # [ 0.638264] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1398lb # [ 0.640649] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001399lb # [ 0.640676] Intel/Sharp Extended Query Table at 0x00311400lb # [ 0.642353] Using buffer write method1401lb # [ 0.643299] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1402client # [ 0.641914] Using buffer write method1403worker2 # [ 0.640948] Using buffer write method1404client # [ 0.641955] Concatenating MTD devices:1405client # [ 0.641959] (0): "0.flash"1406client # [ 0.641964] (1): "0.flash"1407worker2 # [ 0.641037] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1408client # [ 0.641967] into device "0.flash"1409lb # [ 0.656891] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001410lb # [ 0.656925] Intel/Sharp Extended Query Table at 0x00311411lb # [ 0.658606] Using buffer write method1412lb # [ 0.658636] Concatenating MTD devices:1413lb # [ 0.658640] (0): "0.flash"1414lb # [ 0.658644] (1): "0.flash"1415lb # [ 0.658648] into device "0.flash"1416worker2 # [ 0.652319] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417worker2 # [ 0.652360] Intel/Sharp Extended Query Table at 0x00311418worker2 # [ 0.653760] Using buffer write method1419worker2 # [ 0.653789] Concatenating MTD devices:1420worker2 # [ 0.653793] (0): "0.flash"1421worker2 # [ 0.653798] (1): "0.flash"1422worker2 # [ 0.653801] into device "0.flash"1423worker1 # [ 0.922036] Freeing initrd memory: 26148K1424worker1 # [ 0.929287] tun: Universal TUN/TAP device driver, 1.61425worker1 # [ 0.933207] thunder_xcv, ver 1.01426worker1 # [ 0.933251] thunder_bgx, ver 1.01427worker1 # [ 0.933273] nicpf, ver 1.01428worker1 # [ 0.933841] e1000: Intel(R) PRO/1000 Network Driver1429worker1 # [ 0.933852] e1000: Copyright (c) 1999-2006 Intel Corporation.1430worker1 # [ 0.933878] e1000e: Intel(R) PRO/1000 Network Driver1431worker1 # [ 0.933886] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432worker1 # [ 0.933915] igb: Intel(R) Gigabit Ethernet Network Driver1433worker1 # [ 0.933920] igb: Copyright (c) 2007-2014 Intel Corporation.1434worker1 # [ 0.933944] igbvf: Intel(R) Gigabit Virtual Function Network Driver1435client # [ 0.909848] Freeing initrd memory: 26160K1436worker1 # [ 0.933949] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1437worker1 # [ 0.934098] sky2: driver version 1.301438worker1 # [ 0.936258] ehci-pci 0000:00:0a.0: EHCI Host Controller1439worker1 # [ 0.936284] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11440worker1 # [ 0.936494] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001441lb # [ 0.924198] Freeing initrd memory: 26140K1442client # [ 0.916137] tun: Universal TUN/TAP device driver, 1.61443worker2 # [ 0.918407] Freeing initrd memory: 26148K1444worker1 # [ 0.946181] usbcore: registered new interface driver usb-storage1445worker1 # [ 0.946320] usbcore: registered new interface driver usbserial_generic1446worker1 # [ 0.946343] usbserial: USB Serial support registered for generic1447worker1 # [ 0.946987] hv_vmbus: registering driver hyperv_keyboard1448worker1 # [ 0.947578] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001449lb # [ 0.930659] tun: Universal TUN/TAP device driver, 1.61450worker1 # [ 0.947920] hub 1-0:1.0: USB hub found1451worker1 # [ 0.947955] hub 1-0:1.0: 6 ports detected1452client # [ 0.919982] thunder_xcv, ver 1.01453client # [ 0.920029] thunder_bgx, ver 1.01454client # [ 0.920053] nicpf, ver 1.01455worker2 # [ 0.924863] tun: Universal TUN/TAP device driver, 1.61456worker1 # [ 0.953353] rtc-pl031 9010000.pl031: registered as rtc01457client # [ 0.920630] e1000: Intel(R) PRO/1000 Network Driver1458client # [ 0.920642] e1000: Copyright (c) 1999-2006 Intel Corporation.1459worker1 # [ 0.953379] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T13:34:15 UTC (1789911255)1460worker1 # [ 0.953740] i2c_dev: i2c /dev entries driver1461client # [ 0.920678] e1000e: Intel(R) PRO/1000 Network Driver1462client # [ 0.920693] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1463client # [ 0.920730] igb: Intel(R) Gigabit Ethernet Network Driver1464client # [ 0.920735] igb: Copyright (c) 2007-2014 Intel Corporation.1465lb # [ 0.934454] thunder_xcv, ver 1.01466client # [ 0.920759] igbvf: Intel(R) Gigabit Virtual Function Network Driver1467lb # [ 0.934500] thunder_bgx, ver 1.01468lb # [ 0.934522] nicpf, ver 1.01469client # [ 0.920765] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1470lb # [ 0.935099] e1000: Intel(R) PRO/1000 Network Driver1471client # [ 0.920902] sky2: driver version 1.301472lb # [ 0.935106] e1000: Copyright (c) 1999-2006 Intel Corporation.1473client # [ 0.922839] ehci-pci 0000:00:0a.0: EHCI Host Controller1474lb # [ 0.935129] e1000e: Intel(R) PRO/1000 Network Driver1475client # [ 0.922872] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11476worker2 # [ 0.928801] thunder_xcv, ver 1.01477lb # [ 0.935136] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1478client # [ 0.923040] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001479worker2 # [ 0.928845] thunder_bgx, ver 1.01480lb # [ 0.935167] igb: Intel(R) Gigabit Ethernet Network Driver1481worker2 # [ 0.928868] nicpf, ver 1.01482worker2 # [ 0.929433] e1000: Intel(R) PRO/1000 Network Driver1483lb # [ 0.935172] igb: Copyright (c) 2007-2014 Intel Corporation.1484worker1 # [ 0.959899] sdhci: Secure Digital Host Controller Interface driver1485worker2 # [ 0.929441] e1000: Copyright (c) 1999-2006 Intel Corporation.1486worker1 # [ 0.959915] sdhci: Copyright(c) Pierre Ossman1487lb # [ 0.935199] igbvf: Intel(R) Gigabit Virtual Function Network Driver1488worker2 # [ 0.929466] e1000e: Intel(R) PRO/1000 Network Driver1489lb # [ 0.935205] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1490worker1 # [ 0.960207] Synopsys Designware Multimedia Card Interface Driver1491lb # [ 0.935345] sky2: driver version 1.301492worker2 # [ 0.929473] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1493worker1 # [ 0.960588] sdhci-pltfm: SDHCI platform and OF driver helper1494client # [ 0.932846] usbcore: registered new interface driver usb-storage1495lb # [ 0.937566] ehci-pci 0000:00:0a.0: EHCI Host Controller1496worker2 # [ 0.929505] igb: Intel(R) Gigabit Ethernet Network Driver1497worker1 # [ 0.962373] hid: raw HID events driver (C) Jiri Kosina1498client # [ 0.932943] usbcore: registered new interface driver usbserial_generic1499worker2 # [ 0.929511] igb: Copyright (c) 2007-2014 Intel Corporation.1500lb # [ 0.937596] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11501worker1 # [ 0.962623] usbcore: registered new interface driver usbhid1502client # [ 0.932959] usbserial: USB Serial support registered for generic1503worker1 # [ 0.962629] usbhid: USB HID core driver1504worker2 # [ 0.929535] igbvf: Intel(R) Gigabit Virtual Function Network Driver1505lb # [ 0.937871] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001506client # [ 0.933631] hv_vmbus: registering driver hyperv_keyboard1507worker2 # [ 0.929541] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1508client # [ 0.934193] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001509worker2 # [ 0.929695] sky2: driver version 1.301510client # [ 0.934502] hub 1-0:1.0: USB hub found1511worker2 # [ 0.931385] usbcore: registered new interface driver usb-storage1512client # [ 0.934522] hub 1-0:1.0: 6 ports detected1513lb # [ 0.947388] usbcore: registered new interface driver usb-storage1514worker2 # [ 0.931483] usbcore: registered new interface driver usbserial_generic1515worker1 # [ 0.968187] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1516lb # [ 0.947482] usbcore: registered new interface driver usbserial_generic1517worker2 # [ 0.931498] usbserial: USB Serial support registered for generic1518worker1 # [ 0.969721] drop_monitor: Initializing network drop monitor service1519lb # [ 0.947496] usbserial: USB Serial support registered for generic1520worker2 # [ 0.932544] ehci-pci 0000:00:0a.0: EHCI Host Controller1521client # [ 0.939683] rtc-pl031 9010000.pl031: registered as rtc01522worker1 # [ 0.969870] NET: Registered PF_INET6 protocol family1523lb # [ 0.948108] hv_vmbus: registering driver hyperv_keyboard1524worker2 # [ 0.932574] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11525client # [ 0.939714] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T13:34:15 UTC (1789911255)1526worker1 # [ 0.973021] Segment Routing with IPv61527worker2 # [ 0.932769] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001528client # [ 0.940071] i2c_dev: i2c /dev entries driver1529worker1 # [ 0.973042] In-situ OAM (IOAM) with IPv61530lb # [ 0.951734] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001531lb # [ 0.952046] hub 1-0:1.0: USB hub found1532worker1 # [ 0.973074] NET: Registered PF_PACKET protocol family1533lb # [ 0.952114] hub 1-0:1.0: 6 ports detected1534worker2 # [ 0.944276] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001535worker2 # [ 0.944671] hub 1-0:1.0: USB hub found1536worker1 # [ 0.974723] 9pnet: Installing 9P2000 support1537worker2 # [ 0.944708] hub 1-0:1.0: 6 ports detected1538lb # [ 0.954384] rtc-pl031 9010000.pl031: registered as rtc01539client # [ 0.944974] sdhci: Secure Digital Host Controller Interface driver1540worker2 # [ 0.947158] hv_vmbus: registering driver hyperv_keyboard1541client # [ 0.945598] sdhci: Copyright(c) Pierre Ossman1542lb # [ 0.954412] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T13:34:15 UTC (1789911255)1543lb # [ 0.954757] i2c_dev: i2c /dev entries driver1544client # [ 0.945868] Synopsys Designware Multimedia Card Interface Driver1545worker2 # [ 0.948742] rtc-pl031 9010000.pl031: registered as rtc01546worker1 # [ 0.977479] Key type dns_resolver registered1547worker2 # [ 0.948771] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T13:34:15 UTC (1789911255)1548client # [ 0.948247] sdhci-pltfm: SDHCI platform and OF driver helper1549worker2 # [ 0.949085] i2c_dev: i2c /dev entries driver1550client # [ 0.950043] hid: raw HID events driver (C) Jiri Kosina1551lb # [ 0.959889] sdhci: Secure Digital Host Controller Interface driver1552lb # [ 0.959903] sdhci: Copyright(c) Pierre Ossman1553lb # [ 0.960176] Synopsys Designware Multimedia Card Interface Driver1554client # [ 0.951629] usbcore: registered new interface driver usbhid1555client # [ 0.951639] usbhid: USB HID core driver1556lb # [ 0.960558] sdhci-pltfm: SDHCI platform and OF driver helper1557worker2 # [ 0.954175] sdhci: Secure Digital Host Controller Interface driver1558worker2 # [ 0.954190] sdhci: Copyright(c) Pierre Ossman1559worker2 # [ 0.954491] Synopsys Designware Multimedia Card Interface Driver1560worker2 # [ 0.954919] sdhci-pltfm: SDHCI platform and OF driver helper1561lb # [ 0.965095] hid: raw HID events driver (C) Jiri Kosina1562client # [ 0.953952] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1563worker1 # [ 0.984102] registered taskstats version 11564lb # [ 0.965352] usbcore: registered new interface driver usbhid1565worker1 # [ 0.984266] Loading compiled-in X.509 certificates1566lb # [ 0.965362] usbhid: USB HID core driver1567client # [ 0.955509] drop_monitor: Initializing network drop monitor service1568client # [ 0.956473] NET: Registered PF_INET6 protocol family1569worker2 # [ 0.959403] hid: raw HID events driver (C) Jiri Kosina1570worker2 # [ 0.959674] usbcore: registered new interface driver usbhid1571lb # [ 0.968287] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1572client # [ 0.959726] Segment Routing with IPv61573worker2 # [ 0.959684] usbhid: USB HID core driver1574client # [ 0.959746] In-situ OAM (IOAM) with IPv61575client # [ 0.959788] NET: Registered PF_PACKET protocol family1576lb # [ 0.971031] drop_monitor: Initializing network drop monitor service1577client # [ 0.961430] 9pnet: Installing 9P2000 support1578lb # [ 0.971231] NET: Registered PF_INET6 protocol family1579worker1 # [ 0.992805] Demotion targets for Node 0: null1580worker1 # [ 0.992918] Key type .fscrypt registered1581worker2 # [ 0.962635] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1582worker1 # [ 0.992924] Key type fscrypt-provisioning registered1583lb # [ 0.973382] Segment Routing with IPv61584lb # [ 0.973402] In-situ OAM (IOAM) with IPv61585worker1 # [ 0.993024] ima: No TPM chip found, activating TPM-bypass!1586worker2 # [ 0.965344] drop_monitor: Initializing network drop monitor service1587lb # [ 0.973434] NET: Registered PF_PACKET protocol family1588worker1 # [ 0.993044] ima: Allocated hash algorithm: sha11589worker2 # [ 0.965502] NET: Registered PF_INET6 protocol family1590worker1 # [ 0.993066] ima: No architecture policies found1591client # [ 0.964132] Key type dns_resolver registered1592worker2 # [ 0.967581] Segment Routing with IPv61593worker1 # [ 0.997177] input: gpio-keys as /devices/platform/gpio-keys/input/input01594worker2 # [ 0.967600] In-situ OAM (IOAM) with IPv61595lb # [ 0.975089] 9pnet: Installing 9P2000 support1596worker2 # [ 0.967632] NET: Registered PF_PACKET protocol family1597worker2 # [ 0.969483] 9pnet: Installing 9P2000 support1598lb # [ 0.977792] Key type dns_resolver registered1599worker2 # [ 0.971672] Key type dns_resolver registered1600client # [ 0.970541] registered taskstats version 11601client # [ 0.970719] Loading compiled-in X.509 certificates1602lb # [ 0.984197] registered taskstats version 11603lb # [ 0.984373] Loading compiled-in X.509 certificates1604worker2 # [ 0.978773] registered taskstats version 11605worker2 # [ 0.978958] Loading compiled-in X.509 certificates1606client # [ 0.979536] Demotion targets for Node 0: null1607client # [ 0.979656] Key type .fscrypt registered1608client # [ 0.979663] Key type fscrypt-provisioning registered1609client # [ 0.979770] ima: No TPM chip found, activating TPM-bypass!1610client # [ 0.979793] ima: Allocated hash algorithm: sha11611client # [ 0.979816] ima: No architecture policies found1612lb # [ 0.993253] Demotion targets for Node 0: null1613client # [ 0.984009] input: gpio-keys as /devices/platform/gpio-keys/input/input01614lb # [ 0.993370] Key type .fscrypt registered1615worker1 # [ 1.015221] clk: Disabling unused clocks1616lb # [ 0.993376] Key type fscrypt-provisioning registered1617worker1 # [ 1.015268] PM: genpd: Disabling unused power domains1618lb # [ 0.993473] ima: No TPM chip found, activating TPM-bypass!1619lb # [ 0.993493] ima: Allocated hash algorithm: sha11620worker2 # [ 0.987679] Demotion targets for Node 0: null1621lb # [ 0.993515] ima: No architecture policies found1622worker2 # [ 0.987794] Key type .fscrypt registered1623worker2 # [ 0.987804] Key type fscrypt-provisioning registered1624lb # [ 0.998086] input: gpio-keys as /devices/platform/gpio-keys/input/input01625worker2 # [ 0.989883] ima: No TPM chip found, activating TPM-bypass!1626worker1 # [ 1.019747] Freeing unused kernel memory: 4736K1627worker2 # [ 0.989912] ima: Allocated hash algorithm: sha11628worker1 # [ 1.019943] Run /init as init process1629worker2 # [ 0.989937] ima: No architecture policies found1630worker2 # [ 0.992480] input: gpio-keys as /devices/platform/gpio-keys/input/input01631client # [ 1.003137] clk: Disabling unused clocks1632client # [ 1.003169] PM: genpd: Disabling unused power domains1633lb # [ 1.016698] clk: Disabling unused clocks1634worker1 # [ 1.037986] systemd[1]: Successfully made /usr/ read-only.1635lb # [ 1.016736] PM: genpd: Disabling unused power domains1636client # [ 1.007636] Freeing unused kernel memory: 4736K1637client # [ 1.007816] Run /init as init process1638worker2 # [ 1.011557] clk: Disabling unused clocks1639worker2 # [ 1.011596] PM: genpd: Disabling unused power domains1640lb # [ 1.021189] Freeing unused kernel memory: 4736K1641lb # [ 1.021386] Run /init as init process1642worker2 # [ 1.016048] Freeing unused kernel memory: 4736K1643worker2 # [ 1.016257] Run /init as init process1644client # [ 1.024488] systemd[1]: Successfully made /usr/ read-only.1645lb # [ 1.039262] systemd[1]: Successfully made /usr/ read-only.1646worker2 # [ 1.035203] systemd[1]: Successfully made /usr/ read-only.1647worker1 # [ 1.195628] usb 1-1: new high-speed USB device number 2 using ehci-pci1648client # [ 1.182151] usb 1-1: new high-speed USB device number 2 using ehci-pci1649lb # [ 1.196942] usb 1-1: new high-speed USB device number 2 using ehci-pci1650worker2 # [ 1.191913] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker1 # [ 1.348138] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652client # [ 1.334531] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653lb # [ 1.349239] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11654worker2 # [ 1.344256] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11655worker1 # [ 1.372638] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1656worker1 # [ 1.385108] systemd[1]: Detected virtualization qemu.1657worker1 # [ 1.387263] systemd[1]: Detected architecture arm64.1658worker1 # [ 1.389333] systemd[1]: Running in initrd.1659worker1 # [ 1.392288] systemd[1]: Initializing machine ID from random generator.1660worker1 # [ 1.395292] systemd[1]: Hostname set to <worker1>.1661client # [ 1.359429] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1662client # [ 1.371750] systemd[1]: Detected virtualization qemu.1663client # [ 1.373855] systemd[1]: Detected architecture arm64.1664client # [ 1.374702] systemd[1]: Running in initrd.1665lb # [ 1.374273] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1666client # [ 1.375918] systemd[1]: Initializing machine ID from random generator.1667client # [ 1.377165] systemd[1]: Hostname set to <client>.1668lb # [ 1.386807] systemd[1]: Detected virtualization qemu.1669lb # [ 1.388951] systemd[1]: Detected architecture arm64.1670worker2 # [ 1.370368] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1671lb # [ 1.390901] systemd[1]: Running in initrd.1672worker2 # [ 1.382659] systemd[1]: Detected virtualization qemu.1673worker2 # [ 1.385201] systemd[1]: Detected architecture arm64.1674lb # [ 1.393719] systemd[1]: Initializing machine ID from random generator.1675worker2 # [ 1.387180] systemd[1]: Running in initrd.1676lb # [ 1.396630] systemd[1]: Hostname set to <lb>.1677worker2 # [ 1.390107] systemd[1]: Initializing machine ID from random generator.1678worker2 # [ 1.393180] systemd[1]: Hostname set to <worker2>.1679worker1 # [ 1.439938] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01680client # [ 1.422502] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681lb # [ 1.441186] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682worker2 # [ 1.444154] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker1 # [ 1.563571] usb 1-2: new high-speed USB device number 3 using ehci-pci1684client # [ 1.546197] usb 1-2: new high-speed USB device number 3 using ehci-pci1685lb # [ 1.568901] usb 1-2: new high-speed USB device number 3 using ehci-pci1686worker2 # [ 1.567896] usb 1-2: new high-speed USB device number 3 using ehci-pci1687client # [ 1.675479] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688worker1 # [ 1.710438] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689worker1 # [ 1.728468] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21690worker1 # [ 1.733549] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01691client # [ 1.706449] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21692client # [ 1.708503] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01693lb # [ 1.725266] systemd[1]: bpf-restrict-fs: LSM BPF program attached1694worker2 # [ 1.721822] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695lb # [ 1.736905] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21696lb # [ 1.741281] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01697worker2 # [ 1.739694] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21698client # [ 1.742638] systemd[1]: Queued start job for default target Initrd Default Target.1699worker2 # [ 1.748160] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01700client # [ 1.752130] systemd[1]: Created slice Slice /system/modprobe.1701client # [ 1.753087] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1702client # [ 1.754208] systemd[1]: Expecting device /dev/disk/by-label/nixos...1703client # [ 1.755107] systemd[1]: Reached target Path Units.1704client # [ 1.755737] systemd[1]: Reached target Slice Units.1705client # [ 1.756378] systemd[1]: Reached target Swaps.1706client # [ 1.756963] systemd[1]: Reached target Timer Units.1707client # [ 1.757761] systemd[1]: Listening on D-Bus System Message Bus Socket.1708client # [ 1.758775] systemd[1]: Listening on Journal Socket (/dev/log).1709client # [ 1.759646] systemd[1]: Listening on Journal Sockets.1710client # [ 1.760422] systemd[1]: Listening on udev Control Socket.1711client # [ 1.761215] systemd[1]: Listening on udev Kernel Socket.1712client # [ 1.761987] systemd[1]: Reached target Socket Units.1713client # [ 1.764112] systemd[1]: Starting Create List of Static Device Nodes...1714client # [ 1.773649] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1715client # [ 1.774748] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1716client # [ 1.787584] systemd[1]: Mounting Kernel Configuration File System...1717worker1 # [ 1.829908] systemd[1]: Queued start job for default target Initrd Default Target.1718client # [ 1.808323] systemd[1]: Starting Journal Service...1719worker1 # [ 1.842411] systemd[1]: Created slice Slice /system/modprobe.1720worker1 # [ 1.843821] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1721client # [ 1.814712] systemd[1]: Starting Load Kernel Modules...1722worker1 # [ 1.845377] systemd[1]: Expecting device /dev/disk/by-label/nixos...1723worker1 # [ 1.846549] systemd[1]: Reached target Path Units.1724client # [ 1.815559] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1725worker1 # [ 1.847446] systemd[1]: Reached target Slice Units.1726worker1 # [ 1.848454] systemd[1]: Reached target Swaps.1727worker1 # [ 1.849282] systemd[1]: Reached target Timer Units.1728worker1 # [ 1.850496] systemd[1]: Listening on D-Bus System Message Bus Socket.1729worker1 # [ 1.851868] systemd[1]: Listening on Journal Socket (/dev/log).1730worker1 # [ 1.852031] systemd[1]: Listening on Journal Sockets.1731worker1 # [ 1.852215] systemd[1]: Listening on udev Control Socket.1732worker1 # [ 1.852347] systemd[1]: Listening on udev Kernel Socket.1733lb # [ 1.835722] systemd[1]: Queued start job for default target Initrd Default Target.1734worker1 # [ 1.852373] systemd[1]: Reached target Socket Units.1735worker1 # [ 1.858962] systemd[1]: Starting Create List of Static Device Nodes...1736lb # [ 1.847049] systemd[1]: Created slice Slice /system/modprobe.1737worker1 # [ 1.867777] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1738lb # [ 1.848078] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1739worker1 # [ 1.869063] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1740lb # [ 1.849161] systemd[1]: Expecting device /dev/disk/by-label/nixos...1741lb # [ 1.849998] systemd[1]: Reached target Path Units.1742lb # [ 1.850635] systemd[1]: Reached target Slice Units.1743lb # [ 1.851297] systemd[1]: Reached target Swaps.1744lb # [ 1.851984] systemd[1]: Reached target Timer Units.1745lb # [ 1.852801] systemd[1]: Listening on D-Bus System Message Bus Socket.1746lb # [ 1.853858] systemd[1]: Listening on Journal Socket (/dev/log).1747lb # [ 1.854755] systemd[1]: Listening on Journal Sockets.1748worker2 # [ 1.846036] systemd[1]: Queued start job for default target Initrd Default Target.1749lb # [ 1.855591] systemd[1]: Listening on udev Control Socket.1750lb # [ 1.856387] systemd[1]: Listening on udev Kernel Socket.1751lb # [ 1.857133] systemd[1]: Reached target Socket Units.1752lb # [ 1.859262] systemd[1]: Starting Create List of Static Device Nodes...1753client # [ 1.850426] systemd[1]: Starting Coldplug All udev Devices...1754worker2 # [ 1.854274] systemd[1]: Created slice Slice /system/modprobe.1755client # [ 1.854287] systemd[1]: Finished Create List of Static Device Nodes.1756worker2 # [ 1.855455] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1757lb # [ 1.865663] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1758client # [ 1.856093] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1759lb # [ 1.866655] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1760worker2 # [ 1.856849] systemd[1]: Expecting device /dev/disk/by-label/nixos...1761worker2 # [ 1.856884] systemd[1]: Reached target Path Units.1762worker2 # [ 1.856903] systemd[1]: Reached target Slice Units.1763worker2 # [ 1.856923] systemd[1]: Reached target Swaps.1764worker1 # [ 1.888177] systemd[1]: Mounting Kernel Configuration File System...1765worker2 # [ 1.856944] systemd[1]: Reached target Timer Units.1766worker2 # [ 1.857132] systemd[1]: Listening on D-Bus System Message Bus Socket.1767worker2 # [ 1.857272] systemd[1]: Listening on Journal Socket (/dev/log).1768worker2 # [ 1.857401] systemd[1]: Listening on Journal Sockets.1769worker2 # [ 1.857528] systemd[1]: Listening on udev Control Socket.1770worker2 # [ 1.857637] systemd[1]: Listening on udev Kernel Socket.1771client # [ 1.863953] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1772worker2 # [ 1.857659] systemd[1]: Reached target Socket Units.1773lb # [ 1.876642] systemd[1]: Mounting Kernel Configuration File System...1774worker2 # [ 1.868155] systemd[1]: Starting Create List of Static Device Nodes...1775client # [ 1.870713] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1776worker2 # [ 1.875999] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1777worker2 # [ 1.877110] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1778client # [ 1.877055] systemd-journald[73]: Collecting audit messages is disabled.1779client # [ 1.879098] systemd[1]: Mounted Kernel Configuration File System.1780client # [ 1.879943] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1781lb # [ 1.889265] systemd[1]: Starting Journal Service...1782worker1 # [ 1.912384] systemd[1]: Starting Journal Service...1783worker2 # [ 1.883685] systemd[1]: Mounting Kernel Configuration File System...1784worker1 # [ 1.919709] systemd[1]: Starting Load Kernel Modules...1785worker1 # [ 1.920699] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1786client # [ 1.894719] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01787client # [ 1.894971] [drm] features: -virgl +edid -resource_blob -host_visible1788client # [ 1.894984] [drm] features: -context_init1789client # [ 1.895716] [drm] number of scanouts: 11790client # [ 1.895735] [drm] number of cap sets: 01791lb # [ 1.909124] systemd[1]: Starting Load Kernel Modules...1792lb # [ 1.910006] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1793client # [ 1.902618] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1794worker1 # [ 1.939817] systemd[1]: Starting Coldplug All udev Devices...1795lb # [ 1.920964] systemd[1]: Starting Coldplug All udev Devices...1796worker2 # [ 1.912137] systemd[1]: Starting Journal Service...1797worker2 # [ 1.920152] systemd[1]: Starting Load Kernel Modules...1798worker2 # [ 1.921013] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1799worker1 # [ 1.948551] systemd[1]: Finished Create List of Static Device Nodes.1800worker1 # [ 1.949635] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1801worker1 # [ 1.950058] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1802client # [ 1.918518] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1803client # [ 1.918546] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01804lb # [ 1.935198] systemd[1]: Finished Create List of Static Device Nodes.1805worker1 # [ 1.958363] systemd-journald[73]: Collecting audit messages is disabled.1806lb # [ 1.936067] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1807lb # [ 1.936355] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1808lb # [ 1.936654] systemd[1]: Mounted Kernel Configuration File System.1809worker2 # [ 1.940263] systemd[1]: Starting Coldplug All udev Devices...1810lb # [ 1.953306] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1811worker2 # [ 1.956818] systemd[1]: Finished Create List of Static Device Nodes.1812worker2 # [ 1.958908] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1813client # [ 1.950449] Console: switching to colour frame buffer device 160x501814client # [ 1.957363] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1815worker1 # [ 1.987903] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1816worker1 # [ 1.992816] systemd[1]: Mounted Kernel Configuration File System.1817worker2 # [ 1.964582] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1818lb # [ 1.980364] systemd-journald[73]: Collecting audit messages is disabled.1819client # [ 1.970915] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1820worker2 # [ 1.976424] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1821worker2 # [ 1.976909] systemd[1]: Mounted Kernel Configuration File System.1822client # [ 1.978989] systemd[1]: Starting Create Static Device Nodes in /dev...1823worker2 # [ 1.981959] systemd-journald[73]: Collecting audit messages is disabled.1824worker1 # [ 2.010709] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1825worker2 # [ 1.983223] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1826lb # [ 1.991220] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1827worker1 # [ 2.020362] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1828worker1 # [ 2.022744] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1829worker2 # [ 1.988109] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1830worker2 # [ 1.992950] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01831worker2 # [ 1.993193] [drm] features: -virgl +edid -resource_blob -host_visible1832worker2 # [ 1.993202] [drm] features: -context_init1833worker2 # [ 1.993908] [drm] number of scanouts: 11834client # [ 1.994734] systemd[1]: Finished Load Kernel Modules.1835worker2 # [ 1.993926] [drm] number of cap sets: 01836lb # [ 2.008961] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1837worker1 # [ 2.027967] systemd[1]: Starting Create Static Device Nodes in /dev...1838client # [ 2.002532] systemd[1]: Starting Apply Kernel Variables...1839worker1 # [ 2.036689] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01840worker1 # [ 2.036928] [drm] features: -virgl +edid -resource_blob -host_visible1841worker1 # [ 2.036938] [drm] features: -context_init1842worker1 # [ 2.037648] [drm] number of scanouts: 11843worker1 # [ 2.037665] [drm] number of cap sets: 01844lb # [ 2.017588] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1845worker2 # [ 2.016177] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1846lb # [ 2.027996] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01847worker2 # [ 2.016198] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01848lb # [ 2.028249] [drm] features: -virgl +edid -resource_blob -host_visible1849lb # [ 2.028259] [drm] features: -context_init1850worker1 # [ 2.051997] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1851lb # [ 2.029337] systemd[1]: Starting Create Static Device Nodes in /dev...1852worker1 # [ 2.052017] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01853lb # [ 2.037093] [drm] number of scanouts: 11854lb # [ 2.037123] [drm] number of cap sets: 01855lb # [ 2.038716] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1856lb # [ 2.038734] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01857worker2 # [ 2.044833] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1858worker2 # [ 2.047571] systemd[1]: Starting Create Static Device Nodes in /dev...1859client # [ 2.046917] systemd[1]: Finished Create Static Device Nodes in /dev.1860client # [ 2.047151] systemd[1]: Reached target Preparation for Local File Systems.1861client # [ 2.047187] systemd[1]: Reached target Local File Systems.1862worker2 # [ 2.056370] Console: switching to colour frame buffer device 160x501863client # [ 2.058671] systemd[1]: Starting Rule-based Manager for Device Events and Files...1864worker2 # [ 2.063371] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1865worker1 # [ 2.087924] Console: switching to colour frame buffer device 160x501866worker1 # [ 2.094771] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1867lb # [ 2.073191] Console: switching to colour frame buffer device 160x501868lb # [ 2.080117] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1869client # [ 2.078408] systemd[1]: Finished Apply Kernel Variables.1870worker1 # [ 2.108318] systemd[1]: Finished Create Static Device Nodes in /dev.1871worker1 # [ 2.108760] systemd[1]: Reached target Preparation for Local File Systems.1872worker1 # [ 2.108799] systemd[1]: Reached target Local File Systems.1873worker1 # [ 2.114907] systemd[1]: Starting Rule-based Manager for Device Events and Files...1874client # [ 2.072404] systemd-modules-load[74]: Inserted module 'dm_mod'1875client # [ 2.073692] systemd-modules-load[74]: Module 'virtio_balloon' is built in1876client # [ 2.074759] systemd-modules-load[74]: Module 'virtio_console' is built in1877client # [ 2.075842] systemd-modules-load[74]: Inserted module 'virtio_gpu'1878lb # [ 2.097628] systemd[1]: Finished Create Static Device Nodes in /dev.1879lb # [ 2.097898] systemd[1]: Reached target Preparation for Local File Systems.1880lb # [ 2.097939] systemd[1]: Reached target Local File Systems.1881worker2 # [ 2.092802] systemd[1]: Finished Load Kernel Modules.1882client # [ 2.091013] systemd[1]: Started Journal Service.1883lb # [ 2.103191] systemd[1]: Starting Rule-based Manager for Device Events and Files...1884worker2 # [ 2.096739] systemd[1]: Starting Apply Kernel Variables...1885worker1 # [ 2.136144] systemd[1]: Finished Load Kernel Modules.1886client # [ 2.092616] systemd-modules-load[74]: Module 'virtio_rng' is built in1887worker2 # [ 2.114395] systemd[1]: Finished Create Static Device Nodes in /dev.1888lb # [ 2.125209] systemd[1]: Finished Load Kernel Modules.1889worker2 # [ 2.116956] systemd[1]: Reached target Preparation for Local File Systems.1890worker2 # [ 2.120207] systemd[1]: Reached target Local File Systems.1891client # [ 2.106110] systemd[1]: Starting Create System Files and Directories...1892worker1 # [ 2.155878] systemd[1]: Starting Apply Kernel Variables...1893worker1 # [ 2.143418] systemd-modules-load[74]: Inserted module 'dm_mod'1894worker1 # [ 2.160288] systemd[1]: Started Journal Service.1895worker1 # [ 2.147223] systemd-modules-load[74]: Module 'virtio_balloon' is built in1896worker2 # [ 2.132215] systemd[1]: Starting Rule-based Manager for Device Events and Files...1897lb # [ 2.142172] systemd[1]: Starting Apply Kernel Variables...1898worker1 # [ 2.152368] systemd-modules-load[74]: Module 'virtio_console' is built in1899worker1 # [ 2.153480] systemd-modules-load[74]: Inserted module 'virtio_gpu'1900worker1 # [ 2.154481] systemd-modules-load[74]: Module 'virtio_rng' is built in1901worker2 # [ 2.154133] systemd[1]: Finished Apply Kernel Variables.1902worker1 # [ 2.172205] systemd[1]: Starting Create System Files and Directories...1903client # [ 2.148424] systemd-udevd[81]: Using default interface naming scheme 'v261'.1904client # [ 2.151130] systemd[1]: Finished Create System Files and Directories.1905lb # [ 2.182779] systemd-modules-load[74]: Inserted module 'dm_mod'1906worker1 # [ 2.202500] systemd[1]: Finished Apply Kernel Variables.1907lb # [ 2.196930] systemd[1]: Started Journal Service.1908worker2 # [ 2.192045] systemd[1]: Started Journal Service.1909client # [ 2.179331] systemd[1]: Started Rule-based Manager for Device Events and Files.1910worker2 # [ 2.184878] systemd-modules-load[74]: Inserted module 'dm_mod'1911worker2 # [ 2.185995] systemd-modules-load[74]: Module 'virtio_balloon' is built in1912lb # [ 2.200640] systemd-modules-load[74]: Module 'virtio_balloon' is built in1913lb # [ 2.203005] systemd-modules-load[74]: Module 'virtio_console' is built in1914worker2 # [ 2.196338] systemd-modules-load[74]: Module 'virtio_console' is built in1915worker2 # [ 2.200321] systemd-modules-load[74]: Inserted module 'virtio_gpu'1916worker2 # [ 2.201399] systemd-modules-load[74]: Module 'virtio_rng' is built in1917worker1 # [ 2.227940] systemd[1]: Finished Create System Files and Directories.1918worker2 # [ 2.208314] systemd[1]: Starting Create System Files and Directories...1919lb # [ 2.216252] systemd-modules-load[74]: Inserted module 'virtio_gpu'1920worker1 # [ 2.236581] systemd-udevd[80]: Using default interface naming scheme 'v261'.1921lb # [ 2.218871] systemd-modules-load[74]: Module 'virtio_rng' is built in1922lb # [ 2.224418] systemd[1]: Starting Create System Files and Directories...1923lb # [ 2.230841] systemd-udevd[79]: Using default interface naming scheme 'v261'.1924lb # [ 2.234187] systemd[1]: Finished Apply Kernel Variables.1925worker2 # [ 2.229467] systemd-udevd[81]: Using default interface naming scheme 'v261'.1926lb # [ 2.242062] systemd[1]: Finished Create System Files and Directories.1927worker1 # [ 2.266390] systemd[1]: Started Rule-based Manager for Device Events and Files.1928worker2 # [ 2.245877] systemd[1]: Finished Create System Files and Directories.1929client # [ 2.244177] systemd[1]: Starting Virtual Console Setup...1930lb # [ 2.266249] systemd[1]: Started Rule-based Manager for Device Events and Files.1931worker2 # [ 2.270635] systemd[1]: Started Rule-based Manager for Device Events and Files.1932client # [ 2.296519] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1933client # [ 2.300280] systemd[1]: Finished Virtual Console Setup.1934worker1 # [ 2.333209] systemd[1]: Starting Virtual Console Setup...1935lb # [ 2.334310] systemd[1]: Starting Virtual Console Setup...1936worker2 # [ 2.336108] systemd[1]: Starting Virtual Console Setup...1937worker1 # [ 2.392523] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1938worker1 # [ 2.396328] systemd[1]: Finished Virtual Console Setup.1939lb # [ 2.384655] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940lb # [ 2.388387] systemd[1]: Finished Virtual Console Setup.1941worker2 # [ 2.388587] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942worker2 # [ 2.392913] systemd[1]: Finished Virtual Console Setup.1943client # [ 2.936556] systemd[1]: Finished Coldplug All udev Devices.1944client # [ 2.938410] systemd[1]: Reached target System Initialization.1945client # [ 2.944498] systemd[1]: Reached target Basic System.1946worker1 # [ 3.042010] systemd[1]: Finished Coldplug All udev Devices.1947worker1 # [ 3.042965] systemd[1]: Reached target System Initialization.1948worker1 # [ 3.043779] systemd[1]: Reached target Basic System.1949lb # [ 3.034361] systemd[1]: Finished Coldplug All udev Devices.1950lb # [ 3.035335] systemd[1]: Reached target System Initialization.1951lb # [ 3.040202] systemd[1]: Reached target Basic System.1952worker2 # [ 3.054812] systemd[1]: Finished Coldplug All udev Devices.1953worker2 # [ 3.055803] systemd[1]: Reached target System Initialization.1954worker2 # [ 3.056736] systemd[1]: Reached target Basic System.1955client # [ 3.088496] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1956client # [ 3.132640] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1957client # [ 3.135914] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1958lb # [ 3.212171] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1959worker1 # [ 3.233070] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960client # [ 3.207022] systemd[1]: Found device /dev/disk/by-label/nixos.1961worker1 # [ 3.237378] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1962client # [ 3.212066] systemd[1]: Reached target Initrd Root Device.1963client # [ 3.218867] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1964worker1 # [ 3.251560] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1965lb # [ 3.237890] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1966worker2 # [ 3.234125] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1967lb # [ 3.242012] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1968worker2 # [ 3.248191] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969worker2 # [ 3.255912] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1970client # [ 3.276317] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1971client # [ 3.284221] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1972client # [ 3.288526] systemd[1]: Mounting /sysroot...1973lb # [ 3.306536] systemd[1]: Found device /dev/disk/by-label/nixos.1974lb # [ 3.308987] systemd[1]: Reached target Initrd Root Device.1975lb # [ 3.313226] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1976worker2 # [ 3.306813] systemd[1]: Found device /dev/disk/by-label/nixos.1977worker2 # [ 3.310942] systemd[1]: Reached target Initrd Root Device.1978worker1 # [ 3.337795] systemd[1]: Found device /dev/disk/by-label/nixos.1979worker1 # [ 3.340617] systemd[1]: Reached target Initrd Root Device.1980worker2 # [ 3.316703] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1981worker1 # [ 3.346590] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982client # [ 3.342057] EXT4-fs (vda): mounted filesystem 1ee65b1f-83cb-4ae9-a5b4-a341fcdd71bc r/w with ordered data mode. Quota mode: none.1983client # [ 3.333301] systemd[1]: Mounted /sysroot.1984client # [ 3.336877] systemd[1]: Reached target Initrd Root File System.1985client # [ 3.341098] systemd[1]: Starting Mountpoints Configured in the Real Root...1986lb # [ 3.375046] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1987worker1 # [ 3.394402] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1988client # [ 3.370252] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1989worker2 # [ 3.374998] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1990lb # [ 3.383847] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1991worker1 # [ 3.403663] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1992client # [ 3.374861] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1993worker1 # [ 3.408147] systemd[1]: Mounting /sysroot...1994worker2 # [ 3.382458] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1995client # [ 3.379953] systemd[1]: Reloading...1996lb # [ 3.392331] systemd[1]: Mounting /sysroot...1997worker2 # [ 3.388615] systemd[1]: Mounting /sysroot...1998worker1 # [ 3.463011] EXT4-fs (vda): mounted filesystem b0f1d07d-b782-457f-88d9-1c9db3ece5ab r/w with ordered data mode. Quota mode: none.1999lb # [ 3.445677] EXT4-fs (vda): mounted filesystem 8f61555e-e117-4c0c-8ac4-974b6e3e3b5d r/w with ordered data mode. Quota mode: none.2000worker1 # [ 3.455303] systemd[1]: Mounted /sysroot.
loading 1,439 hidden lines…
3440worker1 # [ 20.422165] dhcpcd[627]: eth0: leased 10.0.2.15 for 86400 seconds3441worker1 # [ 20.425729] dhcpcd[627]: eth0: adding route to 10.0.2.0/243442worker1 # [ 20.429086] dhcpcd[627]: eth0: adding default route via 10.0.2.23443worker1 # [ 20.586180] systemd[1]: Started DHCP Client.3444worker1 # [ 20.588131] systemd[1]: Reached target Multi-User System.3445worker1 # [ 20.589037] systemd[1]: Startup finished in 1.009s (kernel) + 4.829s (initrd) + 14.748s (userspace) = 20.587s.3446lb # [ 20.757543] dhcpcd[674]: eth0: leased 10.0.2.15 for 86400 seconds3447lb # [ 20.757912] dhcpcd[674]: eth0: adding route to 10.0.2.0/243448lb # [ 20.758052] dhcpcd[674]: eth0: adding default route via 10.0.2.23449lb # [ 20.981826] systemd[1]: Started DHCP Client.3450lb # [ 20.984479] systemd[1]: Reached target Network is Online.3451lb # [ 20.988785] systemd[1]: Started Envoy reverse proxy.3452lb # [ 20.993185] systemd[1]: Started mock-oidc.service.3453lb # [ 21.095170] mock-oidc-server[922]: Mock OIDC Server running3454lb # [ 21.096416] mock-oidc-server[922]: OIDC Address: 192.168.1.2:80803455lb # [ 21.097666] mock-oidc-server[922]: Issue Address: 0.0.0.0:80813456lb # [ 21.099015] mock-oidc-server[922]: Issuer: http://192.168.1.2:8080/oidc3457lb # [ 21.100586] mock-oidc-server[922]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3458lb # [ 21.101963] mock-oidc-server[922]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3459lb # [ 21.103627] mock-oidc-server[922]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3460lb # [ 21.643351] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120)3461lb # [ 21.645614] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:469] statically linked extensions:3462lb # [ 21.648555] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response3463lb # [ 21.650778] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls3464lb # [ 21.667696] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls3465lb # [ 21.672787] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query3466lb # [ 21.676244] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3467lb # [ 21.679752] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default3468lb # [ 21.683042] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83469lb # [ 21.685293] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory3470lb # [ 21.687974] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router3471lb # [ 21.718798] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3472lb # [ 21.721767] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san3473lb # [ 21.729224] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle3474lb # [ 21.731384] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter3475lb # [ 21.733602] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3476lb # [ 21.736555] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3477lb # [ 21.739730] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3478lb # [ 21.742245] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats3479lb # [ 21.749422] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3480lb # [ 21.758222] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3481lb # [ 21.760747] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router3482lb # [ 21.762591] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd3483lb # [ 21.767941] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input3484lb # [ 21.770629] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple3485lb # [ 21.773272] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3486lb # [ 21.778298] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override3487lb # [ 21.781234] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default3488lb # [ 21.783306] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert3489lb # [ 21.786620] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3490lb # [ 21.789485] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3491lb # [ 21.794374] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3492lb # [ 21.796568] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log3493lb # [ 21.803723] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector3494lb # [ 21.812291] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3495lb # [ 21.814154] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection3496lb # [ 21.819243] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3497lb # [ 21.821851] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3498lb # [ 21.825445] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re23499lb # [ 21.827226] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3500lb # [ 21.834550] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3501lb # [ 21.838149] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3502lb # [ 21.840919] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3503lb # [ 21.842964] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit3504lb # [ 21.846265] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3505lb # [ 21.850550] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo3506lb # [ 21.852378] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3507lb # [ 21.856986] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port3508lb # [ 21.865342] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3509lb # [ 21.867797] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3510lb # [ 21.870094] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy3511lb # [ 21.872946] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3512lb # [ 21.875393] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3513lb # [ 21.880257] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3514lb # [ 21.882168] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3515lb # [ 21.883865] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config3516lb # [ 21.887970] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter3517lb # [ 21.890254] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator3518lb # [ 21.893097] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection3519lb # [ 21.895105] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3520lb # [ 21.897914] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3521lb # [ 21.902725] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3522lb # [ 21.904989] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3523lb # [ 21.907070] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3524lb # [ 21.912215] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3525lb # [ 21.914869] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3526lb # [ 21.917013] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3527lb # [ 21.918917] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3528lb # [ 21.921724] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3529lb # [ 21.923611] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default3530lb # [ 21.925815] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme3531lb # [ 21.929246] envoy[921]: [2026-09-20 13:34:36.200][921][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr3532lb # [ 21.933495] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case3533lb # [ 21.936676] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3534lb # [ 21.938501] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder3535lb # [ 21.941407] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3536lb # [ 21.947132] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3537lb # [ 21.949984] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3538lb # [ 21.952439] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin3539lb # [ 21.955902] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3540lb # [ 21.961490] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable3541lb # [ 21.965245] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default3542lb # [ 21.967394] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy3543lb # [ 21.982311] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23544lb # [ 21.985073] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3545lb # [ 21.987835] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3546lb # [ 21.990666] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3547lb # [ 21.993144] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23548lb # [ 21.995776] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13549lb # [ 22.001585] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic3550lb # [ 22.005851] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3551lb # [ 22.009702] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3552lb # [ 22.013855] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3553lb # [ 22.016347] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3554lb # [ 22.019515] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3555lb # [ 22.023199] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3556lb # [ 22.025337] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift3557lb # [ 22.029146] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action3558lb # [ 22.031274] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3559lb # [ 22.033697] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3560lb # [ 22.037159] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3561lb # [ 22.048912] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3562lb # [ 22.050711] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3563lb # [ 22.055381] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal3564lb # [ 22.057551] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3565lb # [ 22.059615] envoy[921]: [2026-09-20 13:34:36.201][921][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3566lb # [ 22.062151] envoy[921]: [2026-09-20 13:34:36.223][921][info][main] [source/server/server.cc:531] HTTP header map info:3567lb # [ 22.064788] envoy[921]: [2026-09-20 13:34:36.225][921][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3568lb # [ 22.081525] envoy[921]: [2026-09-20 13:34:36.225][921][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3569lb # [ 22.083159] envoy[921]: [2026-09-20 13:34:36.225][921][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3570lb # [ 22.092311] envoy[921]: [2026-09-20 13:34:36.225][921][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3571lb # [ 22.094345] envoy[921]: [2026-09-20 13:34:36.475][921][info][main] [source/server/server.cc:968] runtime: {}3572lb # [ 22.095695] envoy[921]: [2026-09-20 13:34:36.492][921][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3573lb # [ 22.097715] envoy[921]: [2026-09-20 13:34:36.492][921][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013574lb # [ 22.100345] envoy[921]: [2026-09-20 13:34:36.492][921][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013575lb # [ 22.101918] envoy[921]: [2026-09-20 13:34:36.499][921][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3576lb # [ 22.104675] envoy[921]: [2026-09-20 13:34:36.499][921][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3577lb # [ 22.106398] envoy[921]: [2026-09-20 13:34:36.499][921][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3578lb # [ 22.123185] envoy[921]: [2026-09-20 13:34:36.680][921][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3579worker2 # [ 22.119725] systemd[1]: Reached target Network is Online.3580worker1 # [ 22.152898] systemd[1]: Starting Nix worker-protocol over gRPC...3581lb # [ 22.137076] envoy[921]: [2026-09-20 13:34:36.695][921][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3582worker2 # [ 22.131378] systemd[1]: Starting scheduler-token.service...3583lb # [ 22.140986] envoy[921]: [2026-09-20 13:34:36.698][921][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3584lb # [ 22.144701] envoy[921]: [2026-09-20 13:34:36.698][921][info][main] [source/server/server.cc:1077] starting main dispatch loop3585worker2 # [ 22.249686] systemd[1]: scheduler-token.service: Deactivated successfully.3586worker2 # [ 22.256252] systemd[1]: Finished scheduler-token.service.3587worker2 # [ 22.257665] systemd[1]: scheduler-token.service: Consumed 54ms CPU time over 121ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3588worker2 # [ 22.263204] systemd[1]: Starting Nix worker-protocol over gRPC...3589worker1 # [ 22.404625] nix-grpc-daemon[776]: ts=2026-09-20T13:34:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3590worker1 # [ 22.416726] nix-grpc-daemon[776]: ts=2026-09-20T13:34:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3591worker1 # [ 22.418938] nix-grpc-daemon[776]: ts=2026-09-20T13:34:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13592worker1 # [ 22.463953] nix-grpc-daemon[776]: ts=2026-09-20T13:34:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3593worker1 # [ 22.467620] systemd[1]: Started Nix worker-protocol over gRPC.3594worker1 # [ 22.497586] nix-grpc-daemon[776]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3595lb # [ 22.482834] envoy[921]: [2026-09-20 13:34:37.041][921][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3596lb # [ 22.490285] envoy[921]: [2026-09-20 13:34:37.046][921][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3597worker2 # [ 22.489030] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3598lb # [ 22.495860] envoy[921]: [2026-09-20 13:34:37.046][921][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3599lb # [ 22.501170] envoy[921]: [2026-09-20 13:34:37.046][921][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3600worker2 # [ 22.500859] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3601worker2 # [ 22.503105] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13602worker2 # [ 22.549767] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3603worker2 # [ 22.555345] systemd[1]: Started Nix worker-protocol over gRPC.3604worker2 # [ 22.592614] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500513605worker2 # [ 22.597291] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3606worker1 # [ 22.698767] nix-grpc-daemon[776]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3607worker2 # [ 22.799030] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500513608worker2 # [ 22.804378] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3609worker2 # [ 23.005757] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500513610worker2 # [ 23.011067] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3611worker1 # [ 23.099073] nix-grpc-daemon[776]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3612worker2 # [ 23.212315] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500513613worker2 # [ 23.217390] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3614worker2 # [ 23.418543] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500513615worker2 # [ 23.423382] nix-grpc-daemon[777]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3616worker2 # [ 23.626917] nix-grpc-daemon[777]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500513617worker2 # [ 23.631802] nix-grpc-daemon[777]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3618lb # [ 23.680540] envoy[921]: conn peer=192.168.1.3:59900 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=133619worker2 # [ 23.827964] nix-grpc-daemon[777]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500513620worker2 # [ 23.833102] nix-grpc-daemon[777]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3621worker1 # [ 23.903009] nix-grpc-daemon[776]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500513622worker1 # [ 23.907280] nix-grpc-daemon[776]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3623worker2 # [ 24.034550] nix-grpc-daemon[777]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500513624worker2 # [ 24.040078] nix-grpc-daemon[777]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3625worker1 # [ 24.104062] nix-grpc-daemon[776]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500513626worker1 # [ 24.109528] nix-grpc-daemon[776]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3627worker2 # [ 24.241415] nix-grpc-daemon[777]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500513628worker2 # [ 24.246706] nix-grpc-daemon[777]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3629worker1 # [ 24.310513] nix-grpc-daemon[776]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500513630worker1 # [ 24.316129] nix-grpc-daemon[776]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3631worker2 # [ 24.448097] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513632worker2 # [ 24.453375] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3633worker1 # [ 24.517173] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513634worker1 # [ 24.523205] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3635worker2 # [ 24.656608] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513636worker2 # [ 24.660747] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3637worker1 # [ 24.724149] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513638worker1 # [ 24.729436] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3639worker2 # [ 24.857717] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513640worker2 # [ 24.863043] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3641worker1 # [ 24.932280] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513642worker1 # [ 24.936448] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3643worker2 # [ 25.064454] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513644worker2 # [ 25.069650] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3645worker1 # [ 25.132985] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513646worker1 # [ 25.138155] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3647worker2 # [ 25.271440] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513648worker2 # [ 25.276969] nix-grpc-daemon[777]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3649worker1 # [ 25.339098] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500513650worker1 # [ 25.344616] nix-grpc-daemon[776]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3651worker2 # [ 25.478425] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513652worker2 # [ 25.484112] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3653worker1 # [ 25.545527] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513654worker1 # [ 25.551111] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3655worker2 # [ 25.687382] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513656worker2 # [ 25.691688] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3657worker1 # [ 25.752110] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513658worker1 # [ 25.757456] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3659worker2 # [ 25.888422] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513660worker2 # [ 25.893677] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3661worker1 # [ 25.960488] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513662worker1 # [ 25.964646] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3663worker2 # [ 26.095215] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513664worker2 # [ 26.100626] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3665worker1 # [ 26.161049] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513666worker1 # [ 26.166476] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3667worker2 # [ 26.302017] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513668worker2 # [ 26.307345] nix-grpc-daemon[777]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3669worker1 # [ 26.367622] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500513670worker1 # [ 26.373236] nix-grpc-daemon[776]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3671worker2 # [ 26.508947] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513672worker2 # [ 26.514179] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3673worker1 # [ 26.573963] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513674worker1 # [ 26.579288] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3675worker2 # [ 26.715498] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513676worker2 # [ 26.718273] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3677worker1 # [ 26.780233] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513678worker1 # [ 26.785659] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3679worker2 # [ 26.919723] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513680worker2 # [ 26.924986] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3681worker1 # [ 26.988371] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513682worker1 # [ 26.992098] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3683worker2 # [ 27.126101] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513684worker2 # [ 27.131230] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3685worker1 # [ 27.189279] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513686worker1 # [ 27.194664] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3687worker2 # [ 27.332566] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513688worker2 # [ 27.337663] nix-grpc-daemon[777]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3689worker1 # [ 27.395498] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500513690worker1 # [ 27.400927] nix-grpc-daemon[776]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3691worker2 # [ 27.538933] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513692worker2 # [ 27.544140] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3693worker1 # [ 27.601838] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513694worker1 # [ 27.607085] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3695worker2 # [ 27.747658] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513696worker2 # [ 27.751936] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3697worker1 # [ 27.807929] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513698worker1 # [ 27.813249] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3699worker2 # [ 27.948537] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513700worker2 # [ 27.953835] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3701worker1 # [ 28.016311] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513702worker1 # [ 28.020399] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3703worker2 # [ 28.155376] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513704worker2 # [ 28.160953] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3705worker1 # [ 28.216787] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513706worker1 # [ 28.221883] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3707worker2 # [ 28.362240] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513708worker2 # [ 28.367577] nix-grpc-daemon[777]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3709worker1 # [ 28.422954] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500513710worker1 # [ 28.428834] nix-grpc-daemon[776]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3711worker2 # [ 28.568972] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500513712worker2 # [ 28.574502] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3713worker1 # [ 28.629580] nix-grpc-daemon[776]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500513714worker1 # [ 28.634755] nix-grpc-daemon[776]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3715worker2 # [ 28.777084] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500513716worker2 # [ 28.782074] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3717worker1 # [ 28.835702] nix-grpc-daemon[776]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500513718worker1 # [ 28.841025] nix-grpc-daemon[776]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3719worker2 # [ 28.983416] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500513720worker2 # [ 28.988983] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3721worker1 # [ 29.043655] nix-grpc-daemon[776]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500513722worker1 # [ 29.048127] nix-grpc-daemon[776]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3723worker2 # [ 29.190085] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500513724worker2 # [ 29.195328] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3725worker1 # [ 29.244421] nix-grpc-daemon[776]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500513726worker1 # [ 29.249518] nix-grpc-daemon[776]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker2 # [ 29.396693] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500513728worker2 # [ 29.402399] nix-grpc-daemon[777]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3729worker1 # [ 29.450309] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513730worker1 # [ 29.455445] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker2 # [ 29.603455] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513732worker2 # [ 29.608794] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3733worker1 # [ 29.656525] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513734worker1 # [ 29.661618] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker2 # [ 29.811867] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513736worker2 # [ 29.816188] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3737worker1 # [ 29.862448] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513738worker1 # [ 29.867758] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3739worker2 # [ 30.013189] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513740worker2 # [ 30.016956] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3741worker1 # [ 30.070628] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513742worker1 # [ 30.074831] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3743worker2 # [ 30.215451] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513744worker2 # [ 30.220764] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3745worker1 # [ 30.271227] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513746worker1 # [ 30.274786] nix-grpc-daemon[776]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3747worker2 # [ 30.422223] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500513748worker2 # [ 30.427468] nix-grpc-daemon[777]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3749worker1 # [ 30.475900] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500513750worker1 # [ 30.481226] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751lb # [ 30.492828] rustfs-bucket-start[1013]: mb s3://farm3752lb # [ 30.506741] systemd[1]: Finished rustfs-bucket.service.3753lb # [ 30.526026] systemd[1]: Starting niks3 server...3754worker2 # [ 30.628684] nix-grpc-daemon[777]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500513755worker2 # [ 30.633813] nix-grpc-daemon[777]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3756worker1 # [ 30.681816] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500513757worker1 # [ 30.687068] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3758lb # [ 30.756206] postgres[1031]: [1031] ERROR: relation "goose_db_version" does not exist at character 363759lb # [ 30.757517] postgres[1031]: [1031] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3760lb # [ 30.790983] niks3-server[1024]: 2026/09/20 13:34:45 OK 20241026095416_initial_model.sql (20.54ms)3761lb # [ 30.794211] niks3-server[1024]: 2026/09/20 13:34:45 OK 20251210153512_drop_unused_gin_index.sql (3.68ms)3762lb # [ 30.799543] niks3-server[1024]: 2026/09/20 13:34:45 OK 20251218171726_add_pins.sql (5.29ms)3763lb # [ 30.804255] niks3-server[1024]: 2026/09/20 13:34:45 OK 20260628120000_add_object_size_and_stats.sql (4.67ms)3764lb # [ 30.809777] niks3-server[1024]: 2026/09/20 13:34:45 OK 20260905000000_add_claims.sql (5.48ms)3765lb # [ 30.814235] niks3-server[1024]: 2026/09/20 13:34:45 OK 20260920000000_drop_claims.sql (4.32ms)3766lb # [ 30.816317] niks3-server[1024]: 2026/09/20 13:34:45 goose: successfully migrated database to version: 202609200000003767lb # [ 30.820420] niks3-server[1024]: 2026/09/20 13:34:45 OK 1_commit_pending_closure.sql (6.17ms)3768lb # [ 30.823205] niks3-server[1024]: 2026/09/20 13:34:45 OK 2_object_stats_trigger.sql (2.63ms)3769lb # [ 30.825981] niks3-server[1024]: 2026/09/20 13:34:45 goose: up to current file version: 23770lb # [ 30.829635] niks3-server[1024]: 2026/09/20 13:34:45 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3771worker2 # [ 30.837050] nix-grpc-daemon[777]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500513772worker2 # [ 30.841320] nix-grpc-daemon[777]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3773lb # [ 30.873045] niks3-server[1024]: 2026/09/20 13:34:45 INFO Read proxy enabled — serving cache objects from S33774worker1 # [ 30.888090] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500513775lb # [ 30.874823] niks3-server[1024]: 2026/09/20 13:34:45 INFO Using socket-activated listener address=0.0.0.0:57513776lb # [ 30.878356] systemd[1]: Started niks3 server.3777worker1 # [ 30.893163] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3778lb # [ 30.881721] niks3-server[1024]: 2026/09/20 13:34:45 INFO systemd watchdog enabled interval=15s3779lb # [ 30.882941] niks3-server[1024]: 2026/09/20 13:34:45 INFO Starting HTTP server address=0.0.0.0:57513780lb # [ 30.886358] systemd[1]: Reached target Multi-User System.3781lb # [ 30.887455] systemd[1]: Startup finished in 1.013s (kernel) + 5.603s (initrd) + 24.264s (userspace) = 30.881s.3782lb # [ 30.897024] niks3-server[1024]: 2026/09/20 13:34:45 INFO lead: acquired remote=192.168.1.4:351283783worker2 # [ 30.894643] nix-grpc-daemon[777]: ts=2026-09-20T13:34:45Z level=info event=scheduler_take_over3784worker2 # [ 31.037822] nix-grpc-daemon[777]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500513785worker2 # [ 31.043151] nix-grpc-daemon[777]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3786worker1 # [ 31.095866] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500513787worker1 # [ 31.100225] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3788lb: (finished: waiting for unit niks3.service, in 16.08 seconds)3789lb: waiting for TCP port 5751 on localhost3790lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3791lb: (finished: waiting for TCP port 5751 on localhost, in 0.09 seconds)3792lb: waiting for unit envoy.service3793worker2 # [ 31.244352] nix-grpc-daemon[777]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500513794worker2 # [ 31.249510] nix-grpc-daemon[777]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3795worker1 # [ 31.297363] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500513796worker1 # [ 31.302139] nix-grpc-daemon[776]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3797lb: (finished: waiting for unit envoy.service, in 0.07 seconds)3798lb: waiting for TCP port 50051 on localhost3799lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3800lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3801worker2 # [ 31.451030] nix-grpc-daemon[777]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500513802worker2 # [ 31.458067] nix-grpc-daemon[777]: ts=2026-09-20T13:34:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3803worker1 # [ 31.503108] nix-grpc-daemon[776]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500513804worker1 # [ 31.509279] nix-grpc-daemon[776]: ts=2026-09-20T13:34:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3805worker2 # [ 31.511483] systemd[1]: Starting scheduler-token.service...3806worker2 # [ 31.607172] systemd[1]: scheduler-token.service: Deactivated successfully.3807worker2 # [ 31.612451] systemd[1]: Finished scheduler-token.service.3808worker2 # [ 31.613643] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 99ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3809lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3810??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.3811 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1283812lb: must succeed: curl -sf localhost:9901/clusters3813worker2 # [ 31.657552] nix-grpc-daemon[777]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500513814worker2 # [ 31.679932] nix-grpc-daemon[777]: ts=2026-09-20T13:34:46Z level=info event=oidc_jwks provider=mock keys=13815worker1 # [ 31.709934] nix-grpc-daemon[776]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500513816worker2 # [ 31.684290] nix-grpc-daemon[777]: ts=2026-09-20T13:34:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:525323817worker2 # [ 31.690198] nix-grpc-daemon[777]: ts=2026-09-20T13:34:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03818worker2 # [ 31.694897] nix-grpc-daemon[777]: ts=2026-09-20T13:34:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:525323819worker2 # [ 31.698587] nix-grpc-daemon[777]: ts=2026-09-20T13:34:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03820lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3821lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3822subtest: one node schedules and both builders hold a WorkerSession on it3823lb: must succeed: curl -sf localhost:9901/clusters3824lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3825worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3826worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)3827lb: must succeed: curl -sf localhost:9901/clusters3828lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3829lb: must succeed: curl -sf localhost:9901/clusters3830lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3831worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3832worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)3833(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.32 seconds)3834subtest: balancer auth: client cert, bearer token, nothing, unknown CN3835lb # [ 32.121537] envoy[921]: conn peer=[::1]:47848 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33836worker2 # [ 32.383349] nix-grpc-daemon[777]: ts=2026-09-20T13:34:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:525443837worker2 # [ 32.392676] systemd[1]: Started Nix Daemon.3838worker2 # [ 32.695843] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3839worker2 # [ 32.731311] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 1/5)3840worker2 # [ 32.799637] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)3841worker2 # [ 32.962211] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 232 ms (attempt 3/5)3842worker2 # [ 33.201896] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 673 ms (attempt 4/5)3843worker2 # [ 33.878688] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3844worker2 # [ 33.901389] nix-grpc-daemon[777]: ts=2026-09-20T13:34:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:525443845worker2 # [ 33.916167] nix-grpc-daemon[777]: ts=2026-09-20T13:34:48Z level=debug event=path_info_miss path=pf18as2xhcjh3l28kia27rgak8kjgrxc-nixos-system-client-test error="error: path '/nix/store/pf18as2xhcjh3l28kia27rgak8kjgrxc-nixos-system-client-test' is required, but there is no substituter that can build it"3846worker2 # [ 33.930005] nix-grpc-daemon[777]: ts=2026-09-20T13:34:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:525443847worker2 # [ 33.937672] nix-grpc-daemon[777]: ts=2026-09-20T13:34:48Z level=debug event=path_info_miss path=pf18as2xhcjh3l28kia27rgak8kjgrxc-nixos-system-client-test error="error: path '/nix/store/pf18as2xhcjh3l28kia27rgak8kjgrxc-nixos-system-client-test' is required, but there is no substituter that can build it"3848client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3849client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.11 seconds)3850client: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3851client: (finished: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)3852worker1 # [ 34.430383] nix-grpc-daemon[776]: ts=2026-09-20T13:34:48Z level=info event=oidc_jwks provider=mock keys=13853worker1 # [ 34.430825] nix-grpc-daemon[776]: ts=2026-09-20T13:34:48Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:411023854worker1 # [ 34.445911] systemd[1]: Started Nix Daemon.3855worker1 # [ 34.748807] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3856worker1 # [ 34.776858] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)3857worker1 # [ 34.854049] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 2/5)3858worker1 # [ 34.861478] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 331 ms (attempt 3/5)3859worker1 # [ 35.193005] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 655 ms (attempt 4/5)3860worker1 # [ 35.851842] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3861worker2 # [ 35.848876] nix-grpc-daemon[777]: ts=2026-09-20T13:34:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:525443862worker2 # [ 35.862080] nix-grpc-daemon[777]: ts=2026-09-20T13:34:50Z level=debug event=path_info_miss path=pf18as2xhcjh3l28kia27rgak8kjgrxc-nixos-system-client-test error="error: path '/nix/store/pf18as2xhcjh3l28kia27rgak8kjgrxc-nixos-system-client-test' is required, but there is no substituter that can build it"3863worker2 # [ 35.877464] nix-grpc-daemon[777]: ts=2026-09-20T13:34:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:525443864worker2 # [ 35.885686] nix-grpc-daemon[777]: ts=2026-09-20T13:34:50Z level=debug event=path_info_miss path=pf18as2xhcjh3l28kia27rgak8kjgrxc-nixos-system-client-test error="error: path '/nix/store/pf18as2xhcjh3l28kia27rgak8kjgrxc-nixos-system-client-test' is required, but there is no substituter that can build it"3865client: must succeed: rm -r /var/lib/nix-grpc-store3866client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.05 seconds)3867worker2 # [ 36.204058] nix-grpc-daemon[777]: ts=2026-09-20T13:34:50Z level=info event=denied method=QueryMissing cn=- role=none3868worker1 # [ 36.511186] nix-grpc-daemon[776]: ts=2026-09-20T13:34:51Z level=info event=denied method=QueryMissing cn=stranger role=none3869(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.48 seconds)3870subtest: DAG build is scheduled across workers and lands in the cache3871client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23872worker1 # [ 36.768223] nix-grpc-daemon[776]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:411023873client # don't know how to build these paths:3874client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3875worker2 # [ 36.757116] nix-grpc-daemon[777]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525443876worker2 # [ 36.794564] nix-grpc-daemon[777]: ts=2026-09-20T13:34:51Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03877worker2 # [ 36.802245] nix-grpc-daemon[777]: ts=2026-09-20T13:34:51Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=13878worker2 # [ 36.851355] nix-grpc-daemon[777]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525443879worker2 # [ 36.880111] nix-grpc-daemon[777]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525443880client # copying 1 paths to 'lb:50051'...3881worker2 # [ 36.899693] nix-grpc-daemon[777]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:525443882client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3883worker2 # [ 36.926198] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3884worker2 # [ 36.964162] nix-grpc-daemon[777]: ts=2026-09-20T13:34:51Z level=info event=niks3_push_started pid=8583885worker2 # [ 37.248108] nix-grpc-daemon[858]: time=2026-09-20T13:34:51.809Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""3886worker2 # [ 37.447278] nix-daemon[845]: accepted connection from pid 871, user nix-grpc-daemon (trusted)3887worker2 # [ 37.466591] nix-daemon[845]: reaped child process 874, status = succeeded3888worker2 # [ 37.649483] nix-daemon[845]: accepted connection from pid 876, user nix-grpc-daemon (trusted)3889worker2 # [ 37.669947] nix-daemon[845]: reaped child process 879, status = succeeded3890lb # [ 37.689052] niks3-server[1024]: 2026/09/20 13:34:52 INFO Received uploads request method=POST path=/api/pending_closures3891worker2 # [ 37.751007] nix-grpc-daemon[858]: time=2026-09-20T13:34:52.313Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3892worker2 # [ 37.754531] nix-grpc-daemon[858]: time=2026-09-20T13:34:52.314Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3893lb # [ 37.811860] niks3-server[1024]: 2026/09/20 13:34:52 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3894lb # [ 37.819366] niks3-server[1024]: 2026/09/20 13:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3895lb # [ 37.829599] niks3-server[1024]: 2026/09/20 13:34:52 INFO Signed narinfos id=1 count=13896worker2 # [ 37.825732] nix-grpc-daemon[858]: time=2026-09-20T13:34:52.387Z level=INFO msg="Uploading 1 narinfos"3897lb # [ 37.840608] niks3-server[1024]: 2026/09/20 13:34:52 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3898lb # [ 37.849407] niks3-server[1024]: 2026/09/20 13:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3899lb # [ 37.865768] niks3-server[1024]: 2026/09/20 13:34:52 INFO Completed upload id=13900worker2 # [ 37.858932] nix-grpc-daemon[858]: time=2026-09-20T13:34:52.421Z level=INFO msg="Upload complete. (603ms)"3901worker2 # [ 37.871356] nix-grpc-daemon[777]: ts=2026-09-20T13:34:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_in=5043902worker2 # [ 37.893072] nix-grpc-daemon[777]: ts=2026-09-20T13:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525443903worker2 # [ 37.904440] nix-grpc-daemon[858]: time=2026-09-20T13:34:52.443Z level=WARN msg="Failed to register uploaded object" key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo error="server returned 404: object is not pending upload\n"3904worker2 # [ 37.920308] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3905worker2 # [ 37.923406] nix-daemon[845]: reaped child process 856, status = succeeded3906client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3907client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3908worker2 # [ 37.936521] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3909client # farm-a> LOG-farm-a-t13910worker2 # [ 38.777307] nix-daemon[845]: reaped child process 883, status = succeeded3911worker2 # [ 38.957930] nix-daemon[845]: accepted connection from pid 898, user nix-grpc-daemon (trusted)3912worker2 # [ 38.977118] nix-daemon[845]: reaped child process 901, status = succeeded3913lb # [ 38.999832] niks3-server[1024]: 2026/09/20 13:34:53 INFO Received uploads request method=POST path=/api/pending_closures3914worker2 # [ 39.009676] nix-grpc-daemon[858]: time=2026-09-20T13:34:53.572Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3915worker2 # [ 39.012874] nix-grpc-daemon[858]: time=2026-09-20T13:34:53.574Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3916lb # [ 39.069335] niks3-server[1024]: 2026/09/20 13:34:53 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3917lb # [ 39.082308] niks3-server[1024]: 2026/09/20 13:34:53 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3918lb # [ 39.088991] niks3-server[1024]: 2026/09/20 13:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3919lb # [ 39.097505] niks3-server[1024]: 2026/09/20 13:34:53 INFO Signed narinfos id=2 count=13920worker2 # [ 39.091997] nix-grpc-daemon[858]: time=2026-09-20T13:34:53.653Z level=INFO msg="Uploading 1 narinfos"3921lb # [ 39.101959] niks3-server[1024]: 2026/09/20 13:34:53 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3922lb # [ 39.114988] niks3-server[1024]: 2026/09/20 13:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3923lb # [ 39.124799] niks3-server[1024]: 2026/09/20 13:34:53 INFO Completed upload id=23924worker2 # [ 39.117747] nix-grpc-daemon[858]: time=2026-09-20T13:34:53.680Z level=INFO msg="Upload complete. (368ms)"3925lb # [ 39.127953] niks3-server[1024]: 2026/09/20 13:34:53 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo3926worker2 # [ 39.141070] nix-grpc-daemon[777]: ts=2026-09-20T13:34:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=13927worker2 # [ 39.160359] nix-grpc-daemon[777]: ts=2026-09-20T13:34:53Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=03928worker2 # [ 39.165518] nix-grpc-daemon[777]: ts=2026-09-20T13:34:53Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=13929worker2 # [ 39.169491] nix-daemon[845]: reaped child process 881, status = succeeded3930worker2 # [ 39.192054] nix-grpc-daemon[777]: ts=2026-09-20T13:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525443931worker2 # [ 39.207565] nix-grpc-daemon[777]: ts=2026-09-20T13:34:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525443932worker2 # [ 39.223419] nix-grpc-daemon[858]: time=2026-09-20T13:34:53.786Z level=INFO msg="All 1 paths already cached"3933client # copying 1 paths to 'lb:50051'...3934worker2 # [ 39.232427] nix-grpc-daemon[777]: ts=2026-09-20T13:34:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:525443935client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...3936worker2 # [ 39.256780] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3937worker2 # [ 39.469566] nix-daemon[845]: accepted connection from pid 907, user nix-grpc-daemon (trusted)3938worker2 # [ 39.489641] nix-daemon[845]: reaped child process 910, status = succeeded3939worker2 # [ 39.665794] nix-daemon[845]: accepted connection from pid 912, user nix-grpc-daemon (trusted)3940worker2 # [ 39.684598] nix-daemon[845]: reaped child process 915, status = succeeded3941lb # [ 39.702489] niks3-server[1024]: 2026/09/20 13:34:54 INFO Received uploads request method=POST path=/api/pending_closures3942worker2 # [ 39.705786] nix-grpc-daemon[858]: time=2026-09-20T13:34:54.268Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3943worker2 # [ 39.707286] nix-grpc-daemon[858]: time=2026-09-20T13:34:54.268Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"3944lb # [ 39.730523] niks3-server[1024]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst3945worker2 # [ 39.736484] nix-grpc-daemon[858]: time=2026-09-20T13:34:54.299Z level=INFO msg="Uploading 1 narinfos"3946lb # [ 39.747088] niks3-server[1024]: 2026/09/20 13:34:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign3947lb # [ 39.758006] niks3-server[1024]: 2026/09/20 13:34:54 INFO Signed narinfos id=3 count=13948lb # [ 39.763740] niks3-server[1024]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls3949worker2 # [ 39.767851] nix-grpc-daemon[858]: time=2026-09-20T13:34:54.330Z level=INFO msg="Upload complete. (493ms)"3950lb # [ 39.775239] niks3-server[1024]: 2026/09/20 13:34:54 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete3951lb # [ 39.782364] niks3-server[1024]: 2026/09/20 13:34:54 INFO Completed upload id=33952worker2 # [ 39.775264] nix-grpc-daemon[777]: ts=2026-09-20T13:34:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_in=6243953lb # [ 39.786239] niks3-server[1024]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo3954worker2 # [ 39.788491] nix-grpc-daemon[777]: ts=2026-09-20T13:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525443955worker2 # [ 39.794987] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3956worker2 # [ 39.799744] nix-daemon[845]: reaped child process 905, status = succeeded3957client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...3958client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3959worker2 # [ 39.813848] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3960client # farm-b> LOG-farm-b-t13961worker2 # [ 40.659867] nix-daemon[845]: reaped child process 919, status = succeeded3962worker2 # [ 40.836352] nix-daemon[845]: accepted connection from pid 935, user nix-grpc-daemon (trusted)3963worker2 # [ 40.855469] nix-daemon[845]: reaped child process 938, status = succeeded3964lb # [ 40.879264] niks3-server[1024]: 2026/09/20 13:34:55 INFO Received uploads request method=POST path=/api/pending_closures3965worker2 # [ 40.882355] nix-grpc-daemon[858]: time=2026-09-20T13:34:55.444Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3966worker2 # [ 40.885463] nix-grpc-daemon[858]: time=2026-09-20T13:34:55.447Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"3967lb # [ 40.915933] niks3-server[1024]: 2026/09/20 13:34:55 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3968lb # [ 40.937268] niks3-server[1024]: 2026/09/20 13:34:55 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst3969lb # [ 40.947497] niks3-server[1024]: 2026/09/20 13:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign3970worker2 # [ 40.945757] nix-grpc-daemon[858]: time=2026-09-20T13:34:55.508Z level=INFO msg="Uploading 1 narinfos"3971lb # [ 40.957251] niks3-server[1024]: 2026/09/20 13:34:55 INFO Signed narinfos id=4 count=13972lb # [ 40.963316] niks3-server[1024]: 2026/09/20 13:34:55 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls3973lb # [ 40.970538] niks3-server[1024]: 2026/09/20 13:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete3974lb # [ 40.981554] niks3-server[1024]: 2026/09/20 13:34:55 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo3975lb # [ 40.988071] niks3-server[1024]: 2026/09/20 13:34:55 INFO Completed upload id=43976worker2 # [ 40.980982] nix-grpc-daemon[858]: time=2026-09-20T13:34:55.543Z level=INFO msg="Upload complete. (356ms)"3977worker2 # [ 41.001283] nix-grpc-daemon[777]: ts=2026-09-20T13:34:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=13978worker2 # [ 41.016456] nix-grpc-daemon[777]: ts=2026-09-20T13:34:55Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=03979worker2 # [ 41.022757] nix-grpc-daemon[777]: ts=2026-09-20T13:34:55Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=13980worker2 # [ 41.027540] nix-daemon[845]: reaped child process 917, status = succeeded3981worker2 # [ 41.041603] nix-grpc-daemon[777]: ts=2026-09-20T13:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525443982worker2 # [ 41.057647] nix-grpc-daemon[777]: ts=2026-09-20T13:34:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525443983worker2 # [ 41.071761] nix-grpc-daemon[858]: time=2026-09-20T13:34:55.634Z level=INFO msg="All 2 paths already cached"3984client # copying 1 paths to 'lb:50051'...3985worker2 # [ 41.088417] nix-grpc-daemon[777]: ts=2026-09-20T13:34:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:525443986client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...3987worker2 # [ 41.112201] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3988worker2 # [ 41.318907] nix-daemon[845]: accepted connection from pid 944, user nix-grpc-daemon (trusted)3989worker2 # [ 41.337200] nix-daemon[845]: reaped child process 947, status = succeeded3990worker2 # [ 41.511043] nix-daemon[845]: accepted connection from pid 949, user nix-grpc-daemon (trusted)3991worker2 # [ 41.529179] nix-daemon[845]: reaped child process 952, status = succeeded3992lb # [ 41.547341] niks3-server[1024]: 2026/09/20 13:34:56 INFO Received uploads request method=POST path=/api/pending_closures3993worker2 # [ 41.551619] nix-grpc-daemon[858]: time=2026-09-20T13:34:56.114Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"3994worker2 # [ 41.553300] nix-grpc-daemon[858]: time=2026-09-20T13:34:56.114Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"3995lb # [ 41.574934] niks3-server[1024]: 2026/09/20 13:34:56 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst3996lb # [ 41.590443] niks3-server[1024]: 2026/09/20 13:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign3997worker2 # [ 41.589824] nix-grpc-daemon[858]: time=2026-09-20T13:34:56.152Z level=INFO msg="Uploading 1 narinfos"3998lb # [ 41.602833] niks3-server[1024]: 2026/09/20 13:34:56 INFO Signed narinfos id=5 count=13999lb # [ 41.614505] niks3-server[1024]: 2026/09/20 13:34:56 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4000worker2 # [ 41.612616] nix-grpc-daemon[858]: time=2026-09-20T13:34:56.175Z level=INFO msg="Upload complete. (482ms)"4001lb # [ 41.620979] niks3-server[1024]: 2026/09/20 13:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4002lb # [ 41.626118] niks3-server[1024]: 2026/09/20 13:34:56 INFO Completed upload id=54003worker2 # [ 41.620313] nix-grpc-daemon[777]: ts=2026-09-20T13:34:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_in=7524004lb # [ 41.629389] niks3-server[1024]: 2026/09/20 13:34:56 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4005worker2 # [ 41.634305] nix-grpc-daemon[777]: ts=2026-09-20T13:34:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444006worker2 # [ 41.640580] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4007worker2 # [ 41.645585] nix-daemon[845]: reaped child process 942, status = succeeded4008client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4009worker2 # [ 41.656650] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4010client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4011lb # [ 42.125546] envoy[921]: conn peer=[2001:db8:1::1]:49506 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=365 ms=16024012lb # [ 42.130769] envoy[921]: conn peer=[2001:db8:1::1]:49520 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=365 ms=15274013lb # [ 42.135579] envoy[921]: conn peer=[2001:db8:1::1]:49532 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=124014lb # [ 42.140137] envoy[921]: conn peer=[2001:db8:1::1]:49548 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=182 ms=224015client # farm-top> LOG-farm-top-t14016worker2 # [ 42.511942] nix-daemon[845]: reaped child process 956, status = succeeded4017worker2 # [ 42.691990] nix-daemon[845]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4018worker2 # [ 42.713355] nix-daemon[845]: reaped child process 974, status = succeeded4019lb # [ 42.733054] niks3-server[1024]: 2026/09/20 13:34:57 INFO Received uploads request method=POST path=/api/pending_closures4020worker2 # [ 42.738793] nix-grpc-daemon[858]: time=2026-09-20T13:34:57.299Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4021worker2 # [ 42.740444] nix-grpc-daemon[858]: time=2026-09-20T13:34:57.300Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4022lb # [ 42.766188] niks3-server[1024]: 2026/09/20 13:34:57 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4023lb # [ 42.784435] niks3-server[1024]: 2026/09/20 13:34:57 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4024worker2 # [ 42.792339] nix-grpc-daemon[858]: time=2026-09-20T13:34:57.354Z level=INFO msg="Uploading 1 narinfos"4025lb # [ 42.793592] niks3-server[1024]: 2026/09/20 13:34:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4026lb # [ 42.803172] niks3-server[1024]: 2026/09/20 13:34:57 INFO Signed narinfos id=6 count=14027lb # [ 42.808785] niks3-server[1024]: 2026/09/20 13:34:57 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4028lb # [ 42.814236] niks3-server[1024]: 2026/09/20 13:34:57 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4029lb # [ 42.823306] niks3-server[1024]: 2026/09/20 13:34:57 INFO Completed upload id=64030worker2 # [ 42.816249] nix-grpc-daemon[858]: time=2026-09-20T13:34:57.379Z level=INFO msg="Upload complete. (344ms)"4031lb # [ 42.826138] niks3-server[1024]: 2026/09/20 13:34:57 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4032worker2 # [ 42.827497] nix-grpc-daemon[777]: ts=2026-09-20T13:34:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14033worker2 # [ 42.843825] nix-daemon[845]: reaped child process 954, status = succeeded4034client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 6.36 seconds)4035client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4036client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.27 seconds)4037client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4038client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.04 seconds)4039client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4040client # copying 3 paths...4041client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4042client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4043client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4044client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds)4045(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.12 seconds)4046subtest: repeat is answered Cached by the scheduler without building4047worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4048worker1 # finding garbage collector roots...4049worker1 # deleting specified paths...4050worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4051worker1 # deleting unused links...4052worker1 # note: hard linking is currently saving 0.0 KiB4053worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4054worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4055worker2 # finding garbage collector roots...4056worker2 # deleting specified paths...4057worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4058worker2 # deleting unused links...4059worker2 # note: hard linking is currently saving 0.0 KiB4060worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4061client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24062worker2 # [ 44.434990] nix-grpc-daemon[777]: ts=2026-09-20T13:34:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:525444063client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4064client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4065worker2 # [ 44.456317] nix-grpc-daemon[777]: ts=2026-09-20T13:34:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525444066worker2 # [ 44.467493] nix-grpc-daemon[858]: time=2026-09-20T13:34:59.029Z level=INFO msg="All 2 paths already cached"4067worker2 # [ 44.484724] nix-grpc-daemon[777]: ts=2026-09-20T13:34:59Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14068client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.33 seconds)4069client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4070client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4071lb: must succeed: curl -sf localhost:9901/clusters4072lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4073lb: must succeed: curl -sf localhost:9901/clusters4074lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4075worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4076worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4077(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.43 seconds)4078subtest: inputs already in the cache are not sent to the scheduler4079lb: must succeed: curl -sf localhost:9901/clusters4080lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4081lb: must succeed: curl -sf localhost:9901/clusters4082lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4083worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4084worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4085client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24086client # don't know how to build these paths:4087client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4088worker2 # [ 45.516830] nix-grpc-daemon[777]: ts=2026-09-20T13:35:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:525444089worker1 # [ 45.550156] nix-grpc-daemon[776]: ts=2026-09-20T13:35:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411024090worker2 # [ 45.553546] nix-grpc-daemon[777]: ts=2026-09-20T13:35:00Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04091worker2 # [ 45.560862] nix-grpc-daemon[777]: ts=2026-09-20T13:35:00Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14092worker2 # [ 45.601457] nix-grpc-daemon[777]: ts=2026-09-20T13:35:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444093worker2 # [ 45.617176] nix-grpc-daemon[777]: ts=2026-09-20T13:35:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525444094worker2 # [ 45.811164] nix-grpc-daemon[858]: time=2026-09-20T13:35:00.372Z level=INFO msg="All 2 paths already cached"4095client # copying 1 paths to 'lb:50051'...4096client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4097worker2 # [ 45.822510] nix-grpc-daemon[777]: ts=2026-09-20T13:35:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:525444098worker2 # [ 45.837827] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4099worker2 # [ 46.045501] nix-daemon[845]: accepted connection from pid 1004, user nix-grpc-daemon (trusted)4100worker2 # [ 46.066532] nix-daemon[845]: reaped child process 1007, status = succeeded4101worker2 # [ 46.241862] nix-daemon[845]: accepted connection from pid 1009, user nix-grpc-daemon (trusted)4102worker2 # [ 46.260963] nix-daemon[845]: reaped child process 1012, status = succeeded4103lb # [ 46.283478] niks3-server[1024]: 2026/09/20 13:35:00 INFO Received uploads request method=POST path=/api/pending_closures4104worker2 # [ 46.286324] nix-grpc-daemon[858]: time=2026-09-20T13:35:00.848Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4105worker2 # [ 46.287848] nix-grpc-daemon[858]: time=2026-09-20T13:35:00.849Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4106lb # [ 46.323941] niks3-server[1024]: 2026/09/20 13:35:00 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4107worker2 # [ 46.327706] nix-grpc-daemon[858]: time=2026-09-20T13:35:00.890Z level=INFO msg="Uploading 1 narinfos"4108lb # [ 46.337491] niks3-server[1024]: 2026/09/20 13:35:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4109lb # [ 46.348357] niks3-server[1024]: 2026/09/20 13:35:00 INFO Signed narinfos id=7 count=14110lb # [ 46.354708] niks3-server[1024]: 2026/09/20 13:35:00 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4111worker2 # [ 46.358611] nix-grpc-daemon[858]: time=2026-09-20T13:35:00.921Z level=INFO msg="Upload complete. (502ms)"4112lb # [ 46.366556] niks3-server[1024]: 2026/09/20 13:35:00 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4113lb # [ 46.373001] niks3-server[1024]: 2026/09/20 13:35:00 INFO Completed upload id=74114worker2 # [ 46.368422] nix-grpc-daemon[777]: ts=2026-09-20T13:35:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_in=7524115lb # [ 46.377366] niks3-server[1024]: 2026/09/20 13:35:00 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4116worker2 # [ 46.383859] nix-grpc-daemon[777]: ts=2026-09-20T13:35:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444117worker2 # [ 46.390852] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4118worker2 # [ 46.396187] nix-daemon[845]: reaped child process 1002, status = succeeded4119client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4120client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4121worker2 # [ 46.411531] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4122client # farm-top> LOG-farm-top-t1b4123worker2 # [ 47.457032] nix-daemon[845]: reaped child process 1016, status = succeeded4124worker2 # [ 47.644145] nix-daemon[845]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4125worker2 # [ 47.664128] nix-daemon[845]: reaped child process 1035, status = succeeded4126lb # [ 47.686425] niks3-server[1024]: 2026/09/20 13:35:02 INFO Received uploads request method=POST path=/api/pending_closures4127worker2 # [ 47.688511] nix-grpc-daemon[858]: time=2026-09-20T13:35:02.249Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4128worker2 # [ 47.690042] nix-grpc-daemon[858]: time=2026-09-20T13:35:02.249Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4129lb # [ 47.718932] niks3-server[1024]: 2026/09/20 13:35:02 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4130lb # [ 47.737349] niks3-server[1024]: 2026/09/20 13:35:02 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4131lb # [ 47.745260] niks3-server[1024]: 2026/09/20 13:35:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4132worker2 # [ 47.742907] nix-grpc-daemon[858]: time=2026-09-20T13:35:02.305Z level=INFO msg="Uploading 1 narinfos"4133lb # [ 47.753704] niks3-server[1024]: 2026/09/20 13:35:02 INFO Signed narinfos id=8 count=14134lb # [ 47.758754] niks3-server[1024]: 2026/09/20 13:35:02 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4135lb # [ 47.767395] niks3-server[1024]: 2026/09/20 13:35:02 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4136lb # [ 47.773579] niks3-server[1024]: 2026/09/20 13:35:02 INFO Completed upload id=84137worker2 # [ 47.765877] nix-grpc-daemon[858]: time=2026-09-20T13:35:02.328Z level=INFO msg="Upload complete. (343ms)"4138lb # [ 47.775838] niks3-server[1024]: 2026/09/20 13:35:02 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4139worker2 # [ 47.781183] nix-grpc-daemon[777]: ts=2026-09-20T13:35:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14140worker2 # [ 47.794975] nix-daemon[845]: reaped child process 1014, status = succeeded4141client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.55 seconds)4142client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4143client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.24 seconds)4144lb: must succeed: curl -sf localhost:9901/clusters4145lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4146lb: must succeed: curl -sf localhost:9901/clusters4147lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4148worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4149worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4150(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.21 seconds)4151subtest: two clients wanting the same drv build it once4152client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4153client # [ 48.353116] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4154client # Running as unit: dedup1.service; invocation ID: abc89dfd2b6040409a247cfb60e2dd054155client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.09 seconds)4156client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4157client # [ 48.593253] nix[879]: don't know how to build these paths:4158client # [ 48.594214] nix[879]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4159worker1 # [ 48.622361] nix-grpc-daemon[776]: ts=2026-09-20T13:35:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:411024160worker2 # [ 48.609150] nix-grpc-daemon[777]: ts=2026-09-20T13:35:03Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04161worker2 # [ 48.615545] nix-grpc-daemon[777]: ts=2026-09-20T13:35:03Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14162worker1 # [ 48.644279] nix-grpc-daemon[776]: ts=2026-09-20T13:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411024163worker1 # [ 48.663721] nix-grpc-daemon[776]: ts=2026-09-20T13:35:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411024164client # [ 48.648165] nix[879]: copying 1 paths to 'lb:50051'...4165client # [ 48.649388] nix[879]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4166worker1 # [ 48.678289] nix-grpc-daemon[776]: ts=2026-09-20T13:35:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:411024167worker1 # [ 48.693705] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4168worker1 # [ 48.722097] nix-grpc-daemon[776]: ts=2026-09-20T13:35:03Z level=info event=niks3_push_started pid=8614169worker1 # [ 49.001155] nix-grpc-daemon[861]: time=2026-09-20T13:35:03.561Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""4170worker1 # [ 49.218549] nix-daemon[834]: accepted connection from pid 874, user nix-grpc-daemon (trusted)4171worker1 # [ 49.241331] nix-daemon[834]: reaped child process 877, status = succeeded4172worker1 # [ 49.438006] nix-daemon[834]: accepted connection from pid 879, user nix-grpc-daemon (trusted)4173worker1 # [ 49.457572] nix-daemon[834]: reaped child process 882, status = succeeded4174lb # [ 49.455518] niks3-server[1024]: 2026/09/20 13:35:04 INFO Received uploads request method=POST path=/api/pending_closures4175worker1 # [ 49.488100] nix-grpc-daemon[861]: time=2026-09-20T13:35:04.050Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4176worker1 # [ 49.491912] nix-grpc-daemon[861]: time=2026-09-20T13:35:04.051Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4177client # [ 49.474563] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4178client # Running as unit: dedup2.service; invocation ID: 59ef8d0c802a4425a95acef65a8a5ae34179lb # [ 49.508579] niks3-server[1024]: 2026/09/20 13:35:04 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4180client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4181lb # [ 49.523678] niks3-server[1024]: 2026/09/20 13:35:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4182worker1 # [ 49.548205] nix-grpc-daemon[861]: time=2026-09-20T13:35:04.109Z level=INFO msg="Uploading 1 narinfos"4183lb # [ 49.533704] niks3-server[1024]: 2026/09/20 13:35:04 INFO Signed narinfos id=9 count=14184lb # [ 49.539632] niks3-server[1024]: 2026/09/20 13:35:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4185lb # [ 49.555332] niks3-server[1024]: 2026/09/20 13:35:04 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4186lb # [ 49.562664] niks3-server[1024]: 2026/09/20 13:35:04 INFO Completed upload id=94187worker1 # [ 49.579980] nix-grpc-daemon[861]: time=2026-09-20T13:35:04.141Z level=INFO msg="Upload complete. (569ms)"4188lb # [ 49.567288] niks3-server[1024]: 2026/09/20 13:35:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4189worker1 # [ 49.588336] nix-grpc-daemon[776]: ts=2026-09-20T13:35:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41102 duration_s=0 paths=1 nar_bytes_in=4324190worker1 # [ 49.599117] nix-grpc-daemon[776]: ts=2026-09-20T13:35:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411024191worker1 # [ 49.604851] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4192worker1 # [ 49.608824] nix-daemon[834]: reaped child process 859, status = succeeded4193worker1 # [ 49.619772] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4194client # [ 49.592241] nix[879]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4195worker2 # [ 49.806884] nix-grpc-daemon[777]: ts=2026-09-20T13:35:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:525444196client # [ 49.812092] nix[895]: don't know how to build these paths:4197client # [ 49.813193] nix[895]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4198worker2 # [ 49.831334] nix-grpc-daemon[777]: ts=2026-09-20T13:35:04Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04199worker1 # [ 49.865858] nix-grpc-daemon[776]: ts=2026-09-20T13:35:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411024200client # [ 49.844718] nix[895]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4201client: must succeed: sleep 24202lb # [ 52.124661] envoy[921]: conn peer=[2001:db8:1::1]:49564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9523 tx=3867 ms=61154203lb # [ 52.129975] envoy[921]: conn peer=[2001:db8:1::1]:39706 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=476 ms=714204lb # [ 52.130197] envoy[921]: conn peer=[2001:db8:1::1]:39716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1730 ms=22784205client: (finished: must succeed: sleep 2, in 2.08 seconds)4206??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4207 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394208client: waiting for success: ! systemctl is-active dedup1 dedup24209??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4210 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394211worker1 # [ 53.289866] nix-daemon[834]: reaped child process 890, status = succeeded4212worker1 # [ 53.491280] nix-daemon[834]: accepted connection from pid 915, user nix-grpc-daemon (trusted)4213worker1 # [ 53.510459] nix-daemon[834]: reaped child process 918, status = succeeded4214lb # [ 53.507565] niks3-server[1024]: 2026/09/20 13:35:08 INFO Received uploads request method=POST path=/api/pending_closures4215worker1 # [ 53.542088] nix-grpc-daemon[861]: time=2026-09-20T13:35:08.102Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4216worker1 # [ 53.545421] nix-grpc-daemon[861]: time=2026-09-20T13:35:08.102Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4217lb # [ 53.588330] niks3-server[1024]: 2026/09/20 13:35:08 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4218lb # [ 53.597694] niks3-server[1024]: 2026/09/20 13:35:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4219worker1 # [ 53.619294] nix-grpc-daemon[861]: time=2026-09-20T13:35:08.178Z level=INFO msg="Uploading 1 narinfos"4220lb # [ 53.603095] niks3-server[1024]: 2026/09/20 13:35:08 INFO Signed narinfos id=10 count=14221lb # [ 53.608429] niks3-server[1024]: 2026/09/20 13:35:08 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4222lb # [ 53.615552] niks3-server[1024]: 2026/09/20 13:35:08 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4223lb # [ 53.621668] niks3-server[1024]: 2026/09/20 13:35:08 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4224lb # [ 53.631149] niks3-server[1024]: 2026/09/20 13:35:08 INFO Completed upload id=104225worker1 # [ 53.648943] nix-grpc-daemon[861]: time=2026-09-20T13:35:08.210Z level=INFO msg="Upload complete. (411ms)"4226lb # [ 53.633671] niks3-server[1024]: 2026/09/20 13:35:08 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4227worker1 # [ 53.665557] nix-grpc-daemon[776]: ts=2026-09-20T13:35:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41102 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14228worker1 # [ 53.684672] nix-grpc-daemon[776]: ts=2026-09-20T13:35:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41102 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14229worker1 # [ 53.693229] nix-daemon[834]: reaped child process 888, status = succeeded4230client # [ 53.707913] systemd[1]: dedup2.service: Deactivated successfully.4231client # [ 53.712139] systemd[1]: dedup2.service: Consumed 205ms CPU time over 4.239s wall clock time, 30.9M memory peak, 7.4K incoming IP traffic, 6.5K outgoing IP traffic.4232client # [ 53.718898] systemd[1]: dedup1.service: Deactivated successfully.4233client # [ 53.723285] systemd[1]: dedup1.service: Consumed 157ms CPU time over 5.370s wall clock time, 39.2M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.4234client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.16 seconds)4235client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4236client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4237worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4238worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4239worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4240worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4241worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4242worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)4243worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4244worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4245(finished: subtest: two clients wanting the same drv build it once, in 6.43 seconds)4246subtest: build hook: nix-daemon with builders = grpc://lb4247client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4248client # [ 54.947795] systemd[1]: Started Nix Daemon.4249client # [ 55.142281] nix-daemon[937]: accepted connection from pid 934, user root (trusted)4250worker2 # [ 55.354830] nix-grpc-daemon[777]: ts=2026-09-20T13:35:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:525444251worker1 # [ 55.386351] nix-grpc-daemon[776]: ts=2026-09-20T13:35:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411024252worker2 # [ 55.382215] nix-grpc-daemon[777]: ts=2026-09-20T13:35:09Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04253worker2 # [ 55.389644] nix-grpc-daemon[777]: ts=2026-09-20T13:35:09Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14254worker2 # [ 55.429548] nix-grpc-daemon[777]: ts=2026-09-20T13:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444255worker2 # [ 55.449007] nix-grpc-daemon[777]: ts=2026-09-20T13:35:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525444256worker2 # [ 55.465105] nix-grpc-daemon[777]: ts=2026-09-20T13:35:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:525444257worker2 # [ 55.481968] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4258worker2 # [ 55.703508] nix-daemon[845]: accepted connection from pid 1071, user nix-grpc-daemon (trusted)4259worker2 # [ 55.723763] nix-daemon[845]: reaped child process 1074, status = succeeded4260worker2 # [ 55.894023] nix-daemon[845]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)4261worker2 # [ 55.911020] nix-daemon[845]: reaped child process 1079, status = succeeded4262lb # [ 55.933646] niks3-server[1024]: 2026/09/20 13:35:10 INFO Received uploads request method=POST path=/api/pending_closures4263worker2 # [ 55.935774] nix-grpc-daemon[858]: time=2026-09-20T13:35:10.497Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4264worker2 # [ 55.937379] nix-grpc-daemon[858]: time=2026-09-20T13:35:10.498Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4265lb # [ 55.961704] niks3-server[1024]: 2026/09/20 13:35:10 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4266lb # [ 55.976626] niks3-server[1024]: 2026/09/20 13:35:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4267worker2 # [ 55.975381] nix-grpc-daemon[858]: time=2026-09-20T13:35:10.538Z level=INFO msg="Uploading 1 narinfos"4268lb # [ 55.989326] niks3-server[1024]: 2026/09/20 13:35:10 INFO Signed narinfos id=11 count=14269lb # [ 55.999074] niks3-server[1024]: 2026/09/20 13:35:10 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4270worker2 # [ 55.997819] nix-grpc-daemon[858]: time=2026-09-20T13:35:10.560Z level=INFO msg="Upload complete. (497ms)"4271lb # [ 56.005636] niks3-server[1024]: 2026/09/20 13:35:10 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4272lb # [ 56.010517] niks3-server[1024]: 2026/09/20 13:35:10 INFO Completed upload id=114273lb # [ 56.012587] niks3-server[1024]: 2026/09/20 13:35:10 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4274worker2 # [ 56.004252] nix-grpc-daemon[777]: ts=2026-09-20T13:35:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_in=5124275worker2 # [ 56.013067] nix-grpc-daemon[777]: ts=2026-09-20T13:35:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444276worker2 # [ 56.024245] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4277worker2 # [ 56.029363] nix-daemon[845]: reaped child process 1069, status = succeeded4278worker2 # [ 56.042709] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4279worker2 # [ 57.488604] nix-daemon[845]: reaped child process 1083, status = succeeded4280worker2 # [ 57.656231] nix-daemon[845]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)4281worker2 # [ 57.676424] nix-daemon[845]: reaped child process 1101, status = succeeded4282lb # [ 57.698301] niks3-server[1024]: 2026/09/20 13:35:12 INFO Received uploads request method=POST path=/api/pending_closures4283worker2 # [ 57.700418] nix-grpc-daemon[858]: time=2026-09-20T13:35:12.261Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4284worker2 # [ 57.701923] nix-grpc-daemon[858]: time=2026-09-20T13:35:12.261Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4285lb # [ 57.738958] niks3-server[1024]: 2026/09/20 13:35:12 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4286lb # [ 57.752704] niks3-server[1024]: 2026/09/20 13:35:12 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4287lb # [ 57.763726] niks3-server[1024]: 2026/09/20 13:35:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4288lb # [ 57.771046] niks3-server[1024]: 2026/09/20 13:35:12 INFO Signed narinfos id=12 count=14289worker2 # [ 57.764220] nix-grpc-daemon[858]: time=2026-09-20T13:35:12.326Z level=INFO msg="Uploading 1 narinfos"4290lb # [ 57.774730] niks3-server[1024]: 2026/09/20 13:35:12 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4291lb # [ 57.790209] niks3-server[1024]: 2026/09/20 13:35:12 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4292lb # [ 57.797684] niks3-server[1024]: 2026/09/20 13:35:12 INFO Completed upload id=124293lb # [ 57.800864] niks3-server[1024]: 2026/09/20 13:35:12 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4294worker2 # [ 57.789882] nix-grpc-daemon[858]: time=2026-09-20T13:35:12.352Z level=INFO msg="Upload complete. (336ms)"4295worker2 # [ 57.802628] nix-grpc-daemon[777]: ts=2026-09-20T13:35:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14296worker2 # [ 57.823642] nix-grpc-daemon[777]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:525444297worker2 # [ 57.828900] nix-daemon[845]: reaped child process 1081, status = succeeded4298worker1 # [ 57.874915] nix-grpc-daemon[776]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:411024299worker1 # [ 57.886116] nix-grpc-daemon[776]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4300worker1 # [ 57.909187] nix-grpc-daemon[776]: ts=2026-09-20T13:35:12Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41102 duration_s=0 paths=1 nar_bytes_out=1284301worker2 # [ 58.188135] nix-grpc-daemon[777]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:525444302worker1 # [ 58.223731] nix-grpc-daemon[776]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411024303worker1 # [ 58.235148] nix-grpc-daemon[861]: time=2026-09-20T13:35:12.796Z level=INFO msg="All 1 paths already cached"4304worker2 # [ 58.229223] nix-grpc-daemon[777]: ts=2026-09-20T13:35:12Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04305worker2 # [ 58.238573] nix-grpc-daemon[777]: ts=2026-09-20T13:35:12Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14306worker2 # [ 58.247332] nix-grpc-daemon[777]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444307worker2 # [ 58.254892] nix-grpc-daemon[777]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525444308worker2 # [ 58.265378] nix-grpc-daemon[858]: time=2026-09-20T13:35:12.828Z level=INFO msg="All 1 paths already cached"4309worker2 # [ 58.275080] nix-grpc-daemon[777]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:525444310worker2 # [ 58.289393] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4311worker2 # [ 58.493140] nix-daemon[845]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4312worker2 # [ 58.511893] nix-daemon[845]: reaped child process 1110, status = succeeded4313worker2 # [ 58.686544] nix-daemon[845]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)4314worker2 # [ 58.704129] nix-daemon[845]: reaped child process 1115, status = succeeded4315lb # [ 58.723780] niks3-server[1024]: 2026/09/20 13:35:13 INFO Received uploads request method=POST path=/api/pending_closures4316worker2 # [ 58.726885] nix-grpc-daemon[858]: time=2026-09-20T13:35:13.289Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4317worker2 # [ 58.728433] nix-grpc-daemon[858]: time=2026-09-20T13:35:13.289Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4318lb # [ 58.759382] niks3-server[1024]: 2026/09/20 13:35:13 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4319worker2 # [ 58.762489] nix-grpc-daemon[858]: time=2026-09-20T13:35:13.325Z level=INFO msg="Uploading 1 narinfos"4320lb # [ 58.774640] niks3-server[1024]: 2026/09/20 13:35:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4321lb # [ 58.788460] niks3-server[1024]: 2026/09/20 13:35:13 INFO Signed narinfos id=13 count=14322lb # [ 58.791312] niks3-server[1024]: 2026/09/20 13:35:13 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4323worker2 # [ 58.798529] nix-grpc-daemon[858]: time=2026-09-20T13:35:13.361Z level=INFO msg="Upload complete. (493ms)"4324lb # [ 58.806410] niks3-server[1024]: 2026/09/20 13:35:13 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4325lb # [ 58.812175] niks3-server[1024]: 2026/09/20 13:35:13 INFO Completed upload id=134326lb # [ 58.815911] niks3-server[1024]: 2026/09/20 13:35:13 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4327worker2 # [ 58.809759] nix-grpc-daemon[777]: ts=2026-09-20T13:35:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_in=6244328worker2 # [ 58.822018] nix-grpc-daemon[777]: ts=2026-09-20T13:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444329worker2 # [ 58.828386] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4330worker2 # [ 58.833206] nix-daemon[845]: reaped child process 1105, status = succeeded4331worker2 # [ 58.844192] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4332worker2 # [ 60.067364] nix-daemon[845]: reaped child process 1119, status = succeeded4333worker2 # [ 60.228597] nix-daemon[845]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)4334worker2 # [ 60.245786] nix-daemon[845]: reaped child process 1137, status = succeeded4335lb # [ 60.266960] niks3-server[1024]: 2026/09/20 13:35:14 INFO Received uploads request method=POST path=/api/pending_closures4336worker2 # [ 60.271299] nix-grpc-daemon[858]: time=2026-09-20T13:35:14.832Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4337worker2 # [ 60.272945] nix-grpc-daemon[858]: time=2026-09-20T13:35:14.832Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4338lb # [ 60.302551] niks3-server[1024]: 2026/09/20 13:35:14 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4339lb # [ 60.313776] niks3-server[1024]: 2026/09/20 13:35:14 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4340worker2 # [ 60.320567] nix-grpc-daemon[858]: time=2026-09-20T13:35:14.883Z level=INFO msg="Uploading 1 narinfos"4341lb # [ 60.327587] niks3-server[1024]: 2026/09/20 13:35:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4342lb # [ 60.333305] niks3-server[1024]: 2026/09/20 13:35:14 INFO Signed narinfos id=14 count=14343lb # [ 60.335622] niks3-server[1024]: 2026/09/20 13:35:14 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4344lb # [ 60.346126] niks3-server[1024]: 2026/09/20 13:35:14 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4345lb # [ 60.352097] niks3-server[1024]: 2026/09/20 13:35:14 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4346lb # [ 60.357227] niks3-server[1024]: 2026/09/20 13:35:14 INFO Completed upload id=144347worker2 # [ 60.350099] nix-grpc-daemon[858]: time=2026-09-20T13:35:14.913Z level=INFO msg="Upload complete. (293ms)"4348worker2 # [ 60.364334] nix-grpc-daemon[777]: ts=2026-09-20T13:35:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14349worker2 # [ 60.378688] nix-grpc-daemon[777]: ts=2026-09-20T13:35:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:525444350worker2 # [ 60.382614] nix-daemon[845]: reaped child process 1117, status = succeeded4351worker2 # [ 60.387619] nix-grpc-daemon[777]: ts=2026-09-20T13:35:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:525444352worker2 # [ 60.392392] nix-grpc-daemon[777]: ts=2026-09-20T13:35:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_out=1764353worker2 # [ 60.703048] nix-grpc-daemon[777]: ts=2026-09-20T13:35:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:525444354worker2 # [ 60.710573] nix-grpc-daemon[777]: ts=2026-09-20T13:35:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525444355worker2 # [ 60.717296] nix-grpc-daemon[858]: time=2026-09-20T13:35:15.279Z level=INFO msg="All 2 paths already cached"4356worker2 # [ 60.730061] nix-grpc-daemon[777]: ts=2026-09-20T13:35:15Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04357worker2 # [ 60.737153] nix-grpc-daemon[777]: ts=2026-09-20T13:35:15Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14358worker2 # [ 60.746187] nix-grpc-daemon[777]: ts=2026-09-20T13:35:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444359worker2 # [ 60.757194] nix-grpc-daemon[777]: ts=2026-09-20T13:35:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525444360worker2 # [ 60.770642] nix-grpc-daemon[858]: time=2026-09-20T13:35:15.333Z level=INFO msg="All 2 paths already cached"4361worker2 # [ 60.779674] nix-grpc-daemon[777]: ts=2026-09-20T13:35:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:525444362worker2 # [ 60.793414] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4363worker2 # [ 60.998210] nix-daemon[845]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4364worker2 # [ 61.018195] nix-daemon[845]: reaped child process 1146, status = succeeded4365worker2 # [ 61.190707] nix-daemon[845]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)4366worker2 # [ 61.209161] nix-daemon[845]: reaped child process 1151, status = succeeded4367lb # [ 61.231074] niks3-server[1024]: 2026/09/20 13:35:15 INFO Received uploads request method=POST path=/api/pending_closures4368worker2 # [ 61.233782] nix-grpc-daemon[858]: time=2026-09-20T13:35:15.796Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4369worker2 # [ 61.235270] nix-grpc-daemon[858]: time=2026-09-20T13:35:15.796Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4370lb # [ 61.258644] niks3-server[1024]: 2026/09/20 13:35:15 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4371lb # [ 61.274414] niks3-server[1024]: 2026/09/20 13:35:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4372worker2 # [ 61.273616] nix-grpc-daemon[858]: time=2026-09-20T13:35:15.835Z level=INFO msg="Uploading 1 narinfos"4373lb # [ 61.288410] niks3-server[1024]: 2026/09/20 13:35:15 INFO Signed narinfos id=15 count=14374lb # [ 61.293040] niks3-server[1024]: 2026/09/20 13:35:15 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4375lb # [ 61.301318] niks3-server[1024]: 2026/09/20 13:35:15 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4376worker2 # [ 61.298202] nix-grpc-daemon[858]: time=2026-09-20T13:35:15.861Z level=INFO msg="Upload complete. (489ms)"4377lb # [ 61.306241] niks3-server[1024]: 2026/09/20 13:35:15 INFO Completed upload id=154378lb # [ 61.309640] niks3-server[1024]: 2026/09/20 13:35:15 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4379worker2 # [ 61.304233] nix-grpc-daemon[777]: ts=2026-09-20T13:35:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_in=7524380worker2 # [ 61.317888] nix-grpc-daemon[777]: ts=2026-09-20T13:35:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444381worker2 # [ 61.325113] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4382worker2 # [ 61.329756] nix-daemon[845]: reaped child process 1141, status = succeeded4383worker2 # [ 61.343601] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4384worker2 # [ 61.986448] nix-daemon[845]: reaped child process 1155, status = succeeded4385lb # [ 62.122373] envoy[921]: conn peer=[2001:db8:1::1]:39730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1371 ms=50654386lb # [ 62.127302] envoy[921]: conn peer=[2001:db8:1::1]:39736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1087 ms=38644387lb # [ 62.132186] envoy[921]: conn peer=[2001:db8:1::1]:36660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3418 tx=1662 ms=25914388lb # [ 62.136986] envoy[921]: conn peer=[2001:db8:1::1]:36662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=1044389lb # [ 62.141664] envoy[921]: conn peer=[2001:db8:1::1]:36670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=664390lb # [ 62.146354] envoy[921]: conn peer=[2001:db8:1::1]:36666 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3861 tx=1878 ms=22614391worker2 # [ 62.163998] nix-daemon[845]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)4392worker2 # [ 62.180749] nix-daemon[845]: reaped child process 1173, status = succeeded4393lb # [ 62.201993] niks3-server[1024]: 2026/09/20 13:35:16 INFO Received uploads request method=POST path=/api/pending_closures4394worker2 # [ 62.204067] nix-grpc-daemon[858]: time=2026-09-20T13:35:16.766Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4395worker2 # [ 62.205729] nix-grpc-daemon[858]: time=2026-09-20T13:35:16.766Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4396lb # [ 62.240857] niks3-server[1024]: 2026/09/20 13:35:16 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4397worker2 # [ 62.250165] nix-grpc-daemon[858]: time=2026-09-20T13:35:16.812Z level=INFO msg="Uploading 1 narinfos"4398lb # [ 62.258243] niks3-server[1024]: 2026/09/20 13:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4399lb # [ 62.271725] niks3-server[1024]: 2026/09/20 13:35:16 INFO Signed narinfos id=16 count=14400lb # [ 62.279562] niks3-server[1024]: 2026/09/20 13:35:16 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4401lb # [ 62.285282] niks3-server[1024]: 2026/09/20 13:35:16 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4402lb # [ 62.294819] niks3-server[1024]: 2026/09/20 13:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4403lb # [ 62.300670] niks3-server[1024]: 2026/09/20 13:35:16 INFO Completed upload id=164404worker2 # [ 62.294645] nix-grpc-daemon[858]: time=2026-09-20T13:35:16.856Z level=INFO msg="Upload complete. (340ms)"4405lb # [ 62.303961] niks3-server[1024]: 2026/09/20 13:35:16 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4406worker2 # [ 62.309101] nix-grpc-daemon[777]: ts=2026-09-20T13:35:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14407worker1 # [ 62.337726] nix-grpc-daemon[776]: ts=2026-09-20T13:35:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:411024408worker2 # [ 62.327046] nix-daemon[845]: reaped child process 1153, status = succeeded4409worker1 # [ 62.355551] nix-grpc-daemon[776]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4410worker1 # [ 62.372520] nix-grpc-daemon[776]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4411worker2 # [ 62.388174] nix-grpc-daemon[777]: ts=2026-09-20T13:35:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:525444412worker2 # [ 62.393519] nix-grpc-daemon[777]: ts=2026-09-20T13:35:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_out=2324413client # [ 62.475106] nix-daemon[937]: reaped child process 939, status = succeeded4414client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.78 seconds)4415client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4416client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4417client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4418client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4419client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4420client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4421(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.91 seconds)4422subtest: an input only one worker has still reaches the builder4423client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4424client # this derivation will be built:4425client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4426client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4427client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.44 seconds)4428client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4429client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4430worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4431worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4432worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4433worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4434client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24435client # [ 63.788509] nix-daemon[937]: accepted connection from pid 1020, user root (trusted)4436client # this derivation will be built:4437client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4438client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4439worker1 # [ 64.018992] nix-grpc-daemon[776]: ts=2026-09-20T13:35:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:411024440worker1 # [ 64.027013] nix-grpc-daemon[776]: ts=2026-09-20T13:35:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411024441worker1 # [ 64.229880] nix-daemon[834]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4442worker1 # [ 64.247945] nix-daemon[834]: reaped child process 940, status = succeeded4443lb # [ 64.241721] niks3-server[1024]: 2026/09/20 13:35:18 INFO Received uploads request method=POST path=/api/pending_closures4444worker1 # [ 64.269584] nix-grpc-daemon[861]: time=2026-09-20T13:35:18.830Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4445worker1 # [ 64.271050] nix-grpc-daemon[861]: time=2026-09-20T13:35:18.831Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4446lb # [ 64.271076] niks3-server[1024]: 2026/09/20 13:35:18 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4447worker1 # [ 64.304339] nix-grpc-daemon[861]: time=2026-09-20T13:35:18.866Z level=INFO msg="Uploading 1 narinfos"4448lb # [ 64.288415] niks3-server[1024]: 2026/09/20 13:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4449lb # [ 64.299202] niks3-server[1024]: 2026/09/20 13:35:18 INFO Signed narinfos id=17 count=14450worker1 # [ 64.327058] nix-grpc-daemon[861]: time=2026-09-20T13:35:18.889Z level=INFO msg="Upload complete. (301ms)"4451lb # [ 64.304778] niks3-server[1024]: 2026/09/20 13:35:18 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4452lb # [ 64.315435] niks3-server[1024]: 2026/09/20 13:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4453lb # [ 64.320492] niks3-server[1024]: 2026/09/20 13:35:18 INFO Completed upload id=174454lb # [ 64.322997] niks3-server[1024]: 2026/09/20 13:35:18 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4455worker2 # [ 64.329348] nix-grpc-daemon[777]: ts=2026-09-20T13:35:18Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04456worker2 # [ 64.335757] nix-grpc-daemon[777]: ts=2026-09-20T13:35:18Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14457worker1 # [ 64.407315] nix-grpc-daemon[776]: ts=2026-09-20T13:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411024458worker1 # [ 64.423490] nix-grpc-daemon[776]: ts=2026-09-20T13:35:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411024459client # copying 2 paths to 'lb:50051'...4460client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4461worker1 # [ 64.442675] nix-grpc-daemon[776]: ts=2026-09-20T13:35:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:411024462client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4463worker1 # [ 64.457436] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4464worker1 # [ 64.668993] nix-daemon[834]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4465worker1 # [ 64.688105] nix-daemon[834]: reaped child process 947, status = succeeded4466worker1 # [ 64.861072] nix-daemon[834]: accepted connection from pid 949, user nix-grpc-daemon (trusted)4467worker1 # [ 64.878329] nix-daemon[834]: reaped child process 952, status = succeeded4468lb # [ 64.871635] niks3-server[1024]: 2026/09/20 13:35:19 INFO Received uploads request method=POST path=/api/pending_closures4469lb # [ 64.882724] niks3-server[1024]: 2026/09/20 13:35:19 INFO Received uploads request method=POST path=/api/pending_closures4470worker1 # [ 64.910120] nix-grpc-daemon[861]: time=2026-09-20T13:35:19.471Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4471worker1 # [ 64.911731] nix-grpc-daemon[861]: time=2026-09-20T13:35:19.471Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4472worker1 # [ 64.913753] nix-grpc-daemon[861]: time=2026-09-20T13:35:19.471Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4473lb # [ 64.926385] niks3-server[1024]: 2026/09/20 13:35:19 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4474lb # [ 64.940525] niks3-server[1024]: 2026/09/20 13:35:19 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4475lb # [ 64.956321] niks3-server[1024]: 2026/09/20 13:35:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4476lb # [ 64.963701] niks3-server[1024]: 2026/09/20 13:35:19 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4477lb # [ 64.971918] niks3-server[1024]: 2026/09/20 13:35:19 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4478lb # [ 64.978228] niks3-server[1024]: 2026/09/20 13:35:19 INFO Signed narinfos id=19 count=24479lb # [ 64.980966] niks3-server[1024]: 2026/09/20 13:35:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4480lb # [ 64.984641] niks3-server[1024]: 2026/09/20 13:35:19 INFO Signed narinfos id=18 count=14481worker1 # [ 65.000789] nix-grpc-daemon[861]: time=2026-09-20T13:35:19.561Z level=INFO msg="Uploading 3 narinfos"4482lb # [ 65.003676] niks3-server[1024]: 2026/09/20 13:35:19 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4483lb # [ 65.015761] niks3-server[1024]: 2026/09/20 13:35:19 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4484lb # [ 65.022353] niks3-server[1024]: 2026/09/20 13:35:19 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4485lb # [ 65.026913] niks3-server[1024]: 2026/09/20 13:35:19 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4486lb # [ 65.029735] niks3-server[1024]: 2026/09/20 13:35:19 INFO Completed upload id=184487lb # [ 65.032801] niks3-server[1024]: 2026/09/20 13:35:19 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4488lb # [ 65.035754] niks3-server[1024]: 2026/09/20 13:35:19 INFO Completed upload id=194489worker1 # [ 65.054614] nix-grpc-daemon[861]: time=2026-09-20T13:35:19.616Z level=INFO msg="Upload complete. (577ms)"4490worker1 # [ 65.069377] nix-grpc-daemon[776]: ts=2026-09-20T13:35:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41102 duration_s=0 paths=2 nar_bytes_in=9844491worker1 # [ 65.090754] nix-grpc-daemon[776]: ts=2026-09-20T13:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411024492worker1 # [ 65.095433] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4493worker1 # [ 65.103924] nix-daemon[834]: reaped child process 942, status = succeeded4494client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4495worker1 # [ 65.114164] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4496client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4497worker1 # [ 66.350558] nix-daemon[834]: reaped child process 956, status = succeeded4498worker1 # [ 66.523838] nix-daemon[834]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4499worker1 # [ 66.540631] nix-daemon[834]: reaped child process 974, status = succeeded4500lb # [ 66.538110] niks3-server[1024]: 2026/09/20 13:35:21 INFO Received uploads request method=POST path=/api/pending_closures4501worker1 # [ 66.569978] nix-grpc-daemon[861]: time=2026-09-20T13:35:21.131Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4502lb # [ 66.576169] niks3-server[1024]: 2026/09/20 13:35:21 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4503worker1 # [ 66.605980] nix-grpc-daemon[861]: time=2026-09-20T13:35:21.167Z level=INFO msg="Uploading 1 narinfos"4504lb # [ 66.590449] niks3-server[1024]: 2026/09/20 13:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4505lb # [ 66.599358] niks3-server[1024]: 2026/09/20 13:35:21 INFO Signed narinfos id=20 count=14506lb # [ 66.602919] niks3-server[1024]: 2026/09/20 13:35:21 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4507lb # [ 66.611093] niks3-server[1024]: 2026/09/20 13:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4508lb # [ 66.615456] niks3-server[1024]: 2026/09/20 13:35:21 INFO Completed upload id=204509worker1 # [ 66.633595] nix-grpc-daemon[861]: time=2026-09-20T13:35:21.195Z level=INFO msg="Upload complete. (305ms)"4510lb # [ 66.618237] niks3-server[1024]: 2026/09/20 13:35:21 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4511client # copying 1 paths...4512worker2 # [ 66.626469] nix-grpc-daemon[777]: ts=2026-09-20T13:35:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:525444513worker1 # [ 66.645077] nix-grpc-daemon[776]: ts=2026-09-20T13:35:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41102 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14514worker1 # [ 66.659740] nix-daemon[834]: reaped child process 954, status = succeeded4515worker2 # [ 66.637693] nix-grpc-daemon[777]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4516client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4517worker2 # [ 66.686902] nix-grpc-daemon[777]: ts=2026-09-20T13:35:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:525444518worker2 # [ 66.692826] nix-grpc-daemon[777]: ts=2026-09-20T13:35:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_out=1204519client # [ 66.808767] nix-daemon[937]: reaped child process 1023, status = succeeded4520client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.27 seconds)4521client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24522client # [ 67.032917] nix-daemon[937]: accepted connection from pid 1042, user root (trusted)4523client # this derivation will be built:4524client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4525worker2 # [ 67.233067] nix-grpc-daemon[777]: ts=2026-09-20T13:35:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:525444526client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4527worker1 # [ 67.264818] nix-grpc-daemon[776]: ts=2026-09-20T13:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411024528worker1 # [ 67.274732] nix-grpc-daemon[861]: time=2026-09-20T13:35:21.836Z level=INFO msg="All 1 paths already cached"4529worker2 # [ 67.267574] nix-grpc-daemon[777]: ts=2026-09-20T13:35:21Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04530worker2 # [ 67.277162] nix-grpc-daemon[777]: ts=2026-09-20T13:35:21Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14531worker2 # [ 67.286066] nix-grpc-daemon[777]: ts=2026-09-20T13:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444532worker2 # [ 67.293832] nix-grpc-daemon[777]: ts=2026-09-20T13:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:525444533client # copying 1 paths to 'lb:50051'...4534client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4535worker2 # [ 67.307784] nix-grpc-daemon[777]: ts=2026-09-20T13:35:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:525444536worker2 # [ 67.323185] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4537worker2 # [ 67.344333] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)4538worker2 # [ 67.378239] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)4539worker2 # [ 67.408666] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 273 ms (attempt 3/5)4540worker2 # [ 67.690230] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 570 ms (attempt 4/5)4541worker2 # [ 68.260149] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4542worker2 # [ 68.270182] nix-grpc-daemon[777]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4543worker2 # [ 68.517350] nix-daemon[845]: accepted connection from pid 1189, user nix-grpc-daemon (trusted)4544worker2 # [ 68.535774] nix-daemon[845]: reaped child process 1192, status = succeeded4545worker2 # [ 68.714634] nix-daemon[845]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)4546worker2 # [ 68.731937] nix-daemon[845]: reaped child process 1197, status = succeeded4547lb # [ 68.754089] niks3-server[1024]: 2026/09/20 13:35:23 INFO Received uploads request method=POST path=/api/pending_closures4548worker2 # [ 68.754449] nix-grpc-daemon[858]: time=2026-09-20T13:35:23.316Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4549worker2 # [ 68.755936] nix-grpc-daemon[858]: time=2026-09-20T13:35:23.316Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4550lb # [ 68.779441] niks3-server[1024]: 2026/09/20 13:35:23 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4551lb # [ 68.795127] niks3-server[1024]: 2026/09/20 13:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4552worker2 # [ 68.794016] nix-grpc-daemon[858]: time=2026-09-20T13:35:23.356Z level=INFO msg="Uploading 1 narinfos"4553lb # [ 68.808105] niks3-server[1024]: 2026/09/20 13:35:23 INFO Signed narinfos id=21 count=14554lb # [ 68.814721] niks3-server[1024]: 2026/09/20 13:35:23 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4555worker2 # [ 68.817188] nix-grpc-daemon[858]: time=2026-09-20T13:35:23.380Z level=INFO msg="Upload complete. (514ms)"4556lb # [ 68.825270] niks3-server[1024]: 2026/09/20 13:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4557lb # [ 68.829384] niks3-server[1024]: 2026/09/20 13:35:23 INFO Completed upload id=214558lb # [ 68.833015] niks3-server[1024]: 2026/09/20 13:35:23 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4559worker2 # [ 68.824749] nix-grpc-daemon[777]: ts=2026-09-20T13:35:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=1 paths=1 nar_bytes_in=5684560worker2 # [ 68.836092] nix-grpc-daemon[777]: ts=2026-09-20T13:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:525444561worker2 # [ 68.846772] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4562worker2 # [ 68.851462] nix-daemon[845]: reaped child process 1183, status = succeeded4563worker2 # [ 68.866777] nix-grpc-daemon[777]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4564client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4565worker2 # [ 68.885200] nix-daemon[845]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4566client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4567worker2 # [ 70.109341] nix-daemon[845]: reaped child process 1202, status = succeeded4568worker2 # [ 70.278363] nix-daemon[845]: accepted connection from pid 1217, user nix-grpc-daemon (trusted)4569worker2 # [ 70.297671] nix-daemon[845]: reaped child process 1220, status = succeeded4570lb # [ 70.317761] niks3-server[1024]: 2026/09/20 13:35:24 INFO Received uploads request method=POST path=/api/pending_closures4571worker2 # [ 70.322298] nix-grpc-daemon[858]: time=2026-09-20T13:35:24.883Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4572lb # [ 70.351748] niks3-server[1024]: 2026/09/20 13:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4573worker2 # [ 70.355857] nix-grpc-daemon[858]: time=2026-09-20T13:35:24.918Z level=INFO msg="Uploading 1 narinfos"4574lb # [ 70.368221] niks3-server[1024]: 2026/09/20 13:35:24 INFO Signed narinfos id=22 count=14575lb # [ 70.371145] niks3-server[1024]: 2026/09/20 13:35:24 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4576worker2 # [ 70.376766] nix-grpc-daemon[858]: time=2026-09-20T13:35:24.939Z level=INFO msg="Upload complete. (276ms)"4577lb # [ 70.381114] niks3-server[1024]: 2026/09/20 13:35:24 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4578lb # [ 70.388887] niks3-server[1024]: 2026/09/20 13:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4579lb # [ 70.393161] niks3-server[1024]: 2026/09/20 13:35:24 INFO Completed upload id=224580worker2 # [ 70.385706] nix-grpc-daemon[858]: time=2026-09-20T13:35:24.942Z level=WARN msg="Failed to register uploaded object" key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo error="server returned 404: object is not pending upload\n"4581worker2 # [ 70.401129] nix-grpc-daemon[777]: ts=2026-09-20T13:35:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14582worker2 # [ 70.406402] nix-daemon[845]: reaped child process 1199, status = succeeded4583client # copying 1 paths...4584worker1 # [ 70.461890] nix-grpc-daemon[776]: ts=2026-09-20T13:35:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:411024585worker1 # [ 70.477432] nix-grpc-daemon[776]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4586client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4587worker1 # [ 70.524131] nix-grpc-daemon[776]: ts=2026-09-20T13:35:25Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:411024588worker1 # [ 70.530607] nix-grpc-daemon[776]: ts=2026-09-20T13:35:25Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41102 duration_s=0 paths=1 nar_bytes_out=1204589client # [ 70.630752] nix-daemon[937]: reaped child process 1045, status = succeeded4590client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.81 seconds)4591client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24592client # [ 70.851751] nix-daemon[937]: accepted connection from pid 1064, user root (trusted)4593client # this derivation will be built:4594client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4595worker1 # [ 71.080065] nix-grpc-daemon[776]: ts=2026-09-20T13:35:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:411024596client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4597worker1 # [ 71.087739] nix-grpc-daemon[776]: ts=2026-09-20T13:35:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411024598worker1 # [ 71.095385] nix-grpc-daemon[861]: time=2026-09-20T13:35:25.655Z level=INFO msg="All 1 paths already cached"4599worker2 # [ 71.081589] nix-grpc-daemon[777]: ts=2026-09-20T13:35:25Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04600worker2 # [ 71.082111] nix-grpc-daemon[777]: ts=2026-09-20T13:35:25Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14601worker1 # [ 71.123168] nix-grpc-daemon[776]: ts=2026-09-20T13:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411024602worker1 # [ 71.138174] nix-grpc-daemon[776]: ts=2026-09-20T13:35:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411024603worker1 # [ 71.152665] nix-grpc-daemon[861]: time=2026-09-20T13:35:25.714Z level=INFO msg="All 1 paths already cached"4604client # copying 1 paths to 'lb:50051'...4605client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4606worker1 # [ 71.162538] nix-grpc-daemon[776]: ts=2026-09-20T13:35:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:411024607worker1 # [ 71.177586] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4608worker1 # [ 71.389124] nix-daemon[834]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4609worker1 # [ 71.409437] nix-daemon[834]: reaped child process 982, status = succeeded4610worker1 # [ 71.581468] nix-daemon[834]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4611worker1 # [ 71.600960] nix-daemon[834]: reaped child process 987, status = succeeded4612lb # [ 71.597246] niks3-server[1024]: 2026/09/20 13:35:26 INFO Received uploads request method=POST path=/api/pending_closures4613worker1 # [ 71.623719] nix-grpc-daemon[861]: time=2026-09-20T13:35:26.185Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4614worker1 # [ 71.625356] nix-grpc-daemon[861]: time=2026-09-20T13:35:26.185Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4615lb # [ 71.626446] niks3-server[1024]: 2026/09/20 13:35:26 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4616lb # [ 71.639240] niks3-server[1024]: 2026/09/20 13:35:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4617worker1 # [ 71.664654] nix-grpc-daemon[861]: time=2026-09-20T13:35:26.226Z level=INFO msg="Uploading 1 narinfos"4618lb # [ 71.653250] niks3-server[1024]: 2026/09/20 13:35:26 INFO Signed narinfos id=23 count=14619lb # [ 71.661374] niks3-server[1024]: 2026/09/20 13:35:26 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4620lb # [ 71.672231] niks3-server[1024]: 2026/09/20 13:35:26 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4621lb # [ 71.677087] niks3-server[1024]: 2026/09/20 13:35:26 INFO Completed upload id=234622worker1 # [ 71.695476] nix-grpc-daemon[861]: time=2026-09-20T13:35:26.257Z level=INFO msg="Upload complete. (500ms)"4623lb # [ 71.679634] niks3-server[1024]: 2026/09/20 13:35:26 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4624worker1 # [ 71.705840] nix-grpc-daemon[776]: ts=2026-09-20T13:35:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41102 duration_s=0 paths=1 nar_bytes_in=5684625worker1 # [ 71.719775] nix-grpc-daemon[776]: ts=2026-09-20T13:35:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411024626worker1 # [ 71.726421] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4627worker1 # [ 71.731213] nix-daemon[834]: reaped child process 977, status = succeeded4628client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4629worker1 # [ 71.745454] nix-daemon[834]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4630client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4631lb # [ 72.122804] envoy[921]: conn peer=[2001:db8:1::1]:36680 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4296 tx=2183 ms=17024632lb # [ 72.128050] envoy[921]: conn peer=[2001:db8:1::1]:60898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=244633lb # [ 72.132737] envoy[921]: conn peer=[2001:db8:1::1]:60906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3851 tx=1876 ms=27544634lb # [ 72.137615] envoy[921]: conn peer=[2001:db8:1::1]:60922 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=634635lb # [ 72.142317] envoy[921]: conn peer=[2001:db8:1::1]:60928 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=1919 ms=33284636lb # [ 72.147105] envoy[921]: conn peer=[2001:db8:1::1]:60942 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=684637worker1 # [ 72.772999] nix-daemon[834]: reaped child process 991, status = succeeded4638worker1 # [ 72.929246] nix-daemon[834]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4639worker1 # [ 72.948720] nix-daemon[834]: reaped child process 1009, status = succeeded4640lb # [ 72.941976] niks3-server[1024]: 2026/09/20 13:35:27 INFO Received uploads request method=POST path=/api/pending_closures4641worker1 # [ 72.971604] nix-grpc-daemon[861]: time=2026-09-20T13:35:27.532Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4642lb # [ 72.972292] niks3-server[1024]: 2026/09/20 13:35:27 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4643worker1 # [ 73.003945] nix-grpc-daemon[861]: time=2026-09-20T13:35:27.565Z level=INFO msg="Uploading 1 narinfos"4644lb # [ 72.985280] niks3-server[1024]: 2026/09/20 13:35:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4645lb # [ 72.995861] niks3-server[1024]: 2026/09/20 13:35:27 INFO Signed narinfos id=24 count=14646lb # [ 73.008193] niks3-server[1024]: 2026/09/20 13:35:27 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4647worker1 # [ 73.034363] nix-grpc-daemon[861]: time=2026-09-20T13:35:27.596Z level=INFO msg="Upload complete. (278ms)"4648lb # [ 73.016425] niks3-server[1024]: 2026/09/20 13:35:27 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4649lb # [ 73.022032] niks3-server[1024]: 2026/09/20 13:35:27 INFO Completed upload id=244650lb # [ 73.024734] niks3-server[1024]: 2026/09/20 13:35:27 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4651worker1 # [ 73.044833] nix-grpc-daemon[776]: ts=2026-09-20T13:35:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41102 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14652worker1 # [ 73.056227] nix-daemon[834]: reaped child process 989, status = succeeded4653client # copying 1 paths...4654worker1 # [ 73.092384] nix-grpc-daemon[776]: ts=2026-09-20T13:35:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:411024655client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4656worker2 # [ 73.101688] nix-grpc-daemon[777]: ts=2026-09-20T13:35:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:525444657worker2 # [ 73.116424] nix-grpc-daemon[777]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4658worker2 # [ 73.133997] nix-grpc-daemon[777]: ts=2026-09-20T13:35:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52544 duration_s=0 paths=1 nar_bytes_out=1204659client # [ 73.269409] nix-daemon[937]: reaped child process 1067, status = succeeded4660client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.63 seconds)4661(finished: subtest: an input only one worker has still reaches the builder, in 10.65 seconds)4662subtest: upload whose reference the worker lacks is completed from the cache4663client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4664client # this derivation will be built:4665client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4666client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4667client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.46 seconds)4668client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4669client # this derivation will be built:4670client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4671client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4672client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.44 seconds)4673client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4674client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4675worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4676worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds)4677worker2 # [ 74.691755] systemd[1]: Stopping Nix worker-protocol over gRPC...4678worker2 # [ 75.591205] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=signal_exit4679worker2 # [ 75.644936] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=scheduler_yield4680lb # [ 75.659566] niks3-server[1024]: 2026/09/20 13:35:30 INFO lead: released remote=192.168.1.4:351284681worker1 # [ 75.677914] nix-grpc-daemon[776]: ts=2026-09-20T13:35:30Z level=info event=scheduler_restarting addr=lb:500514682worker2 # [ 75.651822] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:525324683worker2 # [ 75.658271] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=worker_gone addr=192.168.1.3:500514684worker2 # [ 75.662421] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=scheduler_restarting addr=lb:500514685worker2 # [ 75.667386] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:525324686worker2 # [ 75.671916] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=worker_gone addr=192.168.1.4:500514687worker1 # [ 75.877823] nix-grpc-daemon[776]: ts=2026-09-20T13:35:30Z level=info event=scheduler_connected addr=lb:500514688worker1 # [ 75.882996] nix-grpc-daemon[776]: ts=2026-09-20T13:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4689worker2 # [ 75.860103] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=scheduler_connected addr=lb:500514690worker2 # [ 75.864986] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4691lb # [ 75.890336] niks3-server[1024]: 2026/09/20 13:35:30 INFO lead: acquired remote=192.168.1.3:547824692worker1 # [ 75.913007] nix-grpc-daemon[776]: ts=2026-09-20T13:35:30Z level=info event=scheduler_take_over4693worker1 # [ 76.083849] nix-grpc-daemon[776]: ts=2026-09-20T13:35:30Z level=info event=scheduler_connected addr=lb:500514694worker1 # [ 76.089205] nix-grpc-daemon[776]: ts=2026-09-20T13:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4695worker2 # [ 76.065950] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=scheduler_connected addr=lb:500514696worker2 # [ 76.071126] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4697worker1 # [ 76.289770] nix-grpc-daemon[776]: ts=2026-09-20T13:35:30Z level=info event=scheduler_connected addr=lb:500514698worker1 # [ 76.295094] nix-grpc-daemon[776]: ts=2026-09-20T13:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4699worker2 # [ 76.272423] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=scheduler_connected addr=lb:500514700worker2 # [ 76.277465] nix-grpc-daemon[777]: ts=2026-09-20T13:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4701worker1 # [ 76.495699] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_connected addr=lb:500514702worker1 # [ 76.501665] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4703worker2 # [ 76.478462] nix-grpc-daemon[777]: ts=2026-09-20T13:35:31Z level=info event=scheduler_connected addr=lb:500514704worker2 # [ 76.484324] nix-grpc-daemon[777]: ts=2026-09-20T13:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4705worker2 # [ 76.657458] nix-grpc-daemon[777]: ts=2026-09-20T13:35:31Z level=info event=scheduler_restarting4706worker1 # [ 76.704160] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_connected addr=lb:500514707worker1 # [ 76.708168] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4708worker2 # [ 76.696411] nix-grpc-daemon[777]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4709worker2 # [ 76.714218] nix-grpc-daemon[777]: E0000 00:00:1789911331.256604 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4710worker2 # [ 76.719229] nix-grpc-daemon[777]: E0000 00:00:1789911331.264077 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true4711worker2 # [ 76.723791] nix-grpc-daemon[777]: ts=2026-09-20T13:35:31Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4712worker2 # [ 76.727746] nix-daemon[845]: reaped child process 847, status = succeeded4713worker2 # [ 76.772755] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4714worker2 # [ 76.774532] systemd[1]: Stopped Nix worker-protocol over gRPC.4715worker2 # [ 76.775893] systemd[1]: nix-grpc-daemon.service: Consumed 3.581s CPU time over 54.505s wall clock time, 63.2M memory peak, 89.3K incoming IP traffic, 202.4K outgoing IP traffic.4716worker2 # [ 76.788450] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4717worker2 # [ 76.790051] systemd[1]: Closed Nix worker-protocol over gRPC.4718worker2 # [ 76.793245] systemd[1]: nix-grpc-daemon.socket: Consumed 720us CPU time over 1min 4.608s wall clock time, 256K memory peak, 89.9K incoming IP traffic, 69.1K outgoing IP traffic.4719lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4720lb: must succeed: curl -sf localhost:9901/clusters4721worker1 # [ 76.904900] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_connected addr=lb:500514722worker1 # [ 76.910107] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4723lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4724lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.10 seconds)4725client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24726worker1 # [ 77.110922] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_connected addr=lb:500514727worker1 # [ 77.116177] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4728worker1 # [ 77.159341] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:411024729client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4730worker1 # [ 77.166533] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:411024731client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.27 seconds)4732worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4733worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4734worker2 # [ 77.274683] systemd[1]: Listening on Nix worker-protocol over gRPC.4735worker1 # [ 77.317162] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_connected addr=lb:500514736worker1 # [ 77.323852] nix-grpc-daemon[776]: ts=2026-09-20T13:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4737worker1 # [ 77.388994] systemd[1]: Stopping Nix worker-protocol over gRPC...4738worker1 # [ 77.501390] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=signal_exit4739worker1 # [ 77.523626] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_connected addr=lb:500514740worker1 # [ 77.529005] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4741worker1 # [ 77.731146] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_connected addr=lb:500514742worker1 # [ 77.750296] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:340264743worker1 # [ 77.756110] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04744worker1 # [ 77.762947] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_yield4745worker1 # [ 77.772240] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_restarting addr=lb:500514746lb # [ 77.753044] niks3-server[1024]: 2026/09/20 13:35:32 INFO lead: released remote=192.168.1.3:547824747worker1 # [ 77.778247] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:340264748worker1 # [ 77.783182] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=worker_gone addr=192.168.1.3:500514749worker2 # [ 77.764304] systemd[1]: Starting scheduler-token.service...4750worker2 # [ 77.868760] systemd[1]: scheduler-token.service: Deactivated successfully.4751worker2 # [ 77.875263] systemd[1]: Finished scheduler-token.service.4752worker2 # [ 77.879814] systemd[1]: scheduler-token.service: Consumed 52ms CPU time over 107ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4753worker2 # [ 77.884344] systemd[1]: Starting Nix worker-protocol over gRPC...4754worker1 # [ 77.973291] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_connected addr=lb:500514755worker1 # [ 77.980333] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4756worker2 # [ 78.071821] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4757worker2 # [ 78.074587] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4758worker2 # [ 78.078805] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14759worker2 # [ 78.087589] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4760worker2 # [ 78.097961] systemd[1]: Started Nix worker-protocol over gRPC.4761worker2 # [ 78.101463] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:32Z level=info event=scheduler_take_over4762lb # [ 78.108940] niks3-server[1024]: 2026/09/20 13:35:32 INFO lead: acquired remote=192.168.1.4:491384763worker2 # [ 78.122871] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:32Z level=info event=scheduler_connected addr=lb:500514764worker2 # [ 78.125522] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4765worker1 # [ 78.181376] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_connected addr=lb:500514766worker1 # [ 78.187781] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4767worker2 # [ 78.327450] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:32Z level=info event=scheduler_connected addr=lb:500514768worker2 # [ 78.333184] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4769worker1 # [ 78.388741] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_connected addr=lb:500514770worker1 # [ 78.395107] nix-grpc-daemon[776]: ts=2026-09-20T13:35:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4771worker2 # [ 78.534200] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:33Z level=info event=scheduler_connected addr=lb:500514772worker2 # [ 78.539721] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4773worker1 # [ 78.595961] nix-grpc-daemon[776]: ts=2026-09-20T13:35:33Z level=info event=scheduler_connected addr=lb:500514774worker1 # [ 78.602336] nix-grpc-daemon[776]: ts=2026-09-20T13:35:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4775worker2 # [ 78.740916] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:33Z level=info event=scheduler_connected addr=lb:500514776worker1 # [ 78.771793] nix-grpc-daemon[776]: ts=2026-09-20T13:35:33Z level=info event=scheduler_restarting4777worker2 # [ 78.758977] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:33Z level=info event=oidc_jwks provider=mock keys=14778worker2 # [ 78.763639] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:33Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:605384779worker2 # [ 78.769069] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:33Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04780worker1 # [ 78.800330] nix-daemon[834]: reaped child process 836, status = succeeded4781worker1 # [ 78.820605] nix-grpc-daemon[776]: ts=2026-09-20T13:35:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4782worker1 # [ 79.592552] nix-grpc-daemon[776]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4783worker1 # [ 79.592967] nix-grpc-daemon[776]: E0000 00:00:1789911334.154340 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 12, started true, shutdown true4784worker1 # [ 79.621829] nix-grpc-daemon[776]: E0000 00:00:1789911334.183737 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4785worker1 # [ 80.720803] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4786worker1 # [ 80.727077] systemd[1]: Stopped Nix worker-protocol over gRPC.4787worker1 # [ 80.733101] systemd[1]: nix-grpc-daemon.service: Consumed 1.937s CPU time over 58.568s wall clock time, 48.6M memory peak, 54.1K incoming IP traffic, 94.9K outgoing IP traffic.4788worker1 # [ 80.746707] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4789worker1 # [ 80.751352] systemd[1]: Closed Nix worker-protocol over gRPC.4790worker1 # [ 80.754442] systemd[1]: nix-grpc-daemon.socket: Consumed 683us CPU time over 1min 9.239s wall clock time, 256K memory peak, 66.1K incoming IP traffic, 57.7K outgoing IP traffic.4791lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4792lb: must succeed: curl -sf localhost:9901/clusters4793lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4794lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4795client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24796worker2 # [ 81.082994] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:371324797worker2 # [ 81.099144] nix-daemon[845]: accepted connection from pid 1247, user nix-grpc-daemon (trusted)4798worker2 # [ 81.124890] nix-grpc-daemon[1247]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 1/5)4799worker2 # [ 81.136442] nix-grpc-daemon[1247]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 194 ms (attempt 2/5)4800worker2 # [ 81.329114] nix-grpc-daemon[1247]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 3/5)4801worker2 # [ 81.419523] nix-grpc-daemon[1247]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 4/5)4802worker2 # [ 81.480791] nix-grpc-daemon[1247]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4803client # copying 2 paths...4804client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4805worker2 # [ 81.507287] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:371324806client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4807worker2 # [ 81.526813] nix-daemon[845]: accepted connection from pid 1247, user nix-grpc-daemon (trusted)4808worker2 # [ 81.555644] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:36Z level=info event=niks3_push_started pid=13034809worker2 # [ 81.781091] nix-grpc-daemon[1303]: time=2026-09-20T13:35:36.343Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4810worker2 # [ 81.957238] nix-daemon[845]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)4811worker2 # [ 81.975128] nix-daemon[845]: reaped child process 1319, status = succeeded4812lb # [ 82.001747] niks3-server[1024]: 2026/09/20 13:35:36 INFO Received uploads request method=POST path=/api/pending_closures4813lb # [ 82.010127] niks3-server[1024]: 2026/09/20 13:35:36 INFO Received uploads request method=POST path=/api/pending_closures4814worker2 # [ 82.012093] nix-grpc-daemon[1303]: time=2026-09-20T13:35:36.573Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4815worker2 # [ 82.013815] nix-grpc-daemon[1303]: time=2026-09-20T13:35:36.573Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4816worker2 # [ 82.015566] nix-grpc-daemon[1303]: time=2026-09-20T13:35:36.574Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4817lb # [ 82.065629] niks3-server[1024]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4818lb # [ 82.076137] niks3-server[1024]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4819lb # [ 82.089855] niks3-server[1024]: 2026/09/20 13:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4820lb # [ 82.100080] niks3-server[1024]: 2026/09/20 13:35:36 INFO Signed narinfos id=26 count=24821lb # [ 82.107103] niks3-server[1024]: 2026/09/20 13:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4822lb # [ 82.118833] niks3-server[1024]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4823lb # [ 82.130338] envoy[921]: conn peer=[2001:db8:1::1]:46004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3713 tx=1803 ms=21394824worker2 # [ 82.124296] nix-grpc-daemon[1303]: time=2026-09-20T13:35:36.684Z level=INFO msg="Uploading 3 narinfos"4825lb # [ 82.140075] envoy[921]: conn peer=[2001:db8:1::1]:46016 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=964826lb # [ 82.142862] envoy[921]: conn peer=[2001:db8:1::4]:36678 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=39206 tx=2485 ms=541104827lb # [ 82.153200] envoy[921]: conn peer=[2001:db8:1::1]:46028 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=224828lb # [ 82.155648] envoy[921]: conn peer=[2001:db8:1::3]:45032 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=8192 tx=2247 ms=551344829lb # [ 82.166268] niks3-server[1024]: 2026/09/20 13:35:36 INFO Signed narinfos id=25 count=14830lb # [ 82.175649] niks3-server[1024]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4831lb # [ 82.180840] niks3-server[1024]: 2026/09/20 13:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4832lb # [ 82.184081] niks3-server[1024]: 2026/09/20 13:35:36 INFO Completed upload id=254833lb # [ 82.186931] niks3-server[1024]: 2026/09/20 13:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4834lb # [ 82.189176] niks3-server[1024]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4835lb # [ 82.192003] niks3-server[1024]: 2026/09/20 13:35:36 INFO Completed upload id=264836worker2 # [ 82.184629] nix-grpc-daemon[1303]: time=2026-09-20T13:35:36.747Z level=INFO msg="Upload complete. (400ms)"4837lb # [ 82.194199] niks3-server[1024]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4838lb # [ 82.200300] niks3-server[1024]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4839worker2 # [ 82.197384] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37132 duration_s=0 paths=2 nar_bytes_in=3124840worker2 # [ 82.217032] nix-daemon[845]: reaped child process 1301, status = succeeded4841client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.42 seconds)4842worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4843worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)4844worker1 # [ 82.396621] systemd[1]: Listening on Nix worker-protocol over gRPC.4845worker1 # [ 82.401864] systemd[1]: Starting Nix worker-protocol over gRPC...4846worker1 # [ 82.598158] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4847worker1 # [ 82.600848] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4848worker1 # [ 82.605020] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14849worker1 # [ 82.613063] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4850worker1 # [ 82.614811] systemd[1]: Started Nix worker-protocol over gRPC.4851lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4852lb: must succeed: curl -sf localhost:9901/clusters4853worker1 # [ 82.640988] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:37Z level=info event=scheduler_connected addr=lb:500514854worker2 # [ 82.618236] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:605384855worker2 # [ 82.624196] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04856lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4857lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)4858(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.42 seconds)4859subtest: scheduler down: the other node takes the lock and keeps it4860lb: must succeed: curl -sf localhost:9901/clusters4861lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4862lb: must succeed: curl -sf localhost:9901/clusters4863lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4864lb: must succeed: curl -sf localhost:9901/clusters4865lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4866lb: must succeed: curl -sf localhost:9901/clusters4867lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4868worker2 # [ 82.973081] systemd[1]: Stopping Nix worker-protocol over gRPC...4869worker2 # [ 83.090199] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=signal_exit4870worker2 # [ 83.102284] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=scheduler_yield4871lb # [ 83.114162] niks3-server[1024]: 2026/09/20 13:35:37 INFO lead: released remote=192.168.1.4:491384872worker1 # [ 83.133804] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:37Z level=info event=scheduler_restarting addr=lb:500514873worker2 # [ 83.117260] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:605384874worker2 # [ 83.123962] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=worker_gone addr=192.168.1.3:500514875worker2 # [ 83.128114] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=scheduler_restarting addr=lb:500514876worker2 # [ 83.131986] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:605384877worker2 # [ 83.136371] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=worker_gone addr=192.168.1.4:500514878worker1 # [ 83.335113] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:37Z level=info event=scheduler_connected addr=lb:500514879worker1 # [ 83.340655] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4880worker2 # [ 83.326458] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=scheduler_connected addr=lb:500514881worker2 # [ 83.330701] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4882worker1 # [ 83.541600] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_connected addr=lb:500514883worker1 # [ 83.546978] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4884worker2 # [ 83.526962] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:38Z level=info event=scheduler_connected addr=lb:500514885worker2 # [ 83.532314] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4886worker1 # [ 83.749629] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_connected addr=lb:500514887worker1 # [ 83.753830] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4888worker2 # [ 83.733335] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:38Z level=info event=scheduler_connected addr=lb:500514889worker2 # [ 83.738541] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4890worker1 # [ 83.950231] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_connected addr=lb:500514891worker1 # [ 83.955545] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4892worker2 # [ 83.939489] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:38Z level=info event=scheduler_connected addr=lb:500514893worker2 # [ 83.944594] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4894worker2 # [ 84.107451] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:38Z level=info event=scheduler_restarting4895worker1 # [ 84.156342] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_connected addr=lb:500514896worker2 # [ 84.136168] nix-daemon[845]: reaped child process 1296, status = succeeded4897worker1 # [ 84.161542] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4898worker2 # [ 84.152704] nix-grpc-daemon[1247]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4899worker2 # [ 84.165807] nix-grpc-daemon[1247]: E0000 00:00:1789911338.715454 1290 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4900worker2 # [ 84.172982] nix-grpc-daemon[1247]: E0000 00:00:1789911338.715636 1290 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true4901worker2 # [ 84.178891] nix-grpc-daemon[1247]: ts=2026-09-20T13:35:38Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4902worker1 # [ 84.362548] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_connected addr=lb:500514903worker1 # [ 84.367963] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4904worker1 # [ 84.569113] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_connected addr=lb:500514905worker1 # [ 84.574529] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4906worker1 # [ 84.777290] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_connected addr=lb:500514907worker1 # [ 84.781489] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4908worker1 # [ 84.977839] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_connected addr=lb:500514909worker1 # [ 84.983195] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4910worker1 # [ 85.184379] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_connected addr=lb:500514911worker1 # [ 85.189807] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4912worker1 # [ 85.390813] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_connected addr=lb:500514913worker1 # [ 85.396498] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4914worker1 # [ 85.597513] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_connected addr=lb:500514915worker1 # [ 85.603108] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4916worker1 # [ 85.805763] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_connected addr=lb:500514917worker1 # [ 85.809836] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4918worker1 # [ 86.006386] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_connected addr=lb:500514919worker1 # [ 86.011732] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4920worker2 # [ 86.129126] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4921worker2 # [ 86.133764] systemd[1]: Stopped Nix worker-protocol over gRPC.4922worker2 # [ 86.140708] systemd[1]: nix-grpc-daemon.service: Consumed 457ms CPU time over 8.253s wall clock time, 30.9M memory peak, 20.1K incoming IP traffic, 29K outgoing IP traffic.4923worker2 # [ 86.153850] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4924worker2 # [ 86.162292] systemd[1]: Closed Nix worker-protocol over gRPC.4925worker2 # [ 86.165052] systemd[1]: nix-grpc-daemon.socket: Consumed 851us CPU time over 8.896s wall clock time, 256K memory peak, 18.7K incoming IP traffic, 13.6K outgoing IP traffic.4926worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over4927worker1 # [ 86.212778] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_connected addr=lb:500514928worker1 # [ 86.222620] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4929worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)4930worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4931worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4932worker1 # [ 86.419711] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_connected addr=lb:500514933worker1 # [ 86.424470] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4934worker1 # [ 86.625257] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:41Z level=info event=scheduler_connected addr=lb:500514935worker1 # [ 86.630474] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4936worker1 # [ 86.832880] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:41Z level=info event=scheduler_connected addr=lb:500514937worker1 # [ 86.837623] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4938worker1 # [ 87.038656] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:41Z level=info event=scheduler_connected addr=lb:500514939worker1 # [ 87.044218] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4940worker1 # [ 87.245075] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:41Z level=info event=scheduler_connected addr=lb:500514941worker1 # [ 87.250584] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4942worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4943worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4944worker1 # [ 87.451479] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:42Z level=info event=scheduler_connected addr=lb:500514945worker1 # [ 87.456897] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4946worker1 # [ 87.625780] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:42Z level=info event=scheduler_take_over4947lb # [ 87.608263] niks3-server[1024]: 2026/09/20 13:35:42 INFO lead: acquired remote=192.168.1.3:405964948worker1 # [ 87.657738] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:42Z level=info event=scheduler_connected addr=lb:500514949worker1 # [ 87.662908] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4950worker1 # [ 87.865289] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:42Z level=info event=scheduler_connected addr=lb:500514951worker1 # [ 87.874331] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:510604952worker1 # [ 87.880100] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04953worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4954worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4955client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&24956worker1 # [ 88.761027] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:510744957worker1 # [ 88.777174] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4958client # don't know how to build these paths:4959client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv4960worker1 # [ 88.798904] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:510744961worker1 # [ 88.815762] nix-grpc-daemon[1032]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5)4962worker1 # [ 88.863625] nix-grpc-daemon[1032]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 2/5)4963worker1 # [ 88.969674] nix-grpc-daemon[1032]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 3/5)4964worker1 # [ 89.116744] nix-grpc-daemon[1032]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 686 ms (attempt 4/5)4965worker1 # [ 89.803977] nix-grpc-daemon[1032]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4966worker1 # [ 89.840169] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:44Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=04967worker1 # [ 89.846896] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:44Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=14968worker1 # [ 89.898190] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510744969worker1 # [ 89.915223] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:510744970client # copying 1 paths to 'lb:50051'...4971client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...4972worker1 # [ 89.930961] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:510744973worker1 # [ 89.948804] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4974worker1 # [ 89.983016] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:44Z level=info event=niks3_push_started pid=11054975worker1 # [ 90.190001] nix-grpc-daemon[1105]: time=2026-09-20T13:35:44.751Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""4976worker1 # [ 90.364546] nix-daemon[834]: accepted connection from pid 1118, user nix-grpc-daemon (trusted)4977worker1 # [ 90.381794] nix-daemon[834]: reaped child process 1121, status = succeeded4978worker1 # [ 90.556766] nix-daemon[834]: accepted connection from pid 1123, user nix-grpc-daemon (trusted)4979worker1 # [ 90.573120] nix-daemon[834]: reaped child process 1126, status = succeeded4980lb # [ 90.569358] niks3-server[1024]: 2026/09/20 13:35:45 INFO Received uploads request method=POST path=/api/pending_closures4981worker1 # [ 90.599537] nix-grpc-daemon[1105]: time=2026-09-20T13:35:45.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4982worker1 # [ 90.601383] nix-grpc-daemon[1105]: time=2026-09-20T13:35:45.160Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"4983lb # [ 90.608612] niks3-server[1024]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst4984worker1 # [ 90.641669] nix-grpc-daemon[1105]: time=2026-09-20T13:35:45.203Z level=INFO msg="Uploading 1 narinfos"4985lb # [ 90.622379] niks3-server[1024]: 2026/09/20 13:35:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4986lb # [ 90.633657] niks3-server[1024]: 2026/09/20 13:35:45 INFO Signed narinfos id=27 count=14987lb # [ 90.640340] niks3-server[1024]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls4988lb # [ 90.650079] niks3-server[1024]: 2026/09/20 13:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4989worker1 # [ 90.676860] nix-grpc-daemon[1105]: time=2026-09-20T13:35:45.238Z level=INFO msg="Upload complete. (483ms)"4990lb # [ 90.659167] niks3-server[1024]: 2026/09/20 13:35:45 INFO Completed upload id=274991lb # [ 90.662162] niks3-server[1024]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo4992worker1 # [ 90.684130] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51074 duration_s=0 paths=1 nar_bytes_in=5204993worker1 # [ 90.699436] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510744994worker1 # [ 90.709577] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4995worker1 # [ 90.712378] nix-daemon[834]: reaped child process 1103, status = succeeded4996client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...4997worker1 # [ 90.723887] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4998client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv4999client # farm-a> LOG-farm-a-on-standby5000lb # [ 92.128133] envoy[921]: conn peer=[2001:db8:1::1]:57716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1008 tx=305 ms=11295001lb # [ 92.135513] envoy[921]: conn peer=[2001:db8:1::4]:36416 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6441 tx=485 ms=60335002worker1 # [ 92.173126] nix-daemon[834]: reaped child process 1130, status = succeeded5003worker1 # [ 92.352790] nix-daemon[834]: accepted connection from pid 1145, user nix-grpc-daemon (trusted)5004worker1 # [ 92.372327] nix-daemon[834]: reaped child process 1148, status = succeeded5005lb # [ 92.365875] niks3-server[1024]: 2026/09/20 13:35:46 INFO Received uploads request method=POST path=/api/pending_closures5006worker1 # [ 92.396409] nix-grpc-daemon[1105]: time=2026-09-20T13:35:46.955Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5007worker1 # [ 92.397882] nix-grpc-daemon[1105]: time=2026-09-20T13:35:46.955Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5008lb # [ 92.408619] niks3-server[1024]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5009lb # [ 92.420953] niks3-server[1024]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5010lb # [ 92.430072] niks3-server[1024]: 2026/09/20 13:35:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5011lb # [ 92.435871] niks3-server[1024]: 2026/09/20 13:35:46 INFO Signed narinfos id=28 count=15012worker1 # [ 92.455086] nix-grpc-daemon[1105]: time=2026-09-20T13:35:47.015Z level=INFO msg="Uploading 1 narinfos"5013lb # [ 92.442161] niks3-server[1024]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5014lb # [ 92.450590] niks3-server[1024]: 2026/09/20 13:35:47 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5015lb # [ 92.456465] niks3-server[1024]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5016lb # [ 92.461688] niks3-server[1024]: 2026/09/20 13:35:47 INFO Completed upload id=285017worker1 # [ 92.480464] nix-grpc-daemon[1105]: time=2026-09-20T13:35:47.042Z level=INFO msg="Upload complete. (338ms)"5018worker1 # [ 92.501095] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51074 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15019worker1 # [ 92.516199] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:47Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05020worker1 # [ 92.519142] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:47Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15021worker1 # [ 92.522791] nix-daemon[834]: reaped child process 1128, status = succeeded5022worker1 # [ 92.547207] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510745023worker1 # [ 92.562076] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:510745024client # copying 1 paths to 'lb:50051'...5025worker1 # [ 92.577649] nix-grpc-daemon[1105]: time=2026-09-20T13:35:47.138Z level=INFO msg="All 1 paths already cached"5026client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5027worker1 # [ 92.583591] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:510745028worker1 # [ 92.601760] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5029worker1 # [ 92.809847] nix-daemon[834]: accepted connection from pid 1154, user nix-grpc-daemon (trusted)5030worker1 # [ 92.827558] nix-daemon[834]: reaped child process 1157, status = succeeded5031worker1 # [ 92.999904] nix-daemon[834]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5032worker1 # [ 93.017509] nix-daemon[834]: reaped child process 1162, status = succeeded5033lb # [ 93.009513] niks3-server[1024]: 2026/09/20 13:35:47 INFO Received uploads request method=POST path=/api/pending_closures5034worker1 # [ 93.036835] nix-grpc-daemon[1105]: time=2026-09-20T13:35:47.598Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5035worker1 # [ 93.038324] nix-grpc-daemon[1105]: time=2026-09-20T13:35:47.598Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5036lb # [ 93.034744] niks3-server[1024]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5037lb # [ 93.048601] niks3-server[1024]: 2026/09/20 13:35:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5038worker1 # [ 93.073789] nix-grpc-daemon[1105]: time=2026-09-20T13:35:47.635Z level=INFO msg="Uploading 1 narinfos"5039lb # [ 93.060229] niks3-server[1024]: 2026/09/20 13:35:47 INFO Signed narinfos id=29 count=15040lb # [ 93.067678] niks3-server[1024]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5041lb # [ 93.076424] niks3-server[1024]: 2026/09/20 13:35:47 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5042worker1 # [ 93.102141] nix-grpc-daemon[1105]: time=2026-09-20T13:35:47.664Z level=INFO msg="Upload complete. (483ms)"5043lb # [ 93.084737] niks3-server[1024]: 2026/09/20 13:35:47 INFO Completed upload id=295044lb # [ 93.088668] niks3-server[1024]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5045worker1 # [ 93.112994] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51074 duration_s=0 paths=1 nar_bytes_in=6405046worker1 # [ 93.122535] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510745047worker1 # [ 93.129497] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5048worker1 # [ 93.134821] nix-daemon[834]: reaped child process 1152, status = succeeded5049client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5050client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5051worker1 # [ 93.147367] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5052client # farm-b> LOG-farm-b-on-standby5053worker1 # [ 94.592951] nix-daemon[834]: reaped child process 1166, status = succeeded5054worker1 # [ 94.778809] nix-daemon[834]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)5055worker1 # [ 94.796241] nix-daemon[834]: reaped child process 1184, status = succeeded5056lb # [ 94.789647] niks3-server[1024]: 2026/09/20 13:35:49 INFO Received uploads request method=POST path=/api/pending_closures5057worker1 # [ 94.819576] nix-grpc-daemon[1105]: time=2026-09-20T13:35:49.380Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5058worker1 # [ 94.821203] nix-grpc-daemon[1105]: time=2026-09-20T13:35:49.380Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5059lb # [ 94.826345] niks3-server[1024]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5060lb # [ 94.837533] niks3-server[1024]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5061lb # [ 94.849231] niks3-server[1024]: 2026/09/20 13:35:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5062worker1 # [ 94.874419] nix-grpc-daemon[1105]: time=2026-09-20T13:35:49.435Z level=INFO msg="Uploading 1 narinfos"5063lb # [ 94.857019] niks3-server[1024]: 2026/09/20 13:35:49 INFO Signed narinfos id=30 count=15064lb # [ 94.859635] niks3-server[1024]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5065lb # [ 94.871106] niks3-server[1024]: 2026/09/20 13:35:49 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5066lb # [ 94.877852] niks3-server[1024]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5067lb # [ 94.882367] niks3-server[1024]: 2026/09/20 13:35:49 INFO Completed upload id=305068worker1 # [ 94.901748] nix-grpc-daemon[1105]: time=2026-09-20T13:35:49.463Z level=INFO msg="Upload complete. (342ms)"5069worker1 # [ 94.919067] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51074 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15070worker1 # [ 94.936100] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:49Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05071worker1 # [ 94.939544] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:49Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15072worker1 # [ 94.943691] nix-daemon[834]: reaped child process 1164, status = succeeded5073worker1 # [ 94.962949] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510745074worker1 # [ 94.978736] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:510745075worker1 # [ 94.992729] nix-grpc-daemon[1105]: time=2026-09-20T13:35:49.554Z level=INFO msg="All 2 paths already cached"5076client # copying 1 paths to 'lb:50051'...5077client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5078worker1 # [ 95.002176] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:510745079worker1 # [ 95.016179] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5080worker1 # [ 95.211306] nix-daemon[834]: accepted connection from pid 1190, user nix-grpc-daemon (trusted)5081worker1 # [ 95.232158] nix-daemon[834]: reaped child process 1193, status = succeeded5082worker1 # [ 95.404079] nix-daemon[834]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)5083worker1 # [ 95.421507] nix-daemon[834]: reaped child process 1198, status = succeeded5084lb # [ 95.416174] niks3-server[1024]: 2026/09/20 13:35:49 INFO Received uploads request method=POST path=/api/pending_closures5085worker1 # [ 95.443959] nix-grpc-daemon[1105]: time=2026-09-20T13:35:50.005Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5086worker1 # [ 95.445686] nix-grpc-daemon[1105]: time=2026-09-20T13:35:50.005Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5087lb # [ 95.443991] niks3-server[1024]: 2026/09/20 13:35:50 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5088worker1 # [ 95.474139] nix-grpc-daemon[1105]: time=2026-09-20T13:35:50.035Z level=INFO msg="Uploading 1 narinfos"5089lb # [ 95.460925] niks3-server[1024]: 2026/09/20 13:35:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5090lb # [ 95.471281] niks3-server[1024]: 2026/09/20 13:35:50 INFO Signed narinfos id=31 count=15091lb # [ 95.475218] niks3-server[1024]: 2026/09/20 13:35:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5092lb # [ 95.484526] niks3-server[1024]: 2026/09/20 13:35:50 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5093lb # [ 95.490238] niks3-server[1024]: 2026/09/20 13:35:50 INFO Completed upload id=315094worker1 # [ 95.508805] nix-grpc-daemon[1105]: time=2026-09-20T13:35:50.070Z level=INFO msg="Upload complete. (475ms)"5095lb # [ 95.493690] niks3-server[1024]: 2026/09/20 13:35:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5096worker1 # [ 95.520257] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51074 duration_s=0 paths=1 nar_bytes_in=7685097worker1 # [ 95.530207] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510745098worker1 # [ 95.537151] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5099worker1 # [ 95.542039] nix-daemon[834]: reaped child process 1188, status = succeeded5100client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5101client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5102worker1 # [ 95.554966] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5103worker1 # [ 96.577483] nix-daemon[834]: reaped child process 1202, status = succeeded5104client # farm-top> LOG-farm-top-on-standby5105worker1 # [ 96.738746] nix-daemon[834]: accepted connection from pid 1217, user nix-grpc-daemon (trusted)5106worker1 # [ 96.760350] nix-daemon[834]: reaped child process 1220, status = succeeded5107lb # [ 96.754432] niks3-server[1024]: 2026/09/20 13:35:51 INFO Received uploads request method=POST path=/api/pending_closures5108worker1 # [ 96.784441] nix-grpc-daemon[1105]: time=2026-09-20T13:35:51.345Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5109worker1 # [ 96.785928] nix-grpc-daemon[1105]: time=2026-09-20T13:35:51.345Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5110lb # [ 96.795611] niks3-server[1024]: 2026/09/20 13:35:51 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5111worker1 # [ 96.828741] nix-grpc-daemon[1105]: time=2026-09-20T13:35:51.390Z level=INFO msg="Uploading 1 narinfos"5112lb # [ 96.810084] niks3-server[1024]: 2026/09/20 13:35:51 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5113lb # [ 96.820076] niks3-server[1024]: 2026/09/20 13:35:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5114lb # [ 96.826817] niks3-server[1024]: 2026/09/20 13:35:51 INFO Signed narinfos id=32 count=15115worker1 # [ 96.851776] nix-grpc-daemon[1105]: time=2026-09-20T13:35:51.413Z level=INFO msg="Upload complete. (286ms)"5116lb # [ 96.833502] niks3-server[1024]: 2026/09/20 13:35:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5117lb # [ 96.841808] niks3-server[1024]: 2026/09/20 13:35:51 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5118worker1 # [ 96.860885] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51074 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15119lb # [ 96.846677] niks3-server[1024]: 2026/09/20 13:35:51 INFO Completed upload id=325120lb # [ 96.848873] niks3-server[1024]: 2026/09/20 13:35:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5121worker1 # [ 96.876835] nix-daemon[834]: reaped child process 1200, status = succeeded5122client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 8.41 seconds)5123client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5124client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5125worker2 # [ 97.211478] systemd[1]: Listening on Nix worker-protocol over gRPC.5126worker2 # [ 97.222128] systemd[1]: Starting scheduler-token.service...5127worker2 # [ 97.324826] systemd[1]: scheduler-token.service: Deactivated successfully.5128worker2 # [ 97.328565] systemd[1]: Finished scheduler-token.service.5129worker2 # [ 97.329899] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 105ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5130worker2 # [ 97.339797] systemd[1]: Starting Nix worker-protocol over gRPC...5131worker2 # [ 97.520540] nix-grpc-daemon[1343]: ts=2026-09-20T13:35:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5132worker2 # [ 97.523123] nix-grpc-daemon[1343]: ts=2026-09-20T13:35:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5133worker2 # [ 97.527246] nix-grpc-daemon[1343]: ts=2026-09-20T13:35:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15134worker2 # [ 97.535195] nix-grpc-daemon[1343]: ts=2026-09-20T13:35:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5135worker2 # [ 97.540835] systemd[1]: Started Nix worker-protocol over gRPC.5136lb: must succeed: curl -sf localhost:9901/clusters5137worker2 # [ 97.560754] nix-grpc-daemon[1343]: ts=2026-09-20T13:35:52Z level=info event=scheduler_connected addr=lb:500515138worker1 # [ 97.598307] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:52Z level=info event=oidc_jwks provider=mock keys=15139worker1 # [ 97.598701] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:510605140worker1 # [ 97.608515] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05141lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5142worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5143worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5144lb: must succeed: curl -sf localhost:9901/clusters5145lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5146lb: must succeed: curl -sf localhost:9901/clusters5147lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5148worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5149worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5150(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 15.22 seconds)5151subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5152client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5153client # [ 97.991529] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5154client # Running as unit: rs.service; invocation ID: 416ab95bd55e4a1897a05b24a2d951bc5155client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds)5156client # [ 98.236191] nix[1168]: don't know how to build these paths:5157client # [ 98.237862] nix[1168]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5158worker1 # [ 98.264881] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:510745159worker1 # [ 98.278250] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:52Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05160worker1 # [ 98.285454] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:52Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15161worker1 # [ 98.322704] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510745162worker1 # [ 98.338755] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:510745163client # [ 98.321284] nix[1168]: copying 1 paths to 'lb:50051'...5164client # [ 98.326924] nix[1168]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5165worker1 # [ 98.355841] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:510745166worker1 # [ 98.371078] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5167worker1 # [ 98.570140] nix-daemon[834]: accepted connection from pid 1234, user nix-grpc-daemon (trusted)5168worker1 # [ 98.590907] nix-daemon[834]: reaped child process 1237, status = succeeded5169worker1 # [ 98.763081] nix-daemon[834]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5170worker1 # [ 98.780474] nix-daemon[834]: reaped child process 1242, status = succeeded5171lb # [ 98.777570] niks3-server[1024]: 2026/09/20 13:35:53 INFO Received uploads request method=POST path=/api/pending_closures5172worker1 # [ 98.808831] nix-grpc-daemon[1105]: time=2026-09-20T13:35:53.370Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5173worker1 # [ 98.810319] nix-grpc-daemon[1105]: time=2026-09-20T13:35:53.370Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5174lb # [ 98.811014] niks3-server[1024]: 2026/09/20 13:35:53 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5175lb # [ 98.822852] niks3-server[1024]: 2026/09/20 13:35:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5176worker1 # [ 98.847447] nix-grpc-daemon[1105]: time=2026-09-20T13:35:53.409Z level=INFO msg="Uploading 1 narinfos"5177lb # [ 98.834420] niks3-server[1024]: 2026/09/20 13:35:53 INFO Signed narinfos id=33 count=15178lb # [ 98.839266] niks3-server[1024]: 2026/09/20 13:35:53 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5179lb # [ 98.848409] niks3-server[1024]: 2026/09/20 13:35:53 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5180lb # [ 98.853402] niks3-server[1024]: 2026/09/20 13:35:53 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5181lb # [ 98.857794] niks3-server[1024]: 2026/09/20 13:35:53 INFO Completed upload id=335182worker1 # [ 98.876476] nix-grpc-daemon[1105]: time=2026-09-20T13:35:53.438Z level=INFO msg="Upload complete. (488ms)"5183worker1 # [ 98.888122] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51074 duration_s=0 paths=1 nar_bytes_in=4165184worker1 # [ 98.899761] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510745185worker1 # [ 98.905841] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5186worker1 # [ 98.910659] nix-daemon[834]: reaped child process 1232, status = succeeded5187client # [ 98.890742] nix[1168]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5188worker1 # [ 98.923959] nix-daemon[834]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5189lb: must succeed: curl -sf localhost:9901/clusters5190lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5191lb: must succeed: curl -sf localhost:9901/clusters5192lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5193lb: must succeed: curl -sf localhost:9901/clusters5194lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5195lb: must succeed: curl -sf localhost:9901/clusters5196lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5197worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5198worker2 # [ 100.966458] systemd-journald[274]: Received client request to rotate journal, rotating.5199worker2 # [ 100.984801] systemd-journald[274]: Vacuuming done, freed 0B of archived journals from /var/log/journal/903e61ebddc04955a1d8083099f08d12.5200worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5201worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5202worker1 # [ 101.068201] systemd[1]: Stopping Nix worker-protocol over gRPC...5203worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5204??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5205 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735206client: must succeed: sleep 5.05207worker1 # [ 101.628832] nix-grpc-daemon[1032]: ts=2026-09-20T13:35:56Z level=info event=draining5208lb # [ 102.121171] envoy[921]: conn peer=[2001:db8:1::1]:57718 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9653 tx=3879 ms=81195209client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5210worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5211worker1 # [ 106.578060] nix-daemon[834]: reaped child process 1246, status = succeeded5212worker1 # [ 106.760188] nix-daemon[834]: accepted connection from pid 1286, user nix-grpc-daemon (trusted)5213worker1 # [ 106.779800] nix-daemon[834]: reaped child process 1289, status = succeeded5214lb # [ 106.774486] niks3-server[1024]: 2026/09/20 13:36:01 INFO Received uploads request method=POST path=/api/pending_closures5215worker1 # [ 106.806972] nix-grpc-daemon[1105]: time=2026-09-20T13:36:01.368Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5216worker1 # [ 106.811804] nix-grpc-daemon[1105]: time=2026-09-20T13:36:01.370Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5217lb # [ 106.823216] niks3-server[1024]: 2026/09/20 13:36:01 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5218lb # [ 106.834604] niks3-server[1024]: 2026/09/20 13:36:01 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5219lb # [ 106.843554] niks3-server[1024]: 2026/09/20 13:36:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5220worker1 # [ 106.866986] nix-grpc-daemon[1105]: time=2026-09-20T13:36:01.428Z level=INFO msg="Uploading 1 narinfos"5221lb # [ 106.851386] niks3-server[1024]: 2026/09/20 13:36:01 INFO Signed narinfos id=34 count=15222lb # [ 106.856210] niks3-server[1024]: 2026/09/20 13:36:01 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5223lb # [ 106.864545] niks3-server[1024]: 2026/09/20 13:36:01 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5224lb # [ 106.871022] niks3-server[1024]: 2026/09/20 13:36:01 INFO Completed upload id=345225worker1 # [ 106.889667] nix-grpc-daemon[1105]: time=2026-09-20T13:36:01.451Z level=INFO msg="Upload complete. (349ms)"5226lb # [ 106.874299] niks3-server[1024]: 2026/09/20 13:36:01 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5227worker1 # [ 106.905219] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51074 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15228worker1 # [ 106.921041] nix-daemon[834]: reaped child process 1244, status = succeeded5229client # [ 106.914367] systemd[1]: rs.service: Deactivated successfully.5230client # [ 106.919579] systemd[1]: rs.service: Consumed 158ms CPU time over 8.926s wall clock time, 37.2M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5231worker1 # [ 107.632976] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=signal_exit5232worker1 # [ 107.847598] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=scheduler_yield5233lb # [ 107.833353] niks3-server[1024]: 2026/09/20 13:36:02 INFO lead: released remote=192.168.1.3:405965234worker1 # [ 107.853725] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=scheduler_restarting addr=lb:500515235worker2 # [ 107.829367] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:02Z level=info event=scheduler_restarting addr=lb:500515236worker1 # [ 107.860129] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:510605237worker1 # [ 107.864592] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=worker_gone addr=192.168.1.3:500515238worker1 # [ 107.868534] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:510605239worker1 # [ 107.872854] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=worker_gone addr=192.168.1.4:500515240worker2 # [ 108.029712] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:02Z level=info event=scheduler_connected addr=lb:500515241worker1 # [ 108.056265] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=scheduler_connected addr=lb:500515242worker2 # [ 108.035107] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5243worker1 # [ 108.061115] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5244worker2 # [ 108.235978] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:02Z level=info event=scheduler_connected addr=lb:500515245worker1 # [ 108.262004] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=scheduler_connected addr=lb:500515246worker2 # [ 108.241142] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5247worker1 # [ 108.267552] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5248worker2 # [ 108.442241] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_connected addr=lb:500515249worker1 # [ 108.468490] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:03Z level=info event=scheduler_connected addr=lb:500515250worker2 # [ 108.447427] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5251worker1 # [ 108.473965] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5252worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.20 seconds)5253client: waiting for success: ! systemctl is-active rs5254worker2 # [ 108.648640] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_connected addr=lb:500515255worker1 # [ 108.674724] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:03Z level=info event=scheduler_connected addr=lb:500515256worker2 # [ 108.653825] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5257worker1 # [ 108.680187] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5258client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5259client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5260client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5261client: must fail: journalctl -u rs | grep -q 'reconnecting'5262client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5263lb: must succeed: curl -sf localhost:9901/clusters5264worker1 # [ 108.853269] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:03Z level=info event=scheduler_restarting5265lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5266worker2 # [ 108.856743] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_connected addr=lb:500515267worker2 # [ 108.860845] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5268worker1 # [ 108.906703] nix-grpc-daemon[1032]: ts=2026-09-20T13:36:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5269worker1 # [ 108.912604] nix-daemon[834]: reaped child process 1096, status = succeeded5270worker2 # [ 109.057423] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_connected addr=lb:500515271worker2 # [ 109.062585] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5272worker2 # [ 109.263563] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_connected addr=lb:500515273worker2 # [ 109.268784] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5274worker2 # [ 109.469882] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_connected addr=lb:500515275worker2 # [ 109.475079] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5276worker2 # [ 109.676209] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_connected addr=lb:500515277worker2 # [ 109.681206] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5278worker1 # [ 109.745222] nix-grpc-daemon[1032]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5279worker1 # [ 109.752460] nix-grpc-daemon[1032]: E0000 00:00:1789911364.305907 1075 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5280worker1 # [ 109.789315] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5281worker1 # [ 109.816582] systemd[1]: Stopped Nix worker-protocol over gRPC.5282worker1 # [ 109.820458] systemd[1]: nix-grpc-daemon.service: Consumed 1.855s CPU time over 27.385s wall clock time, 34.6M memory peak, 50.4K incoming IP traffic, 92.3K outgoing IP traffic.5283worker1 # [ 109.835339] systemd[1]: Starting Nix worker-protocol over gRPC...5284lb: must succeed: curl -sf localhost:9901/clusters5285worker2 # [ 109.883587] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_connected addr=lb:500515286worker2 # [ 109.888741] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5287lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5288worker1 # [ 110.022338] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5289worker1 # [ 110.024928] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5290worker1 # [ 110.028977] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15291worker1 # [ 110.036799] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5292worker1 # [ 110.038555] systemd[1]: Started Nix worker-protocol over gRPC.5293worker1 # [ 110.050285] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:04Z level=info event=scheduler_take_over5294lb # [ 110.033216] niks3-server[1024]: 2026/09/20 13:36:04 INFO lead: acquired remote=192.168.1.3:490325295worker1 # [ 110.065126] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:04Z level=info event=scheduler_connected addr=lb:500515296worker1 # [ 110.066982] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5297worker2 # [ 110.089854] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_connected addr=lb:500515298worker2 # [ 110.094871] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5299worker1 # [ 110.268121] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:04Z level=info event=scheduler_connected addr=lb:500515300worker1 # [ 110.273972] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5301worker2 # [ 110.295968] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_connected addr=lb:500515302worker2 # [ 110.301079] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5303worker1 # [ 110.474663] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=scheduler_connected addr=lb:500515304worker1 # [ 110.479908] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5305worker2 # [ 110.501959] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:05Z level=info event=scheduler_connected addr=lb:500515306worker2 # [ 110.506982] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5307worker1 # [ 110.680763] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=scheduler_connected addr=lb:500515308worker1 # [ 110.685803] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5309worker2 # [ 110.708125] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:05Z level=info event=scheduler_connected addr=lb:500515310worker2 # [ 110.713282] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5311worker1 # [ 110.886579] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=scheduler_connected addr=lb:500515312worker1 # [ 110.895473] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:341185313worker1 # [ 110.900945] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05314worker2 # [ 110.915569] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:05Z level=info event=scheduler_connected addr=lb:500515315worker1 # [ 110.949750] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=oidc_jwks provider=mock keys=15316lb: must succeed: curl -sf localhost:9901/clusters5317worker1 # [ 110.954207] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:341185318worker1 # [ 110.959485] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05319lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5320worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5321worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5322(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.17 seconds)5323subtest: scheduler restart mid-build: build finishes, no double build5324client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5325client # [ 111.157052] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5326client # Running as unit: mid.service; invocation ID: dac4c1ee468044fa960ddf5617ed06985327client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5328worker2 # [ 111.412932] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585925329worker2 # [ 111.429462] nix-daemon[845]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5330client # [ 111.442512] nix[1204]: don't know how to build these paths:5331client # [ 111.443869] nix[1204]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5332worker1 # [ 111.486795] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:06Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05333worker1 # [ 111.494349] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:06Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15334worker1 # [ 111.554120] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:341245335worker1 # [ 111.568602] nix-daemon[834]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5336worker1 # [ 111.592818] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 1/5)5337worker1 # [ 111.667491] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 116 ms (attempt 2/5)5338worker1 # [ 111.783871] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 3/5)5339worker1 # [ 111.864536] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 577 ms (attempt 4/5)5340lb # [ 112.125083] envoy[921]: conn peer=[2001:db8:1::1]:36602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1349 ms=86505341lb # [ 112.130115] envoy[921]: conn peer=[2001:db8:1::3]:48854 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4970 tx=1460 ms=262545342worker1 # [ 112.450021] nix-grpc-daemon[1296]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5343worker1 # [ 112.467140] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:341245344client # [ 112.449202] nix[1204]: copying 1 paths to 'lb:50051'...5345worker1 # [ 112.481211] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:341245346client # [ 112.454491] nix[1204]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5347worker1 # [ 112.496999] nix-daemon[834]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5348worker1 # [ 112.522087] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:07Z level=info event=niks3_push_started pid=13655349worker1 # [ 112.729881] nix-grpc-daemon[1365]: time=2026-09-20T13:36:07.291Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5350worker1 # [ 112.906895] nix-daemon[834]: accepted connection from pid 1378, user nix-grpc-daemon (trusted)5351worker1 # [ 112.924596] nix-daemon[834]: reaped child process 1381, status = succeeded5352worker1 # [ 113.100560] nix-daemon[834]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)5353worker1 # [ 113.119096] nix-daemon[834]: reaped child process 1386, status = succeeded5354lb # [ 113.113330] niks3-server[1024]: 2026/09/20 13:36:07 INFO Received uploads request method=POST path=/api/pending_closures5355worker1 # [ 113.142427] nix-grpc-daemon[1365]: time=2026-09-20T13:36:07.703Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5356worker1 # [ 113.144207] nix-grpc-daemon[1365]: time=2026-09-20T13:36:07.703Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5357lb # [ 113.148467] niks3-server[1024]: 2026/09/20 13:36:07 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5358worker1 # [ 113.176804] nix-grpc-daemon[1365]: time=2026-09-20T13:36:07.738Z level=INFO msg="Uploading 1 narinfos"5359lb # [ 113.158042] niks3-server[1024]: 2026/09/20 13:36:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5360lb # [ 113.168614] niks3-server[1024]: 2026/09/20 13:36:07 INFO Signed narinfos id=35 count=15361lb # [ 113.182090] niks3-server[1024]: 2026/09/20 13:36:07 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5362lb # [ 113.193989] niks3-server[1024]: 2026/09/20 13:36:07 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5363lb # [ 113.226370] niks3-server[1024]: 2026/09/20 13:36:07 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5364lb # [ 113.233380] niks3-server[1024]: 2026/09/20 13:36:07 INFO Completed upload id=355365worker1 # [ 113.252325] nix-grpc-daemon[1365]: time=2026-09-20T13:36:07.814Z level=INFO msg="Upload complete. (518ms)"5366worker1 # [ 113.264336] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34124 duration_s=0 paths=1 nar_bytes_in=4245367worker1 # [ 113.279056] nix-grpc-daemon[1296]: ts=2026-09-20T13:36:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:341245368worker1 # [ 113.289260] nix-daemon[834]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5369worker1 # [ 113.296432] nix-daemon[834]: reaped child process 1363, status = succeeded5370client # [ 113.276324] nix[1204]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5371worker1 # [ 113.309757] nix-daemon[834]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5372lb: must succeed: curl -sf localhost:9901/clusters5373lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5374lb: must succeed: curl -sf localhost:9901/clusters5375lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5376worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5377worker1 # [ 115.213294] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1296 (nix-grpc-daemon) on client request.5378worker1 # [ 115.215022] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5379lb # [ 115.209859] niks3-server[1024]: 2026/09/20 13:36:09 INFO lead: released remote=192.168.1.3:490325380worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)5381client: must succeed: sleep 5.05382client # [ 115.207305] nix[1204]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5383worker2 # [ 115.210578] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:09Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5384client # [ 115.213781] nix[1204]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5385worker1 # [ 115.251697] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5386worker1 # [ 115.261198] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5387worker1 # [ 115.262504] systemd[1]: nix-grpc-daemon.service: Consumed 502ms CPU time over 5.434s wall clock time, 31.5M memory peak, 14.5K incoming IP traffic, 17.2K outgoing IP traffic.5388worker1 # [ 115.270366] nix-daemon[834]: reaped child process 1388, status = succeeded5389worker1 # [ 115.271428] nix-daemon[834]: reaped child process 1354, status = succeeded5390worker1 # [ 115.280524] nix-daemon[1390]: unexpected Nix daemon error: error: interrupted by the user5391worker1 # [ 115.285301] nix-daemon[834]: reaped child process 1390, status = failed with exit code 15392worker2 # [ 115.410900] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:09Z level=info event=scheduler_connected addr=lb:500515393worker1 # [ 115.440681] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5394worker1 # [ 115.453617] systemd[1]: Starting Nix worker-protocol over gRPC...5395worker1 # [ 115.657573] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5396worker1 # [ 115.661082] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5397worker1 # [ 115.663266] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15398worker1 # [ 115.670985] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5399worker1 # [ 115.677373] systemd[1]: Started Nix worker-protocol over gRPC.5400worker1 # [ 115.684315] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=scheduler_take_over5401lb # [ 115.665907] niks3-server[1024]: 2026/09/20 13:36:10 INFO lead: acquired remote=192.168.1.3:476005402worker1 # [ 115.697101] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=oidc_jwks provider=mock keys=15403worker1 # [ 115.698622] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:341345404worker1 # [ 115.702012] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05405worker1 # [ 115.705159] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=scheduler_connected addr=lb:500515406worker1 # [ 115.707125] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:341345407worker1 # [ 115.709025] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05408worker1 # [ 115.746149] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05409worker1 # [ 115.749749] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:10Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15410worker2 # [ 115.765582] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585925411worker2 # [ 115.782442] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 1/5)5412worker2 # [ 115.860277] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 2/5)5413worker2 # [ 115.886571] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 3/5)5414worker2 # [ 115.928069] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 645 ms (attempt 4/5)5415worker2 # [ 116.582164] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5416worker2 # [ 116.601416] nix-grpc-daemon[1343]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5417worker2 # [ 116.625644] nix-daemon[845]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5418client # [ 116.642883] nix[1204]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5419worker2 # [ 116.650836] nix-daemon[845]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5420client: (finished: must succeed: sleep 5.0, in 5.07 seconds)5421client: waiting for success: ! systemctl is-active mid5422worker2 # [ 120.671061] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:15Z level=info event=niks3_push_started pid=14945423worker2 # [ 120.692871] nix-daemon[845]: reaped child process 1470, status = succeeded5424worker2 # [ 120.877278] nix-grpc-daemon[1494]: time=2026-09-20T13:36:15.439Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5425worker2 # [ 121.047819] nix-daemon[845]: accepted connection from pid 1507, user nix-grpc-daemon (trusted)5426worker2 # [ 121.064368] nix-daemon[845]: reaped child process 1510, status = succeeded5427lb # [ 121.087653] niks3-server[1024]: 2026/09/20 13:36:15 INFO Received uploads request method=POST path=/api/pending_closures5428worker2 # [ 121.095034] nix-grpc-daemon[1494]: time=2026-09-20T13:36:15.656Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5429worker2 # [ 121.097831] nix-grpc-daemon[1494]: time=2026-09-20T13:36:15.656Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5430lb # [ 121.140187] niks3-server[1024]: 2026/09/20 13:36:15 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5431lb # [ 121.151623] niks3-server[1024]: 2026/09/20 13:36:15 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5432worker2 # [ 121.152085] nix-grpc-daemon[1494]: time=2026-09-20T13:36:15.713Z level=INFO msg="Uploading 1 narinfos"5433lb # [ 121.160427] niks3-server[1024]: 2026/09/20 13:36:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5434lb # [ 121.167319] niks3-server[1024]: 2026/09/20 13:36:15 INFO Signed narinfos id=36 count=15435lb # [ 121.172762] niks3-server[1024]: 2026/09/20 13:36:15 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5436lb # [ 121.179704] niks3-server[1024]: 2026/09/20 13:36:15 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5437lb # [ 121.187974] niks3-server[1024]: 2026/09/20 13:36:15 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5438lb # [ 121.194393] niks3-server[1024]: 2026/09/20 13:36:15 INFO Completed upload id=365439worker2 # [ 121.188501] nix-grpc-daemon[1494]: time=2026-09-20T13:36:15.751Z level=INFO msg="Upload complete. (307ms)"5440worker2 # [ 121.199433] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58592 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15441worker2 # [ 121.217301] nix-daemon[845]: reaped child process 1468, status = succeeded5442client # [ 121.234450] systemd[1]: mid.service: Deactivated successfully.5443client # [ 121.238681] systemd[1]: mid.service: Consumed 167ms CPU time over 10.081s wall clock time, 39.2M memory peak, 9.8K incoming IP traffic, 10.6K outgoing IP traffic.5444client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5445client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5446client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5447(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.69 seconds)5448subtest: interrupting the client stops the build on the worker5449client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5450client # [ 121.819847] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5451client # Running as unit: intr.service; invocation ID: b148168916204cc38e5b6e5cbb4087175452client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5453worker1 # [ 122.078058] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:460625454worker1 # [ 122.091085] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5455client # [ 122.080051] nix[1239]: don't know how to build these paths:5456client # [ 122.082073] nix[1239]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5457worker1 # [ 122.120286] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:16Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05458worker1 # [ 122.126855] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:16Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15459lb # [ 122.123079] envoy[921]: conn peer=[2001:db8:1::3]:34320 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1015 tx=411 ms=51785460lb # [ 122.128187] envoy[921]: conn peer=[2001:db8:1::1]:48302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3841 tx=1683 ms=98125461worker1 # [ 122.175273] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625462worker1 # [ 122.191930] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 1/5)5463worker1 # [ 122.264510] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)5464worker1 # [ 122.415244] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 3/5)5465worker1 # [ 122.482455] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 575 ms (attempt 4/5)5466worker1 # [ 123.066542] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5467worker1 # [ 123.091108] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460625468client # [ 123.072883] nix[1239]: copying 1 paths to 'lb:50051'...5469client # [ 123.077535] nix[1239]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5470worker1 # [ 123.107767] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460625471worker1 # [ 123.124953] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5472worker1 # [ 123.154743] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:17Z level=info event=niks3_push_started pid=15025473worker1 # [ 123.357266] nix-grpc-daemon[1502]: time=2026-09-20T13:36:17.918Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5474worker1 # [ 123.534425] nix-daemon[834]: accepted connection from pid 1515, user nix-grpc-daemon (trusted)5475worker1 # [ 123.552376] nix-daemon[834]: reaped child process 1518, status = succeeded5476worker1 # [ 123.729680] nix-daemon[834]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)5477worker1 # [ 123.747020] nix-daemon[834]: reaped child process 1523, status = succeeded5478lb # [ 123.741021] niks3-server[1024]: 2026/09/20 13:36:18 INFO Received uploads request method=POST path=/api/pending_closures5479worker1 # [ 123.774382] nix-grpc-daemon[1502]: time=2026-09-20T13:36:18.335Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5480worker1 # [ 123.775913] nix-grpc-daemon[1502]: time=2026-09-20T13:36:18.335Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5481lb # [ 123.782498] niks3-server[1024]: 2026/09/20 13:36:18 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5482worker1 # [ 123.811447] nix-grpc-daemon[1502]: time=2026-09-20T13:36:18.373Z level=INFO msg="Uploading 1 narinfos"5483lb # [ 123.793401] niks3-server[1024]: 2026/09/20 13:36:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5484lb # [ 123.803362] niks3-server[1024]: 2026/09/20 13:36:18 INFO Signed narinfos id=37 count=15485lb # [ 123.812358] niks3-server[1024]: 2026/09/20 13:36:18 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5486lb # [ 123.824729] niks3-server[1024]: 2026/09/20 13:36:18 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5487lb # [ 123.835669] niks3-server[1024]: 2026/09/20 13:36:18 INFO Completed upload id=375488worker1 # [ 123.854479] nix-grpc-daemon[1502]: time=2026-09-20T13:36:18.416Z level=INFO msg="Upload complete. (493ms)"5489lb # [ 123.839346] niks3-server[1024]: 2026/09/20 13:36:18 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5490worker1 # [ 123.863939] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=4245491worker1 # [ 123.880657] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625492worker1 # [ 123.894456] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5493worker1 # [ 123.898336] nix-daemon[834]: reaped child process 1500, status = succeeded5494client # [ 123.883006] nix[1239]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5495worker1 # [ 123.916210] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5496client: must succeed: systemctl kill -s INT intr5497client # [ 125.723278] systemd[1]: intr.service: Sent signal SIGINT to main process 1239 (nix) on client request.5498client: (finished: must succeed: systemctl kill -s INT intr, in 0.10 seconds)5499client # [ 125.738211] nix[1239]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5500client # [ 125.743115] nix[1239]: error: interrupted by the user5501client # [ 125.794138] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5502client # [ 125.801374] systemd[1]: intr.service: Failed with result 'exit-code'.5503client # [ 125.803785] systemd[1]: intr.service: Consumed 153ms CPU time over 3.977s wall clock time, 41.2M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.5504worker1 # [ 125.949790] nix-daemon[1527]: unexpected Nix daemon error: error: interrupted by the user5505worker1 # [ 125.955773] nix-daemon[834]: reaped child process 1525, status = succeeded5506worker1 # [ 125.960402] nix-daemon[834]: reaped child process 1527, status = failed with exit code 15507(finished: subtest: interrupting the client stops the build on the worker, in 5.38 seconds)5508subtest: a deploy mid-build drains: the build finishes, the new generation takes over5509client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5510client # [ 127.250801] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5511client # Running as unit: sw.service; invocation ID: 5dccb3eff2c343739b3737e529a05fce5512client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.12 seconds)5513client # [ 127.545578] nix[1260]: don't know how to build these paths:5514client # [ 127.546596] nix[1260]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5515worker2 # [ 127.547529] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585925516worker1 # [ 127.600986] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:22Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05517worker1 # [ 127.601574] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:22Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15518worker2 # [ 127.625783] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585925519worker2 # [ 127.656396] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585925520client # [ 127.671458] nix[1260]: copying 1 paths to 'lb:50051'...5521client # [ 127.678521] nix[1260]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5522worker2 # [ 127.682650] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585925523worker2 # [ 127.706076] nix-daemon[845]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5524worker2 # [ 127.997216] nix-daemon[845]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5525worker2 # [ 128.021706] nix-daemon[845]: reaped child process 1545, status = succeeded5526worker2 # [ 128.246601] nix-daemon[845]: accepted connection from pid 1547, user nix-grpc-daemon (trusted)5527worker2 # [ 128.268280] nix-daemon[845]: reaped child process 1550, status = succeeded5528lb # [ 128.289967] niks3-server[1024]: 2026/09/20 13:36:22 INFO Received uploads request method=POST path=/api/pending_closures5529worker2 # [ 128.299023] nix-grpc-daemon[1494]: time=2026-09-20T13:36:22.860Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5530worker2 # [ 128.300677] nix-grpc-daemon[1494]: time=2026-09-20T13:36:22.860Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5531lb # [ 128.333964] niks3-server[1024]: 2026/09/20 13:36:22 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5532worker2 # [ 128.337603] nix-grpc-daemon[1494]: time=2026-09-20T13:36:22.900Z level=INFO msg="Uploading 1 narinfos"5533lb # [ 128.344280] niks3-server[1024]: 2026/09/20 13:36:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5534lb # [ 128.347403] niks3-server[1024]: 2026/09/20 13:36:22 INFO Signed narinfos id=38 count=15535lb # [ 128.360460] niks3-server[1024]: 2026/09/20 13:36:22 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5536worker2 # [ 128.359968] nix-grpc-daemon[1494]: time=2026-09-20T13:36:22.922Z level=INFO msg="Upload complete. (627ms)"5537lb # [ 128.371551] niks3-server[1024]: 2026/09/20 13:36:22 INFO Completed upload id=385538lb # [ 128.378106] niks3-server[1024]: 2026/09/20 13:36:22 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5539worker2 # [ 128.370325] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58592 duration_s=0 paths=1 nar_bytes_in=4165540worker2 # [ 128.380857] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585925541worker2 # [ 128.389224] nix-grpc-daemon[1494]: time=2026-09-20T13:36:22.951Z level=WARN msg="Failed to register uploaded object" key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo error="server returned 404: object is not pending upload\n"5542worker2 # [ 128.403400] nix-daemon[845]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5543worker2 # [ 128.406307] nix-daemon[845]: reaped child process 1540, status = succeeded5544client # [ 128.414068] nix[1260]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5545worker2 # [ 128.421406] nix-daemon[845]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5546worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25547worker2 # Checking switch inhibitors... done5548worker2 # [ 130.983841] nixos[1585]: switching to system configuration /nix/store/nbfzs1r0axc1p0hjvzx3birw6xff13xs-nixos-system-worker2-test5549worker2 # activating the configuration...5550worker2 # [ 130.990256] systemd[1]: Stopped target Remote File Systems.5551worker2 # [ 130.991080] systemd[1]: Stopped target Local File Systems.5552worker2 # setting up /etc...5553worker2 # [ 131.772658] systemd[1]: Reload requested from client PID 1585 ('.switch-to-conf') (unit backdoor.service)...5554worker2 # [ 131.775012] systemd[1]: Reloading...5555lb # [ 132.123251] envoy[921]: conn peer=[2001:db8:1::1]:48424 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2944 tx=754 ms=37245556worker2 # [ 132.656322] systemd[1]: Reloading finished in 878 ms.5557worker2 # restarting sysinit-reactivation.target5558worker2 # [ 132.729002] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5559worker2 # [ 132.883078] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5560worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5561worker2 # [ 132.889361] systemd[1]: Reached target Reactivate sysinit units.5562worker2 # [ 132.893297] systemd[1]: Reloading D-Bus System Message Bus...5563worker2 # [ 132.901386] dbus-broker-launch[498]: Caught SIGHUP, trigger reload.5564worker2 # [ 132.904157] systemd[1]: Reloading Nix worker-protocol over gRPC...5565worker2 # [ 132.936652] dbus-broker-launch[498]: Looking up NSS user entry for 'systemd-timesync'...5566worker2 # [ 132.938681] dbus-broker-launch[498]: NSS returned no entry for 'systemd-timesync'5567worker2 # [ 132.939764] dbus-broker-launch[498]: Invalid user-name in /nix/store/c8a4kacp5lnfwrcxrjmwczbl8ba32m4a-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5568worker2 # [ 132.952822] systemd[1]: Reloaded Nix worker-protocol over gRPC.5569worker2 # [ 132.980699] systemd[1]: Reloaded D-Bus System Message Bus.5570worker2 # [ 132.988115] systemd[1]: Reached target Local File Systems.5571worker2 # [ 132.988926] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5572worker2 # [ 132.994426] systemd[1]: Starting Create SUID/SGID Wrappers...5573worker2 # [ 133.001527] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5574worker2 # [ 133.005719] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675575worker2 # [ 133.017264] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5576worker2 # [ 133.018807] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5577worker2 # [ 133.019995] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5578worker2 # [ 133.025586] systemd[1]: Reached target Remote File Systems.5579worker2 # [ 133.027535] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5580worker2 # [ 133.032142] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5581worker2 # [ 133.497144] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5582worker2 # [ 133.504937] systemd[1]: Finished Create SUID/SGID Wrappers.5583worker2 # [ 133.507418] systemd[1]: Starting linger-users.service...5584worker2 # [ 133.557980] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:28Z level=info event=draining5585worker2 # [ 133.565254] systemd[1]: linger-users.service: Deactivated successfully.5586worker2 # [ 133.566240] systemd[1]: Finished linger-users.service.5587worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5588worker2 # [ 133.824508] nixos[1585]: finished switching to system configuration /nix/store/nbfzs1r0axc1p0hjvzx3birw6xff13xs-nixos-system-worker2-test5589worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.72 seconds)5590worker2: must succeed: pgrep -f 'read -t [9]0 x'5591worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5592client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25593worker1 # [ 134.179815] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:460625594client # don't know how to build these paths:5595client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5596worker2 # [ 134.165483] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585925597worker1 # [ 134.221322] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:28Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05598worker1 # [ 134.228540] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:28Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15599worker1 # [ 134.271354] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625600worker1 # [ 134.287605] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460625601client # copying 1 paths to 'lb:50051'...5602worker1 # [ 134.304134] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460625603client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5604worker1 # [ 134.331176] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5605worker1 # [ 134.546383] nix-daemon[834]: accepted connection from pid 1581, user nix-grpc-daemon (trusted)5606worker1 # [ 134.566461] nix-daemon[834]: reaped child process 1584, status = succeeded5607worker1 # [ 134.741522] nix-daemon[834]: accepted connection from pid 1586, user nix-grpc-daemon (trusted)5608worker1 # [ 134.758516] nix-daemon[834]: reaped child process 1589, status = succeeded5609lb # [ 134.751957] niks3-server[1024]: 2026/09/20 13:36:29 INFO Received uploads request method=POST path=/api/pending_closures5610worker1 # [ 134.789369] nix-grpc-daemon[1502]: time=2026-09-20T13:36:29.350Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5611worker1 # [ 134.790992] nix-grpc-daemon[1502]: time=2026-09-20T13:36:29.350Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5612lb # [ 134.799644] niks3-server[1024]: 2026/09/20 13:36:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5613worker1 # [ 134.826041] nix-grpc-daemon[1502]: time=2026-09-20T13:36:29.387Z level=INFO msg="Uploading 1 narinfos"5614lb # [ 134.817176] niks3-server[1024]: 2026/09/20 13:36:29 INFO Signed narinfos id=39 count=15615lb # [ 134.825295] niks3-server[1024]: 2026/09/20 13:36:29 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5616lb # [ 134.840545] niks3-server[1024]: 2026/09/20 13:36:29 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5617worker1 # [ 134.862794] nix-grpc-daemon[1502]: time=2026-09-20T13:36:29.424Z level=INFO msg="Upload complete. (511ms)"5618lb # [ 134.847929] niks3-server[1024]: 2026/09/20 13:36:29 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5619lb # [ 134.854507] niks3-server[1024]: 2026/09/20 13:36:29 INFO Completed upload id=395620worker1 # [ 134.872671] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=5285621lb # [ 134.857401] niks3-server[1024]: 2026/09/20 13:36:29 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5622worker1 # [ 134.888537] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625623worker1 # [ 134.895637] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5624worker1 # [ 134.901610] nix-daemon[834]: reaped child process 1579, status = succeeded5625client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5626client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5627worker1 # [ 134.917117] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5628client # farm-a> LOG-farm-a-during-drain5629worker1 # [ 135.563047] nix-daemon[834]: reaped child process 1593, status = succeeded5630worker1 # [ 135.729497] nix-daemon[834]: accepted connection from pid 1608, user nix-grpc-daemon (trusted)5631worker1 # [ 135.747877] nix-daemon[834]: reaped child process 1611, status = succeeded5632lb # [ 135.743811] niks3-server[1024]: 2026/09/20 13:36:30 INFO Received uploads request method=POST path=/api/pending_closures5633worker1 # [ 135.772085] nix-grpc-daemon[1502]: time=2026-09-20T13:36:30.333Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5634worker1 # [ 135.773866] nix-grpc-daemon[1502]: time=2026-09-20T13:36:30.333Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5635lb # [ 135.782645] niks3-server[1024]: 2026/09/20 13:36:30 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5636lb # [ 135.799627] niks3-server[1024]: 2026/09/20 13:36:30 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5637worker1 # [ 135.831972] nix-grpc-daemon[1502]: time=2026-09-20T13:36:30.391Z level=INFO msg="Uploading 1 narinfos"5638lb # [ 135.818907] niks3-server[1024]: 2026/09/20 13:36:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5639lb # [ 135.829505] niks3-server[1024]: 2026/09/20 13:36:30 INFO Signed narinfos id=40 count=15640lb # [ 135.838320] niks3-server[1024]: 2026/09/20 13:36:30 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5641lb # [ 135.841853] niks3-server[1024]: 2026/09/20 13:36:30 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5642lb # [ 135.846691] niks3-server[1024]: 2026/09/20 13:36:30 INFO Completed upload id=405643worker1 # [ 135.864683] nix-grpc-daemon[1502]: time=2026-09-20T13:36:30.426Z level=INFO msg="Upload complete. (337ms)"5644lb # [ 135.848871] niks3-server[1024]: 2026/09/20 13:36:30 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5645worker1 # [ 135.881672] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15646worker1 # [ 135.900713] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:30Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05647worker1 # [ 135.905584] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:30Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15648worker1 # [ 135.910507] nix-daemon[834]: reaped child process 1591, status = succeeded5649worker1 # [ 135.933036] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625650worker1 # [ 135.948682] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460625651worker1 # [ 135.963955] nix-grpc-daemon[1502]: time=2026-09-20T13:36:30.525Z level=INFO msg="All 1 paths already cached"5652client # copying 1 paths to 'lb:50051'...5653worker1 # [ 135.974189] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460625654client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5655worker1 # [ 135.997337] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5656worker1 # [ 136.194683] nix-daemon[834]: accepted connection from pid 1617, user nix-grpc-daemon (trusted)5657worker1 # [ 136.213878] nix-daemon[834]: reaped child process 1620, status = succeeded5658worker1 # [ 136.382487] nix-daemon[834]: accepted connection from pid 1622, user nix-grpc-daemon (trusted)5659worker1 # [ 136.401446] nix-daemon[834]: reaped child process 1625, status = succeeded5660lb # [ 136.395799] niks3-server[1024]: 2026/09/20 13:36:30 INFO Received uploads request method=POST path=/api/pending_closures5661worker1 # [ 136.423875] nix-grpc-daemon[1502]: time=2026-09-20T13:36:30.985Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5662worker1 # [ 136.425555] nix-grpc-daemon[1502]: time=2026-09-20T13:36:30.985Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5663lb # [ 136.423521] niks3-server[1024]: 2026/09/20 13:36:30 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5664worker1 # [ 136.454250] nix-grpc-daemon[1502]: time=2026-09-20T13:36:31.016Z level=INFO msg="Uploading 1 narinfos"5665lb # [ 136.439008] niks3-server[1024]: 2026/09/20 13:36:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5666lb # [ 136.449446] niks3-server[1024]: 2026/09/20 13:36:30 INFO Signed narinfos id=41 count=15667lb # [ 136.454599] niks3-server[1024]: 2026/09/20 13:36:31 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5668worker1 # [ 136.482539] nix-grpc-daemon[1502]: time=2026-09-20T13:36:31.044Z level=INFO msg="Upload complete. (468ms)"5669lb # [ 136.464939] niks3-server[1024]: 2026/09/20 13:36:31 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5670lb # [ 136.469935] niks3-server[1024]: 2026/09/20 13:36:31 INFO Completed upload id=415671lb # [ 136.473117] niks3-server[1024]: 2026/09/20 13:36:31 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5672worker1 # [ 136.491992] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=6405673worker1 # [ 136.495919] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625674worker1 # [ 136.507736] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5675worker1 # [ 136.511858] nix-daemon[834]: reaped child process 1615, status = succeeded5676client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5677worker1 # [ 136.521975] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5678client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5679client # farm-b> LOG-farm-b-during-drain5680worker1 # [ 137.973378] nix-daemon[834]: reaped child process 1629, status = succeeded5681worker1 # [ 138.149871] nix-daemon[834]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)5682worker1 # [ 138.167081] nix-daemon[834]: reaped child process 1647, status = succeeded5683lb # [ 138.162223] niks3-server[1024]: 2026/09/20 13:36:32 INFO Received uploads request method=POST path=/api/pending_closures5684worker1 # [ 138.191437] nix-grpc-daemon[1502]: time=2026-09-20T13:36:32.752Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5685worker1 # [ 138.193195] nix-grpc-daemon[1502]: time=2026-09-20T13:36:32.752Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5686lb # [ 138.192143] niks3-server[1024]: 2026/09/20 13:36:32 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5687lb # [ 138.206233] niks3-server[1024]: 2026/09/20 13:36:32 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5688worker1 # [ 138.237217] nix-grpc-daemon[1502]: time=2026-09-20T13:36:32.798Z level=INFO msg="Uploading 1 narinfos"5689lb # [ 138.219721] niks3-server[1024]: 2026/09/20 13:36:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5690lb # [ 138.228076] niks3-server[1024]: 2026/09/20 13:36:32 INFO Signed narinfos id=42 count=15691lb # [ 138.233757] niks3-server[1024]: 2026/09/20 13:36:32 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5692lb # [ 138.240575] niks3-server[1024]: 2026/09/20 13:36:32 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5693lb # [ 138.244285] niks3-server[1024]: 2026/09/20 13:36:32 INFO Completed upload id=425694worker1 # [ 138.262633] nix-grpc-daemon[1502]: time=2026-09-20T13:36:32.824Z level=INFO msg="Upload complete. (328ms)"5695lb # [ 138.246815] niks3-server[1024]: 2026/09/20 13:36:32 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5696worker1 # [ 138.279616] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15697worker1 # [ 138.295969] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:32Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05698worker1 # [ 138.299519] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:32Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=15699worker1 # [ 138.303733] nix-daemon[834]: reaped child process 1627, status = succeeded5700worker1 # [ 138.323011] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625701worker1 # [ 138.344222] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460625702worker1 # [ 138.358677] nix-grpc-daemon[1502]: time=2026-09-20T13:36:32.920Z level=INFO msg="All 2 paths already cached"5703client # copying 1 paths to 'lb:50051'...5704worker1 # [ 138.369539] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460625705client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5706worker1 # [ 138.392752] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5707worker1 # [ 138.596121] nix-daemon[834]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)5708worker1 # [ 138.613560] nix-daemon[834]: reaped child process 1656, status = succeeded5709worker1 # [ 138.786665] nix-daemon[834]: accepted connection from pid 1658, user nix-grpc-daemon (trusted)5710worker1 # [ 138.803922] nix-daemon[834]: reaped child process 1661, status = succeeded5711lb # [ 138.796626] niks3-server[1024]: 2026/09/20 13:36:33 INFO Received uploads request method=POST path=/api/pending_closures5712worker1 # [ 138.824568] nix-grpc-daemon[1502]: time=2026-09-20T13:36:33.386Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5713worker1 # [ 138.826040] nix-grpc-daemon[1502]: time=2026-09-20T13:36:33.386Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5714lb # [ 138.823091] niks3-server[1024]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5715worker1 # [ 138.854825] nix-grpc-daemon[1502]: time=2026-09-20T13:36:33.416Z level=INFO msg="Uploading 1 narinfos"5716lb # [ 138.840168] niks3-server[1024]: 2026/09/20 13:36:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5717lb # [ 138.849602] niks3-server[1024]: 2026/09/20 13:36:33 INFO Signed narinfos id=43 count=15718lb # [ 138.858272] niks3-server[1024]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5719lb # [ 138.862071] niks3-server[1024]: 2026/09/20 13:36:33 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5720lb # [ 138.867002] niks3-server[1024]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5721lb # [ 138.871386] niks3-server[1024]: 2026/09/20 13:36:33 INFO Completed upload id=435722worker1 # [ 138.889693] nix-grpc-daemon[1502]: time=2026-09-20T13:36:33.451Z level=INFO msg="Upload complete. (478ms)"5723worker1 # [ 138.896734] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=7685724worker1 # [ 138.907130] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625725worker1 # [ 138.915118] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5726worker1 # [ 138.920799] nix-daemon[834]: reaped child process 1651, status = succeeded5727client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5728worker1 # [ 138.931703] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5729client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5730client # farm-top> LOG-farm-top-during-drain5731worker1 # [ 140.375673] nix-daemon[834]: reaped child process 1665, status = succeeded5732worker1 # [ 140.550945] nix-daemon[834]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)5733worker1 # [ 140.568415] nix-daemon[834]: reaped child process 1683, status = succeeded5734lb # [ 140.562116] niks3-server[1024]: 2026/09/20 13:36:35 INFO Received uploads request method=POST path=/api/pending_closures5735worker1 # [ 140.591803] nix-grpc-daemon[1502]: time=2026-09-20T13:36:35.152Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5736worker1 # [ 140.593465] nix-grpc-daemon[1502]: time=2026-09-20T13:36:35.152Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5737lb # [ 140.595151] niks3-server[1024]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5738lb # [ 140.608407] niks3-server[1024]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5739lb # [ 140.618767] niks3-server[1024]: 2026/09/20 13:36:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5740worker1 # [ 140.644155] nix-grpc-daemon[1502]: time=2026-09-20T13:36:35.205Z level=INFO msg="Uploading 1 narinfos"5741lb # [ 140.625337] niks3-server[1024]: 2026/09/20 13:36:35 INFO Signed narinfos id=44 count=15742lb # [ 140.630603] niks3-server[1024]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5743lb # [ 140.643102] niks3-server[1024]: 2026/09/20 13:36:35 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5744lb # [ 140.650841] niks3-server[1024]: 2026/09/20 13:36:35 INFO Completed upload id=445745worker1 # [ 140.668964] nix-grpc-daemon[1502]: time=2026-09-20T13:36:35.230Z level=INFO msg="Upload complete. (327ms)"5746lb # [ 140.654172] niks3-server[1024]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5747worker1 # [ 140.683439] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15748worker1 # [ 140.699807] nix-daemon[834]: reaped child process 1663, status = succeeded5749client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.81 seconds)5750client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5751client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5752worker2: must succeed: pkill -f 'read -t [9]0 x'5753worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5754client: waiting for success: ! systemctl is-active sw5755worker2 # [ 141.057917] nix-daemon[845]: reaped child process 1554, status = succeeded5756worker2 # [ 141.216389] nix-daemon[845]: accepted connection from pid 1726, user nix-grpc-daemon (trusted)5757worker2 # [ 141.235935] nix-daemon[845]: reaped child process 1729, status = succeeded5758lb # [ 141.254953] niks3-server[1024]: 2026/09/20 13:36:35 INFO Received uploads request method=POST path=/api/pending_closures5759worker2 # [ 141.257368] nix-grpc-daemon[1494]: time=2026-09-20T13:36:35.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5760worker2 # [ 141.258852] nix-grpc-daemon[1494]: time=2026-09-20T13:36:35.819Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5761lb # [ 141.291626] niks3-server[1024]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5762worker2 # [ 141.299621] nix-grpc-daemon[1494]: time=2026-09-20T13:36:35.861Z level=INFO msg="Uploading 1 narinfos"5763lb # [ 141.302951] niks3-server[1024]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5764lb # [ 141.317227] niks3-server[1024]: 2026/09/20 13:36:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5765lb # [ 141.321806] niks3-server[1024]: 2026/09/20 13:36:35 INFO Signed narinfos id=45 count=15766worker2 # [ 141.319947] nix-grpc-daemon[1494]: time=2026-09-20T13:36:35.882Z level=INFO msg="Upload complete. (275ms)"5767lb # [ 141.328409] niks3-server[1024]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5768lb # [ 141.333439] niks3-server[1024]: 2026/09/20 13:36:35 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5769lb # [ 141.337177] niks3-server[1024]: 2026/09/20 13:36:35 INFO Completed upload id=455770worker2 # [ 141.328681] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58592 duration_s=12 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15771lb # [ 141.339858] niks3-server[1024]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5772worker2 # [ 141.342167] nix-daemon[845]: reaped child process 1552, status = succeeded5773client # [ 141.371052] systemd[1]: sw.service: Deactivated successfully.5774client # [ 141.376565] systemd[1]: sw.service: Consumed 202ms CPU time over 14.124s wall clock time, 36.9M memory peak, 8.5K incoming IP traffic, 9.2K outgoing IP traffic.5775worker2 # [ 141.558451] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:36Z level=info event=signal_exit5776lb # [ 142.123472] envoy[921]: conn peer=[2001:db8:1::1]:40308 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9612 tx=3865 ms=65185777lb # [ 142.128597] envoy[921]: conn peer=[2001:db8:1::1]:48438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2939 tx=1352 ms=138135778client: (finished: waiting for success: ! systemctl is-active sw, in 1.14 seconds)5779client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5780client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5781worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5782worker2 # [ 142.286434] systemd[1]: Starting scheduler-token.service...5783worker2 # [ 142.370975] systemd[1]: scheduler-token.service: Deactivated successfully.5784worker2 # [ 142.374510] systemd[1]: Finished scheduler-token.service.5785worker2 # [ 142.375614] systemd[1]: scheduler-token.service: Consumed 30ms CPU time over 87ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5786worker1 # [ 142.825655] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:341345787worker2 # [ 142.801728] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:37Z level=info event=scheduler_restarting5788worker1 # [ 142.830387] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:37Z level=info event=worker_gone addr=192.168.1.4:500515789worker2 # [ 142.821107] nix-grpc-daemon[1343]: ts=2026-09-20T13:36:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5790worker2 # [ 142.846578] nix-daemon[845]: reaped child process 1446, status = succeeded5791worker2 # [ 143.238818] nix-grpc-daemon[1343]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5792worker2 # [ 143.243245] nix-grpc-daemon[1343]: E0000 00:00:1789911397.801390 1386 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5793worker2 # [ 143.249698] nix-grpc-daemon[1343]: E0000 00:00:1789911397.801686 1386 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5794worker2 # [ 143.993818] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5795worker2 # [ 143.996254] systemd[1]: nix-grpc-daemon.service: Consumed 1.013s CPU time over 46.652s wall clock time, 30.7M memory peak, 24.3K incoming IP traffic, 45.2K outgoing IP traffic.5796worker2 # [ 144.008252] systemd[1]: Starting Nix worker-protocol over gRPC...5797worker2 # [ 144.190549] nix-grpc-daemon[1746]: ts=2026-09-20T13:36:38Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5798worker2 # [ 144.193096] nix-grpc-daemon[1746]: ts=2026-09-20T13:36:38Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5799worker2 # [ 144.197273] nix-grpc-daemon[1746]: ts=2026-09-20T13:36:38Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15800worker2 # [ 144.205684] nix-grpc-daemon[1746]: ts=2026-09-20T13:36:38Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5801worker2 # [ 144.207400] systemd[1]: Started Nix worker-protocol over gRPC.5802worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.02 seconds)5803lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5804lb: must succeed: curl -sf localhost:9901/clusters5805worker2 # [ 144.232971] nix-grpc-daemon[1746]: ts=2026-09-20T13:36:38Z level=info event=scheduler_connected addr=lb:500515806worker1 # [ 144.265261] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:38Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:341345807worker1 # [ 144.271117] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05808lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5809lb: must succeed: curl -sf localhost:9901/clusters5810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5811lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5812lb: must succeed: curl -sf localhost:9901/clusters5813lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5814worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5815worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5816(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 18.37 seconds)5817subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5818client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15819worker2 # [ 145.791104] nix-grpc-daemon[1746]: ts=2026-09-20T13:36:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:470065820worker2 # [ 145.806861] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)5821worker2 # [ 145.827271] nix-grpc-daemon[1746]: ts=2026-09-20T13:36:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470065822worker2 # [ 145.845672] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)5823worker2 # [ 145.867761] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)5824worker2 # [ 146.018651] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 3/5)5825worker2 # [ 146.124345] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 699 ms (attempt 4/5)5826worker2 # [ 146.825607] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5827worker1 # [ 146.909418] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:41Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05828worker1 # [ 146.916402] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:41Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15829worker1 # [ 146.923013] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:41Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05830worker1 # [ 146.928364] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:41Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05831worker1 # [ 146.963706] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625832worker1 # [ 146.978419] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460625833worker1 # [ 146.992709] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460625834worker1 # [ 147.007390] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5835worker1 # [ 147.208975] nix-daemon[834]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)5836worker1 # [ 147.228391] nix-daemon[834]: reaped child process 1695, status = succeeded5837worker1 # [ 147.398506] nix-daemon[834]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)5838worker1 # [ 147.416842] nix-daemon[834]: reaped child process 1700, status = succeeded5839lb # [ 147.410769] niks3-server[1024]: 2026/09/20 13:36:41 INFO Received uploads request method=POST path=/api/pending_closures5840worker1 # [ 147.438746] nix-grpc-daemon[1502]: time=2026-09-20T13:36:42.000Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5841worker1 # [ 147.440662] nix-grpc-daemon[1502]: time=2026-09-20T13:36:42.002Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5842lb # [ 147.442241] niks3-server[1024]: 2026/09/20 13:36:42 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5843lb # [ 147.456037] niks3-server[1024]: 2026/09/20 13:36:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5844worker1 # [ 147.480656] nix-grpc-daemon[1502]: time=2026-09-20T13:36:42.042Z level=INFO msg="Uploading 1 narinfos"5845lb # [ 147.466915] niks3-server[1024]: 2026/09/20 13:36:42 INFO Signed narinfos id=46 count=15846lb # [ 147.472513] niks3-server[1024]: 2026/09/20 13:36:42 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5847lb # [ 147.480717] niks3-server[1024]: 2026/09/20 13:36:42 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5848lb # [ 147.486426] niks3-server[1024]: 2026/09/20 13:36:42 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5849lb # [ 147.491870] niks3-server[1024]: 2026/09/20 13:36:42 INFO Completed upload id=465850worker1 # [ 147.510499] nix-grpc-daemon[1502]: time=2026-09-20T13:36:42.072Z level=INFO msg="Upload complete. (486ms)"5851worker1 # [ 147.518571] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=5125852worker1 # [ 147.528794] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625853worker1 # [ 147.539074] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5854worker1 # [ 147.543632] nix-daemon[834]: reaped child process 1690, status = succeeded5855worker1 # [ 147.556353] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5856worker1 # [ 148.786752] nix-daemon[834]: reaped child process 1704, status = succeeded5857worker1 # [ 148.950178] nix-daemon[834]: accepted connection from pid 1719, user nix-grpc-daemon (trusted)5858worker1 # [ 148.966430] nix-daemon[834]: reaped child process 1722, status = succeeded5859lb # [ 148.961072] niks3-server[1024]: 2026/09/20 13:36:43 INFO Received uploads request method=POST path=/api/pending_closures5860worker1 # [ 148.988366] nix-grpc-daemon[1502]: time=2026-09-20T13:36:43.549Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5861worker1 # [ 148.989814] nix-grpc-daemon[1502]: time=2026-09-20T13:36:43.549Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5862lb # [ 148.997187] niks3-server[1024]: 2026/09/20 13:36:43 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5863worker1 # [ 149.032715] nix-grpc-daemon[1502]: time=2026-09-20T13:36:43.594Z level=INFO msg="Uploading 1 narinfos"5864lb # [ 149.014656] niks3-server[1024]: 2026/09/20 13:36:43 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5865lb # [ 149.028203] niks3-server[1024]: 2026/09/20 13:36:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5866lb # [ 149.031596] niks3-server[1024]: 2026/09/20 13:36:43 INFO Signed narinfos id=47 count=15867lb # [ 149.036232] niks3-server[1024]: 2026/09/20 13:36:43 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5868lb # [ 149.042553] niks3-server[1024]: 2026/09/20 13:36:43 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5869lb # [ 149.046913] niks3-server[1024]: 2026/09/20 13:36:43 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5870lb # [ 149.050638] niks3-server[1024]: 2026/09/20 13:36:43 INFO Completed upload id=475871worker1 # [ 149.069449] nix-grpc-daemon[1502]: time=2026-09-20T13:36:43.631Z level=INFO msg="Upload complete. (301ms)"5872worker1 # [ 149.087486] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=15873worker1 # [ 149.101356] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:43Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15874worker1 # [ 149.110491] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625875worker1 # [ 149.114318] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:43Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05876worker1 # [ 149.118038] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460625877worker1 # [ 149.122167] nix-daemon[834]: reaped child process 1702, status = succeeded5878worker1 # [ 149.132128] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460625879worker1 # [ 149.134887] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5880worker1 # [ 149.319310] nix-daemon[834]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)5881worker1 # [ 149.335769] nix-daemon[834]: reaped child process 1731, status = succeeded5882worker1 # [ 149.511728] nix-daemon[834]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)5883worker1 # [ 149.527469] nix-daemon[834]: reaped child process 1736, status = succeeded5884lb # [ 149.523324] niks3-server[1024]: 2026/09/20 13:36:44 INFO Received uploads request method=POST path=/api/pending_closures5885worker1 # [ 149.550765] nix-grpc-daemon[1502]: time=2026-09-20T13:36:44.112Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5886worker1 # [ 149.552342] nix-grpc-daemon[1502]: time=2026-09-20T13:36:44.112Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5887lb # [ 149.547714] niks3-server[1024]: 2026/09/20 13:36:44 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5888worker1 # [ 149.579028] nix-grpc-daemon[1502]: time=2026-09-20T13:36:44.140Z level=INFO msg="Uploading 1 narinfos"5889lb # [ 149.564269] niks3-server[1024]: 2026/09/20 13:36:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5890lb # [ 149.575147] niks3-server[1024]: 2026/09/20 13:36:44 INFO Signed narinfos id=48 count=15891lb # [ 149.581025] niks3-server[1024]: 2026/09/20 13:36:44 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5892lb # [ 149.591529] niks3-server[1024]: 2026/09/20 13:36:44 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5893worker1 # [ 149.616239] nix-grpc-daemon[1502]: time=2026-09-20T13:36:44.178Z level=INFO msg="Upload complete. (476ms)"5894lb # [ 149.597877] niks3-server[1024]: 2026/09/20 13:36:44 INFO Completed upload id=485895lb # [ 149.602030] niks3-server[1024]: 2026/09/20 13:36:44 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5896worker1 # [ 149.623166] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=5125897worker1 # [ 149.634535] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625898worker1 # [ 149.642981] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5899worker1 # [ 149.645778] nix-daemon[834]: reaped child process 1726, status = succeeded5900worker1 # [ 149.657788] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5901worker1 # [ 150.486067] nix-daemon[834]: reaped child process 1740, status = succeeded5902worker1 # [ 150.650380] nix-daemon[834]: accepted connection from pid 1755, user nix-grpc-daemon (trusted)5903worker1 # [ 150.668580] nix-daemon[834]: reaped child process 1758, status = succeeded5904lb # [ 150.664944] niks3-server[1024]: 2026/09/20 13:36:45 INFO Received uploads request method=POST path=/api/pending_closures5905worker1 # [ 150.696438] nix-grpc-daemon[1502]: time=2026-09-20T13:36:45.255Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5906worker1 # [ 150.699374] nix-grpc-daemon[1502]: time=2026-09-20T13:36:45.255Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5907lb # [ 150.706510] niks3-server[1024]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5908worker1 # [ 150.736171] nix-grpc-daemon[1502]: time=2026-09-20T13:36:45.297Z level=INFO msg="Uploading 1 narinfos"5909lb # [ 150.721712] niks3-server[1024]: 2026/09/20 13:36:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5910lb # [ 150.731303] niks3-server[1024]: 2026/09/20 13:36:45 INFO Signed narinfos id=49 count=15911lb # [ 150.737104] niks3-server[1024]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5912worker1 # [ 150.763988] nix-grpc-daemon[1502]: time=2026-09-20T13:36:45.326Z level=INFO msg="Upload complete. (297ms)"5913lb # [ 150.746421] niks3-server[1024]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5914lb # [ 150.751508] niks3-server[1024]: 2026/09/20 13:36:45 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5915lb # [ 150.755955] niks3-server[1024]: 2026/09/20 13:36:45 INFO Completed upload id=495916lb # [ 150.759574] niks3-server[1024]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5917worker1 # [ 150.779091] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=15918worker1 # [ 150.792938] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:45Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15919worker1 # [ 150.803937] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625920worker1 # [ 150.809039] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:45Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05921worker1 # [ 150.813504] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460625922worker1 # [ 150.817764] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460625923worker1 # [ 150.822393] nix-daemon[834]: reaped child process 1738, status = succeeded5924worker1 # [ 150.834861] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5925worker1 # [ 151.013099] nix-daemon[834]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)5926worker1 # [ 151.031409] nix-daemon[834]: reaped child process 1767, status = succeeded5927worker1 # [ 151.201481] nix-daemon[834]: accepted connection from pid 1769, user nix-grpc-daemon (trusted)5928worker1 # [ 151.218176] nix-daemon[834]: reaped child process 1772, status = succeeded5929lb # [ 151.210473] niks3-server[1024]: 2026/09/20 13:36:45 INFO Received uploads request method=POST path=/api/pending_closures5930worker1 # [ 151.237523] nix-grpc-daemon[1502]: time=2026-09-20T13:36:45.799Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5931worker1 # [ 151.239038] nix-grpc-daemon[1502]: time=2026-09-20T13:36:45.799Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"5932lb # [ 151.238342] niks3-server[1024]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst5933worker1 # [ 151.268480] nix-grpc-daemon[1502]: time=2026-09-20T13:36:45.830Z level=INFO msg="Uploading 1 narinfos"5934lb # [ 151.255068] niks3-server[1024]: 2026/09/20 13:36:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5935lb # [ 151.265412] niks3-server[1024]: 2026/09/20 13:36:45 INFO Signed narinfos id=50 count=15936lb # [ 151.272520] niks3-server[1024]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls5937lb # [ 151.276605] niks3-server[1024]: 2026/09/20 13:36:45 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5938worker1 # [ 151.299191] nix-grpc-daemon[1502]: time=2026-09-20T13:36:45.861Z level=INFO msg="Upload complete. (459ms)"5939lb # [ 151.281336] niks3-server[1024]: 2026/09/20 13:36:45 INFO Completed upload id=505940lb # [ 151.285368] niks3-server[1024]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo5941worker1 # [ 151.308198] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=5125942worker1 # [ 151.312417] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625943worker1 # [ 151.322027] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5944worker1 # [ 151.326368] nix-daemon[834]: reaped child process 1762, status = succeeded5945worker1 # [ 151.336111] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5946lb # [ 152.128721] envoy[921]: conn peer=[2001:db8:1::4]:44476 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=13696 tx=951 ms=452495947worker1 # [ 152.566861] nix-daemon[834]: reaped child process 1776, status = succeeded5948worker1 # [ 152.729150] nix-daemon[834]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)5949worker1 # [ 152.747770] nix-daemon[834]: reaped child process 1794, status = succeeded5950lb # [ 152.741814] niks3-server[1024]: 2026/09/20 13:36:47 INFO Received uploads request method=POST path=/api/pending_closures5951worker1 # [ 152.769896] nix-grpc-daemon[1502]: time=2026-09-20T13:36:47.330Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5952worker1 # [ 152.771522] nix-grpc-daemon[1502]: time=2026-09-20T13:36:47.330Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"5953lb # [ 152.775991] niks3-server[1024]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst5954lb # [ 152.789716] niks3-server[1024]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv5955lb # [ 152.798003] niks3-server[1024]: 2026/09/20 13:36:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5956worker1 # [ 152.822810] nix-grpc-daemon[1502]: time=2026-09-20T13:36:47.384Z level=INFO msg="Uploading 1 narinfos"5957lb # [ 152.804247] niks3-server[1024]: 2026/09/20 13:36:47 INFO Signed narinfos id=51 count=15958lb # [ 152.810770] niks3-server[1024]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls5959lb # [ 152.820639] niks3-server[1024]: 2026/09/20 13:36:47 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5960lb # [ 152.827808] niks3-server[1024]: 2026/09/20 13:36:47 INFO Completed upload id=515961worker1 # [ 152.846334] nix-grpc-daemon[1502]: time=2026-09-20T13:36:47.408Z level=INFO msg="Upload complete. (298ms)"5962lb # [ 152.831130] niks3-server[1024]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo5963worker1 # [ 152.856393] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=15964worker1 # [ 152.876271] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:47Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=15965worker1 # [ 152.883737] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625966worker1 # [ 152.888842] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:47Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=05967worker1 # [ 152.893387] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460625968worker1 # [ 152.897344] nix-daemon[834]: reaped child process 1774, status = succeeded5969worker1 # [ 152.899055] nix-grpc-daemon[1502]: time=2026-09-20T13:36:47.458Z level=INFO msg="All 1 paths already cached"5970worker1 # [ 152.902296] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460625971worker1 # [ 152.910897] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5972worker1 # [ 153.087609] nix-daemon[834]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)5973worker1 # [ 153.106049] nix-daemon[834]: reaped child process 1803, status = succeeded5974worker1 # [ 153.276114] nix-daemon[834]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)5975worker1 # [ 153.293717] nix-daemon[834]: reaped child process 1808, status = succeeded5976lb # [ 153.286350] niks3-server[1024]: 2026/09/20 13:36:47 INFO Received uploads request method=POST path=/api/pending_closures5977worker1 # [ 153.313204] nix-grpc-daemon[1502]: time=2026-09-20T13:36:47.875Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5978worker1 # [ 153.314651] nix-grpc-daemon[1502]: time=2026-09-20T13:36:47.875Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"5979lb # [ 153.313183] niks3-server[1024]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst5980worker1 # [ 153.342208] nix-grpc-daemon[1502]: time=2026-09-20T13:36:47.904Z level=INFO msg="Uploading 1 narinfos"5981lb # [ 153.328144] niks3-server[1024]: 2026/09/20 13:36:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5982lb # [ 153.338804] niks3-server[1024]: 2026/09/20 13:36:47 INFO Signed narinfos id=52 count=15983lb # [ 153.346827] niks3-server[1024]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls5984worker1 # [ 153.369482] nix-grpc-daemon[1502]: time=2026-09-20T13:36:47.931Z level=INFO msg="Upload complete. (448ms)"5985lb # [ 153.351766] niks3-server[1024]: 2026/09/20 13:36:47 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5986lb # [ 153.357158] niks3-server[1024]: 2026/09/20 13:36:47 INFO Completed upload id=525987worker1 # [ 153.375538] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=6245988lb # [ 153.360426] niks3-server[1024]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo5989worker1 # [ 153.384551] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460625990worker1 # [ 153.396297] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5991worker1 # [ 153.398573] nix-daemon[834]: reaped child process 1798, status = succeeded5992worker1 # [ 153.410906] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5993worker1 # [ 155.051717] nix-daemon[834]: reaped child process 1812, status = succeeded5994worker1 # [ 155.234963] nix-daemon[834]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)5995worker1 # [ 155.253694] nix-daemon[834]: reaped child process 1830, status = succeeded5996lb # [ 155.247800] niks3-server[1024]: 2026/09/20 13:36:49 INFO Received uploads request method=POST path=/api/pending_closures5997worker1 # [ 155.276408] nix-grpc-daemon[1502]: time=2026-09-20T13:36:49.837Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5998worker1 # [ 155.277905] nix-grpc-daemon[1502]: time=2026-09-20T13:36:49.837Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"5999lb # [ 155.291747] niks3-server[1024]: 2026/09/20 13:36:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6000worker1 # [ 155.320082] nix-grpc-daemon[1502]: time=2026-09-20T13:36:49.881Z level=INFO msg="Uploading 1 narinfos"6001lb # [ 155.306862] niks3-server[1024]: 2026/09/20 13:36:49 INFO Signed narinfos id=53 count=16002lb # [ 155.316340] niks3-server[1024]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6003lb # [ 155.327374] niks3-server[1024]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6004lb # [ 155.335048] niks3-server[1024]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6005lb # [ 155.341886] niks3-server[1024]: 2026/09/20 13:36:49 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6006worker1 # [ 155.364897] nix-grpc-daemon[1502]: time=2026-09-20T13:36:49.926Z level=INFO msg="Upload complete. (343ms)"6007lb # [ 155.350981] niks3-server[1024]: 2026/09/20 13:36:49 INFO Completed upload id=536008lb # [ 155.357638] niks3-server[1024]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6009worker1 # [ 155.384772] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16010worker1 # [ 155.400258] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:49Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16011worker1 # [ 155.408990] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626012worker1 # [ 155.412331] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:49Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06013worker1 # [ 155.417126] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460626014worker1 # [ 155.421589] nix-daemon[834]: reaped child process 1810, status = succeeded6015worker1 # [ 155.424457] nix-grpc-daemon[1502]: time=2026-09-20T13:36:49.986Z level=INFO msg="All 1 paths already cached"6016worker1 # [ 155.431446] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460626017worker1 # [ 155.445508] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6018worker1 # [ 155.638212] nix-daemon[834]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)6019worker1 # [ 155.658379] nix-daemon[834]: reaped child process 1839, status = succeeded6020worker1 # [ 155.831445] nix-daemon[834]: accepted connection from pid 1841, user nix-grpc-daemon (trusted)6021worker1 # [ 155.847130] nix-daemon[834]: reaped child process 1844, status = succeeded6022lb # [ 155.844172] niks3-server[1024]: 2026/09/20 13:36:50 INFO Received uploads request method=POST path=/api/pending_closures6023worker1 # [ 155.871524] nix-grpc-daemon[1502]: time=2026-09-20T13:36:50.433Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6024worker1 # [ 155.873202] nix-grpc-daemon[1502]: time=2026-09-20T13:36:50.433Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6025lb # [ 155.870668] niks3-server[1024]: 2026/09/20 13:36:50 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6026worker1 # [ 155.902198] nix-grpc-daemon[1502]: time=2026-09-20T13:36:50.464Z level=INFO msg="Uploading 1 narinfos"6027lb # [ 155.887226] niks3-server[1024]: 2026/09/20 13:36:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6028lb # [ 155.898742] niks3-server[1024]: 2026/09/20 13:36:50 INFO Signed narinfos id=54 count=16029lb # [ 155.908415] niks3-server[1024]: 2026/09/20 13:36:50 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6030lb # [ 155.913791] niks3-server[1024]: 2026/09/20 13:36:50 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6031worker1 # [ 155.937358] nix-grpc-daemon[1502]: time=2026-09-20T13:36:50.499Z level=INFO msg="Upload complete. (478ms)"6032lb # [ 155.918904] niks3-server[1024]: 2026/09/20 13:36:50 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6033lb # [ 155.924525] niks3-server[1024]: 2026/09/20 13:36:50 INFO Completed upload id=546034worker1 # [ 155.944132] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=6246035worker1 # [ 155.957863] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626036worker1 # [ 155.964272] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6037worker1 # [ 155.968895] nix-daemon[834]: reaped child process 1834, status = succeeded6038worker1 # [ 155.982080] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6039worker1 # [ 157.421728] nix-daemon[834]: reaped child process 1848, status = succeeded6040worker1 # [ 157.592130] nix-daemon[834]: accepted connection from pid 1863, user nix-grpc-daemon (trusted)6041worker1 # [ 157.610609] nix-daemon[834]: reaped child process 1866, status = succeeded6042lb # [ 157.605789] niks3-server[1024]: 2026/09/20 13:36:52 INFO Received uploads request method=POST path=/api/pending_closures6043worker1 # [ 157.633215] nix-grpc-daemon[1502]: time=2026-09-20T13:36:52.193Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6044worker1 # [ 157.634925] nix-grpc-daemon[1502]: time=2026-09-20T13:36:52.194Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6045lb # [ 157.635779] niks3-server[1024]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6046lb # [ 157.650386] niks3-server[1024]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6047lb # [ 157.660472] niks3-server[1024]: 2026/09/20 13:36:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6048worker1 # [ 157.684381] nix-grpc-daemon[1502]: time=2026-09-20T13:36:52.246Z level=INFO msg="Uploading 1 narinfos"6049lb # [ 157.669870] niks3-server[1024]: 2026/09/20 13:36:52 INFO Signed narinfos id=55 count=16050lb # [ 157.675381] niks3-server[1024]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6051lb # [ 157.683070] niks3-server[1024]: 2026/09/20 13:36:52 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6052lb # [ 157.690074] niks3-server[1024]: 2026/09/20 13:36:52 INFO Completed upload id=556053worker1 # [ 157.708800] nix-grpc-daemon[1502]: time=2026-09-20T13:36:52.270Z level=INFO msg="Upload complete. (316ms)"6054lb # [ 157.693360] niks3-server[1024]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6055worker1 # [ 157.728242] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16056worker1 # [ 157.744672] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:52Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16057worker1 # [ 157.748468] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626058worker1 # [ 157.751659] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:52Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06059worker1 # [ 157.756271] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460626060worker1 # [ 157.761611] nix-daemon[834]: reaped child process 1846, status = succeeded6061worker1 # [ 157.764836] nix-grpc-daemon[1502]: time=2026-09-20T13:36:52.326Z level=INFO msg="All 1 paths already cached"6062worker1 # [ 157.771293] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460626063worker1 # [ 157.780463] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6064worker1 # [ 157.963361] nix-daemon[834]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6065worker1 # [ 157.982356] nix-daemon[834]: reaped child process 1875, status = succeeded6066worker1 # [ 158.152192] nix-daemon[834]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)6067worker1 # [ 158.170689] nix-daemon[834]: reaped child process 1880, status = succeeded6068lb # [ 158.164549] niks3-server[1024]: 2026/09/20 13:36:52 INFO Received uploads request method=POST path=/api/pending_closures6069worker1 # [ 158.191544] nix-grpc-daemon[1502]: time=2026-09-20T13:36:52.753Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6070worker1 # [ 158.193167] nix-grpc-daemon[1502]: time=2026-09-20T13:36:52.753Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6071lb # [ 158.191473] niks3-server[1024]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6072worker1 # [ 158.222246] nix-grpc-daemon[1502]: time=2026-09-20T13:36:52.784Z level=INFO msg="Uploading 1 narinfos"6073lb # [ 158.207015] niks3-server[1024]: 2026/09/20 13:36:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6074lb # [ 158.217491] niks3-server[1024]: 2026/09/20 13:36:52 INFO Signed narinfos id=56 count=16075lb # [ 158.226782] niks3-server[1024]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6076lb # [ 158.230632] niks3-server[1024]: 2026/09/20 13:36:52 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6077worker1 # [ 158.253079] nix-grpc-daemon[1502]: time=2026-09-20T13:36:52.815Z level=INFO msg="Upload complete. (463ms)"6078lb # [ 158.235090] niks3-server[1024]: 2026/09/20 13:36:52 INFO Completed upload id=566079lb # [ 158.238606] niks3-server[1024]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6080worker1 # [ 158.260119] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=6246081worker1 # [ 158.271084] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626082worker1 # [ 158.276520] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6083worker1 # [ 158.280914] nix-daemon[834]: reaped child process 1870, status = succeeded6084worker1 # [ 158.293517] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6085worker1 # [ 159.535656] nix-daemon[834]: reaped child process 1884, status = succeeded6086worker1 # [ 159.713309] nix-daemon[834]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6087worker1 # [ 159.732300] nix-daemon[834]: reaped child process 1902, status = succeeded6088lb # [ 159.726301] niks3-server[1024]: 2026/09/20 13:36:54 INFO Received uploads request method=POST path=/api/pending_closures6089worker1 # [ 159.756148] nix-grpc-daemon[1502]: time=2026-09-20T13:36:54.317Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6090worker1 # [ 159.762696] nix-grpc-daemon[1502]: time=2026-09-20T13:36:54.320Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6091lb # [ 159.764471] niks3-server[1024]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6092lb # [ 159.779726] niks3-server[1024]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6093lb # [ 159.789345] niks3-server[1024]: 2026/09/20 13:36:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6094worker1 # [ 159.813760] nix-grpc-daemon[1502]: time=2026-09-20T13:36:54.375Z level=INFO msg="Uploading 1 narinfos"6095lb # [ 159.799316] niks3-server[1024]: 2026/09/20 13:36:54 INFO Signed narinfos id=57 count=16096lb # [ 159.805545] niks3-server[1024]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6097lb # [ 159.812324] niks3-server[1024]: 2026/09/20 13:36:54 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6098lb # [ 159.821267] niks3-server[1024]: 2026/09/20 13:36:54 INFO Completed upload id=576099worker1 # [ 159.839971] nix-grpc-daemon[1502]: time=2026-09-20T13:36:54.401Z level=INFO msg="Upload complete. (336ms)"6100lb # [ 159.823836] niks3-server[1024]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6101worker1 # [ 159.857690] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16102worker1 # [ 159.875064] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:54Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16103worker1 # [ 159.886223] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626104worker1 # [ 159.891023] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:54Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06105worker1 # [ 159.895360] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460626106worker1 # [ 159.900467] nix-daemon[834]: reaped child process 1882, status = succeeded6107worker1 # [ 159.908217] nix-grpc-daemon[1502]: time=2026-09-20T13:36:54.456Z level=INFO msg="All 2 paths already cached"6108worker1 # [ 159.910311] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6109worker1 # [ 159.915651] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460626110worker1 # [ 160.101720] nix-daemon[834]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6111worker1 # [ 160.123177] nix-daemon[834]: reaped child process 1911, status = succeeded6112worker1 # [ 160.298843] nix-daemon[834]: accepted connection from pid 1913, user nix-grpc-daemon (trusted)6113worker1 # [ 160.317726] nix-daemon[834]: reaped child process 1916, status = succeeded6114lb # [ 160.312310] niks3-server[1024]: 2026/09/20 13:36:54 INFO Received uploads request method=POST path=/api/pending_closures6115worker1 # [ 160.339973] nix-grpc-daemon[1502]: time=2026-09-20T13:36:54.901Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6116worker1 # [ 160.341624] nix-grpc-daemon[1502]: time=2026-09-20T13:36:54.901Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6117lb # [ 160.342030] niks3-server[1024]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6118worker1 # [ 160.369653] nix-grpc-daemon[1502]: time=2026-09-20T13:36:54.931Z level=INFO msg="Uploading 1 narinfos"6119lb # [ 160.354967] niks3-server[1024]: 2026/09/20 13:36:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6120lb # [ 160.367670] niks3-server[1024]: 2026/09/20 13:36:54 INFO Signed narinfos id=58 count=16121lb # [ 160.375406] niks3-server[1024]: 2026/09/20 13:36:54 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6122worker1 # [ 160.402889] nix-grpc-daemon[1502]: time=2026-09-20T13:36:54.964Z level=INFO msg="Upload complete. (481ms)"6123lb # [ 160.385109] niks3-server[1024]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6124lb # [ 160.390531] niks3-server[1024]: 2026/09/20 13:36:54 INFO Completed upload id=586125worker1 # [ 160.409855] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=7526126lb # [ 160.394599] niks3-server[1024]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6127worker1 # [ 160.423578] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626128worker1 # [ 160.429593] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6129worker1 # [ 160.434605] nix-daemon[834]: reaped child process 1906, status = succeeded6130worker1 # [ 160.445173] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6131worker1 # [ 161.269410] nix-daemon[834]: reaped child process 1920, status = succeeded6132worker1 # [ 161.425496] nix-daemon[834]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6133worker1 # [ 161.443518] nix-daemon[834]: reaped child process 1938, status = succeeded6134lb # [ 161.441060] niks3-server[1024]: 2026/09/20 13:36:55 INFO Received uploads request method=POST path=/api/pending_closures6135worker1 # [ 161.470703] nix-grpc-daemon[1502]: time=2026-09-20T13:36:56.031Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6136worker1 # [ 161.472281] nix-grpc-daemon[1502]: time=2026-09-20T13:36:56.031Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6137lb # [ 161.471725] niks3-server[1024]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6138lb # [ 161.488512] niks3-server[1024]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6139lb # [ 161.498372] niks3-server[1024]: 2026/09/20 13:36:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6140worker1 # [ 161.522589] nix-grpc-daemon[1502]: time=2026-09-20T13:36:56.084Z level=INFO msg="Uploading 1 narinfos"6141lb # [ 161.508078] niks3-server[1024]: 2026/09/20 13:36:56 INFO Signed narinfos id=59 count=16142lb # [ 161.510801] niks3-server[1024]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6143lb # [ 161.524601] niks3-server[1024]: 2026/09/20 13:36:56 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6144lb # [ 161.532784] niks3-server[1024]: 2026/09/20 13:36:56 INFO Completed upload id=596145worker1 # [ 161.549875] nix-grpc-daemon[1502]: time=2026-09-20T13:36:56.111Z level=INFO msg="Upload complete. (293ms)"6146lb # [ 161.538589] niks3-server[1024]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6147worker1 # [ 161.562819] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16148worker1 # [ 161.578557] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:56Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16149worker1 # [ 161.583031] nix-daemon[834]: reaped child process 1918, status = succeeded6150worker1 # [ 161.603045] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626151worker1 # [ 161.617944] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460626152worker1 # [ 161.633843] nix-grpc-daemon[1502]: time=2026-09-20T13:36:56.195Z level=INFO msg="All 2 paths already cached"6153worker1 # [ 161.642943] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460626154worker1 # [ 161.656532] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6155worker1 # [ 161.854354] nix-daemon[834]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6156worker1 # [ 161.871623] nix-daemon[834]: reaped child process 1945, status = succeeded6157worker1 # [ 162.048353] nix-daemon[834]: accepted connection from pid 1947, user nix-grpc-daemon (trusted)6158worker1 # [ 162.065388] nix-daemon[834]: reaped child process 1950, status = succeeded6159lb # [ 162.059810] niks3-server[1024]: 2026/09/20 13:36:56 INFO Received uploads request method=POST path=/api/pending_closures6160worker1 # [ 162.089331] nix-grpc-daemon[1502]: time=2026-09-20T13:36:56.651Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6161worker1 # [ 162.090807] nix-grpc-daemon[1502]: time=2026-09-20T13:36:56.651Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6162lb # [ 162.092838] niks3-server[1024]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6163lb # [ 162.106420] niks3-server[1024]: 2026/09/20 13:36:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6164worker1 # [ 162.129969] nix-grpc-daemon[1502]: time=2026-09-20T13:36:56.691Z level=INFO msg="Uploading 1 narinfos"6165lb # [ 162.117526] niks3-server[1024]: 2026/09/20 13:36:56 INFO Signed narinfos id=60 count=16166lb # [ 162.123652] niks3-server[1024]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6167worker1 # [ 162.152524] nix-grpc-daemon[1502]: time=2026-09-20T13:36:56.714Z level=INFO msg="Upload complete. (479ms)"6168lb # [ 162.134590] niks3-server[1024]: 2026/09/20 13:36:56 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6169lb # [ 162.139828] niks3-server[1024]: 2026/09/20 13:36:56 INFO Completed upload id=606170lb # [ 162.143124] niks3-server[1024]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6171worker1 # [ 162.160126] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=7526172worker1 # [ 162.175240] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626173worker1 # [ 162.181513] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6174worker1 # [ 162.186482] nix-daemon[834]: reaped child process 1940, status = succeeded6175worker1 # [ 162.197527] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6176worker1 # [ 163.224411] nix-daemon[834]: reaped child process 1954, status = succeeded6177worker1 # [ 163.379259] nix-daemon[834]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)6178worker1 # [ 163.397913] nix-daemon[834]: reaped child process 1972, status = succeeded6179lb # [ 163.393386] niks3-server[1024]: 2026/09/20 13:36:57 INFO Received uploads request method=POST path=/api/pending_closures6180worker1 # [ 163.423786] nix-grpc-daemon[1502]: time=2026-09-20T13:36:57.983Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6181worker1 # [ 163.425515] nix-grpc-daemon[1502]: time=2026-09-20T13:36:57.983Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6182lb # [ 163.429776] niks3-server[1024]: 2026/09/20 13:36:57 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6183worker1 # [ 163.461359] nix-grpc-daemon[1502]: time=2026-09-20T13:36:58.023Z level=INFO msg="Uploading 1 narinfos"6184lb # [ 163.443143] niks3-server[1024]: 2026/09/20 13:36:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6185lb # [ 163.455905] niks3-server[1024]: 2026/09/20 13:36:58 INFO Signed narinfos id=61 count=16186lb # [ 163.461846] niks3-server[1024]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6187lb # [ 163.466801] niks3-server[1024]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6188lb # [ 163.475426] niks3-server[1024]: 2026/09/20 13:36:58 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6189lb # [ 163.481409] niks3-server[1024]: 2026/09/20 13:36:58 INFO Completed upload id=616190worker1 # [ 163.493653] nix-grpc-daemon[1502]: time=2026-09-20T13:36:58.055Z level=INFO msg="Upload complete. (284ms)"6191lb # [ 163.484070] niks3-server[1024]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6192worker1 # [ 163.503694] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16193worker1 # [ 163.519667] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:58Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16194worker1 # [ 163.524413] nix-daemon[834]: reaped child process 1952, status = succeeded6195worker1 # [ 163.546873] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626196worker1 # [ 163.562750] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460626197worker1 # [ 163.576963] nix-grpc-daemon[1502]: time=2026-09-20T13:36:58.138Z level=INFO msg="All 2 paths already cached"6198worker1 # [ 163.586570] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460626199worker1 # [ 163.600682] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6200worker1 # [ 163.800983] nix-daemon[834]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6201worker1 # [ 163.819505] nix-daemon[834]: reaped child process 1979, status = succeeded6202worker1 # [ 164.003530] nix-daemon[834]: accepted connection from pid 1981, user nix-grpc-daemon (trusted)6203worker1 # [ 164.021009] nix-daemon[834]: reaped child process 1984, status = succeeded6204lb # [ 164.016313] niks3-server[1024]: 2026/09/20 13:36:58 INFO Received uploads request method=POST path=/api/pending_closures6205worker1 # [ 164.043644] nix-grpc-daemon[1502]: time=2026-09-20T13:36:58.605Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6206worker1 # [ 164.045317] nix-grpc-daemon[1502]: time=2026-09-20T13:36:58.605Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6207lb # [ 164.043269] niks3-server[1024]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6208lb # [ 164.057111] niks3-server[1024]: 2026/09/20 13:36:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6209worker1 # [ 164.084562] nix-grpc-daemon[1502]: time=2026-09-20T13:36:58.646Z level=INFO msg="Uploading 1 narinfos"6210lb # [ 164.066249] niks3-server[1024]: 2026/09/20 13:36:58 INFO Signed narinfos id=62 count=16211lb # [ 164.073382] niks3-server[1024]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6212lb # [ 164.088159] niks3-server[1024]: 2026/09/20 13:36:58 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6213worker1 # [ 164.112868] nix-grpc-daemon[1502]: time=2026-09-20T13:36:58.674Z level=INFO msg="Upload complete. (495ms)"6214lb # [ 164.095200] niks3-server[1024]: 2026/09/20 13:36:58 INFO Completed upload id=626215worker1 # [ 164.120142] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=0 paths=1 nar_bytes_in=7526216lb # [ 164.101927] niks3-server[1024]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6217worker1 # [ 164.134871] nix-grpc-daemon[1426]: ts=2026-09-20T13:36:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460626218worker1 # [ 164.142209] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6219worker1 # [ 164.147323] nix-daemon[834]: reaped child process 1974, status = succeeded6220worker1 # [ 164.161006] nix-daemon[834]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6221worker1 # [ 165.607770] nix-daemon[834]: reaped child process 1988, status = succeeded6222worker1 # [ 165.796760] nix-daemon[834]: accepted connection from pid 2003, user nix-grpc-daemon (trusted)6223worker1 # [ 165.815918] nix-daemon[834]: reaped child process 2006, status = succeeded6224lb # [ 165.810244] niks3-server[1024]: 2026/09/20 13:37:00 INFO Received uploads request method=POST path=/api/pending_closures6225worker1 # [ 165.841828] nix-grpc-daemon[1502]: time=2026-09-20T13:37:00.402Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6226worker1 # [ 165.843446] nix-grpc-daemon[1502]: time=2026-09-20T13:37:00.402Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6227lb # [ 165.847249] niks3-server[1024]: 2026/09/20 13:37:00 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6228lb # [ 165.865941] niks3-server[1024]: 2026/09/20 13:37:00 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6229lb # [ 165.874454] niks3-server[1024]: 2026/09/20 13:37:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6230worker1 # [ 165.900892] nix-grpc-daemon[1502]: time=2026-09-20T13:37:00.462Z level=INFO msg="Uploading 1 narinfos"6231lb # [ 165.885931] niks3-server[1024]: 2026/09/20 13:37:00 INFO Signed narinfos id=63 count=16232lb # [ 165.890835] niks3-server[1024]: 2026/09/20 13:37:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6233lb # [ 165.908665] niks3-server[1024]: 2026/09/20 13:37:00 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6234lb # [ 165.920678] niks3-server[1024]: 2026/09/20 13:37:00 INFO Completed upload id=636235worker1 # [ 165.939447] nix-grpc-daemon[1502]: time=2026-09-20T13:37:00.501Z level=INFO msg="Upload complete. (367ms)"6236lb # [ 165.923999] niks3-server[1024]: 2026/09/20 13:37:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6237worker1 # [ 165.953523] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46062 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16238worker1 # [ 165.978385] nix-daemon[834]: reaped child process 1986, status = succeeded6239client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 20.49 seconds)6240worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6241worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)6242client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16243worker1 # [ 166.489738] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:460626244worker2 # [ 166.479660] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470066245worker1 # [ 166.542545] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:01Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06246lb # [ 172.132758] envoy[921]: conn peer=[2001:db8:1::1]:34110 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27174 tx=10862 ms=201616247client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.46 seconds)6248(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.04 seconds)6249subtest: low disk drains a worker and builds go to the other6250worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6251worker1 # [ 197.725634] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:32Z level=info event=unhealthy reason=min_free6252worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.17 seconds)6253worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6254worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.04 seconds)6255client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26256worker1 # [ 198.056479] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:460626257client # don't know how to build these paths:6258client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6259worker1 # [ 198.068641] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460626260worker1 # [ 198.096188] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:32Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06261worker1 # [ 198.102232] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:32Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16262worker2 # [ 198.079264] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470066263worker2 # [ 198.098446] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470066264client # copying 1 paths to 'lb:50051'...6265client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6266worker2 # [ 198.114652] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470066267worker2 # [ 198.133932] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6268worker2 # [ 198.165907] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:32Z level=info event=niks3_push_started pid=18026269worker2 # [ 198.380377] nix-grpc-daemon[1802]: time=2026-09-20T13:37:32.942Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""6270worker2 # [ 198.556457] nix-daemon[845]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6271worker2 # [ 198.575185] nix-daemon[845]: reaped child process 1818, status = succeeded6272worker2 # [ 198.751681] nix-daemon[845]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6273worker2 # [ 198.767746] nix-daemon[845]: reaped child process 1823, status = succeeded6274lb # [ 198.790686] niks3-server[1024]: 2026/09/20 13:37:33 INFO Received uploads request method=POST path=/api/pending_closures6275worker2 # [ 198.797549] nix-grpc-daemon[1802]: time=2026-09-20T13:37:33.359Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6276worker2 # [ 198.799037] nix-grpc-daemon[1802]: time=2026-09-20T13:37:33.359Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6277worker2 # [ 198.840725] nix-grpc-daemon[1802]: time=2026-09-20T13:37:33.403Z level=INFO msg="Uploading 1 narinfos"6278lb # [ 198.839998] niks3-server[1024]: 2026/09/20 13:37:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6279lb # [ 198.853235] niks3-server[1024]: 2026/09/20 13:37:33 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6280lb # [ 198.861015] niks3-server[1024]: 2026/09/20 13:37:33 INFO Signed narinfos id=64 count=16281lb # [ 198.867654] niks3-server[1024]: 2026/09/20 13:37:33 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6282worker2 # [ 198.864647] nix-grpc-daemon[1802]: time=2026-09-20T13:37:33.427Z level=INFO msg="Upload complete. (479ms)"6283lb # [ 198.872789] niks3-server[1024]: 2026/09/20 13:37:33 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6284lb # [ 198.877208] niks3-server[1024]: 2026/09/20 13:37:33 INFO Completed upload id=646285worker2 # [ 198.872262] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47006 duration_s=0 paths=1 nar_bytes_in=5126286lb # [ 198.880704] niks3-server[1024]: 2026/09/20 13:37:33 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6287worker2 # [ 198.884414] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470066288worker2 # [ 198.893164] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6289worker2 # [ 198.895832] nix-daemon[845]: reaped child process 1800, status = succeeded6290client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6291worker2 # [ 198.906364] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6292client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6293client # farm-a> LOG-farm-a-drain6294worker2 # [ 200.153608] nix-daemon[845]: reaped child process 1827, status = succeeded6295worker2 # [ 200.324990] nix-daemon[845]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6296worker2 # [ 200.343366] nix-daemon[845]: reaped child process 1845, status = succeeded6297lb # [ 200.362825] niks3-server[1024]: 2026/09/20 13:37:34 INFO Received uploads request method=POST path=/api/pending_closures6298worker2 # [ 200.367418] nix-grpc-daemon[1802]: time=2026-09-20T13:37:34.927Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6299worker2 # [ 200.369533] nix-grpc-daemon[1802]: time=2026-09-20T13:37:34.927Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6300lb # [ 200.398238] niks3-server[1024]: 2026/09/20 13:37:34 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6301lb # [ 200.410161] niks3-server[1024]: 2026/09/20 13:37:34 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6302lb # [ 200.423181] niks3-server[1024]: 2026/09/20 13:37:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6303lb # [ 200.429907] niks3-server[1024]: 2026/09/20 13:37:34 INFO Signed narinfos id=65 count=16304worker2 # [ 200.423778] nix-grpc-daemon[1802]: time=2026-09-20T13:37:34.985Z level=INFO msg="Uploading 1 narinfos"6305lb # [ 200.433479] niks3-server[1024]: 2026/09/20 13:37:34 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6306lb # [ 200.445367] niks3-server[1024]: 2026/09/20 13:37:35 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6307lb # [ 200.451716] niks3-server[1024]: 2026/09/20 13:37:35 INFO Completed upload id=656308worker2 # [ 200.443671] nix-grpc-daemon[1802]: time=2026-09-20T13:37:35.006Z level=INFO msg="Upload complete. (324ms)"6309lb # [ 200.454782] niks3-server[1024]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6310worker2 # [ 200.452942] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47006 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16311worker1 # [ 200.487830] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:35Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06312worker2 # [ 200.470265] nix-daemon[845]: reaped child process 1825, status = succeeded6313worker2 # [ 200.508735] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470066314worker2 # [ 200.525594] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470066315worker2 # [ 200.539790] nix-grpc-daemon[1802]: time=2026-09-20T13:37:35.102Z level=INFO msg="All 1 paths already cached"6316client # copying 1 paths to 'lb:50051'...6317client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6318worker2 # [ 200.550405] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470066319worker2 # [ 200.565203] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6320worker2 # [ 200.761311] nix-daemon[845]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6321worker2 # [ 200.779898] nix-daemon[845]: reaped child process 1852, status = succeeded6322worker2 # [ 200.960791] nix-daemon[845]: accepted connection from pid 1854, user nix-grpc-daemon (trusted)6323worker2 # [ 200.976938] nix-daemon[845]: reaped child process 1857, status = succeeded6324lb # [ 200.996331] niks3-server[1024]: 2026/09/20 13:37:35 INFO Received uploads request method=POST path=/api/pending_closures6325worker2 # [ 200.998241] nix-grpc-daemon[1802]: time=2026-09-20T13:37:35.560Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6326worker2 # [ 200.999712] nix-grpc-daemon[1802]: time=2026-09-20T13:37:35.560Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6327lb # [ 201.022631] niks3-server[1024]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6328lb # [ 201.034812] niks3-server[1024]: 2026/09/20 13:37:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6329worker2 # [ 201.033402] nix-grpc-daemon[1802]: time=2026-09-20T13:37:35.596Z level=INFO msg="Uploading 1 narinfos"6330lb # [ 201.045102] niks3-server[1024]: 2026/09/20 13:37:35 INFO Signed narinfos id=66 count=16331lb # [ 201.052185] niks3-server[1024]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6332worker1 # [ 200.493679] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:35Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16333lb # [ 201.064599] niks3-server[1024]: 2026/09/20 13:37:35 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6334worker2 # [ 201.063513] nix-grpc-daemon[1802]: time=2026-09-20T13:37:35.626Z level=INFO msg="Upload complete. (481ms)"6335lb # [ 201.071244] niks3-server[1024]: 2026/09/20 13:37:35 INFO Completed upload id=666336lb # [ 201.075947] niks3-server[1024]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6337worker2 # [ 201.072374] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47006 duration_s=0 paths=1 nar_bytes_in=6326338worker2 # [ 201.086807] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470066339worker2 # [ 201.094179] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6340worker2 # [ 201.098514] nix-daemon[845]: reaped child process 1847, status = succeeded6341client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6342client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6343worker2 # [ 201.111521] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6344client # farm-b> LOG-farm-b-drain6345worker2 # [ 201.938055] nix-daemon[845]: reaped child process 1861, status = succeeded6346worker2 # [ 202.101292] nix-daemon[845]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6347worker2 # [ 202.119489] nix-daemon[845]: reaped child process 1879, status = succeeded6348lb # [ 202.139184] niks3-server[1024]: 2026/09/20 13:37:36 INFO Received uploads request method=POST path=/api/pending_closures6349lb # [ 202.146970] envoy[921]: conn peer=[2001:db8:1::1]:33494 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300796350worker2 # [ 202.148394] nix-grpc-daemon[1802]: time=2026-09-20T13:37:36.710Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6351worker2 # [ 202.149906] nix-grpc-daemon[1802]: time=2026-09-20T13:37:36.710Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6352worker2 # [ 202.179182] nix-grpc-daemon[1802]: time=2026-09-20T13:37:36.741Z level=INFO msg="Uploading 1 narinfos"6353lb # [ 202.180158] niks3-server[1024]: 2026/09/20 13:37:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6354lb # [ 202.198848] niks3-server[1024]: 2026/09/20 13:37:36 INFO Signed narinfos id=67 count=16355lb # [ 202.207403] niks3-server[1024]: 2026/09/20 13:37:36 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6356lb # [ 202.215386] niks3-server[1024]: 2026/09/20 13:37:36 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6357worker2 # [ 202.217292] nix-grpc-daemon[1802]: time=2026-09-20T13:37:36.780Z level=INFO msg="Upload complete. (296ms)"6358lb # [ 202.225139] niks3-server[1024]: 2026/09/20 13:37:36 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6359lb # [ 202.229725] niks3-server[1024]: 2026/09/20 13:37:36 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6360lb # [ 202.233863] niks3-server[1024]: 2026/09/20 13:37:36 INFO Completed upload id=676361lb # [ 202.236222] niks3-server[1024]: 2026/09/20 13:37:36 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6362worker2 # [ 202.228906] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47006 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16363worker1 # [ 202.263691] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:36Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06364worker2 # [ 202.242745] nix-daemon[845]: reaped child process 1859, status = succeeded6365worker1 # [ 202.269479] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:36Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16366worker2 # [ 202.281488] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470066367worker2 # [ 202.297458] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470066368worker2 # [ 202.311305] nix-grpc-daemon[1802]: time=2026-09-20T13:37:36.874Z level=INFO msg="All 2 paths already cached"6369client # copying 1 paths to 'lb:50051'...6370client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6371worker2 # [ 202.322297] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470066372worker2 # [ 202.336136] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6373worker2 # [ 202.546742] nix-daemon[845]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6374worker2 # [ 202.567454] nix-daemon[845]: reaped child process 1886, status = succeeded6375worker2 # [ 202.742327] nix-daemon[845]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6376worker2 # [ 202.760248] nix-daemon[845]: reaped child process 1891, status = succeeded6377lb # [ 202.779014] niks3-server[1024]: 2026/09/20 13:37:37 INFO Received uploads request method=POST path=/api/pending_closures6378worker2 # [ 202.781644] nix-grpc-daemon[1802]: time=2026-09-20T13:37:37.344Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6379worker2 # [ 202.783113] nix-grpc-daemon[1802]: time=2026-09-20T13:37:37.344Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6380lb # [ 202.809347] niks3-server[1024]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6381worker2 # [ 202.827279] nix-grpc-daemon[1802]: time=2026-09-20T13:37:37.389Z level=INFO msg="Uploading 1 narinfos"6382lb # [ 202.825929] niks3-server[1024]: 2026/09/20 13:37:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6383lb # [ 202.839866] niks3-server[1024]: 2026/09/20 13:37:37 INFO Signed narinfos id=68 count=16384lb # [ 202.847591] niks3-server[1024]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6385lb # [ 202.857640] niks3-server[1024]: 2026/09/20 13:37:37 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6386worker2 # [ 202.855472] nix-grpc-daemon[1802]: time=2026-09-20T13:37:37.418Z level=INFO msg="Upload complete. (501ms)"6387lb # [ 202.863559] niks3-server[1024]: 2026/09/20 13:37:37 INFO Completed upload id=686388lb # [ 202.867922] niks3-server[1024]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6389worker2 # [ 202.864298] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47006 duration_s=0 paths=1 nar_bytes_in=7606390worker2 # [ 202.872367] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470066391worker2 # [ 202.885962] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6392worker2 # [ 202.889275] nix-daemon[845]: reaped child process 1881, status = succeeded6393client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6394client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6395worker2 # [ 202.903873] nix-daemon[845]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6396client # farm-top> LOG-farm-top-drain6397worker2 # [ 204.593854] nix-daemon[845]: reaped child process 1895, status = succeeded6398worker2 # [ 204.874256] nix-daemon[845]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6399worker2 # [ 204.902495] nix-daemon[845]: reaped child process 1913, status = succeeded6400lb # [ 204.933718] niks3-server[1024]: 2026/09/20 13:37:39 INFO Received uploads request method=POST path=/api/pending_closures6401worker2 # [ 204.941074] nix-grpc-daemon[1802]: time=2026-09-20T13:37:39.502Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6402worker2 # [ 204.942723] nix-grpc-daemon[1802]: time=2026-09-20T13:37:39.502Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6403lb # [ 205.003593] niks3-server[1024]: 2026/09/20 13:37:39 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6404lb # [ 205.013960] niks3-server[1024]: 2026/09/20 13:37:39 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6405lb # [ 205.025906] niks3-server[1024]: 2026/09/20 13:37:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6406worker2 # [ 205.023927] nix-grpc-daemon[1802]: time=2026-09-20T13:37:39.586Z level=INFO msg="Uploading 1 narinfos"6407lb # [ 205.033310] niks3-server[1024]: 2026/09/20 13:37:39 INFO Signed narinfos id=69 count=16408lb # [ 205.041714] niks3-server[1024]: 2026/09/20 13:37:39 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6409lb # [ 205.055963] niks3-server[1024]: 2026/09/20 13:37:39 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6410lb # [ 205.062943] niks3-server[1024]: 2026/09/20 13:37:39 INFO Completed upload id=696411worker2 # [ 205.055257] nix-grpc-daemon[1802]: time=2026-09-20T13:37:39.618Z level=INFO msg="Upload complete. (537ms)"6412lb # [ 205.065824] niks3-server[1024]: 2026/09/20 13:37:39 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6413worker2 # [ 205.070629] nix-grpc-daemon[1746]: ts=2026-09-20T13:37:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47006 duration_s=2 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16414worker2 # [ 205.105208] nix-daemon[845]: reaped child process 1893, status = succeeded6415client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 7.37 seconds)6416client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6417client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.36 seconds)6418worker1: must succeed: rm /nix/.rw-store/fill6419worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.23 seconds)6420worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6421worker1 # [ 206.741280] nix-grpc-daemon[1426]: ts=2026-09-20T13:37:41Z level=info event=healthy reason=min_free6422worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.19 seconds)6423(finished: subtest: low disk drains a worker and builds go to the other, in 10.55 seconds)6424(finished: run the VM test script, in 208.07 seconds)6425test script finished in 208.53s6426cleanup6427kill QemuMachine (pid 50)6428client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6429kill QemuMachine (pid 49)6430lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6431kill QemuMachine (pid 51)6432worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6433kill QemuMachine (pid 52)6434worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6435(finished: cleanup, in 1.20 seconds)6436additionally exposed symbols:6437 client, lb, worker1, worker2,6438 vlan1,6439 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh