vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #298
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb # Formatting '/build/vm-state-lb/tmp.2Sfcn603Ib', fmt=raw size=107374182414lb # mke2fs 1.47.4 (6-Mar-2025)15worker1 # Disk image does not exist, creating the virtualisation disk image...16worker2: QEMU running (pid 52)17lb # Discarding device blocks: 0/262144 done18worker2 # Disk image does not exist, creating the virtualisation disk image...19lb: QEMU running (pid 50)20client # Disk image does not exist, creating the virtualisation disk image...21worker1 # Formatting '/build/vm-state-worker1/tmp.kHMZ3BLnbw', fmt=raw size=107374182422worker2 # Formatting '/build/vm-state-worker2/tmp.BMmmDfIhp5', fmt=raw size=107374182423lb # Creating filesystem with 262144 4k blocks and 65536 inodes24client: QEMU running (pid 49)25worker1: QEMU running (pid 51)26client # Formatting '/build/vm-state-client/tmp.w3b4cUN69L', fmt=raw size=107374182427worker1 # mke2fs 1.47.4 (6-Mar-2025)28lb # Filesystem UUID: f59c81de-1e72-4fc0-a790-d26192d7497d29worker2 # mke2fs 1.47.4 (6-Mar-2025)30client # mke2fs 1.47.4 (6-Mar-2025)31lb # Superblock backups stored on blocks:32worker2 # Discarding device blocks: 0/262144 done33client # Discarding device blocks: 0/262144 done34lb # 32768, 98304, 163840, 22937635worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes36client # Creating filesystem with 262144 4k blocks and 65536 inodes37lb # 38worker2 # Filesystem UUID: bcc8f41c-a7d8-43cd-a7a4-2175576925a039client # Filesystem UUID: 789af849-db24-448d-9161-c1adcfa0e2c740lb # Allocating group tables: 0/8 done41worker2 # Superblock backups stored on blocks:42client # Superblock backups stored on blocks:43lb # Writing inode tables: 0/8 done44worker2 # 32768, 98304, 163840, 22937645client # 32768, 98304, 163840, 22937646lb # Creating journal (8192 blocks): done47worker2 # 48client # 49lb # Writing superblocks and filesystem accounting information: 0/8 done50worker2 # Allocating group tables: 0/8 done51client # Allocating group tables: 0/8 done52lb # 53worker2 # Writing inode tables: 0/8 done54client # Writing inode tables: 0/8 done55lb # Virtualisation disk image created.56worker2 # Creating journal (8192 blocks): done57client # Creating journal (8192 blocks): done58(finished: start all VMs, in 0.47 seconds)59worker1 # Discarding device blocks: 0/262144 done60worker2 # Writing superblocks and filesystem accounting information: 0/8 done61client # Writing superblocks and filesystem accounting information: 0/8 done62client: waiting for unit network-addresses-eth1.service63worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes64worker2 # 65client # 66client: waiting for the VM to finish booting67worker1 # Filesystem UUID: c14f5e8f-e049-43a6-abb5-eeb7712a401168worker2 # Virtualisation disk image created.69worker1 # Superblock backups stored on blocks:70client # Virtualisation disk image created.71worker1 # 32768, 98304, 163840, 22937672worker1 # 73worker1 # Allocating group tables: 0/8 done74worker1 # Writing inode tables: 0/8 done75worker1 # Creating journal (8192 blocks): done76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77worker1 # 78worker1 # Virtualisation disk image created.79worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker2 # [ 0.000000] KASLR enabled82worker2 # [ 0.000000] random: crng init done83worker2 # [ 0.000000] Machine model: linux,dummy-virt84lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]85worker2 # [ 0.000000] efi: UEFI not found.86worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]87worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT88lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202689lb # [ 0.000000] KASLR enabled90worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]91lb # [ 0.000000] random: crng init done92worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202693worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]94worker1 # [ 0.000000] KASLR enabled95lb # [ 0.000000] Machine model: linux,dummy-virt96worker2 # [ 0.000000] Zone ranges:97worker1 # [ 0.000000] random: crng init done98lb # [ 0.000000] efi: UEFI not found.99worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]100worker1 # [ 0.000000] Machine model: linux,dummy-virt101worker2 # [ 0.000000] DMA32 empty102worker1 # [ 0.000000] efi: UEFI not found.103lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT104worker2 # [ 0.000000] Normal empty105worker2 # [ 0.000000] Device empty106worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT107lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]108worker2 # [ 0.000000] Movable zone start for each node109worker2 # [ 0.000000] Early memory node ranges110lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]111worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]112lb # [ 0.000000] Zone ranges:113worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]114worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]115lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]116worker1 # [ 0.000000] Zone ranges:117lb # [ 0.000000] DMA32 empty118worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]119lb # [ 0.000000] Normal empty120worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]121lb # [ 0.000000] Device empty122worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000123worker1 # [ 0.000000] DMA32 empty124worker1 # [ 0.000000] Normal empty125lb # [ 0.000000] Movable zone start for each node126worker2 # [ 0.000000] psci: probing for conduit method from DT.127worker1 # [ 0.000000] Device empty128lb # [ 0.000000] Early memory node ranges129worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.130worker1 # [ 0.000000] Movable zone start for each node131lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]132worker1 # [ 0.000000] Early memory node ranges133worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs134worker2 # [ 0.000000] psci: Trusted OS migration not required135worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]136lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]137worker2 # [ 0.000000] psci: SMC Calling Convention v1.1138lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000139worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]140lb # [ 0.000000] psci: probing for conduit method from DT.141worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000142worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)143lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.144worker1 # [ 0.000000] psci: probing for conduit method from DT.145worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296146lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs147worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.148worker2 # [ 0.000000] Detected PIPT I-cache on CPU0149lb # [ 0.000000] psci: Trusted OS migration not required150worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs151lb # [ 0.000000] psci: SMC Calling Convention v1.1152worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)153worker1 # [ 0.000000] psci: Trusted OS migration not required154worker1 # [ 0.000000] psci: SMC Calling Convention v1.1155worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface156lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)157worker2 # [ 0.000000] CPU features: detected: Spectre-v4158lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296159worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)160worker2 # [ 0.000000] CPU features: detected: Spectre-BHB161lb # [ 0.000000] Detected PIPT I-cache on CPU0162worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296164worker1 # [ 0.000000] Detected PIPT I-cache on CPU0165lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)166worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23167worker2 # [ 0.000000] alternatives: applying boot alternatives168lb # [ 0.000000] CPU features: detected: GICv3 CPU interface169worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)170lb # [ 0.000000] CPU features: detected: Spectre-v4171worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface172lb # [ 0.000000] CPU features: detected: Spectre-BHB173worker1 # [ 0.000000] CPU features: detected: Spectre-v4174lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38175worker1 # [ 0.000000] CPU features: detected: Spectre-BHB176lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23177worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38178lb # [ 0.000000] alternatives: applying boot alternatives179worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23180worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r1ia2y0cy1xis9is5mlx80jsgf16xnyi-nixos-system-worker2-test/init regInfo=/nix/store/5cnh0jbr8izzvkvwcahk9cyrkcpp1wz6-closure-info/registration console=ttyAMA0,115200n8 console=tty0181worker1 # [ 0.000000] alternatives: applying boot alternatives182client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]183worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/5cnh0jbr8izzvkvwcahk9cyrkcpp1wz6-closure-info/registration", will be passed to user space.184lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/14bqwb8zl951hxla1qpmchqzwfgwlqnx-nixos-system-lb-test/init regInfo=/nix/store/hxsww7q411z4rrmrw9s7nvsxa10s5ypn-closure-info/registration console=ttyAMA0,115200n8 console=tty0185worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes186client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026187client # [ 0.000000] KASLR enabled188worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)189worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vwj97rlfz3wvxkwi4x07w9a0jsf1swbn-nixos-system-worker1-test/init regInfo=/nix/store/br0dl2xrji3kkcw2qx931i1wbccmjm7l-closure-info/registration console=ttyAMA0,115200n8 console=tty0190client # [ 0.000000] random: crng init done191client # [ 0.000000] Machine model: linux,dummy-virt192lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/hxsww7q411z4rrmrw9s7nvsxa10s5ypn-closure-info/registration", will be passed to user space.193worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)194client # [ 0.000000] efi: UEFI not found.195lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes196worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB197worker2 # [ 0.000000] software IO TLB: area num 1.198client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT199worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/br0dl2xrji3kkcw2qx931i1wbccmjm7l-closure-info/registration", will be passed to user space.200lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)201client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]202worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB203worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes204client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]205lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)206worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)207client # [ 0.000000] Zone ranges:208worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)209lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB210worker2 # [ 0.000000] Fallback order for Node 0: 0211client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]212lb # [ 0.000000] software IO TLB: area num 1.213client # [ 0.000000] DMA32 empty214worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)215worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216216client # [ 0.000000] Normal empty217lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB218worker2 # [ 0.000000] Policy zone: DMA219client # [ 0.000000] Device empty220worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB221client # [ 0.000000] Movable zone start for each node222worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off223worker1 # [ 0.000000] software IO TLB: area num 1.224lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)225client # [ 0.000000] Early memory node ranges226lb # [ 0.000000] Fallback order for Node 0: 0227worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1228worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB229client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]230worker2 # [ 0.000000] allocated 3145728 bytes of page_ext231lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216232worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)233client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]234worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages235worker1 # [ 0.000000] Fallback order for Node 0: 0236client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000237worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups238lb # [ 0.000000] Policy zone: DMA239client # [ 0.000000] psci: probing for conduit method from DT.240worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216241worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.242lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off243worker1 # [ 0.000000] Policy zone: DMA244worker2 # [ 0.000000] rcu: RCU event tracing is enabled.245client # [ 0.000000] psci: PSCIv1.3 detected in firmware.246lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1247client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs248worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off249worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.250lb # [ 0.000000] allocated 3145728 bytes of page_ext251client # [ 0.000000] psci: Trusted OS migration not required252worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.253worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1254lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages255client # [ 0.000000] psci: SMC Calling Convention v1.1256worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.257worker1 # [ 0.000000] allocated 3145728 bytes of page_ext258lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups259worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.260worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages261client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)262lb # [ 0.000000] rcu: Hierarchical RCU implementation.263worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups264worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.265lb # [ 0.000000] rcu: RCU event tracing is enabled.266client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296267worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.268client # [ 0.000000] Detected PIPT I-cache on CPU0269lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.270worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1271worker1 # [ 0.000000] rcu: RCU event tracing is enabled.272lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.273client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)274worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.275worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.276lb # [ 0.000000] Rude variant of Tasks RCU enabled.277client # [ 0.000000] CPU features: detected: GICv3 CPU interface278worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.279lb # [ 0.000000] Tracing variant of Tasks RCU enabled.280client # [ 0.000000] CPU features: detected: Spectre-v4281worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.282worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.283client # [ 0.000000] CPU features: detected: Spectre-BHB284lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.285worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.286worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.287client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38288lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1289worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.290worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0291client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23292worker2 # [ 0.000000] GICv3: 256 SPIs implemented293worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1294lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.295client # [ 0.000000] alternatives: applying boot alternatives296worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented297worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq298lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.299worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.300worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI301worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0302worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.303lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.304worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000305lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0306worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.307worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]308lb # [ 0.000000] GICv3: 256 SPIs implemented309worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0310lb # [ 0.000000] GICv3: 0 Extended SPIs implemented311client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xhp7hdmnyckpg762n4vd2n1aspzkff7a-nixos-system-client-test/init regInfo=/nix/store/x78jf20gad0ij66h5k5b9a75x5gwxfc4-closure-info/registration console=ttyAMA0,115200n8 console=tty0312worker1 # [ 0.000000] GICv3: 256 SPIs implemented313lb # [ 0.000000] Root IRQ handler: gic_handle_irq314worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)315lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI316worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented317worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq318lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0319client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/x78jf20gad0ij66h5k5b9a75x5gwxfc4-closure-info/registration", will be passed to user space.320worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)321worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI322lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000323worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000324client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes325worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0326lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]327worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000328worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000329client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)330worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]331lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)332worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.333client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)334worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)335worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns336lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)337client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB338client # [ 0.000000] software IO TLB: area num 1.339lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000340worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).341worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)342client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB343lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000344worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000345worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns346client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)347lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.348worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000349client # [ 0.000000] Fallback order for Node 0: 0350worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns351worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.352worker2 # [ 0.000029] arm-pv: using stolen time PV353client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216354lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns355client # [ 0.000000] Policy zone: DMA356lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).357client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off358worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns359worker2 # [ 0.000433] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)360worker2 # [ 0.000622] Console: colour dummy device 80x25361client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1362worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).363lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns364worker2 # [ 0.000630] printk: legacy console [tty0] enabled365client # [ 0.000000] allocated 3145728 bytes of page_ext366client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages367lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns368worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns369worker2 # [ 0.000831] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)370lb # [ 0.000031] arm-pv: using stolen time PV371client # [ 0.000000] ftrace: allocated 294 pages with 4 groups372worker2 # [ 0.000838] pid_max: default: 32768 minimum: 301373worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns374client # [ 0.000000] rcu: Hierarchical RCU implementation.375worker1 # [ 0.000030] arm-pv: using stolen time PV376lb # [ 0.000451] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)377client # [ 0.000000] rcu: RCU event tracing is enabled.378worker2 # [ 0.000924] LSM: initializing lsm=capability,landlock,yama,bpf,ima379worker2 # [ 0.001060] landlock: Up and running.380lb # [ 0.000618] Console: colour dummy device 80x25381client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.382worker2 # [ 0.001063] Yama: becoming mindful.383worker1 # [ 0.000517] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)384lb # [ 0.000625] printk: legacy console [tty0] enabled385worker2 # [ 0.001538] LSM support for eBPF active386client # [ 0.000000] Trampoline variant of Tasks RCU enabled.387worker1 # [ 0.000680] Console: colour dummy device 80x25388client # [ 0.000000] Rude variant of Tasks RCU enabled.389worker1 # [ 0.000688] printk: legacy console [tty0] enabled390worker2 # [ 0.001682] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)391lb # [ 0.000817] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)392client # [ 0.000000] Tracing variant of Tasks RCU enabled.393lb # [ 0.000824] pid_max: default: 32768 minimum: 301394worker2 # [ 0.001715] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)395worker1 # [ 0.000887] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)396client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.397lb # [ 0.000924] LSM: initializing lsm=capability,landlock,yama,bpf,ima398worker2 # [ 0.002819] cacheinfo: Unable to detect cache hierarchy for CPU 0399worker1 # [ 0.000894] pid_max: default: 32768 minimum: 301400lb # [ 0.001068] landlock: Up and running.401client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1402lb # [ 0.001071] Yama: becoming mindful.403worker2 # [ 0.003587] rcu: Hierarchical SRCU implementation.404worker1 # [ 0.000964] LSM: initializing lsm=capability,landlock,yama,bpf,ima405lb # [ 0.001595] LSM support for eBPF active406worker2 # [ 0.003591] rcu: Max phase no-delay instances is 1000.407worker1 # [ 0.001107] landlock: Up and running.408client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.409worker2 # [ 0.004781] fsl-mc MSI: its@8080000 domain created410worker1 # [ 0.001109] Yama: becoming mindful.411lb # [ 0.001740] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)412worker1 # [ 0.001614] LSM support for eBPF active413worker2 # [ 0.004872] EFI services will not be available.414client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.415lb # [ 0.001772] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)416worker2 # [ 0.004937] smp: Bringing up secondary CPUs ...417worker1 # [ 0.001764] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)418worker2 # [ 0.004946] smp: Brought up 1 node, 1 CPU419lb # [ 0.002876] cacheinfo: Unable to detect cache hierarchy for CPU 0420client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.421worker2 # [ 0.004949] SMP: Total of 1 processors activated.422worker1 # [ 0.001797] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)423lb # [ 0.003579] rcu: Hierarchical SRCU implementation.424client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0425worker2 # [ 0.004954] CPU: All CPU(s) started at EL1426client # [ 0.000000] GICv3: 256 SPIs implemented427worker1 # [ 0.002881] cacheinfo: Unable to detect cache hierarchy for CPU 0428lb # [ 0.003583] rcu: Max phase no-delay instances is 1000.429worker2 # [ 0.004966] CPU features: detected: Branch Target Identification430client # [ 0.000000] GICv3: 0 Extended SPIs implemented431worker1 # [ 0.003633] rcu: Hierarchical SRCU implementation.432lb # [ 0.004821] fsl-mc MSI: its@8080000 domain created433client # [ 0.000000] Root IRQ handler: gic_handle_irq434worker1 # [ 0.003638] rcu: Max phase no-delay instances is 1000.435lb # [ 0.004920] EFI services will not be available.436worker2 # [ 0.004971] CPU features: detected: ARMv8.4 Translation Table Level437client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI438lb # [ 0.004990] smp: Bringing up secondary CPUs ...439worker1 # [ 0.004831] fsl-mc MSI: its@8080000 domain created440client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0441lb # [ 0.004997] smp: Brought up 1 node, 1 CPU442worker2 # [ 0.004974] CPU features: detected: Instruction cache invalidation not required for I/D coherence443worker1 # [ 0.004921] EFI services will not be available.444lb # [ 0.005001] SMP: Total of 1 processors activated.445worker1 # [ 0.004986] smp: Bringing up secondary CPUs ...446client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000447lb # [ 0.005006] CPU: All CPU(s) started at EL1448worker2 # [ 0.004977] CPU features: detected: Data cache clean to the PoU not required for I/D coherence449worker1 # [ 0.004994] smp: Brought up 1 node, 1 CPU450client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]451lb # [ 0.005018] CPU features: detected: Branch Target Identification452worker2 # [ 0.004981] CPU features: detected: Common not Private translations453worker1 # [ 0.004997] SMP: Total of 1 processors activated.454worker1 # [ 0.005002] CPU: All CPU(s) started at EL1455worker2 # [ 0.004984] CPU features: detected: CRC32 instructions456client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)457lb # [ 0.005023] CPU features: detected: ARMv8.4 Translation Table Level458worker1 # [ 0.005013] CPU features: detected: Branch Target Identification459worker2 # [ 0.004987] CPU features: detected: Data cache clean to Point of Deep Persistence460client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)461lb # [ 0.005026] CPU features: detected: Instruction cache invalidation not required for I/D coherence462worker1 # [ 0.005019] CPU features: detected: ARMv8.4 Translation Table Level463worker2 # [ 0.004991] CPU features: detected: Data cache clean to Point of Persistence464client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000465lb # [ 0.005030] CPU features: detected: Data cache clean to the PoU not required for I/D coherence466worker1 # [ 0.005022] CPU features: detected: Instruction cache invalidation not required for I/D coherence467worker2 # [ 0.004994] CPU features: detected: Data independent timing control (DIT)468client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000469lb # [ 0.005033] CPU features: detected: Common not Private translations470worker2 # [ 0.004997] CPU features: detected: E0PD471worker1 # [ 0.005025] CPU features: detected: Data cache clean to the PoU not required for I/D coherence472client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.473lb # [ 0.005036] CPU features: detected: CRC32 instructions474worker2 # [ 0.005000] CPU features: detected: Enhanced Counter Virtualization475worker1 # [ 0.005029] CPU features: detected: Common not Private translations476lb # [ 0.005039] CPU features: detected: Data cache clean to Point of Deep Persistence477worker2 # [ 0.005003] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)478worker1 # [ 0.005032] CPU features: detected: CRC32 instructions479client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns480lb # [ 0.005043] CPU features: detected: Data cache clean to Point of Persistence481worker2 # [ 0.005006] CPU features: detected: Enhanced Virtualization Traps482client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).483worker1 # [ 0.005035] CPU features: detected: Data cache clean to Point of Deep Persistence484worker2 # [ 0.005009] CPU features: detected: Fine Grained Traps485lb # [ 0.005046] CPU features: detected: Data independent timing control (DIT)486worker1 # [ 0.005039] CPU features: detected: Data cache clean to Point of Persistence487lb # [ 0.005049] CPU features: detected: E0PD488client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns489worker2 # [ 0.005013] CPU features: detected: Generic authentication (architected QARMA5 algorithm)490lb # [ 0.005052] CPU features: detected: Enhanced Counter Virtualization491worker1 # [ 0.005042] CPU features: detected: Data independent timing control (DIT)492worker2 # [ 0.005018] CPU features: detected: RCpc load-acquire (LDAPR)493client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns494worker1 # [ 0.005045] CPU features: detected: E0PD495lb # [ 0.005055] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)496client # [ 0.000029] arm-pv: using stolen time PV497worker2 # [ 0.005021] CPU features: detected: LSE atomic instructions498worker1 # [ 0.005047] CPU features: detected: Enhanced Counter Virtualization499lb # [ 0.005058] CPU features: detected: Enhanced Virtualization Traps500worker2 # [ 0.005024] CPU features: detected: Privileged Access Never501worker2 # [ 0.005027] CPU features: detected: PMUv3502worker1 # [ 0.005050] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)503lb # [ 0.005061] CPU features: detected: Fine Grained Traps504client # [ 0.000405] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)505client # [ 0.000562] Console: colour dummy device 80x25506worker2 # [ 0.005030] CPU features: detected: RAS Extension Support507worker1 # [ 0.005053] CPU features: detected: Enhanced Virtualization Traps508lb # [ 0.005065] CPU features: detected: Generic authentication (architected QARMA5 algorithm)509client # [ 0.000570] printk: legacy console [tty0] enabled510worker2 # [ 0.005033] CPU features: detected: RASv1p1 Extension Support511worker1 # [ 0.005057] CPU features: detected: Fine Grained Traps512lb # [ 0.005071] CPU features: detected: RCpc load-acquire (LDAPR)513worker2 # [ 0.005035] CPU features: detected: Random Number Generator514client # [ 0.000772] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)515lb # [ 0.005074] CPU features: detected: LSE atomic instructions516worker1 # [ 0.005060] CPU features: detected: Generic authentication (architected QARMA5 algorithm)517worker2 # [ 0.005038] CPU features: detected: Speculation barrier (SB)518client # [ 0.000778] pid_max: default: 32768 minimum: 301519worker1 # [ 0.005065] CPU features: detected: RCpc load-acquire (LDAPR)520lb # [ 0.005077] CPU features: detected: Privileged Access Never521worker2 # [ 0.005041] CPU features: detected: Stage-2 Force Write-Back522lb # [ 0.005080] CPU features: detected: PMUv3523client # [ 0.000847] LSM: initializing lsm=capability,landlock,yama,bpf,ima524worker1 # [ 0.005068] CPU features: detected: LSE atomic instructions525client # [ 0.000969] landlock: Up and running.526worker2 # [ 0.005044] CPU features: detected: TLB range maintenance instructions527lb # [ 0.005082] CPU features: detected: RAS Extension Support528worker1 # [ 0.005071] CPU features: detected: Privileged Access Never529client # [ 0.000971] Yama: becoming mindful.530worker1 # [ 0.005074] CPU features: detected: PMUv3531lb # [ 0.005085] CPU features: detected: RASv1p1 Extension Support532client # [ 0.001416] LSM support for eBPF active533worker2 # [ 0.005049] CPU features: detected: Speculative Store Bypassing Safe (SSBS)534worker1 # [ 0.005077] CPU features: detected: RAS Extension Support535lb # [ 0.005088] CPU features: detected: Random Number Generator536worker2 # [ 0.005089] alternatives: applying system-wide alternatives537client # [ 0.001560] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)538worker1 # [ 0.005079] CPU features: detected: RASv1p1 Extension Support539lb # [ 0.005090] CPU features: detected: Speculation barrier (SB)540worker2 # [ 0.008139] CPU features: detected: BBM Level 2 without TLB conflict abort541client # [ 0.001592] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)542worker1 # [ 0.005082] CPU features: detected: Random Number Generator543lb # [ 0.005093] CPU features: detected: Stage-2 Force Write-Back544client # [ 0.002710] cacheinfo: Unable to detect cache hierarchy for CPU 0545worker1 # [ 0.005084] CPU features: detected: Speculation barrier (SB)546lb # [ 0.005096] CPU features: detected: TLB range maintenance instructions547client # [ 0.003427] rcu: Hierarchical SRCU implementation.548worker2 # [ 0.008295] Memory: 1406512K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)549worker1 # [ 0.005088] CPU features: detected: Stage-2 Force Write-Back550worker2 # [ 0.008675] devtmpfs: initialized551lb # [ 0.005101] CPU features: detected: Speculative Store Bypassing Safe (SSBS)552client # [ 0.003431] rcu: Max phase no-delay instances is 1000.553worker1 # [ 0.005091] CPU features: detected: TLB range maintenance instructions554client # [ 0.004608] fsl-mc MSI: its@8080000 domain created555lb # [ 0.005139] alternatives: applying system-wide alternatives556worker2 # [ 0.010504] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)557client # [ 0.004696] EFI services will not be available.558worker1 # [ 0.005096] CPU features: detected: Speculative Store Bypassing Safe (SSBS)559lb # [ 0.008155] CPU features: detected: BBM Level 2 without TLB conflict abort560worker2 # [ 0.010531] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).561client # [ 0.004813] smp: Bringing up secondary CPUs ...562worker1 # [ 0.005133] alternatives: applying system-wide alternatives563client # [ 0.004821] smp: Brought up 1 node, 1 CPU564worker2 # [ 0.010710] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL565client # [ 0.004824] SMP: Total of 1 processors activated.566worker1 # [ 0.008175] CPU features: detected: BBM Level 2 without TLB conflict abort567lb # [ 0.008381] Memory: 1406572K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)568client # [ 0.004830] CPU: All CPU(s) started at EL1569lb # [ 0.008786] devtmpfs: initialized570client # [ 0.004840] CPU features: detected: Branch Target Identification571lb # [ 0.010604] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)572worker1 # [ 0.008327] Memory: 1406752K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)573client # [ 0.004845] CPU features: detected: ARMv8.4 Translation Table Level574worker1 # [ 0.008750] devtmpfs: initialized575lb # [ 0.010626] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).576worker1 # [ 0.010646] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)577client # [ 0.004849] CPU features: detected: Instruction cache invalidation not required for I/D coherence578lb # [ 0.010832] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL579worker1 # [ 0.010669] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).580client # [ 0.004852] CPU features: detected: Data cache clean to the PoU not required for I/D coherence581client # [ 0.004856] CPU features: detected: Common not Private translations582worker1 # [ 0.010860] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL583client # [ 0.004859] CPU features: detected: CRC32 instructions584client # [ 0.004862] CPU features: detected: Data cache clean to Point of Deep Persistence585client # [ 0.004865] CPU features: detected: Data cache clean to Point of Persistence586client # [ 0.004869] CPU features: detected: Data independent timing control (DIT)587worker2 # [ 0.010715] 0 pages in range for non-PLT usage588client # [ 0.004872] CPU features: detected: E0PD589worker2 # [ 0.010716] 508288 pages in range for PLT usage590client # [ 0.004874] CPU features: detected: Enhanced Counter Virtualization591worker2 # [ 0.010823] pinctrl core: initialized pinctrl subsystem592client # [ 0.004877] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)593worker2 # [ 0.011590] DMI not present or invalid.594worker2 # [ 0.015086] NET: Registered PF_NETLINK/PF_ROUTE protocol family595client # [ 0.004880] CPU features: detected: Enhanced Virtualization Traps596client # [ 0.004883] CPU features: detected: Fine Grained Traps597worker2 # [ 0.017328] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations598lb # [ 0.010837] 0 pages in range for non-PLT usage599client # [ 0.004887] CPU features: detected: Generic authentication (architected QARMA5 algorithm)600worker2 # [ 0.017518] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations601lb # [ 0.010838] 508288 pages in range for PLT usage602client # [ 0.004893] CPU features: detected: RCpc load-acquire (LDAPR)603lb # [ 0.010956] pinctrl core: initialized pinctrl subsystem604worker1 # [ 0.010865] 0 pages in range for non-PLT usage605worker2 # [ 0.017726] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations606lb # [ 0.011761] DMI not present or invalid.607client # [ 0.004896] CPU features: detected: LSE atomic instructions608worker1 # [ 0.010866] 508288 pages in range for PLT usage609worker2 # [ 0.017749] audit: initializing netlink subsys (disabled)610lb # [ 0.014974] NET: Registered PF_NETLINK/PF_ROUTE protocol family611client # [ 0.004899] CPU features: detected: Privileged Access Never612worker1 # [ 0.010967] pinctrl core: initialized pinctrl subsystem613worker2 # [ 0.018300] thermal_sys: Registered thermal governor 'fair_share'614client # [ 0.004902] CPU features: detected: PMUv3615worker1 # [ 0.011778] DMI not present or invalid.616lb # [ 0.017333] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations617worker2 # [ 0.018302] thermal_sys: Registered thermal governor 'bang_bang'618client # [ 0.004904] CPU features: detected: RAS Extension Support619worker1 # [ 0.015093] NET: Registered PF_NETLINK/PF_ROUTE protocol family620lb # [ 0.017530] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations621client # [ 0.004907] CPU features: detected: RASv1p1 Extension Support622worker2 # [ 0.018306] thermal_sys: Registered thermal governor 'step_wise'623worker1 # [ 0.017313] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations624client # [ 0.004910] CPU features: detected: Random Number Generator625worker2 # [ 0.018312] thermal_sys: Registered thermal governor 'user_space'626lb # [ 0.017729] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations627client # [ 0.004912] CPU features: detected: Speculation barrier (SB)628worker1 # [ 0.017492] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations629lb # [ 0.017752] audit: initializing netlink subsys (disabled)630worker2 # [ 0.018315] thermal_sys: Registered thermal governor 'power_allocator'631client # [ 0.004915] CPU features: detected: Stage-2 Force Write-Back632lb # [ 0.018274] thermal_sys: Registered thermal governor 'fair_share'633worker1 # [ 0.017693] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations634worker2 # [ 0.018340] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1635client # [ 0.004918] CPU features: detected: TLB range maintenance instructions636worker1 # [ 0.017716] audit: initializing netlink subsys (disabled)637lb # [ 0.018276] thermal_sys: Registered thermal governor 'bang_bang'638worker2 # [ 0.018348] cpuidle: using governor ladder639worker2 # [ 0.018355] cpuidle: using governor menu640client # [ 0.004923] CPU features: detected: Speculative Store Bypassing Safe (SSBS)641worker1 # [ 0.018317] thermal_sys: Registered thermal governor 'fair_share'642lb # [ 0.018280] thermal_sys: Registered thermal governor 'step_wise'643worker2 # [ 0.018543] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.644client # [ 0.004959] alternatives: applying system-wide alternatives645worker1 # [ 0.018319] thermal_sys: Registered thermal governor 'bang_bang'646lb # [ 0.018285] thermal_sys: Registered thermal governor 'user_space'647worker2 # [ 0.018560] ASID allocator initialised with 65536 entries648worker1 # [ 0.018323] thermal_sys: Registered thermal governor 'step_wise'649client # [ 0.007879] CPU features: detected: BBM Level 2 without TLB conflict abort650lb # [ 0.018288] thermal_sys: Registered thermal governor 'power_allocator'651worker2 # [ 0.019754] Serial: AMBA PL011 UART driver652worker1 # [ 0.018328] thermal_sys: Registered thermal governor 'user_space'653lb # [ 0.018325] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1654worker2 # [ 0.024887] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1655lb # [ 0.018334] cpuidle: using governor ladder656worker1 # [ 0.018331] thermal_sys: Registered thermal governor 'power_allocator'657client # [ 0.008090] Memory: 1406544K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)658worker2 # [ 0.025048] printk: console [ttyAMA0] enabled659lb # [ 0.018340] cpuidle: using governor menu660client # [ 0.008462] devtmpfs: initialized661worker1 # [ 0.018363] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1662lb # [ 0.018539] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.663client # [ 0.010234] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)664worker1 # [ 0.018371] cpuidle: using governor ladder665worker2 # [ 0.144651] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages666worker1 # [ 0.018377] cpuidle: using governor menu667lb # [ 0.018555] ASID allocator initialised with 65536 entries668worker2 # [ 0.144672] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page669lb # [ 0.019739] Serial: AMBA PL011 UART driver670client # [ 0.010256] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).671worker1 # [ 0.018592] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.672client # [ 0.010469] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL673worker2 # [ 0.144677] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages674worker1 # [ 0.018608] ASID allocator initialised with 65536 entries675lb # [ 0.025036] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1676client # [ 0.010473] 0 pages in range for non-PLT usage677worker2 # [ 0.144681] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page678lb # [ 0.025190] printk: console [ttyAMA0] enabled679worker1 # [ 0.019820] Serial: AMBA PL011 UART driver680client # [ 0.010474] 508288 pages in range for PLT usage681worker2 # [ 0.144686] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages682client # [ 0.010563] pinctrl core: initialized pinctrl subsystem683worker1 # [ 0.024990] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1684client # [ 0.011287] DMI not present or invalid.685lb # [ 0.145321] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages686worker2 # [ 0.144690] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page687worker1 # [ 0.025112] printk: console [ttyAMA0] enabled688client # [ 0.014482] NET: Registered PF_NETLINK/PF_ROUTE protocol family689lb # [ 0.145341] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page690worker2 # [ 0.144694] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages691worker1 # [ 0.145755] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages692client # [ 0.016648] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations693lb # [ 0.145346] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages694worker2 # [ 0.144698] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page695worker1 # [ 0.145773] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page696lb # [ 0.145350] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page697client # [ 0.016817] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations698worker1 # [ 0.145779] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages699lb # [ 0.145355] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages700client # [ 0.017009] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations701worker1 # [ 0.145783] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page702lb # [ 0.145359] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page703client # [ 0.017031] audit: initializing netlink subsys (disabled)704worker2 # [ 0.151815] fbcon: Taking over console705worker2 # [ 0.151829] ACPI: Interpreter disabled.706worker1 # [ 0.145788] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages707lb # [ 0.145363] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages708client # [ 0.017564] thermal_sys: Registered thermal governor 'fair_share'709worker2 # [ 0.153625] iommu: Default domain type: Translated710worker1 # [ 0.145792] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page711lb # [ 0.145367] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page712client # [ 0.017566] thermal_sys: Registered thermal governor 'bang_bang'713worker2 # [ 0.153634] iommu: DMA domain TLB invalidation policy: strict mode714worker1 # [ 0.145796] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages715client # [ 0.017569] thermal_sys: Registered thermal governor 'step_wise'716worker2 # [ 0.155292] SCSI subsystem initialized717worker1 # [ 0.145800] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page718client # [ 0.017574] thermal_sys: Registered thermal governor 'user_space'719client # [ 0.017577] thermal_sys: Registered thermal governor 'power_allocator'720lb # [ 0.152638] fbcon: Taking over console721lb # [ 0.152653] ACPI: Interpreter disabled.722client # [ 0.017599] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1723lb # [ 0.154501] iommu: Default domain type: Translated724client # [ 0.017609] cpuidle: using governor ladder725worker1 # [ 0.153032] fbcon: Taking over console726client # [ 0.017614] cpuidle: using governor menu727worker1 # [ 0.153045] ACPI: Interpreter disabled.728lb # [ 0.154510] iommu: DMA domain TLB invalidation policy: strict mode729lb # [ 0.156219] SCSI subsystem initialized730client # [ 0.017815] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.731client # [ 0.017829] ASID allocator initialised with 65536 entries732client # [ 0.018945] Serial: AMBA PL011 UART driver733client # [ 0.023993] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1734client # [ 0.024142] printk: console [ttyAMA0] enabled735worker2 # [ 0.162838] usbcore: registered new interface driver usbfs736worker2 # [ 0.162871] usbcore: registered new interface driver hub737worker1 # [ 0.160318] iommu: Default domain type: Translated738worker2 # [ 0.162889] usbcore: registered new device driver usb739worker1 # [ 0.160328] iommu: DMA domain TLB invalidation policy: strict mode740worker2 # [ 0.163188] pps_core: LinuxPPS API ver. 1 registered741worker1 # [ 0.162046] SCSI subsystem initialized742worker2 # [ 0.163194] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>743worker2 # [ 0.163203] PTP clock support registered744worker1 # [ 0.162737] usbcore: registered new interface driver usbfs745worker2 # [ 0.163261] EDAC MC: Ver: 3.0.0746lb # [ 0.163807] usbcore: registered new interface driver usbfs747worker1 # [ 0.162769] usbcore: registered new interface driver hub748lb # [ 0.163840] usbcore: registered new interface driver hub749worker2 # [ 0.167854] scmi_core: SCMI protocol bus registered750worker1 # [ 0.162788] usbcore: registered new device driver usb751lb # [ 0.163857] usbcore: registered new device driver usb752worker1 # [ 0.163059] pps_core: LinuxPPS API ver. 1 registered753worker2 # [ 0.168808] FPGA manager framework754lb # [ 0.164178] pps_core: LinuxPPS API ver. 1 registered755worker2 # [ 0.169773] vgaarb: loaded756worker1 # [ 0.163065] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>757lb # [ 0.164184] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>758worker1 # [ 0.163084] PTP clock support registered759lb # [ 0.164193] PTP clock support registered760worker1 # [ 0.163137] EDAC MC: Ver: 3.0.0761lb # [ 0.164233] EDAC MC: Ver: 3.0.0762worker2 # [ 0.170399] clocksource: Switched to clocksource arch_sys_counter763client # [ 0.144094] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages764worker2 # [ 0.171032] VFS: Disk quotas dquot_6.6.0765worker1 # [ 0.167752] scmi_core: SCMI protocol bus registered766lb # [ 0.168873] scmi_core: SCMI protocol bus registered767client # [ 0.144109] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page768worker2 # [ 0.171059] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)769worker1 # [ 0.168712] FPGA manager framework770lb # [ 0.169803] FPGA manager framework771client # [ 0.144114] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages772worker2 # [ 0.173348] netfs: FS-Cache loaded773worker1 # [ 0.169673] vgaarb: loaded774lb # [ 0.170765] vgaarb: loaded775worker2 # [ 0.173443] pnp: PnP ACPI: disabled776client # [ 0.144119] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page777worker1 # [ 0.170287] clocksource: Switched to clocksource arch_sys_counter778client # [ 0.144123] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages779lb # [ 0.171393] clocksource: Switched to clocksource arch_sys_counter780client # [ 0.144128] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page781lb # [ 0.172015] VFS: Disk quotas dquot_6.6.0782client # [ 0.144132] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages783lb # [ 0.172040] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)784client # [ 0.144136] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page785worker1 # [ 0.173233] VFS: Disk quotas dquot_6.6.0786lb # [ 0.174414] netfs: FS-Cache loaded787lb # [ 0.174517] pnp: PnP ACPI: disabled788worker1 # [ 0.173263] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)789client # [ 0.151432] fbcon: Taking over console790client # [ 0.151447] ACPI: Interpreter disabled.791client # [ 0.153343] iommu: Default domain type: Translated792worker1 # [ 0.176939] netfs: FS-Cache loaded793worker1 # [ 0.177060] pnp: PnP ACPI: disabled794client # [ 0.153352] iommu: DMA domain TLB invalidation policy: strict mode795client # [ 0.155036] SCSI subsystem initialized796worker2 # [ 0.179227] NET: Registered PF_INET protocol family797worker2 # [ 0.179486] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)798worker1 # [ 0.180903] NET: Registered PF_INET protocol family799lb # [ 0.180338] NET: Registered PF_INET protocol family800worker1 # [ 0.181152] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)801lb # [ 0.180608] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)802client # [ 0.160353] usbcore: registered new interface driver usbfs803client # [ 0.160384] usbcore: registered new interface driver hub804client # [ 0.160401] usbcore: registered new device driver usb805client # [ 0.160678] pps_core: LinuxPPS API ver. 1 registered806client # [ 0.160684] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>807client # [ 0.160694] PTP clock support registered808client # [ 0.160741] EDAC MC: Ver: 3.0.0809client # [ 0.165377] scmi_core: SCMI protocol bus registered810client # [ 0.166365] FPGA manager framework811client # [ 0.167287] vgaarb: loaded812client # [ 0.167956] clocksource: Switched to clocksource arch_sys_counter813client # [ 0.171842] VFS: Disk quotas dquot_6.6.0814client # [ 0.171884] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)815client # [ 0.175530] netfs: FS-Cache loaded816client # [ 0.175650] pnp: PnP ACPI: disabled817client # [ 0.179508] NET: Registered PF_INET protocol family818client # [ 0.179788] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)819worker2 # [ 0.208696] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)820worker2 # [ 0.208746] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)821worker2 # [ 0.208773] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)822worker2 # [ 0.208849] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)823worker2 # [ 0.208992] TCP: Hash tables configured (established 16384 bind 16384)824worker2 # [ 0.209076] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)825worker2 # [ 0.209113] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)826worker2 # [ 0.209149] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)827worker2 # [ 0.209252] NET: Registered PF_UNIX/PF_LOCAL protocol family828worker2 # [ 0.209275] NET: Registered PF_XDP protocol family829worker1 # [ 0.210708] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)830worker2 # [ 0.209293] PCI: CLS 0 bytes, default 64831lb # [ 0.210037] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)832worker2 # [ 0.209540] Trying to unpack rootfs image as initramfs...833worker1 # [ 0.210766] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834lb # [ 0.210089] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)835worker1 # [ 0.210800] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)836lb # [ 0.210119] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)837worker1 # [ 0.210887] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)838lb # [ 0.210199] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)839worker1 # [ 0.211044] TCP: Hash tables configured (established 16384 bind 16384)840lb # [ 0.210345] TCP: Hash tables configured (established 16384 bind 16384)841worker1 # [ 0.211129] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)842lb # [ 0.210432] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)843worker1 # [ 0.211164] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)844lb # [ 0.210469] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)845worker1 # [ 0.211201] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)846lb # [ 0.210505] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)847worker1 # [ 0.211299] NET: Registered PF_UNIX/PF_LOCAL protocol family848lb # [ 0.210601] NET: Registered PF_UNIX/PF_LOCAL protocol family849worker1 # [ 0.211342] NET: Registered PF_XDP protocol family850lb # [ 0.210624] NET: Registered PF_XDP protocol family851worker2 # [ 0.224094] kvm [1]: HYP mode not available852worker1 # [ 0.211361] PCI: CLS 0 bytes, default 64853lb # [ 0.210644] PCI: CLS 0 bytes, default 64854worker1 # [ 0.211619] Trying to unpack rootfs image as initramfs...855lb # [ 0.210896] Trying to unpack rootfs image as initramfs...856worker1 # [ 0.228112] kvm [1]: HYP mode not available857lb # [ 0.229359] kvm [1]: HYP mode not available858client # [ 0.209564] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)859client # [ 0.209618] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)860client # [ 0.209651] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)861client # [ 0.209732] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)862client # [ 0.209875] TCP: Hash tables configured (established 16384 bind 16384)863client # [ 0.209962] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)864client # [ 0.209998] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865client # [ 0.210034] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866client # [ 0.210128] NET: Registered PF_UNIX/PF_LOCAL protocol family867client # [ 0.210156] NET: Registered PF_XDP protocol family868client # [ 0.210175] PCI: CLS 0 bytes, default 64869client # [ 0.210437] Trying to unpack rootfs image as initramfs...870client # [ 0.225818] kvm [1]: HYP mode not available871worker2 # [ 0.320236] Initialise system trusted keyrings872worker2 # [ 0.321025] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker2 # [ 0.322283] squashfs: version 4.0 (2009/01/31) Phillip Lougher874lb # [ 0.323959] Initialise system trusted keyrings875lb # [ 0.324747] workingset: timestamp_bits=42 max_order=19 bucket_order=0876lb # [ 0.326021] squashfs: version 4.0 (2009/01/31) Phillip Lougher877lb # [ 0.326791] 9p: Installing v9fs 9p2000 file system support878worker2 # [ 0.323122] 9p: Installing v9fs 9p2000 file system support879worker1 # [ 0.326910] Initialise system trusted keyrings880worker1 # [ 0.327664] workingset: timestamp_bits=42 max_order=19 bucket_order=0881worker1 # [ 0.328992] squashfs: version 4.0 (2009/01/31) Phillip Lougher882worker1 # [ 0.329802] 9p: Installing v9fs 9p2000 file system support883worker2 # [ 0.343063] Key type asymmetric registered884worker2 # [ 0.343088] Asymmetric key parser 'x509' registered885worker2 # [ 0.343168] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)886client # [ 0.320520] Initialise system trusted keyrings887client # [ 0.321311] workingset: timestamp_bits=42 max_order=19 bucket_order=0888client # [ 0.322553] squashfs: version 4.0 (2009/01/31) Phillip Lougher889worker2 # [ 0.350469] io scheduler mq-deadline registered890worker2 # [ 0.350489] io scheduler kyber registered891client # [ 0.323311] 9p: Installing v9fs 9p2000 file system support892lb # [ 0.355483] Key type asymmetric registered893lb # [ 0.355514] Asymmetric key parser 'x509' registered894worker2 # [ 0.358559] pl061_gpio 9030000.pl061: PL061 GPIO chip registered895lb # [ 0.355590] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)896lb # [ 0.357783] io scheduler mq-deadline registered897lb # [ 0.357793] io scheduler kyber registered898worker2 # [ 0.359915] ledtrig-cpu: registered to indicate activity on CPUs899worker1 # [ 0.358614] Key type asymmetric registered900worker1 # [ 0.358644] Asymmetric key parser 'x509' registered901worker2 # [ 0.360295] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:902worker1 # [ 0.358717] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)903worker2 # [ 0.360311] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000904worker1 # [ 0.360941] io scheduler mq-deadline registered905worker1 # [ 0.360952] io scheduler kyber registered906worker2 # [ 0.360331] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000907worker2 # [ 0.360340] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000908worker2 # [ 0.360360] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits909worker2 # [ 0.360383] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]910worker2 # [ 0.360462] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00911worker2 # [ 0.360471] pci_bus 0000:00: root bus resource [bus 00-ff]912worker2 # [ 0.360477] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]913lb # [ 0.367515] pl061_gpio 9030000.pl061: PL061 GPIO chip registered914worker2 # [ 0.360481] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]915worker2 # [ 0.360487] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]916worker2 # [ 0.360581] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint917lb # [ 0.368929] ledtrig-cpu: registered to indicate activity on CPUs918worker2 # [ 0.361010] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint919worker1 # [ 0.370420] pl061_gpio 9030000.pl061: PL061 GPIO chip registered920lb # [ 0.369312] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:921worker2 # [ 0.361196] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]922worker2 # [ 0.361213] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]923lb # [ 0.369329] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000924worker2 # [ 0.361242] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]925lb # [ 0.369356] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000926worker2 # [ 0.361259] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]927worker1 # [ 0.371829] ledtrig-cpu: registered to indicate activity on CPUs928lb # [ 0.369364] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000929worker2 # [ 0.361708] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint930worker1 # [ 0.372226] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:931client # [ 0.352012] Key type asymmetric registered932worker2 # [ 0.361891] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]933lb # [ 0.369383] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits934client # [ 0.352039] Asymmetric key parser 'x509' registered935worker2 # [ 0.361908] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]936worker1 # [ 0.372244] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000937client # [ 0.352120] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)938lb # [ 0.369407] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]939worker2 # [ 0.361937] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]940worker1 # [ 0.372256] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000941client # [ 0.354168] io scheduler mq-deadline registered942lb # [ 0.369486] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00943worker2 # [ 0.362375] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint944client # [ 0.354179] io scheduler kyber registered945lb # [ 0.369495] pci_bus 0000:00: root bus resource [bus 00-ff]946worker1 # [ 0.372265] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000947lb # [ 0.369500] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]948worker1 # [ 0.372287] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits949lb # [ 0.369505] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]950worker1 # [ 0.372312] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]951lb # [ 0.369511] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]952worker1 # [ 0.372394] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00953lb # [ 0.369570] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint954worker1 # [ 0.372404] pci_bus 0000:00: root bus resource [bus 00-ff]955lb # [ 0.370016] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint956worker1 # [ 0.372410] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]957lb # [ 0.370205] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]958worker1 # [ 0.372416] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]959lb # [ 0.370221] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]960worker1 # [ 0.372421] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]961lb # [ 0.370251] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]962lb # [ 0.370266] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]963worker1 # [ 0.372479] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint964lb # [ 0.370723] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint965worker1 # [ 0.372927] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint966lb # [ 0.370908] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]967worker1 # [ 0.373121] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]968client # [ 0.364077] pl061_gpio 9030000.pl061: PL061 GPIO chip registered969lb # [ 0.370923] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]970worker1 # [ 0.373138] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]971lb # [ 0.370952] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]972worker1 # [ 0.373169] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]973worker1 # [ 0.373185] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]974lb # [ 0.390647] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint975worker1 # [ 0.373651] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint976client # [ 0.365423] ledtrig-cpu: registered to indicate activity on CPUs977worker1 # [ 0.373837] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]978lb # [ 0.390834] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]979lb # [ 0.390850] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]980worker1 # [ 0.373854] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]981client # [ 0.365817] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:982lb # [ 0.390879] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]983worker1 # [ 0.373884] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]984client # [ 0.365834] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000985lb # [ 0.391340] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint986client # [ 0.365845] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000987worker2 # [ 0.390630] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]988worker1 # [ 0.393687] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint989worker1 # [ 0.393879] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]990worker2 # [ 0.390663] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]991client # [ 0.365853] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000992worker1 # [ 0.393896] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]993worker2 # [ 0.390693] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994client # [ 0.365874] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits995worker1 # [ 0.393929] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996worker2 # [ 0.391213] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint997client # [ 0.365897] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]998worker2 # [ 0.391397] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]999worker2 # [ 0.391413] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1000client # [ 0.365969] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001001client # [ 0.365979] pci_bus 0000:00: root bus resource [bus 00-ff]1002worker2 # [ 0.391446] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1003client # [ 0.365984] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1004worker2 # [ 0.391911] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1005client # [ 0.365990] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1006worker2 # [ 0.392094] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1007worker2 # [ 0.392110] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1008client # [ 0.365995] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1009worker2 # [ 0.392139] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1010client # [ 0.366049] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1011worker2 # [ 0.392597] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1012client # [ 0.366478] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1013worker2 # [ 0.392780] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1014client # [ 0.366664] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1015worker2 # [ 0.392796] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1016client # [ 0.366680] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1017worker2 # [ 0.392825] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1018client # [ 0.366709] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019lb # [ 0.399564] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1020client # [ 0.366725] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1021worker2 # [ 0.393274] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1022worker1 # [ 0.402458] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1023lb # [ 0.399589] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1024worker2 # [ 0.393457] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1025worker1 # [ 0.402659] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1026client # [ 0.367178] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1027lb # [ 0.399622] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1028worker2 # [ 0.393473] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1029worker1 # [ 0.402676] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1030client # [ 0.367361] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1031worker2 # [ 0.393503] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1032worker1 # [ 0.402709] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033client # [ 0.367376] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1034lb # [ 0.400105] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1035worker2 # [ 0.393519] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1036lb # [ 0.400290] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1037client # [ 0.367406] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1038worker1 # [ 0.403195] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1039lb # [ 0.400306] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1040worker2 # [ 0.394012] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1041worker1 # [ 0.403383] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1042client # [ 0.367877] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1043worker2 # [ 0.394200] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1044lb # [ 0.400336] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1045worker1 # [ 0.403399] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1046client # [ 0.387948] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1047worker2 # [ 0.394230] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1048worker1 # [ 0.403430] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049lb # [ 0.400783] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1050lb # [ 0.400965] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1051worker2 # [ 0.394703] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1052worker1 # [ 0.403892] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1053lb # [ 0.400980] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1054worker2 # [ 0.394895] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1055worker1 # [ 0.404081] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1056lb # [ 0.401010] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057worker2 # [ 0.394924] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1058worker1 # [ 0.404097] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1059worker1 # [ 0.404128] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1060worker2 # [ 0.395314] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1061lb # [ 0.401465] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1062client # [ 0.388577] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1063lb # [ 0.401648] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1064worker2 # [ 0.395504] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1065worker1 # [ 0.404620] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1066client # [ 0.388610] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067lb # [ 0.401663] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1068worker1 # [ 0.404809] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1069worker2 # [ 0.395751] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1070client # [ 0.389070] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1071lb # [ 0.401693] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1072worker2 # [ 0.396019] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1073worker1 # [ 0.404825] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1074client # [ 0.389254] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1075lb # [ 0.401709] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1076worker1 # [ 0.404855] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077worker2 # [ 0.396036] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1078client # [ 0.389269] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1079worker2 # [ 0.396065] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1080lb # [ 0.402165] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1081worker1 # [ 0.404871] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1082client # [ 0.389301] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1083lb # [ 0.402349] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1084worker2 # [ 0.396532] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1085worker1 # [ 0.405333] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1086client # [ 0.389753] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1087worker2 # [ 0.396718] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1088lb # [ 0.402379] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089worker1 # [ 0.405527] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1090client # [ 0.389935] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1091worker2 # [ 0.396734] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1092lb # [ 0.402829] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1093worker1 # [ 0.405557] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094client # [ 0.389950] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1095worker2 # [ 0.396764] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1096lb # [ 0.403015] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1097client # [ 0.389979] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098worker1 # [ 0.406020] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1099worker2 # [ 0.397334] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1100lb # [ 0.403045] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101worker1 # [ 0.406212] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1102client # [ 0.390426] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1103worker2 # [ 0.397346] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1104client # [ 0.390609] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1105lb # [ 0.403446] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1106worker1 # [ 0.406243] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1107worker2 # [ 0.397351] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1108lb # [ 0.403627] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1109client # [ 0.390624] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1110client # [ 0.390653] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111worker2 # [ 0.397396] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1112lb # [ 0.403873] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1113lb # [ 0.404147] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1114worker2 # [ 0.397443] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1115client # [ 0.391101] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1116worker1 # [ 0.424976] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1117lb # [ 0.404164] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1118client # [ 0.391286] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1119worker2 # [ 0.397489] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1120worker1 # [ 0.425163] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1121client # [ 0.391302] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1122lb # [ 0.404193] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123worker2 # [ 0.397536] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1124worker1 # [ 0.425408] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1125client # [ 0.391331] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1126lb # [ 0.404666] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1127worker1 # [ 0.425695] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1128worker2 # [ 0.397582] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1129client # [ 0.391348] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1130lb # [ 0.404849] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1131worker1 # [ 0.425712] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1132worker2 # [ 0.397629] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1133lb # [ 0.404864] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1134client # [ 0.391816] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1135worker1 # [ 0.425742] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1136lb # [ 0.404893] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137worker2 # [ 0.397675] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1138worker1 # [ 0.426212] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1139lb # [ 0.405454] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1140worker2 # [ 0.397722] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1141lb # [ 0.405465] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1142worker2 # [ 0.397770] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1143lb # [ 0.405471] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1144worker2 # [ 0.397843] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1145lb # [ 0.405515] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1146worker2 # [ 0.397890] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1147lb # [ 0.405561] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1148worker2 # [ 0.397911] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1149worker2 # [ 0.397932] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1150lb # [ 0.405607] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1151worker2 # [ 0.397954] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1152lb # [ 0.405653] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1153worker2 # [ 0.397978] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1154lb # [ 0.405698] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1155worker2 # [ 0.398000] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1156lb # [ 0.405744] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1157client # [ 0.412032] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1158worker2 # [ 0.398022] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1159client # [ 0.412071] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1160lb # [ 0.405790] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1161worker2 # [ 0.398043] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1162worker2 # [ 0.398065] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1163client # [ 0.412547] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1164lb # [ 0.405836] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1165client # [ 0.412739] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1166worker2 # [ 0.398088] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1167lb # [ 0.405881] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1168worker2 # [ 0.398111] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1169client # [ 0.412768] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1170lb # [ 0.405942] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1171worker1 # [ 0.438447] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1172worker2 # [ 0.398134] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1173client # [ 0.413167] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1174worker1 # [ 0.438476] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1175lb # [ 0.405988] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1176worker2 # [ 0.398155] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1177client # [ 0.413349] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1178worker1 # [ 0.438508] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1179lb # [ 0.406009] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1180worker2 # [ 0.398176] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1181client # [ 0.413593] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1182worker1 # [ 0.439173] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1183worker2 # [ 0.398197] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1184lb # [ 0.406030] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1185client # [ 0.413851] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1186worker2 # [ 0.398218] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1187lb # [ 0.406051] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1188worker1 # [ 0.439186] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1189client # [ 0.413868] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1190worker2 # [ 0.398240] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1191lb # [ 0.406075] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1192worker1 # [ 0.439192] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1193client # [ 0.413897] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1194worker2 # [ 0.398261] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1195lb # [ 0.406097] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1196worker1 # [ 0.439239] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1197worker2 # [ 0.398282] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1198client # [ 0.414350] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1199lb # [ 0.406118] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1200worker2 # [ 0.398304] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1201client # [ 0.414534] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1202worker1 # [ 0.439288] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1203lb # [ 0.406140] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1204client # [ 0.414549] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1205worker2 # [ 0.398326] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1206worker1 # [ 0.439337] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1207lb # [ 0.406161] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1208client # [ 0.414578] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1209worker2 # [ 0.398352] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1210lb # [ 0.406183] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1211worker2 # [ 0.398362] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1212worker1 # [ 0.439391] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1213client # [ 0.415147] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1214worker2 # [ 0.398367] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1215lb # [ 0.406205] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1216client # [ 0.415159] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1217worker1 # [ 0.439442] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1218lb # [ 0.406227] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1219client # [ 0.415164] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1220worker1 # [ 0.439492] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1221lb # [ 0.406248] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1222client # [ 0.415208] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1223worker1 # [ 0.439541] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1224lb # [ 0.406270] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1225lb # [ 0.406291] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1226client # [ 0.415255] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1227worker1 # [ 0.439590] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1228lb # [ 0.406312] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1229client # [ 0.415302] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1230worker1 # [ 0.439641] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1231lb # [ 0.406333] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1232client # [ 0.415349] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1233lb # [ 0.406354] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1234worker1 # [ 0.439752] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1235lb # [ 0.406376] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1236client # [ 0.415395] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1237worker1 # [ 0.439801] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1238lb # [ 0.406397] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1239worker1 # [ 0.439824] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1240client # [ 0.415443] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1241lb # [ 0.406419] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1242worker1 # [ 0.439847] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1243client # [ 0.415490] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1244lb # [ 0.406444] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1245worker1 # [ 0.439870] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1246lb # [ 0.406453] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1247client # [ 0.415537] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1248worker1 # [ 0.439896] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1249lb # [ 0.406458] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1250worker2 # [ 0.463618] pci 0000:00:0a.0: enabling device (0000 -> 0002)1251client # [ 0.415584] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1252worker1 # [ 0.439919] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1253client # [ 0.415644] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1254worker1 # [ 0.439942] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1255client # [ 0.415702] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1256worker1 # [ 0.439965] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1257client # [ 0.415724] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1258worker1 # [ 0.439988] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1259client # [ 0.415745] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1260worker1 # [ 0.440011] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1261client # [ 0.415766] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1262worker1 # [ 0.440035] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1263client # [ 0.415790] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1264worker1 # [ 0.440058] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1265client # [ 0.415812] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1266worker1 # [ 0.440081] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1267worker1 # [ 0.440103] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1268client # [ 0.415833] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1269lb # [ 0.467659] pci 0000:00:0a.0: enabling device (0000 -> 0002)1270worker1 # [ 0.440125] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1271client # [ 0.415855] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1272worker1 # [ 0.440147] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1273client # [ 0.415877] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1274worker1 # [ 0.440169] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1275client # [ 0.415899] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1276worker1 # [ 0.440192] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1277client # [ 0.415921] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1278worker1 # [ 0.440214] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1279client # [ 0.415943] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1280worker1 # [ 0.440237] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1281worker1 # [ 0.440260] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1282worker1 # [ 0.440288] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1283worker1 # [ 0.440298] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1284worker1 # [ 0.440302] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1285worker1 # [ 0.441500] pci 0000:00:0a.0: enabling device (0000 -> 0002)1286client # [ 0.451986] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1287client # [ 0.452016] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1288client # [ 0.452037] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1289client # [ 0.452059] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1290client # [ 0.452081] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1291client # [ 0.452103] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1292client # [ 0.452125] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1293worker2 # [ 0.483968] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1294client # [ 0.452148] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1295worker2 # [ 0.486074] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1296client # [ 0.452170] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1297client # [ 0.452198] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1298client # [ 0.452207] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1299client # [ 0.452211] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1300client # [ 0.453366] pci 0000:00:0a.0: enabling device (0000 -> 0002)1301lb # [ 0.493572] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1302worker2 # [ 0.497189] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1303worker1 # [ 0.495586] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1304worker1 # [ 0.497669] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1305worker2 # [ 0.503465] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1306lb # [ 0.499614] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1307worker2 # [ 0.505355] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1308lb # [ 0.501606] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1309worker1 # [ 0.508681] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1310client # [ 0.486074] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1311lb # [ 0.511565] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1312worker2 # [ 0.515644] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1313lb # [ 0.513545] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1314worker2 # [ 0.517790] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1315lb # [ 0.517123] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1316lb # [ 0.519120] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1317client # [ 0.492219] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1318client # [ 0.494183] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1319worker1 # [ 0.519123] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1320worker1 # [ 0.521357] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1321worker2 # [ 0.529334] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1322worker2 # [ 0.532323] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1323client # [ 0.504157] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1324worker2 # [ 0.534060] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1325client # [ 0.506125] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1326worker1 # [ 0.532701] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1327lb # [ 0.532880] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1328lb # [ 0.535012] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1329worker1 # [ 0.535613] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1330lb # [ 0.538418] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1331worker1 # [ 0.537681] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1332worker2 # [ 0.543943] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1333lb # [ 0.541566] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1334client # [ 0.516118] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1335client # [ 0.518718] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1336client # [ 0.520783] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1337client # [ 0.522920] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1338worker1 # [ 0.548141] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1339worker1 # [ 0.550201] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1340worker2 # [ 0.559418] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1341worker2 # [ 0.561903] msm_serial: driver initialized1342worker2 # [ 0.562040] SuperH (H)SCI(F) driver initialized1343lb # [ 0.559454] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1344worker2 # [ 0.562093] STM32 USART driver initialized1345client # [ 0.533205] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1346client # [ 0.535565] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1347lb # [ 0.562043] msm_serial: driver initialized1348worker1 # [ 0.561149] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1349lb # [ 0.562188] SuperH (H)SCI(F) driver initialized1350lb # [ 0.562244] STM32 USART driver initialized1351worker1 # [ 0.571614] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1352worker1 # [ 0.574175] msm_serial: driver initialized1353client # [ 0.553153] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1354worker1 # [ 0.574862] SuperH (H)SCI(F) driver initialized1355worker1 # [ 0.574922] STM32 USART driver initialized1356client # [ 0.561187] msm_serial: driver initialized1357client # [ 0.561345] SuperH (H)SCI(F) driver initialized1358client # [ 0.561397] STM32 USART driver initialized1359worker2 # [ 0.592682] loop: module loaded1360worker2 # [ 0.592853] virtio_blk virtio5: 1/0/0 default/read/poll queues1361lb # [ 0.590460] loop: module loaded1362lb # [ 0.590629] virtio_blk virtio5: 1/0/0 default/read/poll queues1363worker2 # [ 0.593597] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1364lb # [ 0.591371] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1365worker2 # [ 0.598996] megasas: 07.734.00.00-rc11366worker2 # [ 0.599691] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1367worker1 # [ 0.608931] loop: module loaded1368lb # [ 0.604007] megasas: 07.734.00.00-rc11369worker1 # [ 0.609129] virtio_blk virtio5: 1/0/0 default/read/poll queues1370lb # [ 0.604705] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1371worker1 # [ 0.609875] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1372worker2 # [ 0.611453] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001373lb # [ 0.606566] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001374worker2 # [ 0.611486] Intel/Sharp Extended Query Table at 0x00311375worker2 # [ 0.613036] Using buffer write method1376lb # [ 0.606593] Intel/Sharp Extended Query Table at 0x00311377worker2 # [ 0.613102] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1378lb # [ 0.608224] Using buffer write method1379client # [ 0.590645] loop: module loaded1380client # [ 0.590826] virtio_blk virtio5: 1/0/0 default/read/poll queues1381lb # [ 0.608289] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1382client # [ 0.591568] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1383lb # [ 0.609949] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001384lb # [ 0.609979] Intel/Sharp Extended Query Table at 0x00311385worker1 # [ 0.614903] megasas: 07.734.00.00-rc11386worker1 # [ 0.615660] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1387lb # [ 0.627463] Using buffer write method1388worker2 # [ 0.628610] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001389lb # [ 0.627501] Concatenating MTD devices:1390client # [ 0.596529] megasas: 07.734.00.00-rc11391lb # [ 0.627505] (0): "0.flash"1392worker2 # [ 0.628648] Intel/Sharp Extended Query Table at 0x00311393lb # [ 0.627509] (1): "0.flash"1394worker2 # [ 0.630198] Using buffer write method1395lb # [ 0.627513] into device "0.flash"1396client # [ 0.597195] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1397worker2 # [ 0.630225] Concatenating MTD devices:1398worker2 # [ 0.630229] (0): "0.flash"1399worker2 # [ 0.630233] (1): "0.flash"1400worker2 # [ 0.630236] into device "0.flash"1401worker1 # [ 0.627418] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402worker1 # [ 0.627464] Intel/Sharp Extended Query Table at 0x00311403worker1 # [ 0.629188] Using buffer write method1404worker1 # [ 0.629271] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1405worker1 # [ 0.644848] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406worker1 # [ 0.644905] Intel/Sharp Extended Query Table at 0x00311407client # [ 0.621286] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408client # [ 0.621345] Intel/Sharp Extended Query Table at 0x00311409worker1 # [ 0.648645] Using buffer write method1410client # [ 0.622895] Using buffer write method1411worker1 # [ 0.648690] Concatenating MTD devices:1412worker1 # [ 0.648694] (0): "0.flash"1413worker1 # [ 0.648699] (1): "0.flash"1414client # [ 0.622955] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1415worker1 # [ 0.648702] into device "0.flash"1416client # [ 0.627878] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417client # [ 0.627927] Intel/Sharp Extended Query Table at 0x00311418client # [ 0.629787] Using buffer write method1419client # [ 0.629812] Concatenating MTD devices:1420client # [ 0.629816] (0): "0.flash"1421client # [ 0.629820] (1): "0.flash"1422client # [ 0.629823] into device "0.flash"1423worker2 # [ 0.867081] Freeing initrd memory: 26148K1424lb # [ 0.865590] Freeing initrd memory: 26140K1425worker2 # [ 0.873067] tun: Universal TUN/TAP device driver, 1.61426lb # [ 0.871807] tun: Universal TUN/TAP device driver, 1.61427worker2 # [ 0.876699] thunder_xcv, ver 1.01428worker2 # [ 0.876745] thunder_bgx, ver 1.01429worker2 # [ 0.876769] nicpf, ver 1.01430worker2 # [ 0.877300] e1000: Intel(R) PRO/1000 Network Driver1431worker2 # [ 0.877307] e1000: Copyright (c) 1999-2006 Intel Corporation.1432worker2 # [ 0.877334] e1000e: Intel(R) PRO/1000 Network Driver1433worker2 # [ 0.877343] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1434worker2 # [ 0.877377] igb: Intel(R) Gigabit Ethernet Network Driver1435lb # [ 0.875725] thunder_xcv, ver 1.01436lb # [ 0.875771] thunder_bgx, ver 1.01437worker2 # [ 0.877382] igb: Copyright (c) 2007-2014 Intel Corporation.1438lb # [ 0.875793] nicpf, ver 1.01439lb # [ 0.876343] e1000: Intel(R) PRO/1000 Network Driver1440worker2 # [ 0.877405] igbvf: Intel(R) Gigabit Virtual Function Network Driver1441worker2 # [ 0.877410] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1442lb # [ 0.876350] e1000: Copyright (c) 1999-2006 Intel Corporation.1443worker2 # [ 0.877542] sky2: driver version 1.301444lb # [ 0.876376] e1000e: Intel(R) PRO/1000 Network Driver1445lb # [ 0.876385] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1446lb # [ 0.876414] igb: Intel(R) Gigabit Ethernet Network Driver1447worker2 # [ 0.886218] usbcore: registered new interface driver usb-storage1448client # [ 0.857905] Freeing initrd memory: 26160K1449lb # [ 0.876419] igb: Copyright (c) 2007-2014 Intel Corporation.1450worker2 # [ 0.886337] usbcore: registered new interface driver usbserial_generic1451lb # [ 0.876441] igbvf: Intel(R) Gigabit Virtual Function Network Driver1452worker2 # [ 0.886352] usbserial: USB Serial support registered for generic1453lb # [ 0.876446] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1454lb # [ 0.876576] sky2: driver version 1.301455worker2 # [ 0.889314] ehci-pci 0000:00:0a.0: EHCI Host Controller1456lb # [ 0.878170] usbcore: registered new interface driver usb-storage1457worker2 # [ 0.889351] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11458lb # [ 0.878268] usbcore: registered new interface driver usbserial_generic1459client # [ 0.863803] tun: Universal TUN/TAP device driver, 1.61460worker2 # [ 0.889624] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001461lb # [ 0.878282] usbserial: USB Serial support registered for generic1462lb # [ 0.878878] hv_vmbus: registering driver hyperv_keyboard1463worker2 # [ 0.892602] hv_vmbus: registering driver hyperv_keyboard1464lb # [ 0.890501] ehci-pci 0000:00:0a.0: EHCI Host Controller1465worker2 # [ 0.894126] rtc-pl031 9010000.pl031: registered as rtc01466lb # [ 0.890536] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11467client # [ 0.867387] thunder_xcv, ver 1.01468client # [ 0.867430] thunder_bgx, ver 1.01469lb # [ 0.890772] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001470worker2 # [ 0.894152] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T10:50:40 UTC (1789901440)1471client # [ 0.867451] nicpf, ver 1.01472worker2 # [ 0.896199] i2c_dev: i2c /dev entries driver1473lb # [ 0.893583] rtc-pl031 9010000.pl031: registered as rtc01474lb # [ 0.893615] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T10:50:40 UTC (1789901440)1475worker1 # [ 0.895531] Freeing initrd memory: 26148K1476lb # [ 0.894023] i2c_dev: i2c /dev entries driver1477client # [ 0.869188] e1000: Intel(R) PRO/1000 Network Driver1478client # [ 0.869199] e1000: Copyright (c) 1999-2006 Intel Corporation.1479client # [ 0.869223] e1000e: Intel(R) PRO/1000 Network Driver1480client # [ 0.869232] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1481worker2 # [ 0.899399] sdhci: Secure Digital Host Controller Interface driver1482client # [ 0.869261] igb: Intel(R) Gigabit Ethernet Network Driver1483worker2 # [ 0.899408] sdhci: Copyright(c) Pierre Ossman1484lb # [ 0.899113] sdhci: Secure Digital Host Controller Interface driver1485client # [ 0.869266] igb: Copyright (c) 2007-2014 Intel Corporation.1486lb # [ 0.899129] sdhci: Copyright(c) Pierre Ossman1487worker2 # [ 0.899674] Synopsys Designware Multimedia Card Interface Driver1488client # [ 0.869288] igbvf: Intel(R) Gigabit Virtual Function Network Driver1489lb # [ 0.899430] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001490worker2 # [ 0.900043] sdhci-pltfm: SDHCI platform and OF driver helper1491lb # [ 0.899778] hub 1-0:1.0: USB hub found1492client # [ 0.869293] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1493worker2 # [ 0.901646] hid: raw HID events driver (C) Jiri Kosina1494lb # [ 0.899799] hub 1-0:1.0: 6 ports detected1495client # [ 0.869427] sky2: driver version 1.301496worker2 # [ 0.901885] usbcore: registered new interface driver usbhid1497worker1 # [ 0.901734] tun: Universal TUN/TAP device driver, 1.61498client # [ 0.871015] usbcore: registered new interface driver usb-storage1499worker2 # [ 0.901891] usbhid: USB HID core driver1500client # [ 0.871065] usbcore: registered new interface driver usbserial_generic1501worker2 # [ 0.906808] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001502client # [ 0.871080] usbserial: USB Serial support registered for generic1503worker2 # [ 0.907852] hub 1-0:1.0: USB hub found1504client # [ 0.871669] hv_vmbus: registering driver hyperv_keyboard1505worker2 # [ 0.908337] hub 1-0:1.0: 6 ports detected1506client # [ 0.872674] ehci-pci 0000:00:0a.0: EHCI Host Controller1507lb # [ 0.903264] Synopsys Designware Multimedia Card Interface Driver1508client # [ 0.872699] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11509worker1 # [ 0.905738] thunder_xcv, ver 1.01510worker1 # [ 0.905792] thunder_bgx, ver 1.01511lb # [ 0.903694] sdhci-pltfm: SDHCI platform and OF driver helper1512client # [ 0.872963] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001513worker1 # [ 0.905814] nicpf, ver 1.01514worker2 # [ 0.909784] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1515lb # [ 0.905471] hid: raw HID events driver (C) Jiri Kosina1516client # [ 0.884416] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001517lb # [ 0.905701] usbcore: registered new interface driver usbhid1518lb # [ 0.905707] usbhid: USB HID core driver1519client # [ 0.885426] hub 1-0:1.0: USB hub found1520worker2 # [ 0.912358] drop_monitor: Initializing network drop monitor service1521client # [ 0.885910] hub 1-0:1.0: 6 ports detected1522worker2 # [ 0.912496] NET: Registered PF_INET6 protocol family1523worker2 # [ 0.914346] Segment Routing with IPv61524worker2 # [ 0.914379] In-situ OAM (IOAM) with IPv61525client # [ 0.886785] rtc-pl031 9010000.pl031: registered as rtc01526worker1 # [ 0.907672] e1000: Intel(R) PRO/1000 Network Driver1527lb # [ 0.910137] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1528worker2 # [ 0.915359] NET: Registered PF_PACKET protocol family1529worker1 # [ 0.907685] e1000: Copyright (c) 1999-2006 Intel Corporation.1530client # [ 0.886811] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T10:50:40 UTC (1789901440)1531lb # [ 0.911720] drop_monitor: Initializing network drop monitor service1532worker1 # [ 0.907712] e1000e: Intel(R) PRO/1000 Network Driver1533client # [ 0.887113] i2c_dev: i2c /dev entries driver1534lb # [ 0.911878] NET: Registered PF_INET6 protocol family1535worker1 # [ 0.907720] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1536lb # [ 0.914891] Segment Routing with IPv61537worker1 # [ 0.907748] igb: Intel(R) Gigabit Ethernet Network Driver1538lb # [ 0.914913] In-situ OAM (IOAM) with IPv61539worker2 # [ 0.916056] 9pnet: Installing 9P2000 support1540worker1 # [ 0.907754] igb: Copyright (c) 2007-2014 Intel Corporation.1541worker2 # [ 0.918186] Key type dns_resolver registered1542lb # [ 0.914946] NET: Registered PF_PACKET protocol family1543worker1 # [ 0.907777] igbvf: Intel(R) Gigabit Virtual Function Network Driver1544worker1 # [ 0.907783] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1545worker1 # [ 0.907921] sky2: driver version 1.301546worker1 # [ 0.909509] usbcore: registered new interface driver usb-storage1547worker1 # [ 0.909610] usbcore: registered new interface driver usbserial_generic1548lb # [ 0.916651] 9pnet: Installing 9P2000 support1549worker1 # [ 0.909625] usbserial: USB Serial support registered for generic1550lb # [ 0.918908] Key type dns_resolver registered1551client # [ 0.892178] sdhci: Secure Digital Host Controller Interface driver1552worker1 # [ 0.910246] hv_vmbus: registering driver hyperv_keyboard1553client # [ 0.892188] sdhci: Copyright(c) Pierre Ossman1554worker1 # [ 0.911067] ehci-pci 0000:00:0a.0: EHCI Host Controller1555client # [ 0.892450] Synopsys Designware Multimedia Card Interface Driver1556worker1 # [ 0.911101] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11557client # [ 0.892812] sdhci-pltfm: SDHCI platform and OF driver helper1558worker1 # [ 0.911335] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001559client # [ 0.894486] hid: raw HID events driver (C) Jiri Kosina1560worker2 # [ 0.924969] registered taskstats version 11561client # [ 0.894722] usbcore: registered new interface driver usbhid1562worker2 # [ 0.925135] Loading compiled-in X.509 certificates1563client # [ 0.894728] usbhid: USB HID core driver1564worker1 # [ 0.922634] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001565worker1 # [ 0.922988] hub 1-0:1.0: USB hub found1566worker1 # [ 0.923009] hub 1-0:1.0: 6 ports detected1567client # [ 0.900526] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1568worker1 # [ 0.926008] rtc-pl031 9010000.pl031: registered as rtc01569lb # [ 0.926163] registered taskstats version 11570client # [ 0.902048] drop_monitor: Initializing network drop monitor service1571lb # [ 0.926320] Loading compiled-in X.509 certificates1572worker1 # [ 0.926042] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T10:50:40 UTC (1789901440)1573client # [ 0.902187] NET: Registered PF_INET6 protocol family1574client # [ 0.905190] Segment Routing with IPv61575worker1 # [ 0.928084] i2c_dev: i2c /dev entries driver1576client # [ 0.905209] In-situ OAM (IOAM) with IPv61577client # [ 0.905238] NET: Registered PF_PACKET protocol family1578worker2 # [ 0.933669] Demotion targets for Node 0: null1579worker2 # [ 0.933777] Key type .fscrypt registered1580client # [ 0.906875] 9pnet: Installing 9P2000 support1581worker2 # [ 0.933784] Key type fscrypt-provisioning registered1582worker2 # [ 0.933874] ima: No TPM chip found, activating TPM-bypass!1583worker2 # [ 0.933893] ima: Allocated hash algorithm: sha11584worker2 # [ 0.933914] ima: No architecture policies found1585lb # [ 0.934819] Demotion targets for Node 0: null1586worker1 # [ 0.931423] sdhci: Secure Digital Host Controller Interface driver1587worker2 # [ 0.938089] input: gpio-keys as /devices/platform/gpio-keys/input/input01588lb # [ 0.934935] Key type .fscrypt registered1589worker1 # [ 0.931434] sdhci: Copyright(c) Pierre Ossman1590client # [ 0.909512] Key type dns_resolver registered1591lb # [ 0.934941] Key type fscrypt-provisioning registered1592worker1 # [ 0.931696] Synopsys Designware Multimedia Card Interface Driver1593lb # [ 0.935036] ima: No TPM chip found, activating TPM-bypass!1594worker1 # [ 0.932069] sdhci-pltfm: SDHCI platform and OF driver helper1595lb # [ 0.935055] ima: Allocated hash algorithm: sha11596worker1 # [ 0.933789] hid: raw HID events driver (C) Jiri Kosina1597lb # [ 0.935079] ima: No architecture policies found1598worker1 # [ 0.934038] usbcore: registered new interface driver usbhid1599worker1 # [ 0.934045] usbhid: USB HID core driver1600lb # [ 0.939354] input: gpio-keys as /devices/platform/gpio-keys/input/input01601client # [ 0.915941] registered taskstats version 11602worker1 # [ 0.939647] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1603worker1 # [ 0.941176] drop_monitor: Initializing network drop monitor service1604worker1 # [ 0.941318] NET: Registered PF_INET6 protocol family1605client # [ 0.916624] Loading compiled-in X.509 certificates1606worker1 # [ 0.944418] Segment Routing with IPv61607worker1 # [ 0.944438] In-situ OAM (IOAM) with IPv61608worker1 # [ 0.944467] NET: Registered PF_PACKET protocol family1609worker1 # [ 0.946134] 9pnet: Installing 9P2000 support1610client # [ 0.924653] Demotion targets for Node 0: null1611client # [ 0.924760] Key type .fscrypt registered1612client # [ 0.924766] Key type fscrypt-provisioning registered1613client # [ 0.924856] ima: No TPM chip found, activating TPM-bypass!1614worker1 # [ 0.949068] Key type dns_resolver registered1615client # [ 0.924875] ima: Allocated hash algorithm: sha11616client # [ 0.924896] ima: No architecture policies found1617client # [ 0.928911] input: gpio-keys as /devices/platform/gpio-keys/input/input01618worker2 # [ 0.956236] clk: Disabling unused clocks1619worker2 # [ 0.956262] PM: genpd: Disabling unused power domains1620worker2 # [ 0.960498] Freeing unused kernel memory: 4736K1621worker2 # [ 0.960689] Run /init as init process1622lb # [ 0.957982] clk: Disabling unused clocks1623worker1 # [ 0.955275] registered taskstats version 11624lb # [ 0.958023] PM: genpd: Disabling unused power domains1625worker1 # [ 0.955422] Loading compiled-in X.509 certificates1626lb # [ 0.962338] Freeing unused kernel memory: 4736K1627lb # [ 0.962560] Run /init as init process1628worker1 # [ 0.963849] Demotion targets for Node 0: null1629worker1 # [ 0.963957] Key type .fscrypt registered1630worker1 # [ 0.963963] Key type fscrypt-provisioning registered1631worker1 # [ 0.964055] ima: No TPM chip found, activating TPM-bypass!1632worker1 # [ 0.964075] ima: Allocated hash algorithm: sha11633worker1 # [ 0.964097] ima: No architecture policies found1634worker1 # [ 0.968219] input: gpio-keys as /devices/platform/gpio-keys/input/input01635client # [ 0.946438] clk: Disabling unused clocks1636client # [ 0.946460] PM: genpd: Disabling unused power domains1637worker2 # [ 0.976672] systemd[1]: Successfully made /usr/ read-only.1638client # [ 0.950612] Freeing unused kernel memory: 4736K1639client # [ 0.950805] Run /init as init process1640lb # [ 0.977692] systemd[1]: Successfully made /usr/ read-only.1641worker1 # [ 0.987437] clk: Disabling unused clocks1642client # [ 0.966138] systemd[1]: Successfully made /usr/ read-only.1643worker1 # [ 0.987468] PM: genpd: Disabling unused power domains1644worker1 # [ 0.991764] Freeing unused kernel memory: 4736K1645worker1 # [ 0.991971] Run /init as init process1646worker1 # [ 1.008562] systemd[1]: Successfully made /usr/ read-only.1647lb # [ 1.147471] usb 1-1: new high-speed USB device number 2 using ehci-pci1648worker2 # [ 1.154489] usb 1-1: new high-speed USB device number 2 using ehci-pci1649client # [ 1.136032] usb 1-1: new high-speed USB device number 2 using ehci-pci1650worker1 # [ 1.170356] usb 1-1: new high-speed USB device number 2 using ehci-pci1651lb # [ 1.299693] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652worker2 # [ 1.306746] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653client # [ 1.290325] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11654worker2 # [ 1.312573] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1655lb # [ 1.312822] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1656worker2 # [ 1.312630] systemd[1]: Detected virtualization qemu.1657worker2 # [ 1.312700] systemd[1]: Detected architecture arm64.1658lb # [ 1.324973] systemd[1]: Detected virtualization qemu.1659worker2 # [ 1.312723] systemd[1]: Running in initrd.1660worker1 # [ 1.322747] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11661lb # [ 1.325051] systemd[1]: Detected architecture arm64.1662worker2 # [ 1.313606] systemd[1]: Initializing machine ID from random generator.1663lb # [ 1.325065] systemd[1]: Running in initrd.1664worker2 # [ 1.313875] systemd[1]: Hostname set to <worker2>.1665lb # [ 1.325971] systemd[1]: Initializing machine ID from random generator.1666lb # [ 1.326337] systemd[1]: Hostname set to <lb>.1667client # [ 1.300920] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668client # [ 1.300979] systemd[1]: Detected virtualization qemu.1669client # [ 1.301047] systemd[1]: Detected architecture arm64.1670client # [ 1.301072] systemd[1]: Running in initrd.1671client # [ 1.301949] systemd[1]: Initializing machine ID from random generator.1672client # [ 1.302215] systemd[1]: Hostname set to <client>.1673worker1 # [ 1.343371] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1674worker1 # [ 1.355370] systemd[1]: Detected virtualization qemu.1675worker1 # [ 1.357334] systemd[1]: Detected architecture arm64.1676worker1 # [ 1.359295] systemd[1]: Running in initrd.1677worker1 # [ 1.361830] systemd[1]: Initializing machine ID from random generator.1678worker1 # [ 1.364716] systemd[1]: Hostname set to <worker1>.1679lb # [ 1.387713] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01680worker2 # [ 1.394662] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681client # [ 1.372218] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682worker1 # [ 1.410586] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683lb # [ 1.511448] usb 1-2: new high-speed USB device number 3 using ehci-pci1684worker2 # [ 1.518452] usb 1-2: new high-speed USB device number 3 using ehci-pci1685client # [ 1.496034] usb 1-2: new high-speed USB device number 3 using ehci-pci1686worker1 # [ 1.534335] usb 1-2: new high-speed USB device number 3 using ehci-pci1687lb # [ 1.644236] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688client # [ 1.631640] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689worker2 # [ 1.659677] systemd[1]: bpf-restrict-fs: LSM BPF program attached1690worker1 # [ 1.667051] systemd[1]: bpf-restrict-fs: LSM BPF program attached1691lb # [ 1.670974] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21692worker2 # [ 1.676550] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21693lb # [ 1.676726] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694worker2 # [ 1.683310] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01695client # [ 1.656771] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21696client # [ 1.662475] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01697worker1 # [ 1.697981] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21698worker1 # [ 1.703714] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01699lb # [ 1.761134] systemd[1]: Queued start job for default target Initrd Default Target.1700worker2 # [ 1.771714] systemd[1]: Queued start job for default target Initrd Default Target.1701lb # [ 1.769804] systemd[1]: Created slice Slice /system/modprobe.1702lb # [ 1.771004] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1703lb # [ 1.772480] systemd[1]: Expecting device /dev/disk/by-label/nixos...1704client # [ 1.748776] systemd[1]: Queued start job for default target Initrd Default Target.1705lb # [ 1.773558] systemd[1]: Reached target Path Units.1706lb # [ 1.774381] systemd[1]: Reached target Slice Units.1707lb # [ 1.775271] systemd[1]: Reached target Swaps.1708lb # [ 1.776106] systemd[1]: Reached target Timer Units.1709lb # [ 1.777127] systemd[1]: Listening on D-Bus System Message Bus Socket.1710worker2 # [ 1.780703] systemd[1]: Created slice Slice /system/modprobe.1711lb # [ 1.778362] systemd[1]: Listening on Journal Socket (/dev/log).1712worker2 # [ 1.781973] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1713lb # [ 1.779536] systemd[1]: Listening on Journal Sockets.1714lb # [ 1.780577] systemd[1]: Listening on udev Control Socket.1715worker2 # [ 1.783398] systemd[1]: Expecting device /dev/disk/by-label/nixos...1716worker2 # [ 1.784465] systemd[1]: Reached target Path Units.1717lb # [ 1.781626] systemd[1]: Listening on udev Kernel Socket.1718worker2 # [ 1.785277] systemd[1]: Reached target Slice Units.1719lb # [ 1.782543] systemd[1]: Reached target Socket Units.1720worker2 # [ 1.786106] systemd[1]: Reached target Swaps.1721worker2 # [ 1.786912] systemd[1]: Reached target Timer Units.1722client # [ 1.759856] systemd[1]: Created slice Slice /system/modprobe.1723worker2 # [ 1.787958] systemd[1]: Listening on D-Bus System Message Bus Socket.1724lb # [ 1.785016] systemd[1]: Starting Create List of Static Device Nodes...1725client # [ 1.761159] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1726worker2 # [ 1.789178] systemd[1]: Listening on Journal Socket (/dev/log).1727client # [ 1.762477] systemd[1]: Expecting device /dev/disk/by-label/nixos...1728worker2 # [ 1.790317] systemd[1]: Listening on Journal Sockets.1729client # [ 1.763632] systemd[1]: Reached target Path Units.1730worker1 # [ 1.787136] systemd[1]: Queued start job for default target Initrd Default Target.1731client # [ 1.764523] systemd[1]: Reached target Slice Units.1732client # [ 1.765353] systemd[1]: Reached target Swaps.1733worker2 # [ 1.791392] systemd[1]: Listening on udev Control Socket.1734client # [ 1.766111] systemd[1]: Reached target Timer Units.1735worker2 # [ 1.791517] systemd[1]: Listening on udev Kernel Socket.1736worker2 # [ 1.791541] systemd[1]: Reached target Socket Units.1737client # [ 1.767146] systemd[1]: Listening on D-Bus System Message Bus Socket.1738client # [ 1.768433] systemd[1]: Listening on Journal Socket (/dev/log).1739worker2 # [ 1.795753] systemd[1]: Starting Create List of Static Device Nodes...1740client # [ 1.769562] systemd[1]: Listening on Journal Sockets.1741client # [ 1.770549] systemd[1]: Listening on udev Control Socket.1742lb # [ 1.795613] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1743client # [ 1.771561] systemd[1]: Listening on udev Kernel Socket.1744lb # [ 1.796805] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1745worker1 # [ 1.796654] systemd[1]: Created slice Slice /system/modprobe.1746client # [ 1.772529] systemd[1]: Reached target Socket Units.1747worker1 # [ 1.797861] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1748client # [ 1.774925] systemd[1]: Starting Create List of Static Device Nodes...1749worker1 # [ 1.799300] systemd[1]: Expecting device /dev/disk/by-label/nixos...1750lb # [ 1.800240] systemd[1]: Mounting Kernel Configuration File System...1751worker1 # [ 1.800376] systemd[1]: Reached target Path Units.1752worker1 # [ 1.801186] systemd[1]: Reached target Slice Units.1753worker1 # [ 1.802013] systemd[1]: Reached target Swaps.1754worker2 # [ 1.806152] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1755worker1 # [ 1.802823] systemd[1]: Reached target Timer Units.1756worker1 # [ 1.803845] systemd[1]: Listening on D-Bus System Message Bus Socket.1757worker2 # [ 1.807526] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1758worker1 # [ 1.805190] systemd[1]: Listening on Journal Socket (/dev/log).1759worker1 # [ 1.806384] systemd[1]: Listening on Journal Sockets.1760worker1 # [ 1.807393] systemd[1]: Listening on udev Control Socket.1761worker1 # [ 1.808410] systemd[1]: Listening on udev Kernel Socket.1762client # [ 1.784115] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1763worker1 # [ 1.809314] systemd[1]: Reached target Socket Units.1764client # [ 1.784200] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1765worker1 # [ 1.811781] systemd[1]: Starting Create List of Static Device Nodes...1766client # [ 1.792223] systemd[1]: Mounting Kernel Configuration File System...1767worker2 # [ 1.820022] systemd[1]: Mounting Kernel Configuration File System...1768worker1 # [ 1.822213] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1769lb # [ 1.823926] systemd[1]: Starting Journal Service...1770worker1 # [ 1.823641] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1771worker1 # [ 1.835849] systemd[1]: Mounting Kernel Configuration File System...1772lb # [ 1.839617] systemd[1]: Starting Load Kernel Modules...1773worker2 # [ 1.842692] systemd[1]: Starting Journal Service...1774lb # [ 1.840581] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1775worker2 # [ 1.846569] systemd[1]: Starting Load Kernel Modules...1776worker2 # [ 1.847511] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1777lb # [ 1.847695] systemd[1]: Starting Coldplug All udev Devices...1778client # [ 1.824115] systemd[1]: Starting Journal Service...1779client # [ 1.831825] systemd[1]: Starting Load Kernel Modules...1780client # [ 1.832910] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1781worker1 # [ 1.858499] systemd[1]: Starting Journal Service...1782worker1 # [ 1.862501] systemd[1]: Starting Load Kernel Modules...1783worker1 # [ 1.863450] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1784lb # [ 1.863536] systemd[1]: Finished Create List of Static Device Nodes.1785lb # [ 1.864430] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1786lb # [ 1.864735] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1787worker1 # [ 1.867577] systemd[1]: Starting Coldplug All udev Devices...1788lb # [ 1.864998] systemd[1]: Mounted Kernel Configuration File System.1789client # [ 1.845543] systemd[1]: Starting Coldplug All udev Devices...1790worker2 # [ 1.878633] systemd[1]: Starting Coldplug All udev Devices...1791worker1 # [ 1.878520] systemd[1]: Finished Create List of Static Device Nodes.1792worker2 # [ 1.882560] systemd[1]: Finished Create List of Static Device Nodes.1793worker2 # [ 1.884220] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1794worker1 # [ 1.880420] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1795worker2 # [ 1.886860] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1796lb # [ 1.883765] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1797client # [ 1.868102] systemd[1]: Finished Create List of Static Device Nodes.1798client # [ 1.869257] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1799client # [ 1.869557] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1800worker1 # [ 1.894816] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1801client # [ 1.869795] systemd[1]: Mounted Kernel Configuration File System.1802worker2 # [ 1.901991] systemd-journald[73]: Collecting audit messages is disabled.1803worker2 # [ 1.902540] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1804client # [ 1.880463] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1805lb # [ 1.907002] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1806worker2 # [ 1.914809] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1807worker1 # [ 1.914969] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1808worker2 # [ 1.920043] systemd[1]: Mounted Kernel Configuration File System.1809worker1 # [ 1.918688] systemd[1]: Mounted Kernel Configuration File System.1810client # [ 1.895376] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1811lb # [ 1.920349] systemd-journald[73]: Collecting audit messages is disabled.1812client # [ 1.898681] systemd-journald[73]: Collecting audit messages is disabled.1813lb # [ 1.920402] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1814worker1 # [ 1.924967] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1815lb # [ 1.924908] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01816lb # [ 1.925138] [drm] features: -virgl +edid -resource_blob -host_visible1817lb # [ 1.925147] [drm] features: -context_init1818lb # [ 1.925875] [drm] number of scanouts: 11819lb # [ 1.925893] [drm] number of cap sets: 01820client # [ 1.908340] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1821worker2 # [ 1.930476] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1822worker2 # [ 1.935007] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01823worker2 # [ 1.935235] [drm] features: -virgl +edid -resource_blob -host_visible1824worker2 # [ 1.935244] [drm] features: -context_init1825worker2 # [ 1.935941] [drm] number of scanouts: 11826worker2 # [ 1.935959] [drm] number of cap sets: 01827client # [ 1.915696] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01828client # [ 1.915929] [drm] features: -virgl +edid -resource_blob -host_visible1829client # [ 1.915938] [drm] features: -context_init1830worker1 # [ 1.941190] systemd-journald[73]: Collecting audit messages is disabled.1831worker1 # [ 1.942737] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1832lb # [ 1.944044] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1833lb # [ 1.946516] systemd[1]: Starting Create Static Device Nodes in /dev...1834worker1 # [ 1.946691] systemd[1]: Starting Create Static Device Nodes in /dev...1835client # [ 1.930395] [drm] number of scanouts: 11836client # [ 1.930427] [drm] number of cap sets: 01837worker2 # [ 1.954840] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1838worker2 # [ 1.954863] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01839client # [ 1.932315] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1840client # [ 1.934749] systemd[1]: Starting Create Static Device Nodes in /dev...1841lb # [ 1.955863] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1842lb # [ 1.955882] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01843worker1 # [ 1.958360] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1844worker2 # [ 1.967090] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1845worker1 # [ 1.963025] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01846worker2 # [ 1.969593] systemd[1]: Starting Create Static Device Nodes in /dev...1847worker1 # [ 1.963258] [drm] features: -virgl +edid -resource_blob -host_visible1848worker1 # [ 1.963267] [drm] features: -context_init1849worker1 # [ 1.964001] [drm] number of scanouts: 11850worker1 # [ 1.964019] [drm] number of cap sets: 01851client # [ 1.940394] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1852client # [ 1.940411] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01853worker1 # [ 1.982787] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1854lb # [ 1.983855] Console: switching to colour frame buffer device 160x501855worker1 # [ 1.982811] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01856worker2 # [ 1.994830] Console: switching to colour frame buffer device 160x501857worker2 # [ 2.001411] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1858lb # [ 1.999515] systemd[1]: Finished Create Static Device Nodes in /dev.1859lb # [ 1.999733] systemd[1]: Reached target Preparation for Local File Systems.1860lb # [ 1.999757] systemd[1]: Reached target Local File Systems.1861lb # [ 2.003748] systemd[1]: Starting Rule-based Manager for Device Events and Files...1862client # [ 1.972299] Console: switching to colour frame buffer device 160x501863client # [ 1.978898] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1864worker1 # [ 2.011075] Console: switching to colour frame buffer device 160x501865client # [ 1.992531] systemd[1]: Finished Create Static Device Nodes in /dev.1866lb # [ 2.020105] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1867client # [ 1.992760] systemd[1]: Reached target Preparation for Local File Systems.1868client # [ 1.992793] systemd[1]: Reached target Local File Systems.1869worker2 # [ 2.023123] systemd[1]: Finished Create Static Device Nodes in /dev.1870worker2 # [ 2.023363] systemd[1]: Reached target Preparation for Local File Systems.1871worker2 # [ 2.023398] systemd[1]: Reached target Local File Systems.1872worker1 # [ 2.022937] systemd[1]: Finished Create Static Device Nodes in /dev.1873client # [ 1.997888] systemd[1]: Starting Rule-based Manager for Device Events and Files...1874worker1 # [ 2.023150] systemd[1]: Reached target Preparation for Local File Systems.1875worker1 # [ 2.023174] systemd[1]: Reached target Local File Systems.1876worker2 # [ 2.029118] systemd[1]: Starting Rule-based Manager for Device Events and Files...1877worker1 # [ 2.027322] systemd[1]: Starting Rule-based Manager for Device Events and Files...1878lb # [ 2.036899] systemd[1]: Finished Load Kernel Modules.1879worker2 # [ 2.043066] systemd[1]: Finished Load Kernel Modules.1880worker1 # [ 2.042919] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1881client # [ 2.024281] systemd[1]: Finished Load Kernel Modules.1882lb # [ 2.051551] systemd[1]: Starting Apply Kernel Variables...1883client # [ 2.032398] systemd[1]: Starting Apply Kernel Variables...1884worker1 # [ 2.059807] systemd[1]: Finished Load Kernel Modules.1885worker2 # [ 2.067405] systemd[1]: Starting Apply Kernel Variables...1886worker1 # [ 2.074441] systemd[1]: Starting Apply Kernel Variables...1887worker2 # [ 2.086559] systemd[1]: Started Journal Service.1888worker2 # [ 2.083446] systemd-modules-load[74]: Inserted module 'dm_mod'1889worker2 # [ 2.088402] systemd-modules-load[74]: Module 'virtio_balloon' is built in1890worker2 # [ 2.089493] systemd-modules-load[74]: Module 'virtio_console' is built in1891worker2 # [ 2.090574] systemd-modules-load[74]: Inserted module 'virtio_gpu'1892worker2 # [ 2.091537] systemd-modules-load[74]: Module 'virtio_rng' is built in1893lb # [ 2.104254] systemd[1]: Finished Apply Kernel Variables.1894lb # [ 2.096671] systemd-modules-load[74]: Inserted module 'dm_mod'1895client # [ 2.087531] systemd[1]: Finished Apply Kernel Variables.1896lb # [ 2.113146] systemd[1]: Started Journal Service.1897worker2 # [ 2.102841] systemd[1]: Starting Create System Files and Directories...1898lb # [ 2.103879] systemd-modules-load[74]: Module 'virtio_balloon' is built in1899client # [ 2.080736] systemd-modules-load[74]: Inserted module 'dm_mod'1900lb # [ 2.106266] systemd-modules-load[74]: Module 'virtio_console' is built in1901worker1 # [ 2.120216] systemd[1]: Started Journal Service.1902client # [ 2.097772] systemd[1]: Started Journal Service.1903lb # [ 2.112711] systemd-modules-load[74]: Inserted module 'virtio_gpu'1904worker2 # [ 2.117173] systemd[1]: Finished Apply Kernel Variables.1905client # [ 2.088273] systemd-modules-load[74]: Module 'virtio_balloon' is built in1906lb # [ 2.115231] systemd-modules-load[74]: Module 'virtio_rng' is built in1907client # [ 2.092084] systemd-modules-load[74]: Module 'virtio_console' is built in1908client # [ 2.093212] systemd-modules-load[74]: Inserted module 'virtio_gpu'1909worker1 # [ 2.120338] systemd-modules-load[74]: Inserted module 'dm_mod'1910lb # [ 2.120464] systemd-udevd[79]: Using default interface naming scheme 'v261'.1911client # [ 2.096262] systemd-modules-load[74]: Module 'virtio_rng' is built in1912worker1 # [ 2.121656] systemd-modules-load[74]: Module 'virtio_balloon' is built in1913worker1 # [ 2.122748] systemd-modules-load[74]: Module 'virtio_console' is built in1914worker1 # [ 2.123814] systemd-modules-load[74]: Inserted module 'virtio_gpu'1915lb # [ 2.124296] systemd[1]: Starting Create System Files and Directories...1916client # [ 2.100774] systemd-udevd[80]: Using default interface naming scheme 'v261'.1917worker2 # [ 2.133173] systemd[1]: Finished Create System Files and Directories.1918client # [ 2.105835] systemd[1]: Starting Create System Files and Directories...1919worker2 # [ 2.141373] systemd-udevd[80]: Using default interface naming scheme 'v261'.1920worker1 # [ 2.140580] systemd-modules-load[74]: Module 'virtio_rng' is built in1921worker1 # [ 2.141669] systemd[1]: Starting Create System Files and Directories...1922worker1 # [ 2.142953] systemd-udevd[80]: Using default interface naming scheme 'v261'.1923worker1 # [ 2.152401] systemd[1]: Finished Apply Kernel Variables.1924lb # [ 2.160787] systemd[1]: Started Rule-based Manager for Device Events and Files.1925worker2 # [ 2.169189] systemd[1]: Started Rule-based Manager for Device Events and Files.1926client # [ 2.141045] systemd[1]: Finished Create System Files and Directories.1927lb # [ 2.166057] systemd[1]: Finished Create System Files and Directories.1928worker1 # [ 2.167123] systemd[1]: Finished Create System Files and Directories.1929client # [ 2.144698] systemd[1]: Started Rule-based Manager for Device Events and Files.1930worker1 # [ 2.179029] systemd[1]: Started Rule-based Manager for Device Events and Files.1931lb # [ 2.220275] systemd[1]: Starting Virtual Console Setup...1932client # [ 2.203365] systemd[1]: Starting Virtual Console Setup...1933worker2 # [ 2.233492] systemd[1]: Starting Virtual Console Setup...1934worker1 # [ 2.236116] systemd[1]: Starting Virtual Console Setup...1935lb # [ 2.272534] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1936lb # [ 2.276040] systemd[1]: Finished Virtual Console Setup.1937worker2 # [ 2.280535] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1938worker2 # [ 2.283944] systemd[1]: Finished Virtual Console Setup.1939worker1 # [ 2.288547] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940client # [ 2.264509] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1941worker1 # [ 2.292041] systemd[1]: Finished Virtual Console Setup.1942client # [ 2.267960] systemd[1]: Finished Virtual Console Setup.1943lb # [ 2.872752] systemd[1]: Finished Coldplug All udev Devices.1944lb # [ 2.874559] systemd[1]: Reached target System Initialization.1945lb # [ 2.880651] systemd[1]: Reached target Basic System.1946worker2 # [ 2.889430] systemd[1]: Finished Coldplug All udev Devices.1947worker2 # [ 2.890407] systemd[1]: Reached target System Initialization.1948worker2 # [ 2.891214] systemd[1]: Reached target Basic System.1949client # [ 2.864882] systemd[1]: Finished Coldplug All udev Devices.1950client # [ 2.865861] systemd[1]: Reached target System Initialization.1951client # [ 2.866681] systemd[1]: Reached target Basic System.1952worker1 # [ 2.900805] systemd[1]: Finished Coldplug All udev Devices.1953worker1 # [ 2.902594] systemd[1]: Reached target System Initialization.1954worker1 # [ 2.904409] systemd[1]: Reached target Basic System.1955worker2 # [ 3.027877] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1956lb # [ 3.023470] (udev-worker)[102]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1957client # [ 2.999916] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1958worker1 # [ 3.039705] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1959worker2 # [ 3.053643] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960worker2 # [ 3.058572] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1961lb # [ 3.056954] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1962lb # [ 3.059601] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1963client # [ 3.048881] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1964client # [ 3.056074] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1965worker1 # [ 3.090320] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1966worker1 # [ 3.096151] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1967worker2 # [ 3.124097] systemd[1]: Found device /dev/disk/by-label/nixos.1968worker2 # [ 3.127596] systemd[1]: Reached target Initrd Root Device.1969worker2 # [ 3.131107] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1970client # [ 3.104299] systemd[1]: Found device /dev/disk/by-label/nixos.1971lb # [ 3.132704] systemd[1]: Found device /dev/disk/by-label/nixos.1972client # [ 3.109553] systemd[1]: Reached target Initrd Root Device.1973lb # [ 3.135043] systemd[1]: Reached target Initrd Root Device.1974client # [ 3.113658] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1975lb # [ 3.140546] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1976worker1 # [ 3.149693] systemd[1]: Found device /dev/disk/by-label/nixos.1977worker1 # [ 3.152640] systemd[1]: Reached target Initrd Root Device.1978worker1 # [ 3.155031] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1979worker2 # [ 3.188672] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1980worker2 # [ 3.194897] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1981worker2 # [ 3.199372] systemd[1]: Mounting /sysroot...1982lb # [ 3.195125] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1983client # [ 3.171825] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1984client # [ 3.178192] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1985lb # [ 3.203944] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1986lb # [ 3.206330] systemd[1]: Mounting /sysroot...1987client # [ 3.183166] systemd[1]: Mounting /sysroot...1988worker1 # [ 3.208502] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1989worker1 # [ 3.214979] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1990worker1 # [ 3.219906] systemd[1]: Mounting /sysroot...1991worker2 # [ 3.249680] EXT4-fs (vda): mounted filesystem bcc8f41c-a7d8-43cd-a7a4-2175576925a0 r/w with ordered data mode. Quota mode: none.1992worker2 # [ 3.240959] systemd[1]: Mounted /sysroot.1993worker2 # [ 3.243297] systemd[1]: Reached target Initrd Root File System.1994client # [ 3.233434] EXT4-fs (vda): mounted filesystem 789af849-db24-448d-9161-c1adcfa0e2c7 r/w with ordered data mode. Quota mode: none.1995worker2 # [ 3.248227] systemd[1]: Starting Mountpoints Configured in the Real Root...1996client # [ 3.224826] systemd[1]: Mounted /sysroot.1997lb # [ 3.264137] EXT4-fs (vda): mounted filesystem f59c81de-1e72-4fc0-a790-d26192d7497d r/w with ordered data mode. Quota mode: none.1998client # [ 3.227068] systemd[1]: Reached target Initrd Root File System.1999lb # [ 3.254037] systemd[1]: Mounted /sysroot.2000worker1 # [ 3.267787] EXT4-fs (vda): mounted filesystem c14f5e8f-e049-43a6-abb5-eeb7712a4011 r/w with ordered data mode. Quota mode: none.
loading 1,477 hidden lines…
3478lb # [ 19.980994] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log3479lb # [ 19.988908] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config3480lb # [ 19.992885] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action3481lb # [ 19.995329] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal3482lb # [ 19.997435] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response3483lb # [ 19.999798] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3484lb # [ 20.002725] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3485lb # [ 20.005474] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3486lb # [ 20.007714] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls3487lb # [ 20.015495] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re23488lb # [ 20.017572] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3489lb # [ 20.021655] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator3490lb # [ 20.028153] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift3491lb # [ 20.031496] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input3492lb # [ 20.033516] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy3493lb # [ 20.036374] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3494lb # [ 20.038720] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert3495lb # [ 20.043913] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3496lb # [ 20.049329] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3497lb # [ 20.052743] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection3498lb # [ 20.058249] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3499lb # [ 20.061312] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23500lb # [ 20.063416] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3501lb # [ 20.065617] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13502lb # [ 20.067938] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3503lb # [ 20.071764] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3504lb # [ 20.075378] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3505lb # [ 20.078268] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3506lb # [ 20.080538] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3507lb # [ 20.082497] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3508lb # [ 20.085137] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3509lb # [ 20.087039] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3510lb # [ 20.090465] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3511lb # [ 20.092974] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3512lb # [ 20.097934] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme3513lb # [ 20.101651] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3514lb # [ 20.104653] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default3515lb # [ 20.106477] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter3516lb # [ 20.109013] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy3517lb # [ 20.123487] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3518lb # [ 20.131490] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3519lb # [ 20.134886] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic3520lb # [ 20.137196] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3521lb # [ 20.139620] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3522lb # [ 20.141809] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle3523lb # [ 20.144492] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector3524lb # [ 20.152594] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo3525lb # [ 20.154732] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san3526lb # [ 20.162371] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls3527lb # [ 20.167502] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3528lb # [ 20.171213] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3529lb # [ 20.173988] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3530lb # [ 20.179266] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3531lb # [ 20.187455] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder3532lb # [ 20.190289] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3533lb # [ 20.197778] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3534lb # [ 20.200740] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3535lb # [ 20.206910] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3536lb # [ 20.210651] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3537lb # [ 20.212935] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3538lb # [ 20.225168] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3539lb # [ 20.227660] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3540lb # [ 20.230019] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3541lb # [ 20.233688] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3542lb # [ 20.239296] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3543lb # [ 20.244433] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable3544lb # [ 20.247770] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override3545lb # [ 20.252252] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple3546lb # [ 20.254712] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3547lb # [ 20.257497] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3548lb # [ 20.260091] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23549lb # [ 20.262639] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3550lb # [ 20.265250] envoy[913]: [2026-09-20 10:50:59.508][913][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3551lb # [ 20.267962] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3552lb # [ 20.270924] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin3553lb # [ 20.274891] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router3554lb # [ 20.277398] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default3555lb # [ 20.281087] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3556lb # [ 20.283835] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3557lb # [ 20.287121] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case3558lb # [ 20.290193] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection3559lb # [ 20.292517] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats3560lb # [ 20.295242] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3561lb # [ 20.298088] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83562lb # [ 20.301376] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3563lb # [ 20.304739] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3564lb # [ 20.310390] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3565lb # [ 20.313315] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default3566lb # [ 20.315352] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3567lb # [ 20.318230] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3568lb # [ 20.321728] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port3569lb # [ 20.330360] envoy[913]: [2026-09-20 10:50:59.509][913][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3570lb # [ 20.332606] envoy[913]: [2026-09-20 10:50:59.577][913][info][main] [source/server/server.cc:531] HTTP header map info:3571lb # [ 20.334319] envoy[913]: [2026-09-20 10:50:59.579][913][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3572lb # [ 20.352064] envoy[913]: [2026-09-20 10:50:59.579][913][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3573lb # [ 20.353925] envoy[913]: [2026-09-20 10:50:59.579][913][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3574lb # [ 20.364193] envoy[913]: [2026-09-20 10:50:59.579][913][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3575lb # [ 20.366170] envoy[913]: [2026-09-20 10:50:59.766][913][info][main] [source/server/server.cc:968] runtime: {}3576lb # [ 20.367615] envoy[913]: [2026-09-20 10:50:59.795][913][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3577lb # [ 20.369786] envoy[913]: [2026-09-20 10:50:59.795][913][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013578lb # [ 20.372649] envoy[913]: [2026-09-20 10:50:59.795][913][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013579lb # [ 20.378419] envoy[913]: [2026-09-20 10:50:59.800][913][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3580lb # [ 20.380381] envoy[913]: [2026-09-20 10:50:59.800][913][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3581lb # [ 20.385543] envoy[913]: [2026-09-20 10:50:59.800][913][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3582lb # [ 20.403545] envoy[913]: [2026-09-20 10:51:00.024][913][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3583lb # [ 20.419009] envoy[913]: [2026-09-20 10:51:00.040][913][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3584worker1 # [ 20.420247] systemd[1]: Starting Nix worker-protocol over gRPC...3585worker2 # [ 20.425447] systemd[1]: Starting Nix worker-protocol over gRPC...3586lb # [ 20.422329] envoy[913]: [2026-09-20 10:51:00.043][913][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3587lb # [ 20.426465] envoy[913]: [2026-09-20 10:51:00.043][913][info][main] [source/server/server.cc:1077] starting main dispatch loop3588worker1 # [ 20.703795] nix-grpc-daemon[778]: ts=2026-09-20T10:51:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3589worker2 # [ 20.707485] nix-grpc-daemon[777]: ts=2026-09-20T10:51:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3590worker1 # [ 20.716781] nix-grpc-daemon[778]: ts=2026-09-20T10:51:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13591worker2 # [ 20.720663] nix-grpc-daemon[777]: ts=2026-09-20T10:51:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13592worker2 # [ 20.768102] nix-grpc-daemon[777]: ts=2026-09-20T10:51:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3593worker1 # [ 20.763138] nix-grpc-daemon[778]: ts=2026-09-20T10:51:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3594worker1 # [ 20.767570] systemd[1]: Started Nix worker-protocol over gRPC.3595worker2 # [ 20.773948] systemd[1]: Started Nix worker-protocol over gRPC.3596lb # [ 20.805587] envoy[913]: [2026-09-20 10:51:00.427][913][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3597lb # [ 20.814182] envoy[913]: [2026-09-20 10:51:00.432][913][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3598lb # [ 20.820241] envoy[913]: [2026-09-20 10:51:00.432][913][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3599lb # [ 20.832349] envoy[913]: [2026-09-20 10:51:00.432][913][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3600worker2 # [ 20.851692] nix-grpc-daemon[777]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500513601worker1 # [ 20.849262] nix-grpc-daemon[778]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500513602worker2 # [ 20.857297] nix-grpc-daemon[777]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3603worker1 # [ 20.853637] nix-grpc-daemon[778]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3604worker2 # [ 21.058896] nix-grpc-daemon[777]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500513605worker1 # [ 21.055240] nix-grpc-daemon[778]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500513606worker1 # [ 21.061059] nix-grpc-daemon[778]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3607worker2 # [ 21.064891] nix-grpc-daemon[777]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3608worker1 # [ 21.262105] nix-grpc-daemon[778]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500513609worker2 # [ 21.265996] nix-grpc-daemon[777]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500513610worker1 # [ 21.267631] nix-grpc-daemon[778]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3611worker2 # [ 21.271723] nix-grpc-daemon[777]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3612worker1 # [ 21.468767] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513613worker2 # [ 21.473054] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513614worker1 # [ 21.474691] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3615worker2 # [ 21.478939] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3616worker1 # [ 21.675682] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513617worker2 # [ 21.679982] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513618worker1 # [ 21.681499] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3619worker2 # [ 21.685696] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3620worker1 # [ 21.884956] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513621worker2 # [ 21.888881] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513622worker2 # [ 21.893181] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3623worker1 # [ 21.889379] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3624worker2 # [ 22.090019] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513625worker1 # [ 22.086385] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513626worker1 # [ 22.091976] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3627worker2 # [ 22.095703] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3628worker1 # [ 22.293356] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513629worker2 # [ 22.297290] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500513630worker1 # [ 22.299183] nix-grpc-daemon[778]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3631worker2 # [ 22.303502] nix-grpc-daemon[777]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3632worker2 # [ 22.504077] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513633worker1 # [ 22.503590] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513634worker2 # [ 22.509677] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3635worker1 # [ 22.508724] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3636worker1 # [ 22.705297] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513637worker2 # [ 22.710958] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513638worker1 # [ 22.711168] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3639worker2 # [ 22.716542] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3640worker1 # [ 22.915115] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513641worker2 # [ 22.919876] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513642worker1 # [ 22.919549] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3643worker2 # [ 22.924370] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3644worker1 # [ 23.115651] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513645worker2 # [ 23.120057] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513646worker2 # [ 23.125705] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3647worker1 # [ 23.121911] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3648worker2 # [ 23.326508] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513649worker1 # [ 23.323214] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500513650worker2 # [ 23.332113] nix-grpc-daemon[777]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3651worker1 # [ 23.329125] nix-grpc-daemon[778]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3652worker2 # [ 23.533247] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513653worker1 # [ 23.530929] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513654worker2 # [ 23.539025] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3655worker1 # [ 23.536815] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3656worker2 # [ 23.739894] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513657worker1 # [ 23.737895] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513658worker2 # [ 23.746189] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3659worker1 # [ 23.743807] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3660worker2 # [ 23.949738] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513661worker1 # [ 23.947195] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513662worker2 # [ 23.954087] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3663worker1 # [ 23.951770] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker2 # [ 24.150546] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513665worker1 # [ 24.147577] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513666worker2 # [ 24.156568] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3667worker1 # [ 24.153435] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker2 # [ 24.357688] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 24.354171] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500513670worker1 # [ 24.359792] nix-grpc-daemon[778]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3671worker2 # [ 24.363471] nix-grpc-daemon[777]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker2 # [ 24.564342] nix-grpc-daemon[777]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500513673worker1 # [ 24.561159] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500513674worker2 # [ 24.567584] nix-grpc-daemon[777]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3675worker1 # [ 24.567408] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker2 # [ 24.768941] nix-grpc-daemon[777]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500513677worker1 # [ 24.768458] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500513678worker2 # [ 24.775488] nix-grpc-daemon[777]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3679worker1 # [ 24.774008] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker2 # [ 24.979496] nix-grpc-daemon[777]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500513681worker1 # [ 24.977304] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500513682worker2 # [ 24.984200] nix-grpc-daemon[777]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3683worker1 # [ 24.981711] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker1 # [ 25.177710] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500513685worker2 # [ 25.180738] nix-grpc-daemon[777]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500513686worker2 # [ 25.186766] nix-grpc-daemon[777]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3687worker1 # [ 25.183500] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker1 # [ 25.384656] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500513689worker2 # [ 25.387641] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500513690worker2 # [ 25.393412] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3691worker1 # [ 25.390352] nix-grpc-daemon[778]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker2 # [ 25.596626] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500513693worker1 # [ 25.592158] nix-grpc-daemon[778]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500513694worker1 # [ 25.597837] nix-grpc-daemon[778]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3695worker2 # [ 25.604394] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3696worker2 # [ 25.798099] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500513697worker1 # [ 25.794670] nix-grpc-daemon[778]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500513698worker2 # [ 25.803959] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3699worker1 # [ 25.800453] nix-grpc-daemon[778]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker1 # [ 26.002528] nix-grpc-daemon[778]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500513701worker2 # [ 26.006573] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500513702worker1 # [ 26.007856] nix-grpc-daemon[778]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3703worker2 # [ 26.012155] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3704worker1 # [ 26.209301] nix-grpc-daemon[778]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500513705worker2 # [ 26.214810] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500513706worker1 # [ 26.214941] nix-grpc-daemon[778]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3707worker2 # [ 26.219323] nix-grpc-daemon[777]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3708worker2 # [ 26.416167] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513709worker1 # [ 26.416097] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513710worker2 # [ 26.421917] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3711worker1 # [ 26.421723] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3712worker2 # [ 26.622658] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513713worker1 # [ 26.623087] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513714worker2 # [ 26.629102] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3715worker1 # [ 26.629179] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716worker2 # [ 26.829756] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513717worker1 # [ 26.830769] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513718worker2 # [ 26.835438] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3719worker1 # [ 26.836696] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3720worker2 # [ 27.039796] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513721worker1 # [ 27.040691] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513722worker2 # [ 27.044687] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3723worker1 # [ 27.045046] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3724worker2 # [ 27.241468] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513725worker1 # [ 27.241622] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_connected addr=lb:500513726worker2 # [ 27.248193] nix-grpc-daemon[777]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker1 # [ 27.247451] nix-grpc-daemon[778]: ts=2026-09-20T10:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3728worker2 # [ 27.449303] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513729worker1 # [ 27.448900] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513730worker2 # [ 27.456199] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker1 # [ 27.455116] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3732worker2 # [ 27.657343] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 27.655794] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513734worker2 # [ 27.663177] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker1 # [ 27.661535] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3736worker2 # [ 27.864474] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513737worker1 # [ 27.862307] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513738worker2 # [ 27.870436] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3739worker1 # [ 27.868073] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3740worker2 # [ 28.073954] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 28.071671] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513742worker2 # [ 28.079484] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3743worker1 # [ 28.076301] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744worker1 # [ 28.272133] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513745worker2 # [ 28.280646] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_connected addr=lb:500513746worker1 # [ 28.277617] nix-grpc-daemon[778]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3747worker2 # [ 28.286207] nix-grpc-daemon[777]: ts=2026-09-20T10:51:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3748worker1 # [ 28.479042] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513749worker2 # [ 28.487581] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513750worker1 # [ 28.485395] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751worker2 # [ 28.493799] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3752worker1 # [ 28.686475] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513753worker2 # [ 28.694600] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513754worker1 # [ 28.691954] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3755worker2 # [ 28.700515] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3756worker1 # [ 28.893095] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513757worker1 # [ 28.898466] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3758worker2 # [ 28.903211] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513759worker2 # [ 28.910409] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3760worker1 # [ 29.102546] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513761worker1 # [ 29.106981] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3762worker2 # [ 29.114162] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513763worker2 # [ 29.118603] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3764worker1 # [ 29.302808] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513765worker1 # [ 29.307078] nix-grpc-daemon[778]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3766worker2 # [ 29.314586] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_connected addr=lb:500513767worker2 # [ 29.320896] nix-grpc-daemon[777]: ts=2026-09-20T10:51:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3768worker1 # [ 29.508437] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513769worker1 # [ 29.513952] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3770worker2 # [ 29.521699] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513771worker2 # [ 29.527420] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3772worker1 # [ 29.715337] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513773worker1 # [ 29.721074] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774worker2 # [ 29.728871] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513775worker2 # [ 29.734413] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3776worker1 # [ 29.922010] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513777worker1 # [ 29.927727] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3778worker2 # [ 29.935507] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513779worker2 # [ 29.941346] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3780worker1 # [ 30.131533] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513781worker1 # [ 30.136163] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3782worker2 # [ 30.145921] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513783worker2 # [ 30.150332] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3784worker1 # [ 30.332051] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513785worker1 # [ 30.337572] nix-grpc-daemon[778]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3786worker2 # [ 30.346072] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500513787worker2 # [ 30.351649] nix-grpc-daemon[777]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3788worker1 # [ 30.539311] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513789worker1 # [ 30.545197] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3790worker2 # [ 30.553011] nix-grpc-daemon[777]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513791worker2 # [ 30.558814] nix-grpc-daemon[777]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3792worker1 # [ 30.746138] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513793worker1 # [ 30.751685] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3794worker2 # [ 30.760270] nix-grpc-daemon[777]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513795worker2 # [ 30.766104] nix-grpc-daemon[777]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3796worker1 # [ 30.952861] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513797worker1 # [ 30.958635] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3798worker2 # [ 30.967521] nix-grpc-daemon[777]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513799worker2 # [ 30.973338] nix-grpc-daemon[777]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3800worker1 # [ 31.161598] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513801worker1 # [ 31.165785] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3802worker2 # [ 31.176123] nix-grpc-daemon[777]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513803worker2 # [ 31.180286] nix-grpc-daemon[777]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3804worker1 # [ 31.362614] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513805worker1 # [ 31.367841] nix-grpc-daemon[778]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3806worker2 # [ 31.376448] nix-grpc-daemon[777]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500513807worker2 # [ 31.381611] nix-grpc-daemon[777]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3808worker1 # [ 31.568678] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500513809worker1 # [ 31.573804] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3810worker2 # [ 31.582427] nix-grpc-daemon[777]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500513811worker2 # [ 31.587978] nix-grpc-daemon[777]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3812worker1 # [ 31.774719] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500513813worker1 # [ 31.780123] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3814worker2 # [ 31.788877] nix-grpc-daemon[777]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500513815worker2 # [ 31.794199] nix-grpc-daemon[777]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3816worker1 # [ 31.980689] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500513817worker1 # [ 31.985836] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3818worker2 # [ 31.994976] nix-grpc-daemon[777]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500513819worker2 # [ 32.000352] nix-grpc-daemon[777]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3820worker1 # [ 32.188557] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500513821worker1 # [ 32.192853] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3822worker2 # [ 32.203106] nix-grpc-daemon[777]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500513823worker2 # [ 32.207348] nix-grpc-daemon[777]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3824worker1 # [ 32.389317] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500513825worker1 # [ 32.394474] nix-grpc-daemon[778]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3826worker2 # [ 32.403494] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500513827worker2 # [ 32.408946] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3828worker1 # [ 32.595367] nix-grpc-daemon[778]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500513829worker1 # [ 32.599209] nix-grpc-daemon[778]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3830worker2 # [ 32.609767] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500513831worker2 # [ 32.615096] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3832lb # [ 32.769551] rustfs-bucket-start[1033]: mb s3://farm3833lb # [ 32.783058] systemd[1]: Finished rustfs-bucket.service.3834lb # [ 32.793344] systemd[1]: Starting niks3 server...3835worker1 # [ 32.800061] nix-grpc-daemon[778]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500513836worker1 # [ 32.805789] nix-grpc-daemon[778]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3837worker2 # [ 32.815873] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500513838worker2 # [ 32.821144] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3839worker1 # [ 33.006841] nix-grpc-daemon[778]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500513840worker1 # [ 33.012642] nix-grpc-daemon[778]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3841worker2 # [ 33.022313] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500513842worker2 # [ 33.027942] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3843lb # [ 33.067419] postgres[1053]: [1053] ERROR: relation "goose_db_version" does not exist at character 363844lb # [ 33.072241] postgres[1053]: [1053] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3845lb # [ 33.099950] niks3-server[1042]: 2026/09/20 10:51:12 OK 20241026095416_initial_model.sql (18.55ms)3846lb # [ 33.103378] niks3-server[1042]: 2026/09/20 10:51:12 OK 20251210153512_drop_unused_gin_index.sql (1.83ms)3847lb # [ 33.108788] niks3-server[1042]: 2026/09/20 10:51:12 OK 20251218171726_add_pins.sql (5.34ms)3848lb # [ 33.113501] niks3-server[1042]: 2026/09/20 10:51:12 OK 20260628120000_add_object_size_and_stats.sql (4.67ms)3849lb # [ 33.118973] niks3-server[1042]: 2026/09/20 10:51:12 OK 20260905000000_add_claims.sql (5.43ms)3850lb # [ 33.123616] niks3-server[1042]: 2026/09/20 10:51:12 OK 20260920000000_drop_claims.sql (4.51ms)3851lb # [ 33.125799] niks3-server[1042]: 2026/09/20 10:51:12 goose: successfully migrated database to version: 202609200000003852lb # [ 33.130501] niks3-server[1042]: 2026/09/20 10:51:12 OK 1_commit_pending_closure.sql (6.87ms)3853lb # [ 33.133216] niks3-server[1042]: 2026/09/20 10:51:12 OK 2_object_stats_trigger.sql (2.58ms)3854lb # [ 33.134578] niks3-server[1042]: 2026/09/20 10:51:12 goose: up to current file version: 23855lb # [ 33.138336] niks3-server[1042]: 2026/09/20 10:51:12 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3856lb # [ 33.181442] niks3-server[1042]: 2026/09/20 10:51:12 INFO Read proxy enabled — serving cache objects from S33857lb # [ 33.183066] niks3-server[1042]: 2026/09/20 10:51:12 INFO Using socket-activated listener address=0.0.0.0:57513858lb # [ 33.186952] systemd[1]: Started niks3 server.3859lb # [ 33.189218] niks3-server[1042]: 2026/09/20 10:51:12 INFO systemd watchdog enabled interval=15s3860lb # [ 33.191856] niks3-server[1042]: 2026/09/20 10:51:12 INFO Starting HTTP server address=0.0.0.0:57513861lb # [ 33.194680] systemd[1]: Reached target Multi-User System.3862lb # [ 33.198864] systemd[1]: Startup finished in 951ms (kernel) + 5.471s (initrd) + 26.766s (userspace) = 33.190s.3863lb # [ 33.202536] niks3-server[1042]: 2026/09/20 10:51:12 INFO lead: acquired remote=192.168.1.3:463503864worker1 # [ 33.209011] nix-grpc-daemon[778]: ts=2026-09-20T10:51:12Z level=info event=scheduler_take_over3865worker1 # [ 33.214044] nix-grpc-daemon[778]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500513866worker1 # [ 33.219296] nix-grpc-daemon[778]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3867worker2 # [ 33.230674] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500513868worker2 # [ 33.234849] nix-grpc-daemon[777]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3869worker1 # [ 33.420207] nix-grpc-daemon[778]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500513870worker1 # [ 33.425410] nix-grpc-daemon[778]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3871worker2 # [ 33.431124] nix-grpc-daemon[777]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500513872worker2 # [ 33.436439] nix-grpc-daemon[777]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3873worker1 # [ 33.626184] nix-grpc-daemon[778]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500513874worker1 # [ 33.631498] nix-grpc-daemon[778]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3875worker2 # [ 33.637235] nix-grpc-daemon[777]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500513876worker2 # [ 33.642403] nix-grpc-daemon[777]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3877worker1 # [ 33.832686] nix-grpc-daemon[778]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500513878worker2 # [ 33.843065] nix-grpc-daemon[777]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500513879worker1 # [ 33.839365] nix-grpc-daemon[778]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3880worker1 # [ 33.863697] nix-grpc-daemon[778]: ts=2026-09-20T10:51:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03881lb: (finished: waiting for unit niks3.service, in 19.46 seconds)3882lb: waiting for TCP port 5751 on localhost3883worker1 # [ 34.040442] nix-grpc-daemon[778]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500513884worker1 # [ 34.046768] nix-grpc-daemon[778]: ts=2026-09-20T10:51:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03885lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3886lb: (finished: waiting for TCP port 5751 on localhost, in 0.09 seconds)3887lb: waiting for unit envoy.service3888lb: (finished: waiting for unit envoy.service, in 0.07 seconds)3889lb: waiting for TCP port 50051 on localhost3890lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3891lb # [ 34.221766] envoy[913]: conn peer=[::1]:49228 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33892lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3893lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3894lb: must succeed: curl -sf localhost:9901/clusters3895lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3896lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3897subtest: one node schedules and both builders hold a WorkerSession on it3898lb: must succeed: curl -sf localhost:9901/clusters3899lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3900worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3901worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3902lb: must succeed: curl -sf localhost:9901/clusters3903lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3904lb: must succeed: curl -sf localhost:9901/clusters3905lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3906worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3907worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3908lb: must succeed: curl -sf localhost:9901/clusters3909lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3910lb: must succeed: curl -sf localhost:9901/clusters3911lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3912worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3913worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3914worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3915worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3916worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3917worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3918worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3919worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3920(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.76 seconds)3921subtest: balancer auth: client cert, bearer token, nothing, unknown CN3922worker1 # [ 35.645510] nix-grpc-daemon[778]: ts=2026-09-20T10:51:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:503323923worker1 # [ 35.663120] systemd[1]: Started Nix Daemon.3924worker1 # [ 35.989071] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)3925worker1 # [ 36.028296] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5)3926worker1 # [ 36.086679] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 2/5)3927worker1 # [ 36.176205] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 3/5)3928worker1 # [ 36.301705] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 145 ms (attempt 4/5)3929worker1 # [ 36.456736] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3930worker2 # [ 36.505644] nix-grpc-daemon[777]: ts=2026-09-20T10:51:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453943931worker2 # [ 36.523492] systemd[1]: Started Nix Daemon.3932worker2 # [ 36.855149] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3933worker2 # [ 36.885494] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5)3934worker2 # [ 36.976601] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 2/5)3935worker2 # [ 37.087872] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 346 ms (attempt 3/5)3936worker2 # [ 37.434427] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 204 ms (attempt 4/5)3937worker2 # [ 37.642574] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3938worker2 # [ 37.659870] nix-grpc-daemon[777]: ts=2026-09-20T10:51:17Z level=debug event=path_info_miss path=xhp7hdmnyckpg762n4vd2n1aspzkff7a-nixos-system-client-test error="error: path '/nix/store/xhp7hdmnyckpg762n4vd2n1aspzkff7a-nixos-system-client-test' is required, but there is no substituter that can build it"3939worker2 # [ 37.674504] nix-grpc-daemon[777]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453943940worker2 # [ 37.687160] nix-grpc-daemon[777]: ts=2026-09-20T10:51:17Z level=debug event=path_info_miss path=xhp7hdmnyckpg762n4vd2n1aspzkff7a-nixos-system-client-test error="error: path '/nix/store/xhp7hdmnyckpg762n4vd2n1aspzkff7a-nixos-system-client-test' is required, but there is no substituter that can build it"3941client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3942client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.12 seconds)3943client: must succeed: install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3944client: (finished: must succeed: install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.06 seconds)3945worker2 # [ 38.156258] nix-grpc-daemon[777]: ts=2026-09-20T10:51:17Z level=info event=oidc_jwks provider=mock keys=13946worker2 # [ 38.162933] nix-grpc-daemon[777]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:453943947worker2 # [ 38.178570] nix-grpc-daemon[777]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:453943948worker2 # [ 38.191263] nix-grpc-daemon[777]: ts=2026-09-20T10:51:17Z level=debug event=path_info_miss path=xhp7hdmnyckpg762n4vd2n1aspzkff7a-nixos-system-client-test error="error: path '/nix/store/xhp7hdmnyckpg762n4vd2n1aspzkff7a-nixos-system-client-test' is required, but there is no substituter that can build it"3949worker1 # [ 38.205323] nix-grpc-daemon[778]: ts=2026-09-20T10:51:17Z level=info event=oidc_jwks provider=mock keys=13950worker1 # [ 38.212557] nix-grpc-daemon[778]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:503323951worker1 # [ 38.225780] nix-grpc-daemon[778]: ts=2026-09-20T10:51:17Z level=debug event=path_info_miss path=xhp7hdmnyckpg762n4vd2n1aspzkff7a-nixos-system-client-test error="error: path '/nix/store/xhp7hdmnyckpg762n4vd2n1aspzkff7a-nixos-system-client-test' is required, but there is no substituter that can build it"3952client: must succeed: rm -r /var/lib/nix-grpc-store3953client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.05 seconds)3954worker1 # [ 38.536197] nix-grpc-daemon[778]: ts=2026-09-20T10:51:18Z level=info event=denied method=QueryMissing cn=- role=none3955worker2 # [ 38.791507] nix-grpc-daemon[777]: ts=2026-09-20T10:51:18Z level=info event=denied method=QueryMissing cn=stranger role=none3956(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.54 seconds)3957subtest: DAG build is scheduled across workers and lands in the cache3958client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23959worker2 # [ 39.036705] nix-grpc-daemon[777]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:453943960client # don't know how to build these paths:3961client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3962worker1 # [ 39.045883] nix-grpc-daemon[778]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503323963worker1 # [ 39.083825] nix-grpc-daemon[778]: ts=2026-09-20T10:51:18Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03964worker1 # [ 39.090052] nix-grpc-daemon[778]: ts=2026-09-20T10:51:18Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=13965worker2 # [ 39.133243] nix-grpc-daemon[777]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453943966worker2 # [ 39.154169] nix-grpc-daemon[777]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453943967client # copying 1 paths to 'lb:50051'...3968worker2 # [ 39.170994] nix-grpc-daemon[777]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453943969client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3970worker2 # [ 39.210130] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3971worker2 # [ 39.249083] nix-grpc-daemon[777]: ts=2026-09-20T10:51:18Z level=info event=niks3_push_started pid=8493972worker2 # [ 39.720845] nix-daemon[839]: accepted connection from pid 862, user nix-grpc-daemon (trusted)3973worker2 # [ 39.742406] nix-daemon[839]: reaped child process 865, status = succeeded3974worker2 # [ 39.919091] nix-daemon[839]: accepted connection from pid 867, user nix-grpc-daemon (trusted)3975worker2 # [ 39.937637] nix-daemon[839]: reaped child process 870, status = succeeded3976lb # [ 39.948467] niks3-server[1042]: 2026/09/20 10:51:19 INFO Received uploads request method=POST path=/api/pending_closures3977worker2 # [ 39.981263] nix-grpc-daemon[849]: time=2026-09-20T10:51:19.598Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3978worker2 # [ 39.983098] nix-grpc-daemon[849]: time=2026-09-20T10:51:19.599Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3979lb # [ 40.028783] niks3-server[1042]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3980lb # [ 40.038442] niks3-server[1042]: 2026/09/20 10:51:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3981lb # [ 40.046862] niks3-server[1042]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3982lb # [ 40.071136] niks3-server[1042]: 2026/09/20 10:51:19 INFO Signed narinfos id=1 count=13983worker2 # [ 40.079114] nix-grpc-daemon[849]: time=2026-09-20T10:51:19.698Z level=INFO msg="Uploading 1 narinfos"3984lb # [ 40.086763] niks3-server[1042]: 2026/09/20 10:51:19 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3985lb # [ 40.097418] niks3-server[1042]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3986lb # [ 40.112467] niks3-server[1042]: 2026/09/20 10:51:19 INFO Completed upload id=13987worker2 # [ 40.117088] nix-grpc-daemon[849]: time=2026-09-20T10:51:19.737Z level=INFO msg="Upload complete. (572ms)"3988worker2 # [ 40.132827] nix-grpc-daemon[777]: ts=2026-09-20T10:51:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_in=5043989worker2 # [ 40.150985] nix-grpc-daemon[777]: ts=2026-09-20T10:51:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453943990worker2 # [ 40.161542] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3991worker2 # [ 40.171050] nix-daemon[839]: reaped child process 847, status = succeeded3992client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3993worker2 # [ 40.180596] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3994client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3995lb # [ 40.402557] envoy[913]: conn peer=[2001:db8:1::1]:34354 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=20923996lb # [ 40.407575] envoy[913]: conn peer=[2001:db8:1::1]:34358 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=1023997lb # [ 40.412259] envoy[913]: conn peer=[2001:db8:1::1]:34362 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=123998lb # [ 40.419711] envoy[913]: conn peer=[2001:db8:1::1]:34372 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=133999client # farm-a> LOG-farm-a-t14000worker2 # [ 41.217314] nix-daemon[839]: reaped child process 874, status = succeeded4001worker2 # [ 41.389007] nix-daemon[839]: accepted connection from pid 889, user nix-grpc-daemon (trusted)4002worker2 # [ 41.409100] nix-daemon[839]: reaped child process 892, status = succeeded4003lb # [ 41.413362] niks3-server[1042]: 2026/09/20 10:51:21 INFO Received uploads request method=POST path=/api/pending_closures4004worker2 # [ 41.435664] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.054Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4005worker2 # [ 41.438531] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.054Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4006lb # [ 41.475639] niks3-server[1042]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4007worker2 # [ 41.495494] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.113Z level=INFO msg="Uploading 1 narinfos"4008lb # [ 41.491852] niks3-server[1042]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4009lb # [ 41.499801] niks3-server[1042]: 2026/09/20 10:51:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4010lb # [ 41.507390] niks3-server[1042]: 2026/09/20 10:51:21 INFO Signed narinfos id=2 count=14011lb # [ 41.515777] niks3-server[1042]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4012lb # [ 41.519759] niks3-server[1042]: 2026/09/20 10:51:21 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4013lb # [ 41.524958] niks3-server[1042]: 2026/09/20 10:51:21 INFO Completed upload id=24014worker2 # [ 41.529020] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.149Z level=INFO msg="Upload complete. (336ms)"4015lb # [ 41.527834] niks3-server[1042]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4016worker2 # [ 41.541291] nix-grpc-daemon[777]: ts=2026-09-20T10:51:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14017worker1 # [ 41.550598] nix-grpc-daemon[778]: ts=2026-09-20T10:51:21Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04018worker2 # [ 41.562386] nix-daemon[839]: reaped child process 872, status = succeeded4019worker1 # [ 41.557022] nix-grpc-daemon[778]: ts=2026-09-20T10:51:21Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=14020worker2 # [ 41.606706] nix-grpc-daemon[777]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944021worker2 # [ 41.622913] nix-grpc-daemon[777]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944022worker2 # [ 41.639344] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.259Z level=INFO msg="All 1 paths already cached"4023client # copying 1 paths to 'lb:50051'...4024worker2 # [ 41.649787] nix-grpc-daemon[777]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453944025client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4026worker2 # [ 41.672744] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4027worker2 # [ 41.884853] nix-daemon[839]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4028worker2 # [ 41.904298] nix-daemon[839]: reaped child process 899, status = succeeded4029worker2 # [ 42.078740] nix-daemon[839]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4030worker2 # [ 42.097407] nix-daemon[839]: reaped child process 904, status = succeeded4031lb # [ 42.104409] niks3-server[1042]: 2026/09/20 10:51:21 INFO Received uploads request method=POST path=/api/pending_closures4032worker2 # [ 42.119296] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.738Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4033worker2 # [ 42.121010] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.739Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4034lb # [ 42.134427] niks3-server[1042]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4035worker2 # [ 42.153485] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.773Z level=INFO msg="Uploading 1 narinfos"4036lb # [ 42.150637] niks3-server[1042]: 2026/09/20 10:51:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4037lb # [ 42.162433] niks3-server[1042]: 2026/09/20 10:51:21 INFO Signed narinfos id=3 count=14038lb # [ 42.168411] niks3-server[1042]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4039lb # [ 42.174491] niks3-server[1042]: 2026/09/20 10:51:21 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4040lb # [ 42.187963] niks3-server[1042]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4041lb # [ 42.192989] niks3-server[1042]: 2026/09/20 10:51:21 INFO Completed upload id=34042worker2 # [ 42.197226] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.817Z level=INFO msg="Upload complete. (506ms)"4043worker2 # [ 42.208601] nix-grpc-daemon[777]: ts=2026-09-20T10:51:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_in=6244044worker2 # [ 42.223949] nix-grpc-daemon[777]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944045worker2 # [ 42.236064] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4046worker2 # [ 42.238660] nix-daemon[839]: reaped child process 894, status = succeeded4047client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4048client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4049worker2 # [ 42.258852] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4050client # farm-b> LOG-farm-b-t14051worker2 # [ 43.493297] nix-daemon[839]: reaped child process 908, status = succeeded4052worker2 # [ 43.654447] nix-daemon[839]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4053worker2 # [ 43.675349] nix-daemon[839]: reaped child process 927, status = succeeded4054lb # [ 43.684699] niks3-server[1042]: 2026/09/20 10:51:23 INFO Received uploads request method=POST path=/api/pending_closures4055worker2 # [ 43.699540] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.318Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4056worker2 # [ 43.701445] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.318Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4057lb # [ 43.720914] niks3-server[1042]: 2026/09/20 10:51:23 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4058lb # [ 43.734684] niks3-server[1042]: 2026/09/20 10:51:23 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4059lb # [ 43.746982] niks3-server[1042]: 2026/09/20 10:51:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4060worker2 # [ 43.755722] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.375Z level=INFO msg="Uploading 1 narinfos"4061lb # [ 43.754511] niks3-server[1042]: 2026/09/20 10:51:23 INFO Signed narinfos id=4 count=14062lb # [ 43.760280] niks3-server[1042]: 2026/09/20 10:51:23 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4063lb # [ 43.769855] niks3-server[1042]: 2026/09/20 10:51:23 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4064lb # [ 43.777735] niks3-server[1042]: 2026/09/20 10:51:23 INFO Completed upload id=44065worker2 # [ 43.781987] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.402Z level=INFO msg="Upload complete. (303ms)"4066lb # [ 43.780549] niks3-server[1042]: 2026/09/20 10:51:23 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4067worker1 # [ 43.795384] nix-grpc-daemon[778]: ts=2026-09-20T10:51:23Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04068worker1 # [ 43.801688] nix-grpc-daemon[778]: ts=2026-09-20T10:51:23Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=14069worker2 # [ 43.793151] nix-grpc-daemon[777]: ts=2026-09-20T10:51:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14070worker2 # [ 43.813025] nix-daemon[839]: reaped child process 906, status = succeeded4071worker2 # [ 43.849823] nix-grpc-daemon[777]: ts=2026-09-20T10:51:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944072worker2 # [ 43.866759] nix-grpc-daemon[777]: ts=2026-09-20T10:51:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944073worker2 # [ 43.882288] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.502Z level=INFO msg="All 2 paths already cached"4074client # copying 1 paths to 'lb:50051'...4075worker2 # [ 43.892895] nix-grpc-daemon[777]: ts=2026-09-20T10:51:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453944076client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4077worker2 # [ 43.912246] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4078worker2 # [ 44.129558] nix-daemon[839]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4079worker2 # [ 44.148333] nix-daemon[839]: reaped child process 934, status = succeeded4080worker2 # [ 44.320951] nix-daemon[839]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4081worker2 # [ 44.340345] nix-daemon[839]: reaped child process 939, status = succeeded4082lb # [ 44.348733] niks3-server[1042]: 2026/09/20 10:51:23 INFO Received uploads request method=POST path=/api/pending_closures4083worker2 # [ 44.363274] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.983Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4084worker2 # [ 44.365021] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.983Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4085lb # [ 44.379173] niks3-server[1042]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4086worker2 # [ 44.397227] nix-grpc-daemon[849]: time=2026-09-20T10:51:24.017Z level=INFO msg="Uploading 1 narinfos"4087lb # [ 44.394869] niks3-server[1042]: 2026/09/20 10:51:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4088lb # [ 44.406999] niks3-server[1042]: 2026/09/20 10:51:24 INFO Signed narinfos id=5 count=14089lb # [ 44.413117] niks3-server[1042]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4090lb # [ 44.422631] niks3-server[1042]: 2026/09/20 10:51:24 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4091lb # [ 44.429429] niks3-server[1042]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4092lb # [ 44.433781] niks3-server[1042]: 2026/09/20 10:51:24 INFO Completed upload id=54093worker2 # [ 44.438091] nix-grpc-daemon[849]: time=2026-09-20T10:51:24.058Z level=INFO msg="Upload complete. (503ms)"4094worker2 # [ 44.446451] nix-grpc-daemon[777]: ts=2026-09-20T10:51:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_in=7524095worker2 # [ 44.463389] nix-grpc-daemon[777]: ts=2026-09-20T10:51:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944096worker2 # [ 44.469767] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4097worker2 # [ 44.474850] nix-daemon[839]: reaped child process 929, status = succeeded4098client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4099client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4100worker2 # [ 44.491071] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4101worker2 # [ 45.915539] nix-daemon[839]: reaped child process 943, status = succeeded4102client # farm-top> LOG-farm-top-t14103worker2 # [ 46.074953] nix-daemon[839]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4104worker2 # [ 46.096400] nix-daemon[839]: reaped child process 962, status = succeeded4105lb # [ 46.105407] niks3-server[1042]: 2026/09/20 10:51:25 INFO Received uploads request method=POST path=/api/pending_closures4106worker2 # [ 46.127219] nix-grpc-daemon[849]: time=2026-09-20T10:51:25.745Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4107worker2 # [ 46.128906] nix-grpc-daemon[849]: time=2026-09-20T10:51:25.745Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4108lb # [ 46.160422] niks3-server[1042]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4109lb # [ 46.177352] niks3-server[1042]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4110lb # [ 46.191232] niks3-server[1042]: 2026/09/20 10:51:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4111worker2 # [ 46.198852] nix-grpc-daemon[849]: time=2026-09-20T10:51:25.818Z level=INFO msg="Uploading 1 narinfos"4112lb # [ 46.198186] niks3-server[1042]: 2026/09/20 10:51:25 INFO Signed narinfos id=6 count=14113lb # [ 46.202287] niks3-server[1042]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4114lb # [ 46.207138] niks3-server[1042]: 2026/09/20 10:51:25 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4115lb # [ 46.213852] niks3-server[1042]: 2026/09/20 10:51:25 INFO Completed upload id=64116worker2 # [ 46.217920] nix-grpc-daemon[849]: time=2026-09-20T10:51:25.838Z level=INFO msg="Upload complete. (315ms)"4117lb # [ 46.216332] niks3-server[1042]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4118worker2 # [ 46.229306] nix-grpc-daemon[777]: ts=2026-09-20T10:51:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14119worker2 # [ 46.248766] nix-daemon[839]: reaped child process 941, status = succeeded4120client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.47 seconds)4121client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4122client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.24 seconds)4123client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4124client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4125client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4126client # copying 3 paths...4127client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4128client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4129client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4130client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4131(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.14 seconds)4132subtest: repeat is answered Cached by the scheduler without building4133worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4134worker1 # finding garbage collector roots...4135worker1 # deleting specified paths...4136worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4137worker1 # deleting unused links...4138worker1 # note: hard linking is currently saving 0.0 KiB4139worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.29 seconds)4140worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4141worker2 # finding garbage collector roots...4142worker2 # deleting specified paths...4143worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4144worker2 # deleting unused links...4145worker2 # note: hard linking is currently saving 0.0 KiB4146worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4147client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24148worker2 # [ 47.779312] nix-grpc-daemon[777]: ts=2026-09-20T10:51:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:453944149client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4150client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4151worker2 # [ 47.797962] nix-grpc-daemon[777]: ts=2026-09-20T10:51:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944152worker2 # [ 47.809780] nix-grpc-daemon[849]: time=2026-09-20T10:51:27.428Z level=INFO msg="All 2 paths already cached"4153worker1 # [ 47.825847] nix-grpc-daemon[778]: ts=2026-09-20T10:51:27Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14154client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.34 seconds)4155client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4156client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.22 seconds)4157lb: must succeed: curl -sf localhost:9901/clusters4158lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4159lb: must succeed: curl -sf localhost:9901/clusters4160lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4161worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4162worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4163(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.46 seconds)4164subtest: inputs already in the cache are not sent to the scheduler4165lb: must succeed: curl -sf localhost:9901/clusters4166lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4167lb: must succeed: curl -sf localhost:9901/clusters4168lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4169worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4170worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4171client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24172worker1 # [ 48.859884] nix-grpc-daemon[778]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:503324173client # don't know how to build these paths:4174client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4175worker2 # [ 48.874634] nix-grpc-daemon[777]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944176worker2 # [ 48.888696] nix-grpc-daemon[849]: time=2026-09-20T10:51:28.507Z level=INFO msg="All 2 paths already cached"4177worker1 # [ 48.901987] nix-grpc-daemon[778]: ts=2026-09-20T10:51:28Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04178worker1 # [ 48.908577] nix-grpc-daemon[778]: ts=2026-09-20T10:51:28Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14179worker2 # [ 48.953272] nix-grpc-daemon[777]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944180worker2 # [ 48.969642] nix-grpc-daemon[777]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944181worker2 # [ 48.984978] nix-grpc-daemon[849]: time=2026-09-20T10:51:28.605Z level=INFO msg="All 2 paths already cached"4182client # copying 1 paths to 'lb:50051'...4183client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4184worker2 # [ 48.995458] nix-grpc-daemon[777]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453944185worker2 # [ 49.010703] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4186worker2 # [ 49.215213] nix-daemon[839]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4187worker2 # [ 49.232841] nix-daemon[839]: reaped child process 982, status = succeeded4188worker2 # [ 49.409186] nix-daemon[839]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4189worker2 # [ 49.426882] nix-daemon[839]: reaped child process 987, status = succeeded4190lb # [ 49.435962] niks3-server[1042]: 2026/09/20 10:51:29 INFO Received uploads request method=POST path=/api/pending_closures4191worker2 # [ 49.449428] nix-grpc-daemon[849]: time=2026-09-20T10:51:29.069Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4192worker2 # [ 49.450870] nix-grpc-daemon[849]: time=2026-09-20T10:51:29.069Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4193lb # [ 49.466650] niks3-server[1042]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4194lb # [ 49.477092] niks3-server[1042]: 2026/09/20 10:51:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4195worker2 # [ 49.488921] nix-grpc-daemon[849]: time=2026-09-20T10:51:29.108Z level=INFO msg="Uploading 1 narinfos"4196lb # [ 49.488492] niks3-server[1042]: 2026/09/20 10:51:29 INFO Signed narinfos id=7 count=14197lb # [ 49.491176] niks3-server[1042]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4198lb # [ 49.502138] niks3-server[1042]: 2026/09/20 10:51:29 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4199lb # [ 49.508991] niks3-server[1042]: 2026/09/20 10:51:29 INFO Completed upload id=74200worker2 # [ 49.513196] nix-grpc-daemon[849]: time=2026-09-20T10:51:29.133Z level=INFO msg="Upload complete. (484ms)"4201lb # [ 49.511741] niks3-server[1042]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4202worker2 # [ 49.524112] nix-grpc-daemon[777]: ts=2026-09-20T10:51:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_in=7524203worker2 # [ 49.534672] nix-grpc-daemon[777]: ts=2026-09-20T10:51:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944204worker2 # [ 49.542900] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4205worker2 # [ 49.547307] nix-daemon[839]: reaped child process 977, status = succeeded4206client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4207worker2 # [ 49.557618] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4208client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4209worker2 # [ 50.380964] nix-daemon[839]: reaped child process 991, status = succeeded4210client # farm-top> LOG-farm-top-t1b4211lb # [ 50.404411] envoy[913]: conn peer=[2001:db8:1::1]:34376 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9553 tx=3858 ms=72174212lb # [ 50.404790] envoy[913]: conn peer=[2001:db8:1::1]:37324 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=744213worker2 # [ 50.545628] nix-daemon[839]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4214worker2 # [ 50.565402] nix-daemon[839]: reaped child process 1009, status = succeeded4215lb # [ 50.574030] niks3-server[1042]: 2026/09/20 10:51:30 INFO Received uploads request method=POST path=/api/pending_closures4216worker2 # [ 50.588461] nix-grpc-daemon[849]: time=2026-09-20T10:51:30.207Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4217worker2 # [ 50.589955] nix-grpc-daemon[849]: time=2026-09-20T10:51:30.207Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4218lb # [ 50.614034] niks3-server[1042]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4219lb # [ 50.633611] niks3-server[1042]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4220worker2 # [ 50.645111] nix-grpc-daemon[849]: time=2026-09-20T10:51:30.264Z level=INFO msg="Uploading 1 narinfos"4221lb # [ 50.644600] niks3-server[1042]: 2026/09/20 10:51:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4222lb # [ 50.653917] niks3-server[1042]: 2026/09/20 10:51:30 INFO Signed narinfos id=8 count=14223lb # [ 50.659051] niks3-server[1042]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4224worker2 # [ 50.672954] nix-grpc-daemon[849]: time=2026-09-20T10:51:30.293Z level=INFO msg="Upload complete. (304ms)"4225lb # [ 50.669302] niks3-server[1042]: 2026/09/20 10:51:30 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4226lb # [ 50.673130] niks3-server[1042]: 2026/09/20 10:51:30 INFO Completed upload id=84227lb # [ 50.677242] niks3-server[1042]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4228worker2 # [ 50.683848] nix-grpc-daemon[777]: ts=2026-09-20T10:51:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14229worker2 # [ 50.702752] nix-daemon[839]: reaped child process 989, status = succeeded4230client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.12 seconds)4231client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4232client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.24 seconds)4233lb: must succeed: curl -sf localhost:9901/clusters4234lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4235lb: must succeed: curl -sf localhost:9901/clusters4236lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4237worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4238worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4239(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.76 seconds)4240subtest: two clients wanting the same drv build it once4241client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4242client # [ 51.229040] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4243client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4244client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4245client # Running as unit: dedup1.service; invocation ID: 9476a1763b5b4ab788ca6cf383a6bf5f4246client # [ 51.480300] nix[880]: don't know how to build these paths:4247client # [ 51.482972] nix[880]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4248worker1 # [ 51.504095] nix-grpc-daemon[778]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:503324249worker1 # [ 51.519414] nix-grpc-daemon[778]: ts=2026-09-20T10:51:31Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04250worker1 # [ 51.527233] nix-grpc-daemon[778]: ts=2026-09-20T10:51:31Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14251worker1 # [ 51.535999] nix-grpc-daemon[778]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503324252worker1 # [ 51.556444] nix-grpc-daemon[778]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503324253client # [ 51.547060] nix[880]: copying 1 paths to 'lb:50051'...4254client # [ 51.549232] nix[880]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4255worker1 # [ 51.572744] nix-grpc-daemon[778]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:503324256worker1 # [ 51.589162] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4257worker1 # [ 51.615769] nix-grpc-daemon[778]: ts=2026-09-20T10:51:31Z level=info event=niks3_push_started pid=9024258worker1 # [ 52.064270] nix-daemon[851]: accepted connection from pid 915, user nix-grpc-daemon (trusted)4259worker1 # [ 52.086918] nix-daemon[851]: reaped child process 918, status = succeeded4260worker1 # [ 52.264719] nix-daemon[851]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4261worker1 # [ 52.287160] nix-daemon[851]: reaped child process 923, status = succeeded4262lb # [ 52.299759] niks3-server[1042]: 2026/09/20 10:51:31 INFO Received uploads request method=POST path=/api/pending_closures4263worker1 # [ 52.313003] nix-grpc-daemon[902]: time=2026-09-20T10:51:31.899Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4264worker1 # [ 52.314587] nix-grpc-daemon[902]: time=2026-09-20T10:51:31.899Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4265lb # [ 52.338322] niks3-server[1042]: 2026/09/20 10:51:31 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4266worker1 # [ 52.359518] nix-grpc-daemon[902]: time=2026-09-20T10:51:31.947Z level=INFO msg="Uploading 1 narinfos"4267lb # [ 52.353221] niks3-server[1042]: 2026/09/20 10:51:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4268lb # [ 52.363253] niks3-server[1042]: 2026/09/20 10:51:31 INFO Signed narinfos id=9 count=14269lb # [ 52.372789] niks3-server[1042]: 2026/09/20 10:51:31 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4270client # [ 52.346877] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4271lb # [ 52.384793] niks3-server[1042]: 2026/09/20 10:51:32 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4272client # Running as unit: dedup2.service; invocation ID: 336a124789e242b18a04ad902471f7344273worker1 # [ 52.392650] nix-grpc-daemon[902]: time=2026-09-20T10:51:31.980Z level=INFO msg="Upload complete. (515ms)"4274lb # [ 52.393446] niks3-server[1042]: 2026/09/20 10:51:32 INFO Completed upload id=94275lb # [ 52.396773] niks3-server[1042]: 2026/09/20 10:51:32 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4276worker1 # [ 52.404321] nix-grpc-daemon[778]: ts=2026-09-20T10:51:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50332 duration_s=0 paths=1 nar_bytes_in=4324277client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4278worker1 # [ 52.415538] nix-grpc-daemon[778]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503324279worker1 # [ 52.426977] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4280worker1 # [ 52.432590] nix-daemon[851]: reaped child process 900, status = succeeded4281client # [ 52.427847] nix[880]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4282worker1 # [ 52.457437] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4283worker1 # [ 52.705530] nix-grpc-daemon[778]: ts=2026-09-20T10:51:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:503324284client # [ 52.696047] nix[896]: this derivation will be built:4285worker2 # [ 52.730787] nix-grpc-daemon[777]: ts=2026-09-20T10:51:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453944286worker2 # [ 52.744455] nix-grpc-daemon[777]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' from 'http://lb:5751'...4287client # [ 52.738280] nix[896]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4288worker1 # [ 52.778227] nix-grpc-daemon[778]: ts=2026-09-20T10:51:32Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04289worker1 # [ 52.787786] nix-grpc-daemon[778]: ts=2026-09-20T10:51:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503324290client # [ 52.770234] nix[896]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4291client: must succeed: sleep 24292client: (finished: must succeed: sleep 2, in 2.06 seconds)4293client: waiting for success: ! systemctl is-active dedup1 dedup24294worker1 # [ 56.113618] nix-daemon[851]: reaped child process 931, status = succeeded4295worker1 # [ 56.290978] nix-daemon[851]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4296worker1 # [ 56.309251] nix-daemon[851]: reaped child process 961, status = succeeded4297lb # [ 56.322013] niks3-server[1042]: 2026/09/20 10:51:35 INFO Received uploads request method=POST path=/api/pending_closures4298worker1 # [ 56.333047] nix-grpc-daemon[902]: time=2026-09-20T10:51:35.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4299worker1 # [ 56.334747] nix-grpc-daemon[902]: time=2026-09-20T10:51:35.919Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4300lb # [ 56.366457] niks3-server[1042]: 2026/09/20 10:51:35 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4301lb # [ 56.379766] niks3-server[1042]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4302lb # [ 56.388625] niks3-server[1042]: 2026/09/20 10:51:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4303lb # [ 56.394542] niks3-server[1042]: 2026/09/20 10:51:36 INFO Signed narinfos id=10 count=14304worker1 # [ 56.396196] nix-grpc-daemon[902]: time=2026-09-20T10:51:35.982Z level=INFO msg="Uploading 1 narinfos"4305lb # [ 56.397330] niks3-server[1042]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4306lb # [ 56.410064] niks3-server[1042]: 2026/09/20 10:51:36 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4307lb # [ 56.414637] niks3-server[1042]: 2026/09/20 10:51:36 INFO Completed upload id=104308worker1 # [ 56.415203] nix-grpc-daemon[902]: time=2026-09-20T10:51:36.003Z level=INFO msg="Upload complete. (341ms)"4309lb # [ 56.417035] niks3-server[1042]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4310worker1 # [ 56.433358] nix-grpc-daemon[778]: ts=2026-09-20T10:51:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50332 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14311worker1 # [ 56.447359] nix-grpc-daemon[778]: ts=2026-09-20T10:51:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50332 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14312worker1 # [ 56.453291] nix-daemon[851]: reaped child process 928, status = succeeded4313client # [ 56.461429] systemd[1]: dedup2.service: Deactivated successfully.4314client # [ 56.466943] systemd[1]: dedup2.service: Consumed 197ms CPU time over 4.119s wall clock time, 31M memory peak, 7.9K incoming IP traffic, 6.6K outgoing IP traffic.4315client # [ 56.477091] systemd[1]: dedup1.service: Deactivated successfully.4316client # [ 56.479092] systemd[1]: dedup1.service: Consumed 158ms CPU time over 5.245s wall clock time, 37.2M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.4317client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.15 seconds)4318client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4319client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4320worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4321worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4322worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4323worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4324worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4325worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4326worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4327worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4328(finished: subtest: two clients wanting the same drv build it once, in 6.31 seconds)4329subtest: build hook: nix-daemon with builders = grpc://lb4330client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4331client # [ 57.683689] systemd[1]: Started Nix Daemon.4332client # [ 57.874029] nix-daemon[938]: accepted connection from pid 935, user root (trusted)4333worker1 # [ 58.103605] nix-grpc-daemon[778]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:503324334worker2 # [ 58.113575] nix-grpc-daemon[777]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944335worker1 # [ 58.122493] nix-grpc-daemon[778]: ts=2026-09-20T10:51:37Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04336worker1 # [ 58.128650] nix-grpc-daemon[778]: ts=2026-09-20T10:51:37Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14337worker2 # [ 58.172143] nix-grpc-daemon[777]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944338worker2 # [ 58.188469] nix-grpc-daemon[777]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944339worker2 # [ 58.205593] nix-grpc-daemon[777]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453944340worker2 # [ 58.221362] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4341worker2 # [ 58.444336] nix-daemon[839]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)4342worker2 # [ 58.465273] nix-daemon[839]: reaped child process 1038, status = succeeded4343worker2 # [ 58.642567] nix-daemon[839]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)4344worker2 # [ 58.660657] nix-daemon[839]: reaped child process 1043, status = succeeded4345lb # [ 58.669183] niks3-server[1042]: 2026/09/20 10:51:38 INFO Received uploads request method=POST path=/api/pending_closures4346worker2 # [ 58.683540] nix-grpc-daemon[849]: time=2026-09-20T10:51:38.303Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4347worker2 # [ 58.685175] nix-grpc-daemon[849]: time=2026-09-20T10:51:38.303Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4348lb # [ 58.702363] niks3-server[1042]: 2026/09/20 10:51:38 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4349worker2 # [ 58.723104] nix-grpc-daemon[849]: time=2026-09-20T10:51:38.343Z level=INFO msg="Uploading 1 narinfos"4350lb # [ 58.718863] niks3-server[1042]: 2026/09/20 10:51:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4351lb # [ 58.729089] niks3-server[1042]: 2026/09/20 10:51:38 INFO Signed narinfos id=11 count=14352lb # [ 58.736196] niks3-server[1042]: 2026/09/20 10:51:38 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4353lb # [ 58.748202] niks3-server[1042]: 2026/09/20 10:51:38 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4354lb # [ 58.756485] niks3-server[1042]: 2026/09/20 10:51:38 INFO Completed upload id=114355worker2 # [ 58.759691] nix-grpc-daemon[849]: time=2026-09-20T10:51:38.379Z level=INFO msg="Upload complete. (517ms)"4356lb # [ 58.758658] niks3-server[1042]: 2026/09/20 10:51:38 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4357worker2 # [ 58.768361] nix-grpc-daemon[777]: ts=2026-09-20T10:51:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_in=5124358worker2 # [ 58.783939] nix-grpc-daemon[777]: ts=2026-09-20T10:51:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944359worker2 # [ 58.790793] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4360worker2 # [ 58.796851] nix-daemon[839]: reaped child process 1033, status = succeeded4361worker2 # [ 58.811599] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4362worker2 # [ 60.037081] nix-daemon[839]: reaped child process 1047, status = succeeded4363worker2 # [ 60.199570] nix-daemon[839]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)4364worker2 # [ 60.222388] nix-daemon[839]: reaped child process 1065, status = succeeded4365lb # [ 60.230861] niks3-server[1042]: 2026/09/20 10:51:39 INFO Received uploads request method=POST path=/api/pending_closures4366worker2 # [ 60.248520] nix-grpc-daemon[849]: time=2026-09-20T10:51:39.865Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4367worker2 # [ 60.250081] nix-grpc-daemon[849]: time=2026-09-20T10:51:39.865Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4368lb # [ 60.275246] niks3-server[1042]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4369lb # [ 60.288666] niks3-server[1042]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4370lb # [ 60.297741] niks3-server[1042]: 2026/09/20 10:51:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4371worker2 # [ 60.307456] nix-grpc-daemon[849]: time=2026-09-20T10:51:39.927Z level=INFO msg="Uploading 1 narinfos"4372lb # [ 60.307502] niks3-server[1042]: 2026/09/20 10:51:39 INFO Signed narinfos id=12 count=14373lb # [ 60.312807] niks3-server[1042]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4374lb # [ 60.321328] niks3-server[1042]: 2026/09/20 10:51:39 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4375lb # [ 60.327992] niks3-server[1042]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4376lb # [ 60.334073] niks3-server[1042]: 2026/09/20 10:51:39 INFO Completed upload id=124377worker2 # [ 60.338403] nix-grpc-daemon[849]: time=2026-09-20T10:51:39.958Z level=INFO msg="Upload complete. (315ms)"4378worker2 # [ 60.349325] nix-grpc-daemon[777]: ts=2026-09-20T10:51:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14379worker2 # [ 60.368614] nix-grpc-daemon[777]: ts=2026-09-20T10:51:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453944380worker2 # [ 60.373879] nix-daemon[839]: reaped child process 1045, status = succeeded4381worker2 # [ 60.392197] nix-grpc-daemon[777]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:453944382worker2 # [ 60.399238] nix-grpc-daemon[777]: ts=2026-09-20T10:51:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_out=1284383lb # [ 60.405638] envoy[913]: conn peer=[2001:db8:1::1]:37338 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4159 tx=1727 ms=18424384lb # [ 60.409309] envoy[913]: conn peer=[2001:db8:1::1]:35754 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1348 ms=49424385lb # [ 60.412815] envoy[913]: conn peer=[2001:db8:1::1]:35756 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1592 tx=1487 ms=37444386worker1 # [ 60.704307] nix-grpc-daemon[778]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:503324387worker2 # [ 60.712300] nix-grpc-daemon[777]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944388worker2 # [ 60.722436] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.342Z level=INFO msg="All 1 paths already cached"4389worker1 # [ 60.737578] nix-grpc-daemon[778]: ts=2026-09-20T10:51:40Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04390worker1 # [ 60.743859] nix-grpc-daemon[778]: ts=2026-09-20T10:51:40Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14391worker2 # [ 60.751091] nix-grpc-daemon[777]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944392worker2 # [ 60.766988] nix-grpc-daemon[777]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944393worker2 # [ 60.781903] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.401Z level=INFO msg="All 1 paths already cached"4394worker2 # [ 60.791998] nix-grpc-daemon[777]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453944395worker2 # [ 60.807555] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4396worker2 # [ 61.035670] nix-daemon[839]: accepted connection from pid 1069, user nix-grpc-daemon (trusted)4397worker2 # [ 61.057658] nix-daemon[839]: reaped child process 1072, status = succeeded4398worker2 # [ 61.236103] nix-daemon[839]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4399worker2 # [ 61.253364] nix-daemon[839]: reaped child process 1077, status = succeeded4400lb # [ 61.263766] niks3-server[1042]: 2026/09/20 10:51:40 INFO Received uploads request method=POST path=/api/pending_closures4401worker2 # [ 61.277846] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.897Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4402worker2 # [ 61.279414] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.897Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4403lb # [ 61.292504] niks3-server[1042]: 2026/09/20 10:51:40 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4404worker2 # [ 61.308026] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.927Z level=INFO msg="Uploading 1 narinfos"4405lb # [ 61.308702] niks3-server[1042]: 2026/09/20 10:51:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4406lb # [ 61.319229] niks3-server[1042]: 2026/09/20 10:51:40 INFO Signed narinfos id=13 count=14407lb # [ 61.323545] niks3-server[1042]: 2026/09/20 10:51:40 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4408lb # [ 61.331454] niks3-server[1042]: 2026/09/20 10:51:40 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4409lb # [ 61.335794] niks3-server[1042]: 2026/09/20 10:51:40 INFO Completed upload id=134410worker2 # [ 61.339352] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.959Z level=INFO msg="Upload complete. (512ms)"4411lb # [ 61.338301] niks3-server[1042]: 2026/09/20 10:51:40 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4412worker2 # [ 61.348254] nix-grpc-daemon[777]: ts=2026-09-20T10:51:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_in=6244413worker2 # [ 61.360439] nix-grpc-daemon[777]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944414worker2 # [ 61.365981] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4415worker2 # [ 61.370792] nix-daemon[839]: reaped child process 1067, status = succeeded4416worker2 # [ 61.384121] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4417worker2 # [ 62.022134] nix-daemon[839]: reaped child process 1081, status = succeeded4418worker2 # [ 62.205319] nix-daemon[839]: accepted connection from pid 1096, user nix-grpc-daemon (trusted)4419worker2 # [ 62.223843] nix-daemon[839]: reaped child process 1099, status = succeeded4420lb # [ 62.231237] niks3-server[1042]: 2026/09/20 10:51:41 INFO Received uploads request method=POST path=/api/pending_closures4421worker2 # [ 62.245426] nix-grpc-daemon[849]: time=2026-09-20T10:51:41.864Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4422worker2 # [ 62.246964] nix-grpc-daemon[849]: time=2026-09-20T10:51:41.864Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4423lb # [ 62.263809] niks3-server[1042]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4424lb # [ 62.280350] niks3-server[1042]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4425lb # [ 62.289623] niks3-server[1042]: 2026/09/20 10:51:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4426lb # [ 62.295925] niks3-server[1042]: 2026/09/20 10:51:41 INFO Signed narinfos id=14 count=14427worker2 # [ 62.300896] nix-grpc-daemon[849]: time=2026-09-20T10:51:41.920Z level=INFO msg="Uploading 1 narinfos"4428lb # [ 62.303563] niks3-server[1042]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4429lb # [ 62.313192] niks3-server[1042]: 2026/09/20 10:51:41 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4430lb # [ 62.320534] niks3-server[1042]: 2026/09/20 10:51:41 INFO Completed upload id=144431worker2 # [ 62.324754] nix-grpc-daemon[849]: time=2026-09-20T10:51:41.944Z level=INFO msg="Upload complete. (327ms)"4432lb # [ 62.323759] niks3-server[1042]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4433worker2 # [ 62.334877] nix-grpc-daemon[777]: ts=2026-09-20T10:51:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14434worker2 # [ 62.353224] nix-grpc-daemon[777]: ts=2026-09-20T10:51:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453944435worker2 # [ 62.357768] nix-daemon[839]: reaped child process 1079, status = succeeded4436worker2 # [ 62.371549] nix-grpc-daemon[777]: ts=2026-09-20T10:51:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:453944437worker2 # [ 62.376376] nix-grpc-daemon[777]: ts=2026-09-20T10:51:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_out=1764438worker2 # [ 62.674453] nix-grpc-daemon[777]: ts=2026-09-20T10:51:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:453944439worker1 # [ 62.679153] nix-grpc-daemon[778]: ts=2026-09-20T10:51:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503324440worker1 # [ 62.700828] nix-grpc-daemon[778]: ts=2026-09-20T10:51:42Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04441worker1 # [ 62.707283] nix-grpc-daemon[778]: ts=2026-09-20T10:51:42Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14442worker2 # [ 62.714069] nix-grpc-daemon[777]: ts=2026-09-20T10:51:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944443worker2 # [ 62.729747] nix-grpc-daemon[777]: ts=2026-09-20T10:51:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944444worker2 # [ 62.744607] nix-grpc-daemon[849]: time=2026-09-20T10:51:42.364Z level=INFO msg="All 2 paths already cached"4445worker2 # [ 62.754330] nix-grpc-daemon[777]: ts=2026-09-20T10:51:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453944446worker2 # [ 62.768229] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4447worker2 # [ 62.983174] nix-daemon[839]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)4448worker2 # [ 63.002031] nix-daemon[839]: reaped child process 1106, status = succeeded4449worker2 # [ 63.179088] nix-daemon[839]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)4450worker2 # [ 63.200223] nix-daemon[839]: reaped child process 1111, status = succeeded4451lb # [ 63.207867] niks3-server[1042]: 2026/09/20 10:51:42 INFO Received uploads request method=POST path=/api/pending_closures4452worker2 # [ 63.221168] nix-grpc-daemon[849]: time=2026-09-20T10:51:42.841Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4453worker2 # [ 63.222729] nix-grpc-daemon[849]: time=2026-09-20T10:51:42.841Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4454lb # [ 63.235188] niks3-server[1042]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4455worker2 # [ 63.253060] nix-grpc-daemon[849]: time=2026-09-20T10:51:42.873Z level=INFO msg="Uploading 1 narinfos"4456lb # [ 63.253756] niks3-server[1042]: 2026/09/20 10:51:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4457lb # [ 63.263659] niks3-server[1042]: 2026/09/20 10:51:42 INFO Signed narinfos id=15 count=14458lb # [ 63.268543] niks3-server[1042]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4459worker2 # [ 63.279037] nix-grpc-daemon[849]: time=2026-09-20T10:51:42.899Z level=INFO msg="Upload complete. (490ms)"4460lb # [ 63.276191] niks3-server[1042]: 2026/09/20 10:51:42 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4461lb # [ 63.281421] niks3-server[1042]: 2026/09/20 10:51:42 INFO Completed upload id=154462worker2 # [ 63.285241] nix-grpc-daemon[777]: ts=2026-09-20T10:51:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_in=7524463lb # [ 63.284590] niks3-server[1042]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4464worker2 # [ 63.298883] nix-grpc-daemon[777]: ts=2026-09-20T10:51:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944465worker2 # [ 63.305079] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4466worker2 # [ 63.309655] nix-daemon[839]: reaped child process 1101, status = succeeded4467worker2 # [ 63.322494] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4468worker2 # [ 64.159550] nix-daemon[839]: reaped child process 1115, status = succeeded4469worker2 # [ 64.337264] nix-daemon[839]: accepted connection from pid 1130, user nix-grpc-daemon (trusted)4470worker2 # [ 64.358114] nix-daemon[839]: reaped child process 1133, status = succeeded4471lb # [ 64.369461] niks3-server[1042]: 2026/09/20 10:51:43 INFO Received uploads request method=POST path=/api/pending_closures4472worker2 # [ 64.392201] nix-grpc-daemon[849]: time=2026-09-20T10:51:44.008Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4473worker2 # [ 64.393863] nix-grpc-daemon[849]: time=2026-09-20T10:51:44.008Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4474lb # [ 64.423249] niks3-server[1042]: 2026/09/20 10:51:44 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4475lb # [ 64.433802] niks3-server[1042]: 2026/09/20 10:51:44 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4476lb # [ 64.441816] niks3-server[1042]: 2026/09/20 10:51:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4477worker2 # [ 64.450143] nix-grpc-daemon[849]: time=2026-09-20T10:51:44.069Z level=INFO msg="Uploading 1 narinfos"4478lb # [ 64.449209] niks3-server[1042]: 2026/09/20 10:51:44 INFO Signed narinfos id=16 count=14479lb # [ 64.455300] niks3-server[1042]: 2026/09/20 10:51:44 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4480lb # [ 64.465536] niks3-server[1042]: 2026/09/20 10:51:44 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4481lb # [ 64.473858] niks3-server[1042]: 2026/09/20 10:51:44 INFO Completed upload id=164482worker2 # [ 64.476177] nix-grpc-daemon[849]: time=2026-09-20T10:51:44.096Z level=INFO msg="Upload complete. (344ms)"4483lb # [ 64.476398] niks3-server[1042]: 2026/09/20 10:51:44 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4484worker2 # [ 64.489508] nix-grpc-daemon[777]: ts=2026-09-20T10:51:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14485worker2 # [ 64.511196] nix-grpc-daemon[777]: ts=2026-09-20T10:51:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453944486worker2 # [ 64.517081] nix-daemon[839]: reaped child process 1113, status = succeeded4487worker1 # [ 64.526526] nix-grpc-daemon[778]: ts=2026-09-20T10:51:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:503324488worker1 # [ 64.543902] nix-grpc-daemon[778]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4489worker1 # [ 64.567578] nix-grpc-daemon[778]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4490worker1 # [ 64.583643] nix-grpc-daemon[778]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4491worker1 # [ 64.600653] nix-grpc-daemon[778]: ts=2026-09-20T10:51:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50332 duration_s=0 paths=1 nar_bytes_out=2324492client # [ 64.719240] nix-daemon[938]: reaped child process 940, status = succeeded4493client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.28 seconds)4494client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4495client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4496client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4497client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4498client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4499client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4500(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.42 seconds)4501subtest: an input only one worker has still reaches the builder4502client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4503client # this derivation will be built:4504client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4505client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4506client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.46 seconds)4507client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4508client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.21 seconds)4509worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4510worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.27 seconds)4511worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4512worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4513client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24514client # [ 66.098688] nix-daemon[938]: accepted connection from pid 1020, user root (trusted)4515client # this derivation will be built:4516client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4517worker2 # [ 66.338441] nix-grpc-daemon[777]: ts=2026-09-20T10:51:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:453944518client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux'...4519worker1 # [ 66.341710] nix-grpc-daemon[778]: ts=2026-09-20T10:51:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503324520worker1 # [ 66.551697] nix-daemon[851]: accepted connection from pid 986, user nix-grpc-daemon (trusted)4521worker1 # [ 66.570019] nix-daemon[851]: reaped child process 989, status = succeeded4522lb # [ 66.586752] niks3-server[1042]: 2026/09/20 10:51:46 INFO Received uploads request method=POST path=/api/pending_closures4523worker1 # [ 66.599417] nix-grpc-daemon[902]: time=2026-09-20T10:51:46.186Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4524worker1 # [ 66.601246] nix-grpc-daemon[902]: time=2026-09-20T10:51:46.186Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4525lb # [ 66.625422] niks3-server[1042]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4526worker1 # [ 66.642224] nix-grpc-daemon[902]: time=2026-09-20T10:51:46.229Z level=INFO msg="Uploading 1 narinfos"4527lb # [ 66.641675] niks3-server[1042]: 2026/09/20 10:51:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4528lb # [ 66.648798] niks3-server[1042]: 2026/09/20 10:51:46 INFO Signed narinfos id=17 count=14529lb # [ 66.655010] niks3-server[1042]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4530lb # [ 66.668392] niks3-server[1042]: 2026/09/20 10:51:46 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4531worker1 # [ 66.675531] nix-grpc-daemon[902]: time=2026-09-20T10:51:46.263Z level=INFO msg="Upload complete. (334ms)"4532lb # [ 66.679289] niks3-server[1042]: 2026/09/20 10:51:46 INFO Completed upload id=174533lb # [ 66.683370] niks3-server[1042]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4534worker1 # [ 66.704348] nix-grpc-daemon[778]: ts=2026-09-20T10:51:46Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04535worker1 # [ 66.712692] nix-grpc-daemon[778]: ts=2026-09-20T10:51:46Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14536worker1 # [ 66.760408] nix-grpc-daemon[778]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503324537worker1 # [ 66.779253] nix-grpc-daemon[778]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503324538client # copying 2 paths to 'lb:50051'...4539client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4540worker1 # [ 66.802883] nix-grpc-daemon[778]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:503324541client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4542worker1 # [ 66.822352] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4543worker1 # [ 67.050627] nix-daemon[851]: accepted connection from pid 995, user nix-grpc-daemon (trusted)4544worker1 # [ 67.072601] nix-daemon[851]: reaped child process 998, status = succeeded4545worker1 # [ 67.252525] nix-daemon[851]: accepted connection from pid 1000, user nix-grpc-daemon (trusted)4546worker1 # [ 67.273979] nix-daemon[851]: reaped child process 1003, status = succeeded4547lb # [ 67.284545] niks3-server[1042]: 2026/09/20 10:51:46 INFO Received uploads request method=POST path=/api/pending_closures4548lb # [ 67.295210] niks3-server[1042]: 2026/09/20 10:51:46 INFO Received uploads request method=POST path=/api/pending_closures4549worker1 # [ 67.305183] nix-grpc-daemon[902]: time=2026-09-20T10:51:46.893Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4550worker1 # [ 67.307484] nix-grpc-daemon[902]: time=2026-09-20T10:51:46.895Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4551worker1 # [ 67.309361] nix-grpc-daemon[902]: time=2026-09-20T10:51:46.895Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4552lb # [ 67.342179] niks3-server[1042]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4553lb # [ 67.355263] niks3-server[1042]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4554lb # [ 67.365696] niks3-server[1042]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4555worker1 # [ 67.373457] nix-grpc-daemon[902]: time=2026-09-20T10:51:46.960Z level=INFO msg="Uploading 3 narinfos"4556lb # [ 67.373675] niks3-server[1042]: 2026/09/20 10:51:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4557lb # [ 67.380525] niks3-server[1042]: 2026/09/20 10:51:46 INFO Signed narinfos id=18 count=14558lb # [ 67.382425] niks3-server[1042]: 2026/09/20 10:51:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4559lb # [ 67.390339] niks3-server[1042]: 2026/09/20 10:51:46 INFO Signed narinfos id=19 count=24560lb # [ 67.396216] niks3-server[1042]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4561lb # [ 67.404713] niks3-server[1042]: 2026/09/20 10:51:47 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4562lb # [ 67.408362] niks3-server[1042]: 2026/09/20 10:51:47 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4563lb # [ 67.416278] niks3-server[1042]: 2026/09/20 10:51:47 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4564lb # [ 67.422475] niks3-server[1042]: 2026/09/20 10:51:47 INFO Completed upload id=184565lb # [ 67.425242] niks3-server[1042]: 2026/09/20 10:51:47 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4566lb # [ 67.428804] niks3-server[1042]: 2026/09/20 10:51:47 INFO Completed upload id=194567lb # [ 67.430778] niks3-server[1042]: 2026/09/20 10:51:47 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4568worker1 # [ 67.429412] nix-grpc-daemon[902]: time=2026-09-20T10:51:47.017Z level=INFO msg="Upload complete. (585ms)"4569worker1 # [ 67.441654] nix-grpc-daemon[778]: ts=2026-09-20T10:51:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50332 duration_s=0 paths=2 nar_bytes_in=9844570worker1 # [ 67.464387] nix-grpc-daemon[778]: ts=2026-09-20T10:51:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503324571worker1 # [ 67.470219] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4572worker1 # [ 67.477001] nix-daemon[851]: reaped child process 993, status = succeeded4573client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4574client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4575worker1 # [ 67.492352] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4576worker1 # [ 68.548560] nix-daemon[851]: reaped child process 1007, status = succeeded4577worker1 # [ 68.721601] nix-daemon[851]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)4578worker1 # [ 68.740241] nix-daemon[851]: reaped child process 1025, status = succeeded4579lb # [ 68.754735] niks3-server[1042]: 2026/09/20 10:51:48 INFO Received uploads request method=POST path=/api/pending_closures4580worker1 # [ 68.763436] nix-grpc-daemon[902]: time=2026-09-20T10:51:48.350Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4581lb # [ 68.787501] niks3-server[1042]: 2026/09/20 10:51:48 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4582worker1 # [ 68.797275] nix-grpc-daemon[902]: time=2026-09-20T10:51:48.385Z level=INFO msg="Uploading 1 narinfos"4583lb # [ 68.802626] niks3-server[1042]: 2026/09/20 10:51:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4584lb # [ 68.813059] niks3-server[1042]: 2026/09/20 10:51:48 INFO Signed narinfos id=20 count=14585lb # [ 68.815512] niks3-server[1042]: 2026/09/20 10:51:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4586lb # [ 68.821652] niks3-server[1042]: 2026/09/20 10:51:48 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4587lb # [ 68.828856] niks3-server[1042]: 2026/09/20 10:51:48 INFO Completed upload id=204588worker1 # [ 68.828993] nix-grpc-daemon[902]: time=2026-09-20T10:51:48.417Z level=INFO msg="Upload complete. (323ms)"4589lb # [ 68.831167] niks3-server[1042]: 2026/09/20 10:51:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4590client # copying 1 paths...4591worker1 # [ 68.845185] nix-grpc-daemon[778]: ts=2026-09-20T10:51:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50332 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14592client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4593worker1 # [ 68.860993] nix-grpc-daemon[778]: ts=2026-09-20T10:51:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:503324594worker1 # [ 68.868290] nix-daemon[851]: reaped child process 1005, status = succeeded4595worker2 # [ 68.896832] nix-grpc-daemon[777]: ts=2026-09-20T10:51:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:453944596worker2 # [ 68.917480] nix-grpc-daemon[777]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4597worker2 # [ 68.939096] nix-grpc-daemon[777]: ts=2026-09-20T10:51:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_out=1204598client # [ 69.045672] nix-daemon[938]: reaped child process 1023, status = succeeded4599client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.20 seconds)4600client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24601client # [ 69.268056] nix-daemon[938]: accepted connection from pid 1042, user root (trusted)4602client # this derivation will be built:4603client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4604worker1 # [ 69.499234] nix-grpc-daemon[778]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:503324605client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux'...4606worker1 # [ 69.507513] nix-grpc-daemon[778]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503324607worker1 # [ 69.514697] nix-grpc-daemon[902]: time=2026-09-20T10:51:49.101Z level=INFO msg="All 1 paths already cached"4608worker1 # [ 69.528190] nix-grpc-daemon[778]: ts=2026-09-20T10:51:49Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04609worker1 # [ 69.534904] nix-grpc-daemon[778]: ts=2026-09-20T10:51:49Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14610worker2 # [ 69.549895] nix-grpc-daemon[777]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944611worker2 # [ 69.568897] nix-grpc-daemon[777]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944612client # copying 1 paths to 'lb:50051'...4613client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4614worker2 # [ 69.593136] nix-grpc-daemon[777]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453944615worker2 # [ 69.609785] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4616worker2 # [ 69.637826] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)4617worker2 # [ 69.649980] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 2/5)4618worker2 # [ 69.719723] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 3/5)4619worker2 # [ 69.803100] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 4/5)4620worker2 # [ 70.009771] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4621worker2 # [ 70.013584] nix-grpc-daemon[777]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4622worker2 # [ 70.251418] nix-daemon[839]: accepted connection from pid 1145, user nix-grpc-daemon (trusted)4623worker2 # [ 70.272125] nix-daemon[839]: reaped child process 1148, status = succeeded4624lb # [ 70.410338] envoy[913]: conn peer=[2001:db8:1::1]:35772 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3375 tx=1662 ms=23514625lb # [ 70.410731] envoy[913]: conn peer=[2001:db8:1::1]:35786 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=754626lb # [ 70.410843] envoy[913]: conn peer=[2001:db8:1::1]:35800 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3845 tx=1837 ms=17234627lb # [ 70.410953] envoy[913]: conn peer=[2001:db8:1::1]:46148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=634628lb # [ 70.411139] envoy[913]: conn peer=[2001:db8:1::1]:46156 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=1374629lb # [ 70.411258] envoy[913]: conn peer=[2001:db8:1::1]:46150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2088 ms=19884630lb # [ 70.411358] envoy[913]: conn peer=[2001:db8:1::1]:46160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3835 tx=1789 ms=26634631lb # [ 70.411490] envoy[913]: conn peer=[2001:db8:1::1]:46176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1114632worker2 # [ 70.452796] nix-daemon[839]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)4633worker2 # [ 70.474907] nix-daemon[839]: reaped child process 1153, status = succeeded4634lb # [ 70.483425] niks3-server[1042]: 2026/09/20 10:51:50 INFO Received uploads request method=POST path=/api/pending_closures4635worker2 # [ 70.497618] nix-grpc-daemon[849]: time=2026-09-20T10:51:50.117Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4636worker2 # [ 70.499176] nix-grpc-daemon[849]: time=2026-09-20T10:51:50.117Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4637lb # [ 70.514744] niks3-server[1042]: 2026/09/20 10:51:50 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4638lb # [ 70.527856] niks3-server[1042]: 2026/09/20 10:51:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4639worker2 # [ 70.537663] nix-grpc-daemon[849]: time=2026-09-20T10:51:50.157Z level=INFO msg="Uploading 1 narinfos"4640lb # [ 70.539232] niks3-server[1042]: 2026/09/20 10:51:50 INFO Signed narinfos id=21 count=14641lb # [ 70.544703] niks3-server[1042]: 2026/09/20 10:51:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4642lb # [ 70.552635] niks3-server[1042]: 2026/09/20 10:51:50 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4643lb # [ 70.558052] niks3-server[1042]: 2026/09/20 10:51:50 INFO Completed upload id=214644worker2 # [ 70.561562] nix-grpc-daemon[849]: time=2026-09-20T10:51:50.181Z level=INFO msg="Upload complete. (522ms)"4645lb # [ 70.560983] niks3-server[1042]: 2026/09/20 10:51:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4646worker2 # [ 70.572453] nix-grpc-daemon[777]: ts=2026-09-20T10:51:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_in=5684647worker2 # [ 70.580200] nix-grpc-daemon[777]: ts=2026-09-20T10:51:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453944648worker2 # [ 70.593741] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4649worker2 # [ 70.596501] nix-daemon[839]: reaped child process 1139, status = succeeded4650worker2 # [ 70.611740] nix-grpc-daemon[777]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4651client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4652worker2 # [ 70.626926] nix-daemon[839]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4653client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4654worker2 # [ 71.683107] nix-daemon[839]: reaped child process 1158, status = succeeded4655worker2 # [ 71.868109] nix-daemon[839]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)4656worker2 # [ 71.889958] nix-daemon[839]: reaped child process 1176, status = succeeded4657lb # [ 71.898345] niks3-server[1042]: 2026/09/20 10:51:51 INFO Received uploads request method=POST path=/api/pending_closures4658worker2 # [ 71.917374] nix-grpc-daemon[849]: time=2026-09-20T10:51:51.537Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4659lb # [ 71.940429] niks3-server[1042]: 2026/09/20 10:51:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4660worker2 # [ 71.954405] nix-grpc-daemon[849]: time=2026-09-20T10:51:51.574Z level=INFO msg="Uploading 1 narinfos"4661lb # [ 71.956468] niks3-server[1042]: 2026/09/20 10:51:51 INFO Signed narinfos id=22 count=14662lb # [ 71.964376] niks3-server[1042]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4663lb # [ 71.972342] niks3-server[1042]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4664lb # [ 71.981159] niks3-server[1042]: 2026/09/20 10:51:51 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4665lb # [ 71.984614] niks3-server[1042]: 2026/09/20 10:51:51 INFO Completed upload id=224666lb # [ 71.988116] niks3-server[1042]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4667worker2 # [ 71.984827] nix-grpc-daemon[849]: time=2026-09-20T10:51:51.604Z level=INFO msg="Upload complete. (339ms)"4668worker2 # [ 71.997452] nix-grpc-daemon[777]: ts=2026-09-20T10:51:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14669worker2 # [ 72.015264] nix-daemon[839]: reaped child process 1155, status = succeeded4670client # copying 1 paths...4671worker2 # [ 72.041884] nix-grpc-daemon[777]: ts=2026-09-20T10:51:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453944672client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4673worker1 # [ 72.078163] nix-grpc-daemon[778]: ts=2026-09-20T10:51:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:503324674worker1 # [ 72.092216] nix-grpc-daemon[778]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4675worker1 # [ 72.112756] nix-grpc-daemon[778]: ts=2026-09-20T10:51:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50332 duration_s=0 paths=1 nar_bytes_out=1204676client # [ 72.221012] nix-daemon[938]: reaped child process 1045, status = succeeded4677client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.17 seconds)4678client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24679client # [ 72.451227] nix-daemon[938]: accepted connection from pid 1063, user root (trusted)4680client # this derivation will be built:4681client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4682worker1 # [ 72.682990] nix-grpc-daemon[778]: ts=2026-09-20T10:51:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:503324683client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux'...4684worker2 # [ 72.691465] nix-grpc-daemon[777]: ts=2026-09-20T10:51:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453944685worker2 # [ 72.703039] nix-grpc-daemon[849]: time=2026-09-20T10:51:52.323Z level=INFO msg="All 1 paths already cached"4686worker1 # [ 72.718769] nix-grpc-daemon[778]: ts=2026-09-20T10:51:52Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04687worker1 # [ 72.726746] nix-grpc-daemon[778]: ts=2026-09-20T10:51:52Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14688worker1 # [ 72.765014] nix-grpc-daemon[778]: ts=2026-09-20T10:51:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503324689worker1 # [ 72.782780] nix-grpc-daemon[778]: ts=2026-09-20T10:51:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503324690worker1 # [ 72.802457] nix-grpc-daemon[902]: time=2026-09-20T10:51:52.389Z level=INFO msg="All 1 paths already cached"4691client # copying 1 paths to 'lb:50051'...4692client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4693worker1 # [ 72.814417] nix-grpc-daemon[778]: ts=2026-09-20T10:51:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:503324694worker1 # [ 72.830742] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4695worker1 # [ 73.053589] nix-daemon[851]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)4696worker1 # [ 73.076658] nix-daemon[851]: reaped child process 1037, status = succeeded4697worker1 # [ 73.252588] nix-daemon[851]: accepted connection from pid 1039, user nix-grpc-daemon (trusted)4698worker1 # [ 73.272055] nix-daemon[851]: reaped child process 1042, status = succeeded4699lb # [ 73.284246] niks3-server[1042]: 2026/09/20 10:51:52 INFO Received uploads request method=POST path=/api/pending_closures4700worker1 # [ 73.295902] nix-grpc-daemon[902]: time=2026-09-20T10:51:52.883Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4701worker1 # [ 73.297651] nix-grpc-daemon[902]: time=2026-09-20T10:51:52.883Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4702lb # [ 73.317361] niks3-server[1042]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4703lb # [ 73.334209] niks3-server[1042]: 2026/09/20 10:51:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4704worker1 # [ 73.340856] nix-grpc-daemon[902]: time=2026-09-20T10:51:52.928Z level=INFO msg="Uploading 1 narinfos"4705lb # [ 73.345201] niks3-server[1042]: 2026/09/20 10:51:52 INFO Signed narinfos id=23 count=14706lb # [ 73.350462] niks3-server[1042]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4707lb # [ 73.360104] niks3-server[1042]: 2026/09/20 10:51:52 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4708lb # [ 73.367292] niks3-server[1042]: 2026/09/20 10:51:52 INFO Completed upload id=234709worker1 # [ 73.367850] nix-grpc-daemon[902]: time=2026-09-20T10:51:52.955Z level=INFO msg="Upload complete. (517ms)"4710lb # [ 73.369967] niks3-server[1042]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4711worker1 # [ 73.378088] nix-grpc-daemon[778]: ts=2026-09-20T10:51:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50332 duration_s=0 paths=1 nar_bytes_in=5684712worker1 # [ 73.392943] nix-grpc-daemon[778]: ts=2026-09-20T10:51:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503324713worker1 # [ 73.401811] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4714worker1 # [ 73.408569] nix-daemon[851]: reaped child process 1032, status = succeeded4715client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4716worker1 # [ 73.419224] nix-daemon[851]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4717client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4718worker1 # [ 74.647596] nix-daemon[851]: reaped child process 1046, status = succeeded4719worker1 # [ 74.827868] nix-daemon[851]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)4720worker1 # [ 74.847191] nix-daemon[851]: reaped child process 1064, status = succeeded4721lb # [ 74.863701] niks3-server[1042]: 2026/09/20 10:51:54 INFO Received uploads request method=POST path=/api/pending_closures4722worker1 # [ 74.874163] nix-grpc-daemon[902]: time=2026-09-20T10:51:54.460Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4723lb # [ 74.893373] niks3-server[1042]: 2026/09/20 10:51:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4724worker1 # [ 74.902977] nix-grpc-daemon[902]: time=2026-09-20T10:51:54.490Z level=INFO msg="Uploading 1 narinfos"4725lb # [ 74.907555] niks3-server[1042]: 2026/09/20 10:51:54 INFO Signed narinfos id=24 count=14726lb # [ 74.915394] niks3-server[1042]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4727lb # [ 74.926134] niks3-server[1042]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4728worker1 # [ 74.930847] nix-grpc-daemon[902]: time=2026-09-20T10:51:54.518Z level=INFO msg="Upload complete. (301ms)"4729lb # [ 74.931356] niks3-server[1042]: 2026/09/20 10:51:54 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4730lb # [ 74.936722] niks3-server[1042]: 2026/09/20 10:51:54 INFO Completed upload id=244731lb # [ 74.939177] niks3-server[1042]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4732worker1 # [ 74.940911] nix-grpc-daemon[778]: ts=2026-09-20T10:51:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50332 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14733worker1 # [ 74.953394] nix-daemon[851]: reaped child process 1044, status = succeeded4734client # copying 1 paths...4735worker1 # [ 74.986095] nix-grpc-daemon[778]: ts=2026-09-20T10:51:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:503324736client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4737worker2 # [ 75.025675] nix-grpc-daemon[777]: ts=2026-09-20T10:51:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:453944738worker2 # [ 75.041479] nix-grpc-daemon[777]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4739worker2 # [ 75.059078] nix-grpc-daemon[777]: ts=2026-09-20T10:51:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45394 duration_s=0 paths=1 nar_bytes_out=1204740client # [ 75.158518] nix-daemon[938]: reaped child process 1066, status = succeeded4741client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.94 seconds)4742(finished: subtest: an input only one worker has still reaches the builder, in 10.29 seconds)4743subtest: upload whose reference the worker lacks is completed from the cache4744client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4745client # this derivation will be built:4746client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4747client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4748client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.47 seconds)4749client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4750client # this derivation will be built:4751client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4752client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4753client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.44 seconds)4754client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4755client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4756worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4757worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4758worker2 # [ 76.630883] systemd[1]: Stopping Nix worker-protocol over gRPC...4759worker2 # [ 76.816655] nix-grpc-daemon[777]: ts=2026-09-20T10:51:56Z level=info event=signal_exit4760worker1 # [ 77.866680] nix-grpc-daemon[778]: ts=2026-09-20T10:51:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503244761worker2 # [ 77.871388] nix-grpc-daemon[777]: ts=2026-09-20T10:51:57Z level=info event=scheduler_restarting4762worker1 # [ 77.871539] nix-grpc-daemon[778]: ts=2026-09-20T10:51:57Z level=info event=worker_gone addr=192.168.1.4:500514763worker2 # [ 77.907053] nix-grpc-daemon[777]: ts=2026-09-20T10:51:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4764worker2 # [ 77.916438] nix-grpc-daemon[777]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4765worker2 # [ 77.928316] nix-grpc-daemon[777]: E0000 00:00:1789901517.532809 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4766worker2 # [ 77.934915] nix-grpc-daemon[777]: E0000 00:00:1789901517.541698 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4767worker2 # [ 77.942126] nix-daemon[839]: reaped child process 841, status = succeeded4768worker2 # [ 79.555958] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4769worker2 # [ 79.559322] systemd[1]: Stopped Nix worker-protocol over gRPC.4770worker2 # [ 79.566950] systemd[1]: nix-grpc-daemon.service: Consumed 3.605s CPU time over 59.130s wall clock time, 62.9M memory peak, 86.5K incoming IP traffic, 158.9K outgoing IP traffic.4771worker2 # [ 79.580810] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4772worker2 # [ 79.582910] systemd[1]: Closed Nix worker-protocol over gRPC.4773worker2 # [ 79.588176] systemd[1]: nix-grpc-daemon.socket: Consumed 574us CPU time over 1min 8.641s wall clock time, 256K memory peak, 77.3K incoming IP traffic, 63.3K outgoing IP traffic.4774lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4775lb: must succeed: curl -sf localhost:9901/clusters4776lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4777lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4778client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24779worker1 # [ 79.917225] nix-grpc-daemon[778]: ts=2026-09-20T10:51:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:503324780client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4781worker1 # [ 79.924385] nix-grpc-daemon[778]: ts=2026-09-20T10:51:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:503324782client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.28 seconds)4783worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4784worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4785worker2 # [ 80.062492] systemd[1]: Listening on Nix worker-protocol over gRPC.4786worker1 # [ 80.142344] systemd[1]: Stopping Nix worker-protocol over gRPC...4787lb # [ 80.412191] envoy[913]: conn peer=[2001:db8:1::1]:46180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3711 tx=1806 ms=26754788lb # [ 80.417167] envoy[913]: conn peer=[2001:db8:1::1]:51350 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1044789lb # [ 80.422609] envoy[913]: conn peer=[2001:db8:1::1]:51364 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3754 tx=1796 ms=24334790lb # [ 80.427916] envoy[913]: conn peer=[2001:db8:1::1]:51368 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=984791lb # [ 80.432983] envoy[913]: conn peer=[2001:db8:1::4]:49362 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10194 tx=2939 ms=570704792lb # [ 80.438316] envoy[913]: conn peer=[2001:db8:1::1]:51384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=244793worker2 # [ 80.540745] systemd[1]: Starting Nix worker-protocol over gRPC...4794worker2 # [ 80.759889] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4795worker2 # [ 80.762569] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14796worker2 # [ 80.771824] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4797worker2 # [ 80.781739] systemd[1]: Started Nix worker-protocol over gRPC.4798worker1 # [ 80.803551] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=signal_exit4799worker2 # [ 80.815502] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:00Z level=info event=scheduler_connected addr=lb:500514800worker1 # [ 80.814375] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04801worker1 # [ 81.065433] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=scheduler_yield4802worker2 # [ 81.078998] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:00Z level=info event=scheduler_restarting addr=lb:500514803lb # [ 81.076685] niks3-server[1042]: 2026/09/20 10:52:00 INFO lead: released remote=192.168.1.3:463504804worker1 # [ 81.076328] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503244805worker1 # [ 81.084173] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=worker_gone addr=192.168.1.4:500514806worker1 # [ 81.088279] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=scheduler_restarting addr=lb:500514807worker1 # [ 81.092194] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503244808worker1 # [ 81.096648] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=worker_gone addr=192.168.1.3:500514809worker2 # [ 81.279896] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:00Z level=info event=scheduler_connected addr=lb:500514810worker2 # [ 81.285469] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4811worker1 # [ 81.283616] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=scheduler_connected addr=lb:500514812worker1 # [ 81.289121] nix-grpc-daemon[778]: ts=2026-09-20T10:52:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4813worker2 # [ 81.486346] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_connected addr=lb:500514814worker2 # [ 81.491636] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4815worker1 # [ 81.489980] nix-grpc-daemon[778]: ts=2026-09-20T10:52:01Z level=info event=scheduler_connected addr=lb:500514816worker1 # [ 81.495257] nix-grpc-daemon[778]: ts=2026-09-20T10:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4817worker2 # [ 81.692507] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_connected addr=lb:500514818worker2 # [ 81.697634] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4819worker1 # [ 81.695998] nix-grpc-daemon[778]: ts=2026-09-20T10:52:01Z level=info event=scheduler_connected addr=lb:500514820worker1 # [ 81.701128] nix-grpc-daemon[778]: ts=2026-09-20T10:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4821worker2 # [ 81.900196] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_connected addr=lb:500514822worker2 # [ 81.904306] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4823worker1 # [ 81.902026] nix-grpc-daemon[778]: ts=2026-09-20T10:52:01Z level=info event=scheduler_connected addr=lb:500514824worker1 # [ 81.907288] nix-grpc-daemon[778]: ts=2026-09-20T10:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4825worker1 # [ 82.077655] nix-grpc-daemon[778]: ts=2026-09-20T10:52:01Z level=info event=scheduler_restarting4826worker2 # [ 82.100555] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_connected addr=lb:500514827worker2 # [ 82.105744] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4828worker1 # [ 82.105716] nix-daemon[851]: reaped child process 854, status = succeeded4829worker1 # [ 82.131963] nix-grpc-daemon[778]: ts=2026-09-20T10:52:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4830worker2 # [ 82.306534] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_connected addr=lb:500514831worker2 # [ 82.311725] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4832worker2 # [ 82.512615] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_connected addr=lb:500514833worker2 # [ 82.517778] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4834worker2 # [ 82.718721] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_connected addr=lb:500514835worker2 # [ 82.723995] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4836worker2 # [ 82.926567] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_connected addr=lb:500514837worker2 # [ 82.930771] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4838worker1 # [ 82.950893] nix-grpc-daemon[778]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4839worker2 # [ 83.127282] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_connected addr=lb:500514840worker2 # [ 83.132822] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4841worker2 # [ 83.333744] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_connected addr=lb:500514842worker2 # [ 83.339188] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4843worker1 # [ 82.955617] nix-grpc-daemon[778]: E0000 00:00:1789901522.538661 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4844worker1 # [ 82.955769] nix-grpc-daemon[778]: E0000 00:00:1789901522.538971 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4845worker1 # [ 83.480454] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4846worker1 # [ 83.507649] systemd[1]: Stopped Nix worker-protocol over gRPC.4847worker1 # [ 83.512852] systemd[1]: nix-grpc-daemon.service: Consumed 2.159s CPU time over 1min 3.065s wall clock time, 54.3M memory peak, 62.1K incoming IP traffic, 130.4K outgoing IP traffic.4848worker1 # [ 83.517087] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4849worker1 # [ 83.518737] systemd[1]: Closed Nix worker-protocol over gRPC.4850worker1 # [ 83.520329] systemd[1]: nix-grpc-daemon.socket: Consumed 603us CPU time over 1min 12.822s wall clock time, 256K memory peak, 78.2K incoming IP traffic, 68.2K outgoing IP traffic.4851lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4852lb: must succeed: curl -sf localhost:9901/clusters4853worker2 # [ 83.539958] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500514854worker2 # [ 83.545219] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4855lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4856lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4857client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24858worker2 # [ 83.746022] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500514859worker2 # [ 83.751559] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4860worker2 # [ 83.872267] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388344861worker2 # [ 83.889615] nix-daemon[839]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)4862worker2 # [ 83.915856] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 1/5)4863worker2 # [ 83.954410] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500514864worker2 # [ 83.958617] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4865worker2 # [ 83.992421] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 2/5)4866worker2 # [ 84.040675] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 250 ms (attempt 3/5)4867worker2 # [ 84.155200] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500514868worker2 # [ 84.160880] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4869worker2 # [ 84.298456] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 643 ms (attempt 4/5)4870worker2 # [ 84.361800] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500514871worker2 # [ 84.367249] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4872worker2 # [ 84.568516] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=info event=scheduler_connected addr=lb:500514873worker2 # [ 84.574028] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4874worker2 # [ 84.774877] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=info event=scheduler_connected addr=lb:500514875worker2 # [ 84.781126] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4876worker2 # [ 84.950032] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4877client # copying 2 paths...4878client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4879client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4880worker2 # [ 84.980461] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:388344881worker2 # [ 84.990190] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=info event=scheduler_connected addr=lb:500514882worker2 # [ 84.995710] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4883worker2 # [ 85.008683] nix-daemon[839]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)4884worker2 # [ 85.040196] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=info event=niks3_push_started pid=12544885worker2 # [ 85.189188] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=info event=scheduler_connected addr=lb:500514886worker2 # [ 85.191723] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4887worker2 # [ 85.392303] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500514888worker2 # [ 85.412283] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4889worker2 # [ 85.437213] nix-daemon[839]: accepted connection from pid 1267, user nix-grpc-daemon (trusted)4890worker2 # [ 85.455951] nix-daemon[839]: reaped child process 1270, status = succeeded4891lb # [ 85.465522] niks3-server[1042]: 2026/09/20 10:52:05 INFO Received uploads request method=POST path=/api/pending_closures4892lb # [ 85.476230] niks3-server[1042]: 2026/09/20 10:52:05 INFO Received uploads request method=POST path=/api/pending_closures4893worker2 # [ 85.491482] nix-grpc-daemon[1254]: time=2026-09-20T10:52:05.109Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4894worker2 # [ 85.494345] nix-grpc-daemon[1254]: time=2026-09-20T10:52:05.109Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4895worker2 # [ 85.496336] nix-grpc-daemon[1254]: time=2026-09-20T10:52:05.109Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4896lb # [ 85.531262] niks3-server[1042]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4897lb # [ 85.543414] niks3-server[1042]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4898lb # [ 85.553433] niks3-server[1042]: 2026/09/20 10:52:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4899worker2 # [ 85.564113] nix-grpc-daemon[1254]: time=2026-09-20T10:52:05.183Z level=INFO msg="Uploading 3 narinfos"4900lb # [ 85.560201] niks3-server[1042]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4901lb # [ 85.562945] niks3-server[1042]: 2026/09/20 10:52:05 INFO Signed narinfos id=25 count=14902lb # [ 85.569310] niks3-server[1042]: 2026/09/20 10:52:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4903lb # [ 85.574931] niks3-server[1042]: 2026/09/20 10:52:05 INFO Signed narinfos id=26 count=24904lb # [ 85.579048] niks3-server[1042]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4905lb # [ 85.592780] niks3-server[1042]: 2026/09/20 10:52:05 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4906worker2 # [ 85.597623] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500514907lb # [ 85.601171] niks3-server[1042]: 2026/09/20 10:52:05 INFO Completed upload id=254908lb # [ 85.603140] niks3-server[1042]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4909worker2 # [ 85.605850] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4910lb # [ 85.614356] niks3-server[1042]: 2026/09/20 10:52:05 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4911worker2 # [ 85.621978] nix-grpc-daemon[1254]: time=2026-09-20T10:52:05.242Z level=INFO msg="Upload complete. (359ms)"4912lb # [ 85.621362] niks3-server[1042]: 2026/09/20 10:52:05 INFO Completed upload id=264913lb # [ 85.623962] niks3-server[1042]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4914lb # [ 85.631162] niks3-server[1042]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4915worker2 # [ 85.632462] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38834 duration_s=0 paths=2 nar_bytes_in=3124916worker2 # [ 85.649632] nix-daemon[839]: reaped child process 1252, status = succeeded4917client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.05 seconds)4918worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4919worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)4920lb # [ 85.786748] niks3-server[1042]: 2026/09/20 10:52:05 INFO lead: acquired remote=192.168.1.4:533284921worker2 # [ 85.795246] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_take_over4922worker1 # [ 85.792964] systemd[1]: Listening on Nix worker-protocol over gRPC.4923worker1 # [ 85.801307] systemd[1]: Starting Nix worker-protocol over gRPC...4924worker2 # [ 85.803582] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500514925worker2 # [ 85.808965] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4926worker1 # [ 85.998993] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4927worker1 # [ 86.002431] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14928worker2 # [ 86.011606] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500514929worker1 # [ 86.010445] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4930worker2 # [ 86.015926] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4931worker1 # [ 86.018976] systemd[1]: Started Nix worker-protocol over gRPC.4932lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4933lb: must succeed: curl -sf localhost:9901/clusters4934worker1 # [ 86.056855] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500514935worker1 # [ 86.059302] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4936lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4937worker2 # [ 86.212439] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500514938worker2 # [ 86.217554] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4939worker1 # [ 86.260786] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500514940worker1 # [ 86.266534] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4941worker2 # [ 86.418432] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500514942worker2 # [ 86.423638] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4943worker1 # [ 86.468522] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500514944worker1 # [ 86.477142] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4945worker2 # [ 86.624507] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500514946worker2 # [ 86.629625] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4947worker1 # [ 86.678058] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500514948worker1 # [ 86.683233] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4949worker2 # [ 86.830348] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500514950worker2 # [ 86.849248] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04951worker1 # [ 86.883929] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500514952worker2 # [ 86.889657] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04953lb: must succeed: curl -sf localhost:9901/clusters4954lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4955lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)4956(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.01 seconds)4957subtest: scheduler down: the other node takes the lock and keeps it4958lb: must succeed: curl -sf localhost:9901/clusters4959lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4960lb: must succeed: curl -sf localhost:9901/clusters4961lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4962lb: must succeed: curl -sf localhost:9901/clusters4963lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4964lb: must succeed: curl -sf localhost:9901/clusters4965lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4966worker2 # [ 87.477217] systemd[1]: Stopping Nix worker-protocol over gRPC...4967worker2 # [ 87.784316] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:07Z level=info event=signal_exit4968worker2 # [ 87.992528] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:07Z level=info event=scheduler_yield4969lb # [ 87.999479] niks3-server[1042]: 2026/09/20 10:52:07 INFO lead: released remote=192.168.1.4:533284970worker1 # [ 88.000330] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:07Z level=info event=scheduler_restarting addr=lb:500514971worker2 # [ 88.003755] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:388424972worker2 # [ 88.011623] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:07Z level=info event=worker_gone addr=192.168.1.3:500514973worker2 # [ 88.015974] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:07Z level=info event=scheduler_restarting addr=lb:500514974worker2 # [ 88.019951] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:388424975worker2 # [ 88.024390] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:07Z level=info event=worker_gone addr=192.168.1.4:500514976worker1 # [ 88.199942] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:07Z level=info event=scheduler_connected addr=lb:500514977worker2 # [ 88.210467] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:07Z level=info event=scheduler_connected addr=lb:500514978worker1 # [ 88.214090] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4979worker2 # [ 88.218676] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4980worker1 # [ 88.415239] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_connected addr=lb:500514981worker2 # [ 88.419908] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:08Z level=info event=scheduler_connected addr=lb:500514982worker1 # [ 88.424075] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4983worker2 # [ 88.427892] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4984worker1 # [ 88.625045] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_connected addr=lb:500514985worker2 # [ 88.629124] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:08Z level=info event=scheduler_connected addr=lb:500514986worker2 # [ 88.636767] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4987worker1 # [ 88.633474] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4988worker1 # [ 88.834726] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_connected addr=lb:500514989worker2 # [ 88.840207] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:08Z level=info event=scheduler_connected addr=lb:500514990worker1 # [ 88.840266] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4991worker2 # [ 88.844456] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4992worker2 # [ 88.998065] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:08Z level=info event=scheduler_restarting4993worker2 # [ 89.029110] nix-daemon[839]: reaped child process 1247, status = succeeded4994worker1 # [ 89.042873] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_connected addr=lb:500514995worker1 # [ 89.047106] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4996worker2 # [ 89.060883] nix-grpc-daemon[1198]: ts=2026-09-20T10:52:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4997worker1 # [ 89.243524] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_connected addr=lb:500514998worker1 # [ 89.248956] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4999worker1 # [ 89.449733] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_connected addr=lb:500515000worker1 # [ 89.454895] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5001worker1 # [ 89.655731] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_connected addr=lb:500515002worker1 # [ 89.661093] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5003worker2 # [ 89.811221] nix-grpc-daemon[1198]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5004worker2 # [ 89.811616] nix-grpc-daemon[1198]: E0000 00:00:1789901529.430861 1241 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 11, started true, shutdown true5005worker2 # [ 89.811763] nix-grpc-daemon[1198]: E0000 00:00:1789901529.431299 1241 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5006worker1 # [ 89.861801] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_connected addr=lb:500515007worker1 # [ 89.866931] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5008worker1 # [ 90.069614] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_connected addr=lb:500515009worker1 # [ 90.073794] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5010worker1 # [ 90.270510] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_connected addr=lb:500515011worker1 # [ 90.276144] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5012lb # [ 90.413880] envoy[913]: conn peer=[2001:db8:1::3]:39904 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10296 tx=2760 ms=612835013lb # [ 90.419236] envoy[913]: conn peer=[2001:db8:1::1]:49008 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1007 tx=305 ms=17745014lb # [ 90.426679] envoy[913]: conn peer=[2001:db8:1::4]:39400 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4699 tx=1344 ms=82345015worker1 # [ 90.477046] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_connected addr=lb:500515016worker1 # [ 90.482616] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5017worker1 # [ 90.683337] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_connected addr=lb:500515018worker1 # [ 90.689264] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5019worker2 # [ 90.809384] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5020worker2 # [ 90.815853] systemd[1]: Stopped Nix worker-protocol over gRPC.5021worker2 # [ 90.820270] systemd[1]: nix-grpc-daemon.service: Consumed 548ms CPU time over 10.265s wall clock time, 33.3M memory peak, 23.2K incoming IP traffic, 33.1K outgoing IP traffic.5022worker2 # [ 90.833484] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5023worker2 # [ 90.835592] systemd[1]: Closed Nix worker-protocol over gRPC.5024worker2 # [ 90.839992] systemd[1]: nix-grpc-daemon.socket: Consumed 701us CPU time over 10.779s wall clock time, 256K memory peak, 25.2K incoming IP traffic, 20.4K outgoing IP traffic.5025worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5026worker1 # [ 90.890205] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_connected addr=lb:500515027worker1 # [ 90.896558] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5028worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.10 seconds)5029worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5030worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5031lb # [ 91.020764] niks3-server[1042]: 2026/09/20 10:52:10 INFO lead: acquired remote=192.168.1.3:443045032worker1 # [ 91.025316] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_take_over5033worker1 # [ 91.098784] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_connected addr=lb:500515034worker1 # [ 91.102983] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5035worker1 # [ 91.299839] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_connected addr=lb:500515036worker1 # [ 91.305400] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5037worker1 # [ 91.506020] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:11Z level=info event=scheduler_connected addr=lb:500515038worker1 # [ 91.525185] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05039worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5040worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5041client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25042worker1 # [ 92.342999] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:387805043worker1 # [ 92.358136] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5044client # don't know how to build these paths:5045client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5046worker1 # [ 92.382679] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387805047worker1 # [ 92.403777] nix-grpc-daemon[1087]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)5048worker1 # [ 92.465535] nix-grpc-daemon[1087]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 128 ms (attempt 2/5)5049worker1 # [ 92.601349] nix-grpc-daemon[1087]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 3/5)5050worker1 # [ 92.643064] nix-grpc-daemon[1087]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 507 ms (attempt 4/5)5051worker1 # [ 93.158708] nix-grpc-daemon[1087]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5052worker1 # [ 93.222011] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:12Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05053worker1 # [ 93.229608] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:12Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15054worker1 # [ 93.269549] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387805055worker1 # [ 93.287018] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387805056client # copying 1 paths to 'lb:50051'...5057client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5058worker1 # [ 93.302602] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387805059worker1 # [ 93.320116] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5060worker1 # [ 93.349717] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:12Z level=info event=niks3_push_started pid=11575061worker1 # [ 93.741259] nix-daemon[851]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)5062worker1 # [ 93.759198] nix-daemon[851]: reaped child process 1173, status = succeeded5063worker1 # [ 93.931449] nix-daemon[851]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)5064worker1 # [ 93.947972] nix-daemon[851]: reaped child process 1178, status = succeeded5065lb # [ 93.962255] niks3-server[1042]: 2026/09/20 10:52:13 INFO Received uploads request method=POST path=/api/pending_closures5066worker1 # [ 93.978937] nix-grpc-daemon[1157]: time=2026-09-20T10:52:13.566Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5067worker1 # [ 93.982442] nix-grpc-daemon[1157]: time=2026-09-20T10:52:13.568Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5068lb # [ 94.021381] niks3-server[1042]: 2026/09/20 10:52:13 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5069worker1 # [ 94.034161] nix-grpc-daemon[1157]: time=2026-09-20T10:52:13.621Z level=INFO msg="Uploading 1 narinfos"5070lb # [ 94.037079] niks3-server[1042]: 2026/09/20 10:52:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5071lb # [ 94.050786] niks3-server[1042]: 2026/09/20 10:52:13 INFO Signed narinfos id=27 count=15072lb # [ 94.063667] niks3-server[1042]: 2026/09/20 10:52:13 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5073lb # [ 94.071351] niks3-server[1042]: 2026/09/20 10:52:13 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5074lb # [ 94.077368] niks3-server[1042]: 2026/09/20 10:52:13 INFO Completed upload id=275075worker1 # [ 94.078081] nix-grpc-daemon[1157]: time=2026-09-20T10:52:13.666Z level=INFO msg="Upload complete. (507ms)"5076lb # [ 94.079642] niks3-server[1042]: 2026/09/20 10:52:13 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5077worker1 # [ 94.088839] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=0 paths=1 nar_bytes_in=5205078worker1 # [ 94.104726] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387805079worker1 # [ 94.115318] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5080worker1 # [ 94.121247] nix-daemon[851]: reaped child process 1155, status = succeeded5081client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5082client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5083worker1 # [ 94.135134] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5084client # farm-a> LOG-farm-a-on-standby5085worker1 # [ 95.359850] nix-daemon[851]: reaped child process 1182, status = succeeded5086worker1 # [ 95.518996] nix-daemon[851]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5087worker1 # [ 95.539195] nix-daemon[851]: reaped child process 1200, status = succeeded5088lb # [ 95.551269] niks3-server[1042]: 2026/09/20 10:52:15 INFO Received uploads request method=POST path=/api/pending_closures5089worker1 # [ 95.564429] nix-grpc-daemon[1157]: time=2026-09-20T10:52:15.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5090worker1 # [ 95.565965] nix-grpc-daemon[1157]: time=2026-09-20T10:52:15.151Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5091lb # [ 95.596555] niks3-server[1042]: 2026/09/20 10:52:15 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5092lb # [ 95.605839] niks3-server[1042]: 2026/09/20 10:52:15 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5093lb # [ 95.615416] niks3-server[1042]: 2026/09/20 10:52:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5094lb # [ 95.621261] niks3-server[1042]: 2026/09/20 10:52:15 INFO Signed narinfos id=28 count=15095worker1 # [ 95.622087] nix-grpc-daemon[1157]: time=2026-09-20T10:52:15.208Z level=INFO msg="Uploading 1 narinfos"5096lb # [ 95.625347] niks3-server[1042]: 2026/09/20 10:52:15 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5097lb # [ 95.633297] niks3-server[1042]: 2026/09/20 10:52:15 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5098lb # [ 95.640335] niks3-server[1042]: 2026/09/20 10:52:15 INFO Completed upload id=285099lb # [ 95.642788] niks3-server[1042]: 2026/09/20 10:52:15 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5100worker1 # [ 95.640892] nix-grpc-daemon[1157]: time=2026-09-20T10:52:15.228Z level=INFO msg="Upload complete. (292ms)"5101worker1 # [ 95.661069] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15102worker1 # [ 95.672766] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:15Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05103worker1 # [ 95.675638] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:15Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15104worker1 # [ 95.679141] nix-daemon[851]: reaped child process 1180, status = succeeded5105worker1 # [ 95.704763] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387805106worker1 # [ 95.719865] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387805107worker1 # [ 95.735434] nix-grpc-daemon[1157]: time=2026-09-20T10:52:15.323Z level=INFO msg="All 1 paths already cached"5108client # copying 1 paths to 'lb:50051'...5109client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5110worker1 # [ 95.746349] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387805111worker1 # [ 95.760677] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5112worker1 # [ 95.955879] nix-daemon[851]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5113worker1 # [ 95.975943] nix-daemon[851]: reaped child process 1209, status = succeeded5114worker1 # [ 96.143667] nix-daemon[851]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5115worker1 # [ 96.162862] nix-daemon[851]: reaped child process 1214, status = succeeded5116lb # [ 96.173462] niks3-server[1042]: 2026/09/20 10:52:15 INFO Received uploads request method=POST path=/api/pending_closures5117worker1 # [ 96.183422] nix-grpc-daemon[1157]: time=2026-09-20T10:52:15.771Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5118worker1 # [ 96.185737] nix-grpc-daemon[1157]: time=2026-09-20T10:52:15.773Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5119lb # [ 96.202553] niks3-server[1042]: 2026/09/20 10:52:15 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5120worker1 # [ 96.216874] nix-grpc-daemon[1157]: time=2026-09-20T10:52:15.804Z level=INFO msg="Uploading 1 narinfos"5121lb # [ 96.217380] niks3-server[1042]: 2026/09/20 10:52:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5122lb # [ 96.230990] niks3-server[1042]: 2026/09/20 10:52:15 INFO Signed narinfos id=29 count=15123lb # [ 96.237284] niks3-server[1042]: 2026/09/20 10:52:15 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5124lb # [ 96.243409] niks3-server[1042]: 2026/09/20 10:52:15 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5125lb # [ 96.251555] niks3-server[1042]: 2026/09/20 10:52:15 INFO Completed upload id=295126worker1 # [ 96.250336] nix-grpc-daemon[1157]: time=2026-09-20T10:52:15.837Z level=INFO msg="Upload complete. (473ms)"5127lb # [ 96.254784] niks3-server[1042]: 2026/09/20 10:52:15 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5128worker1 # [ 96.258058] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=0 paths=1 nar_bytes_in=6405129worker1 # [ 96.272218] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387805130worker1 # [ 96.278722] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5131worker1 # [ 96.280800] nix-daemon[851]: reaped child process 1204, status = succeeded5132client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5133client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5134worker1 # [ 96.290597] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5135client # farm-b> LOG-farm-b-on-standby5136worker1 # [ 97.135757] nix-daemon[851]: reaped child process 1218, status = succeeded5137worker1 # [ 97.302587] nix-daemon[851]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)5138worker1 # [ 97.321390] nix-daemon[851]: reaped child process 1236, status = succeeded5139lb # [ 97.334883] niks3-server[1042]: 2026/09/20 10:52:16 INFO Received uploads request method=POST path=/api/pending_closures5140worker1 # [ 97.345689] nix-grpc-daemon[1157]: time=2026-09-20T10:52:16.932Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5141worker1 # [ 97.347140] nix-grpc-daemon[1157]: time=2026-09-20T10:52:16.933Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5142lb # [ 97.373714] niks3-server[1042]: 2026/09/20 10:52:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5143worker1 # [ 97.386298] nix-grpc-daemon[1157]: time=2026-09-20T10:52:16.974Z level=INFO msg="Uploading 1 narinfos"5144lb # [ 97.389935] niks3-server[1042]: 2026/09/20 10:52:17 INFO Signed narinfos id=30 count=15145lb # [ 97.399154] niks3-server[1042]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5146lb # [ 97.408593] niks3-server[1042]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5147lb # [ 97.418414] niks3-server[1042]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5148worker1 # [ 97.417308] nix-grpc-daemon[1157]: time=2026-09-20T10:52:17.005Z level=INFO msg="Upload complete. (315ms)"5149lb # [ 97.424276] niks3-server[1042]: 2026/09/20 10:52:17 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5150lb # [ 97.428705] niks3-server[1042]: 2026/09/20 10:52:17 INFO Completed upload id=305151lb # [ 97.431015] niks3-server[1042]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5152worker1 # [ 97.431710] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15153worker1 # [ 97.445122] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:17Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05154worker1 # [ 97.452573] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:17Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15155worker1 # [ 97.456889] nix-daemon[851]: reaped child process 1216, status = succeeded5156worker1 # [ 97.492599] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387805157worker1 # [ 97.509043] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387805158worker1 # [ 97.529122] nix-grpc-daemon[1157]: time=2026-09-20T10:52:17.117Z level=INFO msg="All 2 paths already cached"5159client # copying 1 paths to 'lb:50051'...5160client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5161worker1 # [ 97.539953] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387805162worker1 # [ 97.553814] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5163worker1 # [ 97.755813] nix-daemon[851]: accepted connection from pid 1242, user nix-grpc-daemon (trusted)5164worker1 # [ 97.776305] nix-daemon[851]: reaped child process 1245, status = succeeded5165worker1 # [ 97.949461] nix-daemon[851]: accepted connection from pid 1247, user nix-grpc-daemon (trusted)5166worker1 # [ 97.966848] nix-daemon[851]: reaped child process 1250, status = succeeded5167lb # [ 97.977643] niks3-server[1042]: 2026/09/20 10:52:17 INFO Received uploads request method=POST path=/api/pending_closures5168worker1 # [ 97.989286] nix-grpc-daemon[1157]: time=2026-09-20T10:52:17.577Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5169worker1 # [ 97.990755] nix-grpc-daemon[1157]: time=2026-09-20T10:52:17.577Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5170lb # [ 98.010158] niks3-server[1042]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5171worker1 # [ 98.022547] nix-grpc-daemon[1157]: time=2026-09-20T10:52:17.610Z level=INFO msg="Uploading 1 narinfos"5172lb # [ 98.026938] niks3-server[1042]: 2026/09/20 10:52:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5173lb # [ 98.037150] niks3-server[1042]: 2026/09/20 10:52:17 INFO Signed narinfos id=31 count=15174lb # [ 98.040748] niks3-server[1042]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5175lb # [ 98.050071] niks3-server[1042]: 2026/09/20 10:52:17 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5176lb # [ 98.054981] niks3-server[1042]: 2026/09/20 10:52:17 INFO Completed upload id=315177worker1 # [ 98.055394] nix-grpc-daemon[1157]: time=2026-09-20T10:52:17.643Z level=INFO msg="Upload complete. (485ms)"5178lb # [ 98.057881] niks3-server[1042]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5179worker1 # [ 98.064326] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=0 paths=1 nar_bytes_in=7685180worker1 # [ 98.078205] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387805181worker1 # [ 98.083948] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5182worker1 # [ 98.089162] nix-daemon[851]: reaped child process 1240, status = succeeded5183client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5184client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5185worker1 # [ 98.102632] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5186client # farm-top> LOG-farm-top-on-standby5187worker1 # [ 99.344789] nix-daemon[851]: reaped child process 1254, status = succeeded5188worker1 # [ 99.523074] nix-daemon[851]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)5189worker1 # [ 99.542842] nix-daemon[851]: reaped child process 1272, status = succeeded5190lb # [ 99.555949] niks3-server[1042]: 2026/09/20 10:52:19 INFO Received uploads request method=POST path=/api/pending_closures5191worker1 # [ 99.568558] nix-grpc-daemon[1157]: time=2026-09-20T10:52:19.154Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5192worker1 # [ 99.570851] nix-grpc-daemon[1157]: time=2026-09-20T10:52:19.154Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5193lb # [ 99.597832] niks3-server[1042]: 2026/09/20 10:52:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5194worker1 # [ 99.608462] nix-grpc-daemon[1157]: time=2026-09-20T10:52:19.196Z level=INFO msg="Uploading 1 narinfos"5195lb # [ 99.614537] niks3-server[1042]: 2026/09/20 10:52:19 INFO Signed narinfos id=32 count=15196lb # [ 99.622398] niks3-server[1042]: 2026/09/20 10:52:19 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5197lb # [ 99.627686] niks3-server[1042]: 2026/09/20 10:52:19 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5198lb # [ 99.634478] niks3-server[1042]: 2026/09/20 10:52:19 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5199lb # [ 99.638624] niks3-server[1042]: 2026/09/20 10:52:19 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5200lb # [ 99.642303] niks3-server[1042]: 2026/09/20 10:52:19 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5201lb # [ 99.645934] niks3-server[1042]: 2026/09/20 10:52:19 INFO Completed upload id=325202worker1 # [ 99.646210] nix-grpc-daemon[1157]: time=2026-09-20T10:52:19.234Z level=INFO msg="Upload complete. (332ms)"5203worker1 # [ 99.661274] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15204worker1 # [ 99.678095] nix-daemon[851]: reaped child process 1252, status = succeeded5205client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 7.62 seconds)5206client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5207client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5208worker2 # [ 100.024907] systemd[1]: Listening on Nix worker-protocol over gRPC.5209worker2 # [ 100.031648] systemd[1]: Starting Nix worker-protocol over gRPC...5210worker2 # [ 100.227058] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5211worker2 # [ 100.230355] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15212worker2 # [ 100.237969] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5213worker2 # [ 100.239723] systemd[1]: Started Nix worker-protocol over gRPC.5214lb: must succeed: curl -sf localhost:9901/clusters5215worker2 # [ 100.281631] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:19Z level=info event=scheduler_connected addr=lb:500515216worker1 # [ 100.280753] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05217lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5218worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5219lb # [ 100.417533] envoy[913]: conn peer=[2001:db8:1::1]:58558 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9636 tx=3884 ms=73315220worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5221lb: must succeed: curl -sf localhost:9901/clusters5222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5223lb: must succeed: curl -sf localhost:9901/clusters5224lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5225worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5226worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5227(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.43 seconds)5228subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5229client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5230client # [ 100.677259] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5231client # Running as unit: rs.service; invocation ID: d8160bdd886f47b0873ebede2b83dba45232client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5233worker1 # [ 100.946400] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:387805234client # [ 100.929022] nix[1166]: don't know how to build these paths:5235client # [ 100.930272] nix[1166]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5236worker1 # [ 100.966684] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:20Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05237worker1 # [ 100.973907] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:20Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15238worker1 # [ 101.012407] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387805239worker1 # [ 101.028813] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387805240client # [ 101.015884] nix[1166]: copying 1 paths to 'lb:50051'...5241client # [ 101.021596] nix[1166]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5242worker1 # [ 101.046156] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387805243worker1 # [ 101.061118] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5244worker1 # [ 101.259338] nix-daemon[851]: accepted connection from pid 1285, user nix-grpc-daemon (trusted)5245worker1 # [ 101.279683] nix-daemon[851]: reaped child process 1288, status = succeeded5246worker1 # [ 101.453039] nix-daemon[851]: accepted connection from pid 1290, user nix-grpc-daemon (trusted)5247worker1 # [ 101.470723] nix-daemon[851]: reaped child process 1293, status = succeeded5248lb # [ 101.482127] niks3-server[1042]: 2026/09/20 10:52:21 INFO Received uploads request method=POST path=/api/pending_closures5249worker1 # [ 101.492386] nix-grpc-daemon[1157]: time=2026-09-20T10:52:21.080Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5250worker1 # [ 101.494254] nix-grpc-daemon[1157]: time=2026-09-20T10:52:21.082Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5251lb # [ 101.511446] niks3-server[1042]: 2026/09/20 10:52:21 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5252worker1 # [ 101.533246] nix-grpc-daemon[1157]: time=2026-09-20T10:52:21.121Z level=INFO msg="Uploading 1 narinfos"5253lb # [ 101.525754] niks3-server[1042]: 2026/09/20 10:52:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5254lb # [ 101.539110] niks3-server[1042]: 2026/09/20 10:52:21 INFO Signed narinfos id=33 count=15255lb # [ 101.545524] niks3-server[1042]: 2026/09/20 10:52:21 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5256lb # [ 101.551965] niks3-server[1042]: 2026/09/20 10:52:21 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5257worker1 # [ 101.559575] nix-grpc-daemon[1157]: time=2026-09-20T10:52:21.147Z level=INFO msg="Upload complete. (482ms)"5258lb # [ 101.559892] niks3-server[1042]: 2026/09/20 10:52:21 INFO Completed upload id=335259lb # [ 101.563255] niks3-server[1042]: 2026/09/20 10:52:21 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5260worker1 # [ 101.566620] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=0 paths=1 nar_bytes_in=4165261worker1 # [ 101.579577] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387805262worker1 # [ 101.583702] nix-daemon[851]: reaped child process 1283, status = succeeded5263worker1 # [ 101.588002] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5264worker1 # [ 101.601662] nix-daemon[851]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5265client # [ 101.575249] nix[1166]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5266lb: must succeed: curl -sf localhost:9901/clusters5267lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5268lb: must succeed: curl -sf localhost:9901/clusters5269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5270lb: must succeed: curl -sf localhost:9901/clusters5271lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5272lb: must succeed: curl -sf localhost:9901/clusters5273lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5274worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5275worker2 # [ 103.672663] systemd-journald[267]: Received client request to rotate journal, rotating.5276worker2 # [ 103.698685] systemd-journald[267]: Vacuuming done, freed 0B of archived journals from /var/log/journal/3c82eb8cd832414a98a9be4f74b0507a.5277worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5278worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5279worker1 # [ 103.746433] systemd[1]: Stopping Nix worker-protocol over gRPC...5280worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5281client: must succeed: sleep 5.05282worker1 # [ 104.019798] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:23Z level=info event=draining5283client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5284worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5285worker1 # [ 109.257589] nix-daemon[851]: reaped child process 1297, status = succeeded5286worker1 # [ 109.448198] nix-daemon[851]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5287worker1 # [ 109.466404] nix-daemon[851]: reaped child process 1340, status = succeeded5288lb # [ 109.481778] niks3-server[1042]: 2026/09/20 10:52:29 INFO Received uploads request method=POST path=/api/pending_closures5289worker1 # [ 109.495985] nix-grpc-daemon[1157]: time=2026-09-20T10:52:29.080Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5290worker1 # [ 109.497693] nix-grpc-daemon[1157]: time=2026-09-20T10:52:29.080Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5291lb # [ 109.519264] niks3-server[1042]: 2026/09/20 10:52:29 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5292lb # [ 109.539117] niks3-server[1042]: 2026/09/20 10:52:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5293worker1 # [ 109.545803] nix-grpc-daemon[1157]: time=2026-09-20T10:52:29.133Z level=INFO msg="Uploading 1 narinfos"5294lb # [ 109.549695] niks3-server[1042]: 2026/09/20 10:52:29 INFO Signed narinfos id=34 count=15295lb # [ 109.558332] niks3-server[1042]: 2026/09/20 10:52:29 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5296lb # [ 109.564948] niks3-server[1042]: 2026/09/20 10:52:29 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5297lb # [ 109.571131] niks3-server[1042]: 2026/09/20 10:52:29 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5298worker1 # [ 109.571437] nix-grpc-daemon[1157]: time=2026-09-20T10:52:29.159Z level=INFO msg="Upload complete. (353ms)"5299lb # [ 109.575218] niks3-server[1042]: 2026/09/20 10:52:29 INFO Completed upload id=345300lb # [ 109.578973] niks3-server[1042]: 2026/09/20 10:52:29 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5301worker1 # [ 109.589091] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15302worker1 # [ 109.602415] nix-daemon[851]: reaped child process 1295, status = succeeded5303client # [ 109.603153] systemd[1]: rs.service: Deactivated successfully.5304client # [ 109.607668] systemd[1]: rs.service: Consumed 160ms CPU time over 8.929s wall clock time, 39.2M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.5305worker1 # [ 110.023025] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:29Z level=info event=signal_exit5306worker1 # [ 110.244528] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:29Z level=info event=scheduler_yield5307worker2 # [ 110.251945] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:29Z level=info event=scheduler_restarting addr=lb:500515308lb # [ 110.247021] niks3-server[1042]: 2026/09/20 10:52:29 INFO lead: released remote=192.168.1.3:443045309worker1 # [ 110.254731] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:387745310worker2 # [ 110.261180] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:29Z level=info event=scheduler_take_over5311lb # [ 110.257381] niks3-server[1042]: 2026/09/20 10:52:29 INFO lead: acquired remote=192.168.1.4:336445312worker1 # [ 110.260790] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:29Z level=info event=worker_gone addr=192.168.1.4:500515313worker1 # [ 110.264908] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:29Z level=info event=scheduler_restarting addr=lb:500515314worker1 # [ 110.269958] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:387745315worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.14 seconds)5316client: waiting for success: ! systemctl is-active rs5317worker1 # [ 110.274472] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:29Z level=info event=worker_gone addr=192.168.1.3:500515318client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5319client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5320client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5321client: must fail: journalctl -u rs | grep -q 'reconnecting'5322lb # [ 110.419187] envoy[913]: conn peer=[2001:db8:1::1]:54298 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2940 tx=1349 ms=86525323client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5324lb: must succeed: curl -sf localhost:9901/clusters5325worker2 # [ 110.452515] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_connected addr=lb:500515326worker2 # [ 110.458670] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5327worker1 # [ 110.461680] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=scheduler_connected addr=lb:500515328worker1 # [ 110.469231] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5329lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5330worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5331worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5332worker2 # [ 110.659699] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_connected addr=lb:500515333worker2 # [ 110.665406] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5334worker1 # [ 110.672502] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=scheduler_connected addr=lb:500515335worker1 # [ 110.676809] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5336worker2 # [ 110.866235] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_connected addr=lb:500515337worker2 # [ 110.871968] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5338worker1 # [ 110.872735] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=scheduler_connected addr=lb:500515339worker1 # [ 110.878392] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5340worker2 # [ 111.073056] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_connected addr=lb:500515341worker2 # [ 111.078499] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5342worker1 # [ 111.079429] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=scheduler_connected addr=lb:500515343worker1 # [ 111.085063] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5344worker1 # [ 111.251042] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=scheduler_restarting5345worker2 # [ 111.281817] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_connected addr=lb:500515346worker2 # [ 111.286168] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5347worker1 # [ 111.309291] nix-daemon[851]: reaped child process 1148, status = succeeded5348worker1 # [ 111.311944] nix-grpc-daemon[1087]: ts=2026-09-20T10:52:30Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5349worker2 # [ 111.482417] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515350worker2 # [ 111.488267] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5351worker1 # [ 111.559489] nix-grpc-daemon[1087]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5352worker1 # [ 111.564391] nix-grpc-daemon[1087]: E0000 00:00:1789901551.147128 1130 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5353worker1 # [ 111.571081] nix-grpc-daemon[1087]: E0000 00:00:1789901551.147432 1130 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5354lb: must succeed: curl -sf localhost:9901/clusters5355worker2 # [ 111.689113] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515356worker2 # [ 111.694636] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5358worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5359worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5360worker2 # [ 111.895415] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515361worker2 # [ 111.906931] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05362worker1 # [ 112.664970] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5363worker1 # [ 112.668552] systemd[1]: Stopped Nix worker-protocol over gRPC.5364worker1 # [ 112.671173] systemd[1]: nix-grpc-daemon.service: Consumed 1.851s CPU time over 26.861s wall clock time, 36.3M memory peak, 48.9K incoming IP traffic, 104.8K outgoing IP traffic.5365worker1 # [ 112.691739] systemd[1]: Starting Nix worker-protocol over gRPC...5366lb: must succeed: curl -sf localhost:9901/clusters5367lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5368worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5369worker1 # [ 112.903081] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5370worker1 # [ 112.905802] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15371worker1 # [ 112.914047] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5372worker1 # [ 112.915919] systemd[1]: Started Nix worker-protocol over gRPC.5373worker1 # [ 112.952525] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515374worker2 # [ 112.959327] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05375worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5376lb: must succeed: curl -sf localhost:9901/clusters5377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5378worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5379worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5380(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.50 seconds)5381subtest: scheduler restart mid-build: build finishes, no double build5382client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5383client # [ 114.182036] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5384client # Running as unit: mid.service; invocation ID: 5dfa1344eebc4cd188f983ad597115845385client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.10 seconds)5386worker1 # [ 114.461351] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:483165387worker1 # [ 114.478512] nix-daemon[851]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5388client # [ 114.474265] nix[1202]: don't know how to build these paths:5389client # [ 114.477340] nix[1202]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5390worker2 # [ 114.520414] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:34Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05391worker2 # [ 114.525957] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:34Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15392worker2 # [ 114.581897] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335405393worker2 # [ 114.597913] nix-daemon[839]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5394worker2 # [ 114.626946] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 1/5)5395worker2 # [ 114.690092] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 2/5)5396worker2 # [ 114.818213] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 296 ms (attempt 3/5)5397worker2 # [ 115.114852] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 784 ms (attempt 4/5)5398worker2 # [ 115.900970] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5399worker2 # [ 115.926086] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335405400client # [ 115.911629] nix[1202]: copying 1 paths to 'lb:50051'...5401worker2 # [ 115.943997] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335405402client # [ 115.919640] nix[1202]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5403worker2 # [ 115.966337] nix-daemon[839]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5404worker2 # [ 115.997460] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:35Z level=info event=niks3_push_started pid=14105405worker2 # [ 116.413274] nix-daemon[839]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5406worker2 # [ 116.433586] nix-daemon[839]: reaped child process 1426, status = succeeded5407worker2 # [ 116.628395] nix-daemon[839]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)5408worker2 # [ 116.647325] nix-daemon[839]: reaped child process 1431, status = succeeded5409lb # [ 116.657688] niks3-server[1042]: 2026/09/20 10:52:36 INFO Received uploads request method=POST path=/api/pending_closures5410worker2 # [ 116.674231] nix-grpc-daemon[1410]: time=2026-09-20T10:52:36.293Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5411worker2 # [ 116.675843] nix-grpc-daemon[1410]: time=2026-09-20T10:52:36.293Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5412lb # [ 116.708162] niks3-server[1042]: 2026/09/20 10:52:36 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5413worker2 # [ 116.723770] nix-grpc-daemon[1410]: time=2026-09-20T10:52:36.343Z level=INFO msg="Uploading 1 narinfos"5414lb # [ 116.725306] niks3-server[1042]: 2026/09/20 10:52:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5415lb # [ 116.737679] niks3-server[1042]: 2026/09/20 10:52:36 INFO Signed narinfos id=35 count=15416lb # [ 116.743465] niks3-server[1042]: 2026/09/20 10:52:36 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5417lb # [ 116.754211] niks3-server[1042]: 2026/09/20 10:52:36 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5418lb # [ 116.760751] niks3-server[1042]: 2026/09/20 10:52:36 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5419worker2 # [ 116.771670] nix-grpc-daemon[1410]: time=2026-09-20T10:52:36.391Z level=INFO msg="Upload complete. (536ms)"5420lb # [ 116.767399] niks3-server[1042]: 2026/09/20 10:52:36 INFO Completed upload id=355421worker2 # [ 116.779311] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33540 duration_s=0 paths=1 nar_bytes_in=4245422worker2 # [ 116.786576] nix-grpc-daemon[1291]: ts=2026-09-20T10:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335405423worker2 # [ 116.791785] nix-daemon[839]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5424worker2 # [ 116.795364] nix-daemon[839]: reaped child process 1408, status = succeeded5425worker2 # [ 116.803513] nix-daemon[839]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5426client # [ 116.774448] nix[1202]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5427lb: must succeed: curl -sf localhost:9901/clusters5428lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5429lb: must succeed: curl -sf localhost:9901/clusters5430lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5431worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5432worker2 # [ 118.248449] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1291 (nix-grpc-daemon) on client request.5433worker2 # [ 118.253040] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5434lb # [ 118.254905] niks3-server[1042]: 2026/09/20 10:52:37 INFO lead: released remote=192.168.1.4:336445435client # [ 118.245110] nix[1202]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5436worker1 # [ 118.269796] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5437worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.09 seconds)5438client: must succeed: sleep 5.05439client # [ 118.251789] nix[1202]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5440worker2 # [ 118.291850] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5441worker2 # [ 118.295635] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5442worker2 # [ 118.300273] systemd[1]: nix-grpc-daemon.service: Consumed 625ms CPU time over 18.260s wall clock time, 34M memory peak, 15.1K incoming IP traffic, 26.5K outgoing IP traffic.5443worker2 # [ 118.309001] nix-daemon[839]: reaped child process 1399, status = succeeded5444worker2 # [ 118.310271] nix-daemon[839]: reaped child process 1437, status = succeeded5445worker2 # [ 118.316387] nix-daemon[1439]: unexpected Nix daemon error: error: interrupted by the user5446worker2 # [ 118.323633] nix-daemon[839]: reaped child process 1439, status = failed with exit code 15447worker1 # [ 118.470036] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500515448worker2 # [ 118.478379] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5449worker2 # [ 118.491545] systemd[1]: Starting Nix worker-protocol over gRPC...5450worker2 # [ 118.696846] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:38Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5451worker2 # [ 118.699214] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:38Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15452worker2 # [ 118.709484] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:38Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5453worker2 # [ 118.711228] systemd[1]: Started Nix worker-protocol over gRPC.5454lb # [ 118.716878] niks3-server[1042]: 2026/09/20 10:52:38 INFO lead: acquired remote=192.168.1.4:457125455worker2 # [ 118.724923] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:38Z level=info event=scheduler_take_over5456worker2 # [ 118.738796] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:38Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05457worker2 # [ 118.752239] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500515458worker2 # [ 118.754276] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05459worker2 # [ 118.782337] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:38Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05460worker2 # [ 118.785210] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:38Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15461worker1 # [ 118.825052] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483165462worker1 # [ 118.847528] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)5463worker1 # [ 118.862251] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 196 ms (attempt 2/5)5464worker1 # [ 119.066839] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 257 ms (attempt 3/5)5465worker1 # [ 119.332834] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 678 ms (attempt 4/5)5466worker1 # [ 120.019258] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5467worker1 # [ 120.043220] nix-grpc-daemon[1350]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5468worker1 # [ 120.080227] nix-daemon[851]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5469client # [ 120.072539] nix[1202]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5470worker1 # [ 120.101360] nix-daemon[851]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5471lb # [ 120.420203] envoy[913]: conn peer=[2001:db8:1::3]:54264 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4700 tx=1479 ms=252435472lb # [ 120.420500] envoy[913]: conn peer=[2001:db8:1::4]:43948 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1566 tx=618 ms=180015473client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5474client: waiting for success: ! systemctl is-active mid5475worker1 # [ 123.725870] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:43Z level=info event=niks3_push_started pid=14515476worker1 # [ 123.775064] nix-daemon[851]: reaped child process 1427, status = succeeded5477worker1 # [ 124.134127] nix-daemon[851]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)5478worker1 # [ 124.153216] nix-daemon[851]: reaped child process 1467, status = succeeded5479lb # [ 124.167981] niks3-server[1042]: 2026/09/20 10:52:43 INFO Received uploads request method=POST path=/api/pending_closures5480worker1 # [ 124.180504] nix-grpc-daemon[1451]: time=2026-09-20T10:52:43.766Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5481worker1 # [ 124.182018] nix-grpc-daemon[1451]: time=2026-09-20T10:52:43.766Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5482lb # [ 124.215003] niks3-server[1042]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5483lb # [ 124.228414] niks3-server[1042]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5484lb # [ 124.236199] niks3-server[1042]: 2026/09/20 10:52:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5485worker1 # [ 124.243118] nix-grpc-daemon[1451]: time=2026-09-20T10:52:43.829Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 124.245670] niks3-server[1042]: 2026/09/20 10:52:43 INFO Signed narinfos id=36 count=15487lb # [ 124.258854] niks3-server[1042]: 2026/09/20 10:52:43 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5488worker1 # [ 124.263196] nix-grpc-daemon[1451]: time=2026-09-20T10:52:43.851Z level=INFO msg="Upload complete. (298ms)"5489lb # [ 124.264998] niks3-server[1042]: 2026/09/20 10:52:43 INFO Completed upload id=365490worker1 # [ 124.272258] nix-grpc-daemon[1451]: time=2026-09-20T10:52:43.854Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo error="server returned 404: object is not pending upload\n"5491lb # [ 124.277276] niks3-server[1042]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5492worker1 # [ 124.285246] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48316 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15493worker1 # [ 124.290699] nix-daemon[851]: reaped child process 1425, status = succeeded5494client # [ 124.301252] systemd[1]: mid.service: Deactivated successfully.5495client # [ 124.305565] systemd[1]: mid.service: Consumed 183ms CPU time over 10.123s wall clock time, 37M memory peak, 9.7K incoming IP traffic, 10.7K outgoing IP traffic.5496client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5497client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5498client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5499(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.70 seconds)5500subtest: interrupting the client stops the build on the worker5501client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5502client # [ 124.852621] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5503client # Running as unit: intr.service; invocation ID: 3c7eac97f7774b1e8a6b918dbd8b5d9e5504client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5505worker2 # [ 125.118391] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:578085506worker2 # [ 125.131604] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5507client # [ 125.121952] nix[1238]: don't know how to build these paths:5508client # [ 125.123556] nix[1238]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5509worker2 # [ 125.161040] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:44Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05510worker2 # [ 125.167798] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:44Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15511worker2 # [ 125.215291] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578085512worker2 # [ 125.233106] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 1/5)5513worker2 # [ 125.316566] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 2/5)5514worker2 # [ 125.358902] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 286 ms (attempt 3/5)5515worker2 # [ 125.652183] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 564 ms (attempt 4/5)5516worker2 # [ 126.221745] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5517worker2 # [ 126.240984] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578085518worker2 # [ 126.255829] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578085519client # [ 126.229254] nix[1238]: copying 1 paths to 'lb:50051'...5520client # [ 126.232537] nix[1238]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5521worker2 # [ 126.276289] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5522worker2 # [ 126.300148] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:45Z level=info event=niks3_push_started pid=15475523worker2 # [ 126.693058] nix-daemon[839]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)5524worker2 # [ 126.710933] nix-daemon[839]: reaped child process 1563, status = succeeded5525worker2 # [ 126.887620] nix-daemon[839]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5526worker2 # [ 126.905204] nix-daemon[839]: reaped child process 1568, status = succeeded5527lb # [ 126.918144] niks3-server[1042]: 2026/09/20 10:52:46 INFO Received uploads request method=POST path=/api/pending_closures5528worker2 # [ 126.936277] nix-grpc-daemon[1547]: time=2026-09-20T10:52:46.554Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5529worker2 # [ 126.938058] nix-grpc-daemon[1547]: time=2026-09-20T10:52:46.554Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5530lb # [ 126.962129] niks3-server[1042]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5531worker2 # [ 126.977919] nix-grpc-daemon[1547]: time=2026-09-20T10:52:46.597Z level=INFO msg="Uploading 1 narinfos"5532lb # [ 126.973794] niks3-server[1042]: 2026/09/20 10:52:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5533lb # [ 126.987772] niks3-server[1042]: 2026/09/20 10:52:46 INFO Signed narinfos id=37 count=15534lb # [ 126.992405] niks3-server[1042]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5535lb # [ 126.998969] niks3-server[1042]: 2026/09/20 10:52:46 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5536lb # [ 127.004719] niks3-server[1042]: 2026/09/20 10:52:46 INFO Completed upload id=375537worker2 # [ 127.008286] nix-grpc-daemon[1547]: time=2026-09-20T10:52:46.628Z level=INFO msg="Upload complete. (489ms)"5538lb # [ 127.007353] niks3-server[1042]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5539worker2 # [ 127.016162] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=0 paths=1 nar_bytes_in=4245540worker2 # [ 127.031702] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578085541worker2 # [ 127.038384] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5542worker2 # [ 127.043199] nix-daemon[839]: reaped child process 1545, status = succeeded5543worker2 # [ 127.054810] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5544client # [ 127.024743] nix[1238]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5545client: must succeed: systemctl kill -s INT intr5546client # [ 128.589747] systemd[1]: intr.service: Sent signal SIGINT to main process 1238 (nix) on client request.5547worker2 # [ 128.627311] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:48Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5548client # [ 128.599184] nix[1238]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5549client # [ 128.607499] nix[1238]: error: interrupted by the user5550client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5551client # [ 128.637384] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5552client # [ 128.641140] systemd[1]: intr.service: Failed with result 'exit-code'.5553client # [ 128.642885] systemd[1]: intr.service: Consumed 152ms CPU time over 3.787s wall clock time, 39.2M memory peak, 7.6K incoming IP traffic, 8.7K outgoing IP traffic.5554worker2 # [ 128.682709] nix-daemon[1572]: unexpected Nix daemon error: error: interrupted by the user5555worker2 # [ 128.691629] nix-daemon[839]: reaped child process 1570, status = succeeded5556worker2 # [ 128.692816] nix-daemon[839]: reaped child process 1572, status = failed with exit code 15557(finished: subtest: interrupting the client stops the build on the worker, in 3.93 seconds)5558subtest: a deploy mid-build drains: the build finishes, the new generation takes over5559client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5560client # [ 128.811897] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5561client # Running as unit: sw.service; invocation ID: 3047eefab6fb4554b3e508111e1691d95562client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5563client # [ 129.046106] nix[1258]: don't know how to build these paths:5564client # [ 129.048048] nix[1258]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5565worker2 # [ 129.072139] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:578085566worker2 # [ 129.087956] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:48Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05567worker2 # [ 129.093938] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:48Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15568worker1 # [ 129.134380] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483165569worker1 # [ 129.151255] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483165570client # [ 129.140159] nix[1258]: copying 1 paths to 'lb:50051'...5571worker1 # [ 129.167009] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483165572client # [ 129.145525] nix[1258]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5573worker1 # [ 129.182719] nix-daemon[851]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5574worker1 # [ 129.388536] nix-daemon[851]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)5575worker1 # [ 129.407478] nix-daemon[851]: reaped child process 1498, status = succeeded5576worker1 # [ 129.582781] nix-daemon[851]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)5577worker1 # [ 129.601025] nix-daemon[851]: reaped child process 1503, status = succeeded5578lb # [ 129.614538] niks3-server[1042]: 2026/09/20 10:52:49 INFO Received uploads request method=POST path=/api/pending_closures5579worker1 # [ 129.625172] nix-grpc-daemon[1451]: time=2026-09-20T10:52:49.213Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5580worker1 # [ 129.627256] nix-grpc-daemon[1451]: time=2026-09-20T10:52:49.214Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5581lb # [ 129.651569] niks3-server[1042]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5582worker1 # [ 129.662390] nix-grpc-daemon[1451]: time=2026-09-20T10:52:49.250Z level=INFO msg="Uploading 1 narinfos"5583lb # [ 129.666872] niks3-server[1042]: 2026/09/20 10:52:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5584lb # [ 129.676274] niks3-server[1042]: 2026/09/20 10:52:49 INFO Signed narinfos id=38 count=15585lb # [ 129.679617] niks3-server[1042]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5586lb # [ 129.691024] niks3-server[1042]: 2026/09/20 10:52:49 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5587lb # [ 129.696631] niks3-server[1042]: 2026/09/20 10:52:49 INFO Completed upload id=385588worker1 # [ 129.697218] nix-grpc-daemon[1451]: time=2026-09-20T10:52:49.285Z level=INFO msg="Upload complete. (497ms)"5589lb # [ 129.699980] niks3-server[1042]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5590worker1 # [ 129.704169] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48316 duration_s=0 paths=1 nar_bytes_in=4165591worker1 # [ 129.718938] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483165592worker1 # [ 129.730776] nix-daemon[851]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5593worker1 # [ 129.734030] nix-daemon[851]: reaped child process 1493, status = succeeded5594worker1 # [ 129.746409] nix-daemon[851]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5595client # [ 129.719717] nix[1258]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5596lb # [ 130.421577] envoy[913]: conn peer=[2001:db8:1::1]:41012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3871 tx=1726 ms=98425597lb # [ 130.427029] envoy[913]: conn peer=[2001:db8:1::1]:55468 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2931 tx=757 ms=35375598worker1: must succeed: systemctl reload nix-grpc-daemon.service5599worker1 # [ 131.532263] systemd[1]: Reloading Nix worker-protocol over gRPC...5600worker1 # [ 131.580504] systemd[1]: Reloaded Nix worker-protocol over gRPC.5601worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.10 seconds)5602worker1: must succeed: pgrep -f 'read -t [9]0 x'5603worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5604client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25605client # don't know how to build these paths:5606client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5607worker2 # [ 131.899212] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:578085608worker2 # [ 131.907378] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578085609worker1 # [ 131.927580] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:51Z level=info event=draining5610worker2 # [ 131.930415] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:51Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05611worker2 # [ 131.938736] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:51Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15612worker2 # [ 131.949308] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578085613worker2 # [ 131.965372] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578085614client # copying 1 paths to 'lb:50051'...5615worker2 # [ 131.981736] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578085616client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5617worker2 # [ 132.004409] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5618worker2 # [ 132.212190] nix-daemon[839]: accepted connection from pid 1622, user nix-grpc-daemon (trusted)5619worker2 # [ 132.230499] nix-daemon[839]: reaped child process 1625, status = succeeded5620worker2 # [ 132.408189] nix-daemon[839]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)5621worker2 # [ 132.427596] nix-daemon[839]: reaped child process 1630, status = succeeded5622lb # [ 132.438398] niks3-server[1042]: 2026/09/20 10:52:52 INFO Received uploads request method=POST path=/api/pending_closures5623worker2 # [ 132.451811] nix-grpc-daemon[1547]: time=2026-09-20T10:52:52.071Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5624worker2 # [ 132.453495] nix-grpc-daemon[1547]: time=2026-09-20T10:52:52.071Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5625lb # [ 132.464634] niks3-server[1042]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5626worker2 # [ 132.483200] nix-grpc-daemon[1547]: time=2026-09-20T10:52:52.103Z level=INFO msg="Uploading 1 narinfos"5627lb # [ 132.479708] niks3-server[1042]: 2026/09/20 10:52:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5628lb # [ 132.489293] niks3-server[1042]: 2026/09/20 10:52:52 INFO Signed narinfos id=39 count=15629lb # [ 132.498223] niks3-server[1042]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5630lb # [ 132.508377] niks3-server[1042]: 2026/09/20 10:52:52 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5631lb # [ 132.513699] niks3-server[1042]: 2026/09/20 10:52:52 INFO Completed upload id=395632worker2 # [ 132.517727] nix-grpc-daemon[1547]: time=2026-09-20T10:52:52.137Z level=INFO msg="Upload complete. (496ms)"5633lb # [ 132.516714] niks3-server[1042]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5634worker2 # [ 132.524336] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=0 paths=1 nar_bytes_in=5285635worker2 # [ 132.536737] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578085636worker2 # [ 132.546776] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5637worker2 # [ 132.551943] nix-daemon[839]: reaped child process 1620, status = succeeded5638client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5639client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5640worker2 # [ 132.566612] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5641client # farm-a> LOG-farm-a-during-drain5642worker2 # [ 133.594584] nix-daemon[839]: reaped child process 1634, status = succeeded5643worker2 # [ 133.758635] nix-daemon[839]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)5644worker2 # [ 133.777626] nix-daemon[839]: reaped child process 1652, status = succeeded5645lb # [ 133.787337] niks3-server[1042]: 2026/09/20 10:52:53 INFO Received uploads request method=POST path=/api/pending_closures5646worker2 # [ 133.804287] nix-grpc-daemon[1547]: time=2026-09-20T10:52:53.420Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5647worker2 # [ 133.805845] nix-grpc-daemon[1547]: time=2026-09-20T10:52:53.420Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5648lb # [ 133.821248] niks3-server[1042]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5649lb # [ 133.834695] niks3-server[1042]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5650lb # [ 133.848195] niks3-server[1042]: 2026/09/20 10:52:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5651worker2 # [ 133.858571] nix-grpc-daemon[1547]: time=2026-09-20T10:52:53.477Z level=INFO msg="Uploading 1 narinfos"5652lb # [ 133.854216] niks3-server[1042]: 2026/09/20 10:52:53 INFO Signed narinfos id=40 count=15653lb # [ 133.859779] niks3-server[1042]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5654lb # [ 133.871118] niks3-server[1042]: 2026/09/20 10:52:53 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5655lb # [ 133.877887] niks3-server[1042]: 2026/09/20 10:52:53 INFO Completed upload id=405656worker2 # [ 133.881648] nix-grpc-daemon[1547]: time=2026-09-20T10:52:53.501Z level=INFO msg="Upload complete. (305ms)"5657lb # [ 133.880925] niks3-server[1042]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5658worker2 # [ 133.895980] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15659worker2 # [ 133.914315] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:53Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05660worker2 # [ 133.918923] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:53Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15661worker2 # [ 133.922783] nix-daemon[839]: reaped child process 1632, status = succeeded5662worker2 # [ 133.944187] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578085663worker2 # [ 133.958243] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578085664worker2 # [ 133.978289] nix-grpc-daemon[1547]: time=2026-09-20T10:52:53.598Z level=INFO msg="All 1 paths already cached"5665client # copying 1 paths to 'lb:50051'...5666worker2 # [ 133.989234] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578085667client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5668worker2 # [ 134.008190] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5669worker2 # [ 134.218482] nix-daemon[839]: accepted connection from pid 1658, user nix-grpc-daemon (trusted)5670worker2 # [ 134.236718] nix-daemon[839]: reaped child process 1661, status = succeeded5671worker2 # [ 134.409120] nix-daemon[839]: accepted connection from pid 1663, user nix-grpc-daemon (trusted)5672worker2 # [ 134.427795] nix-daemon[839]: reaped child process 1666, status = succeeded5673lb # [ 134.434906] niks3-server[1042]: 2026/09/20 10:52:54 INFO Received uploads request method=POST path=/api/pending_closures5674worker2 # [ 134.448789] nix-grpc-daemon[1547]: time=2026-09-20T10:52:54.068Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5675worker2 # [ 134.450308] nix-grpc-daemon[1547]: time=2026-09-20T10:52:54.068Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5676lb # [ 134.461874] niks3-server[1042]: 2026/09/20 10:52:54 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5677lb # [ 134.476383] niks3-server[1042]: 2026/09/20 10:52:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5678worker2 # [ 134.486693] nix-grpc-daemon[1547]: time=2026-09-20T10:52:54.106Z level=INFO msg="Uploading 1 narinfos"5679lb # [ 134.488987] niks3-server[1042]: 2026/09/20 10:52:54 INFO Signed narinfos id=41 count=15680lb # [ 134.495460] niks3-server[1042]: 2026/09/20 10:52:54 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5681worker2 # [ 134.507080] nix-grpc-daemon[1547]: time=2026-09-20T10:52:54.127Z level=INFO msg="Upload complete. (477ms)"5682lb # [ 134.504463] niks3-server[1042]: 2026/09/20 10:52:54 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5683worker2 # [ 134.512269] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=0 paths=1 nar_bytes_in=6405684lb # [ 134.510101] niks3-server[1042]: 2026/09/20 10:52:54 INFO Completed upload id=415685lb # [ 134.513531] niks3-server[1042]: 2026/09/20 10:52:54 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5686worker2 # [ 134.521212] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578085687worker2 # [ 134.533218] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5688worker2 # [ 134.535787] nix-daemon[839]: reaped child process 1656, status = succeeded5689client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5690worker2 # [ 134.547984] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5691client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5692client # farm-b> LOG-farm-b-during-drain5693worker2 # [ 135.576334] nix-daemon[839]: reaped child process 1670, status = succeeded5694worker2 # [ 135.741828] nix-daemon[839]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)5695worker2 # [ 135.760134] nix-daemon[839]: reaped child process 1688, status = succeeded5696lb # [ 135.771443] niks3-server[1042]: 2026/09/20 10:52:55 INFO Received uploads request method=POST path=/api/pending_closures5697worker2 # [ 135.785632] nix-grpc-daemon[1547]: time=2026-09-20T10:52:55.404Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5698worker2 # [ 135.787199] nix-grpc-daemon[1547]: time=2026-09-20T10:52:55.404Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5699lb # [ 135.813843] niks3-server[1042]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5700worker2 # [ 135.828337] nix-grpc-daemon[1547]: time=2026-09-20T10:52:55.447Z level=INFO msg="Uploading 1 narinfos"5701lb # [ 135.823427] niks3-server[1042]: 2026/09/20 10:52:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5702lb # [ 135.835498] niks3-server[1042]: 2026/09/20 10:52:55 INFO Signed narinfos id=42 count=15703lb # [ 135.844273] niks3-server[1042]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5704lb # [ 135.850073] niks3-server[1042]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5705lb # [ 135.858381] niks3-server[1042]: 2026/09/20 10:52:55 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5706lb # [ 135.863631] niks3-server[1042]: 2026/09/20 10:52:55 INFO Completed upload id=425707worker2 # [ 135.867777] nix-grpc-daemon[1547]: time=2026-09-20T10:52:55.487Z level=INFO msg="Upload complete. (311ms)"5708lb # [ 135.866687] niks3-server[1042]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5709worker2 # [ 135.881286] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15710worker2 # [ 135.900398] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:55Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05711worker2 # [ 135.905177] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:55Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15712worker2 # [ 135.909492] nix-daemon[839]: reaped child process 1668, status = succeeded5713worker2 # [ 135.936388] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578085714worker2 # [ 135.951544] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578085715worker2 # [ 135.964833] nix-grpc-daemon[1547]: time=2026-09-20T10:52:55.584Z level=INFO msg="All 2 paths already cached"5716client # copying 1 paths to 'lb:50051'...5717worker2 # [ 135.975409] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578085718client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5719worker2 # [ 135.998198] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5720worker2 # [ 136.203695] nix-daemon[839]: accepted connection from pid 1694, user nix-grpc-daemon (trusted)5721worker2 # [ 136.223337] nix-daemon[839]: reaped child process 1697, status = succeeded5722worker2 # [ 136.397797] nix-daemon[839]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)5723worker2 # [ 136.415707] nix-daemon[839]: reaped child process 1702, status = succeeded5724lb # [ 136.423139] niks3-server[1042]: 2026/09/20 10:52:56 INFO Received uploads request method=POST path=/api/pending_closures5725worker2 # [ 136.436857] nix-grpc-daemon[1547]: time=2026-09-20T10:52:56.056Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5726worker2 # [ 136.438394] nix-grpc-daemon[1547]: time=2026-09-20T10:52:56.056Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5727lb # [ 136.455068] niks3-server[1042]: 2026/09/20 10:52:56 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5728worker2 # [ 136.471374] nix-grpc-daemon[1547]: time=2026-09-20T10:52:56.091Z level=INFO msg="Uploading 1 narinfos"5729lb # [ 136.471001] niks3-server[1042]: 2026/09/20 10:52:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5730lb # [ 136.481961] niks3-server[1042]: 2026/09/20 10:52:56 INFO Signed narinfos id=43 count=15731worker2 # [ 136.494072] nix-grpc-daemon[1547]: time=2026-09-20T10:52:56.114Z level=INFO msg="Upload complete. (477ms)"5732lb # [ 136.490561] niks3-server[1042]: 2026/09/20 10:52:56 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5733worker2 # [ 136.500324] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=0 paths=1 nar_bytes_in=7685734lb # [ 136.495630] niks3-server[1042]: 2026/09/20 10:52:56 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5735lb # [ 136.501188] niks3-server[1042]: 2026/09/20 10:52:56 INFO Completed upload id=435736lb # [ 136.503827] niks3-server[1042]: 2026/09/20 10:52:56 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5737worker2 # [ 136.514416] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578085738worker2 # [ 136.519927] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5739worker2 # [ 136.524733] nix-daemon[839]: reaped child process 1692, status = succeeded5740client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5741worker2 # [ 136.537243] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5742client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5743client # farm-top> LOG-farm-top-during-drain5744worker2 # [ 138.182428] nix-daemon[839]: reaped child process 1706, status = succeeded5745worker2 # [ 138.360779] nix-daemon[839]: accepted connection from pid 1721, user nix-grpc-daemon (trusted)5746worker2 # [ 138.380908] nix-daemon[839]: reaped child process 1724, status = succeeded5747lb # [ 138.389504] niks3-server[1042]: 2026/09/20 10:52:58 INFO Received uploads request method=POST path=/api/pending_closures5748worker2 # [ 138.404393] nix-grpc-daemon[1547]: time=2026-09-20T10:52:58.023Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5749worker2 # [ 138.405855] nix-grpc-daemon[1547]: time=2026-09-20T10:52:58.023Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5750lb # [ 138.428544] niks3-server[1042]: 2026/09/20 10:52:58 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5751lb # [ 138.440456] niks3-server[1042]: 2026/09/20 10:52:58 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5752lb # [ 138.451486] niks3-server[1042]: 2026/09/20 10:52:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5753lb # [ 138.457107] niks3-server[1042]: 2026/09/20 10:52:58 INFO Signed narinfos id=44 count=15754worker2 # [ 138.460727] nix-grpc-daemon[1547]: time=2026-09-20T10:52:58.080Z level=INFO msg="Uploading 1 narinfos"5755lb # [ 138.463015] niks3-server[1042]: 2026/09/20 10:52:58 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5756lb # [ 138.472366] niks3-server[1042]: 2026/09/20 10:52:58 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5757worker2 # [ 138.482171] nix-grpc-daemon[1547]: time=2026-09-20T10:52:58.102Z level=INFO msg="Upload complete. (334ms)"5758lb # [ 138.479749] niks3-server[1042]: 2026/09/20 10:52:58 INFO Completed upload id=445759lb # [ 138.482952] niks3-server[1042]: 2026/09/20 10:52:58 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5760worker2 # [ 138.494388] nix-grpc-daemon[1471]: ts=2026-09-20T10:52:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15761worker2 # [ 138.512901] nix-daemon[839]: reaped child process 1704, status = succeeded5762client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.91 seconds)5763client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5764client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5765worker1: must succeed: pkill -f 'read -t [9]0 x'5766worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.07 seconds)5767client: waiting for success: ! systemctl is-active sw5768worker1 # [ 138.998726] nix-daemon[851]: reaped child process 1507, status = succeeded5769worker1 # [ 139.183720] nix-daemon[851]: accepted connection from pid 1547, user nix-grpc-daemon (trusted)5770worker1 # [ 139.202529] nix-daemon[851]: reaped child process 1550, status = succeeded5771lb # [ 139.218043] niks3-server[1042]: 2026/09/20 10:52:58 INFO Received uploads request method=POST path=/api/pending_closures5772worker1 # [ 139.228394] nix-grpc-daemon[1451]: time=2026-09-20T10:52:58.815Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5773worker1 # [ 139.229916] nix-grpc-daemon[1451]: time=2026-09-20T10:52:58.815Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5774lb # [ 139.255773] niks3-server[1042]: 2026/09/20 10:52:58 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5775worker1 # [ 139.270643] nix-grpc-daemon[1451]: time=2026-09-20T10:52:58.857Z level=INFO msg="Uploading 1 narinfos"5776lb # [ 139.267065] niks3-server[1042]: 2026/09/20 10:52:58 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5777lb # [ 139.281763] niks3-server[1042]: 2026/09/20 10:52:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5778lb # [ 139.292319] niks3-server[1042]: 2026/09/20 10:52:58 INFO Signed narinfos id=45 count=15779lb # [ 139.295106] niks3-server[1042]: 2026/09/20 10:52:58 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5780worker1 # [ 139.299699] nix-grpc-daemon[1451]: time=2026-09-20T10:52:58.887Z level=INFO msg="Upload complete. (333ms)"5781lb # [ 139.300112] niks3-server[1042]: 2026/09/20 10:52:58 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5782lb # [ 139.307053] niks3-server[1042]: 2026/09/20 10:52:58 INFO Completed upload id=455783lb # [ 139.309843] niks3-server[1042]: 2026/09/20 10:52:58 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5784worker1 # [ 139.308911] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48316 duration_s=9 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15785worker1 # [ 139.324479] nix-daemon[851]: reaped child process 1505, status = succeeded5786client # [ 139.333500] systemd[1]: sw.service: Deactivated successfully.5787client # [ 139.338408] systemd[1]: sw.service: Consumed 156ms CPU time over 10.525s wall clock time, 37M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5788worker1 # [ 139.931443] nix-grpc-daemon[1350]: ts=2026-09-20T10:52:59Z level=info event=signal_exit5789client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds)5790client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5791client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5792worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5793lb # [ 140.424876] envoy[913]: conn peer=[2001:db8:1::1]:42154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9625 tx=3822 ms=66285794lb # [ 140.427436] envoy[913]: conn peer=[2001:db8:1::1]:55478 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1350 ms=102595795worker2 # [ 141.168351] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:335425796worker2 # [ 141.168645] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:00Z level=info event=worker_gone addr=192.168.1.3:500515797worker1 # [ 141.174229] nix-grpc-daemon[1350]: ts=2026-09-20T10:53:00Z level=info event=scheduler_restarting5798worker1 # [ 141.190734] nix-grpc-daemon[1350]: ts=2026-09-20T10:53:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5799worker1 # [ 141.217783] nix-daemon[851]: reaped child process 1403, status = succeeded5800worker1 # [ 141.690452] nix-grpc-daemon[1350]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5801worker1 # [ 141.692752] nix-grpc-daemon[1350]: E0000 00:00:1789901581.278430 1393 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5802worker1 # [ 141.743784] nix-grpc-daemon[1350]: E0000 00:00:1789901581.331739 1393 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5803worker1 # [ 142.976948] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5804worker1 # [ 142.980173] systemd[1]: nix-grpc-daemon.service: Consumed 927ms CPU time over 30.282s wall clock time, 33.4M memory peak, 21K incoming IP traffic, 27.3K outgoing IP traffic.5805worker1 # [ 143.002626] systemd[1]: Starting Nix worker-protocol over gRPC...5806worker1 # [ 143.203258] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:02Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5807worker1 # [ 143.206622] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:02Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15808worker1 # [ 143.214681] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:02Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5809worker1 # [ 143.222684] systemd[1]: Started Nix worker-protocol over gRPC.5810worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.21 seconds)5811lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5812lb: must succeed: curl -sf localhost:9901/clusters5813worker1 # [ 143.257034] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:02Z level=info event=scheduler_connected addr=lb:500515814worker2 # [ 143.265263] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:02Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05815lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5816lb: must succeed: curl -sf localhost:9901/clusters5817lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5818lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5819lb: must succeed: curl -sf localhost:9901/clusters5820lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5821worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5822worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5823(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.78 seconds)5824subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5825client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15826worker2 # [ 144.805588] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:578085827worker2 # [ 144.812119] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578085828worker2 # [ 144.858537] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:04Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05829worker2 # [ 144.864639] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:04Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15830worker2 # [ 144.870598] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:04Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05831worker2 # [ 144.875996] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:04Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05832worker1 # [ 144.906913] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423425833worker1 # [ 144.918304] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5834worker1 # [ 144.947032] nix-grpc-daemon[1564]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)5835worker1 # [ 144.991727] nix-grpc-daemon[1564]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 2/5)5836worker1 # [ 145.093470] nix-grpc-daemon[1564]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 377 ms (attempt 3/5)5837worker1 # [ 145.478429] nix-grpc-daemon[1564]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 307 ms (attempt 4/5)5838worker1 # [ 145.794521] nix-grpc-daemon[1564]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5839worker1 # [ 145.815685] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423425840worker1 # [ 145.833342] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:423425841worker1 # [ 145.848497] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5842worker1 # [ 145.878083] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:05Z level=info event=niks3_push_started pid=16205843worker1 # [ 146.265035] nix-daemon[851]: accepted connection from pid 1633, user nix-grpc-daemon (trusted)5844worker1 # [ 146.283041] nix-daemon[851]: reaped child process 1636, status = succeeded5845worker1 # [ 146.458233] nix-daemon[851]: accepted connection from pid 1638, user nix-grpc-daemon (trusted)5846worker1 # [ 146.475914] nix-daemon[851]: reaped child process 1641, status = succeeded5847lb # [ 146.490001] niks3-server[1042]: 2026/09/20 10:53:06 INFO Received uploads request method=POST path=/api/pending_closures5848worker1 # [ 146.502209] nix-grpc-daemon[1620]: time=2026-09-20T10:53:06.088Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5849worker1 # [ 146.503768] nix-grpc-daemon[1620]: time=2026-09-20T10:53:06.088Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5850lb # [ 146.525067] niks3-server[1042]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5851lb # [ 146.537299] niks3-server[1042]: 2026/09/20 10:53:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5852worker1 # [ 146.545208] nix-grpc-daemon[1620]: time=2026-09-20T10:53:06.132Z level=INFO msg="Uploading 1 narinfos"5853lb # [ 146.548440] niks3-server[1042]: 2026/09/20 10:53:06 INFO Signed narinfos id=46 count=15854lb # [ 146.556228] niks3-server[1042]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5855lb # [ 146.566673] niks3-server[1042]: 2026/09/20 10:53:06 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5856worker1 # [ 146.573390] nix-grpc-daemon[1620]: time=2026-09-20T10:53:06.161Z level=INFO msg="Upload complete. (481ms)"5857lb # [ 146.575055] niks3-server[1042]: 2026/09/20 10:53:06 INFO Completed upload id=465858lb # [ 146.578805] niks3-server[1042]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5859worker1 # [ 146.584697] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=0 paths=1 nar_bytes_in=5125860worker1 # [ 146.594899] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423425861worker1 # [ 146.601771] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5862worker1 # [ 146.606870] nix-daemon[851]: reaped child process 1618, status = succeeded5863worker1 # [ 146.617264] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5864worker1 # [ 148.268382] nix-daemon[851]: reaped child process 1645, status = succeeded5865worker1 # [ 148.449625] nix-daemon[851]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)5866worker1 # [ 148.467595] nix-daemon[851]: reaped child process 1663, status = succeeded5867lb # [ 148.479336] niks3-server[1042]: 2026/09/20 10:53:08 INFO Received uploads request method=POST path=/api/pending_closures5868worker1 # [ 148.492504] nix-grpc-daemon[1620]: time=2026-09-20T10:53:08.077Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5869worker1 # [ 148.494448] nix-grpc-daemon[1620]: time=2026-09-20T10:53:08.077Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5870lb # [ 148.524443] niks3-server[1042]: 2026/09/20 10:53:08 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5871worker1 # [ 148.538558] nix-grpc-daemon[1620]: time=2026-09-20T10:53:08.125Z level=INFO msg="Uploading 1 narinfos"5872lb # [ 148.541765] niks3-server[1042]: 2026/09/20 10:53:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5873lb # [ 148.551042] niks3-server[1042]: 2026/09/20 10:53:08 INFO Signed narinfos id=47 count=15874worker1 # [ 148.561675] nix-grpc-daemon[1620]: time=2026-09-20T10:53:08.149Z level=INFO msg="Upload complete. (328ms)"5875lb # [ 148.561289] niks3-server[1042]: 2026/09/20 10:53:08 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5876lb # [ 148.565282] niks3-server[1042]: 2026/09/20 10:53:08 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5877lb # [ 148.569919] niks3-server[1042]: 2026/09/20 10:53:08 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5878lb # [ 148.574131] niks3-server[1042]: 2026/09/20 10:53:08 INFO Completed upload id=475879worker2 # [ 148.576413] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:08Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15880lb # [ 148.576217] niks3-server[1042]: 2026/09/20 10:53:08 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5881worker2 # [ 148.590295] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:08Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05882worker1 # [ 148.573128] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=15883worker1 # [ 148.592515] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423425884worker1 # [ 148.599078] nix-daemon[851]: reaped child process 1643, status = succeeded5885worker1 # [ 148.603113] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423425886worker1 # [ 148.612060] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:423425887worker1 # [ 148.622206] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5888worker1 # [ 148.807688] nix-daemon[851]: accepted connection from pid 1667, user nix-grpc-daemon (trusted)5889worker1 # [ 148.825591] nix-daemon[851]: reaped child process 1670, status = succeeded5890worker1 # [ 149.000971] nix-daemon[851]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5891worker1 # [ 149.019322] nix-daemon[851]: reaped child process 1675, status = succeeded5892lb # [ 149.030315] niks3-server[1042]: 2026/09/20 10:53:08 INFO Received uploads request method=POST path=/api/pending_closures5893worker1 # [ 149.039467] nix-grpc-daemon[1620]: time=2026-09-20T10:53:08.627Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5894worker1 # [ 149.041693] nix-grpc-daemon[1620]: time=2026-09-20T10:53:08.629Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5895lb # [ 149.057701] niks3-server[1042]: 2026/09/20 10:53:08 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5896worker1 # [ 149.073518] nix-grpc-daemon[1620]: time=2026-09-20T10:53:08.661Z level=INFO msg="Uploading 1 narinfos"5897lb # [ 149.072910] niks3-server[1042]: 2026/09/20 10:53:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5898lb # [ 149.085124] niks3-server[1042]: 2026/09/20 10:53:08 INFO Signed narinfos id=48 count=15899lb # [ 149.092500] niks3-server[1042]: 2026/09/20 10:53:08 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5900lb # [ 149.100500] niks3-server[1042]: 2026/09/20 10:53:08 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5901lb # [ 149.107680] niks3-server[1042]: 2026/09/20 10:53:08 INFO Completed upload id=485902worker1 # [ 149.108273] nix-grpc-daemon[1620]: time=2026-09-20T10:53:08.696Z level=INFO msg="Upload complete. (474ms)"5903lb # [ 149.111184] niks3-server[1042]: 2026/09/20 10:53:08 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5904worker1 # [ 149.116293] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=0 paths=1 nar_bytes_in=5125905worker1 # [ 149.131707] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423425906worker1 # [ 149.138644] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5907worker1 # [ 149.143913] nix-daemon[851]: reaped child process 1665, status = succeeded5908worker1 # [ 149.155693] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5909worker1 # [ 150.385403] nix-daemon[851]: reaped child process 1679, status = succeeded5910lb # [ 150.424605] envoy[913]: conn peer=[2001:db8:1::3]:43558 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=986 tx=448 ms=282205911worker1 # [ 150.549911] nix-daemon[851]: accepted connection from pid 1694, user nix-grpc-daemon (trusted)5912worker1 # [ 150.570383] nix-daemon[851]: reaped child process 1697, status = succeeded5913lb # [ 150.583079] niks3-server[1042]: 2026/09/20 10:53:10 INFO Received uploads request method=POST path=/api/pending_closures5914worker1 # [ 150.594024] nix-grpc-daemon[1620]: time=2026-09-20T10:53:10.181Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5915worker1 # [ 150.595529] nix-grpc-daemon[1620]: time=2026-09-20T10:53:10.181Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5916lb # [ 150.616938] niks3-server[1042]: 2026/09/20 10:53:10 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5917worker1 # [ 150.634480] nix-grpc-daemon[1620]: time=2026-09-20T10:53:10.222Z level=INFO msg="Uploading 1 narinfos"5918lb # [ 150.634792] niks3-server[1042]: 2026/09/20 10:53:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5919lb # [ 150.644948] niks3-server[1042]: 2026/09/20 10:53:10 INFO Signed narinfos id=49 count=15920lb # [ 150.647356] niks3-server[1042]: 2026/09/20 10:53:10 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5921lb # [ 150.655611] niks3-server[1042]: 2026/09/20 10:53:10 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5922lb # [ 150.662657] niks3-server[1042]: 2026/09/20 10:53:10 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5923lb # [ 150.666764] niks3-server[1042]: 2026/09/20 10:53:10 INFO Completed upload id=495924worker1 # [ 150.667273] nix-grpc-daemon[1620]: time=2026-09-20T10:53:10.255Z level=INFO msg="Upload complete. (299ms)"5925lb # [ 150.669190] niks3-server[1042]: 2026/09/20 10:53:10 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5926worker2 # [ 150.679236] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:10Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15927worker1 # [ 150.677090] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=15928worker2 # [ 150.692402] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:10Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05929worker1 # [ 150.694701] nix-daemon[851]: reaped child process 1677, status = succeeded5930worker1 # [ 150.729083] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423425931worker1 # [ 150.745438] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423425932worker1 # [ 150.761082] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:423425933worker1 # [ 150.775176] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5934worker1 # [ 150.974615] nix-daemon[851]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)5935worker1 # [ 150.996797] nix-daemon[851]: reaped child process 1704, status = succeeded5936worker1 # [ 151.169107] nix-daemon[851]: accepted connection from pid 1706, user nix-grpc-daemon (trusted)5937worker1 # [ 151.188594] nix-daemon[851]: reaped child process 1709, status = succeeded5938lb # [ 151.199550] niks3-server[1042]: 2026/09/20 10:53:10 INFO Received uploads request method=POST path=/api/pending_closures5939worker1 # [ 151.209864] nix-grpc-daemon[1620]: time=2026-09-20T10:53:10.797Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5940worker1 # [ 151.211381] nix-grpc-daemon[1620]: time=2026-09-20T10:53:10.797Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"5941lb # [ 151.226388] niks3-server[1042]: 2026/09/20 10:53:10 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst5942lb # [ 151.240172] niks3-server[1042]: 2026/09/20 10:53:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5943worker1 # [ 151.246371] nix-grpc-daemon[1620]: time=2026-09-20T10:53:10.834Z level=INFO msg="Uploading 1 narinfos"5944lb # [ 151.252353] niks3-server[1042]: 2026/09/20 10:53:10 INFO Signed narinfos id=50 count=15945lb # [ 151.259040] niks3-server[1042]: 2026/09/20 10:53:10 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls5946lb # [ 151.267953] niks3-server[1042]: 2026/09/20 10:53:10 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5947worker1 # [ 151.272847] nix-grpc-daemon[1620]: time=2026-09-20T10:53:10.860Z level=INFO msg="Upload complete. (481ms)"5948lb # [ 151.272956] niks3-server[1042]: 2026/09/20 10:53:10 INFO Completed upload id=505949lb # [ 151.276862] niks3-server[1042]: 2026/09/20 10:53:10 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo5950worker1 # [ 151.280123] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=0 paths=1 nar_bytes_in=5125951worker1 # [ 151.291435] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423425952worker1 # [ 151.296923] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5953worker1 # [ 151.301513] nix-daemon[851]: reaped child process 1699, status = succeeded5954worker1 # [ 151.312119] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5955worker1 # [ 152.544637] nix-daemon[851]: reaped child process 1713, status = succeeded5956worker1 # [ 152.725017] nix-daemon[851]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)5957worker1 # [ 152.742874] nix-daemon[851]: reaped child process 1731, status = succeeded5958lb # [ 152.754564] niks3-server[1042]: 2026/09/20 10:53:12 INFO Received uploads request method=POST path=/api/pending_closures5959worker1 # [ 152.766581] nix-grpc-daemon[1620]: time=2026-09-20T10:53:12.353Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5960worker1 # [ 152.768236] nix-grpc-daemon[1620]: time=2026-09-20T10:53:12.353Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"5961lb # [ 152.792333] niks3-server[1042]: 2026/09/20 10:53:12 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst5962lb # [ 152.803572] niks3-server[1042]: 2026/09/20 10:53:12 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv5963lb # [ 152.814095] niks3-server[1042]: 2026/09/20 10:53:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5964worker1 # [ 152.821873] nix-grpc-daemon[1620]: time=2026-09-20T10:53:12.409Z level=INFO msg="Uploading 1 narinfos"5965lb # [ 152.824790] niks3-server[1042]: 2026/09/20 10:53:12 INFO Signed narinfos id=51 count=15966lb # [ 152.827988] niks3-server[1042]: 2026/09/20 10:53:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls5967lb # [ 152.840206] niks3-server[1042]: 2026/09/20 10:53:12 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5968lb # [ 152.846127] niks3-server[1042]: 2026/09/20 10:53:12 INFO Completed upload id=515969worker1 # [ 152.846247] nix-grpc-daemon[1620]: time=2026-09-20T10:53:12.434Z level=INFO msg="Upload complete. (322ms)"5970lb # [ 152.849460] niks3-server[1042]: 2026/09/20 10:53:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo5971worker2 # [ 152.861000] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:12Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=15972worker1 # [ 152.858367] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=15973worker2 # [ 152.873697] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:12Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=05974worker1 # [ 152.876162] nix-daemon[851]: reaped child process 1711, status = succeeded5975worker1 # [ 152.901200] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423425976worker1 # [ 152.917178] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423425977worker1 # [ 152.931501] nix-grpc-daemon[1620]: time=2026-09-20T10:53:12.519Z level=INFO msg="All 1 paths already cached"5978worker1 # [ 152.940569] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:423425979worker1 # [ 152.954301] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5980worker1 # [ 153.163288] nix-daemon[851]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)5981worker1 # [ 153.180244] nix-daemon[851]: reaped child process 1738, status = succeeded5982worker1 # [ 153.356086] nix-daemon[851]: accepted connection from pid 1740, user nix-grpc-daemon (trusted)5983worker1 # [ 153.374650] nix-daemon[851]: reaped child process 1743, status = succeeded5984lb # [ 153.385310] niks3-server[1042]: 2026/09/20 10:53:13 INFO Received uploads request method=POST path=/api/pending_closures5985worker1 # [ 153.394348] nix-grpc-daemon[1620]: time=2026-09-20T10:53:12.982Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5986worker1 # [ 153.396271] nix-grpc-daemon[1620]: time=2026-09-20T10:53:12.984Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"5987lb # [ 153.413192] niks3-server[1042]: 2026/09/20 10:53:13 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst5988lb # [ 153.427338] niks3-server[1042]: 2026/09/20 10:53:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5989worker1 # [ 153.434410] nix-grpc-daemon[1620]: time=2026-09-20T10:53:13.022Z level=INFO msg="Uploading 1 narinfos"5990lb # [ 153.440099] niks3-server[1042]: 2026/09/20 10:53:13 INFO Signed narinfos id=52 count=15991lb # [ 153.450910] niks3-server[1042]: 2026/09/20 10:53:13 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls5992worker1 # [ 153.457289] nix-grpc-daemon[1620]: time=2026-09-20T10:53:13.045Z level=INFO msg="Upload complete. (485ms)"5993lb # [ 153.457646] niks3-server[1042]: 2026/09/20 10:53:13 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5994lb # [ 153.463065] niks3-server[1042]: 2026/09/20 10:53:13 INFO Completed upload id=525995worker1 # [ 153.463285] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=0 paths=1 nar_bytes_in=6245996lb # [ 153.465673] niks3-server[1042]: 2026/09/20 10:53:13 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo5997worker1 # [ 153.471028] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423425998worker1 # [ 153.481390] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5999worker1 # [ 153.485926] nix-daemon[851]: reaped child process 1733, status = succeeded6000worker1 # [ 153.497836] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6001worker1 # [ 154.523058] nix-daemon[851]: reaped child process 1747, status = succeeded6002worker1 # [ 154.686514] nix-daemon[851]: accepted connection from pid 1762, user nix-grpc-daemon (trusted)6003worker1 # [ 154.704483] nix-daemon[851]: reaped child process 1765, status = succeeded6004lb # [ 154.717946] niks3-server[1042]: 2026/09/20 10:53:14 INFO Received uploads request method=POST path=/api/pending_closures6005worker1 # [ 154.728408] nix-grpc-daemon[1620]: time=2026-09-20T10:53:14.313Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6006worker1 # [ 154.729936] nix-grpc-daemon[1620]: time=2026-09-20T10:53:14.313Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6007lb # [ 154.749872] niks3-server[1042]: 2026/09/20 10:53:14 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6008lb # [ 154.764499] niks3-server[1042]: 2026/09/20 10:53:14 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6009worker1 # [ 154.782999] nix-grpc-daemon[1620]: time=2026-09-20T10:53:14.370Z level=INFO msg="Uploading 1 narinfos"6010lb # [ 154.777484] niks3-server[1042]: 2026/09/20 10:53:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6011lb # [ 154.786448] niks3-server[1042]: 2026/09/20 10:53:14 INFO Signed narinfos id=53 count=16012lb # [ 154.792116] niks3-server[1042]: 2026/09/20 10:53:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6013lb # [ 154.799705] niks3-server[1042]: 2026/09/20 10:53:14 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6014lb # [ 154.806827] niks3-server[1042]: 2026/09/20 10:53:14 INFO Completed upload id=536015worker1 # [ 154.807330] nix-grpc-daemon[1620]: time=2026-09-20T10:53:14.395Z level=INFO msg="Upload complete. (298ms)"6016lb # [ 154.809975] niks3-server[1042]: 2026/09/20 10:53:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6017worker2 # [ 154.821476] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:14Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16018worker1 # [ 154.821015] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16019worker2 # [ 154.834670] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:14Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06020worker1 # [ 154.836616] nix-daemon[851]: reaped child process 1745, status = succeeded6021worker1 # [ 154.873251] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426022worker1 # [ 154.888684] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423426023worker1 # [ 154.902600] nix-grpc-daemon[1620]: time=2026-09-20T10:53:14.490Z level=INFO msg="All 1 paths already cached"6024worker1 # [ 154.911586] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:423426025worker1 # [ 154.925927] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6026worker1 # [ 155.127957] nix-daemon[851]: accepted connection from pid 1769, user nix-grpc-daemon (trusted)6027worker1 # [ 155.145228] nix-daemon[851]: reaped child process 1772, status = succeeded6028worker1 # [ 155.318403] nix-daemon[851]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6029worker1 # [ 155.335355] nix-daemon[851]: reaped child process 1777, status = succeeded6030lb # [ 155.346147] niks3-server[1042]: 2026/09/20 10:53:14 INFO Received uploads request method=POST path=/api/pending_closures6031worker1 # [ 155.354979] nix-grpc-daemon[1620]: time=2026-09-20T10:53:14.942Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6032worker1 # [ 155.356619] nix-grpc-daemon[1620]: time=2026-09-20T10:53:14.942Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6033lb # [ 155.376945] niks3-server[1042]: 2026/09/20 10:53:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6034worker1 # [ 155.384021] nix-grpc-daemon[1620]: time=2026-09-20T10:53:14.972Z level=INFO msg="Uploading 1 narinfos"6035lb # [ 155.390793] niks3-server[1042]: 2026/09/20 10:53:15 INFO Signed narinfos id=54 count=16036lb # [ 155.398809] niks3-server[1042]: 2026/09/20 10:53:15 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6037lb # [ 155.410636] niks3-server[1042]: 2026/09/20 10:53:15 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6038lb # [ 155.415278] niks3-server[1042]: 2026/09/20 10:53:15 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6039worker1 # [ 155.420790] nix-grpc-daemon[1620]: time=2026-09-20T10:53:15.008Z level=INFO msg="Upload complete. (478ms)"6040lb # [ 155.422444] niks3-server[1042]: 2026/09/20 10:53:15 INFO Completed upload id=546041lb # [ 155.426418] niks3-server[1042]: 2026/09/20 10:53:15 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6042worker1 # [ 155.428260] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=0 paths=1 nar_bytes_in=6246043worker1 # [ 155.440462] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426044worker1 # [ 155.447028] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6045worker1 # [ 155.451865] nix-daemon[851]: reaped child process 1767, status = succeeded6046worker1 # [ 155.464775] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6047worker1 # [ 156.286698] nix-daemon[851]: reaped child process 1781, status = succeeded6048worker1 # [ 156.440370] nix-daemon[851]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6049worker1 # [ 156.459421] nix-daemon[851]: reaped child process 1799, status = succeeded6050lb # [ 156.472906] niks3-server[1042]: 2026/09/20 10:53:16 INFO Received uploads request method=POST path=/api/pending_closures6051worker1 # [ 156.483924] nix-grpc-daemon[1620]: time=2026-09-20T10:53:16.071Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6052worker1 # [ 156.488916] nix-grpc-daemon[1620]: time=2026-09-20T10:53:16.073Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6053lb # [ 156.524887] niks3-server[1042]: 2026/09/20 10:53:16 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6054lb # [ 156.537400] niks3-server[1042]: 2026/09/20 10:53:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6055lb # [ 156.548623] niks3-server[1042]: 2026/09/20 10:53:16 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6056lb # [ 156.556100] niks3-server[1042]: 2026/09/20 10:53:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6057lb # [ 156.561418] niks3-server[1042]: 2026/09/20 10:53:16 INFO Signed narinfos id=55 count=16058worker1 # [ 156.562364] nix-grpc-daemon[1620]: time=2026-09-20T10:53:16.150Z level=INFO msg="Uploading 1 narinfos"6059lb # [ 156.568889] niks3-server[1042]: 2026/09/20 10:53:16 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6060lb # [ 156.576348] niks3-server[1042]: 2026/09/20 10:53:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6061lb # [ 156.588717] niks3-server[1042]: 2026/09/20 10:53:16 INFO Completed upload id=556062worker1 # [ 156.589257] nix-grpc-daemon[1620]: time=2026-09-20T10:53:16.177Z level=INFO msg="Upload complete. (316ms)"6063worker2 # [ 156.601413] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:16Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16064worker1 # [ 156.601241] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16065worker2 # [ 156.618413] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:16Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06066worker1 # [ 156.620571] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426067worker1 # [ 156.632326] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423426068worker1 # [ 156.638457] nix-daemon[851]: reaped child process 1779, status = succeeded6069worker1 # [ 156.643168] nix-grpc-daemon[1620]: time=2026-09-20T10:53:16.231Z level=INFO msg="All 1 paths already cached"6070worker1 # [ 156.651050] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:423426071worker1 # [ 156.664714] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6072worker1 # [ 156.859360] nix-daemon[851]: accepted connection from pid 1803, user nix-grpc-daemon (trusted)6073worker1 # [ 156.878457] nix-daemon[851]: reaped child process 1806, status = succeeded6074worker1 # [ 157.053105] nix-daemon[851]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6075worker1 # [ 157.070033] nix-daemon[851]: reaped child process 1811, status = succeeded6076lb # [ 157.081910] niks3-server[1042]: 2026/09/20 10:53:16 INFO Received uploads request method=POST path=/api/pending_closures6077worker1 # [ 157.095874] nix-grpc-daemon[1620]: time=2026-09-20T10:53:16.683Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6078worker1 # [ 157.097512] nix-grpc-daemon[1620]: time=2026-09-20T10:53:16.683Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6079lb # [ 157.121946] niks3-server[1042]: 2026/09/20 10:53:16 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6080worker1 # [ 157.133635] nix-grpc-daemon[1620]: time=2026-09-20T10:53:16.721Z level=INFO msg="Uploading 1 narinfos"6081lb # [ 157.138699] niks3-server[1042]: 2026/09/20 10:53:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6082lb # [ 157.147613] niks3-server[1042]: 2026/09/20 10:53:16 INFO Signed narinfos id=56 count=16083worker1 # [ 157.153549] nix-grpc-daemon[1620]: time=2026-09-20T10:53:16.741Z level=INFO msg="Upload complete. (473ms)"6084lb # [ 157.154669] niks3-server[1042]: 2026/09/20 10:53:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6085lb # [ 157.159454] niks3-server[1042]: 2026/09/20 10:53:16 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6086lb # [ 157.163968] niks3-server[1042]: 2026/09/20 10:53:16 INFO Completed upload id=566087worker1 # [ 157.163546] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=0 paths=1 nar_bytes_in=6246088lb # [ 157.166086] niks3-server[1042]: 2026/09/20 10:53:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6089worker1 # [ 157.175149] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426090worker1 # [ 157.181059] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6091worker1 # [ 157.185822] nix-daemon[851]: reaped child process 1801, status = succeeded6092worker1 # [ 157.199845] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6093worker1 # [ 158.029412] nix-daemon[851]: reaped child process 1815, status = succeeded6094worker1 # [ 158.199748] nix-daemon[851]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6095worker1 # [ 158.218970] nix-daemon[851]: reaped child process 1833, status = succeeded6096lb # [ 158.231078] niks3-server[1042]: 2026/09/20 10:53:17 INFO Received uploads request method=POST path=/api/pending_closures6097worker1 # [ 158.243139] nix-grpc-daemon[1620]: time=2026-09-20T10:53:17.830Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6098worker1 # [ 158.244760] nix-grpc-daemon[1620]: time=2026-09-20T10:53:17.830Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6099lb # [ 158.264773] niks3-server[1042]: 2026/09/20 10:53:17 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6100lb # [ 158.278591] niks3-server[1042]: 2026/09/20 10:53:17 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6101lb # [ 158.292108] niks3-server[1042]: 2026/09/20 10:53:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6102worker1 # [ 158.298371] nix-grpc-daemon[1620]: time=2026-09-20T10:53:17.886Z level=INFO msg="Uploading 1 narinfos"6103lb # [ 158.302274] niks3-server[1042]: 2026/09/20 10:53:17 INFO Signed narinfos id=57 count=16104lb # [ 158.308316] niks3-server[1042]: 2026/09/20 10:53:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6105lb # [ 158.316560] niks3-server[1042]: 2026/09/20 10:53:17 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6106lb # [ 158.326142] niks3-server[1042]: 2026/09/20 10:53:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6107lb # [ 158.330458] niks3-server[1042]: 2026/09/20 10:53:17 INFO Completed upload id=576108worker1 # [ 158.330794] nix-grpc-daemon[1620]: time=2026-09-20T10:53:17.918Z level=INFO msg="Upload complete. (320ms)"6109worker2 # [ 158.343418] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:17Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16110worker2 # [ 158.356869] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:17Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06111worker1 # [ 158.343139] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16112worker1 # [ 158.363113] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426113worker1 # [ 158.369024] nix-daemon[851]: reaped child process 1813, status = succeeded6114worker1 # [ 158.377579] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423426115worker1 # [ 158.390818] nix-grpc-daemon[1620]: time=2026-09-20T10:53:17.978Z level=INFO msg="All 2 paths already cached"6116worker1 # [ 158.399730] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:423426117worker1 # [ 158.412542] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6118worker1 # [ 158.610802] nix-daemon[851]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6119worker1 # [ 158.629981] nix-daemon[851]: reaped child process 1840, status = succeeded6120worker1 # [ 158.809384] nix-daemon[851]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6121worker1 # [ 158.827018] nix-daemon[851]: reaped child process 1845, status = succeeded6122lb # [ 158.837829] niks3-server[1042]: 2026/09/20 10:53:18 INFO Received uploads request method=POST path=/api/pending_closures6123worker1 # [ 158.847899] nix-grpc-daemon[1620]: time=2026-09-20T10:53:18.435Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6124worker1 # [ 158.849585] nix-grpc-daemon[1620]: time=2026-09-20T10:53:18.435Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6125lb # [ 158.865132] niks3-server[1042]: 2026/09/20 10:53:18 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6126worker1 # [ 158.878355] nix-grpc-daemon[1620]: time=2026-09-20T10:53:18.466Z level=INFO msg="Uploading 1 narinfos"6127lb # [ 158.882648] niks3-server[1042]: 2026/09/20 10:53:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6128lb # [ 158.892559] niks3-server[1042]: 2026/09/20 10:53:18 INFO Signed narinfos id=58 count=16129lb # [ 158.897964] niks3-server[1042]: 2026/09/20 10:53:18 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6130lb # [ 158.907477] niks3-server[1042]: 2026/09/20 10:53:18 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6131lb # [ 158.912916] niks3-server[1042]: 2026/09/20 10:53:18 INFO Completed upload id=586132worker1 # [ 158.912525] nix-grpc-daemon[1620]: time=2026-09-20T10:53:18.500Z level=INFO msg="Upload complete. (484ms)"6133lb # [ 158.915858] niks3-server[1042]: 2026/09/20 10:53:18 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6134worker1 # [ 158.921047] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=0 paths=1 nar_bytes_in=7526135worker1 # [ 158.934721] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426136worker1 # [ 158.940801] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6137worker1 # [ 158.946311] nix-daemon[851]: reaped child process 1835, status = succeeded6138worker1 # [ 158.960165] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6139worker1 # [ 159.983670] nix-daemon[851]: reaped child process 1849, status = succeeded6140worker1 # [ 160.143302] nix-daemon[851]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6141worker1 # [ 160.162001] nix-daemon[851]: reaped child process 1867, status = succeeded6142lb # [ 160.174377] niks3-server[1042]: 2026/09/20 10:53:19 INFO Received uploads request method=POST path=/api/pending_closures6143worker1 # [ 160.187549] nix-grpc-daemon[1620]: time=2026-09-20T10:53:19.772Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6144worker1 # [ 160.189160] nix-grpc-daemon[1620]: time=2026-09-20T10:53:19.772Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6145worker1 # [ 160.227198] nix-grpc-daemon[1620]: time=2026-09-20T10:53:19.815Z level=INFO msg="Uploading 1 narinfos"6146lb # [ 160.217898] niks3-server[1042]: 2026/09/20 10:53:19 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6147lb # [ 160.229862] niks3-server[1042]: 2026/09/20 10:53:19 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6148lb # [ 160.241506] niks3-server[1042]: 2026/09/20 10:53:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6149worker1 # [ 160.247574] nix-grpc-daemon[1620]: time=2026-09-20T10:53:19.835Z level=INFO msg="Upload complete. (277ms)"6150lb # [ 160.247642] niks3-server[1042]: 2026/09/20 10:53:19 INFO Signed narinfos id=59 count=16151lb # [ 160.250228] niks3-server[1042]: 2026/09/20 10:53:19 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6152lb # [ 160.254106] niks3-server[1042]: 2026/09/20 10:53:19 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6153lb # [ 160.257722] niks3-server[1042]: 2026/09/20 10:53:19 INFO Completed upload id=596154worker1 # [ 160.256743] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16155lb # [ 160.259471] niks3-server[1042]: 2026/09/20 10:53:19 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6156worker2 # [ 160.260812] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:19Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16157worker1 # [ 160.268609] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426158worker1 # [ 160.273096] nix-daemon[851]: reaped child process 1847, status = succeeded6159worker1 # [ 160.279977] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423426160worker1 # [ 160.290536] nix-grpc-daemon[1620]: time=2026-09-20T10:53:19.878Z level=INFO msg="All 2 paths already cached"6161worker1 # [ 160.297878] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:423426162worker1 # [ 160.310536] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6163worker1 # [ 160.498935] nix-daemon[851]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6164worker1 # [ 160.519138] nix-daemon[851]: reaped child process 1874, status = succeeded6165worker1 # [ 160.694200] nix-daemon[851]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6166worker1 # [ 160.711735] nix-daemon[851]: reaped child process 1879, status = succeeded6167lb # [ 160.723161] niks3-server[1042]: 2026/09/20 10:53:20 INFO Received uploads request method=POST path=/api/pending_closures6168worker1 # [ 160.732808] nix-grpc-daemon[1620]: time=2026-09-20T10:53:20.320Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6169worker1 # [ 160.734308] nix-grpc-daemon[1620]: time=2026-09-20T10:53:20.320Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6170lb # [ 160.749575] niks3-server[1042]: 2026/09/20 10:53:20 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6171lb # [ 160.763729] niks3-server[1042]: 2026/09/20 10:53:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6172worker1 # [ 160.770316] nix-grpc-daemon[1620]: time=2026-09-20T10:53:20.358Z level=INFO msg="Uploading 1 narinfos"6173lb # [ 160.777260] niks3-server[1042]: 2026/09/20 10:53:20 INFO Signed narinfos id=60 count=16174lb # [ 160.785723] niks3-server[1042]: 2026/09/20 10:53:20 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6175worker1 # [ 160.791905] nix-grpc-daemon[1620]: time=2026-09-20T10:53:20.380Z level=INFO msg="Upload complete. (469ms)"6176lb # [ 160.792556] niks3-server[1042]: 2026/09/20 10:53:20 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6177lb # [ 160.797501] niks3-server[1042]: 2026/09/20 10:53:20 INFO Completed upload id=606178lb # [ 160.800734] niks3-server[1042]: 2026/09/20 10:53:20 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6179worker1 # [ 160.800222] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=0 paths=1 nar_bytes_in=7526180worker1 # [ 160.811680] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426181worker1 # [ 160.820294] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6182worker1 # [ 160.823005] nix-daemon[851]: reaped child process 1869, status = succeeded6183worker1 # [ 160.835998] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6184worker1 # [ 161.677153] nix-daemon[851]: reaped child process 1883, status = succeeded6185worker1 # [ 161.848675] nix-daemon[851]: accepted connection from pid 1898, user nix-grpc-daemon (trusted)6186worker1 # [ 161.868117] nix-daemon[851]: reaped child process 1901, status = succeeded6187lb # [ 161.881998] niks3-server[1042]: 2026/09/20 10:53:21 INFO Received uploads request method=POST path=/api/pending_closures6188worker1 # [ 161.891353] nix-grpc-daemon[1620]: time=2026-09-20T10:53:21.478Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6189worker1 # [ 161.892960] nix-grpc-daemon[1620]: time=2026-09-20T10:53:21.478Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6190lb # [ 161.917381] niks3-server[1042]: 2026/09/20 10:53:21 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6191worker1 # [ 161.931456] nix-grpc-daemon[1620]: time=2026-09-20T10:53:21.519Z level=INFO msg="Uploading 1 narinfos"6192lb # [ 161.932416] niks3-server[1042]: 2026/09/20 10:53:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6193lb # [ 161.942842] niks3-server[1042]: 2026/09/20 10:53:21 INFO Signed narinfos id=61 count=16194lb # [ 161.949949] niks3-server[1042]: 2026/09/20 10:53:21 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6195lb # [ 161.954704] niks3-server[1042]: 2026/09/20 10:53:21 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6196worker1 # [ 161.962848] nix-grpc-daemon[1620]: time=2026-09-20T10:53:21.550Z level=INFO msg="Upload complete. (317ms)"6197lb # [ 161.962513] niks3-server[1042]: 2026/09/20 10:53:21 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6198lb # [ 161.966597] niks3-server[1042]: 2026/09/20 10:53:21 INFO Completed upload id=616199lb # [ 161.969654] niks3-server[1042]: 2026/09/20 10:53:21 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6200worker2 # [ 161.975677] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:21Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16201worker1 # [ 161.973121] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16202worker1 # [ 161.996825] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426203worker1 # [ 162.000628] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423426204worker1 # [ 162.005330] nix-daemon[851]: reaped child process 1881, status = succeeded6205worker1 # [ 162.011748] nix-grpc-daemon[1620]: time=2026-09-20T10:53:21.599Z level=INFO msg="All 2 paths already cached"6206worker1 # [ 162.019796] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:423426207worker1 # [ 162.030476] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6208worker1 # [ 162.216439] nix-daemon[851]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6209worker1 # [ 162.233600] nix-daemon[851]: reaped child process 1908, status = succeeded6210worker1 # [ 162.407448] nix-daemon[851]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6211worker1 # [ 162.424107] nix-daemon[851]: reaped child process 1913, status = succeeded6212lb # [ 162.438304] niks3-server[1042]: 2026/09/20 10:53:22 INFO Received uploads request method=POST path=/api/pending_closures6213worker1 # [ 162.447697] nix-grpc-daemon[1620]: time=2026-09-20T10:53:22.035Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6214worker1 # [ 162.449366] nix-grpc-daemon[1620]: time=2026-09-20T10:53:22.035Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6215lb # [ 162.464165] niks3-server[1042]: 2026/09/20 10:53:22 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6216lb # [ 162.477764] niks3-server[1042]: 2026/09/20 10:53:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6217worker1 # [ 162.485653] nix-grpc-daemon[1620]: time=2026-09-20T10:53:22.073Z level=INFO msg="Uploading 1 narinfos"6218lb # [ 162.490135] niks3-server[1042]: 2026/09/20 10:53:22 INFO Signed narinfos id=62 count=16219lb # [ 162.496302] niks3-server[1042]: 2026/09/20 10:53:22 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6220lb # [ 162.505468] niks3-server[1042]: 2026/09/20 10:53:22 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6221lb # [ 162.510274] niks3-server[1042]: 2026/09/20 10:53:22 INFO Completed upload id=626222worker1 # [ 162.510748] nix-grpc-daemon[1620]: time=2026-09-20T10:53:22.098Z level=INFO msg="Upload complete. (468ms)"6223lb # [ 162.512868] niks3-server[1042]: 2026/09/20 10:53:22 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6224worker1 # [ 162.520167] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=0 paths=1 nar_bytes_in=7526225worker1 # [ 162.530652] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:423426226worker1 # [ 162.536638] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6227worker1 # [ 162.541390] nix-daemon[851]: reaped child process 1903, status = succeeded6228worker1 # [ 162.554258] nix-daemon[851]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6229worker1 # [ 163.983848] nix-daemon[851]: reaped child process 1917, status = succeeded6230worker1 # [ 164.155812] nix-daemon[851]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6231worker1 # [ 164.178576] nix-daemon[851]: reaped child process 1935, status = succeeded6232lb # [ 164.191534] niks3-server[1042]: 2026/09/20 10:53:23 INFO Received uploads request method=POST path=/api/pending_closures6233worker1 # [ 164.202950] nix-grpc-daemon[1620]: time=2026-09-20T10:53:23.789Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6234worker1 # [ 164.205745] nix-grpc-daemon[1620]: time=2026-09-20T10:53:23.789Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6235lb # [ 164.231398] niks3-server[1042]: 2026/09/20 10:53:23 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6236lb # [ 164.243558] niks3-server[1042]: 2026/09/20 10:53:23 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6237lb # [ 164.255890] niks3-server[1042]: 2026/09/20 10:53:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6238worker1 # [ 164.261283] nix-grpc-daemon[1620]: time=2026-09-20T10:53:23.849Z level=INFO msg="Uploading 1 narinfos"6239lb # [ 164.266846] niks3-server[1042]: 2026/09/20 10:53:23 INFO Signed narinfos id=63 count=16240lb # [ 164.272749] niks3-server[1042]: 2026/09/20 10:53:23 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6241lb # [ 164.282408] niks3-server[1042]: 2026/09/20 10:53:23 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6242lb # [ 164.288399] niks3-server[1042]: 2026/09/20 10:53:23 INFO Completed upload id=636243worker1 # [ 164.288970] nix-grpc-daemon[1620]: time=2026-09-20T10:53:23.877Z level=INFO msg="Upload complete. (324ms)"6244lb # [ 164.291486] niks3-server[1042]: 2026/09/20 10:53:23 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6245worker1 # [ 164.301116] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42342 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16246worker1 # [ 164.318960] nix-daemon[851]: reaped child process 1915, status = succeeded6247client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 19.83 seconds)6248worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6249worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6250client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6251client # [ 164.502524] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6252client # Running as unit: gpu.service; invocation ID: 7ef765cd37fc45a7a1e8c5e014de40a86253client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6254lb: must succeed: curl -sf localhost:9901/clusters6255lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6256lb: must succeed: curl -sf localhost:9901/clusters6257lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6258worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6259worker2 # [ 164.775006] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:578086260client # [ 164.752270] nix[1337]: don't know how to build these paths:6261client # [ 164.753688] nix[1337]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6262worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6263worker1 # [ 164.783684] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:423426264worker2 # [ 164.814014] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:24Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06265client # [ 164.792864] nix[1337]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6266lb: must succeed: curl -sf localhost:9901/clusters6267lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6268lb: must succeed: curl -sf localhost:9901/clusters6269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6270worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6271worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6272client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6273lb # [ 170.423624] envoy[913]: conn peer=[2001:db8:1::1]:43956 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27118 tx=10678 ms=195226274client # [ 194.795166] nix[1337]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6275client # [ 194.809250] nix[1337]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6276client # [ 194.854587] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6277client # [ 194.860844] systemd[1]: gpu.service: Failed with result 'exit-code'.6278client # [ 194.867153] systemd[1]: gpu.service: Consumed 172ms CPU time over 30.357s wall clock time, 33.2M memory peak, 6.7K incoming IP traffic, 5.8K outgoing IP traffic.6279client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.39 seconds)6280client: must succeed: journalctl -u gpu -o cat6281client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6282lb: must succeed: curl -sf localhost:9901/clusters6283lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6284lb: must succeed: curl -sf localhost:9901/clusters6285lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6286worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6287worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6288(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.10 seconds)6289subtest: low disk drains a worker and builds go to the other6290worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6291worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.13 seconds)6292worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6293worker1 # [ 197.246773] nix-grpc-daemon[1564]: ts=2026-09-20T10:53:56Z level=info event=unhealthy reason=min_free6294worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.13 seconds)6295client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26296client # don't know how to build these paths:6297client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6298worker2 # [ 198.169207] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:578086299worker2 # [ 198.176984] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578086300worker2 # [ 198.208958] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:57Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06301worker2 # [ 198.216216] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:57Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16302worker2 # [ 198.256172] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578086303worker2 # [ 198.272401] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578086304client # copying 1 paths to 'lb:50051'...6305client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6306worker2 # [ 198.288111] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578086307worker2 # [ 198.305958] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6308worker2 # [ 198.517768] nix-daemon[839]: accepted connection from pid 1758, user nix-grpc-daemon (trusted)6309worker2 # [ 198.537168] nix-daemon[839]: reaped child process 1761, status = succeeded6310worker2 # [ 198.708640] nix-daemon[839]: accepted connection from pid 1763, user nix-grpc-daemon (trusted)6311worker2 # [ 198.726239] nix-daemon[839]: reaped child process 1766, status = succeeded6312lb # [ 198.739568] niks3-server[1042]: 2026/09/20 10:53:58 INFO Received uploads request method=POST path=/api/pending_closures6313worker2 # [ 198.753615] nix-grpc-daemon[1547]: time=2026-09-20T10:53:58.373Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6314worker2 # [ 198.755205] nix-grpc-daemon[1547]: time=2026-09-20T10:53:58.373Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6315lb # [ 198.770054] niks3-server[1042]: 2026/09/20 10:53:58 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6316lb # [ 198.781642] niks3-server[1042]: 2026/09/20 10:53:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6317worker2 # [ 198.792646] nix-grpc-daemon[1547]: time=2026-09-20T10:53:58.412Z level=INFO msg="Uploading 1 narinfos"6318lb # [ 198.796333] niks3-server[1042]: 2026/09/20 10:53:58 INFO Signed narinfos id=64 count=16319lb # [ 198.802430] niks3-server[1042]: 2026/09/20 10:53:58 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6320lb # [ 198.812699] niks3-server[1042]: 2026/09/20 10:53:58 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6321lb # [ 198.817875] niks3-server[1042]: 2026/09/20 10:53:58 INFO Completed upload id=646322worker2 # [ 198.822074] nix-grpc-daemon[1547]: time=2026-09-20T10:53:58.442Z level=INFO msg="Upload complete. (496ms)"6323lb # [ 198.820417] niks3-server[1042]: 2026/09/20 10:53:58 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6324worker2 # [ 198.831789] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=0 paths=1 nar_bytes_in=5126325worker2 # [ 198.841123] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578086326worker2 # [ 198.851843] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6327worker2 # [ 198.857150] nix-daemon[839]: reaped child process 1756, status = succeeded6328client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6329client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6330worker2 # [ 198.871420] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6331client # farm-a> LOG-farm-a-drain6332worker2 # [ 199.714169] nix-daemon[839]: reaped child process 1770, status = succeeded6333worker2 # [ 199.893876] nix-daemon[839]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6334worker2 # [ 199.911398] nix-daemon[839]: reaped child process 1788, status = succeeded6335lb # [ 199.923385] niks3-server[1042]: 2026/09/20 10:53:59 INFO Received uploads request method=POST path=/api/pending_closures6336worker2 # [ 199.939285] nix-grpc-daemon[1547]: time=2026-09-20T10:53:59.556Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6337worker2 # [ 199.940913] nix-grpc-daemon[1547]: time=2026-09-20T10:53:59.556Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6338lb # [ 199.968232] niks3-server[1042]: 2026/09/20 10:53:59 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6339worker2 # [ 199.982905] nix-grpc-daemon[1547]: time=2026-09-20T10:53:59.602Z level=INFO msg="Uploading 1 narinfos"6340lb # [ 199.983573] niks3-server[1042]: 2026/09/20 10:53:59 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6341lb # [ 199.992990] niks3-server[1042]: 2026/09/20 10:53:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6342lb # [ 200.002287] niks3-server[1042]: 2026/09/20 10:53:59 INFO Signed narinfos id=65 count=16343worker2 # [ 200.006226] nix-grpc-daemon[1547]: time=2026-09-20T10:53:59.626Z level=INFO msg="Upload complete. (323ms)"6344lb # [ 200.004973] niks3-server[1042]: 2026/09/20 10:53:59 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6345lb # [ 200.009196] niks3-server[1042]: 2026/09/20 10:53:59 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6346lb # [ 200.012965] niks3-server[1042]: 2026/09/20 10:53:59 INFO Completed upload id=656347lb # [ 200.015022] niks3-server[1042]: 2026/09/20 10:53:59 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6348worker2 # [ 200.019270] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16349worker2 # [ 200.033395] nix-daemon[839]: reaped child process 1768, status = succeeded6350worker2 # [ 200.036572] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:59Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06351worker2 # [ 200.041025] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:59Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16352worker2 # [ 200.084362] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578086353worker2 # [ 200.099727] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578086354worker2 # [ 200.115359] nix-grpc-daemon[1547]: time=2026-09-20T10:53:59.735Z level=INFO msg="All 1 paths already cached"6355client # copying 1 paths to 'lb:50051'...6356client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6357worker2 # [ 200.125646] nix-grpc-daemon[1471]: ts=2026-09-20T10:53:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578086358worker2 # [ 200.140906] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6359worker2 # [ 200.339918] nix-daemon[839]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6360worker2 # [ 200.358847] nix-daemon[839]: reaped child process 1797, status = succeeded6361lb # [ 200.422445] envoy[913]: conn peer=[2001:db8:1::1]:39420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300746362worker2 # [ 200.528940] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6363worker2 # [ 200.545201] nix-daemon[839]: reaped child process 1802, status = succeeded6364lb # [ 200.553870] niks3-server[1042]: 2026/09/20 10:54:00 INFO Received uploads request method=POST path=/api/pending_closures6365worker2 # [ 200.566693] nix-grpc-daemon[1547]: time=2026-09-20T10:54:00.186Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6366worker2 # [ 200.568356] nix-grpc-daemon[1547]: time=2026-09-20T10:54:00.186Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6367lb # [ 200.579300] niks3-server[1042]: 2026/09/20 10:54:00 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6368worker2 # [ 200.597738] nix-grpc-daemon[1547]: time=2026-09-20T10:54:00.217Z level=INFO msg="Uploading 1 narinfos"6369lb # [ 200.593614] niks3-server[1042]: 2026/09/20 10:54:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6370lb # [ 200.605545] niks3-server[1042]: 2026/09/20 10:54:00 INFO Signed narinfos id=66 count=16371lb # [ 200.612394] niks3-server[1042]: 2026/09/20 10:54:00 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6372lb # [ 200.622440] niks3-server[1042]: 2026/09/20 10:54:00 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6373worker2 # [ 200.631051] nix-grpc-daemon[1547]: time=2026-09-20T10:54:00.251Z level=INFO msg="Upload complete. (473ms)"6374lb # [ 200.627542] niks3-server[1042]: 2026/09/20 10:54:00 INFO Completed upload id=666375lb # [ 200.630543] niks3-server[1042]: 2026/09/20 10:54:00 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6376worker2 # [ 200.640237] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=0 paths=1 nar_bytes_in=6326377worker2 # [ 200.651880] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578086378worker2 # [ 200.659147] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6379worker2 # [ 200.664193] nix-daemon[839]: reaped child process 1792, status = succeeded6380client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6381client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6382worker2 # [ 200.677680] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6383client # farm-b> LOG-farm-b-drain6384worker2 # [ 201.522159] nix-daemon[839]: reaped child process 1806, status = succeeded6385worker2 # [ 201.695153] nix-daemon[839]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6386worker2 # [ 201.713778] nix-daemon[839]: reaped child process 1824, status = succeeded6387lb # [ 201.723452] niks3-server[1042]: 2026/09/20 10:54:01 INFO Received uploads request method=POST path=/api/pending_closures6388worker2 # [ 201.740396] nix-grpc-daemon[1547]: time=2026-09-20T10:54:01.358Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6389worker2 # [ 201.741872] nix-grpc-daemon[1547]: time=2026-09-20T10:54:01.358Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6390lb # [ 201.765120] niks3-server[1042]: 2026/09/20 10:54:01 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6391lb # [ 201.777748] niks3-server[1042]: 2026/09/20 10:54:01 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6392lb # [ 201.788571] niks3-server[1042]: 2026/09/20 10:54:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6393worker2 # [ 201.798162] nix-grpc-daemon[1547]: time=2026-09-20T10:54:01.417Z level=INFO msg="Uploading 1 narinfos"6394lb # [ 201.794210] niks3-server[1042]: 2026/09/20 10:54:01 INFO Signed narinfos id=67 count=16395lb # [ 201.800472] niks3-server[1042]: 2026/09/20 10:54:01 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6396lb # [ 201.812102] niks3-server[1042]: 2026/09/20 10:54:01 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6397lb # [ 201.818745] niks3-server[1042]: 2026/09/20 10:54:01 INFO Completed upload id=676398worker2 # [ 201.823020] nix-grpc-daemon[1547]: time=2026-09-20T10:54:01.443Z level=INFO msg="Upload complete. (335ms)"6399lb # [ 201.822103] niks3-server[1042]: 2026/09/20 10:54:01 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6400worker2 # [ 201.844242] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16401worker2 # [ 201.856189] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:01Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06402worker2 # [ 201.859988] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:01Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16403worker2 # [ 201.864164] nix-daemon[839]: reaped child process 1804, status = succeeded6404worker2 # [ 201.887028] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578086405worker2 # [ 201.901565] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578086406worker2 # [ 201.915248] nix-grpc-daemon[1547]: time=2026-09-20T10:54:01.535Z level=INFO msg="All 2 paths already cached"6407client # copying 1 paths to 'lb:50051'...6408client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6409worker2 # [ 201.925421] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578086410worker2 # [ 201.938701] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6411worker2 # [ 202.136565] nix-daemon[839]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6412worker2 # [ 202.154146] nix-daemon[839]: reaped child process 1833, status = succeeded6413worker2 # [ 202.322367] nix-daemon[839]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)6414worker2 # [ 202.338429] nix-daemon[839]: reaped child process 1838, status = succeeded6415lb # [ 202.347141] niks3-server[1042]: 2026/09/20 10:54:01 INFO Received uploads request method=POST path=/api/pending_closures6416worker2 # [ 202.360028] nix-grpc-daemon[1547]: time=2026-09-20T10:54:01.979Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6417worker2 # [ 202.361523] nix-grpc-daemon[1547]: time=2026-09-20T10:54:01.979Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6418lb # [ 202.373806] niks3-server[1042]: 2026/09/20 10:54:01 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6419worker2 # [ 202.389410] nix-grpc-daemon[1547]: time=2026-09-20T10:54:02.009Z level=INFO msg="Uploading 1 narinfos"6420lb # [ 202.389866] niks3-server[1042]: 2026/09/20 10:54:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6421lb # [ 202.400305] niks3-server[1042]: 2026/09/20 10:54:02 INFO Signed narinfos id=68 count=16422lb # [ 202.403177] niks3-server[1042]: 2026/09/20 10:54:02 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6423lb # [ 202.409652] niks3-server[1042]: 2026/09/20 10:54:02 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6424worker2 # [ 202.420286] nix-grpc-daemon[1547]: time=2026-09-20T10:54:02.040Z level=INFO msg="Upload complete. (465ms)"6425lb # [ 202.416936] niks3-server[1042]: 2026/09/20 10:54:02 INFO Completed upload id=686426lb # [ 202.420798] niks3-server[1042]: 2026/09/20 10:54:02 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6427worker2 # [ 202.428124] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=0 paths=1 nar_bytes_in=7606428worker2 # [ 202.439807] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578086429worker2 # [ 202.445913] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6430worker2 # [ 202.450512] nix-daemon[839]: reaped child process 1828, status = succeeded6431client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6432worker2 # [ 202.463209] nix-daemon[839]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6433client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6434client # farm-top> LOG-farm-top-drain6435worker2 # [ 203.510539] nix-daemon[839]: reaped child process 1842, status = succeeded6436worker2 # [ 203.685484] nix-daemon[839]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6437worker2 # [ 203.704223] nix-daemon[839]: reaped child process 1860, status = succeeded6438lb # [ 203.717529] niks3-server[1042]: 2026/09/20 10:54:03 INFO Received uploads request method=POST path=/api/pending_closures6439worker2 # [ 203.732473] nix-grpc-daemon[1547]: time=2026-09-20T10:54:03.351Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6440worker2 # [ 203.734175] nix-grpc-daemon[1547]: time=2026-09-20T10:54:03.351Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6441lb # [ 203.764244] niks3-server[1042]: 2026/09/20 10:54:03 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6442worker2 # [ 203.781879] nix-grpc-daemon[1547]: time=2026-09-20T10:54:03.401Z level=INFO msg="Uploading 1 narinfos"6443lb # [ 203.779724] niks3-server[1042]: 2026/09/20 10:54:03 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6444lb # [ 203.789477] niks3-server[1042]: 2026/09/20 10:54:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6445lb # [ 203.794882] niks3-server[1042]: 2026/09/20 10:54:03 INFO Signed narinfos id=69 count=16446lb # [ 203.800837] niks3-server[1042]: 2026/09/20 10:54:03 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6447lb # [ 203.805088] niks3-server[1042]: 2026/09/20 10:54:03 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6448lb # [ 203.808770] niks3-server[1042]: 2026/09/20 10:54:03 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6449worker2 # [ 203.812775] nix-grpc-daemon[1547]: time=2026-09-20T10:54:03.432Z level=INFO msg="Upload complete. (338ms)"6450lb # [ 203.811764] niks3-server[1042]: 2026/09/20 10:54:03 INFO Completed upload id=696451worker2 # [ 203.825168] nix-grpc-daemon[1471]: ts=2026-09-20T10:54:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57808 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16452worker2 # [ 203.842291] nix-daemon[839]: reaped child process 1840, status = succeeded6453client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 5.96 seconds)6454client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6455client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.24 seconds)6456worker1: must succeed: rm /nix/.rw-store/fill6457worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6458worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6459worker1 # [ 205.257181] nix-grpc-daemon[1564]: ts=2026-09-20T10:54:04Z level=info event=healthy reason=min_free6460worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.13 seconds)6461(finished: subtest: low disk drains a worker and builds go to the other, in 9.83 seconds)6462(finished: run the VM test script, in 206.36 seconds)6463test script finished in 206.65s6464cleanup6465kill QemuMachine (pid 49)6466client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6467kill QemuMachine (pid 50)6468lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6469kill QemuMachine (pid 51)6470worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6471kill QemuMachine (pid 52)6472worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6473(finished: cleanup, in 1.00 seconds)6474additionally exposed symbols:6475 client, lb, worker1, worker2,6476 vlan1,6477 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh