vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #492
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.01 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1: QEMU running (pid 51)14worker2 # Disk image does not exist, creating the virtualisation disk image...15worker2: QEMU running (pid 52)16worker1 # Formatting '/build/vm-state-worker1/tmp.dXiobIir6P', fmt=raw size=107374182417client # Disk image does not exist, creating the virtualisation disk image...18lb # Disk image does not exist, creating the virtualisation disk image...19lb: QEMU running (pid 49)20worker2 # Formatting '/build/vm-state-worker2/tmp.M9cLLboM0W', fmt=raw size=107374182421client: QEMU running (pid 50)22worker1 # mke2fs 1.47.4 (6-Mar-2025)23worker2 # mke2fs 1.47.4 (6-Mar-2025)24client # Formatting '/build/vm-state-client/tmp.tezcDG5AML', fmt=raw size=107374182425lb # Formatting '/build/vm-state-lb/tmp.r3reIge0GV', fmt=raw size=107374182426lb # mke2fs 1.47.4 (6-Mar-2025)27worker2 # Discarding device blocks: 0/262144 done28worker1 # Discarding device blocks: 0/262144 done29(finished: start all VMs, in 0.51 seconds)30client # mke2fs 1.47.4 (6-Mar-2025)31lb # Discarding device blocks: 0/262144 done32worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes33worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes34lb # Creating filesystem with 262144 4k blocks and 65536 inodes35worker2 # Filesystem UUID: 1901d721-ca8c-4062-ae20-6ee1aeb92fa236worker1 # Filesystem UUID: b7494a4a-0c71-4154-867d-11aa1912d09337lb # Filesystem UUID: 81debc9d-e7d6-4829-8d96-4b8ab9939e2738worker2 # Superblock backups stored on blocks:39worker1 # Superblock backups stored on blocks:40lb # Superblock backups stored on blocks:41worker2 # 32768, 98304, 163840, 22937642worker1 # 32768, 98304, 163840, 22937643lb # 32768, 98304, 163840, 22937644worker2 # 45worker1 # 46lb # 47worker2 # Allocating group tables: 0/8 done48client: waiting for unit network-addresses-eth1.service49lb # Allocating group tables: 0/8 done50worker2 # Writing inode tables: 0/8 done51client: waiting for the VM to finish booting52lb # Writing inode tables: 0/8 done53worker2 # Creating journal (8192 blocks): done54client # Discarding device blocks: 0/262144 done55worker1 # Allocating group tables: 0/8 done56worker2 # Writing superblocks and filesystem accounting information: 0/8 done57client # Creating filesystem with 262144 4k blocks and 65536 inodes58lb # Creating journal (8192 blocks): done59client # Filesystem UUID: bbd434e5-d1a0-43bf-a370-65bcbd9a72f560lb # Writing superblocks and filesystem accounting information: 0/8 done61worker2 # 62worker1 # Writing inode tables: 0/8 done63client # Superblock backups stored on blocks:64lb # 65worker2 # Virtualisation disk image created.66worker1 # Creating journal (8192 blocks): done67client # 32768, 98304, 163840, 22937668worker1 # Writing superblocks and filesystem accounting information: 0/8 done69client # 70worker1 # 71client # Allocating group tables: 0/8 done72worker1 # Virtualisation disk image created.73client # Writing inode tables: 0/8 done74lb # Virtualisation disk image created.75client # Creating journal (8192 blocks): done76client # Writing superblocks and filesystem accounting information: 0/8 done77client # 78client # Virtualisation disk image created.79worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker1 # [ 0.000000] KASLR enabled82worker1 # [ 0.000000] random: crng init done83worker1 # [ 0.000000] Machine model: linux,dummy-virt84worker1 # [ 0.000000] efi: UEFI not found.85worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88worker1 # [ 0.000000] Zone ranges:89worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90worker1 # [ 0.000000] DMA32 empty91worker1 # [ 0.000000] Normal empty92worker1 # [ 0.000000] Device empty93worker1 # [ 0.000000] Movable zone start for each node94worker1 # [ 0.000000] Early memory node ranges95worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098worker1 # [ 0.000000] psci: probing for conduit method from DT.99worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.100worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101worker1 # [ 0.000000] psci: Trusted OS migration not required102worker1 # [ 0.000000] psci: SMC Calling Convention v1.1103worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105worker1 # [ 0.000000] Detected PIPT I-cache on CPU0106worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface108worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]109worker1 # [ 0.000000] CPU features: detected: Spectre-v4110worker1 # [ 0.000000] CPU features: detected: Spectre-BHB111worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38112worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026113worker2 # [ 0.000000] KASLR enabled114worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23115worker2 # [ 0.000000] random: crng init done116worker1 # [ 0.000000] alternatives: applying boot alternatives117worker2 # [ 0.000000] Machine model: linux,dummy-virt118worker2 # [ 0.000000] efi: UEFI not found.119worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT120worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]121worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]122worker2 # [ 0.000000] Zone ranges:123worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hl1ynmh5snhcjq7al2fl9h42d6chq0rx-nixos-system-worker1-test/init regInfo=/nix/store/z498f4bnd0bm1y7j2vivsc2m8rqscg04-closure-info/registration console=ttyAMA0,115200n8 console=tty0124worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]125worker2 # [ 0.000000] DMA32 empty126worker2 # [ 0.000000] Normal empty127worker2 # [ 0.000000] Device empty128worker2 # [ 0.000000] Movable zone start for each node129worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/z498f4bnd0bm1y7j2vivsc2m8rqscg04-closure-info/registration", will be passed to user space.130worker2 # [ 0.000000] Early memory node ranges131worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes132worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]133worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)134worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]135worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000136worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)137worker2 # [ 0.000000] psci: probing for conduit method from DT.138worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB139worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.140worker1 # [ 0.000000] software IO TLB: area num 1.141worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs142worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB143worker2 # [ 0.000000] psci: Trusted OS migration not required144worker2 # [ 0.000000] psci: SMC Calling Convention v1.1145worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)146worker1 # [ 0.000000] Fallback order for Node 0: 0147worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)148worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216149worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296150worker1 # [ 0.000000] Policy zone: DMA151worker2 # [ 0.000000] Detected PIPT I-cache on CPU0152worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off153worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1154worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)155worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface156worker1 # [ 0.000000] allocated 3145728 bytes of page_ext157worker2 # [ 0.000000] CPU features: detected: Spectre-v4158worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages159worker2 # [ 0.000000] CPU features: detected: Spectre-BHB160lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]161worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups162worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.164worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23165worker1 # [ 0.000000] rcu: RCU event tracing is enabled.166lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026167worker2 # [ 0.000000] alternatives: applying boot alternatives168lb # [ 0.000000] KASLR enabled169worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.170lb # [ 0.000000] random: crng init done171worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.172lb # [ 0.000000] Machine model: linux,dummy-virt173lb # [ 0.000000] efi: UEFI not found.174worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.175worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.176lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT177worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.178lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]179worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/02j81afdj8xzihnmsy7rp3hq1zfspx7w-nixos-system-worker2-test/init regInfo=/nix/store/b12amaslgxhmmal4fdpicv0pb8fqziw4-closure-info/registration console=ttyAMA0,115200n8 console=tty0180worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1181lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]182lb # [ 0.000000] Zone ranges:183worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.184lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]185lb # [ 0.000000] DMA32 empty186worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/b12amaslgxhmmal4fdpicv0pb8fqziw4-closure-info/registration", will be passed to user space.187lb # [ 0.000000] Normal empty188worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.189lb # [ 0.000000] Device empty190worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes191lb # [ 0.000000] Movable zone start for each node192lb # [ 0.000000] Early memory node ranges193worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.194worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)195worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0196lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]197worker1 # [ 0.000000] GICv3: 256 SPIs implemented198worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)199lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]200worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented201worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB202worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq203lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000204worker2 # [ 0.000000] software IO TLB: area num 1.205worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI206lb # [ 0.000000] psci: probing for conduit method from DT.207worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB208worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0209lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.210lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs211worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)212worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000213lb # [ 0.000000] psci: Trusted OS migration not required214worker2 # [ 0.000000] Fallback order for Node 0: 0215worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]216lb # [ 0.000000] psci: SMC Calling Convention v1.1217worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216218worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)219worker2 # [ 0.000000] Policy zone: DMA220lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)221worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off222lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296223worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)224client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]225worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1226lb # [ 0.000000] Detected PIPT I-cache on CPU0227worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000228worker2 # [ 0.000000] allocated 3145728 bytes of page_ext229lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)230worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages231worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000232client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026233client # [ 0.000000] KASLR enabled234lb # [ 0.000000] CPU features: detected: GICv3 CPU interface235worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups236worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.237client # [ 0.000000] random: crng init done238lb # [ 0.000000] CPU features: detected: Spectre-v4239worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.240client # [ 0.000000] Machine model: linux,dummy-virt241lb # [ 0.000000] CPU features: detected: Spectre-BHB242worker2 # [ 0.000000] rcu: RCU event tracing is enabled.243client # [ 0.000000] efi: UEFI not found.244worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns245lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38246worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.247client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT248worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).249lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23250worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.251client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]252worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.253lb # [ 0.000000] alternatives: applying boot alternatives254worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns255client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]256worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.257client # [ 0.000000] Zone ranges:258worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns259worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.260client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]261worker1 # [ 0.000038] arm-pv: using stolen time PV262client # [ 0.000000] DMA32 empty263worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1264client # [ 0.000000] Normal empty265client # [ 0.000000] Device empty266worker1 # [ 0.000793] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)267client # [ 0.000000] Movable zone start for each node268worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.269worker1 # [ 0.000989] Console: colour dummy device 80x25270client # [ 0.000000] Early memory node ranges271lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mmnw0b9xdzgvlxb4mzaa4gjdldj887b1-nixos-system-lb-test/init regInfo=/nix/store/mmqsdvj4977m1gdgnvij2v3i1inliqqm-closure-info/registration console=ttyAMA0,115200n8 console=tty0272worker1 # [ 0.000998] printk: legacy console [tty0] enabled273worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.274client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]275client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]276worker1 # [ 0.001203] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)277lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/mmqsdvj4977m1gdgnvij2v3i1inliqqm-closure-info/registration", will be passed to user space.278worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.279client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000280worker1 # [ 0.001210] pid_max: default: 32768 minimum: 301281worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0282lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes283client # [ 0.000000] psci: probing for conduit method from DT.284worker2 # [ 0.000000] GICv3: 256 SPIs implemented285worker1 # [ 0.001287] LSM: initializing lsm=capability,landlock,yama,bpf,ima286client # [ 0.000000] psci: PSCIv1.3 detected in firmware.287worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented288worker1 # [ 0.001507] landlock: Up and running.289lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)290worker1 # [ 0.001510] Yama: becoming mindful.291worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq292client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs293worker1 # [ 0.002214] LSM support for eBPF active294client # [ 0.000000] psci: Trusted OS migration not required295worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI296lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)297worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0298client # [ 0.000000] psci: SMC Calling Convention v1.1299worker1 # [ 0.002375] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)300lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB301lb # [ 0.000000] software IO TLB: area num 1.302worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000303client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)304worker1 # [ 0.002406] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)305worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]306lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB307client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296308worker1 # [ 0.003641] cacheinfo: Unable to detect cache hierarchy for CPU 0309client # [ 0.000000] Detected PIPT I-cache on CPU0310lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)311worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)312worker1 # [ 0.004453] rcu: Hierarchical SRCU implementation.313lb # [ 0.000000] Fallback order for Node 0: 0314worker1 # [ 0.004459] rcu: Max phase no-delay instances is 1000.315client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)316worker1 # [ 0.005724] fsl-mc MSI: its@8080000 domain created317lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216318worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)319client # [ 0.000000] CPU features: detected: GICv3 CPU interface320lb # [ 0.000000] Policy zone: DMA321worker1 # [ 0.005818] EFI services will not be available.322worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000323client # [ 0.000000] CPU features: detected: Spectre-v4324worker1 # [ 0.005897] smp: Bringing up secondary CPUs ...325lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off326client # [ 0.000000] CPU features: detected: Spectre-BHB327worker1 # [ 0.005907] smp: Brought up 1 node, 1 CPU328worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000329lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1330client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38331worker1 # [ 0.005910] SMP: Total of 1 processors activated.332worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.333lb # [ 0.000000] allocated 3145728 bytes of page_ext334worker1 # [ 0.005917] CPU: All CPU(s) started at EL1335client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23336lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages337worker1 # [ 0.005930] CPU features: detected: Branch Target Identification338client # [ 0.000000] alternatives: applying boot alternatives339worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns340lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups341worker1 # [ 0.005935] CPU features: detected: ARMv8.4 Translation Table Level342worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).343lb # [ 0.000000] rcu: Hierarchical RCU implementation.344lb # [ 0.000000] rcu: RCU event tracing is enabled.345worker1 # [ 0.005939] CPU features: detected: Instruction cache invalidation not required for I/D coherence346worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns347lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.348worker1 # [ 0.005942] CPU features: detected: Data cache clean to the PoU not required for I/D coherence349lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.350worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns351lb # [ 0.000000] Rude variant of Tasks RCU enabled.352worker1 # [ 0.005945] CPU features: detected: Common not Private translations353worker2 # [ 0.000033] arm-pv: using stolen time PV354client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/k6qpk00c7qa52lid1b7zqxvp2lykb521-nixos-system-client-test/init regInfo=/nix/store/55zbzi2q0wlj27byxrsggqmmpdy1hwi9-closure-info/registration console=ttyAMA0,115200n8 console=tty0355lb # [ 0.000000] Tracing variant of Tasks RCU enabled.356worker1 # [ 0.005949] CPU features: detected: CRC32 instructions357worker2 # [ 0.000585] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)358lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.359worker1 # [ 0.005951] CPU features: detected: Data cache clean to Point of Deep Persistence360worker2 # [ 0.000785] Console: colour dummy device 80x25361client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/55zbzi2q0wlj27byxrsggqmmpdy1hwi9-closure-info/registration", will be passed to user space.362lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1363worker2 # [ 0.000793] printk: legacy console [tty0] enabled364worker1 # [ 0.005955] CPU features: detected: Data cache clean to Point of Persistence365client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes366worker1 # [ 0.005958] CPU features: detected: Data independent timing control (DIT)367lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.368worker2 # [ 0.000991] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)369client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)370worker1 # [ 0.005962] CPU features: detected: E0PD371worker2 # [ 0.000998] pid_max: default: 32768 minimum: 301372lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.373worker1 # [ 0.005964] CPU features: detected: Enhanced Counter Virtualization374client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)375worker2 # [ 0.001079] LSM: initializing lsm=capability,landlock,yama,bpf,ima376worker2 # [ 0.001293] landlock: Up and running.377worker1 # [ 0.005967] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)378client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB379lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.380worker2 # [ 0.001296] Yama: becoming mindful.381client # [ 0.000000] software IO TLB: area num 1.382lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0383worker1 # [ 0.005970] CPU features: detected: Enhanced Virtualization Traps384worker2 # [ 0.001872] LSM support for eBPF active385lb # [ 0.000000] GICv3: 256 SPIs implemented386client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB387worker1 # [ 0.005973] CPU features: detected: Fine Grained Traps388worker2 # [ 0.002024] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)389lb # [ 0.000000] GICv3: 0 Extended SPIs implemented390client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)391lb # [ 0.000000] Root IRQ handler: gic_handle_irq392worker1 # [ 0.005978] CPU features: detected: Generic authentication (architected QARMA5 algorithm)393client # [ 0.000000] Fallback order for Node 0: 0394worker2 # [ 0.002056] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)395lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI396worker1 # [ 0.005983] CPU features: detected: RCpc load-acquire (LDAPR)397worker2 # [ 0.003194] cacheinfo: Unable to detect cache hierarchy for CPU 0398client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216399lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0400worker1 # [ 0.005986] CPU features: detected: LSE atomic instructions401client # [ 0.000000] Policy zone: DMA402worker2 # [ 0.003959] rcu: Hierarchical SRCU implementation.403lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000404worker1 # [ 0.005989] CPU features: detected: Privileged Access Never405worker2 # [ 0.003963] rcu: Max phase no-delay instances is 1000.406client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off407lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]408worker1 # [ 0.005992] CPU features: detected: PMUv3409worker2 # [ 0.005271] fsl-mc MSI: its@8080000 domain created410client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1411worker1 # [ 0.005994] CPU features: detected: RAS Extension Support412worker2 # [ 0.005367] EFI services will not be available.413lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)414client # [ 0.000000] allocated 3145728 bytes of page_ext415worker1 # [ 0.005997] CPU features: detected: RASv1p1 Extension Support416worker2 # [ 0.005446] smp: Bringing up secondary CPUs ...417client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages418worker2 # [ 0.005456] smp: Brought up 1 node, 1 CPU419worker1 # [ 0.006000] CPU features: detected: Random Number Generator420lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)421client # [ 0.000000] ftrace: allocated 294 pages with 4 groups422worker2 # [ 0.005460] SMP: Total of 1 processors activated.423worker1 # [ 0.006002] CPU features: detected: Speculation barrier (SB)424client # [ 0.000000] rcu: Hierarchical RCU implementation.425lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000426worker2 # [ 0.005466] CPU: All CPU(s) started at EL1427worker1 # [ 0.006005] CPU features: detected: Stage-2 Force Write-Back428client # [ 0.000000] rcu: RCU event tracing is enabled.429worker2 # [ 0.005478] CPU features: detected: Branch Target Identification430lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000431worker1 # [ 0.006008] CPU features: detected: TLB range maintenance instructions432client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.433worker2 # [ 0.005483] CPU features: detected: ARMv8.4 Translation Table Level434lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.435client # [ 0.000000] Trampoline variant of Tasks RCU enabled.436worker1 # [ 0.006013] CPU features: detected: Speculative Store Bypassing Safe (SSBS)437client # [ 0.000000] Rude variant of Tasks RCU enabled.438worker2 # [ 0.005487] CPU features: detected: Instruction cache invalidation not required for I/D coherence439worker1 # [ 0.006057] alternatives: applying system-wide alternatives440lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns441client # [ 0.000000] Tracing variant of Tasks RCU enabled.442worker1 # [ 0.009268] CPU features: detected: BBM Level 2 without TLB conflict abort443lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).444worker2 # [ 0.005490] CPU features: detected: Data cache clean to the PoU not required for I/D coherence445client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.446worker2 # [ 0.005494] CPU features: detected: Common not Private translations447client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1448lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns449worker2 # [ 0.005497] CPU features: detected: CRC32 instructions450worker1 # [ 0.009509] Memory: 1406508K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)451worker1 # [ 0.009981] devtmpfs: initialized452client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.453lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns454worker2 # [ 0.005500] CPU features: detected: Data cache clean to Point of Deep Persistence455lb # [ 0.000033] arm-pv: using stolen time PV456worker1 # [ 0.011885] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)457worker2 # [ 0.005504] CPU features: detected: Data cache clean to Point of Persistence458client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.459worker1 # [ 0.011911] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).460worker2 # [ 0.005507] CPU features: detected: Data independent timing control (DIT)461lb # [ 0.000484] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)462client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.463worker2 # [ 0.005510] CPU features: detected: E0PD464lb # [ 0.000840] Console: colour dummy device 80x25465worker1 # [ 0.012118] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL466lb # [ 0.000848] printk: legacy console [tty0] enabled467client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0468worker2 # [ 0.005513] CPU features: detected: Enhanced Counter Virtualization469worker1 # [ 0.012170] 0 pages in range for non-PLT usage470client # [ 0.000000] GICv3: 256 SPIs implemented471worker1 # [ 0.012171] 508288 pages in range for PLT usage472worker2 # [ 0.005516] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)473client # [ 0.000000] GICv3: 0 Extended SPIs implemented474lb # [ 0.001051] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)475worker1 # [ 0.012314] pinctrl core: initialized pinctrl subsystem476client # [ 0.000000] Root IRQ handler: gic_handle_irq477worker2 # [ 0.005520] CPU features: detected: Enhanced Virtualization Traps478lb # [ 0.001058] pid_max: default: 32768 minimum: 301479worker1 # [ 0.013160] DMI not present or invalid.480client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI481worker2 # [ 0.005523] CPU features: detected: Fine Grained Traps482lb # [ 0.001161] LSM: initializing lsm=capability,landlock,yama,bpf,ima483worker1 # [ 0.016741] NET: Registered PF_NETLINK/PF_ROUTE protocol family484client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0485lb # [ 0.001356] landlock: Up and running.486worker2 # [ 0.005528] CPU features: detected: Generic authentication (architected QARMA5 algorithm)487worker1 # [ 0.019069] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations488lb # [ 0.001359] Yama: becoming mindful.489client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000490lb # [ 0.001895] LSM support for eBPF active491worker2 # [ 0.005533] CPU features: detected: RCpc load-acquire (LDAPR)492client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]493worker1 # [ 0.019266] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations494worker2 # [ 0.005536] CPU features: detected: LSE atomic instructions495lb # [ 0.002057] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)496client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)497worker2 # [ 0.005540] CPU features: detected: Privileged Access Never498worker1 # [ 0.019487] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations499lb # [ 0.002087] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)500worker2 # [ 0.005542] CPU features: detected: PMUv3501worker1 # [ 0.019514] audit: initializing netlink subsys (disabled)502client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)503worker2 # [ 0.005545] CPU features: detected: RAS Extension Support504lb # [ 0.003276] cacheinfo: Unable to detect cache hierarchy for CPU 0505worker1 # [ 0.020364] thermal_sys: Registered thermal governor 'fair_share'506client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000507lb # [ 0.004032] rcu: Hierarchical SRCU implementation.508worker2 # [ 0.005547] CPU features: detected: RASv1p1 Extension Support509worker1 # [ 0.020367] thermal_sys: Registered thermal governor 'bang_bang'510lb # [ 0.004036] rcu: Max phase no-delay instances is 1000.511worker2 # [ 0.005550] CPU features: detected: Random Number Generator512client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000513worker1 # [ 0.020371] thermal_sys: Registered thermal governor 'step_wise'514lb # [ 0.005293] fsl-mc MSI: its@8080000 domain created515worker2 # [ 0.005553] CPU features: detected: Speculation barrier (SB)516client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.517lb # [ 0.005393] EFI services will not be available.518worker1 # [ 0.020376] thermal_sys: Registered thermal governor 'user_space'519worker2 # [ 0.005556] CPU features: detected: Stage-2 Force Write-Back520lb # [ 0.005473] smp: Bringing up secondary CPUs ...521worker2 # [ 0.005559] CPU features: detected: TLB range maintenance instructions522client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns523lb # [ 0.005482] smp: Brought up 1 node, 1 CPU524lb # [ 0.005485] SMP: Total of 1 processors activated.525worker1 # [ 0.020379] thermal_sys: Registered thermal governor 'power_allocator'526client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).527worker2 # [ 0.005564] CPU features: detected: Speculative Store Bypassing Safe (SSBS)528lb # [ 0.005491] CPU: All CPU(s) started at EL1529worker2 # [ 0.005607] alternatives: applying system-wide alternatives530worker1 # [ 0.020418] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1531lb # [ 0.005503] CPU features: detected: Branch Target Identification532client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns533worker1 # [ 0.020427] cpuidle: using governor ladder534worker2 # [ 0.008813] CPU features: detected: BBM Level 2 without TLB conflict abort535lb # [ 0.005508] CPU features: detected: ARMv8.4 Translation Table Level536worker1 # [ 0.020433] cpuidle: using governor menu537client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns538worker1 # [ 0.020643] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.539client # [ 0.000035] arm-pv: using stolen time PV540lb # [ 0.005511] CPU features: detected: Instruction cache invalidation not required for I/D coherence541worker2 # [ 0.009011] Memory: 1406716K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)542worker1 # [ 0.020661] ASID allocator initialised with 65536 entries543worker2 # [ 0.009465] devtmpfs: initialized544lb # [ 0.005515] CPU features: detected: Data cache clean to the PoU not required for I/D coherence545worker1 # [ 0.021944] Serial: AMBA PL011 UART driver546client # [ 0.000413] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)547worker2 # [ 0.011324] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)548client # [ 0.000618] Console: colour dummy device 80x25549lb # [ 0.005518] CPU features: detected: Common not Private translations550worker1 # [ 0.027395] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1551client # [ 0.000627] printk: legacy console [tty0] enabled552lb # [ 0.005522] CPU features: detected: CRC32 instructions553worker1 # [ 0.027564] printk: console [ttyAMA0] enabled554worker2 # [ 0.011346] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).555lb # [ 0.005525] CPU features: detected: Data cache clean to Point of Deep Persistence556worker2 # [ 0.011554] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL557client # [ 0.000817] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)558worker2 # [ 0.011559] 0 pages in range for non-PLT usage559client # [ 0.000824] pid_max: default: 32768 minimum: 301560worker2 # [ 0.011560] 508288 pages in range for PLT usage561client # [ 0.000896] LSM: initializing lsm=capability,landlock,yama,bpf,ima562lb # [ 0.005529] CPU features: detected: Data cache clean to Point of Persistence563worker2 # [ 0.011665] pinctrl core: initialized pinctrl subsystem564client # [ 0.001057] landlock: Up and running.565worker2 # [ 0.012465] DMI not present or invalid.566client # [ 0.001059] Yama: becoming mindful.567lb # [ 0.005532] CPU features: detected: Data independent timing control (DIT)568client # [ 0.001516] LSM support for eBPF active569lb # [ 0.005535] CPU features: detected: E0PD570worker2 # [ 0.015619] NET: Registered PF_NETLINK/PF_ROUTE protocol family571client # [ 0.001670] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)572lb # [ 0.005538] CPU features: detected: Enhanced Counter Virtualization573worker2 # [ 0.018105] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations574lb # [ 0.005540] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)575client # [ 0.001702] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)576worker2 # [ 0.018316] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations577client # [ 0.002808] cacheinfo: Unable to detect cache hierarchy for CPU 0578lb # [ 0.005544] CPU features: detected: Enhanced Virtualization Traps579worker2 # [ 0.018532] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations580lb # [ 0.005547] CPU features: detected: Fine Grained Traps581client # [ 0.003621] rcu: Hierarchical SRCU implementation.582worker1 # [ 0.147155] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages583worker2 # [ 0.018567] audit: initializing netlink subsys (disabled)584client # [ 0.003626] rcu: Max phase no-delay instances is 1000.585lb # [ 0.005550] CPU features: detected: Generic authentication (architected QARMA5 algorithm)586worker1 # [ 0.147178] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page587worker2 # [ 0.019214] thermal_sys: Registered thermal governor 'fair_share'588client # [ 0.004850] fsl-mc MSI: its@8080000 domain created589lb # [ 0.005556] CPU features: detected: RCpc load-acquire (LDAPR)590client # [ 0.004965] EFI services will not be available.591worker2 # [ 0.019216] thermal_sys: Registered thermal governor 'bang_bang'592worker1 # [ 0.147183] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages593lb # [ 0.005559] CPU features: detected: LSE atomic instructions594client # [ 0.005054] smp: Bringing up secondary CPUs ...595worker2 # [ 0.019220] thermal_sys: Registered thermal governor 'step_wise'596worker1 # [ 0.147188] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page597client # [ 0.005064] smp: Brought up 1 node, 1 CPU598lb # [ 0.005562] CPU features: detected: Privileged Access Never599worker2 # [ 0.019226] thermal_sys: Registered thermal governor 'user_space'600client # [ 0.005068] SMP: Total of 1 processors activated.601worker1 # [ 0.147192] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages602lb # [ 0.005564] CPU features: detected: PMUv3603client # [ 0.005074] CPU: All CPU(s) started at EL1604lb # [ 0.005567] CPU features: detected: RAS Extension Support605worker1 # [ 0.147196] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page606client # [ 0.005087] CPU features: detected: Branch Target Identification607lb # [ 0.005570] CPU features: detected: RASv1p1 Extension Support608worker2 # [ 0.019229] thermal_sys: Registered thermal governor 'power_allocator'609worker1 # [ 0.147201] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages610client # [ 0.005092] CPU features: detected: ARMv8.4 Translation Table Level611lb # [ 0.005572] CPU features: detected: Random Number Generator612worker1 # [ 0.147205] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page613worker2 # [ 0.019256] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1614lb # [ 0.005575] CPU features: detected: Speculation barrier (SB)615client # [ 0.005095] CPU features: detected: Instruction cache invalidation not required for I/D coherence616worker2 # [ 0.019265] cpuidle: using governor ladder617lb # [ 0.005578] CPU features: detected: Stage-2 Force Write-Back618worker2 # [ 0.019271] cpuidle: using governor menu619client # [ 0.005098] CPU features: detected: Data cache clean to the PoU not required for I/D coherence620lb # [ 0.005581] CPU features: detected: TLB range maintenance instructions621worker2 # [ 0.019475] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.622worker1 # [ 0.154534] fbcon: Taking over console623client # [ 0.005102] CPU features: detected: Common not Private translations624worker2 # [ 0.019493] ASID allocator initialised with 65536 entries625lb # [ 0.005585] CPU features: detected: Speculative Store Bypassing Safe (SSBS)626worker1 # [ 0.154554] ACPI: Interpreter disabled.627client # [ 0.005105] CPU features: detected: CRC32 instructions628worker2 # [ 0.020729] Serial: AMBA PL011 UART driver629lb # [ 0.005627] alternatives: applying system-wide alternatives630client # [ 0.005108] CPU features: detected: Data cache clean to Point of Deep Persistence631worker2 # [ 0.026177] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1632lb # [ 0.008866] CPU features: detected: BBM Level 2 without TLB conflict abort633worker2 # [ 0.026351] printk: console [ttyAMA0] enabled634client # [ 0.005112] CPU features: detected: Data cache clean to Point of Persistence635client # [ 0.005115] CPU features: detected: Data independent timing control (DIT)636lb # [ 0.009050] Memory: 1406508K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)637client # [ 0.005118] CPU features: detected: E0PD638lb # [ 0.009508] devtmpfs: initialized639client # [ 0.005121] CPU features: detected: Enhanced Counter Virtualization640lb # [ 0.011360] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)641client # [ 0.005124] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)642lb # [ 0.011387] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).643client # [ 0.005128] CPU features: detected: Enhanced Virtualization Traps644lb # [ 0.011593] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL645client # [ 0.005131] CPU features: detected: Fine Grained Traps646lb # [ 0.011599] 0 pages in range for non-PLT usage647client # [ 0.005135] CPU features: detected: Generic authentication (architected QARMA5 algorithm)648lb # [ 0.011600] 508288 pages in range for PLT usage649client # [ 0.005140] CPU features: detected: RCpc load-acquire (LDAPR)650lb # [ 0.011733] pinctrl core: initialized pinctrl subsystem651lb # [ 0.012514] DMI not present or invalid.652client # [ 0.005143] CPU features: detected: LSE atomic instructions653worker1 # [ 0.163598] iommu: Default domain type: Translated654worker2 # [ 0.143879] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages655lb # [ 0.015655] NET: Registered PF_NETLINK/PF_ROUTE protocol family656client # [ 0.005146] CPU features: detected: Privileged Access Never657worker1 # [ 0.163610] iommu: DMA domain TLB invalidation policy: strict mode658worker2 # [ 0.143902] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page659client # [ 0.005149] CPU features: detected: PMUv3660worker1 # [ 0.163967] SCSI subsystem initialized661lb # [ 0.018043] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations662worker2 # [ 0.143908] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages663lb # [ 0.018232] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations664worker2 # [ 0.143912] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page665worker1 # [ 0.165934] usbcore: registered new interface driver usbfs666client # [ 0.005152] CPU features: detected: RAS Extension Support667worker1 # [ 0.165965] usbcore: registered new interface driver hub668lb # [ 0.018447] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations669worker2 # [ 0.143917] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages670client # [ 0.005155] CPU features: detected: RASv1p1 Extension Support671worker1 # [ 0.165983] usbcore: registered new device driver usb672lb # [ 0.018475] audit: initializing netlink subsys (disabled)673client # [ 0.005157] CPU features: detected: Random Number Generator674worker2 # [ 0.143921] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page675worker1 # [ 0.166265] pps_core: LinuxPPS API ver. 1 registered676lb # [ 0.019089] thermal_sys: Registered thermal governor 'fair_share'677client # [ 0.005160] CPU features: detected: Speculation barrier (SB)678worker2 # [ 0.143925] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages679lb # [ 0.019091] thermal_sys: Registered thermal governor 'bang_bang'680client # [ 0.005163] CPU features: detected: Stage-2 Force Write-Back681worker1 # [ 0.166271] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>682worker2 # [ 0.143930] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page683worker1 # [ 0.166282] PTP clock support registered684lb # [ 0.019094] thermal_sys: Registered thermal governor 'step_wise'685client # [ 0.005166] CPU features: detected: TLB range maintenance instructions686worker1 # [ 0.166331] EDAC MC: Ver: 3.0.0687lb # [ 0.019100] thermal_sys: Registered thermal governor 'user_space'688client # [ 0.005171] CPU features: detected: Speculative Store Bypassing Safe (SSBS)689worker1 # [ 0.170859] scmi_core: SCMI protocol bus registered690lb # [ 0.019103] thermal_sys: Registered thermal governor 'power_allocator'691client # [ 0.005217] alternatives: applying system-wide alternatives692worker2 # [ 0.151287] fbcon: Taking over console693worker2 # [ 0.151307] ACPI: Interpreter disabled.694worker1 # [ 0.171848] FPGA manager framework695client # [ 0.008407] CPU features: detected: BBM Level 2 without TLB conflict abort696lb # [ 0.019130] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1697worker1 # [ 0.172871] vgaarb: loaded698lb # [ 0.019139] cpuidle: using governor ladder699lb # [ 0.019145] cpuidle: using governor menu700worker1 # [ 0.173548] clocksource: Switched to clocksource arch_sys_counter701client # [ 0.008583] Memory: 1406516K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)702lb # [ 0.019343] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.703client # [ 0.009060] devtmpfs: initialized704lb # [ 0.019362] ASID allocator initialised with 65536 entries705lb # [ 0.020560] Serial: AMBA PL011 UART driver706client # [ 0.010958] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)707lb # [ 0.026051] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1708client # [ 0.010982] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).709lb # [ 0.026242] printk: console [ttyAMA0] enabled710client # [ 0.011180] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL711client # [ 0.011185] 0 pages in range for non-PLT usage712client # [ 0.011186] 508288 pages in range for PLT usage713client # [ 0.011294] pinctrl core: initialized pinctrl subsystem714client # [ 0.012055] DMI not present or invalid.715worker1 # [ 0.179779] VFS: Disk quotas dquot_6.6.0716client # [ 0.015429] NET: Registered PF_NETLINK/PF_ROUTE protocol family717worker2 # [ 0.160379] iommu: Default domain type: Translated718worker1 # [ 0.179842] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)719client # [ 0.017814] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations720worker2 # [ 0.160393] iommu: DMA domain TLB invalidation policy: strict mode721worker1 # [ 0.181572] netfs: FS-Cache loaded722client # [ 0.018008] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations723worker2 # [ 0.160766] SCSI subsystem initialized724worker1 # [ 0.181691] pnp: PnP ACPI: disabled725client # [ 0.018226] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations726worker2 # [ 0.162765] usbcore: registered new interface driver usbfs727client # [ 0.018254] audit: initializing netlink subsys (disabled)728worker2 # [ 0.162796] usbcore: registered new interface driver hub729client # [ 0.018882] thermal_sys: Registered thermal governor 'fair_share'730worker2 # [ 0.162814] usbcore: registered new device driver usb731client # [ 0.018884] thermal_sys: Registered thermal governor 'bang_bang'732worker2 # [ 0.163084] pps_core: LinuxPPS API ver. 1 registered733client # [ 0.018888] thermal_sys: Registered thermal governor 'step_wise'734worker2 # [ 0.163090] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>735client # [ 0.018894] thermal_sys: Registered thermal governor 'user_space'736lb # [ 0.146831] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages737worker2 # [ 0.163100] PTP clock support registered738client # [ 0.018897] thermal_sys: Registered thermal governor 'power_allocator'739worker2 # [ 0.163148] EDAC MC: Ver: 3.0.0740lb # [ 0.146854] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page741lb # [ 0.146859] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages742client # [ 0.018932] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1743worker2 # [ 0.167791] scmi_core: SCMI protocol bus registered744client # [ 0.018940] cpuidle: using governor ladder745lb # [ 0.146864] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page746client # [ 0.018946] cpuidle: using governor menu747worker2 # [ 0.168832] FPGA manager framework748lb # [ 0.146868] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages749worker2 # [ 0.169834] vgaarb: loaded750client # [ 0.019151] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.751worker1 # [ 0.185586] NET: Registered PF_INET protocol family752lb # [ 0.146872] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page753client # [ 0.019168] ASID allocator initialised with 65536 entries754worker2 # [ 0.170481] clocksource: Switched to clocksource arch_sys_counter755worker1 # [ 0.185838] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)756client # [ 0.020394] Serial: AMBA PL011 UART driver757lb # [ 0.146877] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages758lb # [ 0.146881] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page759client # [ 0.025669] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1760client # [ 0.025826] printk: console [ttyAMA0] enabled761lb # [ 0.154394] fbcon: Taking over console762lb # [ 0.154412] ACPI: Interpreter disabled.763lb # [ 0.156340] iommu: Default domain type: Translated764lb # [ 0.156350] iommu: DMA domain TLB invalidation policy: strict mode765worker2 # [ 0.176614] VFS: Disk quotas dquot_6.6.0766lb # [ 0.158082] SCSI subsystem initialized767worker2 # [ 0.176672] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)768worker2 # [ 0.178382] netfs: FS-Cache loaded769worker2 # [ 0.178958] pnp: PnP ACPI: disabled770client # [ 0.145582] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages771client # [ 0.145607] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page772worker2 # [ 0.182469] NET: Registered PF_INET protocol family773client # [ 0.145613] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages774lb # [ 0.163180] usbcore: registered new interface driver usbfs775lb # [ 0.163213] usbcore: registered new interface driver hub776client # [ 0.145617] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page777lb # [ 0.163239] usbcore: registered new device driver usb778client # [ 0.145621] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages779lb # [ 0.163536] pps_core: LinuxPPS API ver. 1 registered780client # [ 0.145625] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page781lb # [ 0.163542] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>782client # [ 0.145630] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages783lb # [ 0.163553] PTP clock support registered784worker2 # [ 0.183364] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)785client # [ 0.145634] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page786lb # [ 0.163602] EDAC MC: Ver: 3.0.0787lb # [ 0.168252] scmi_core: SCMI protocol bus registered788lb # [ 0.169242] FPGA manager framework789client # [ 0.153170] fbcon: Taking over console790lb # [ 0.170222] vgaarb: loaded791client # [ 0.153189] ACPI: Interpreter disabled.792lb # [ 0.170864] clocksource: Switched to clocksource arch_sys_counter793lb # [ 0.175212] VFS: Disk quotas dquot_6.6.0794lb # [ 0.175250] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)795client # [ 0.161867] iommu: Default domain type: Translated796lb # [ 0.179083] netfs: FS-Cache loaded797client # [ 0.161882] iommu: DMA domain TLB invalidation policy: strict mode798lb # [ 0.179222] pnp: PnP ACPI: disabled799client # [ 0.162327] SCSI subsystem initialized800client # [ 0.164329] usbcore: registered new interface driver usbfs801client # [ 0.164362] usbcore: registered new interface driver hub802client # [ 0.164381] usbcore: registered new device driver usb803client # [ 0.164708] pps_core: LinuxPPS API ver. 1 registered804client # [ 0.164715] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>805client # [ 0.164725] PTP clock support registered806client # [ 0.164784] EDAC MC: Ver: 3.0.0807client # [ 0.169467] scmi_core: SCMI protocol bus registered808lb # [ 0.183211] NET: Registered PF_INET protocol family809client # [ 0.170475] FPGA manager framework810worker1 # [ 0.217855] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)811client # [ 0.171457] vgaarb: loaded812lb # [ 0.183481] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)813worker1 # [ 0.217916] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)814client # [ 0.172094] clocksource: Switched to clocksource arch_sys_counter815worker1 # [ 0.217952] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)816worker1 # [ 0.218042] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)817worker1 # [ 0.218186] TCP: Hash tables configured (established 16384 bind 16384)818worker1 # [ 0.218283] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)819worker1 # [ 0.218323] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)820client # [ 0.176715] VFS: Disk quotas dquot_6.6.0821worker1 # [ 0.218362] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)822client # [ 0.176778] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)823worker1 # [ 0.218462] NET: Registered PF_UNIX/PF_LOCAL protocol family824worker1 # [ 0.218485] NET: Registered PF_XDP protocol family825client # [ 0.178499] netfs: FS-Cache loaded826worker1 # [ 0.218504] PCI: CLS 0 bytes, default 64827client # [ 0.178641] pnp: PnP ACPI: disabled828worker1 # [ 0.218767] Trying to unpack rootfs image as initramfs...829worker2 # [ 0.214463] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)830client # [ 0.182498] NET: Registered PF_INET protocol family831client # [ 0.182769] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)832worker2 # [ 0.215748] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)833worker2 # [ 0.215799] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)834worker2 # [ 0.215892] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)835worker2 # [ 0.216034] TCP: Hash tables configured (established 16384 bind 16384)836worker1 # [ 0.241605] kvm [1]: HYP mode not available837worker2 # [ 0.216134] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)838worker2 # [ 0.216174] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)839worker2 # [ 0.216213] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)840worker2 # [ 0.216370] NET: Registered PF_UNIX/PF_LOCAL protocol family841worker2 # [ 0.216399] NET: Registered PF_XDP protocol family842worker2 # [ 0.216420] PCI: CLS 0 bytes, default 64843worker2 # [ 0.216673] Trying to unpack rootfs image as initramfs...844lb # [ 0.216081] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)845lb # [ 0.216141] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)846lb # [ 0.216173] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)847lb # [ 0.216269] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)848lb # [ 0.216412] TCP: Hash tables configured (established 16384 bind 16384)849worker2 # [ 0.238562] kvm [1]: HYP mode not available850lb # [ 0.216510] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)851lb # [ 0.216549] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)852lb # [ 0.216587] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)853lb # [ 0.216697] NET: Registered PF_UNIX/PF_LOCAL protocol family854lb # [ 0.216728] NET: Registered PF_XDP protocol family855lb # [ 0.216745] PCI: CLS 0 bytes, default 64856lb # [ 0.216998] Trying to unpack rootfs image as initramfs...857client # [ 0.214450] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)858client # [ 0.214515] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)859client # [ 0.214552] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)860lb # [ 0.232576] kvm [1]: HYP mode not available861client # [ 0.214646] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)862client # [ 0.214791] TCP: Hash tables configured (established 16384 bind 16384)863client # [ 0.214887] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)864client # [ 0.214930] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865client # [ 0.214968] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866client # [ 0.215088] NET: Registered PF_UNIX/PF_LOCAL protocol family867client # [ 0.215119] NET: Registered PF_XDP protocol family868client # [ 0.215142] PCI: CLS 0 bytes, default 64869client # [ 0.215409] Trying to unpack rootfs image as initramfs...870client # [ 0.230099] kvm [1]: HYP mode not available871worker1 # [ 0.442129] Initialise system trusted keyrings872worker1 # [ 0.442912] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker1 # [ 0.444274] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker1 # [ 0.445057] 9p: Installing v9fs 9p2000 file system support875worker2 # [ 0.432530] Initialise system trusted keyrings876worker2 # [ 0.433316] workingset: timestamp_bits=42 max_order=19 bucket_order=0877worker2 # [ 0.434661] squashfs: version 4.0 (2009/01/31) Phillip Lougher878worker2 # [ 0.435437] 9p: Installing v9fs 9p2000 file system support879worker1 # [ 0.473861] Key type asymmetric registered880worker1 # [ 0.473896] Asymmetric key parser 'x509' registered881lb # [ 0.435508] Initialise system trusted keyrings882worker1 # [ 0.473976] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)883lb # [ 0.436258] workingset: timestamp_bits=42 max_order=19 bucket_order=0884worker1 # [ 0.476109] io scheduler mq-deadline registered885worker1 # [ 0.476168] io scheduler kyber registered886lb # [ 0.437617] squashfs: version 4.0 (2009/01/31) Phillip Lougher887client # [ 0.421507] Initialise system trusted keyrings888client # [ 0.422267] workingset: timestamp_bits=42 max_order=19 bucket_order=0889lb # [ 0.438381] 9p: Installing v9fs 9p2000 file system support890client # [ 0.423613] squashfs: version 4.0 (2009/01/31) Phillip Lougher891worker2 # [ 0.459444] Key type asymmetric registered892worker2 # [ 0.459477] Asymmetric key parser 'x509' registered893worker2 # [ 0.459555] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)894worker2 # [ 0.461710] io scheduler mq-deadline registered895worker2 # [ 0.461722] io scheduler kyber registered896client # [ 0.424415] 9p: Installing v9fs 9p2000 file system support897worker1 # [ 0.485750] pl061_gpio 9030000.pl061: PL061 GPIO chip registered898worker1 # [ 0.487227] ledtrig-cpu: registered to indicate activity on CPUs899worker1 # [ 0.487657] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:900worker1 # [ 0.487677] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000901worker1 # [ 0.487689] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000902worker1 # [ 0.487697] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000903worker1 # [ 0.487728] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits904worker2 # [ 0.474988] pl061_gpio 9030000.pl061: PL061 GPIO chip registered905worker1 # [ 0.487756] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]906worker1 # [ 0.487849] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00907worker1 # [ 0.487859] pci_bus 0000:00: root bus resource [bus 00-ff]908worker1 # [ 0.487865] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]909worker1 # [ 0.487870] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]910worker1 # [ 0.487875] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]911worker1 # [ 0.487933] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint912worker1 # [ 0.488397] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint913worker2 # [ 0.476452] ledtrig-cpu: registered to indicate activity on CPUs914worker1 # [ 0.488581] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]915worker2 # [ 0.476887] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:916worker1 # [ 0.488604] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]917worker1 # [ 0.488633] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]918worker2 # [ 0.476905] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000919worker1 # [ 0.488649] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]920worker2 # [ 0.476918] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000921worker1 # [ 0.489115] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint922lb # [ 0.467168] Key type asymmetric registered923worker2 # [ 0.476926] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000924worker1 # [ 0.489298] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]925lb # [ 0.467204] Asymmetric key parser 'x509' registered926worker1 # [ 0.489314] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]927worker2 # [ 0.476947] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits928client # [ 0.452401] Key type asymmetric registered929lb # [ 0.467285] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)930worker1 # [ 0.489343] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]931client # [ 0.452435] Asymmetric key parser 'x509' registered932lb # [ 0.469446] io scheduler mq-deadline registered933worker2 # [ 0.476971] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]934lb # [ 0.469458] io scheduler kyber registered935client # [ 0.452516] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)936worker2 # [ 0.477053] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00937client # [ 0.454620] io scheduler mq-deadline registered938worker1 # [ 0.509017] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint939worker2 # [ 0.477063] pci_bus 0000:00: root bus resource [bus 00-ff]940client # [ 0.454634] io scheduler kyber registered941worker1 # [ 0.509206] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]942worker2 # [ 0.477069] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]943worker1 # [ 0.509222] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]944worker2 # [ 0.477074] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]945worker1 # [ 0.509251] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]946worker2 # [ 0.477079] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]947worker2 # [ 0.477138] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint948worker2 # [ 0.477580] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint949worker2 # [ 0.477777] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]950worker2 # [ 0.477794] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]951worker2 # [ 0.477824] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]952worker2 # [ 0.477840] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]953worker2 # [ 0.478312] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint954lb # [ 0.479040] pl061_gpio 9030000.pl061: PL061 GPIO chip registered955worker2 # [ 0.478538] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]956worker2 # [ 0.478555] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]957client # [ 0.464313] pl061_gpio 9030000.pl061: PL061 GPIO chip registered958worker2 # [ 0.478585] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]959worker2 # [ 0.479051] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint960lb # [ 0.480525] ledtrig-cpu: registered to indicate activity on CPUs961worker2 # [ 0.479234] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]962worker2 # [ 0.479250] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]963lb # [ 0.480958] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:964client # [ 0.465914] ledtrig-cpu: registered to indicate activity on CPUs965worker2 # [ 0.479279] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]966lb # [ 0.480976] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000967client # [ 0.466338] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:968worker2 # [ 0.479746] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint969lb # [ 0.480998] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000970client # [ 0.466355] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000971worker2 # [ 0.479928] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]972worker1 # [ 0.517913] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint973worker2 # [ 0.479943] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]974lb # [ 0.481007] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000975client # [ 0.466369] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000976worker1 # [ 0.518131] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]977worker2 # [ 0.479976] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]978lb # [ 0.481027] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits979worker1 # [ 0.518148] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]980client # [ 0.466378] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000981worker2 # [ 0.480463] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint982worker1 # [ 0.518181] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]983lb # [ 0.481055] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]984client # [ 0.466409] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits985worker2 # [ 0.480646] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]986worker1 # [ 0.518652] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint987worker2 # [ 0.480662] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]988lb # [ 0.481179] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00989client # [ 0.466437] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]990worker1 # [ 0.518837] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]991lb # [ 0.481189] pci_bus 0000:00: root bus resource [bus 00-ff]992worker2 # [ 0.480692] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]993client # [ 0.466527] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00994worker1 # [ 0.518853] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]995lb # [ 0.481195] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]996client # [ 0.466537] pci_bus 0000:00: root bus resource [bus 00-ff]997worker2 # [ 0.481141] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint998worker1 # [ 0.518882] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]999lb # [ 0.481200] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1000worker2 # [ 0.481323] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1001client # [ 0.466543] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1002lb # [ 0.481205] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1003worker1 # [ 0.519331] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1004worker2 # [ 0.481339] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1005client # [ 0.466548] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1006worker1 # [ 0.519515] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1007worker2 # [ 0.481369] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1008lb # [ 0.481303] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1009client # [ 0.466553] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1010worker1 # [ 0.519530] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1011worker2 # [ 0.481849] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1012client # [ 0.466612] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1013lb # [ 0.481760] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1014worker1 # [ 0.519560] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1015worker2 # [ 0.482033] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1016lb # [ 0.481950] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1017worker2 # [ 0.482049] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1018worker1 # [ 0.520056] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1019client # [ 0.467049] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1020lb # [ 0.481966] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1021client # [ 0.467238] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1022worker1 # [ 0.520273] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1023worker2 # [ 0.482079] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1024lb # [ 0.481995] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025client # [ 0.467255] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1026worker1 # [ 0.520290] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1027worker2 # [ 0.482095] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1028lb # [ 0.482011] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1029client # [ 0.467285] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1030worker1 # [ 0.520319] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1031lb # [ 0.482465] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1032client # [ 0.467301] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1033worker1 # [ 0.520335] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1034lb # [ 0.482648] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1035client # [ 0.467768] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1036worker1 # [ 0.520809] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1037lb # [ 0.482663] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1038client # [ 0.467952] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1039worker1 # [ 0.520997] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1040lb # [ 0.482692] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1041client # [ 0.467969] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1042worker1 # [ 0.521027] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1043client # [ 0.467999] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1044worker1 # [ 0.521476] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1045worker1 # [ 0.521678] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1046lb # [ 0.501716] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1047worker1 # [ 0.521707] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1048lb # [ 0.501905] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1049lb # [ 0.501921] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1050worker1 # [ 0.522096] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1051lb # [ 0.501950] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1052worker1 # [ 0.522276] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1053lb # [ 0.502397] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1054worker1 # [ 0.522521] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1055lb # [ 0.502579] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1056worker1 # [ 0.522858] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1057worker2 # [ 0.522673] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1058lb # [ 0.502594] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1059worker1 # [ 0.522876] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1060worker2 # [ 0.522899] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1061lb # [ 0.502626] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1062worker1 # [ 0.522906] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1063worker2 # [ 0.522930] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064worker1 # [ 0.523374] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1065worker2 # [ 0.523468] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1066worker1 # [ 0.523556] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1067worker2 # [ 0.523656] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1068worker1 # [ 0.523571] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1069worker2 # [ 0.523686] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070worker1 # [ 0.523601] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1071worker2 # [ 0.524083] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1072worker1 # [ 0.524217] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1073worker2 # [ 0.524265] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1074worker1 # [ 0.524228] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1075worker2 # [ 0.524544] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1076worker1 # [ 0.524234] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1077worker2 # [ 0.524854] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1078worker1 # [ 0.524280] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1079worker2 # [ 0.524872] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1080worker1 # [ 0.524327] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1081client # [ 0.492613] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1082worker2 # [ 0.524902] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1083client # [ 0.492828] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1084worker1 # [ 0.524374] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1085worker2 # [ 0.525388] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1086client # [ 0.492846] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1087worker2 # [ 0.525593] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1088worker1 # [ 0.524420] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1089client # [ 0.492876] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1090worker2 # [ 0.525610] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1091worker1 # [ 0.524466] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1092client # [ 0.493365] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1093worker2 # [ 0.525639] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094client # [ 0.493548] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1095worker1 # [ 0.524513] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1096worker2 # [ 0.526243] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1097client # [ 0.493564] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1098worker2 # [ 0.526256] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1099worker1 # [ 0.524559] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1100client # [ 0.493596] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101worker2 # [ 0.526261] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1102worker1 # [ 0.524606] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1103client # [ 0.494056] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1104worker2 # [ 0.526308] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1105lb # [ 0.515281] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1106worker1 # [ 0.524652] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1107client # [ 0.494239] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1108lb # [ 0.515499] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1109client # [ 0.494255] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1110worker2 # [ 0.526356] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1111worker1 # [ 0.524744] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1112lb # [ 0.515517] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1113client # [ 0.494284] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114worker1 # [ 0.524790] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1115worker2 # [ 0.526405] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1116lb # [ 0.515547] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1117worker1 # [ 0.524811] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1118client # [ 0.494735] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1119worker2 # [ 0.526452] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1120lb # [ 0.516020] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1121client # [ 0.494916] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1122worker1 # [ 0.524832] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1123lb # [ 0.516209] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1124client # [ 0.494933] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1125worker1 # [ 0.524854] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1126lb # [ 0.516224] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1127client # [ 0.494962] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1128worker1 # [ 0.524878] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1129lb # [ 0.516254] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1130client # [ 0.495481] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1131worker1 # [ 0.524900] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1132client # [ 0.495673] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1133lb # [ 0.516707] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1134worker1 # [ 0.524922] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1135lb # [ 0.516893] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1136client # [ 0.495690] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1137worker1 # [ 0.524943] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1138lb # [ 0.516908] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1139client # [ 0.495720] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1140worker1 # [ 0.524965] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1141client # [ 0.495738] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1142lb # [ 0.516938] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1143worker1 # [ 0.524987] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1144lb # [ 0.516953] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1145client # [ 0.496221] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1146worker1 # [ 0.525009] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1147client # [ 0.496408] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1148lb # [ 0.517468] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1149worker1 # [ 0.525032] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1150lb # [ 0.517656] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1151client # [ 0.496438] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1152worker1 # [ 0.525053] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1153lb # [ 0.517686] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1154client # [ 0.496885] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1155worker1 # [ 0.525075] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1156client # [ 0.497083] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1157lb # [ 0.518138] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1158worker1 # [ 0.525096] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1159client # [ 0.497114] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1160lb # [ 0.518326] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1161worker1 # [ 0.525117] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1162worker1 # [ 0.525138] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1163lb # [ 0.518355] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164client # [ 0.497509] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1165worker1 # [ 0.525159] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1166client # [ 0.497692] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1167lb # [ 0.518743] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1168worker1 # [ 0.525180] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1169lb # [ 0.518945] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1170client # [ 0.497940] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1171worker1 # [ 0.525202] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1172worker2 # [ 0.554629] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1173client # [ 0.498234] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1174lb # [ 0.519192] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1175worker1 # [ 0.525224] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1176client # [ 0.498252] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1177lb # [ 0.519488] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1178worker2 # [ 0.554716] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1179worker1 # [ 0.525252] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1180lb # [ 0.519505] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1181client # [ 0.498281] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1182worker2 # [ 0.554767] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1183worker1 # [ 0.525262] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1184lb # [ 0.519534] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1185client # [ 0.498737] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1186worker1 # [ 0.525266] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1187worker2 # [ 0.554817] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1188client # [ 0.498922] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1189lb # [ 0.520007] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1190worker2 # [ 0.554867] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1191client # [ 0.498938] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1192lb # [ 0.520191] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1193client # [ 0.498968] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1194lb # [ 0.520207] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1195worker2 # [ 0.555025] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1196lb # [ 0.520236] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1197client # [ 0.499550] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1198worker2 # [ 0.555076] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1199lb # [ 0.520846] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1200client # [ 0.499561] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1201worker2 # [ 0.555106] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1202worker2 # [ 0.555129] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1203lb # [ 0.520857] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1204client # [ 0.499567] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1205worker2 # [ 0.555153] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1206lb # [ 0.520863] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1207client # [ 0.499612] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1208worker2 # [ 0.555181] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1209lb # [ 0.520908] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1210client # [ 0.499660] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1211worker2 # [ 0.555203] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1212lb # [ 0.520957] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1213client # [ 0.499706] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1214worker2 # [ 0.555225] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1215lb # [ 0.521004] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1216client # [ 0.499753] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1217worker2 # [ 0.555247] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1218lb # [ 0.521051] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1219client # [ 0.499799] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1220worker2 # [ 0.555270] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1221worker2 # [ 0.555293] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1222lb # [ 0.521124] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1223client # [ 0.499846] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1224worker2 # [ 0.555317] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1225lb # [ 0.521177] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1226worker1 # [ 0.590609] pci 0000:00:0a.0: enabling device (0000 -> 0002)1227client # [ 0.499893] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1228worker2 # [ 0.555341] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1229lb # [ 0.521224] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1230client # [ 0.499940] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1231worker2 # [ 0.555362] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1232lb # [ 0.521270] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1233client # [ 0.499987] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1234worker2 # [ 0.555385] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1235worker2 # [ 0.555407] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1236lb # [ 0.521318] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1237client # [ 0.500049] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1238worker2 # [ 0.555429] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1239lb # [ 0.521401] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1240worker2 # [ 0.555450] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1241lb # [ 0.521447] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1242worker2 # [ 0.555472] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1243lb # [ 0.521469] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1244worker2 # [ 0.555494] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1245lb # [ 0.521490] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1246worker2 # [ 0.555516] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1247worker2 # [ 0.555539] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1248lb # [ 0.521512] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1249worker2 # [ 0.555574] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1250lb # [ 0.521537] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1251worker2 # [ 0.555584] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1252lb # [ 0.521560] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1253worker2 # [ 0.555589] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1254lb # [ 0.521582] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1255worker2 # [ 0.556805] pci 0000:00:0a.0: enabling device (0000 -> 0002)1256lb # [ 0.521604] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1257lb # [ 0.521625] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1258lb # [ 0.521647] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1259lb # [ 0.521670] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1260lb # [ 0.521692] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1261lb # [ 0.521714] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1262client # [ 0.548187] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1263lb # [ 0.521736] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1264client # [ 0.548281] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1265lb # [ 0.521757] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1266client # [ 0.548309] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1267lb # [ 0.521778] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1268client # [ 0.548333] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1269lb # [ 0.521800] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1270client # [ 0.548361] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1271lb # [ 0.521821] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1272client # [ 0.548385] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1273lb # [ 0.521843] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1274lb # [ 0.521865] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1275client # [ 0.548409] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1276lb # [ 0.521886] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1277client # [ 0.548432] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1278lb # [ 0.521915] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1279client # [ 0.548456] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1280worker1 # [ 0.611628] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1281lb # [ 0.521924] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1282client # [ 0.548481] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1283lb # [ 0.521929] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1284client # [ 0.548505] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1285client # [ 0.548529] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1286client # [ 0.548552] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1287client # [ 0.548574] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1288client # [ 0.548596] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1289client # [ 0.548617] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1290client # [ 0.548640] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1291client # [ 0.548662] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1292client # [ 0.548683] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1293client # [ 0.548706] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1294worker2 # [ 0.601775] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1295client # [ 0.548728] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1296client # [ 0.548760] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1297lb # [ 0.583252] pci 0000:00:0a.0: enabling device (0000 -> 0002)1298client # [ 0.548770] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1299client # [ 0.548775] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1300worker1 # [ 0.621911] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1301client # [ 0.549975] pci 0000:00:0a.0: enabling device (0000 -> 0002)1302worker1 # [ 0.626457] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1303worker1 # [ 0.628525] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1304worker2 # [ 0.612027] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1305worker2 # [ 0.616611] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1306worker1 # [ 0.639663] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1307client # [ 0.586326] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1308lb # [ 0.604511] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1309worker2 # [ 0.622794] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1310lb # [ 0.606731] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1311worker2 # [ 0.625078] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1312worker1 # [ 0.645794] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1313worker1 # [ 0.648340] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1314lb # [ 0.617694] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1315worker2 # [ 0.636259] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1316worker1 # [ 0.659178] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1317client # [ 0.604432] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1318worker2 # [ 0.642646] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1319client # [ 0.606731] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1320worker2 # [ 0.645195] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1321client # [ 0.608875] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1322client # [ 0.611051] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1323worker1 # [ 0.665738] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1324worker1 # [ 0.667791] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1325lb # [ 0.632237] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1326lb # [ 0.634428] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1327lb # [ 0.638251] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1328worker2 # [ 0.655648] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1329worker2 # [ 0.657779] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1330client # [ 0.621637] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1331client # [ 0.623742] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1332worker1 # [ 0.678280] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1333lb # [ 0.641399] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1334worker2 # [ 0.669537] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1335client # [ 0.635428] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1336worker1 # [ 0.691704] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1337lb # [ 0.652551] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1338worker1 # [ 0.694459] msm_serial: driver initialized1339worker1 # [ 0.694607] SuperH (H)SCI(F) driver initialized1340worker1 # [ 0.694666] STM32 USART driver initialized1341worker2 # [ 0.679818] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1342client # [ 0.642058] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1343client # [ 0.643955] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1344lb # [ 0.658991] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1345lb # [ 0.660975] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1346worker2 # [ 0.688106] msm_serial: driver initialized1347worker2 # [ 0.688318] SuperH (H)SCI(F) driver initialized1348worker2 # [ 0.688375] STM32 USART driver initialized1349lb # [ 0.671212] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1350client # [ 0.657857] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1351lb # [ 0.684606] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1352client # [ 0.668953] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1353client # [ 0.671722] msm_serial: driver initialized1354client # [ 0.671899] SuperH (H)SCI(F) driver initialized1355client # [ 0.671954] STM32 USART driver initialized1356lb # [ 0.687307] msm_serial: driver initialized1357lb # [ 0.687449] SuperH (H)SCI(F) driver initialized1358lb # [ 0.687505] STM32 USART driver initialized1359worker1 # [ 0.733849] loop: module loaded1360worker1 # [ 0.734084] virtio_blk virtio5: 1/0/0 default/read/poll queues1361worker1 # [ 0.734925] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1362worker2 # [ 0.722805] loop: module loaded1363worker2 # [ 0.723051] virtio_blk virtio5: 1/0/0 default/read/poll queues1364worker2 # [ 0.723877] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1365worker1 # [ 0.742398] megasas: 07.734.00.00-rc11366worker1 # [ 0.743147] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1367worker1 # [ 0.745141] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001368worker1 # [ 0.745177] Intel/Sharp Extended Query Table at 0x00311369worker1 # [ 0.746814] Using buffer write method1370worker1 # [ 0.746909] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1371worker1 # [ 0.748730] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001372worker1 # [ 0.748769] Intel/Sharp Extended Query Table at 0x00311373worker1 # [ 0.757933] Using buffer write method1374worker2 # [ 0.731322] megasas: 07.734.00.00-rc11375worker1 # [ 0.757974] Concatenating MTD devices:1376worker1 # [ 0.757978] (0): "0.flash"1377worker1 # [ 0.757982] (1): "0.flash"1378client # [ 0.703158] loop: module loaded1379worker2 # [ 0.732108] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1380worker1 # [ 0.757985] into device "0.flash"1381client # [ 0.703375] virtio_blk virtio5: 1/0/0 default/read/poll queues1382worker2 # [ 0.734197] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001383worker2 # [ 0.734238] Intel/Sharp Extended Query Table at 0x00311384lb # [ 0.720476] loop: module loaded1385worker2 # [ 0.735902] Using buffer write method1386lb # [ 0.720706] virtio_blk virtio5: 1/0/0 default/read/poll queues1387lb # [ 0.721534] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1388worker2 # [ 0.735998] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1389worker2 # [ 0.737755] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001390worker2 # [ 0.737797] Intel/Sharp Extended Query Table at 0x00311391worker2 # [ 0.746869] Using buffer write method1392worker2 # [ 0.746907] Concatenating MTD devices:1393worker2 # [ 0.746911] (0): "0.flash"1394worker2 # [ 0.746916] (1): "0.flash"1395worker2 # [ 0.746919] into device "0.flash"1396client # [ 0.712285] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1397lb # [ 0.727665] megasas: 07.734.00.00-rc11398lb # [ 0.728431] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399client # [ 0.715862] megasas: 07.734.00.00-rc11400client # [ 0.716673] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1401lb # [ 0.748214] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402lb # [ 0.748266] Intel/Sharp Extended Query Table at 0x00311403lb # [ 0.750012] Using buffer write method1404lb # [ 0.750119] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1405lb # [ 0.755376] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406lb # [ 0.755420] Intel/Sharp Extended Query Table at 0x00311407lb # [ 0.757098] Using buffer write method1408lb # [ 0.757173] Concatenating MTD devices:1409lb # [ 0.757177] (0): "0.flash"1410lb # [ 0.757181] (1): "0.flash"1411client # [ 0.741385] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001412lb # [ 0.757185] into device "0.flash"1413client # [ 0.741439] Intel/Sharp Extended Query Table at 0x00311414client # [ 0.743124] Using buffer write method1415client # [ 0.743239] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1416client # [ 0.748323] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417client # [ 0.748363] Intel/Sharp Extended Query Table at 0x00311418client # [ 0.750053] Using buffer write method1419client # [ 0.750092] Concatenating MTD devices:1420client # [ 0.750096] (0): "0.flash"1421client # [ 0.750100] (1): "0.flash"1422client # [ 0.750103] into device "0.flash"1423worker1 # [ 1.017476] Freeing initrd memory: 26148K1424worker1 # [ 1.023750] tun: Universal TUN/TAP device driver, 1.61425worker1 # [ 1.027864] thunder_xcv, ver 1.01426worker1 # [ 1.027912] thunder_bgx, ver 1.01427worker1 # [ 1.027934] nicpf, ver 1.01428worker1 # [ 1.028512] e1000: Intel(R) PRO/1000 Network Driver1429worker1 # [ 1.028520] e1000: Copyright (c) 1999-2006 Intel Corporation.1430worker1 # [ 1.028544] e1000e: Intel(R) PRO/1000 Network Driver1431worker1 # [ 1.028554] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432worker1 # [ 1.028583] igb: Intel(R) Gigabit Ethernet Network Driver1433worker1 # [ 1.028589] igb: Copyright (c) 2007-2014 Intel Corporation.1434worker1 # [ 1.028617] igbvf: Intel(R) Gigabit Virtual Function Network Driver1435worker1 # [ 1.028623] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1436worker1 # [ 1.028756] sky2: driver version 1.301437worker1 # [ 1.037599] usbcore: registered new interface driver usb-storage1438worker1 # [ 1.037732] usbcore: registered new interface driver usbserial_generic1439worker1 # [ 1.037748] usbserial: USB Serial support registered for generic1440worker1 # [ 1.038367] hv_vmbus: registering driver hyperv_keyboard1441worker1 # [ 1.039830] ehci-pci 0000:00:0a.0: EHCI Host Controller1442worker1 # [ 1.039863] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11443worker1 # [ 1.040048] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001444worker1 # [ 1.045529] rtc-pl031 9010000.pl031: registered as rtc01445worker2 # [ 1.024196] Freeing initrd memory: 26148K1446worker1 # [ 1.046217] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:56:51 UTC (1790074611)1447worker1 # [ 1.046626] i2c_dev: i2c /dev entries driver1448worker2 # [ 1.030676] tun: Universal TUN/TAP device driver, 1.61449worker1 # [ 1.051008] sdhci: Secure Digital Host Controller Interface driver1450worker1 # [ 1.051021] sdhci: Copyright(c) Pierre Ossman1451worker1 # [ 1.051286] Synopsys Designware Multimedia Card Interface Driver1452worker1 # [ 1.051655] sdhci-pltfm: SDHCI platform and OF driver helper1453worker1 # [ 1.053413] hid: raw HID events driver (C) Jiri Kosina1454worker1 # [ 1.056865] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001455worker1 # [ 1.057811] usbcore: registered new interface driver usbhid1456worker2 # [ 1.034843] thunder_xcv, ver 1.01457worker1 # [ 1.057820] usbhid: USB HID core driver1458worker2 # [ 1.034894] thunder_bgx, ver 1.01459worker2 # [ 1.034919] nicpf, ver 1.01460worker1 # [ 1.060376] hub 1-0:1.0: USB hub found1461worker2 # [ 1.035476] e1000: Intel(R) PRO/1000 Network Driver1462worker1 # [ 1.060869] hub 1-0:1.0: 6 ports detected1463worker2 # [ 1.035483] e1000: Copyright (c) 1999-2006 Intel Corporation.1464worker2 # [ 1.035507] e1000e: Intel(R) PRO/1000 Network Driver1465worker2 # [ 1.035516] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1466worker2 # [ 1.035547] igb: Intel(R) Gigabit Ethernet Network Driver1467worker2 # [ 1.035553] igb: Copyright (c) 2007-2014 Intel Corporation.1468worker1 # [ 1.061847] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1469worker2 # [ 1.035575] igbvf: Intel(R) Gigabit Virtual Function Network Driver1470worker1 # [ 1.063403] drop_monitor: Initializing network drop monitor service1471lb # [ 1.026137] Freeing initrd memory: 26140K1472worker2 # [ 1.035581] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1473worker1 # [ 1.063570] NET: Registered PF_INET6 protocol family1474worker2 # [ 1.035713] sky2: driver version 1.301475worker2 # [ 1.037330] usbcore: registered new interface driver usb-storage1476worker1 # [ 1.066651] Segment Routing with IPv61477worker1 # [ 1.066671] In-situ OAM (IOAM) with IPv61478worker2 # [ 1.037425] usbcore: registered new interface driver usbserial_generic1479worker1 # [ 1.066702] NET: Registered PF_PACKET protocol family1480worker2 # [ 1.037439] usbserial: USB Serial support registered for generic1481worker2 # [ 1.038051] hv_vmbus: registering driver hyperv_keyboard1482worker2 # [ 1.039260] ehci-pci 0000:00:0a.0: EHCI Host Controller1483worker1 # [ 1.068350] 9pnet: Installing 9P2000 support1484worker2 # [ 1.039292] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11485worker2 # [ 1.039525] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001486client # [ 1.014482] Freeing initrd memory: 26160K1487lb # [ 1.032620] tun: Universal TUN/TAP device driver, 1.61488worker2 # [ 1.051610] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001489worker2 # [ 1.052006] hub 1-0:1.0: USB hub found1490worker2 # [ 1.052028] hub 1-0:1.0: 6 ports detected1491worker1 # [ 1.071242] Key type dns_resolver registered1492worker2 # [ 1.054460] rtc-pl031 9010000.pl031: registered as rtc01493client # [ 1.020935] tun: Universal TUN/TAP device driver, 1.61494worker2 # [ 1.055148] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:56:51 UTC (1790074611)1495worker2 # [ 1.055605] i2c_dev: i2c /dev entries driver1496lb # [ 1.036564] thunder_xcv, ver 1.01497lb # [ 1.036616] thunder_bgx, ver 1.01498lb # [ 1.036638] nicpf, ver 1.01499lb # [ 1.037219] e1000: Intel(R) PRO/1000 Network Driver1500lb # [ 1.037227] e1000: Copyright (c) 1999-2006 Intel Corporation.1501lb # [ 1.037250] e1000e: Intel(R) PRO/1000 Network Driver1502worker1 # [ 1.077770] registered taskstats version 11503lb # [ 1.037259] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1504worker1 # [ 1.077939] Loading compiled-in X.509 certificates1505worker2 # [ 1.060148] sdhci: Secure Digital Host Controller Interface driver1506lb # [ 1.037287] igb: Intel(R) Gigabit Ethernet Network Driver1507worker2 # [ 1.060162] sdhci: Copyright(c) Pierre Ossman1508lb # [ 1.037292] igb: Copyright (c) 2007-2014 Intel Corporation.1509worker2 # [ 1.060444] Synopsys Designware Multimedia Card Interface Driver1510client # [ 1.024965] thunder_xcv, ver 1.01511lb # [ 1.037315] igbvf: Intel(R) Gigabit Virtual Function Network Driver1512worker2 # [ 1.060821] sdhci-pltfm: SDHCI platform and OF driver helper1513client # [ 1.025011] thunder_bgx, ver 1.01514client # [ 1.025036] nicpf, ver 1.01515lb # [ 1.037320] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1516lb # [ 1.037459] sky2: driver version 1.301517client # [ 1.025591] e1000: Intel(R) PRO/1000 Network Driver1518lb # [ 1.039583] ehci-pci 0000:00:0a.0: EHCI Host Controller1519client # [ 1.025598] e1000: Copyright (c) 1999-2006 Intel Corporation.1520worker2 # [ 1.065207] hid: raw HID events driver (C) Jiri Kosina1521client # [ 1.025622] e1000e: Intel(R) PRO/1000 Network Driver1522lb # [ 1.039613] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11523worker2 # [ 1.065467] usbcore: registered new interface driver usbhid1524client # [ 1.025631] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1525worker2 # [ 1.065474] usbhid: USB HID core driver1526lb # [ 1.039836] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001527worker1 # [ 1.087125] Demotion targets for Node 0: null1528client # [ 1.025663] igb: Intel(R) Gigabit Ethernet Network Driver1529worker1 # [ 1.087259] Key type .fscrypt registered1530worker1 # [ 1.087265] Key type fscrypt-provisioning registered1531client # [ 1.025669] igb: Copyright (c) 2007-2014 Intel Corporation.1532worker1 # [ 1.087374] ima: No TPM chip found, activating TPM-bypass!1533client # [ 1.025691] igbvf: Intel(R) Gigabit Virtual Function Network Driver1534worker1 # [ 1.087393] ima: Allocated hash algorithm: sha11535lb # [ 1.049454] usbcore: registered new interface driver usb-storage1536client # [ 1.025697] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1537worker1 # [ 1.087416] ima: No architecture policies found1538client # [ 1.025835] sky2: driver version 1.301539worker2 # [ 1.068411] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1540lb # [ 1.049519] usbcore: registered new interface driver usbserial_generic1541client # [ 1.027480] usbcore: registered new interface driver usb-storage1542worker2 # [ 1.069966] drop_monitor: Initializing network drop monitor service1543lb # [ 1.049536] usbserial: USB Serial support registered for generic1544worker1 # [ 1.091597] input: gpio-keys as /devices/platform/gpio-keys/input/input01545worker2 # [ 1.070175] NET: Registered PF_INET6 protocol family1546client # [ 1.027574] usbcore: registered new interface driver usbserial_generic1547lb # [ 1.050139] hv_vmbus: registering driver hyperv_keyboard1548client # [ 1.027588] usbserial: USB Serial support registered for generic1549worker2 # [ 1.073270] Segment Routing with IPv61550client # [ 1.028765] ehci-pci 0000:00:0a.0: EHCI Host Controller1551worker2 # [ 1.073303] In-situ OAM (IOAM) with IPv61552lb # [ 1.053720] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001553lb # [ 1.054110] hub 1-0:1.0: USB hub found1554worker2 # [ 1.073334] NET: Registered PF_PACKET protocol family1555client # [ 1.028793] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11556lb # [ 1.054131] hub 1-0:1.0: 6 ports detected1557client # [ 1.028983] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001558lb # [ 1.056364] rtc-pl031 9010000.pl031: registered as rtc01559client # [ 1.040544] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001560client # [ 1.040965] hub 1-0:1.0: USB hub found1561client # [ 1.041019] hub 1-0:1.0: 6 ports detected1562lb # [ 1.056395] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:56:51 UTC (1790074611)1563worker2 # [ 1.075151] 9pnet: Installing 9P2000 support1564lb # [ 1.056755] i2c_dev: i2c /dev entries driver1565worker2 # [ 1.077460] Key type dns_resolver registered1566client # [ 1.043287] hv_vmbus: registering driver hyperv_keyboard1567client # [ 1.044996] rtc-pl031 9010000.pl031: registered as rtc01568lb # [ 1.061855] sdhci: Secure Digital Host Controller Interface driver1569client # [ 1.045029] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:56:51 UTC (1790074611)1570lb # [ 1.061869] sdhci: Copyright(c) Pierre Ossman1571client # [ 1.045360] i2c_dev: i2c /dev entries driver1572lb # [ 1.062140] Synopsys Designware Multimedia Card Interface Driver1573lb # [ 1.062517] sdhci-pltfm: SDHCI platform and OF driver helper1574worker2 # [ 1.084261] registered taskstats version 11575worker2 # [ 1.084528] Loading compiled-in X.509 certificates1576lb # [ 1.067035] hid: raw HID events driver (C) Jiri Kosina1577client # [ 1.050462] sdhci: Secure Digital Host Controller Interface driver1578client # [ 1.050475] sdhci: Copyright(c) Pierre Ossman1579lb # [ 1.067300] usbcore: registered new interface driver usbhid1580lb # [ 1.067307] usbhid: USB HID core driver1581client # [ 1.050749] Synopsys Designware Multimedia Card Interface Driver1582client # [ 1.051129] sdhci-pltfm: SDHCI platform and OF driver helper1583lb # [ 1.070318] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1584client # [ 1.055768] hid: raw HID events driver (C) Jiri Kosina1585client # [ 1.056030] usbcore: registered new interface driver usbhid1586client # [ 1.056037] usbhid: USB HID core driver1587lb # [ 1.072992] drop_monitor: Initializing network drop monitor service1588lb # [ 1.073241] NET: Registered PF_INET6 protocol family1589worker2 # [ 1.093459] Demotion targets for Node 0: null1590worker2 # [ 1.093599] Key type .fscrypt registered1591lb # [ 1.075518] Segment Routing with IPv61592worker2 # [ 1.093606] Key type fscrypt-provisioning registered1593lb # [ 1.075539] In-situ OAM (IOAM) with IPv61594worker2 # [ 1.093715] ima: No TPM chip found, activating TPM-bypass!1595lb # [ 1.075588] NET: Registered PF_PACKET protocol family1596client # [ 1.059023] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1597worker2 # [ 1.093735] ima: Allocated hash algorithm: sha11598worker1 # [ 1.114059] clk: Disabling unused clocks1599worker2 # [ 1.093758] ima: No architecture policies found1600worker1 # [ 1.114101] PM: genpd: Disabling unused power domains1601client # [ 1.061653] drop_monitor: Initializing network drop monitor service1602lb # [ 1.077204] 9pnet: Installing 9P2000 support1603client # [ 1.061818] NET: Registered PF_INET6 protocol family1604worker2 # [ 1.097929] input: gpio-keys as /devices/platform/gpio-keys/input/input01605worker1 # [ 1.118489] Freeing unused kernel memory: 4736K1606client # [ 1.063875] Segment Routing with IPv61607worker1 # [ 1.118718] Run /init as init process1608client # [ 1.063897] In-situ OAM (IOAM) with IPv61609client # [ 1.063968] NET: Registered PF_PACKET protocol family1610lb # [ 1.080128] Key type dns_resolver registered1611client # [ 1.065866] 9pnet: Installing 9P2000 support1612lb # [ 1.086783] registered taskstats version 11613client # [ 1.068676] Key type dns_resolver registered1614lb # [ 1.087497] Loading compiled-in X.509 certificates1615client # [ 1.075294] registered taskstats version 11616client # [ 1.075451] Loading compiled-in X.509 certificates1617lb # [ 1.096078] Demotion targets for Node 0: null1618lb # [ 1.096224] Key type .fscrypt registered1619worker1 # [ 1.136075] systemd[1]: Successfully made /usr/ read-only.1620lb # [ 1.096230] Key type fscrypt-provisioning registered1621lb # [ 1.096332] ima: No TPM chip found, activating TPM-bypass!1622lb # [ 1.096352] ima: Allocated hash algorithm: sha11623lb # [ 1.096376] ima: No architecture policies found1624lb # [ 1.100551] input: gpio-keys as /devices/platform/gpio-keys/input/input01625client # [ 1.084289] Demotion targets for Node 0: null1626client # [ 1.084434] Key type .fscrypt registered1627client # [ 1.084441] Key type fscrypt-provisioning registered1628client # [ 1.084545] ima: No TPM chip found, activating TPM-bypass!1629worker2 # [ 1.120246] clk: Disabling unused clocks1630client # [ 1.084564] ima: Allocated hash algorithm: sha11631worker2 # [ 1.120328] PM: genpd: Disabling unused power domains1632client # [ 1.084590] ima: No architecture policies found1633client # [ 1.089204] input: gpio-keys as /devices/platform/gpio-keys/input/input01634worker2 # [ 1.124675] Freeing unused kernel memory: 4736K1635worker2 # [ 1.124894] Run /init as init process1636worker2 # [ 1.141123] systemd[1]: Successfully made /usr/ read-only.1637lb # [ 1.123249] clk: Disabling unused clocks1638lb # [ 1.123287] PM: genpd: Disabling unused power domains1639lb # [ 1.127687] Freeing unused kernel memory: 4736K1640client # [ 1.111187] clk: Disabling unused clocks1641lb # [ 1.127931] Run /init as init process1642client # [ 1.111232] PM: genpd: Disabling unused power domains1643client # [ 1.115631] Freeing unused kernel memory: 4736K1644client # [ 1.115836] Run /init as init process1645lb # [ 1.145459] systemd[1]: Successfully made /usr/ read-only.1646client # [ 1.134691] systemd[1]: Successfully made /usr/ read-only.1647worker1 # [ 1.309634] usb 1-1: new high-speed USB device number 2 using ehci-pci1648worker2 # [ 1.298565] usb 1-1: new high-speed USB device number 2 using ehci-pci1649lb # [ 1.298943] usb 1-1: new high-speed USB device number 2 using ehci-pci1650client # [ 1.288175] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker1 # [ 1.464171] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652worker2 # [ 1.450536] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653worker1 # [ 1.471055] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1654worker1 # [ 1.483496] systemd[1]: Detected virtualization qemu.1655worker1 # [ 1.483599] systemd[1]: Detected architecture arm64.1656worker1 # [ 1.483623] systemd[1]: Running in initrd.1657worker1 # [ 1.484655] systemd[1]: Initializing machine ID from random generator.1658worker1 # [ 1.484939] systemd[1]: Hostname set to <worker1>.1659lb # [ 1.451331] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660client # [ 1.442533] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11661worker2 # [ 1.475773] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1662worker2 # [ 1.488295] systemd[1]: Detected virtualization qemu.1663worker2 # [ 1.490425] systemd[1]: Detected architecture arm64.1664worker2 # [ 1.492485] systemd[1]: Running in initrd.1665worker2 # [ 1.495219] systemd[1]: Initializing machine ID from random generator.1666worker2 # [ 1.498321] systemd[1]: Hostname set to <worker2>.1667lb # [ 1.480533] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668lb # [ 1.492869] systemd[1]: Detected virtualization qemu.1669lb # [ 1.495039] systemd[1]: Detected architecture arm64.1670client # [ 1.469650] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1671lb # [ 1.496979] systemd[1]: Running in initrd.1672client # [ 1.481967] systemd[1]: Detected virtualization qemu.1673client # [ 1.483993] systemd[1]: Detected architecture arm64.1674lb # [ 1.499671] systemd[1]: Initializing machine ID from random generator.1675client # [ 1.486073] systemd[1]: Running in initrd.1676lb # [ 1.502668] systemd[1]: Hostname set to <lb>.1677client # [ 1.488826] systemd[1]: Initializing machine ID from random generator.1678client # [ 1.491898] systemd[1]: Hostname set to <client>.1679worker1 # [ 1.549857] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01680worker2 # [ 1.538780] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681lb # [ 1.539346] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682client # [ 1.532527] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker1 # [ 1.673641] usb 1-2: new high-speed USB device number 3 using ehci-pci1684worker2 # [ 1.658627] usb 1-2: new high-speed USB device number 3 using ehci-pci1685lb # [ 1.662976] usb 1-2: new high-speed USB device number 3 using ehci-pci1686client # [ 1.660155] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker2 # [ 1.779916] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688worker1 # [ 1.806678] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689worker1 # [ 1.835985] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21690worker2 # [ 1.817218] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21691worker2 # [ 1.819548] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01692worker1 # [ 1.841875] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01693lb # [ 1.824362] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21694lb # [ 1.830328] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01695worker2 # [ 1.858841] systemd[1]: Queued start job for default target Initrd Default Target.1696client # [ 1.826069] systemd[1]: bpf-restrict-fs: LSM BPF program attached1697lb # [ 1.842653] systemd[1]: bpf-restrict-fs: LSM BPF program attached1698worker2 # [ 1.869270] systemd[1]: Created slice Slice /system/modprobe.1699client # [ 1.829423] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21700worker2 # [ 1.870375] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1701worker2 # [ 1.871538] systemd[1]: Expecting device /dev/disk/by-label/nixos...1702worker2 # [ 1.872515] systemd[1]: Reached target Path Units.1703worker1 # [ 1.892000] systemd[1]: Queued start job for default target Initrd Default Target.1704worker2 # [ 1.873155] systemd[1]: Reached target Slice Units.1705worker2 # [ 1.873821] systemd[1]: Reached target Swaps.1706client # [ 1.829800] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01707worker2 # [ 1.874420] systemd[1]: Reached target Timer Units.1708worker2 # [ 1.875367] systemd[1]: Listening on D-Bus System Message Bus Socket.1709worker2 # [ 1.876369] systemd[1]: Listening on Journal Socket (/dev/log).1710worker2 # [ 1.877304] systemd[1]: Listening on Journal Sockets.1711worker2 # [ 1.878094] systemd[1]: Listening on udev Control Socket.1712worker2 # [ 1.878991] systemd[1]: Listening on udev Kernel Socket.1713worker2 # [ 1.879684] systemd[1]: Reached target Socket Units.1714worker1 # [ 1.899674] systemd[1]: Created slice Slice /system/modprobe.1715worker1 # [ 1.900733] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1716worker1 # [ 1.901866] systemd[1]: Expecting device /dev/disk/by-label/nixos...1717worker2 # [ 1.882189] systemd[1]: Starting Create List of Static Device Nodes...1718worker1 # [ 1.902730] systemd[1]: Reached target Path Units.1719worker1 # [ 1.903396] systemd[1]: Reached target Slice Units.1720worker1 # [ 1.904050] systemd[1]: Reached target Swaps.1721worker1 # [ 1.904660] systemd[1]: Reached target Timer Units.1722worker1 # [ 1.905504] systemd[1]: Listening on D-Bus System Message Bus Socket.1723worker1 # [ 1.906717] systemd[1]: Listening on Journal Socket (/dev/log).1724worker1 # [ 1.907639] systemd[1]: Listening on Journal Sockets.1725worker1 # [ 1.908472] systemd[1]: Listening on udev Control Socket.1726worker1 # [ 1.909295] systemd[1]: Listening on udev Kernel Socket.1727worker1 # [ 1.910089] systemd[1]: Reached target Socket Units.1728worker2 # [ 1.890781] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1729worker2 # [ 1.890899] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1730worker1 # [ 1.912273] systemd[1]: Starting Create List of Static Device Nodes...1731worker2 # [ 1.898860] systemd[1]: Mounting Kernel Configuration File System...1732worker1 # [ 1.921793] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1733worker1 # [ 1.922782] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1734worker1 # [ 1.929910] systemd[1]: Mounting Kernel Configuration File System...1735worker2 # [ 1.931444] systemd[1]: Starting Journal Service...1736worker2 # [ 1.938796] systemd[1]: Starting Load Kernel Modules...1737worker2 # [ 1.939719] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1738worker1 # [ 1.961902] systemd[1]: Starting Journal Service...1739worker1 # [ 1.969753] systemd[1]: Starting Load Kernel Modules...1740worker1 # [ 1.970625] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1741lb # [ 1.935724] systemd[1]: Queued start job for default target Initrd Default Target.1742worker1 # [ 1.977911] systemd[1]: Starting Coldplug All udev Devices...1743worker2 # [ 1.958704] systemd[1]: Starting Coldplug All udev Devices...1744lb # [ 1.945421] systemd[1]: Created slice Slice /system/modprobe.1745client # [ 1.930055] systemd[1]: Queued start job for default target Initrd Default Target.1746lb # [ 1.946483] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1747lb # [ 1.947642] systemd[1]: Expecting device /dev/disk/by-label/nixos...1748lb # [ 1.948495] systemd[1]: Reached target Path Units.1749lb # [ 1.949140] systemd[1]: Reached target Slice Units.1750lb # [ 1.949788] systemd[1]: Reached target Swaps.1751lb # [ 1.950364] systemd[1]: Reached target Timer Units.1752lb # [ 1.951240] systemd[1]: Listening on D-Bus System Message Bus Socket.1753lb # [ 1.952202] systemd[1]: Listening on Journal Socket (/dev/log).1754lb # [ 1.953113] systemd[1]: Listening on Journal Sockets.1755lb # [ 1.953895] systemd[1]: Listening on udev Control Socket.1756lb # [ 1.954676] systemd[1]: Listening on udev Kernel Socket.1757client # [ 1.938486] systemd[1]: Created slice Slice /system/modprobe.1758worker2 # [ 1.970900] systemd[1]: Finished Create List of Static Device Nodes.1759lb # [ 1.955455] systemd[1]: Reached target Socket Units.1760worker2 # [ 1.972087] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1761client # [ 1.939670] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1762worker2 # [ 1.972493] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1763worker2 # [ 1.972778] systemd[1]: Mounted Kernel Configuration File System.1764client # [ 1.941050] systemd[1]: Expecting device /dev/disk/by-label/nixos...1765client # [ 1.942186] systemd[1]: Reached target Path Units.1766client # [ 1.942879] systemd[1]: Reached target Slice Units.1767client # [ 1.943576] systemd[1]: Reached target Swaps.1768client # [ 1.944244] systemd[1]: Reached target Timer Units.1769client # [ 1.945227] systemd[1]: Listening on D-Bus System Message Bus Socket.1770worker1 # [ 1.997813] systemd[1]: Finished Create List of Static Device Nodes.1771client # [ 1.946333] systemd[1]: Listening on Journal Socket (/dev/log).1772worker1 # [ 1.998888] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1773client # [ 1.947351] systemd[1]: Listening on Journal Sockets.1774worker1 # [ 1.999258] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1775worker1 # [ 1.999522] systemd[1]: Mounted Kernel Configuration File System.1776client # [ 1.948273] systemd[1]: Listening on udev Control Socket.1777client # [ 1.949232] systemd[1]: Listening on udev Kernel Socket.1778lb # [ 1.957710] systemd[1]: Starting Create List of Static Device Nodes...1779client # [ 1.950048] systemd[1]: Reached target Socket Units.1780lb # [ 1.967143] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1781lb # [ 1.968159] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1782client # [ 1.952498] systemd[1]: Starting Create List of Static Device Nodes...1783worker2 # [ 1.991066] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1784worker1 # [ 2.014142] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1785lb # [ 1.975280] systemd[1]: Mounting Kernel Configuration File System...1786client # [ 1.960355] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1787client # [ 1.961455] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1788client # [ 1.968524] systemd[1]: Mounting Kernel Configuration File System...1789worker2 # [ 2.011697] systemd-journald[73]: Collecting audit messages is disabled.1790lb # [ 1.995854] systemd[1]: Starting Journal Service...1791worker1 # [ 2.050724] systemd-journald[73]: Collecting audit messages is disabled.1792client # [ 1.996506] systemd[1]: Starting Journal Service...1793client # [ 1.998965] systemd[1]: Starting Load Kernel Modules...1794lb # [ 2.015189] systemd[1]: Starting Load Kernel Modules...1795client # [ 1.999809] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1796lb # [ 2.016066] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1797worker1 # [ 2.056146] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1798worker2 # [ 2.037312] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1799lb # [ 2.028137] systemd[1]: Starting Coldplug All udev Devices...1800client # [ 2.014339] systemd[1]: Starting Coldplug All udev Devices...1801worker1 # [ 2.073650] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1802worker2 # [ 2.054568] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1803worker1 # [ 2.078385] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1804worker2 # [ 2.063637] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01805worker1 # [ 2.085424] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01806worker2 # [ 2.063909] [drm] features: -virgl +edid -resource_blob -host_visible1807worker2 # [ 2.063921] [drm] features: -context_init1808lb # [ 2.047074] systemd[1]: Finished Create List of Static Device Nodes.1809worker2 # [ 2.064737] [drm] number of scanouts: 11810worker2 # [ 2.064760] [drm] number of cap sets: 01811client # [ 2.032988] systemd[1]: Finished Create List of Static Device Nodes.1812lb # [ 2.049329] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1813worker2 # [ 2.070748] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1814client # [ 2.037386] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1815lb # [ 2.055399] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1816lb # [ 2.056629] systemd[1]: Mounted Kernel Configuration File System.1817worker1 # [ 2.094044] systemd[1]: Starting Create Static Device Nodes in /dev...1818worker1 # [ 2.101797] [drm] features: -virgl +edid -resource_blob -host_visible1819worker1 # [ 2.101822] [drm] features: -context_init1820worker1 # [ 2.102653] [drm] number of scanouts: 11821worker1 # [ 2.102676] [drm] number of cap sets: 01822client # [ 2.048796] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1823worker2 # [ 2.083025] systemd[1]: Starting Create Static Device Nodes in /dev...1824client # [ 2.049631] systemd[1]: Mounted Kernel Configuration File System.1825lb # [ 2.069637] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1826worker2 # [ 2.088002] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1827worker2 # [ 2.088034] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01828lb # [ 2.074546] systemd-journald[73]: Collecting audit messages is disabled.1829worker1 # [ 2.110970] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1830worker1 # [ 2.111003] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01831client # [ 2.056714] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1832lb # [ 2.079489] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1833client # [ 2.077629] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1834lb # [ 2.092019] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1835client # [ 2.085115] systemd-journald[73]: Collecting audit messages is disabled.1836lb # [ 2.109250] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01837lb # [ 2.109511] [drm] features: -virgl +edid -resource_blob -host_visible1838lb # [ 2.109522] [drm] features: -context_init1839lb # [ 2.110287] [drm] number of scanouts: 11840lb # [ 2.110306] [drm] number of cap sets: 01841worker2 # [ 2.126840] Console: switching to colour frame buffer device 160x501842worker2 # [ 2.133335] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1843client # [ 2.096577] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1844client # [ 2.102927] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01845worker1 # [ 2.149920] Console: switching to colour frame buffer device 160x501846client # [ 2.103187] [drm] features: -virgl +edid -resource_blob -host_visible1847client # [ 2.103199] [drm] features: -context_init1848lb # [ 2.120687] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1849client # [ 2.104050] [drm] number of scanouts: 11850client # [ 2.104070] [drm] number of cap sets: 01851lb # [ 2.120720] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01852worker2 # [ 2.155550] systemd[1]: Finished Create Static Device Nodes in /dev.1853worker2 # [ 2.158143] systemd[1]: Reached target Preparation for Local File Systems.1854worker1 # [ 2.178507] systemd[1]: Finished Create Static Device Nodes in /dev.1855worker1 # [ 2.178975] systemd[1]: Reached target Preparation for Local File Systems.1856worker1 # [ 2.179013] systemd[1]: Reached target Local File Systems.1857worker1 # [ 2.183490] systemd[1]: Starting Rule-based Manager for Device Events and Files...1858worker2 # [ 2.162796] systemd[1]: Reached target Local File Systems.1859lb # [ 2.151679] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1860client # [ 2.133406] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1861client # [ 2.133439] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01862worker2 # [ 2.175036] systemd[1]: Starting Rule-based Manager for Device Events and Files...1863worker1 # [ 2.194236] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1864worker2 # [ 2.178338] systemd[1]: Finished Load Kernel Modules.1865client # [ 2.142691] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1866client # [ 2.145671] systemd[1]: Starting Create Static Device Nodes in /dev...1867lb # [ 2.159256] Console: switching to colour frame buffer device 160x501868worker1 # [ 2.210277] systemd[1]: Finished Load Kernel Modules.1869lb # [ 2.170312] systemd[1]: Starting Create Static Device Nodes in /dev...1870worker2 # [ 2.187840] systemd-modules-load[74]: Inserted module 'dm_mod'1871worker1 # [ 2.230099] systemd[1]: Starting Apply Kernel Variables...1872worker2 # [ 2.197120] systemd-modules-load[74]: Module 'virtio_balloon' is built in1873worker2 # [ 2.198391] systemd-modules-load[74]: Module 'virtio_console' is built in1874worker2 # [ 2.199504] systemd-modules-load[74]: Inserted module 'virtio_gpu'1875client # [ 2.168407] Console: switching to colour frame buffer device 160x501876lb # [ 2.195604] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1877client # [ 2.177314] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1878worker2 # [ 2.215154] systemd[1]: Starting Apply Kernel Variables...1879worker2 # [ 2.215434] systemd[1]: Started Journal Service.1880worker2 # [ 2.208944] systemd-modules-load[74]: Module 'virtio_rng' is built in1881worker1 # [ 2.228519] systemd-modules-load[74]: Inserted module 'dm_mod'1882lb # [ 2.207632] systemd[1]: Finished Load Kernel Modules.1883worker1 # [ 2.232364] systemd-modules-load[74]: Module 'virtio_balloon' is built in1884worker1 # [ 2.233601] systemd-modules-load[74]: Module 'virtio_console' is built in1885worker1 # [ 2.234756] systemd-modules-load[74]: Inserted module 'virtio_gpu'1886worker1 # [ 2.235734] systemd-modules-load[74]: Module 'virtio_rng' is built in1887worker1 # [ 2.254490] systemd[1]: Started Journal Service.1888worker2 # [ 2.224206] systemd[1]: Starting Create System Files and Directories...1889lb # [ 2.220093] systemd[1]: Starting Apply Kernel Variables...1890client # [ 2.205209] systemd[1]: Finished Create Static Device Nodes in /dev.1891client # [ 2.205476] systemd[1]: Reached target Preparation for Local File Systems.1892client # [ 2.205515] systemd[1]: Reached target Local File Systems.1893client # [ 2.211265] systemd[1]: Starting Rule-based Manager for Device Events and Files...1894lb # [ 2.247552] systemd[1]: Finished Create Static Device Nodes in /dev.1895lb # [ 2.247934] systemd[1]: Reached target Preparation for Local File Systems.1896lb # [ 2.247978] systemd[1]: Reached target Local File Systems.1897client # [ 2.237311] systemd[1]: Finished Load Kernel Modules.1898worker1 # [ 2.277335] systemd[1]: Starting Create System Files and Directories...1899client # [ 2.248824] systemd[1]: Starting Apply Kernel Variables...1900lb # [ 2.263392] systemd[1]: Starting Rule-based Manager for Device Events and Files...1901worker1 # [ 2.290221] systemd[1]: Finished Apply Kernel Variables.1902worker2 # [ 2.274262] systemd[1]: Finished Apply Kernel Variables.1903lb # [ 2.283517] systemd[1]: Started Journal Service.1904worker1 # [ 2.310394] systemd-udevd[79]: Using default interface naming scheme 'v261'.1905lb # [ 2.267991] systemd-modules-load[74]: Inserted module 'dm_mod'1906worker2 # [ 2.300413] systemd[1]: Finished Create System Files and Directories.1907lb # [ 2.284500] systemd-modules-load[74]: Module 'virtio_balloon' is built in1908lb # [ 2.285744] systemd-modules-load[74]: Module 'virtio_console' is built in1909client # [ 2.272558] systemd-modules-load[74]: Inserted module 'dm_mod'1910worker2 # [ 2.305692] systemd-udevd[79]: Using default interface naming scheme 'v261'.1911lb # [ 2.286897] systemd-modules-load[74]: Inserted module 'virtio_gpu'1912client # [ 2.273798] systemd-modules-load[74]: Module 'virtio_balloon' is built in1913lb # [ 2.287952] systemd-modules-load[74]: Module 'virtio_rng' is built in1914client # [ 2.274925] systemd-modules-load[74]: Module 'virtio_console' is built in1915client # [ 2.288380] systemd[1]: Started Journal Service.1916worker1 # [ 2.336583] systemd[1]: Finished Create System Files and Directories.1917lb # [ 2.302601] systemd[1]: Starting Create System Files and Directories...1918client # [ 2.292719] systemd-modules-load[74]: Inserted module 'virtio_gpu'1919client # [ 2.296097] systemd-modules-load[74]: Module 'virtio_rng' is built in1920lb # [ 2.309790] systemd[1]: Finished Apply Kernel Variables.1921client # [ 2.300084] systemd[1]: Starting Create System Files and Directories...1922worker2 # [ 2.338586] systemd[1]: Started Rule-based Manager for Device Events and Files.1923worker1 # [ 2.357035] systemd[1]: Started Rule-based Manager for Device Events and Files.1924client # [ 2.320287] systemd[1]: Finished Apply Kernel Variables.1925client # [ 2.330607] systemd-udevd[79]: Using default interface naming scheme 'v261'.1926lb # [ 2.347503] systemd[1]: Finished Create System Files and Directories.1927lb # [ 2.360355] systemd-udevd[81]: Using default interface naming scheme 'v261'.1928client # [ 2.371652] systemd[1]: Finished Create System Files and Directories.1929worker1 # [ 2.424813] systemd[1]: Starting Virtual Console Setup...1930worker2 # [ 2.408836] systemd[1]: Starting Virtual Console Setup...1931lb # [ 2.391022] systemd[1]: Started Rule-based Manager for Device Events and Files.1932client # [ 2.380341] systemd[1]: Started Rule-based Manager for Device Events and Files.1933worker1 # [ 2.480597] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1934worker1 # [ 2.484554] systemd[1]: Finished Virtual Console Setup.1935worker2 # [ 2.468643] systemd-vconsole-setup[104]: Configuration of first virtual console was skipped, ignoring remaining ones.1936worker2 # [ 2.472531] systemd[1]: Finished Virtual Console Setup.1937client # [ 2.441855] systemd[1]: Starting Virtual Console Setup...1938lb # [ 2.460942] systemd[1]: Starting Virtual Console Setup...1939client # [ 2.496608] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940lb # [ 2.516555] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1941client # [ 2.504269] systemd[1]: Finished Virtual Console Setup.1942lb # [ 2.520303] systemd[1]: Finished Virtual Console Setup.1943worker1 # [ 3.068494] systemd[1]: Finished Coldplug All udev Devices.1944worker1 # [ 3.070168] systemd[1]: Reached target System Initialization.1945worker1 # [ 3.072129] systemd[1]: Reached target Basic System.1946worker2 # [ 3.073569] systemd[1]: Finished Coldplug All udev Devices.1947worker2 # [ 3.076168] systemd[1]: Reached target System Initialization.1948worker2 # [ 3.080169] systemd[1]: Reached target Basic System.1949lb # [ 3.116497] systemd[1]: Finished Coldplug All udev Devices.1950client # [ 3.104472] systemd[1]: Finished Coldplug All udev Devices.1951lb # [ 3.118669] systemd[1]: Reached target System Initialization.1952client # [ 3.106274] systemd[1]: Reached target System Initialization.1953lb # [ 3.120156] systemd[1]: Reached target Basic System.1954client # [ 3.107162] systemd[1]: Reached target Basic System.1955worker1 # [ 3.246504] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1956worker1 # [ 3.258326] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1957worker2 # [ 3.243667] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1958worker1 # [ 3.263288] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1959worker2 # [ 3.269764] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960lb # [ 3.251881] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1961worker2 # [ 3.276242] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1962client # [ 3.248158] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1963client # [ 3.275124] (udev-worker)[102]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1964worker1 # [ 3.336650] systemd[1]: Found device /dev/disk/by-label/nixos.1965worker1 # [ 3.339617] systemd[1]: Reached target Initrd Root Device.1966client # [ 3.293451] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1967worker1 # [ 3.344127] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1968lb # [ 3.306695] (udev-worker)[107]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969lb # [ 3.311428] (udev-worker)[107]: Network interface NamePolicy= disabled on kernel command line.1970worker2 # [ 3.332447] systemd[1]: Found device /dev/disk/by-label/nixos.1971worker2 # [ 3.335722] systemd[1]: Reached target Initrd Root Device.1972worker2 # [ 3.339114] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1973lb # [ 3.369650] systemd[1]: Found device /dev/disk/by-label/nixos.1974worker1 # [ 3.408397] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1975lb # [ 3.372790] systemd[1]: Reached target Initrd Root Device.1976lb # [ 3.376605] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1977client # [ 3.364518] systemd[1]: Found device /dev/disk/by-label/nixos.1978worker1 # [ 3.416342] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1979worker2 # [ 3.401639] systemd-fsck[113]: nixos: clean, 12/65536 files, 13019/262144 blocks1980client # [ 3.371636] systemd[1]: Reached target Initrd Root Device.1981worker1 # [ 3.425192] systemd[1]: Mounting /sysroot...1982worker2 # [ 3.412069] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1983client # [ 3.379023] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1984worker2 # [ 3.415446] systemd[1]: Mounting /sysroot...1985worker1 # [ 3.483072] EXT4-fs (vda): mounted filesystem b7494a4a-0c71-4154-867d-11aa1912d093 r/w with ordered data mode. Quota mode: none.1986worker1 # [ 3.473103] systemd[1]: Mounted /sysroot.1987lb # [ 3.437415] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1988worker1 # [ 3.475550] systemd[1]: Reached target Initrd Root File System.1989worker1 # [ 3.480354] systemd[1]: Starting Mountpoints Configured in the Real Root...1990worker2 # [ 3.476565] EXT4-fs (vda): mounted filesystem 1901d721-ca8c-4062-ae20-6ee1aeb92fa2 r/w with ordered data mode. Quota mode: none.1991lb # [ 3.448601] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1992worker2 # [ 3.469553] systemd[1]: Mounted /sysroot.1993worker2 # [ 3.471968] systemd[1]: Reached target Initrd Root File System.1994client # [ 3.438736] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1995lb # [ 3.455227] systemd[1]: Mounting /sysroot...1996client # [ 3.446135] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1997worker2 # [ 3.480149] systemd[1]: Starting Mountpoints Configured in the Real Root...1998client # [ 3.459720] systemd[1]: Mounting /sysroot...1999worker1 # [ 3.513119] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.2000worker1 # [ 3.516797] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...
loading 2,301 hidden lines…
4302(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.60 seconds)4303subtest: repeat is answered Cached by the scheduler without building4304worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4305worker1 # finding garbage collector roots...4306worker1 # deleting specified paths...4307worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4308worker1 # deleting unused links...4309worker1 # note: hard linking is currently saving 0.0 KiB4310worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4311worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4312worker2 # finding garbage collector roots...4313worker2 # deleting specified paths...4314worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4315worker2 # deleting unused links...4316worker2 # note: hard linking is currently saving 0.0 KiB4317worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4318client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24319client # don't know how to build these paths:4320client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4321worker2 # [ 60.066144] nix-grpc-daemon[780]: ts=2026-09-22T10:57:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401324322worker1 # [ 60.088968] nix-grpc-daemon[778]: ts=2026-09-22T10:57:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384323worker1 # [ 60.094010] nix-grpc-daemon[857]: time=2026-09-22T10:57:50.564Z level=INFO msg="All 2 paths already cached"4324worker2 # [ 60.094960] nix-grpc-daemon[780]: ts=2026-09-22T10:57:50Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14325client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.33 seconds)4326client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4327client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4328lb: must succeed: curl -sf localhost:9901/clusters4329lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4330lb: must succeed: curl -sf localhost:9901/clusters4331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4332worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4333worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4334(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.45 seconds)4335subtest: inputs already in the cache are not sent to the scheduler4336lb: must succeed: curl -sf localhost:9901/clusters4337lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4338lb: must succeed: curl -sf localhost:9901/clusters4339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4340worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4341worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4342client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24343client # don't know how to build these paths:4344client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4345worker2 # [ 61.138460] nix-grpc-daemon[780]: ts=2026-09-22T10:57:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401324346worker1 # [ 61.160494] nix-grpc-daemon[778]: ts=2026-09-22T10:57:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384347worker1 # [ 61.174351] nix-grpc-daemon[857]: time=2026-09-22T10:57:51.642Z level=INFO msg="All 2 paths already cached"4348worker2 # [ 61.167693] nix-grpc-daemon[780]: ts=2026-09-22T10:57:51Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04349worker2 # [ 61.173925] nix-grpc-daemon[780]: ts=2026-09-22T10:57:51Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14350worker1 # [ 61.232390] nix-grpc-daemon[778]: ts=2026-09-22T10:57:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384351worker1 # [ 61.249157] nix-grpc-daemon[778]: ts=2026-09-22T10:57:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384352worker1 # [ 61.264073] nix-grpc-daemon[857]: time=2026-09-22T10:57:51.735Z level=INFO msg="All 2 paths already cached"4353client # copying 1 paths to 'lb:50051'...4354client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4355worker1 # [ 61.275408] nix-grpc-daemon[778]: ts=2026-09-22T10:57:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413384356worker1 # [ 61.290420] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4357worker1 # [ 61.493119] nix-daemon[844]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4358worker1 # [ 61.514095] nix-daemon[844]: reaped child process 991, status = succeeded4359worker1 # [ 61.682968] nix-daemon[844]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4360worker1 # [ 61.702702] nix-daemon[844]: reaped child process 996, status = succeeded4361lb # [ 61.679713] niks3-server[1035]: 2026/09/22 10:57:52 INFO Received uploads request method=POST path=/api/pending_closures4362worker1 # [ 61.726632] nix-grpc-daemon[857]: time=2026-09-22T10:57:52.197Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4363worker1 # [ 61.728195] nix-grpc-daemon[857]: time=2026-09-22T10:57:52.197Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4364lb # [ 61.710680] niks3-server[1035]: 2026/09/22 10:57:52 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4365lb # [ 61.726844] niks3-server[1035]: 2026/09/22 10:57:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4366worker1 # [ 61.769276] nix-grpc-daemon[857]: time=2026-09-22T10:57:52.240Z level=INFO msg="Uploading 1 narinfos"4367lb # [ 61.739867] niks3-server[1035]: 2026/09/22 10:57:52 INFO Signed narinfos id=7 count=14368lb # [ 61.746620] niks3-server[1035]: 2026/09/22 10:57:52 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4369worker1 # [ 61.793819] nix-grpc-daemon[857]: time=2026-09-22T10:57:52.265Z level=INFO msg="Upload complete. (485ms)"4370lb # [ 61.757795] niks3-server[1035]: 2026/09/22 10:57:52 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4371lb # [ 61.761241] niks3-server[1035]: 2026/09/22 10:57:52 INFO Completed upload id=74372lb # [ 61.764428] niks3-server[1035]: 2026/09/22 10:57:52 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4373worker1 # [ 61.803651] nix-grpc-daemon[778]: ts=2026-09-22T10:57:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_in=7524374worker1 # [ 61.817960] nix-grpc-daemon[778]: ts=2026-09-22T10:57:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384375worker1 # [ 61.824305] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4376worker1 # [ 61.829899] nix-daemon[844]: reaped child process 986, status = succeeded4377client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4378worker1 # [ 61.842369] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4379client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4380client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4381client # farm-top> LOG-farm-top-t1b4382worker1 # [ 63.490127] nix-daemon[844]: reaped child process 1000, status = succeeded4383worker1 # [ 63.661066] nix-daemon[844]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)4384worker1 # [ 63.679276] nix-daemon[844]: reaped child process 1019, status = succeeded4385lb # [ 63.657092] niks3-server[1035]: 2026/09/22 10:57:54 INFO Received uploads request method=POST path=/api/pending_closures4386worker1 # [ 63.708427] nix-grpc-daemon[857]: time=2026-09-22T10:57:54.177Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4387worker1 # [ 63.710177] nix-grpc-daemon[857]: time=2026-09-22T10:57:54.177Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4388lb # [ 63.693952] niks3-server[1035]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4389lb # [ 63.711102] niks3-server[1035]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4390lb # [ 63.721950] niks3-server[1035]: 2026/09/22 10:57:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4391worker1 # [ 63.762660] nix-grpc-daemon[857]: time=2026-09-22T10:57:54.233Z level=INFO msg="Uploading 1 narinfos"4392lb # [ 63.729700] niks3-server[1035]: 2026/09/22 10:57:54 INFO Signed narinfos id=8 count=14393lb # [ 63.731411] niks3-server[1035]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4394lb # [ 63.739016] niks3-server[1035]: 2026/09/22 10:57:54 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4395lb # [ 63.744341] niks3-server[1035]: 2026/09/22 10:57:54 INFO Completed upload id=84396worker1 # [ 63.781123] nix-grpc-daemon[857]: time=2026-09-22T10:57:54.252Z level=INFO msg="Upload complete. (324ms)"4397lb # [ 63.747275] niks3-server[1035]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4398worker1 # [ 63.793085] nix-grpc-daemon[778]: ts=2026-09-22T10:57:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14399worker1 # [ 63.808360] nix-daemon[844]: reaped child process 998, status = succeeded4400client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.95 seconds)4401client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4402client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds)4403lb: must succeed: curl -sf localhost:9901/clusters4404lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4405lb: must succeed: curl -sf localhost:9901/clusters4406lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4407worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4408lb # [ 64.193636] envoy[933]: conn peer=[2001:db8:1::1]:57940 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9536 tx=4410 ms=76274409lb # [ 64.196692] envoy[933]: conn peer=[2001:db8:1::1]:59212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=514410lb # [ 64.199643] envoy[933]: conn peer=[2001:db8:1::1]:60148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4215 tx=1935 ms=26524411worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4412(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.58 seconds)4413subtest: outputs read back through any worker carry the cache signature4414client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag sig >&24415worker2 # [ 64.521880] nix-grpc-daemon[780]: ts=2026-09-22T10:57:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401324416client # don't know how to build these paths:4417client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4418worker2 # [ 64.531707] nix-grpc-daemon[780]: ts=2026-09-22T10:57:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401324419worker2 # [ 64.558267] nix-grpc-daemon[780]: ts=2026-09-22T10:57:55Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04420worker2 # [ 64.566230] nix-grpc-daemon[780]: ts=2026-09-22T10:57:55Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14421worker2 # [ 64.574001] nix-grpc-daemon[780]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324422worker2 # [ 64.594762] nix-grpc-daemon[780]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401324423client # copying 1 paths to 'lb:50051'...4424client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4425worker2 # [ 64.611115] nix-grpc-daemon[780]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401324426worker2 # [ 64.627812] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4427worker2 # [ 64.655327] nix-grpc-daemon[780]: ts=2026-09-22T10:57:55Z level=info event=niks3_push_started pid=9174428worker2 # [ 64.928347] nix-grpc-daemon[917]: time=2026-09-22T10:57:55.385Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4429worker2 # [ 65.151654] nix-daemon[866]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4430worker2 # [ 65.175400] nix-daemon[866]: reaped child process 933, status = succeeded4431worker2 # [ 65.357339] nix-daemon[866]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4432worker2 # [ 65.377549] nix-daemon[866]: reaped child process 938, status = succeeded4433lb # [ 65.374027] niks3-server[1035]: 2026/09/22 10:57:55 INFO Received uploads request method=POST path=/api/pending_closures4434worker2 # [ 65.402989] nix-grpc-daemon[917]: time=2026-09-22T10:57:55.862Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4435worker2 # [ 65.406332] nix-grpc-daemon[917]: time=2026-09-22T10:57:55.864Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4436lb # [ 65.411178] niks3-server[1035]: 2026/09/22 10:57:55 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4437worker2 # [ 65.443440] nix-grpc-daemon[917]: time=2026-09-22T10:57:55.902Z level=INFO msg="Uploading 1 narinfos"4438lb # [ 65.423450] niks3-server[1035]: 2026/09/22 10:57:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4439lb # [ 65.432934] niks3-server[1035]: 2026/09/22 10:57:55 INFO Signed narinfos id=9 count=14440lb # [ 65.441169] niks3-server[1035]: 2026/09/22 10:57:55 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4441lb # [ 65.452517] niks3-server[1035]: 2026/09/22 10:57:55 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4442worker2 # [ 65.478279] nix-grpc-daemon[917]: time=2026-09-22T10:57:55.937Z level=INFO msg="Upload complete. (542ms)"4443lb # [ 65.460430] niks3-server[1035]: 2026/09/22 10:57:55 INFO Completed upload id=94444lb # [ 65.464731] niks3-server[1035]: 2026/09/22 10:57:55 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4445worker2 # [ 65.488304] nix-grpc-daemon[780]: ts=2026-09-22T10:57:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=0 paths=1 nar_bytes_in=5124446worker2 # [ 65.500204] nix-grpc-daemon[780]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324447worker2 # [ 65.507579] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4448worker2 # [ 65.513636] nix-daemon[866]: reaped child process 915, status = succeeded4449client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4450client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4451worker2 # [ 65.527789] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4452client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4453client # farm-a> LOG-farm-a-sig4454worker2 # [ 66.570419] nix-daemon[866]: reaped child process 942, status = succeeded4455worker2 # [ 66.748619] nix-daemon[866]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4456worker2 # [ 66.767638] nix-daemon[866]: reaped child process 961, status = succeeded4457lb # [ 66.760758] niks3-server[1035]: 2026/09/22 10:57:57 INFO Received uploads request method=POST path=/api/pending_closures4458worker2 # [ 66.791475] nix-grpc-daemon[917]: time=2026-09-22T10:57:57.249Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4459worker2 # [ 66.795023] nix-grpc-daemon[917]: time=2026-09-22T10:57:57.249Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4460lb # [ 66.804696] niks3-server[1035]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4461lb # [ 66.816828] niks3-server[1035]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4462lb # [ 66.833370] niks3-server[1035]: 2026/09/22 10:57:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4463worker2 # [ 66.860197] nix-grpc-daemon[917]: time=2026-09-22T10:57:57.317Z level=INFO msg="Uploading 1 narinfos"4464lb # [ 66.843289] niks3-server[1035]: 2026/09/22 10:57:57 INFO Signed narinfos id=10 count=14465lb # [ 66.852387] niks3-server[1035]: 2026/09/22 10:57:57 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4466lb # [ 66.888748] niks3-server[1035]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4467lb # [ 66.895801] niks3-server[1035]: 2026/09/22 10:57:57 INFO Completed upload id=104468worker2 # [ 66.914752] nix-grpc-daemon[917]: time=2026-09-22T10:57:57.373Z level=INFO msg="Upload complete. (375ms)"4469lb # [ 66.899963] niks3-server[1035]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4470worker2 # [ 66.938541] nix-grpc-daemon[780]: ts=2026-09-22T10:57:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14471worker2 # [ 66.955234] nix-grpc-daemon[780]: ts=2026-09-22T10:57:57Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04472worker2 # [ 66.959624] nix-grpc-daemon[780]: ts=2026-09-22T10:57:57Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14473worker2 # [ 66.963816] nix-daemon[866]: reaped child process 940, status = succeeded4474worker2 # [ 66.985451] nix-grpc-daemon[780]: ts=2026-09-22T10:57:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324475worker2 # [ 67.001496] nix-grpc-daemon[780]: ts=2026-09-22T10:57:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401324476worker2 # [ 67.017348] nix-grpc-daemon[917]: time=2026-09-22T10:57:57.476Z level=INFO msg="All 1 paths already cached"4477client # copying 1 paths to 'lb:50051'...4478client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4479worker2 # [ 67.029003] nix-grpc-daemon[780]: ts=2026-09-22T10:57:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401324480worker2 # [ 67.043982] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4481worker2 # [ 67.252159] nix-daemon[866]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4482worker2 # [ 67.274079] nix-daemon[866]: reaped child process 970, status = succeeded4483worker2 # [ 67.451690] nix-daemon[866]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4484worker2 # [ 67.472980] nix-daemon[866]: reaped child process 975, status = succeeded4485lb # [ 67.464718] niks3-server[1035]: 2026/09/22 10:57:57 INFO Received uploads request method=POST path=/api/pending_closures4486worker2 # [ 67.497674] nix-grpc-daemon[917]: time=2026-09-22T10:57:57.956Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4487worker2 # [ 67.499180] nix-grpc-daemon[917]: time=2026-09-22T10:57:57.956Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4488lb # [ 67.497198] niks3-server[1035]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4489lb # [ 67.511337] niks3-server[1035]: 2026/09/22 10:57:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4490worker2 # [ 67.539573] nix-grpc-daemon[917]: time=2026-09-22T10:57:57.998Z level=INFO msg="Uploading 1 narinfos"4491lb # [ 67.519677] niks3-server[1035]: 2026/09/22 10:57:57 INFO Signed narinfos id=11 count=14492lb # [ 67.530000] niks3-server[1035]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4493lb # [ 67.539384] niks3-server[1035]: 2026/09/22 10:57:57 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4494lb # [ 67.549705] niks3-server[1035]: 2026/09/22 10:57:58 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4495lb # [ 67.554501] niks3-server[1035]: 2026/09/22 10:57:58 INFO Completed upload id=114496worker2 # [ 67.574469] nix-grpc-daemon[917]: time=2026-09-22T10:57:58.033Z level=INFO msg="Upload complete. (513ms)"4497worker2 # [ 67.583840] nix-grpc-daemon[780]: ts=2026-09-22T10:57:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=0 paths=1 nar_bytes_in=6244498worker2 # [ 67.599278] nix-grpc-daemon[780]: ts=2026-09-22T10:57:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324499worker2 # [ 67.606471] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4500worker2 # [ 67.612354] nix-daemon[866]: reaped child process 965, status = succeeded4501client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4502client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4503worker2 # [ 67.627209] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4504client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4505client # farm-b> LOG-farm-b-sig4506worker2 # [ 68.455353] nix-daemon[866]: reaped child process 979, status = succeeded4507worker2 # [ 68.627443] nix-daemon[866]: accepted connection from pid 995, user nix-grpc-daemon (trusted)4508worker2 # [ 68.647288] nix-daemon[866]: reaped child process 998, status = succeeded4509lb # [ 68.640813] niks3-server[1035]: 2026/09/22 10:57:59 INFO Received uploads request method=POST path=/api/pending_closures4510worker2 # [ 68.671539] nix-grpc-daemon[917]: time=2026-09-22T10:57:59.129Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4511worker2 # [ 68.673189] nix-grpc-daemon[917]: time=2026-09-22T10:57:59.129Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4512lb # [ 68.680982] niks3-server[1035]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4513worker2 # [ 68.714987] nix-grpc-daemon[917]: time=2026-09-22T10:57:59.173Z level=INFO msg="Uploading 1 narinfos"4514lb # [ 68.696576] niks3-server[1035]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4515lb # [ 68.706305] niks3-server[1035]: 2026/09/22 10:57:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4516lb # [ 68.712525] niks3-server[1035]: 2026/09/22 10:57:59 INFO Signed narinfos id=12 count=14517lb # [ 68.719254] niks3-server[1035]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4518lb # [ 68.722727] niks3-server[1035]: 2026/09/22 10:57:59 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4519lb # [ 68.727299] niks3-server[1035]: 2026/09/22 10:57:59 INFO Completed upload id=124520worker2 # [ 68.746544] nix-grpc-daemon[917]: time=2026-09-22T10:57:59.205Z level=INFO msg="Upload complete. (309ms)"4521lb # [ 68.730167] niks3-server[1035]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4522worker2 # [ 68.755964] nix-grpc-daemon[780]: ts=2026-09-22T10:57:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14523worker2 # [ 68.777995] nix-grpc-daemon[780]: ts=2026-09-22T10:57:59Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04524worker2 # [ 68.781846] nix-grpc-daemon[780]: ts=2026-09-22T10:57:59Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14525worker2 # [ 68.786150] nix-daemon[866]: reaped child process 977, status = succeeded4526worker2 # [ 68.816055] nix-grpc-daemon[780]: ts=2026-09-22T10:57:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324527worker2 # [ 68.833195] nix-grpc-daemon[780]: ts=2026-09-22T10:57:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401324528worker2 # [ 68.848501] nix-grpc-daemon[917]: time=2026-09-22T10:57:59.307Z level=INFO msg="All 2 paths already cached"4529client # copying 1 paths to 'lb:50051'...4530client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4531worker2 # [ 68.860472] nix-grpc-daemon[780]: ts=2026-09-22T10:57:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401324532worker2 # [ 68.878323] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4533worker2 # [ 69.078944] nix-daemon[866]: accepted connection from pid 1004, user nix-grpc-daemon (trusted)4534worker2 # [ 69.101039] nix-daemon[866]: reaped child process 1007, status = succeeded4535worker2 # [ 69.268093] nix-daemon[866]: accepted connection from pid 1009, user nix-grpc-daemon (trusted)4536worker2 # [ 69.287109] nix-daemon[866]: reaped child process 1012, status = succeeded4537lb # [ 69.278142] niks3-server[1035]: 2026/09/22 10:57:59 INFO Received uploads request method=POST path=/api/pending_closures4538worker2 # [ 69.305675] nix-grpc-daemon[917]: time=2026-09-22T10:57:59.764Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4539worker2 # [ 69.307152] nix-grpc-daemon[917]: time=2026-09-22T10:57:59.764Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4540lb # [ 69.300350] niks3-server[1035]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4541lb # [ 69.306850] niks3-server[1035]: 2026/09/22 10:57:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4542worker2 # [ 69.330603] nix-grpc-daemon[917]: time=2026-09-22T10:57:59.789Z level=INFO msg="Uploading 1 narinfos"4543lb # [ 69.313251] niks3-server[1035]: 2026/09/22 10:57:59 INFO Signed narinfos id=13 count=14544lb # [ 69.314891] niks3-server[1035]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4545lb # [ 69.322806] niks3-server[1035]: 2026/09/22 10:57:59 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4546lb # [ 69.327716] niks3-server[1035]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4547lb # [ 69.330644] niks3-server[1035]: 2026/09/22 10:57:59 INFO Completed upload id=134548worker2 # [ 69.350497] nix-grpc-daemon[917]: time=2026-09-22T10:57:59.809Z level=INFO msg="Upload complete. (454ms)"4549worker2 # [ 69.356238] nix-grpc-daemon[780]: ts=2026-09-22T10:57:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=0 paths=1 nar_bytes_in=7524550worker2 # [ 69.367439] nix-grpc-daemon[780]: ts=2026-09-22T10:57:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324551worker2 # [ 69.373457] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4552worker2 # [ 69.377418] nix-daemon[866]: reaped child process 1002, status = succeeded4553client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4554worker2 # [ 69.388900] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4555client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4556client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4557client # farm-top> LOG-farm-top-sig4558worker2 # [ 70.813070] nix-daemon[866]: reaped child process 1016, status = succeeded4559worker2 # [ 70.982736] nix-daemon[866]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4560worker2 # [ 71.004368] nix-daemon[866]: reaped child process 1035, status = succeeded4561lb # [ 70.997289] niks3-server[1035]: 2026/09/22 10:58:01 INFO Received uploads request method=POST path=/api/pending_closures4562worker2 # [ 71.029937] nix-grpc-daemon[917]: time=2026-09-22T10:58:01.487Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4563worker2 # [ 71.031437] nix-grpc-daemon[917]: time=2026-09-22T10:58:01.487Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4564lb # [ 71.040331] niks3-server[1035]: 2026/09/22 10:58:01 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4565worker2 # [ 71.078923] nix-grpc-daemon[917]: time=2026-09-22T10:58:01.537Z level=INFO msg="Uploading 1 narinfos"4566lb # [ 71.051844] niks3-server[1035]: 2026/09/22 10:58:01 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4567lb # [ 71.063921] niks3-server[1035]: 2026/09/22 10:58:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4568lb # [ 71.071932] niks3-server[1035]: 2026/09/22 10:58:01 INFO Signed narinfos id=14 count=14569worker2 # [ 71.098924] nix-grpc-daemon[917]: time=2026-09-22T10:58:01.558Z level=INFO msg="Upload complete. (301ms)"4570lb # [ 71.079707] niks3-server[1035]: 2026/09/22 10:58:01 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4571worker2 # [ 71.105575] nix-grpc-daemon[780]: ts=2026-09-22T10:58:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14572lb # [ 71.085284] niks3-server[1035]: 2026/09/22 10:58:01 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4573lb # [ 71.089785] niks3-server[1035]: 2026/09/22 10:58:01 INFO Completed upload id=144574lb # [ 71.092791] niks3-server[1035]: 2026/09/22 10:58:01 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4575worker2 # [ 71.115649] nix-daemon[866]: reaped child process 1014, status = succeeded4576client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag sig >&2, in 6.92 seconds)4577client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag sig outPath4578client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag sig outPath, in 0.24 seconds)4579client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&ca-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4580worker1 # [ 71.654196] nix-grpc-daemon[778]: ts=2026-09-22T10:58:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:522524581client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4582client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4583client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4584client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4585worker1 # [ 71.697063] nix-grpc-daemon[778]: ts=2026-09-22T10:58:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:522524586worker1 # [ 71.710418] nix-grpc-daemon[778]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4587worker1 # [ 71.729269] nix-grpc-daemon[778]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4588worker1 # [ 71.744163] nix-grpc-daemon[778]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4589client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&ca-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.36 seconds)4590client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&ca-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4591worker2 # [ 72.015683] nix-grpc-daemon[780]: ts=2026-09-22T10:58:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:438744592worker2 # [ 72.025122] nix-grpc-daemon[780]: ts=2026-09-22T10:58:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:438744593client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&ca-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.32 seconds)4594client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4595client # finding garbage collector roots...4596client # removing stale temporary roots file "/nix/var/nix/temproots/877"4597client # removing stale temporary roots file "/nix/var/nix/temproots/856"4598client # removing stale temporary roots file "/nix/var/nix/temproots/844"4599client # removing stale temporary roots file "/nix/var/nix/temproots/823"4600client # removing stale temporary roots file "/nix/var/nix/temproots/885"4601client # removing stale temporary roots file "/nix/var/nix/temproots/900"4602client # removing stale temporary roots file "/nix/var/nix/temproots/864"4603client # removing stale temporary roots file "/nix/var/nix/temproots/808"4604client # deleting specified paths...4605client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4606client # deleting unused links...4607client # note: hard linking is currently saving 0.0 KiB4608worker1 # [ 72.581550] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:413384609worker2 # [ 72.579210] nix-grpc-daemon[780]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401324610worker2 # [ 72.591357] nix-grpc-daemon[917]: time=2026-09-22T10:58:03.049Z level=INFO msg="All 1 paths already cached"4611worker1 # [ 72.618420] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413384612worker1 # [ 72.628303] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413384613client # copying 3 paths...4614worker1 # [ 72.643479] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413384615client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4616worker1 # [ 72.682041] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413384617worker1 # [ 72.690344] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_out=2324618worker1 # [ 72.698243] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413384619worker1 # [ 72.705113] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_out=1764620worker1 # [ 72.714741] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413384621worker1 # [ 72.721972] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_out=1284622client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4623client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4624client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.71 seconds)4625(finished: subtest: outputs read back through any worker carry the cache signature, in 8.55 seconds)4626subtest: two clients wanting the same drv build it once4627client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4628client # [ 72.831018] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4629client # Running as unit: dedup1.service; invocation ID: eb3f46277fbf44c4aa764ebb416a456e4630client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.07 seconds)4631client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4632worker1 # [ 73.136140] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:413384633client # [ 73.094348] nix[953]: don't know how to build these paths:4634client # [ 73.095328] nix[953]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4635worker2 # [ 73.137445] nix-grpc-daemon[780]: ts=2026-09-22T10:58:03Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04636worker2 # [ 73.143882] nix-grpc-daemon[780]: ts=2026-09-22T10:58:03Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14637worker1 # [ 73.201643] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384638worker1 # [ 73.217312] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384639client # [ 73.180417] nix[953]: copying 1 paths to 'lb:50051'...4640worker1 # [ 73.230568] nix-grpc-daemon[778]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413384641client # [ 73.184194] nix[953]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4642worker1 # [ 73.249624] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4643worker1 # [ 73.469228] nix-daemon[844]: accepted connection from pid 1026, user nix-grpc-daemon (trusted)4644worker1 # [ 73.489643] nix-daemon[844]: reaped child process 1029, status = succeeded4645worker1 # [ 73.673386] nix-daemon[844]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4646worker1 # [ 73.692473] nix-daemon[844]: reaped child process 1034, status = succeeded4647lb # [ 73.669020] niks3-server[1035]: 2026/09/22 10:58:04 INFO Received uploads request method=POST path=/api/pending_closures4648worker1 # [ 73.719587] nix-grpc-daemon[857]: time=2026-09-22T10:58:04.190Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4649worker1 # [ 73.721385] nix-grpc-daemon[857]: time=2026-09-22T10:58:04.190Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4650lb # [ 73.702619] niks3-server[1035]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4651lb # [ 73.720116] niks3-server[1035]: 2026/09/22 10:58:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4652worker1 # [ 73.762893] nix-grpc-daemon[857]: time=2026-09-22T10:58:04.233Z level=INFO msg="Uploading 1 narinfos"4653lb # [ 73.733520] niks3-server[1035]: 2026/09/22 10:58:04 INFO Signed narinfos id=15 count=14654lb # [ 73.740511] niks3-server[1035]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4655lb # [ 73.750362] niks3-server[1035]: 2026/09/22 10:58:04 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4656lb # [ 73.755178] niks3-server[1035]: 2026/09/22 10:58:04 INFO Completed upload id=154657worker1 # [ 73.791130] nix-grpc-daemon[857]: time=2026-09-22T10:58:04.262Z level=INFO msg="Upload complete. (523ms)"4658lb # [ 73.758147] niks3-server[1035]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4659worker1 # [ 73.801112] nix-grpc-daemon[778]: ts=2026-09-22T10:58:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_in=4324660worker1 # [ 73.820009] nix-grpc-daemon[778]: ts=2026-09-22T10:58:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384661worker1 # [ 73.827313] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4662worker1 # [ 73.835705] nix-daemon[844]: reaped child process 1024, status = succeeded4663client # [ 73.790534] nix[953]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4664worker1 # [ 73.844181] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4665client # [ 73.937905] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4666client # Running as unit: dedup2.service; invocation ID: 66ec79723e234ad58d1631bf24ca34224667client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4668lb # [ 74.196993] envoy[933]: conn peer=[2001:db8:1::1]:60154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9545 tx=4389 ms=66084669lb # [ 74.199666] envoy[933]: conn peer=[2001:db8:1::1]:44412 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2885 ms=2124670lb # [ 74.202298] envoy[933]: conn peer=[2001:db8:1::1]:44438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=814671lb # [ 74.204707] envoy[933]: conn peer=[2001:db8:1::1]:44432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=994672lb # [ 74.207064] envoy[933]: conn peer=[2001:db8:1::1]:44418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=1204673worker2 # [ 74.307165] nix-grpc-daemon[780]: ts=2026-09-22T10:58:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401324674client # [ 74.282082] nix[969]: don't know how to build these paths:4675client # [ 74.284384] nix[969]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4676worker2 # [ 74.327289] nix-grpc-daemon[780]: ts=2026-09-22T10:58:04Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04677worker1 # [ 74.350869] nix-grpc-daemon[778]: ts=2026-09-22T10:58:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384678client # [ 74.309370] nix[969]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4679client: must succeed: sleep 24680client: (finished: must succeed: sleep 2, in 2.07 seconds)4681client: waiting for success: ! systemctl is-active dedup1 dedup24682worker1 # [ 77.691680] nix-daemon[844]: reaped child process 1038, status = succeeded4683worker1 # [ 77.876872] nix-daemon[844]: accepted connection from pid 1068, user nix-grpc-daemon (trusted)4684worker1 # [ 77.900187] nix-daemon[844]: reaped child process 1071, status = succeeded4685lb # [ 77.875401] niks3-server[1035]: 2026/09/22 10:58:08 INFO Received uploads request method=POST path=/api/pending_closures4686worker1 # [ 77.928260] nix-grpc-daemon[857]: time=2026-09-22T10:58:08.395Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4687worker1 # [ 77.929820] nix-grpc-daemon[857]: time=2026-09-22T10:58:08.395Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4688lb # [ 77.924467] niks3-server[1035]: 2026/09/22 10:58:08 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4689lb # [ 77.937981] niks3-server[1035]: 2026/09/22 10:58:08 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4690worker1 # [ 77.989109] nix-grpc-daemon[857]: time=2026-09-22T10:58:08.460Z level=INFO msg="Uploading 1 narinfos"4691lb # [ 77.947341] niks3-server[1035]: 2026/09/22 10:58:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4692lb # [ 77.957321] niks3-server[1035]: 2026/09/22 10:58:08 INFO Signed narinfos id=16 count=14693lb # [ 77.959479] niks3-server[1035]: 2026/09/22 10:58:08 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4694lb # [ 77.972158] niks3-server[1035]: 2026/09/22 10:58:08 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4695worker1 # [ 78.013485] nix-grpc-daemon[857]: time=2026-09-22T10:58:08.484Z level=INFO msg="Upload complete. (355ms)"4696lb # [ 77.977730] niks3-server[1035]: 2026/09/22 10:58:08 INFO Completed upload id=164697lb # [ 77.980399] niks3-server[1035]: 2026/09/22 10:58:08 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4698worker1 # [ 78.020605] nix-grpc-daemon[778]: ts=2026-09-22T10:58:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14699worker1 # [ 78.032209] nix-grpc-daemon[778]: ts=2026-09-22T10:58:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14700worker1 # [ 78.036087] nix-daemon[844]: reaped child process 1036, status = succeeded4701client # [ 78.032883] systemd[1]: dedup1.service: Deactivated successfully.4702client # [ 78.040259] systemd[1]: dedup1.service: Consumed 174ms CPU time over 5.207s wall clock time, 39.3M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4703client # [ 78.077819] systemd[1]: dedup2.service: Deactivated successfully.4704client # [ 78.080618] systemd[1]: dedup2.service: Consumed 204ms CPU time over 4.143s wall clock time, 33.2M memory peak, 6.1K incoming IP traffic, 5.6K outgoing IP traffic.4705client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.17 seconds)4706client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4707client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4708worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4709worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.09 seconds)4710worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4711worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.09 seconds)4712worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4713worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4714worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4715worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.11 seconds)4716(finished: subtest: two clients wanting the same drv build it once, in 6.38 seconds)4717subtest: --timeout and --max-silent-time reach the builder4718worker1 # [ 79.558486] nix-grpc-daemon[778]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:413384719worker2 # [ 79.563749] nix-grpc-daemon[780]: ts=2026-09-22T10:58:10Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04720worker2 # [ 79.569855] nix-grpc-daemon[780]: ts=2026-09-22T10:58:10Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14721worker2 # [ 79.610385] nix-grpc-daemon[780]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324722worker2 # [ 79.633619] nix-grpc-daemon[780]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401324723worker2 # [ 79.651751] nix-grpc-daemon[780]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401324724worker2 # [ 79.673280] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4725worker2 # [ 79.942605] nix-daemon[866]: accepted connection from pid 1066, user nix-grpc-daemon (trusted)4726worker2 # [ 79.964192] nix-daemon[866]: reaped child process 1069, status = succeeded4727worker2 # [ 80.169714] nix-daemon[866]: accepted connection from pid 1071, user nix-grpc-daemon (trusted)4728worker2 # [ 80.192911] nix-daemon[866]: reaped child process 1074, status = succeeded4729lb # [ 80.194557] niks3-server[1035]: 2026/09/22 10:58:10 INFO Received uploads request method=POST path=/api/pending_closures4730worker2 # [ 80.224838] nix-grpc-daemon[917]: time=2026-09-22T10:58:10.683Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4731worker2 # [ 80.226374] nix-grpc-daemon[917]: time=2026-09-22T10:58:10.683Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4732lb # [ 80.234808] niks3-server[1035]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4733worker2 # [ 80.282009] nix-grpc-daemon[917]: time=2026-09-22T10:58:10.740Z level=INFO msg="Uploading 1 narinfos"4734lb # [ 80.255545] niks3-server[1035]: 2026/09/22 10:58:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4735lb # [ 80.270359] niks3-server[1035]: 2026/09/22 10:58:10 INFO Signed narinfos id=17 count=14736lb # [ 80.275959] niks3-server[1035]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4737lb # [ 80.286849] niks3-server[1035]: 2026/09/22 10:58:10 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4738worker2 # [ 80.312875] nix-grpc-daemon[917]: time=2026-09-22T10:58:10.772Z level=INFO msg="Upload complete. (617ms)"4739lb # [ 80.296263] niks3-server[1035]: 2026/09/22 10:58:10 INFO Completed upload id=174740lb # [ 80.299183] niks3-server[1035]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4741worker2 # [ 80.324473] nix-grpc-daemon[780]: ts=2026-09-22T10:58:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=0 paths=1 nar_bytes_in=4244742worker2 # [ 80.340835] nix-grpc-daemon[780]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324743worker2 # [ 80.360769] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4744worker2 # [ 80.368882] nix-daemon[866]: reaped child process 1064, status = succeeded4745worker2 # [ 80.390693] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4746lb # [ 84.203542] envoy[933]: conn peer=[2001:db8:1::1]:44452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1562 ms=49034747lb # [ 84.209407] envoy[933]: conn peer=[2001:db8:1::1]:44462 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1194 ms=37704748worker2 # [ 86.418425] nix-grpc-daemon[780]: ts=2026-09-22T10:58:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04749worker2 # [ 86.449580] nix-daemon[866]: reaped child process 1076, status = succeeded4750worker2 # [ 86.451758] nix-daemon[866]: reaped child process 1078, status = succeeded4751worker2 # [ 86.847879] nix-grpc-daemon[780]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401324752worker2 # [ 86.868926] nix-grpc-daemon[780]: ts=2026-09-22T10:58:17Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04753worker2 # [ 86.875881] nix-grpc-daemon[780]: ts=2026-09-22T10:58:17Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14754worker1 # [ 86.905158] nix-grpc-daemon[778]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384755worker1 # [ 86.931179] nix-grpc-daemon[778]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384756worker1 # [ 86.953470] nix-grpc-daemon[778]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413384757worker1 # [ 86.982696] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4758worker1 # [ 87.248873] nix-daemon[844]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)4759worker1 # [ 87.273321] nix-daemon[844]: reaped child process 1087, status = succeeded4760worker1 # [ 87.478574] nix-daemon[844]: accepted connection from pid 1089, user nix-grpc-daemon (trusted)4761worker1 # [ 87.498779] nix-daemon[844]: reaped child process 1092, status = succeeded4762lb # [ 87.482466] niks3-server[1035]: 2026/09/22 10:58:17 INFO Received uploads request method=POST path=/api/pending_closures4763worker1 # [ 87.533168] nix-grpc-daemon[857]: time=2026-09-22T10:58:18.003Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4764worker1 # [ 87.534694] nix-grpc-daemon[857]: time=2026-09-22T10:58:18.003Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4765lb # [ 87.535742] niks3-server[1035]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4766lb # [ 87.550973] niks3-server[1035]: 2026/09/22 10:58:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4767worker1 # [ 87.593921] nix-grpc-daemon[857]: time=2026-09-22T10:58:18.064Z level=INFO msg="Uploading 1 narinfos"4768lb # [ 87.561213] niks3-server[1035]: 2026/09/22 10:58:18 INFO Signed narinfos id=18 count=14769lb # [ 87.563402] niks3-server[1035]: 2026/09/22 10:58:18 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4770lb # [ 87.574623] niks3-server[1035]: 2026/09/22 10:58:18 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4771lb # [ 87.585267] niks3-server[1035]: 2026/09/22 10:58:18 INFO Completed upload id=184772worker1 # [ 87.622235] nix-grpc-daemon[857]: time=2026-09-22T10:58:18.093Z level=INFO msg="Upload complete. (611ms)"4773lb # [ 87.589700] niks3-server[1035]: 2026/09/22 10:58:18 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4774worker1 # [ 87.637123] nix-grpc-daemon[778]: ts=2026-09-22T10:58:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_in=4244775worker1 # [ 87.656993] nix-grpc-daemon[778]: ts=2026-09-22T10:58:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384776worker1 # [ 87.666055] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4777worker1 # [ 87.673958] nix-daemon[844]: reaped child process 1082, status = succeeded4778worker1 # [ 87.691998] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4779worker1 # [ 93.912838] nix-grpc-daemon[778]: ts=2026-09-22T10:58:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04780worker1 # [ 93.955733] nix-daemon[844]: reaped child process 1094, status = succeeded4781worker1 # [ 93.958693] nix-daemon[844]: reaped child process 1096, status = succeeded4782(finished: subtest: --timeout and --max-silent-time reach the builder, in 14.91 seconds)4783subtest: the first client leaving does not fail the second4784client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share4785client # [ 94.151382] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share.4786client # Running as unit: share1.service; invocation ID: 04ecdfc3afc2448b8f6f40fe5ea3fd514787client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share, in 0.11 seconds)4788client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share4789lb # [ 94.204607] envoy[933]: conn peer=[2001:db8:1::1]:44468 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=995 ms=69474790client # [ 94.431405] nix[1042]: don't know how to build these paths:4791client # [ 94.433145] nix[1042]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4792worker2 # [ 94.462454] nix-grpc-daemon[780]: ts=2026-09-22T10:58:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401324793worker2 # [ 94.481512] nix-grpc-daemon[780]: ts=2026-09-22T10:58:24Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04794worker2 # [ 94.490156] nix-grpc-daemon[780]: ts=2026-09-22T10:58:24Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14795worker2 # [ 94.501769] nix-grpc-daemon[780]: ts=2026-09-22T10:58:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324796worker2 # [ 94.514975] nix-grpc-daemon[780]: ts=2026-09-22T10:58:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401324797client # [ 94.496160] nix[1042]: copying 1 paths to 'lb:50051'...4798client # [ 94.497045] nix[1042]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4799worker2 # [ 94.530549] nix-grpc-daemon[780]: ts=2026-09-22T10:58:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401324800worker2 # [ 94.549648] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4801worker2 # [ 94.772826] nix-daemon[866]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)4802worker2 # [ 94.796447] nix-daemon[866]: reaped child process 1108, status = succeeded4803lb # [ 94.210195] envoy[933]: conn peer=[2001:db8:1::1]:56592 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2976 tx=953 ms=70784804worker2 # [ 94.975638] nix-daemon[866]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)4805worker2 # [ 94.995580] nix-daemon[866]: reaped child process 1113, status = succeeded4806lb # [ 94.993170] niks3-server[1035]: 2026/09/22 10:58:25 INFO Received uploads request method=POST path=/api/pending_closures4807worker2 # [ 95.022939] nix-grpc-daemon[917]: time=2026-09-22T10:58:25.481Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4808worker2 # [ 95.024626] nix-grpc-daemon[917]: time=2026-09-22T10:58:25.481Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4809lb # [ 95.026351] niks3-server[1035]: 2026/09/22 10:58:25 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4810worker2 # [ 95.061413] nix-grpc-daemon[917]: time=2026-09-22T10:58:25.520Z level=INFO msg="Uploading 1 narinfos"4811lb # [ 95.041724] niks3-server[1035]: 2026/09/22 10:58:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4812lb # [ 95.053452] niks3-server[1035]: 2026/09/22 10:58:25 INFO Signed narinfos id=19 count=14813lb # [ 95.060643] niks3-server[1035]: 2026/09/22 10:58:25 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4814lb # [ 95.069745] niks3-server[1035]: 2026/09/22 10:58:25 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4815lb # [ 95.076647] niks3-server[1035]: 2026/09/22 10:58:25 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4816lb # [ 95.082609] niks3-server[1035]: 2026/09/22 10:58:25 INFO Completed upload id=194817worker2 # [ 95.102481] nix-grpc-daemon[917]: time=2026-09-22T10:58:25.561Z level=INFO msg="Upload complete. (533ms)"4818worker2 # [ 95.108306] nix-grpc-daemon[780]: ts=2026-09-22T10:58:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=0 paths=1 nar_bytes_in=4324819worker2 # [ 95.116651] nix-grpc-daemon[780]: ts=2026-09-22T10:58:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324820worker2 # [ 95.125933] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4821worker2 # [ 95.130280] nix-daemon[866]: reaped child process 1103, status = succeeded4822worker2 # [ 95.139761] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4823client # [ 95.106438] nix[1042]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4824client # [ 95.271975] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share.4825client # Running as unit: share2.service; invocation ID: a05c60e8f9b84ff8910216599d78a8184826client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share, in 1.13 seconds)4827worker1 # [ 95.610571] nix-grpc-daemon[778]: ts=2026-09-22T10:58:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:413384828client # [ 95.569057] nix[1058]: don't know how to build these paths:4829client # [ 95.569979] nix[1058]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4830worker2 # [ 95.615159] nix-grpc-daemon[780]: ts=2026-09-22T10:58:26Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04831client # [ 95.591531] nix[1058]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4832worker2 # [ 95.623862] nix-grpc-daemon[780]: ts=2026-09-22T10:58:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401324833client: must succeed: sleep 2; systemctl kill -s INT share14834client # [ 98.722610] systemd[1]: share1.service: Sent signal SIGINT to main process 1042 (nix) on client request.4835client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.10 seconds)4836client: waiting for success: ! systemctl is-active share14837client # [ 98.734908] nix[1042]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4838client # [ 98.739503] nix[1042]: error: interrupted by the user4839client # [ 98.794247] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4840client # [ 98.802640] systemd[1]: share1.service: Failed with result 'exit-code'.4841client # [ 98.804775] systemd[1]: share1.service: Consumed 183ms CPU time over 4.643s wall clock time, 41.2M memory peak, 6.1K incoming IP traffic, 8.2K outgoing IP traffic.4842client: (finished: waiting for success: ! systemctl is-active share1, in 0.09 seconds)4843client: waiting for success: ! systemctl is-active share24844worker2 # [ 99.370815] nix-daemon[866]: reaped child process 1117, status = succeeded4845worker2 # [ 99.544917] nix-daemon[866]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)4846worker2 # [ 99.566984] nix-daemon[866]: reaped child process 1156, status = succeeded4847lb # [ 99.561024] niks3-server[1035]: 2026/09/22 10:58:30 INFO Received uploads request method=POST path=/api/pending_closures4848worker2 # [ 99.595638] nix-grpc-daemon[917]: time=2026-09-22T10:58:30.051Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4849worker2 # [ 99.597701] nix-grpc-daemon[917]: time=2026-09-22T10:58:30.051Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4850lb # [ 99.602280] niks3-server[1035]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4851lb # [ 99.618040] niks3-server[1035]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4852lb # [ 99.631237] niks3-server[1035]: 2026/09/22 10:58:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4853worker2 # [ 99.657497] nix-grpc-daemon[917]: time=2026-09-22T10:58:30.115Z level=INFO msg="Uploading 1 narinfos"4854lb # [ 99.637779] niks3-server[1035]: 2026/09/22 10:58:30 INFO Signed narinfos id=20 count=14855lb # [ 99.643043] niks3-server[1035]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4856lb # [ 99.655215] niks3-server[1035]: 2026/09/22 10:58:30 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4857worker2 # [ 99.680640] nix-grpc-daemon[917]: time=2026-09-22T10:58:30.139Z level=INFO msg="Upload complete. (327ms)"4858lb # [ 99.662243] niks3-server[1035]: 2026/09/22 10:58:30 INFO Completed upload id=204859lb # [ 99.665157] niks3-server[1035]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4860worker2 # [ 99.697344] nix-grpc-daemon[780]: ts=2026-09-22T10:58:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14861worker2 # [ 99.712514] nix-grpc-daemon[780]: ts=2026-09-22T10:58:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14862worker2 # [ 99.721064] nix-daemon[866]: reaped child process 1115, status = succeeded4863client # [ 99.748564] systemd[1]: share2.service: Deactivated successfully.4864client # [ 99.752932] systemd[1]: share2.service: Consumed 189ms CPU time over 4.480s wall clock time, 35.2M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4865client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4866client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4867client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds)4868(finished: subtest: the first client leaving does not fail the second, in 6.20 seconds)4869subtest: build hook: nix-daemon with builders = grpc://lb4870client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4871client # [ 100.460519] systemd[1]: Started Nix Daemon.4872client # [ 100.650282] nix-daemon[1105]: accepted connection from pid 1102, user root (trusted)4873worker2 # [ 100.898247] nix-grpc-daemon[780]: ts=2026-09-22T10:58:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401324874worker1 # [ 100.926865] nix-grpc-daemon[778]: ts=2026-09-22T10:58:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384875worker2 # [ 100.933231] nix-grpc-daemon[780]: ts=2026-09-22T10:58:31Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04876worker2 # [ 100.939863] nix-grpc-daemon[780]: ts=2026-09-22T10:58:31Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14877worker1 # [ 100.960507] nix-grpc-daemon[778]: ts=2026-09-22T10:58:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384878worker1 # [ 100.978508] nix-grpc-daemon[778]: ts=2026-09-22T10:58:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384879worker1 # [ 100.994810] nix-grpc-daemon[778]: ts=2026-09-22T10:58:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413384880worker1 # [ 101.013202] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4881worker1 # [ 101.220386] nix-daemon[844]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)4882worker1 # [ 101.241686] nix-daemon[844]: reaped child process 1138, status = succeeded4883worker1 # [ 101.414448] nix-daemon[844]: accepted connection from pid 1140, user nix-grpc-daemon (trusted)4884worker1 # [ 101.433148] nix-daemon[844]: reaped child process 1143, status = succeeded4885lb # [ 101.407575] niks3-server[1035]: 2026/09/22 10:58:31 INFO Received uploads request method=POST path=/api/pending_closures4886worker1 # [ 101.450567] nix-grpc-daemon[857]: time=2026-09-22T10:58:31.921Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4887worker1 # [ 101.452253] nix-grpc-daemon[857]: time=2026-09-22T10:58:31.921Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4888lb # [ 101.429138] niks3-server[1035]: 2026/09/22 10:58:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4889worker1 # [ 101.470561] nix-grpc-daemon[857]: time=2026-09-22T10:58:31.941Z level=INFO msg="Uploading 1 narinfos"4890lb # [ 101.436259] niks3-server[1035]: 2026/09/22 10:58:31 INFO Signed narinfos id=21 count=14891lb # [ 101.439913] niks3-server[1035]: 2026/09/22 10:58:31 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4892lb # [ 101.444733] niks3-server[1035]: 2026/09/22 10:58:31 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4893lb # [ 101.451262] niks3-server[1035]: 2026/09/22 10:58:31 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4894lb # [ 101.455514] niks3-server[1035]: 2026/09/22 10:58:31 INFO Completed upload id=214895worker1 # [ 101.491820] nix-grpc-daemon[857]: time=2026-09-22T10:58:31.963Z level=INFO msg="Upload complete. (460ms)"4896lb # [ 101.457516] niks3-server[1035]: 2026/09/22 10:58:31 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4897worker1 # [ 101.498135] nix-grpc-daemon[778]: ts=2026-09-22T10:58:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_in=5124898worker1 # [ 101.504365] nix-grpc-daemon[778]: ts=2026-09-22T10:58:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384899worker1 # [ 101.514928] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4900worker1 # [ 101.516639] nix-daemon[844]: reaped child process 1133, status = succeeded4901worker1 # [ 101.527272] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4902worker1 # [ 102.968209] nix-daemon[844]: reaped child process 1147, status = succeeded4903worker1 # [ 103.152377] nix-daemon[844]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)4904worker1 # [ 103.170409] nix-daemon[844]: reaped child process 1166, status = succeeded4905lb # [ 103.146848] niks3-server[1035]: 2026/09/22 10:58:33 INFO Received uploads request method=POST path=/api/pending_closures4906worker1 # [ 103.192420] nix-grpc-daemon[857]: time=2026-09-22T10:58:33.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4907worker1 # [ 103.193957] nix-grpc-daemon[857]: time=2026-09-22T10:58:33.661Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4908lb # [ 103.178070] niks3-server[1035]: 2026/09/22 10:58:33 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4909lb # [ 103.185249] niks3-server[1035]: 2026/09/22 10:58:33 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4910lb # [ 103.191925] niks3-server[1035]: 2026/09/22 10:58:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4911worker1 # [ 103.232564] nix-grpc-daemon[857]: time=2026-09-22T10:58:33.703Z level=INFO msg="Uploading 1 narinfos"4912lb # [ 103.197970] niks3-server[1035]: 2026/09/22 10:58:33 INFO Signed narinfos id=22 count=14913lb # [ 103.201348] niks3-server[1035]: 2026/09/22 10:58:33 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4914lb # [ 103.208980] niks3-server[1035]: 2026/09/22 10:58:33 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4915lb # [ 103.212988] niks3-server[1035]: 2026/09/22 10:58:33 INFO Completed upload id=224916worker1 # [ 103.249232] nix-grpc-daemon[857]: time=2026-09-22T10:58:33.720Z level=INFO msg="Upload complete. (312ms)"4917lb # [ 103.215225] niks3-server[1035]: 2026/09/22 10:58:33 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4918worker1 # [ 103.260845] nix-grpc-daemon[778]: ts=2026-09-22T10:58:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14919worker1 # [ 103.274388] nix-grpc-daemon[778]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413384920worker1 # [ 103.277985] nix-daemon[844]: reaped child process 1145, status = succeeded4921worker1 # [ 103.291226] nix-grpc-daemon[778]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413384922worker1 # [ 103.296364] nix-grpc-daemon[778]: ts=2026-09-22T10:58:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_out=1284923worker2 # [ 103.579122] nix-grpc-daemon[780]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401324924worker2 # [ 103.586346] nix-grpc-daemon[780]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401324925worker2 # [ 103.614664] nix-grpc-daemon[780]: ts=2026-09-22T10:58:34Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04926worker1 # [ 103.634306] nix-grpc-daemon[778]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384927worker2 # [ 103.620716] nix-grpc-daemon[780]: ts=2026-09-22T10:58:34Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14928worker1 # [ 103.645319] nix-grpc-daemon[778]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384929worker1 # [ 103.656187] nix-grpc-daemon[857]: time=2026-09-22T10:58:34.127Z level=INFO msg="All 1 paths already cached"4930worker1 # [ 103.664067] nix-grpc-daemon[778]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413384931worker1 # [ 103.678283] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4932worker1 # [ 103.897230] nix-daemon[844]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)4933worker1 # [ 103.916544] nix-daemon[844]: reaped child process 1173, status = succeeded4934worker1 # [ 104.095588] nix-daemon[844]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)4935worker1 # [ 104.113566] nix-daemon[844]: reaped child process 1178, status = succeeded4936lb # [ 104.091402] niks3-server[1035]: 2026/09/22 10:58:34 INFO Received uploads request method=POST path=/api/pending_closures4937worker1 # [ 104.139115] nix-grpc-daemon[857]: time=2026-09-22T10:58:34.609Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4938worker1 # [ 104.142080] nix-grpc-daemon[857]: time=2026-09-22T10:58:34.610Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4939lb # [ 104.120668] niks3-server[1035]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4940lb # [ 104.135757] niks3-server[1035]: 2026/09/22 10:58:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4941worker1 # [ 104.178457] nix-grpc-daemon[857]: time=2026-09-22T10:58:34.649Z level=INFO msg="Uploading 1 narinfos"4942lb # [ 104.144974] niks3-server[1035]: 2026/09/22 10:58:34 INFO Signed narinfos id=23 count=14943lb # [ 104.157107] niks3-server[1035]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4944lb # [ 104.167681] niks3-server[1035]: 2026/09/22 10:58:34 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4945lb # [ 104.176183] niks3-server[1035]: 2026/09/22 10:58:34 INFO Completed upload id=234946worker1 # [ 104.213581] nix-grpc-daemon[857]: time=2026-09-22T10:58:34.684Z level=INFO msg="Upload complete. (517ms)"4947lb # [ 104.178941] niks3-server[1035]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4948worker1 # [ 104.221889] nix-grpc-daemon[778]: ts=2026-09-22T10:58:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_in=6244949worker1 # [ 104.235996] nix-grpc-daemon[778]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384950lb # [ 104.203261] envoy[933]: conn peer=[2001:db8:1::1]:42174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2990 tx=814 ms=43304951worker1 # [ 104.243328] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4952lb # [ 104.208420] envoy[933]: conn peer=[2001:db8:1::1]:42180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1198 ms=41614953worker1 # [ 104.248768] nix-daemon[844]: reaped child process 1168, status = succeeded4954lb # [ 104.213507] envoy[933]: conn peer=[2001:db8:1::1]:39802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1942 ms=24414955lb # [ 104.219260] envoy[933]: conn peer=[2001:db8:1::1]:39812 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=634956worker1 # [ 104.263445] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4957worker1 # [ 105.686255] nix-daemon[844]: reaped child process 1182, status = succeeded4958worker1 # [ 105.845947] nix-daemon[844]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)4959worker1 # [ 105.866536] nix-daemon[844]: reaped child process 1201, status = succeeded4960lb # [ 105.843254] niks3-server[1035]: 2026/09/22 10:58:36 INFO Received uploads request method=POST path=/api/pending_closures4961worker1 # [ 105.891287] nix-grpc-daemon[857]: time=2026-09-22T10:58:36.359Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4962worker1 # [ 105.893060] nix-grpc-daemon[857]: time=2026-09-22T10:58:36.359Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4963lb # [ 105.886722] niks3-server[1035]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4964lb # [ 105.898849] niks3-server[1035]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4965lb # [ 105.907483] niks3-server[1035]: 2026/09/22 10:58:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4966worker1 # [ 105.949368] nix-grpc-daemon[857]: time=2026-09-22T10:58:36.420Z level=INFO msg="Uploading 1 narinfos"4967lb # [ 105.915421] niks3-server[1035]: 2026/09/22 10:58:36 INFO Signed narinfos id=24 count=14968lb # [ 105.920666] niks3-server[1035]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4969lb # [ 105.931314] niks3-server[1035]: 2026/09/22 10:58:36 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4970lb # [ 105.937156] niks3-server[1035]: 2026/09/22 10:58:36 INFO Completed upload id=244971worker1 # [ 105.972929] nix-grpc-daemon[857]: time=2026-09-22T10:58:36.444Z level=INFO msg="Upload complete. (299ms)"4972lb # [ 105.939745] niks3-server[1035]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4973worker1 # [ 105.985111] nix-grpc-daemon[778]: ts=2026-09-22T10:58:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14974worker1 # [ 106.002653] nix-grpc-daemon[778]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413384975worker1 # [ 106.008162] nix-daemon[844]: reaped child process 1180, status = succeeded4976worker2 # [ 106.013362] nix-grpc-daemon[780]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:401324977worker2 # [ 106.029535] nix-grpc-daemon[780]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4978worker2 # [ 106.049145] nix-grpc-daemon[780]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4979worker2 # [ 106.067803] nix-grpc-daemon[780]: ts=2026-09-22T10:58:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=0 paths=1 nar_bytes_out=1764980worker2 # [ 106.376982] nix-grpc-daemon[780]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401324981worker1 # [ 106.398632] nix-grpc-daemon[778]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384982worker1 # [ 106.409082] nix-grpc-daemon[857]: time=2026-09-22T10:58:36.880Z level=INFO msg="All 2 paths already cached"4983worker2 # [ 106.408859] nix-grpc-daemon[780]: ts=2026-09-22T10:58:36Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04984worker2 # [ 106.411892] nix-grpc-daemon[780]: ts=2026-09-22T10:58:36Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14985worker1 # [ 106.473520] nix-grpc-daemon[778]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413384986worker1 # [ 106.489348] nix-grpc-daemon[778]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413384987worker1 # [ 106.506707] nix-grpc-daemon[857]: time=2026-09-22T10:58:36.977Z level=INFO msg="All 2 paths already cached"4988worker1 # [ 106.516719] nix-grpc-daemon[778]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413384989worker1 # [ 106.531446] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4990worker1 # [ 106.732591] nix-daemon[844]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)4991worker1 # [ 106.753295] nix-daemon[844]: reaped child process 1208, status = succeeded4992worker1 # [ 106.931877] nix-daemon[844]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)4993worker1 # [ 106.949924] nix-daemon[844]: reaped child process 1213, status = succeeded4994lb # [ 106.927002] niks3-server[1035]: 2026/09/22 10:58:37 INFO Received uploads request method=POST path=/api/pending_closures4995worker1 # [ 106.970348] nix-grpc-daemon[857]: time=2026-09-22T10:58:37.440Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4996worker1 # [ 106.972057] nix-grpc-daemon[857]: time=2026-09-22T10:58:37.440Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4997lb # [ 106.946365] niks3-server[1035]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4998lb # [ 106.955336] niks3-server[1035]: 2026/09/22 10:58:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4999worker1 # [ 106.995488] nix-grpc-daemon[857]: time=2026-09-22T10:58:37.466Z level=INFO msg="Uploading 1 narinfos"5000lb # [ 106.962592] niks3-server[1035]: 2026/09/22 10:58:37 INFO Signed narinfos id=25 count=15001lb # [ 106.966350] niks3-server[1035]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls5002lb # [ 106.972384] niks3-server[1035]: 2026/09/22 10:58:37 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5003worker1 # [ 107.014904] nix-grpc-daemon[857]: time=2026-09-22T10:58:37.486Z level=INFO msg="Upload complete. (465ms)"5004lb # [ 106.978994] niks3-server[1035]: 2026/09/22 10:58:37 INFO Completed upload id=255005lb # [ 106.983489] niks3-server[1035]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo5006worker1 # [ 107.024491] nix-grpc-daemon[778]: ts=2026-09-22T10:58:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_in=7525007worker1 # [ 107.027984] nix-grpc-daemon[778]: ts=2026-09-22T10:58:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413385008worker1 # [ 107.039281] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)5009worker1 # [ 107.040971] nix-daemon[844]: reaped child process 1203, status = succeeded5010worker1 # [ 107.051571] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)5011worker1 # [ 108.288549] nix-daemon[844]: reaped child process 1217, status = succeeded5012worker1 # [ 108.452808] nix-daemon[844]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)5013worker1 # [ 108.473145] nix-daemon[844]: reaped child process 1236, status = succeeded5014lb # [ 108.448993] niks3-server[1035]: 2026/09/22 10:58:38 INFO Received uploads request method=POST path=/api/pending_closures5015worker1 # [ 108.498738] nix-grpc-daemon[857]: time=2026-09-22T10:58:38.967Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5016worker1 # [ 108.500385] nix-grpc-daemon[857]: time=2026-09-22T10:58:38.967Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"5017lb # [ 108.489237] niks3-server[1035]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst5018lb # [ 108.502635] niks3-server[1035]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv5019worker1 # [ 108.548243] nix-grpc-daemon[857]: time=2026-09-22T10:58:39.019Z level=INFO msg="Uploading 1 narinfos"5020lb # [ 108.513155] niks3-server[1035]: 2026/09/22 10:58:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5021lb # [ 108.521623] niks3-server[1035]: 2026/09/22 10:58:38 INFO Signed narinfos id=26 count=15022lb # [ 108.526460] niks3-server[1035]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls5023lb # [ 108.534488] niks3-server[1035]: 2026/09/22 10:58:38 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5024worker1 # [ 108.571035] nix-grpc-daemon[857]: time=2026-09-22T10:58:39.042Z level=INFO msg="Upload complete. (310ms)"5025lb # [ 108.538074] niks3-server[1035]: 2026/09/22 10:58:38 INFO Completed upload id=265026lb # [ 108.539894] niks3-server[1035]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo5027worker1 # [ 108.581035] nix-grpc-daemon[778]: ts=2026-09-22T10:58:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=15028worker1 # [ 108.596893] nix-daemon[844]: reaped child process 1215, status = succeeded5029worker2 # [ 108.613780] nix-grpc-daemon[780]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:401325030worker2 # [ 108.628668] nix-grpc-daemon[780]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...5031worker1 # [ 108.690206] nix-grpc-daemon[778]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413385032worker1 # [ 108.696363] nix-grpc-daemon[778]: ts=2026-09-22T10:58:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_out=2325033client # [ 108.731343] nix-daemon[1105]: reaped child process 1107, status = succeeded5034client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.48 seconds)5035client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top5036client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.03 seconds)5037client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5038client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)5039client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5040client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5041(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.59 seconds)5042subtest: an input only one worker has still reaches the builder5043client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only5044client # this derivation will be built:5045client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv5046client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...5047client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.44 seconds)5048client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure5049client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)5050worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5051worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)5052worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only5053worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)5054client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&25055client # [ 110.032465] nix-daemon[1105]: accepted connection from pid 1188, user root (trusted)5056client # this derivation will be built:5057client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5058client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...5059worker1 # [ 110.301854] nix-grpc-daemon[778]: ts=2026-09-22T10:58:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:413385060worker1 # [ 110.308774] nix-grpc-daemon[778]: ts=2026-09-22T10:58:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413385061worker1 # [ 110.519415] nix-daemon[844]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5062worker1 # [ 110.536407] nix-daemon[844]: reaped child process 1246, status = succeeded5063lb # [ 110.514840] niks3-server[1035]: 2026/09/22 10:58:40 INFO Received uploads request method=POST path=/api/pending_closures5064worker1 # [ 110.561284] nix-grpc-daemon[857]: time=2026-09-22T10:58:41.031Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5065worker1 # [ 110.562734] nix-grpc-daemon[857]: time=2026-09-22T10:58:41.031Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"5066lb # [ 110.543260] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5067worker1 # [ 110.592938] nix-grpc-daemon[857]: time=2026-09-22T10:58:41.064Z level=INFO msg="Uploading 1 narinfos"5068lb # [ 110.558108] niks3-server[1035]: 2026/09/22 10:58:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5069lb # [ 110.568934] niks3-server[1035]: 2026/09/22 10:58:41 INFO Signed narinfos id=27 count=15070lb # [ 110.574882] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls5071worker1 # [ 110.619022] nix-grpc-daemon[857]: time=2026-09-22T10:58:41.090Z level=INFO msg="Upload complete. (305ms)"5072lb # [ 110.583925] niks3-server[1035]: 2026/09/22 10:58:41 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5073lb # [ 110.589795] niks3-server[1035]: 2026/09/22 10:58:41 INFO Completed upload id=275074lb # [ 110.592941] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo5075worker2 # [ 110.630431] nix-grpc-daemon[780]: ts=2026-09-22T10:58:41Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=05076worker2 # [ 110.638041] nix-grpc-daemon[780]: ts=2026-09-22T10:58:41Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15077worker2 # [ 110.645946] nix-grpc-daemon[780]: ts=2026-09-22T10:58:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401325078worker2 # [ 110.657926] nix-grpc-daemon[780]: ts=2026-09-22T10:58:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401325079client # copying 2 paths to 'lb:50051'...5080client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...5081worker2 # [ 110.678589] nix-grpc-daemon[780]: ts=2026-09-22T10:58:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401325082client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...5083worker2 # [ 110.694757] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5084worker2 # [ 110.900415] nix-daemon[866]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5085worker2 # [ 110.919858] nix-daemon[866]: reaped child process 1177, status = succeeded5086worker2 # [ 111.098353] nix-daemon[866]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)5087worker2 # [ 111.115505] nix-daemon[866]: reaped child process 1182, status = succeeded5088lb # [ 111.111129] niks3-server[1035]: 2026/09/22 10:58:41 INFO Received uploads request method=POST path=/api/pending_closures5089lb # [ 111.120945] niks3-server[1035]: 2026/09/22 10:58:41 INFO Received uploads request method=POST path=/api/pending_closures5090worker2 # [ 111.149790] nix-grpc-daemon[917]: time=2026-09-22T10:58:41.608Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5091worker2 # [ 111.151489] nix-grpc-daemon[917]: time=2026-09-22T10:58:41.610Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"5092worker2 # [ 111.154200] nix-grpc-daemon[917]: time=2026-09-22T10:58:41.612Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"5093lb # [ 111.163248] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst5094lb # [ 111.174784] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst5095lb # [ 111.187051] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls5096lb # [ 111.195038] niks3-server[1035]: 2026/09/22 10:58:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5097worker2 # [ 111.215446] nix-grpc-daemon[917]: time=2026-09-22T10:58:41.673Z level=INFO msg="Uploading 3 narinfos"5098lb # [ 111.200198] niks3-server[1035]: 2026/09/22 10:58:41 INFO Signed narinfos id=29 count=25099lb # [ 111.202535] niks3-server[1035]: 2026/09/22 10:58:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5100lb # [ 111.211044] niks3-server[1035]: 2026/09/22 10:58:41 INFO Signed narinfos id=28 count=15101lb # [ 111.215440] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls5102lb # [ 111.230107] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo5103lb # [ 111.236209] niks3-server[1035]: 2026/09/22 10:58:41 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5104lb # [ 111.241769] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5105lb # [ 111.245362] niks3-server[1035]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5106lb # [ 111.248515] niks3-server[1035]: 2026/09/22 10:58:41 INFO Completed upload id=285107lb # [ 111.251189] niks3-server[1035]: 2026/09/22 10:58:41 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5108lb # [ 111.254198] niks3-server[1035]: 2026/09/22 10:58:41 INFO Completed upload id=295109worker2 # [ 111.274045] nix-grpc-daemon[917]: time=2026-09-22T10:58:41.733Z level=INFO msg="Upload complete. (560ms)"5110worker2 # [ 111.285543] nix-grpc-daemon[780]: ts=2026-09-22T10:58:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=0 paths=2 nar_bytes_in=9845111worker2 # [ 111.300801] nix-grpc-daemon[780]: ts=2026-09-22T10:58:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401325112worker2 # [ 111.313405] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5113worker2 # [ 111.320396] nix-daemon[866]: reaped child process 1172, status = succeeded5114worker2 # [ 111.339764] nix-grpc-daemon[780]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...5115client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...5116client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5117worker2 # [ 111.363084] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5118client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5119worker2 # [ 112.388498] nix-daemon[866]: reaped child process 1187, status = succeeded5120worker2 # [ 112.543672] nix-daemon[866]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5121worker2 # [ 112.562114] nix-daemon[866]: reaped child process 1206, status = succeeded5122lb # [ 112.554731] niks3-server[1035]: 2026/09/22 10:58:43 INFO Received uploads request method=POST path=/api/pending_closures5123worker2 # [ 112.585720] nix-grpc-daemon[917]: time=2026-09-22T10:58:43.043Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5124lb # [ 112.584671] niks3-server[1035]: 2026/09/22 10:58:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5125worker2 # [ 112.615406] nix-grpc-daemon[917]: time=2026-09-22T10:58:43.074Z level=INFO msg="Uploading 1 narinfos"5126lb # [ 112.600691] niks3-server[1035]: 2026/09/22 10:58:43 INFO Signed narinfos id=30 count=15127lb # [ 112.603666] niks3-server[1035]: 2026/09/22 10:58:43 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5128lb # [ 112.614514] niks3-server[1035]: 2026/09/22 10:58:43 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5129lb # [ 112.621998] niks3-server[1035]: 2026/09/22 10:58:43 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5130lb # [ 112.627014] niks3-server[1035]: 2026/09/22 10:58:43 INFO Completed upload id=305131worker2 # [ 112.646979] nix-grpc-daemon[917]: time=2026-09-22T10:58:43.106Z level=INFO msg="Upload complete. (270ms)"5132lb # [ 112.629934] niks3-server[1035]: 2026/09/22 10:58:43 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5133worker2 # [ 112.655046] nix-grpc-daemon[780]: ts=2026-09-22T10:58:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15134worker2 # [ 112.670641] nix-daemon[866]: reaped child process 1184, status = succeeded5135client # copying 1 paths...5136worker2 # [ 112.705455] nix-grpc-daemon[780]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:401325137client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5138worker1 # [ 112.758386] nix-grpc-daemon[778]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413385139worker1 # [ 112.772138] nix-grpc-daemon[778]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...5140worker1 # [ 112.791264] nix-grpc-daemon[778]: ts=2026-09-22T10:58:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_out=1205141client # [ 112.876433] nix-daemon[1105]: reaped child process 1191, status = succeeded5142client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.09 seconds)5143client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&25144client # [ 113.095281] nix-daemon[1105]: accepted connection from pid 1210, user root (trusted)5145client # this derivation will be built:5146client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5147worker2 # [ 113.346629] nix-grpc-daemon[780]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401325148client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...5149worker1 # [ 113.368352] nix-grpc-daemon[778]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413385150worker1 # [ 113.381064] nix-grpc-daemon[857]: time=2026-09-22T10:58:43.851Z level=INFO msg="All 1 paths already cached"5151worker2 # [ 113.382862] nix-grpc-daemon[780]: ts=2026-09-22T10:58:43Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05152worker2 # [ 113.389377] nix-grpc-daemon[780]: ts=2026-09-22T10:58:43Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15153worker1 # [ 113.445038] nix-grpc-daemon[778]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413385154worker1 # [ 113.460676] nix-grpc-daemon[778]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413385155client # copying 1 paths to 'lb:50051'...5156client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5157worker1 # [ 113.482917] nix-grpc-daemon[778]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413385158worker1 # [ 113.498642] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)5159worker1 # [ 113.522297] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)5160worker1 # [ 113.596507] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 2/5)5161worker1 # [ 113.742216] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 282 ms (attempt 3/5)5162worker1 # [ 114.032883] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 722 ms (attempt 4/5)5163lb # [ 114.204870] envoy[933]: conn peer=[2001:db8:1::1]:39828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=2176 ms=25485164lb # [ 114.205261] envoy[933]: conn peer=[2001:db8:1::1]:39844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=1155165lb # [ 114.205398] envoy[933]: conn peer=[2001:db8:1::1]:39848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4356 tx=2427 ms=23195166lb # [ 114.205531] envoy[933]: conn peer=[2001:db8:1::1]:39850 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=275167lb # [ 114.224866] envoy[933]: conn peer=[2001:db8:1::1]:59866 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=965168lb # [ 114.229887] envoy[933]: conn peer=[2001:db8:1::1]:39858 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3892 tx=2086 ms=25535169worker1 # [ 114.754741] nix-grpc-daemon[778]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5170worker1 # [ 114.755151] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5171worker1 # [ 114.767009] nix-grpc-daemon[778]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5172worker1 # [ 114.991174] nix-daemon[844]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5173worker1 # [ 115.010615] nix-daemon[844]: reaped child process 1259, status = succeeded5174worker1 # [ 115.182611] nix-daemon[844]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5175worker1 # [ 115.199184] nix-daemon[844]: reaped child process 1264, status = succeeded5176lb # [ 115.174488] niks3-server[1035]: 2026/09/22 10:58:45 INFO Received uploads request method=POST path=/api/pending_closures5177worker1 # [ 115.222024] nix-grpc-daemon[857]: time=2026-09-22T10:58:45.692Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5178worker1 # [ 115.223444] nix-grpc-daemon[857]: time=2026-09-22T10:58:45.692Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5179lb # [ 115.209777] niks3-server[1035]: 2026/09/22 10:58:45 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5180lb # [ 115.223704] niks3-server[1035]: 2026/09/22 10:58:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5181worker1 # [ 115.265614] nix-grpc-daemon[857]: time=2026-09-22T10:58:45.736Z level=INFO msg="Uploading 1 narinfos"5182lb # [ 115.233407] niks3-server[1035]: 2026/09/22 10:58:45 INFO Signed narinfos id=31 count=15183lb # [ 115.235834] niks3-server[1035]: 2026/09/22 10:58:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5184worker1 # [ 115.284076] nix-grpc-daemon[857]: time=2026-09-22T10:58:45.755Z level=INFO msg="Upload complete. (492ms)"5185lb # [ 115.248767] niks3-server[1035]: 2026/09/22 10:58:45 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5186lb # [ 115.253487] niks3-server[1035]: 2026/09/22 10:58:45 INFO Completed upload id=315187lb # [ 115.256492] niks3-server[1035]: 2026/09/22 10:58:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5188worker1 # [ 115.293675] nix-grpc-daemon[778]: ts=2026-09-22T10:58:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=1 paths=1 nar_bytes_in=5685189worker1 # [ 115.302480] nix-grpc-daemon[778]: ts=2026-09-22T10:58:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413385190worker1 # [ 115.312358] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)5191worker1 # [ 115.316491] nix-daemon[844]: reaped child process 1249, status = succeeded5192client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5193worker1 # [ 115.327783] nix-daemon[844]: accepted connection from pid 778, user nix-grpc-daemon (trusted)5194client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5195client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5196worker1 # [ 116.163639] nix-daemon[844]: reaped child process 1268, status = succeeded5197worker1 # [ 116.335518] nix-daemon[844]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)5198worker1 # [ 116.354949] nix-daemon[844]: reaped child process 1287, status = succeeded5199lb # [ 116.330220] niks3-server[1035]: 2026/09/22 10:58:46 INFO Received uploads request method=POST path=/api/pending_closures5200worker1 # [ 116.377635] nix-grpc-daemon[857]: time=2026-09-22T10:58:46.846Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5201lb # [ 116.360336] niks3-server[1035]: 2026/09/22 10:58:46 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5202worker1 # [ 116.411159] nix-grpc-daemon[857]: time=2026-09-22T10:58:46.882Z level=INFO msg="Uploading 1 narinfos"5203lb # [ 116.368690] niks3-server[1035]: 2026/09/22 10:58:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5204lb # [ 116.381122] niks3-server[1035]: 2026/09/22 10:58:46 INFO Signed narinfos id=32 count=15205lb # [ 116.387041] niks3-server[1035]: 2026/09/22 10:58:46 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5206lb # [ 116.398440] niks3-server[1035]: 2026/09/22 10:58:46 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5207lb # [ 116.406580] niks3-server[1035]: 2026/09/22 10:58:46 INFO Completed upload id=325208worker1 # [ 116.443296] nix-grpc-daemon[857]: time=2026-09-22T10:58:46.914Z level=INFO msg="Upload complete. (306ms)"5209lb # [ 116.410249] niks3-server[1035]: 2026/09/22 10:58:46 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5210client # copying 1 paths...5211worker1 # [ 116.452989] nix-grpc-daemon[778]: ts=2026-09-22T10:58:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15212worker2 # [ 116.447617] nix-grpc-daemon[780]: ts=2026-09-22T10:58:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:401325213worker1 # [ 116.470450] nix-daemon[844]: reaped child process 1266, status = succeeded5214worker2 # [ 116.455553] nix-grpc-daemon[780]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5215client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5216worker1 # [ 116.514639] nix-grpc-daemon[778]: ts=2026-09-22T10:58:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413385217worker1 # [ 116.521909] nix-grpc-daemon[778]: ts=2026-09-22T10:58:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_out=1205218client # [ 116.558777] nix-daemon[1105]: reaped child process 1213, status = succeeded5219client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.68 seconds)5220client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&25221client # [ 116.776135] nix-daemon[1105]: accepted connection from pid 1232, user root (trusted)5222client # this derivation will be built:5223client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5224worker2 # [ 117.021846] nix-grpc-daemon[780]: ts=2026-09-22T10:58:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401325225client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...5226worker1 # [ 117.042679] nix-grpc-daemon[778]: ts=2026-09-22T10:58:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413385227worker1 # [ 117.052744] nix-grpc-daemon[857]: time=2026-09-22T10:58:47.523Z level=INFO msg="All 1 paths already cached"5228worker2 # [ 117.054721] nix-grpc-daemon[780]: ts=2026-09-22T10:58:47Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05229worker2 # [ 117.064530] nix-grpc-daemon[780]: ts=2026-09-22T10:58:47Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15230worker2 # [ 117.073405] nix-grpc-daemon[780]: ts=2026-09-22T10:58:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401325231worker2 # [ 117.080955] nix-grpc-daemon[780]: ts=2026-09-22T10:58:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401325232client # copying 1 paths to 'lb:50051'...5233worker2 # [ 117.093563] nix-grpc-daemon[917]: time=2026-09-22T10:58:47.550Z level=INFO msg="All 1 paths already cached"5234client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5235worker2 # [ 117.099018] nix-grpc-daemon[780]: ts=2026-09-22T10:58:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401325236worker2 # [ 117.114120] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5237worker2 # [ 117.312969] nix-daemon[866]: accepted connection from pid 1215, user nix-grpc-daemon (trusted)5238worker2 # [ 117.331374] nix-daemon[866]: reaped child process 1218, status = succeeded5239worker2 # [ 117.502334] nix-daemon[866]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5240worker2 # [ 117.519191] nix-daemon[866]: reaped child process 1223, status = succeeded5241lb # [ 117.513735] niks3-server[1035]: 2026/09/22 10:58:47 INFO Received uploads request method=POST path=/api/pending_closures5242worker2 # [ 117.541553] nix-grpc-daemon[917]: time=2026-09-22T10:58:48.000Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5243worker2 # [ 117.543027] nix-grpc-daemon[917]: time=2026-09-22T10:58:48.000Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5244lb # [ 117.541828] niks3-server[1035]: 2026/09/22 10:58:47 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5245worker2 # [ 117.575349] nix-grpc-daemon[917]: time=2026-09-22T10:58:48.034Z level=INFO msg="Uploading 1 narinfos"5246lb # [ 117.557844] niks3-server[1035]: 2026/09/22 10:58:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5247lb # [ 117.568120] niks3-server[1035]: 2026/09/22 10:58:48 INFO Signed narinfos id=33 count=15248lb # [ 117.570734] niks3-server[1035]: 2026/09/22 10:58:48 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5249lb # [ 117.576781] niks3-server[1035]: 2026/09/22 10:58:48 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5250lb # [ 117.581829] niks3-server[1035]: 2026/09/22 10:58:48 INFO Completed upload id=335251worker2 # [ 117.600876] nix-grpc-daemon[917]: time=2026-09-22T10:58:48.060Z level=INFO msg="Upload complete. (471ms)"5252worker2 # [ 117.612205] nix-grpc-daemon[780]: ts=2026-09-22T10:58:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=0 paths=1 nar_bytes_in=5685253worker2 # [ 117.622101] nix-grpc-daemon[780]: ts=2026-09-22T10:58:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401325254worker2 # [ 117.627695] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5255worker2 # [ 117.631988] nix-daemon[866]: reaped child process 1213, status = succeeded5256client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5257worker2 # [ 117.644336] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5258client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5259client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5260worker2 # [ 118.891625] nix-daemon[866]: reaped child process 1227, status = succeeded5261worker2 # [ 119.081809] nix-daemon[866]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5262worker2 # [ 119.101096] nix-daemon[866]: reaped child process 1246, status = succeeded5263lb # [ 119.095752] niks3-server[1035]: 2026/09/22 10:58:49 INFO Received uploads request method=POST path=/api/pending_closures5264worker2 # [ 119.128471] nix-grpc-daemon[917]: time=2026-09-22T10:58:49.586Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5265lb # [ 119.140470] niks3-server[1035]: 2026/09/22 10:58:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5266worker2 # [ 119.164812] nix-grpc-daemon[917]: time=2026-09-22T10:58:49.623Z level=INFO msg="Uploading 1 narinfos"5267lb # [ 119.152211] niks3-server[1035]: 2026/09/22 10:58:49 INFO Signed narinfos id=34 count=15268lb # [ 119.162034] niks3-server[1035]: 2026/09/22 10:58:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5269lb # [ 119.171173] niks3-server[1035]: 2026/09/22 10:58:49 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5270lb # [ 119.176916] niks3-server[1035]: 2026/09/22 10:58:49 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5271lb # [ 119.184441] niks3-server[1035]: 2026/09/22 10:58:49 INFO Completed upload id=345272worker2 # [ 119.203808] nix-grpc-daemon[917]: time=2026-09-22T10:58:49.662Z level=INFO msg="Upload complete. (348ms)"5273lb # [ 119.187534] niks3-server[1035]: 2026/09/22 10:58:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5274client # copying 1 paths...5275worker2 # [ 119.216689] nix-grpc-daemon[780]: ts=2026-09-22T10:58:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40132 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15276worker1 # [ 119.242402] nix-grpc-daemon[778]: ts=2026-09-22T10:58:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413385277worker2 # [ 119.237070] nix-daemon[866]: reaped child process 1225, status = succeeded5278client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5279worker1 # [ 119.250545] nix-grpc-daemon[778]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5280worker1 # [ 119.286076] nix-grpc-daemon[778]: ts=2026-09-22T10:58:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413385281worker1 # [ 119.297880] nix-grpc-daemon[778]: ts=2026-09-22T10:58:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41338 duration_s=0 paths=1 nar_bytes_out=1205282client # [ 119.375620] nix-daemon[1105]: reaped child process 1235, status = succeeded5283client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.82 seconds)5284(finished: subtest: an input only one worker has still reaches the builder, in 10.53 seconds)5285subtest: upload whose reference the worker lacks is completed from the cache5286client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache5287client # this derivation will be built:5288client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5289client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5290client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.45 seconds)5291client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache5292client # this derivation will be built:5293client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5294client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5295client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds)5296client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5297client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)5298worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5299worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds)5300worker2 # [ 120.823785] systemd[1]: Stopping Nix worker-protocol over gRPC...5301worker2 # [ 121.741739] nix-grpc-daemon[780]: ts=2026-09-22T10:58:52Z level=info event=signal_exit5302lb # [ 122.002608] niks3-server[1035]: 2026/09/22 10:58:52 INFO lead: released remote=192.168.1.4:350205303lb # [ 122.759671] niks3-server[1035]: 2026/09/22 10:58:53 INFO lead: acquired remote=192.168.1.3:434805304worker1 # [ 122.798887] nix-grpc-daemon[778]: ts=2026-09-22T10:58:53Z level=info event=scheduler_take_over5305worker2 # [ 123.014724] nix-grpc-daemon[780]: ts=2026-09-22T10:58:53Z level=info event=scheduler_restarting5306worker1 # [ 123.040817] nix-grpc-daemon[778]: ts=2026-09-22T10:58:53Z level=info event=scheduler_restarting addr=lb:500515307worker2 # [ 123.023627] nix-grpc-daemon[780]: ts=2026-09-22T10:58:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:401245308worker2 # [ 123.034813] nix-grpc-daemon[780]: ts=2026-09-22T10:58:53Z level=info event=worker_gone addr=192.168.1.4:500515309worker2 # [ 123.040294] nix-grpc-daemon[780]: ts=2026-09-22T10:58:53Z level=info event=scheduler_restarting addr=lb:500515310worker2 # [ 123.044677] nix-grpc-daemon[780]: ts=2026-09-22T10:58:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:401245311worker2 # [ 123.049939] nix-grpc-daemon[780]: ts=2026-09-22T10:58:53Z level=info event=worker_gone addr=192.168.1.3:500515312worker2 # [ 123.079926] nix-daemon[866]: reaped child process 869, status = succeeded5313worker1 # [ 123.242998] nix-grpc-daemon[778]: ts=2026-09-22T10:58:53Z level=info event=scheduler_connected addr=lb:500515314worker1 # [ 123.248297] nix-grpc-daemon[778]: ts=2026-09-22T10:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5315worker2 # [ 123.254625] nix-grpc-daemon[780]: ts=2026-09-22T10:58:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5316worker1 # [ 123.449191] nix-grpc-daemon[778]: ts=2026-09-22T10:58:53Z level=info event=scheduler_connected addr=lb:500515317worker1 # [ 123.454703] nix-grpc-daemon[778]: ts=2026-09-22T10:58:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5318worker1 # [ 123.655520] nix-grpc-daemon[778]: ts=2026-09-22T10:58:54Z level=info event=scheduler_connected addr=lb:500515319worker1 # [ 123.661095] nix-grpc-daemon[778]: ts=2026-09-22T10:58:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5320worker1 # [ 123.863011] nix-grpc-daemon[778]: ts=2026-09-22T10:58:54Z level=info event=scheduler_connected addr=lb:500515321worker1 # [ 123.885213] nix-grpc-daemon[778]: ts=2026-09-22T10:58:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:502945322worker1 # [ 123.890507] nix-grpc-daemon[778]: ts=2026-09-22T10:58:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05323worker2 # [ 123.902924] nix-grpc-daemon[780]: E0000 00:00:1790074734.361712 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5324worker2 # [ 123.909860] nix-grpc-daemon[780]: E0000 00:00:1790074734.362049 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5325lb # [ 124.204917] envoy[933]: conn peer=[2001:db8:1::1]:59886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=265326lb # [ 124.209857] envoy[933]: conn peer=[2001:db8:1::1]:59872 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3752 tx=2125 ms=31755327lb # [ 124.209998] envoy[933]: conn peer=[2001:db8:1::1]:59888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2102 ms=23185328lb # [ 124.210212] envoy[933]: conn peer=[2001:db8:1::1]:59896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=735329lb # [ 124.210321] envoy[933]: conn peer=[2001:db8:1::4]:33630 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=73274 tx=3902 ms=984665330worker2 # [ 125.285411] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5331worker2 # [ 125.293063] systemd[1]: Stopped Nix worker-protocol over gRPC.5332worker2 # [ 125.295429] systemd[1]: nix-grpc-daemon.service: Consumed 3.578s CPU time over 1min 40.900s wall clock time, 61.1M memory peak, 93.5K incoming IP traffic, 245.2K outgoing IP traffic.5333worker2 # [ 125.310579] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5334worker2 # [ 125.313621] systemd[1]: Closed Nix worker-protocol over gRPC.5335worker2 # [ 125.315198] systemd[1]: nix-grpc-daemon.socket: Consumed 784us CPU time over 1min 53.055s wall clock time, 256K memory peak, 120.1K incoming IP traffic, 100.6K outgoing IP traffic.5336lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5337lb: must succeed: curl -sf localhost:9901/clusters5338lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5339lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.10 seconds)5340client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25341worker1 # [ 125.707892] nix-grpc-daemon[778]: ts=2026-09-22T10:58:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:413385342worker1 # [ 125.716277] nix-grpc-daemon[778]: ts=2026-09-22T10:58:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413385343client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5344client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.31 seconds)5345worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5346worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)5347worker2 # [ 125.841775] systemd[1]: Listening on Nix worker-protocol over gRPC.5348worker1 # [ 125.939541] systemd[1]: Stopping Nix worker-protocol over gRPC...5349worker2 # [ 126.272189] systemd[1]: Starting scheduler-token.service...5350worker2 # [ 126.390191] systemd[1]: scheduler-token.service: Deactivated successfully.5351worker2 # [ 126.397091] systemd[1]: Finished scheduler-token.service.5352worker2 # [ 126.398904] systemd[1]: scheduler-token.service: Consumed 57ms CPU time over 121ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5353worker2 # [ 126.408968] systemd[1]: Starting Nix worker-protocol over gRPC...5354worker2 # [ 126.594398] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:57Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5355worker2 # [ 126.596956] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:57Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5356worker2 # [ 126.601307] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:57Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15357worker2 # [ 126.608062] nix-grpc-daemon[1270]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5358worker2 # [ 126.609526] nix-grpc-daemon[1270]: E0000 00:00:1790074737.067088 1270 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5359worker2 # [ 126.624166] nix-grpc-daemon[1270]: E0000 00:00:1790074737.073751 1270 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5360worker2 # [ 126.628479] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:57Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5361worker2 # [ 126.632814] systemd[1]: Started Nix worker-protocol over gRPC.5362worker2 # [ 126.640068] nix-grpc-daemon[1270]: E0000 00:00:1790074737.095528 1309 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5363worker1 # [ 126.667976] nix-grpc-daemon[778]: ts=2026-09-22T10:58:57Z level=info event=signal_exit5364worker2 # [ 126.664055] nix-grpc-daemon[1270]: E0000 00:00:1790074737.113004 1309 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5365worker2 # [ 126.666398] nix-grpc-daemon[1270]: E0000 00:00:1790074737.113178 1309 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5366worker2 # [ 126.673923] nix-grpc-daemon[1270]: E0000 00:00:1790074737.132341 1309 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5367worker2 # [ 126.686289] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:57Z level=info event=scheduler_connected addr=lb:500515368worker1 # [ 126.705653] nix-grpc-daemon[778]: ts=2026-09-22T10:58:57Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:502945369worker1 # [ 126.712246] nix-grpc-daemon[778]: ts=2026-09-22T10:58:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05370lb # [ 126.881342] niks3-server[1035]: 2026/09/22 10:58:57 INFO lead: released remote=192.168.1.3:434805371worker1 # [ 127.917263] nix-grpc-daemon[778]: ts=2026-09-22T10:58:58Z level=info event=scheduler_restarting5372worker2 # [ 127.902802] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:58Z level=info event=scheduler_restarting addr=lb:500515373worker1 # [ 127.930242] nix-grpc-daemon[778]: ts=2026-09-22T10:58:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:502945374worker1 # [ 127.935278] nix-grpc-daemon[778]: ts=2026-09-22T10:58:58Z level=info event=worker_gone addr=192.168.1.4:500515375worker1 # [ 127.942042] nix-grpc-daemon[778]: ts=2026-09-22T10:58:58Z level=info event=scheduler_restarting addr=lb:500515376worker1 # [ 127.946225] nix-grpc-daemon[778]: ts=2026-09-22T10:58:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:502945377worker1 # [ 127.950966] nix-grpc-daemon[778]: ts=2026-09-22T10:58:58Z level=info event=worker_gone addr=192.168.1.3:500515378worker1 # [ 127.975794] nix-daemon[844]: reaped child process 846, status = succeeded5379worker2 # [ 128.103077] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:58Z level=info event=scheduler_connected addr=lb:500515380worker2 # [ 128.108355] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5381worker1 # [ 128.151001] nix-grpc-daemon[778]: ts=2026-09-22T10:58:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5382worker2 # [ 128.309594] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:58Z level=info event=scheduler_connected addr=lb:500515383worker2 # [ 128.314842] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker2 # [ 128.515803] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:58Z level=info event=scheduler_connected addr=lb:500515385worker2 # [ 128.521062] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5386worker2 # [ 128.722071] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:59Z level=info event=scheduler_connected addr=lb:500515387worker2 # [ 128.727181] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5388worker1 # [ 128.807357] nix-grpc-daemon[778]: E0000 00:00:1790074739.278296 823 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5389worker1 # [ 128.852431] nix-grpc-daemon[778]: E0000 00:00:1790074739.323517 823 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5390worker2 # [ 128.930061] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:59Z level=info event=scheduler_connected addr=lb:500515391worker2 # [ 128.934317] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5392worker2 # [ 129.131005] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:59Z level=info event=scheduler_connected addr=lb:500515393worker2 # [ 129.136403] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5394worker1 # [ 129.196694] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5395worker1 # [ 129.204894] systemd[1]: Stopped Nix worker-protocol over gRPC.5396worker1 # [ 129.207372] systemd[1]: nix-grpc-daemon.service: Consumed 4.704s CPU time over 1min 44.953s wall clock time, 62.9M memory peak, 109.9K incoming IP traffic, 198.6K outgoing IP traffic.5397worker1 # [ 129.225646] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5398worker1 # [ 129.227526] systemd[1]: Closed Nix worker-protocol over gRPC.5399worker1 # [ 129.229167] systemd[1]: nix-grpc-daemon.socket: Consumed 799us CPU time over 1min 57.469s wall clock time, 256K memory peak, 118.5K incoming IP traffic, 103.4K outgoing IP traffic.5400lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5401lb: must succeed: curl -sf localhost:9901/clusters5402lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5403lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)5404client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25405worker2 # [ 129.337457] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:59Z level=info event=scheduler_connected addr=lb:500515406worker2 # [ 129.342554] nix-grpc-daemon[1270]: ts=2026-09-22T10:58:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker2 # [ 129.544003] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_connected addr=lb:500515408worker2 # [ 129.549295] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5409worker2 # [ 129.573090] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505410worker2 # [ 129.588879] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5411worker2 # [ 129.620315] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5)5412worker2 # [ 129.685892] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 2/5)5413worker2 # [ 129.750465] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_connected addr=lb:500515414worker2 # [ 129.755672] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5415worker2 # [ 129.811065] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 270 ms (attempt 3/5)5416worker2 # [ 129.958881] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_connected addr=lb:500515417worker2 # [ 129.963087] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5418worker2 # [ 130.088898] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 258 ms (attempt 4/5)5419worker2 # [ 130.159836] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_connected addr=lb:500515420worker2 # [ 130.165089] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5421worker2 # [ 130.347720] nix-grpc-daemon[1270]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5422worker2 # [ 130.353946] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5423worker2 # [ 130.366160] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_connected addr=lb:500515424worker2 # [ 130.373211] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5425client # copying 2 paths...5426client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5427worker2 # [ 130.385518] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555505428client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5429worker2 # [ 130.404962] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5430worker2 # [ 130.435391] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:00Z level=info event=niks3_push_started pid=13305431worker2 # [ 130.573974] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_connected addr=lb:500515432worker2 # [ 130.576302] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker2 # [ 130.655499] nix-grpc-daemon[1330]: time=2026-09-22T10:59:01.114Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5434worker2 # [ 130.777163] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_connected addr=lb:500515435worker2 # [ 130.779676] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5436worker2 # [ 130.838054] nix-daemon[866]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5437worker2 # [ 130.858189] nix-daemon[866]: reaped child process 1346, status = succeeded5438lb # [ 130.851731] niks3-server[1035]: 2026/09/22 10:59:01 INFO Received uploads request method=POST path=/api/pending_closures5439lb # [ 130.861293] niks3-server[1035]: 2026/09/22 10:59:01 INFO Received uploads request method=POST path=/api/pending_closures5440worker2 # [ 130.890025] nix-grpc-daemon[1330]: time=2026-09-22T10:59:01.349Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5441worker2 # [ 130.894800] nix-grpc-daemon[1330]: time=2026-09-22T10:59:01.350Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5442worker2 # [ 130.897077] nix-grpc-daemon[1330]: time=2026-09-22T10:59:01.351Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5443lb # [ 130.906750] niks3-server[1035]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5444lb # [ 130.922552] niks3-server[1035]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5445lb # [ 130.933447] niks3-server[1035]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5446lb # [ 130.941507] niks3-server[1035]: 2026/09/22 10:59:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5447lb # [ 130.952109] niks3-server[1035]: 2026/09/22 10:59:01 INFO Signed narinfos id=35 count=15448worker2 # [ 130.972351] nix-grpc-daemon[1330]: time=2026-09-22T10:59:01.430Z level=INFO msg="Uploading 3 narinfos"5449lb # [ 130.955546] niks3-server[1035]: 2026/09/22 10:59:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5450lb # [ 130.964199] niks3-server[1035]: 2026/09/22 10:59:01 INFO Signed narinfos id=36 count=25451worker2 # [ 130.984851] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_connected addr=lb:500515452lb # [ 130.966811] niks3-server[1035]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5453worker2 # [ 130.989163] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5454lb # [ 130.986881] niks3-server[1035]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5455lb # [ 130.994225] niks3-server[1035]: 2026/09/22 10:59:01 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5456lb # [ 130.998793] niks3-server[1035]: 2026/09/22 10:59:01 INFO Completed upload id=355457lb # [ 131.004794] niks3-server[1035]: 2026/09/22 10:59:01 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5458lb # [ 131.007539] niks3-server[1035]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5459worker2 # [ 131.033599] nix-grpc-daemon[1330]: time=2026-09-22T10:59:01.492Z level=INFO msg="Upload complete. (373ms)"5460lb # [ 131.019073] niks3-server[1035]: 2026/09/22 10:59:01 INFO Completed upload id=365461worker2 # [ 131.044609] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=0 paths=2 nar_bytes_in=3125462worker2 # [ 131.063239] nix-daemon[866]: reaped child process 1328, status = succeeded5463client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.78 seconds)5464worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5465worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5466worker2 # [ 131.183835] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_connected addr=lb:500515467worker2 # [ 131.189080] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5468worker1 # [ 131.224967] systemd[1]: Listening on Nix worker-protocol over gRPC.5469worker1 # [ 131.232748] systemd[1]: Starting Nix worker-protocol over gRPC...5470worker2 # [ 131.389835] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_connected addr=lb:500515471worker2 # [ 131.395015] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5472worker1 # [ 131.430919] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5473worker1 # [ 131.433444] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5474worker1 # [ 131.437622] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15475worker1 # [ 131.446747] nix-grpc-daemon[1311]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5476worker1 # [ 131.455561] nix-grpc-daemon[1311]: E0000 00:00:1790074741.915333 1311 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5477lb # [ 131.421238] niks3-server[1035]: 2026/09/22 10:59:01 INFO lead: acquired remote=192.168.1.3:344465478worker1 # [ 131.460163] nix-grpc-daemon[1311]: E0000 00:00:1790074741.917789 1311 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5479worker1 # [ 131.463228] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5480lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5481lb: must succeed: curl -sf localhost:9901/clusters5482worker1 # [ 131.466122] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:01Z level=info event=scheduler_take_over5483worker1 # [ 131.467492] systemd[1]: Started Nix worker-protocol over gRPC.5484worker1 # [ 131.481467] nix-grpc-daemon[1311]: E0000 00:00:1790074741.939995 1350 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5485worker1 # [ 131.502019] nix-grpc-daemon[1311]: E0000 00:00:1790074741.957855 1350 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5486worker1 # [ 131.504899] nix-grpc-daemon[1311]: E0000 00:00:1790074741.958040 1350 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5487worker1 # [ 131.507344] nix-grpc-daemon[1311]: E0000 00:00:1790074741.972334 1350 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5488worker1 # [ 131.516277] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:01Z level=info event=scheduler_connected addr=lb:500515489worker1 # [ 131.518156] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5490lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5491worker2 # [ 131.596242] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:02Z level=info event=scheduler_connected addr=lb:500515492worker2 # [ 131.601408] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5493worker1 # [ 131.719062] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=scheduler_connected addr=lb:500515494worker1 # [ 131.724007] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5495worker2 # [ 131.802652] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:02Z level=info event=scheduler_connected addr=lb:500515496worker2 # [ 131.807925] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker1 # [ 131.924799] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=scheduler_connected addr=lb:500515498worker1 # [ 131.929769] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker2 # [ 132.010929] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:02Z level=info event=scheduler_connected addr=lb:500515500worker2 # [ 132.015111] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5501worker1 # [ 132.130509] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=scheduler_connected addr=lb:500515502worker1 # [ 132.135530] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5503worker2 # [ 132.211481] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:02Z level=info event=scheduler_connected addr=lb:500515504worker1 # [ 132.240232] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=oidc_jwks provider=mock keys=15505worker1 # [ 132.245137] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:440065506worker1 # [ 132.250461] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05507worker1 # [ 132.336338] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=scheduler_connected addr=lb:500515508worker1 # [ 132.341971] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:440065509worker1 # [ 132.346825] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:02Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05510lb: must succeed: curl -sf localhost:9901/clusters5511lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5512lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5513(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 13.20 seconds)5514subtest: a niks3 restart does not cost the leader its role5515lb: must succeed: curl -sf localhost:9901/clusters5516lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5517lb: must succeed: curl -sf localhost:9901/clusters5518lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5519lb # [ 132.753409] niks3-server[1035]: 2026/09/22 10:59:03 INFO Shutdown signal received, draining in-flight requests timeout=10s5520lb # [ 132.758696] systemd[1]: Stopping niks3 server...5521worker2 # [ 132.779021] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:03Z level=info event=scheduler_lock_lost status=2005522worker1 # [ 132.796355] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:03Z level=info event=scheduler_lock_lost status=2005523lb # [ 132.763406] niks3-server[1035]: 2026/09/22 10:59:03 INFO lead: released remote=192.168.1.3:344465524lb # [ 134.208607] envoy[933]: conn peer=[2001:db8:1::1]:46820 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=295525lb # [ 134.213511] envoy[933]: conn peer=[2001:db8:1::3]:35386 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15540 tx=4302 ms=1034515526lb # [ 134.218541] envoy[933]: conn peer=[2001:db8:1::1]:46822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=304 ms=14875527lb # [ 136.462701] systemd[1]: niks3.service: Deactivated successfully.5528lb # [ 136.469238] systemd[1]: Stopped niks3 server.5529lb # [ 136.472912] systemd[1]: niks3.service: Consumed 714ms CPU time over 1min 44.164s wall clock time, 23.8M memory peak, 95.3K incoming IP traffic, 137.4K outgoing IP traffic.5530lb # [ 136.499089] systemd[1]: Starting niks3 server...5531lb # [ 136.624501] niks3-server[1182]: 2026/09/22 10:59:07 goose: no migrations to run. current version: 202609200000005532lb # [ 136.633623] niks3-server[1182]: 2026/09/22 10:59:07 OK 1_commit_pending_closure.sql (9.99ms)5533lb # [ 136.637962] niks3-server[1182]: 2026/09/22 10:59:07 OK 2_object_stats_trigger.sql (3.66ms)5534lb # [ 136.639839] niks3-server[1182]: 2026/09/22 10:59:07 goose: up to current file version: 25535lb # [ 136.642914] niks3-server[1182]: 2026/09/22 10:59:07 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key5536lb # [ 136.658038] niks3-server[1182]: 2026/09/22 10:59:07 INFO Read proxy enabled — serving cache objects from S35537lb # [ 136.659766] niks3-server[1182]: 2026/09/22 10:59:07 INFO Using socket-activated listener address=0.0.0.0:57515538lb # [ 136.666545] niks3-server[1182]: 2026/09/22 10:59:07 INFO systemd watchdog enabled interval=15s5539lb # [ 136.669286] niks3-server[1182]: 2026/09/22 10:59:07 INFO Starting HTTP server address=0.0.0.0:57515540lb # [ 136.670847] niks3-server[1182]: 2026/09/22 10:59:07 INFO lead: acquired remote=192.168.1.3:344545541lb # [ 136.673823] systemd[1]: Started niks3 server.5542lb: waiting for unit niks3.service5543lb: (finished: waiting for unit niks3.service, in 0.05 seconds)5544lb: must succeed: curl -sf localhost:9901/clusters5545lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5546worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5547worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5548lb: must succeed: curl -sf localhost:9901/clusters5549lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5550lb: must succeed: curl -sf localhost:9901/clusters5551lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5552worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5553worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5554client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&25555worker2 # [ 137.322920] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555505556client # don't know how to build these paths:5557client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5558worker2 # [ 137.334403] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505559worker1 # [ 137.395994] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:07Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05560worker1 # [ 137.397489] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:07Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15561worker2 # [ 137.424117] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505562worker2 # [ 137.439482] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505563client # copying 1 paths to 'lb:50051'...5564client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5565worker2 # [ 137.455474] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555505566worker2 # [ 137.471283] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5567worker2 # [ 137.685173] nix-daemon[866]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)5568worker2 # [ 137.703587] nix-daemon[866]: reaped child process 1358, status = succeeded5569worker2 # [ 137.878876] nix-daemon[866]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5570worker2 # [ 137.896527] nix-daemon[866]: reaped child process 1363, status = succeeded5571lb # [ 137.889742] niks3-server[1182]: 2026/09/22 10:59:08 INFO Received uploads request method=POST path=/api/pending_closures5572worker2 # [ 137.925352] nix-grpc-daemon[1330]: time=2026-09-22T10:59:08.383Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5573worker2 # [ 137.926862] nix-grpc-daemon[1330]: time=2026-09-22T10:59:08.384Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5574lb # [ 137.935733] niks3-server[1182]: 2026/09/22 10:59:08 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5575lb # [ 137.952864] niks3-server[1182]: 2026/09/22 10:59:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5576worker2 # [ 137.983721] nix-grpc-daemon[1330]: time=2026-09-22T10:59:08.442Z level=INFO msg="Uploading 1 narinfos"5577lb # [ 137.969710] niks3-server[1182]: 2026/09/22 10:59:08 INFO Signed narinfos id=37 count=15578lb # [ 137.982633] niks3-server[1182]: 2026/09/22 10:59:08 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5579lb # [ 137.996196] niks3-server[1182]: 2026/09/22 10:59:08 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5580lb # [ 137.998818] niks3-server[1182]: 2026/09/22 10:59:08 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5581lb # [ 138.004048] niks3-server[1182]: 2026/09/22 10:59:08 INFO Completed upload id=375582worker2 # [ 138.024039] nix-grpc-daemon[1330]: time=2026-09-22T10:59:08.483Z level=INFO msg="Upload complete. (534ms)"5583worker2 # [ 138.036161] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=0 paths=1 nar_bytes_in=5445584worker2 # [ 138.052633] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505585worker2 # [ 138.060981] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5586worker2 # [ 138.067420] nix-daemon[866]: reaped child process 1353, status = succeeded5587client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5588client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5589worker2 # [ 138.080507] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5590client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5591client # farm-a> LOG-farm-a-after-niks3-restart5592worker2 # [ 139.522800] nix-daemon[866]: reaped child process 1367, status = succeeded5593worker2 # [ 139.708848] nix-daemon[866]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)5594worker2 # [ 139.727357] nix-daemon[866]: reaped child process 1386, status = succeeded5595lb # [ 139.721607] niks3-server[1182]: 2026/09/22 10:59:10 INFO Received uploads request method=POST path=/api/pending_closures5596worker2 # [ 139.754325] nix-grpc-daemon[1330]: time=2026-09-22T10:59:10.211Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5597worker2 # [ 139.755838] nix-grpc-daemon[1330]: time=2026-09-22T10:59:10.211Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5598lb # [ 139.763392] niks3-server[1182]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5599lb # [ 139.776766] niks3-server[1182]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5600lb # [ 139.787836] niks3-server[1182]: 2026/09/22 10:59:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5601lb # [ 139.794541] niks3-server[1182]: 2026/09/22 10:59:10 INFO Signed narinfos id=38 count=15602worker2 # [ 139.815027] nix-grpc-daemon[1330]: time=2026-09-22T10:59:10.273Z level=INFO msg="Uploading 1 narinfos"5603lb # [ 139.799116] niks3-server[1182]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5604lb # [ 139.807127] niks3-server[1182]: 2026/09/22 10:59:10 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5605lb # [ 139.818982] niks3-server[1182]: 2026/09/22 10:59:10 INFO Completed upload id=385606lb # [ 139.821900] niks3-server[1182]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5607worker2 # [ 139.838943] nix-grpc-daemon[1330]: time=2026-09-22T10:59:10.298Z level=INFO msg="Upload complete. (347ms)"5608worker1 # [ 139.878430] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:10Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05609worker2 # [ 139.853445] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15610worker1 # [ 139.884174] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:10Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15611worker2 # [ 139.872932] nix-daemon[866]: reaped child process 1365, status = succeeded5612worker2 # [ 139.907821] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505613worker2 # [ 139.923852] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505614worker2 # [ 139.938325] nix-grpc-daemon[1330]: time=2026-09-22T10:59:10.397Z level=INFO msg="All 1 paths already cached"5615client # copying 1 paths to 'lb:50051'...5616client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5617worker2 # [ 139.948779] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555505618worker2 # [ 139.962108] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5619worker2 # [ 140.162354] nix-daemon[866]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5620worker2 # [ 140.182059] nix-daemon[866]: reaped child process 1393, status = succeeded5621worker2 # [ 140.357567] nix-daemon[866]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5622worker2 # [ 140.375933] nix-daemon[866]: reaped child process 1398, status = succeeded5623lb # [ 140.372601] niks3-server[1182]: 2026/09/22 10:59:10 INFO Received uploads request method=POST path=/api/pending_closures5624worker2 # [ 140.402386] nix-grpc-daemon[1330]: time=2026-09-22T10:59:10.861Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5625worker2 # [ 140.403875] nix-grpc-daemon[1330]: time=2026-09-22T10:59:10.861Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5626lb # [ 140.400407] niks3-server[1182]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5627lb # [ 140.414115] niks3-server[1182]: 2026/09/22 10:59:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5628worker2 # [ 140.440520] nix-grpc-daemon[1330]: time=2026-09-22T10:59:10.899Z level=INFO msg="Uploading 1 narinfos"5629lb # [ 140.427064] niks3-server[1182]: 2026/09/22 10:59:10 INFO Signed narinfos id=39 count=15630lb # [ 140.432443] niks3-server[1182]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5631lb # [ 140.441133] niks3-server[1182]: 2026/09/22 10:59:10 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5632worker2 # [ 140.465215] nix-grpc-daemon[1330]: time=2026-09-22T10:59:10.924Z level=INFO msg="Upload complete. (487ms)"5633lb # [ 140.445974] niks3-server[1182]: 2026/09/22 10:59:10 INFO Completed upload id=395634lb # [ 140.449019] niks3-server[1182]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5635worker2 # [ 140.472258] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=0 paths=1 nar_bytes_in=6565636worker2 # [ 140.486085] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505637worker2 # [ 140.492397] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5638worker2 # [ 140.497268] nix-daemon[866]: reaped child process 1388, status = succeeded5639client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5640client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5641worker2 # [ 140.510490] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5642client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5643client # farm-b> LOG-farm-b-after-niks3-restart5644worker2 # [ 141.742570] nix-daemon[866]: reaped child process 1402, status = succeeded5645worker2 # [ 141.911816] nix-daemon[866]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5646worker2 # [ 141.932206] nix-daemon[866]: reaped child process 1421, status = succeeded5647lb # [ 141.924871] niks3-server[1182]: 2026/09/22 10:59:12 INFO Received uploads request method=POST path=/api/pending_closures5648worker2 # [ 141.954585] nix-grpc-daemon[1330]: time=2026-09-22T10:59:12.412Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5649worker2 # [ 141.956118] nix-grpc-daemon[1330]: time=2026-09-22T10:59:12.412Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5650lb # [ 141.958685] niks3-server[1182]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5651lb # [ 141.972683] niks3-server[1182]: 2026/09/22 10:59:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5652worker2 # [ 142.001301] nix-grpc-daemon[1330]: time=2026-09-22T10:59:12.460Z level=INFO msg="Uploading 1 narinfos"5653lb # [ 141.985000] niks3-server[1182]: 2026/09/22 10:59:12 INFO Signed narinfos id=40 count=15654lb # [ 141.987914] niks3-server[1182]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5655lb # [ 142.007454] niks3-server[1182]: 2026/09/22 10:59:12 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5656worker2 # [ 142.031252] nix-grpc-daemon[1330]: time=2026-09-22T10:59:12.490Z level=INFO msg="Upload complete. (310ms)"5657lb # [ 142.015103] niks3-server[1182]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5658lb # [ 142.023167] niks3-server[1182]: 2026/09/22 10:59:12 INFO Completed upload id=405659worker2 # [ 142.041066] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15660worker1 # [ 142.066158] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:12Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05661worker1 # [ 142.072144] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:12Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15662worker2 # [ 142.060635] nix-daemon[866]: reaped child process 1400, status = succeeded5663worker2 # [ 142.096175] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505664worker2 # [ 142.111988] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505665worker2 # [ 142.128361] nix-grpc-daemon[1330]: time=2026-09-22T10:59:12.587Z level=INFO msg="All 2 paths already cached"5666client # copying 1 paths to 'lb:50051'...5667client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5668worker2 # [ 142.139693] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555505669worker2 # [ 142.153871] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5670worker2 # [ 142.351391] nix-daemon[866]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)5671worker2 # [ 142.372725] nix-daemon[866]: reaped child process 1428, status = succeeded5672worker2 # [ 142.547273] nix-daemon[866]: accepted connection from pid 1430, user nix-grpc-daemon (trusted)5673worker2 # [ 142.564312] nix-daemon[866]: reaped child process 1433, status = succeeded5674lb # [ 142.558837] niks3-server[1182]: 2026/09/22 10:59:13 INFO Received uploads request method=POST path=/api/pending_closures5675worker2 # [ 142.618620] nix-grpc-daemon[1330]: time=2026-09-22T10:59:13.077Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5676worker2 # [ 142.620583] nix-grpc-daemon[1330]: time=2026-09-22T10:59:13.077Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5677worker2 # [ 142.657551] nix-grpc-daemon[1330]: time=2026-09-22T10:59:13.116Z level=INFO msg="Uploading 1 narinfos"5678lb # [ 142.630982] niks3-server[1182]: 2026/09/22 10:59:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5679lb # [ 142.645396] niks3-server[1182]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5680lb # [ 142.654606] niks3-server[1182]: 2026/09/22 10:59:13 INFO Signed narinfos id=41 count=15681lb # [ 142.664266] niks3-server[1182]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5682lb # [ 142.671301] niks3-server[1182]: 2026/09/22 10:59:13 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5683lb # [ 142.676390] niks3-server[1182]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5684worker2 # [ 142.696424] nix-grpc-daemon[1330]: time=2026-09-22T10:59:13.155Z level=INFO msg="Upload complete. (525ms)"5685lb # [ 142.680833] niks3-server[1182]: 2026/09/22 10:59:13 INFO Completed upload id=415686worker2 # [ 142.707410] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=0 paths=1 nar_bytes_in=7845687worker2 # [ 142.720854] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505688worker2 # [ 142.733113] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5689worker2 # [ 142.739758] nix-daemon[866]: reaped child process 1423, status = succeeded5690client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5691client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5692worker2 # [ 142.753440] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5693client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5694client # farm-top> LOG-farm-top-after-niks3-restart5695worker2 # [ 143.987287] nix-daemon[866]: reaped child process 1437, status = succeeded5696worker2 # [ 144.150498] nix-daemon[866]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)5697worker2 # [ 144.168123] nix-daemon[866]: reaped child process 1456, status = succeeded5698lb # [ 144.160862] niks3-server[1182]: 2026/09/22 10:59:14 INFO Received uploads request method=POST path=/api/pending_closures5699worker2 # [ 144.191933] nix-grpc-daemon[1330]: time=2026-09-22T10:59:14.649Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5700worker2 # [ 144.193771] nix-grpc-daemon[1330]: time=2026-09-22T10:59:14.649Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5701lb # [ 144.206698] niks3-server[1182]: 2026/09/22 10:59:14 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5702worker2 # [ 144.238866] nix-grpc-daemon[1330]: time=2026-09-22T10:59:14.697Z level=INFO msg="Uploading 1 narinfos"5703lb # [ 144.221171] niks3-server[1182]: 2026/09/22 10:59:14 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5704lb # [ 144.229711] niks3-server[1182]: 2026/09/22 10:59:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5705lb # [ 144.233862] niks3-server[1182]: 2026/09/22 10:59:14 INFO Signed narinfos id=42 count=15706lb # [ 144.239927] niks3-server[1182]: 2026/09/22 10:59:14 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5707lb # [ 144.244457] niks3-server[1182]: 2026/09/22 10:59:14 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5708worker2 # [ 144.263868] nix-grpc-daemon[1330]: time=2026-09-22T10:59:14.723Z level=INFO msg="Upload complete. (296ms)"5709lb # [ 144.248037] niks3-server[1182]: 2026/09/22 10:59:14 INFO Completed upload id=425710lb # [ 144.249789] niks3-server[1182]: 2026/09/22 10:59:14 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5711worker2 # [ 144.277168] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15712worker2 # [ 144.295293] nix-daemon[866]: reaped child process 1435, status = succeeded5713client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&2, in 7.28 seconds)5714client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath5715client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath, in 0.24 seconds)5716(finished: subtest: a niks3 restart does not cost the leader its role, in 11.96 seconds)5717subtest: scheduler down: the other node takes the lock and keeps it5718lb: must succeed: curl -sf localhost:9901/clusters5719lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5720lb: must succeed: curl -sf localhost:9901/clusters5721lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5722lb: must succeed: curl -sf localhost:9901/clusters5723lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5724lb: must succeed: curl -sf localhost:9901/clusters5725lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5726worker1 # [ 144.894174] systemd[1]: Stopping Nix worker-protocol over gRPC...5727worker1 # [ 145.455413] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:15Z level=info event=signal_exit5728lb # [ 145.981971] niks3-server[1182]: 2026/09/22 10:59:16 INFO lead: released remote=192.168.1.3:344545729lb # [ 146.664161] niks3-server[1182]: 2026/09/22 10:59:17 INFO lead: acquired remote=192.168.1.4:477565730worker2 # [ 146.689572] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:17Z level=info event=scheduler_take_over5731worker2 # [ 147.001485] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:17Z level=info event=scheduler_restarting addr=lb:500515732worker1 # [ 147.022427] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:17Z level=info event=scheduler_restarting5733worker1 # [ 147.022783] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:17Z level=info event=scheduler_restarting addr=lb:500515734worker1 # [ 147.022962] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:440065735worker1 # [ 147.023092] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:17Z level=info event=worker_gone addr=192.168.1.4:500515736worker1 # [ 147.023250] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:440065737worker1 # [ 147.023362] nix-grpc-daemon[1311]: ts=2026-09-22T10:59:17Z level=info event=worker_gone addr=192.168.1.3:500515738worker2 # [ 147.202796] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:17Z level=info event=scheduler_connected addr=lb:500515739worker1 # [ 147.222021] nix-grpc-daemon[1311]: E0000 00:00:1790074757.693102 1356 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5740worker2 # [ 147.208410] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5741worker1 # [ 147.238212] nix-grpc-daemon[1311]: E0000 00:00:1790074757.701071 1356 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5742worker2 # [ 147.409543] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:17Z level=info event=scheduler_connected addr=lb:500515743worker2 # [ 147.414826] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5744worker2 # [ 147.615970] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:18Z level=info event=scheduler_connected addr=lb:500515745worker2 # [ 147.621416] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5746worker2 # [ 147.824227] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:18Z level=info event=scheduler_connected addr=lb:500515747worker2 # [ 147.852198] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:18Z level=info event=oidc_jwks provider=mock keys=15748worker2 # [ 147.856966] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:18Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:374565749worker2 # [ 147.862313] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:18Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05750worker1 # [ 148.933019] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5751worker1 # [ 148.938595] systemd[1]: Stopped Nix worker-protocol over gRPC.5752worker1 # [ 148.944267] systemd[1]: nix-grpc-daemon.service: Consumed 291ms CPU time over 17.698s wall clock time, 14.7M memory peak, 10.8K incoming IP traffic, 14K outgoing IP traffic.5753worker1 # [ 148.957677] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5754worker1 # [ 148.959864] systemd[1]: Closed Nix worker-protocol over gRPC.5755worker1 # [ 148.965093] systemd[1]: nix-grpc-daemon.socket: Consumed 977us CPU time over 17.744s wall clock time, 256K memory peak, 24.2K incoming IP traffic, 20.2K outgoing IP traffic.5756worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5757worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5758worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5759worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5760client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25761worker2 # [ 149.362347] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555505762client # don't know how to build these paths:5763client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5764worker2 # [ 149.373609] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505765worker2 # [ 149.401065] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:19Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05766worker2 # [ 149.408557] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:19Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15767worker2 # [ 149.448212] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505768worker2 # [ 149.463684] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505769client # copying 1 paths to 'lb:50051'...5770client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5771worker2 # [ 149.479543] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555505772worker2 # [ 149.495392] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5773worker2 # [ 149.703388] nix-daemon[866]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5774worker2 # [ 149.722734] nix-daemon[866]: reaped child process 1474, status = succeeded5775worker2 # [ 149.904935] nix-daemon[866]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)5776worker2 # [ 149.922571] nix-daemon[866]: reaped child process 1479, status = succeeded5777lb # [ 149.917161] niks3-server[1182]: 2026/09/22 10:59:20 INFO Received uploads request method=POST path=/api/pending_closures5778worker2 # [ 149.949139] nix-grpc-daemon[1330]: time=2026-09-22T10:59:20.407Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5779worker2 # [ 149.950626] nix-grpc-daemon[1330]: time=2026-09-22T10:59:20.407Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5780lb # [ 149.950981] niks3-server[1182]: 2026/09/22 10:59:20 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5781lb # [ 149.965846] niks3-server[1182]: 2026/09/22 10:59:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5782worker2 # [ 149.991243] nix-grpc-daemon[1330]: time=2026-09-22T10:59:20.450Z level=INFO msg="Uploading 1 narinfos"5783lb # [ 149.975989] niks3-server[1182]: 2026/09/22 10:59:20 INFO Signed narinfos id=43 count=15784lb # [ 149.978407] niks3-server[1182]: 2026/09/22 10:59:20 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5785lb # [ 149.991309] niks3-server[1182]: 2026/09/22 10:59:20 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5786lb # [ 149.996983] niks3-server[1182]: 2026/09/22 10:59:20 INFO Completed upload id=435787worker2 # [ 150.016897] nix-grpc-daemon[1330]: time=2026-09-22T10:59:20.476Z level=INFO msg="Upload complete. (504ms)"5788lb # [ 149.999754] niks3-server[1182]: 2026/09/22 10:59:20 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5789worker2 # [ 150.026466] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=0 paths=1 nar_bytes_in=5205790worker2 # [ 150.036701] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505791worker2 # [ 150.048651] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5792worker2 # [ 150.051226] nix-daemon[866]: reaped child process 1469, status = succeeded5793client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5794client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5795worker2 # [ 150.063331] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5796client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5797client # farm-a> LOG-farm-a-on-standby5798worker2 # [ 151.109377] nix-daemon[866]: reaped child process 1483, status = succeeded5799worker2 # [ 151.277302] nix-daemon[866]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5800worker2 # [ 151.295705] nix-daemon[866]: reaped child process 1502, status = succeeded5801lb # [ 151.289002] niks3-server[1182]: 2026/09/22 10:59:21 INFO Received uploads request method=POST path=/api/pending_closures5802worker2 # [ 151.323901] nix-grpc-daemon[1330]: time=2026-09-22T10:59:21.779Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5803worker2 # [ 151.325999] nix-grpc-daemon[1330]: time=2026-09-22T10:59:21.779Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5804lb # [ 151.326452] niks3-server[1182]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5805lb # [ 151.343286] niks3-server[1182]: 2026/09/22 10:59:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5806worker2 # [ 151.368162] nix-grpc-daemon[1330]: time=2026-09-22T10:59:21.827Z level=INFO msg="Uploading 1 narinfos"5807lb # [ 151.354023] niks3-server[1182]: 2026/09/22 10:59:21 INFO Signed narinfos id=44 count=15808lb # [ 151.359016] niks3-server[1182]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5809lb # [ 151.364801] niks3-server[1182]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5810lb # [ 151.372302] niks3-server[1182]: 2026/09/22 10:59:21 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5811lb # [ 151.376151] niks3-server[1182]: 2026/09/22 10:59:21 INFO Completed upload id=445812worker2 # [ 151.395762] nix-grpc-daemon[1330]: time=2026-09-22T10:59:21.854Z level=INFO msg="Upload complete. (323ms)"5813lb # [ 151.378757] niks3-server[1182]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5814worker2 # [ 151.417146] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15815worker2 # [ 151.432368] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:21Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05816worker2 # [ 151.435844] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:21Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15817worker2 # [ 151.442251] nix-daemon[866]: reaped child process 1481, status = succeeded5818worker2 # [ 151.459967] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505819worker2 # [ 151.475247] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505820worker2 # [ 151.489622] nix-grpc-daemon[1330]: time=2026-09-22T10:59:21.948Z level=INFO msg="All 1 paths already cached"5821client # copying 1 paths to 'lb:50051'...5822client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5823worker2 # [ 151.500521] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555505824worker2 # [ 151.515349] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5825worker2 # [ 151.720156] nix-daemon[866]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)5826worker2 # [ 151.737874] nix-daemon[866]: reaped child process 1511, status = succeeded5827worker2 # [ 151.912931] nix-daemon[866]: accepted connection from pid 1513, user nix-grpc-daemon (trusted)5828worker2 # [ 151.930716] nix-daemon[866]: reaped child process 1516, status = succeeded5829lb # [ 151.922616] niks3-server[1182]: 2026/09/22 10:59:22 INFO Received uploads request method=POST path=/api/pending_closures5830worker2 # [ 151.951925] nix-grpc-daemon[1330]: time=2026-09-22T10:59:22.410Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5831worker2 # [ 151.953682] nix-grpc-daemon[1330]: time=2026-09-22T10:59:22.410Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5832lb # [ 151.947971] niks3-server[1182]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5833lb # [ 151.963605] niks3-server[1182]: 2026/09/22 10:59:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5834worker2 # [ 151.989021] nix-grpc-daemon[1330]: time=2026-09-22T10:59:22.448Z level=INFO msg="Uploading 1 narinfos"5835lb # [ 151.978830] niks3-server[1182]: 2026/09/22 10:59:22 INFO Signed narinfos id=45 count=15836lb # [ 151.985609] niks3-server[1182]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5837lb # [ 151.992285] niks3-server[1182]: 2026/09/22 10:59:22 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5838worker2 # [ 152.018560] nix-grpc-daemon[1330]: time=2026-09-22T10:59:22.477Z level=INFO msg="Upload complete. (487ms)"5839lb # [ 151.999506] niks3-server[1182]: 2026/09/22 10:59:22 INFO Completed upload id=455840lb # [ 152.002425] niks3-server[1182]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5841worker2 # [ 152.028270] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=0 paths=1 nar_bytes_in=6405842worker2 # [ 152.039522] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505843worker2 # [ 152.046904] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5844worker2 # [ 152.051957] nix-daemon[866]: reaped child process 1506, status = succeeded5845client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5846client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5847worker2 # [ 152.066432] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5848client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5849client # farm-b> LOG-farm-b-on-standby5850worker2 # [ 153.115656] nix-daemon[866]: reaped child process 1520, status = succeeded5851worker2 # [ 153.292129] nix-daemon[866]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5852worker2 # [ 153.312588] nix-daemon[866]: reaped child process 1539, status = succeeded5853lb # [ 153.305192] niks3-server[1182]: 2026/09/22 10:59:23 INFO Received uploads request method=POST path=/api/pending_closures5854worker2 # [ 153.334744] nix-grpc-daemon[1330]: time=2026-09-22T10:59:23.793Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5855worker2 # [ 153.339844] nix-grpc-daemon[1330]: time=2026-09-22T10:59:23.795Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5856lb # [ 153.338253] niks3-server[1182]: 2026/09/22 10:59:23 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5857worker2 # [ 153.378285] nix-grpc-daemon[1330]: time=2026-09-22T10:59:23.837Z level=INFO msg="Uploading 1 narinfos"5858lb # [ 153.358133] niks3-server[1182]: 2026/09/22 10:59:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5859lb # [ 153.370297] niks3-server[1182]: 2026/09/22 10:59:23 INFO Signed narinfos id=46 count=15860lb # [ 153.377656] niks3-server[1182]: 2026/09/22 10:59:23 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5861lb # [ 153.384140] niks3-server[1182]: 2026/09/22 10:59:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5862lb # [ 153.392867] niks3-server[1182]: 2026/09/22 10:59:23 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5863lb # [ 153.398038] niks3-server[1182]: 2026/09/22 10:59:23 INFO Completed upload id=465864worker2 # [ 153.417345] nix-grpc-daemon[1330]: time=2026-09-22T10:59:23.876Z level=INFO msg="Upload complete. (339ms)"5865lb # [ 153.401271] niks3-server[1182]: 2026/09/22 10:59:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5866worker2 # [ 153.436736] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15867worker2 # [ 153.452335] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:23Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05868worker2 # [ 153.457751] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:23Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15869worker2 # [ 153.461952] nix-daemon[866]: reaped child process 1518, status = succeeded5870worker2 # [ 153.480221] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505871worker2 # [ 153.495326] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505872worker2 # [ 153.510917] nix-grpc-daemon[1330]: time=2026-09-22T10:59:23.970Z level=INFO msg="All 2 paths already cached"5873client # copying 1 paths to 'lb:50051'...5874client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5875worker2 # [ 153.522493] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555505876worker2 # [ 153.536550] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5877worker2 # [ 153.740130] nix-daemon[866]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)5878worker2 # [ 153.759683] nix-daemon[866]: reaped child process 1548, status = succeeded5879worker2 # [ 153.930053] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5880worker2 # [ 153.949369] nix-daemon[866]: reaped child process 1553, status = succeeded5881lb # [ 153.940736] niks3-server[1182]: 2026/09/22 10:59:24 INFO Received uploads request method=POST path=/api/pending_closures5882worker2 # [ 153.970710] nix-grpc-daemon[1330]: time=2026-09-22T10:59:24.429Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5883worker2 # [ 153.972280] nix-grpc-daemon[1330]: time=2026-09-22T10:59:24.429Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5884lb # [ 153.967245] niks3-server[1182]: 2026/09/22 10:59:24 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5885lb # [ 153.980738] niks3-server[1182]: 2026/09/22 10:59:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5886worker2 # [ 154.008781] nix-grpc-daemon[1330]: time=2026-09-22T10:59:24.467Z level=INFO msg="Uploading 1 narinfos"5887lb # [ 153.989131] niks3-server[1182]: 2026/09/22 10:59:24 INFO Signed narinfos id=47 count=15888lb # [ 153.997482] niks3-server[1182]: 2026/09/22 10:59:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5889lb # [ 154.006116] niks3-server[1182]: 2026/09/22 10:59:24 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5890lb # [ 154.012610] niks3-server[1182]: 2026/09/22 10:59:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5891worker2 # [ 154.035879] nix-grpc-daemon[1330]: time=2026-09-22T10:59:24.495Z level=INFO msg="Upload complete. (482ms)"5892lb # [ 154.018057] niks3-server[1182]: 2026/09/22 10:59:24 INFO Completed upload id=475893worker2 # [ 154.043373] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=0 paths=1 nar_bytes_in=7685894worker2 # [ 154.055112] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505895worker2 # [ 154.060970] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5896worker2 # [ 154.065690] nix-daemon[866]: reaped child process 1543, status = succeeded5897client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5898client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5899worker2 # [ 154.078364] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5900lb # [ 154.217166] envoy[933]: conn peer=[2001:db8:1::1]:43708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9766 tx=4444 ms=69705901lb # [ 154.222184] envoy[933]: conn peer=[2001:db8:1::3]:39560 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1028 tx=372 ms=157165902client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5903client # farm-top> LOG-farm-top-on-standby5904worker2 # [ 155.527319] nix-daemon[866]: reaped child process 1557, status = succeeded5905worker2 # [ 155.714932] nix-daemon[866]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)5906worker2 # [ 155.733991] nix-daemon[866]: reaped child process 1576, status = succeeded5907lb # [ 155.728680] niks3-server[1182]: 2026/09/22 10:59:26 INFO Received uploads request method=POST path=/api/pending_closures5908worker2 # [ 155.757070] nix-grpc-daemon[1330]: time=2026-09-22T10:59:26.214Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5909worker2 # [ 155.758578] nix-grpc-daemon[1330]: time=2026-09-22T10:59:26.214Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5910lb # [ 155.762702] niks3-server[1182]: 2026/09/22 10:59:26 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5911lb # [ 155.776362] niks3-server[1182]: 2026/09/22 10:59:26 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5912lb # [ 155.783578] niks3-server[1182]: 2026/09/22 10:59:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5913worker2 # [ 155.809096] nix-grpc-daemon[1330]: time=2026-09-22T10:59:26.267Z level=INFO msg="Uploading 1 narinfos"5914lb # [ 155.793168] niks3-server[1182]: 2026/09/22 10:59:26 INFO Signed narinfos id=48 count=15915lb # [ 155.799555] niks3-server[1182]: 2026/09/22 10:59:26 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5916lb # [ 155.807281] niks3-server[1182]: 2026/09/22 10:59:26 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5917lb # [ 155.815106] niks3-server[1182]: 2026/09/22 10:59:26 INFO Completed upload id=485918worker2 # [ 155.835065] nix-grpc-daemon[1330]: time=2026-09-22T10:59:26.294Z level=INFO msg="Upload complete. (345ms)"5919lb # [ 155.818280] niks3-server[1182]: 2026/09/22 10:59:26 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5920worker2 # [ 155.849213] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15921worker2 # [ 155.866224] nix-daemon[866]: reaped child process 1555, status = succeeded5922client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 6.80 seconds)5923client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5924client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5925worker1 # [ 156.242766] systemd[1]: Listening on Nix worker-protocol over gRPC.5926worker1 # [ 156.253402] systemd[1]: Starting Nix worker-protocol over gRPC...5927worker1 # [ 156.452600] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5928worker1 # [ 156.454615] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5929worker1 # [ 156.458718] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15930worker1 # [ 156.467028] nix-grpc-daemon[1394]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5931worker1 # [ 156.475663] nix-grpc-daemon[1394]: E0000 00:00:1790074766.936115 1394 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5932worker1 # [ 156.480476] nix-grpc-daemon[1394]: E0000 00:00:1790074766.938133 1394 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5933worker1 # [ 156.483583] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5934worker1 # [ 156.485958] systemd[1]: Started Nix worker-protocol over gRPC.5935lb: must succeed: curl -sf localhost:9901/clusters5936worker1 # [ 156.496288] nix-grpc-daemon[1394]: E0000 00:00:1790074766.958995 1433 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5937worker1 # [ 156.520783] nix-grpc-daemon[1394]: E0000 00:00:1790074766.977263 1433 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5938worker1 # [ 156.523143] nix-grpc-daemon[1394]: E0000 00:00:1790074766.977465 1433 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5939worker1 # [ 156.526619] nix-grpc-daemon[1394]: E0000 00:00:1790074766.991978 1433 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5940worker1 # [ 156.543519] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:27Z level=info event=scheduler_connected addr=lb:500515941worker2 # [ 156.527577] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:374565942worker2 # [ 156.533913] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05943lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5944worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5945worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5946lb: must succeed: curl -sf localhost:9901/clusters5947lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5948lb: must succeed: curl -sf localhost:9901/clusters5949lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5950worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5951worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5952(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.28 seconds)5953subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5954client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5955client # [ 156.888723] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5956client # Running as unit: rs.service; invocation ID: 367018a80363475093cc772703a3d2dc5957client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5958client # [ 157.150268] nix[1359]: don't know how to build these paths:5959client # [ 157.152105] nix[1359]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5960worker2 # [ 157.182945] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555505961worker2 # [ 157.195847] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:27Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05962worker2 # [ 157.202655] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:27Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15963worker2 # [ 157.245059] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505964worker2 # [ 157.261247] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555505965client # [ 157.244172] nix[1359]: copying 1 paths to 'lb:50051'...5966worker2 # [ 157.277198] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555505967client # [ 157.246562] nix[1359]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5968worker2 # [ 157.294968] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5969worker2 # [ 157.499377] nix-daemon[866]: accepted connection from pid 1589, user nix-grpc-daemon (trusted)5970worker2 # [ 157.521167] nix-daemon[866]: reaped child process 1592, status = succeeded5971worker2 # [ 157.697155] nix-daemon[866]: accepted connection from pid 1594, user nix-grpc-daemon (trusted)5972worker2 # [ 157.717353] nix-daemon[866]: reaped child process 1597, status = succeeded5973lb # [ 157.711689] niks3-server[1182]: 2026/09/22 10:59:28 INFO Received uploads request method=POST path=/api/pending_closures5974worker2 # [ 157.741757] nix-grpc-daemon[1330]: time=2026-09-22T10:59:28.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5975worker2 # [ 157.743292] nix-grpc-daemon[1330]: time=2026-09-22T10:59:28.200Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5976lb # [ 157.740479] niks3-server[1182]: 2026/09/22 10:59:28 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5977worker2 # [ 157.786350] nix-grpc-daemon[1330]: time=2026-09-22T10:59:28.245Z level=INFO msg="Uploading 1 narinfos"5978lb # [ 157.758742] niks3-server[1182]: 2026/09/22 10:59:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5979lb # [ 157.770928] niks3-server[1182]: 2026/09/22 10:59:28 INFO Signed narinfos id=49 count=15980lb # [ 157.776081] niks3-server[1182]: 2026/09/22 10:59:28 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5981lb # [ 157.784432] niks3-server[1182]: 2026/09/22 10:59:28 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5982lb # [ 157.790308] niks3-server[1182]: 2026/09/22 10:59:28 INFO Completed upload id=495983worker2 # [ 157.809770] nix-grpc-daemon[1330]: time=2026-09-22T10:59:28.268Z level=INFO msg="Upload complete. (497ms)"5984lb # [ 157.792802] niks3-server[1182]: 2026/09/22 10:59:28 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5985worker2 # [ 157.819772] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=0 paths=1 nar_bytes_in=4165986worker2 # [ 157.828270] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555505987worker2 # [ 157.840998] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5988worker2 # [ 157.846288] nix-daemon[866]: reaped child process 1587, status = succeeded5989client # [ 157.823442] nix[1359]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5990worker2 # [ 157.860456] nix-daemon[866]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5991lb: must succeed: curl -sf localhost:9901/clusters5992lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5993lb: must succeed: curl -sf localhost:9901/clusters5994lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5995lb: must succeed: curl -sf localhost:9901/clusters5996lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5997lb: must succeed: curl -sf localhost:9901/clusters5998lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5999worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6000worker1 # [ 159.932888] systemd-journald[267]: Received client request to rotate journal, rotating.6001worker1 # [ 159.954808] systemd-journald[267]: Vacuuming done, freed 0B of archived journals from /var/log/journal/3457c29705e74d53b4ef05a587961acb.6002worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)6003worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6004worker2 # [ 159.994711] systemd[1]: Stopping Nix worker-protocol over gRPC...6005worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)6006client: must succeed: sleep 5.06007worker2 # [ 160.635969] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:31Z level=info event=draining6008lb # [ 164.216572] envoy[933]: conn peer=[2001:db8:1::1]:34140 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9623 tx=4454 ms=64976009client: (finished: must succeed: sleep 5.0, in 5.06 seconds)6010worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6011worker2 # [ 165.495487] nix-daemon[866]: reaped child process 1601, status = succeeded6012worker2 # [ 165.666370] nix-daemon[866]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6013worker2 # [ 165.685189] nix-daemon[866]: reaped child process 1645, status = succeeded6014lb # [ 165.676967] niks3-server[1182]: 2026/09/22 10:59:36 INFO Received uploads request method=POST path=/api/pending_closures6015worker2 # [ 165.711882] nix-grpc-daemon[1330]: time=2026-09-22T10:59:36.167Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6016worker2 # [ 165.713548] nix-grpc-daemon[1330]: time=2026-09-22T10:59:36.167Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"6017lb # [ 165.723584] niks3-server[1182]: 2026/09/22 10:59:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6018worker2 # [ 165.754843] nix-grpc-daemon[1330]: time=2026-09-22T10:59:36.213Z level=INFO msg="Uploading 1 narinfos"6019lb # [ 165.741118] niks3-server[1182]: 2026/09/22 10:59:36 INFO Signed narinfos id=50 count=16020lb # [ 165.762571] niks3-server[1182]: 2026/09/22 10:59:36 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv6021lb # [ 165.774421] niks3-server[1182]: 2026/09/22 10:59:36 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls6022lb # [ 165.782898] niks3-server[1182]: 2026/09/22 10:59:36 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6023lb # [ 165.791002] niks3-server[1182]: 2026/09/22 10:59:36 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo6024lb # [ 165.797185] niks3-server[1182]: 2026/09/22 10:59:36 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6025lb # [ 165.803236] niks3-server[1182]: 2026/09/22 10:59:36 INFO Completed upload id=506026worker2 # [ 165.823219] nix-grpc-daemon[1330]: time=2026-09-22T10:59:36.282Z level=INFO msg="Upload complete. (346ms)"6027worker2 # [ 165.837515] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55550 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=16028worker2 # [ 165.857632] nix-daemon[866]: reaped child process 1599, status = succeeded6029client # [ 165.845766] systemd[1]: rs.service: Deactivated successfully.6030client # [ 165.850143] systemd[1]: rs.service: Consumed 184ms CPU time over 8.960s wall clock time, 41.2M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6031worker2 # [ 166.641230] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:37Z level=info event=signal_exit6032lb # [ 166.668215] niks3-server[1182]: 2026/09/22 10:59:37 INFO lead: released remote=192.168.1.4:477566033worker1 # [ 167.706036] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:38Z level=info event=scheduler_restarting addr=lb:500516034worker2 # [ 167.693578] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:38Z level=info event=scheduler_restarting6035worker2 # [ 167.698792] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:38Z level=info event=scheduler_restarting addr=lb:500516036worker2 # [ 167.699012] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:374566037worker2 # [ 167.699147] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:38Z level=info event=worker_gone addr=192.168.1.4:500516038worker2 # [ 167.699260] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:374566039worker2 # [ 167.699368] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:38Z level=info event=worker_gone addr=192.168.1.3:500516040worker2 # [ 167.725105] nix-daemon[866]: reaped child process 1322, status = succeeded6041worker2 # [ 167.749210] nix-grpc-daemon[1270]: E0000 00:00:1790074778.208116 1315 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6042worker1 # [ 167.905748] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:38Z level=info event=scheduler_connected addr=lb:500516043worker1 # [ 167.911171] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6044worker2 # [ 167.914239] nix-grpc-daemon[1270]: ts=2026-09-22T10:59:38Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6045worker1 # [ 168.111881] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:38Z level=info event=scheduler_connected addr=lb:500516046worker1 # [ 168.117046] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6047worker1 # [ 168.317820] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:38Z level=info event=scheduler_connected addr=lb:500516048worker1 # [ 168.322933] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6049worker1 # [ 168.523874] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:38Z level=info event=scheduler_connected addr=lb:500516050worker1 # [ 168.529034] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6051worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.30 seconds)6052client: waiting for success: ! systemctl is-active rs6053worker2 # [ 168.692293] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6054worker2 # [ 168.695498] systemd[1]: Stopped Nix worker-protocol over gRPC.6055worker2 # [ 168.702690] systemd[1]: nix-grpc-daemon.service: Consumed 3.087s CPU time over 42.277s wall clock time, 44.2M memory peak, 83.5K incoming IP traffic, 174.4K outgoing IP traffic.6056worker2 # [ 168.716282] systemd[1]: Starting scheduler-token.service...6057worker1 # [ 168.731762] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:39Z level=info event=scheduler_connected addr=lb:500516058worker1 # [ 168.736169] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6059client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)6060client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6061client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)6062client: must fail: journalctl -u rs | grep -q 'reconnecting'6063worker2 # [ 168.814366] systemd[1]: scheduler-token.service: Deactivated successfully.6064worker2 # [ 168.815694] systemd[1]: Finished scheduler-token.service.6065worker2 # [ 168.817136] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 95ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6066worker2 # [ 168.821167] systemd[1]: Starting Nix worker-protocol over gRPC...6067client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6068lb: must succeed: curl -sf localhost:9901/clusters6069worker1 # [ 168.932508] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:39Z level=info event=scheduler_connected addr=lb:500516070lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6071worker1 # [ 168.937767] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6072worker2 # [ 168.994872] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6073worker2 # [ 168.997550] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6074worker2 # [ 169.001601] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16075worker2 # [ 169.008057] nix-grpc-daemon[1655]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6076worker2 # [ 169.017794] nix-grpc-daemon[1655]: E0000 00:00:1790074779.467182 1655 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6077worker2 # [ 169.023678] nix-grpc-daemon[1655]: E0000 00:00:1790074779.470765 1655 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6078worker2 # [ 169.026721] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6079worker2 # [ 169.029561] systemd[1]: Started Nix worker-protocol over gRPC.6080lb # [ 169.007046] niks3-server[1182]: 2026/09/22 10:59:39 INFO lead: acquired remote=192.168.1.4:423846081worker2 # [ 169.030509] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=scheduler_take_over6082worker2 # [ 169.036068] nix-grpc-daemon[1655]: E0000 00:00:1790074779.492488 1695 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6083worker2 # [ 169.060055] nix-grpc-daemon[1655]: E0000 00:00:1790074779.509910 1695 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6084worker2 # [ 169.062398] nix-grpc-daemon[1655]: E0000 00:00:1790074779.510056 1695 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6085worker2 # [ 169.070949] nix-grpc-daemon[1655]: E0000 00:00:1790074779.529456 1695 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6086worker2 # [ 169.083391] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=scheduler_connected addr=lb:500516087worker2 # [ 169.085046] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6088worker1 # [ 169.138488] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:39Z level=info event=scheduler_connected addr=lb:500516089worker1 # [ 169.143730] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6090worker2 # [ 169.286628] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=scheduler_connected addr=lb:500516091worker2 # [ 169.291569] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6092worker1 # [ 169.344585] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:39Z level=info event=scheduler_connected addr=lb:500516093worker1 # [ 169.349766] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6094worker2 # [ 169.492853] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=scheduler_connected addr=lb:500516095worker2 # [ 169.498010] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6096worker1 # [ 169.550498] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:40Z level=info event=scheduler_connected addr=lb:500516097worker1 # [ 169.555609] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6098worker2 # [ 169.698888] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=info event=scheduler_connected addr=lb:500516099worker2 # [ 169.714587] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=info event=oidc_jwks provider=mock keys=16100worker2 # [ 169.719134] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:485026101worker2 # [ 169.724652] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06102worker1 # [ 169.757861] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:40Z level=info event=scheduler_connected addr=lb:500516103worker2 # [ 169.741476] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:485026104worker2 # [ 169.746395] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06105lb: must succeed: curl -sf localhost:9901/clusters6106lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6107worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6108worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6109(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.24 seconds)6110subtest: scheduler restart mid-build: build finishes, no double build6111client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid6112client # [ 170.124963] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.6113client # Running as unit: mid.service; invocation ID: f7587752f37d42bbb55ae726def577a26114client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)6115worker2 # [ 170.418153] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485146116worker2 # [ 170.431340] nix-daemon[866]: accepted connection from pid 1655, user nix-grpc-daemon (trusted)6117client # [ 170.415916] nix[1395]: don't know how to build these paths:6118client # [ 170.417093] nix[1395]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6119worker2 # [ 170.463509] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06120worker2 # [ 170.470736] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=16121worker2 # [ 170.508542] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485146122worker2 # [ 170.525702] nix-grpc-daemon[1655]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)6123worker2 # [ 170.553212] nix-grpc-daemon[1655]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)6124worker2 # [ 170.583779] nix-grpc-daemon[1655]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 364 ms (attempt 3/5)6125worker2 # [ 170.948298] nix-grpc-daemon[1655]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 124 ms (attempt 4/5)6126worker2 # [ 171.073903] nix-grpc-daemon[1655]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6127worker2 # [ 171.080612] nix-grpc-daemon[1655]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6128worker2 # [ 171.098592] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485146129client # [ 171.078388] nix[1395]: copying 1 paths to 'lb:50051'...6130worker2 # [ 171.117755] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485146131worker2 # [ 171.132337] nix-daemon[866]: accepted connection from pid 1655, user nix-grpc-daemon (trusted)6132worker2 # [ 171.161310] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:41Z level=info event=niks3_push_started pid=17246133worker2 # [ 171.376377] nix-grpc-daemon[1724]: time=2026-09-22T10:59:41.834Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""6134worker2 # [ 171.563934] nix-daemon[866]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6135worker2 # [ 171.579700] nix-daemon[866]: reaped child process 1744, status = succeeded6136client # [ 171.084227] nix[1395]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...6137worker2 # [ 171.755790] nix-daemon[866]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6138worker2 # [ 171.771134] nix-daemon[866]: reaped child process 1749, status = succeeded6139lb # [ 171.767515] niks3-server[1182]: 2026/09/22 10:59:42 INFO Received uploads request method=POST path=/api/pending_closures6140worker2 # [ 171.804067] nix-grpc-daemon[1724]: time=2026-09-22T10:59:42.262Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6141worker2 # [ 171.805561] nix-grpc-daemon[1724]: time=2026-09-22T10:59:42.262Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"6142lb # [ 171.813948] niks3-server[1182]: 2026/09/22 10:59:42 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst6143worker2 # [ 171.844064] nix-grpc-daemon[1724]: time=2026-09-22T10:59:42.302Z level=INFO msg="Uploading 1 narinfos"6144lb # [ 171.829373] niks3-server[1182]: 2026/09/22 10:59:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6145lb # [ 171.839893] niks3-server[1182]: 2026/09/22 10:59:42 INFO Signed narinfos id=51 count=16146lb # [ 171.850162] niks3-server[1182]: 2026/09/22 10:59:42 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls6147lb # [ 171.855188] niks3-server[1182]: 2026/09/22 10:59:42 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6148lb # [ 171.860043] niks3-server[1182]: 2026/09/22 10:59:42 INFO Completed upload id=516149worker2 # [ 171.879242] nix-grpc-daemon[1724]: time=2026-09-22T10:59:42.338Z level=INFO msg="Upload complete. (499ms)"6150lb # [ 171.862179] niks3-server[1182]: 2026/09/22 10:59:42 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo6151worker2 # [ 171.890335] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48514 duration_s=0 paths=1 nar_bytes_in=4246152worker2 # [ 171.903706] nix-grpc-daemon[1655]: ts=2026-09-22T10:59:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485146153worker2 # [ 171.911967] nix-daemon[866]: accepted connection from pid 1655, user nix-grpc-daemon (trusted)6154worker2 # [ 171.914070] nix-daemon[866]: reaped child process 1722, status = succeeded6155client # [ 171.891624] nix[1395]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6156worker2 # [ 171.927185] nix-daemon[866]: accepted connection from pid 1655, user nix-grpc-daemon (trusted)6157lb: must succeed: curl -sf localhost:9901/clusters6158lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6159lb: must succeed: curl -sf localhost:9901/clusters6160lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6161worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6162worker2 # [ 174.165452] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1655 (nix-grpc-daemon) on client request.6163worker2 # [ 174.167025] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6164lb # [ 174.154058] niks3-server[1182]: 2026/09/22 10:59:44 INFO lead: released remote=192.168.1.4:423846165worker2 # [ 174.190809] systemd[1]: Starting scheduler-token.service...6166client # [ 174.156662] nix[1395]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6167worker1 # [ 174.206203] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:44Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6168client # [ 174.163142] nix[1395]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6169worker2 # [ 174.207044] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6170worker2 # [ 174.212509] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6171worker2 # [ 174.213783] systemd[1]: nix-grpc-daemon.service: Consumed 515ms CPU time over 5.381s wall clock time, 34.1M memory peak, 13.9K incoming IP traffic, 21.1K outgoing IP traffic.6172worker2 # [ 174.230470] nix-daemon[866]: reaped child process 1714, status = succeeded6173worker2 # [ 174.231913] nix-daemon[866]: reaped child process 1751, status = succeeded6174lb # [ 174.214261] envoy[933]: conn peer=[2001:db8:1::1]:57544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1518 ms=86656175worker2 # [ 174.240564] nix-daemon[1753]: unexpected Nix daemon error: error: interrupted by the user6176lb # [ 174.219392] envoy[933]: conn peer=[2001:db8:1::4]:49774 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=21617 tx=1696 ms=412136177lb # [ 174.224627] envoy[933]: conn peer=[2001:db8:1::4]:46340 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3384 tx=383 ms=51126178worker2 # [ 174.248132] nix-daemon[866]: reaped child process 1753, status = failed with exit code 16179worker2 # [ 174.295693] systemd[1]: scheduler-token.service: Deactivated successfully.6180worker2 # [ 174.297752] systemd[1]: Finished scheduler-token.service.6181worker2 # [ 174.298509] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 105ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6182worker2 # [ 174.302777] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6183worker2 # [ 174.306368] systemd[1]: Starting scheduler-token.service...6184worker2 # [ 174.378125] systemd[1]: scheduler-token.service: Deactivated successfully.6185worker2 # [ 174.381777] systemd[1]: Finished scheduler-token.service.6186worker2 # [ 174.382565] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 72ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6187worker2 # [ 174.386276] systemd[1]: Starting Nix worker-protocol over gRPC...6188worker1 # [ 174.406091] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:44Z level=info event=scheduler_connected addr=lb:500516189worker2 # [ 174.554335] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6190worker2 # [ 174.556960] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6191worker2 # [ 174.561039] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16192worker2 # [ 174.568064] nix-grpc-daemon[1796]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6193worker2 # [ 174.580175] nix-grpc-daemon[1796]: E0000 00:00:1790074785.026761 1796 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6194worker2 # [ 174.586416] nix-grpc-daemon[1796]: E0000 00:00:1790074785.030405 1796 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6195lb # [ 174.568330] niks3-server[1182]: 2026/09/22 10:59:45 INFO lead: acquired remote=192.168.1.4:473546196worker2 # [ 174.589964] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6197worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.48 seconds)6198worker1 # [ 174.604789] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6199client: must succeed: sleep 5.06200worker2 # [ 174.591962] systemd[1]: Started Nix worker-protocol over gRPC.6201worker2 # [ 174.593840] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=scheduler_take_over6202worker2 # [ 174.596102] nix-grpc-daemon[1796]: E0000 00:00:1790074785.054932 1836 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6203worker2 # [ 174.624061] nix-grpc-daemon[1796]: E0000 00:00:1790074785.072377 1836 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6204worker2 # [ 174.626401] nix-grpc-daemon[1796]: E0000 00:00:1790074785.072524 1836 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6205worker2 # [ 174.633343] nix-grpc-daemon[1796]: E0000 00:00:1790074785.091786 1836 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6206worker2 # [ 174.645788] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=scheduler_connected addr=lb:500516207worker2 # [ 174.651616] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=oidc_jwks provider=mock keys=16208worker2 # [ 174.653194] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:535606209worker2 # [ 174.654859] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06210worker2 # [ 174.697200] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06211worker2 # [ 174.699416] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=16212worker2 # [ 174.756055] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535766213worker2 # [ 174.767608] nix-daemon[866]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6214worker2 # [ 174.784195] nix-daemon[866]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6215worker1 # [ 174.805794] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:45Z level=info event=scheduler_connected addr=lb:500516216worker2 # [ 174.790579] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:535606217worker2 # [ 174.797460] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06218client # [ 174.765516] nix[1395]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6219worker2 # [ 174.805268] nix-daemon[866]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6220client: (finished: must succeed: sleep 5.0, in 5.06 seconds)6221client: waiting for success: ! systemctl is-active mid6222worker2 # [ 180.022879] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:50Z level=info event=niks3_push_started pid=18796223worker2 # [ 180.041429] nix-daemon[866]: reaped child process 1854, status = succeeded6224worker2 # [ 180.224075] nix-grpc-daemon[1879]: time=2026-09-22T10:59:50.682Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""6225worker2 # [ 180.399810] nix-daemon[866]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6226worker2 # [ 180.417523] nix-daemon[866]: reaped child process 1895, status = succeeded6227lb # [ 180.411066] niks3-server[1182]: 2026/09/22 10:59:50 INFO Received uploads request method=POST path=/api/pending_closures6228worker2 # [ 180.442681] nix-grpc-daemon[1879]: time=2026-09-22T10:59:50.900Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6229worker2 # [ 180.445484] nix-grpc-daemon[1879]: time=2026-09-22T10:59:50.900Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6230lb # [ 180.450787] niks3-server[1182]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6231lb # [ 180.463770] niks3-server[1182]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6232lb # [ 180.478669] niks3-server[1182]: 2026/09/22 10:59:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6233worker2 # [ 180.506149] nix-grpc-daemon[1879]: time=2026-09-22T10:59:50.963Z level=INFO msg="Uploading 1 narinfos"6234lb # [ 180.488569] niks3-server[1182]: 2026/09/22 10:59:50 INFO Signed narinfos id=52 count=16235lb # [ 180.497469] niks3-server[1182]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6236lb # [ 180.507157] niks3-server[1182]: 2026/09/22 10:59:50 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6237lb # [ 180.516579] niks3-server[1182]: 2026/09/22 10:59:50 INFO Completed upload id=526238worker2 # [ 180.536517] nix-grpc-daemon[1879]: time=2026-09-22T10:59:50.995Z level=INFO msg="Upload complete. (308ms)"6239worker2 # [ 180.552079] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53576 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16240worker2 # [ 180.567216] nix-daemon[866]: reaped child process 1852, status = succeeded6241client # [ 180.557632] systemd[1]: mid.service: Deactivated successfully.6242client # [ 180.561892] systemd[1]: mid.service: Consumed 191ms CPU time over 10.437s wall clock time, 37.2M memory peak, 8.7K incoming IP traffic, 10.4K outgoing IP traffic.6243client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)6244client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6245client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)6246(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.07 seconds)6247subtest: interrupting the client stops the build on the worker6248client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr6249client # [ 181.168289] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.6250client # Running as unit: intr.service; invocation ID: 0248c014c0c342a0afffb908baca19ce6251client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.05 seconds)6252worker1 # [ 181.469581] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:602046253worker1 # [ 181.485295] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6254client # [ 181.454997] nix[1431]: don't know how to build these paths:6255client # [ 181.456841] nix[1431]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6256worker2 # [ 181.498605] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:51Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06257worker2 # [ 181.504628] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:51Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=16258worker1 # [ 181.561065] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046259worker1 # [ 181.584220] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5)6260worker1 # [ 181.624687] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 112 ms (attempt 2/5)6261worker1 # [ 181.744312] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 388 ms (attempt 3/5)6262worker1 # [ 182.132716] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 376 ms (attempt 4/5)6263worker1 # [ 182.510574] nix-grpc-daemon[1394]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6264worker1 # [ 182.517181] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6265worker1 # [ 182.536299] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046266client # [ 182.501233] nix[1431]: copying 1 paths to 'lb:50051'...6267worker1 # [ 182.551717] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046268client # [ 182.506605] nix[1431]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6269worker1 # [ 182.567762] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6270worker1 # [ 182.595033] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:53Z level=info event=niks3_push_started pid=15466271worker1 # [ 182.803238] nix-grpc-daemon[1546]: time=2026-09-22T10:59:53.274Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""6272worker1 # [ 182.978273] nix-daemon[844]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6273worker1 # [ 182.995419] nix-daemon[844]: reaped child process 1562, status = succeeded6274worker1 # [ 183.171725] nix-daemon[844]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6275worker1 # [ 183.187548] nix-daemon[844]: reaped child process 1567, status = succeeded6276lb # [ 183.166843] niks3-server[1182]: 2026/09/22 10:59:53 INFO Received uploads request method=POST path=/api/pending_closures6277worker1 # [ 183.219891] nix-grpc-daemon[1546]: time=2026-09-22T10:59:53.689Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6278worker1 # [ 183.221673] nix-grpc-daemon[1546]: time=2026-09-22T10:59:53.689Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6279lb # [ 183.228597] niks3-server[1182]: 2026/09/22 10:59:53 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6280worker1 # [ 183.278460] nix-grpc-daemon[1546]: time=2026-09-22T10:59:53.749Z level=INFO msg="Uploading 1 narinfos"6281lb # [ 183.241762] niks3-server[1182]: 2026/09/22 10:59:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6282lb # [ 183.250117] niks3-server[1182]: 2026/09/22 10:59:53 INFO Signed narinfos id=53 count=16283lb # [ 183.255197] niks3-server[1182]: 2026/09/22 10:59:53 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6284lb # [ 183.268198] niks3-server[1182]: 2026/09/22 10:59:53 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6285lb # [ 183.272471] niks3-server[1182]: 2026/09/22 10:59:53 INFO Completed upload id=536286worker1 # [ 183.309337] nix-grpc-daemon[1546]: time=2026-09-22T10:59:53.780Z level=INFO msg="Upload complete. (501ms)"6287lb # [ 183.276856] niks3-server[1182]: 2026/09/22 10:59:53 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6288worker1 # [ 183.320365] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=4246289worker1 # [ 183.337020] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046290worker1 # [ 183.349863] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6291worker1 # [ 183.353726] nix-daemon[844]: reaped child process 1544, status = succeeded6292client # [ 183.315214] nix[1431]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6293worker1 # [ 183.369847] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6294lb # [ 184.215929] envoy[933]: conn peer=[2001:db8:1::1]:50556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1942 ms=101506295client: must succeed: systemctl kill -s INT intr6296worker1 # [ 184.933141] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:55Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6297client # [ 184.881256] systemd[1]: intr.service: Sent signal SIGINT to main process 1431 (nix) on client request.6298client # [ 184.888847] nix[1431]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6299client # [ 184.900639] nix[1431]: error: interrupted by the user6300client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6301client # [ 184.933608] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6302client # [ 184.937666] systemd[1]: intr.service: Failed with result 'exit-code'.6303client # [ 184.939636] systemd[1]: intr.service: Consumed 159ms CPU time over 3.768s wall clock time, 39.2M memory peak, 6.5K incoming IP traffic, 8.4K outgoing IP traffic.6304worker1 # [ 185.012676] nix-daemon[844]: reaped child process 1569, status = succeeded6305worker1 # [ 185.018239] nix-daemon[1571]: unexpected Nix daemon error: error: interrupted by the user6306worker1 # [ 185.026617] nix-daemon[844]: reaped child process 1571, status = failed with exit code 16307(finished: subtest: interrupting the client stops the build on the worker, in 3.94 seconds)6308subtest: a deploy mid-build drains: the build finishes, the new generation takes over6309client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw6310client # [ 185.129597] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.6311client # Running as unit: sw.service; invocation ID: f28d13f892e84d8f976b8fd05b8404ce6312client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)6313client # [ 185.383750] nix[1452]: don't know how to build these paths:6314client # [ 185.384920] nix[1452]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6315worker2 # [ 185.415859] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:535766316worker2 # [ 185.429545] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:55Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06317worker2 # [ 185.436587] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:55Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=16318worker2 # [ 185.476158] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535766319worker2 # [ 185.493312] nix-grpc-daemon[1796]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5)6320worker2 # [ 185.588879] nix-grpc-daemon[1796]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 2/5)6321worker2 # [ 185.718460] nix-grpc-daemon[1796]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 119 ms (attempt 3/5)6322worker2 # [ 185.845121] nix-grpc-daemon[1796]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 524 ms (attempt 4/5)6323worker2 # [ 186.376919] nix-grpc-daemon[1796]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6324worker2 # [ 186.382724] nix-grpc-daemon[1796]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6325worker2 # [ 186.401985] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535766326client # [ 186.385253] nix[1452]: copying 1 paths to 'lb:50051'...6327worker2 # [ 186.418619] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535766328client # [ 186.385567] nix[1452]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6329worker2 # [ 186.442299] nix-daemon[866]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6330worker2 # [ 186.656974] nix-daemon[866]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6331worker2 # [ 186.676734] nix-daemon[866]: reaped child process 1938, status = succeeded6332worker2 # [ 186.857674] nix-daemon[866]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6333worker2 # [ 186.874687] nix-daemon[866]: reaped child process 1943, status = succeeded6334lb # [ 186.867947] niks3-server[1182]: 2026/09/22 10:59:57 INFO Received uploads request method=POST path=/api/pending_closures6335worker2 # [ 186.899199] nix-grpc-daemon[1879]: time=2026-09-22T10:59:57.357Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6336worker2 # [ 186.900777] nix-grpc-daemon[1879]: time=2026-09-22T10:59:57.357Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6337lb # [ 186.898809] niks3-server[1182]: 2026/09/22 10:59:57 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6338lb # [ 186.915616] niks3-server[1182]: 2026/09/22 10:59:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6339worker2 # [ 186.941073] nix-grpc-daemon[1879]: time=2026-09-22T10:59:57.399Z level=INFO msg="Uploading 1 narinfos"6340lb # [ 186.927936] niks3-server[1182]: 2026/09/22 10:59:57 INFO Signed narinfos id=54 count=16341lb # [ 186.934793] niks3-server[1182]: 2026/09/22 10:59:57 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6342lb # [ 186.944286] niks3-server[1182]: 2026/09/22 10:59:57 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6343lb # [ 186.950254] niks3-server[1182]: 2026/09/22 10:59:57 INFO Completed upload id=546344lb # [ 186.952832] niks3-server[1182]: 2026/09/22 10:59:57 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6345worker2 # [ 186.969216] nix-grpc-daemon[1879]: time=2026-09-22T10:59:57.428Z level=INFO msg="Upload complete. (497ms)"6346worker2 # [ 186.980741] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53576 duration_s=0 paths=1 nar_bytes_in=4166347worker2 # [ 186.991640] nix-grpc-daemon[1796]: ts=2026-09-22T10:59:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535766348worker2 # [ 186.998889] nix-daemon[866]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6349worker2 # [ 187.003692] nix-daemon[866]: reaped child process 1932, status = succeeded6350client # [ 186.980345] nix[1452]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6351worker2 # [ 187.016992] nix-daemon[866]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6352worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26353worker2 # Checking switch inhibitors... done6354worker2 # activating the configuration...6355worker2 # [ 189.834261] nixos[1979]: switching to system configuration /nix/store/3y6vfndpcj9aa9jcwmk2jinamvf8rfcl-nixos-system-worker2-test6356worker2 # [ 189.840138] systemd[1]: Stopped target Remote File Systems.6357worker2 # [ 189.840933] systemd[1]: Stopped target Local File Systems.6358worker2 # setting up /etc...6359worker2 # [ 190.613272] systemd[1]: Reload requested from client PID 1979 ('.switch-to-conf') (unit backdoor.service)...6360worker2 # [ 190.615126] systemd[1]: Reloading...6361worker2 # [ 191.516517] systemd[1]: Reloading finished in 897 ms.6362worker2 # restarting sysinit-reactivation.target6363worker2 # [ 191.597915] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6364worker2 # [ 191.749852] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6365worker2 # [ 191.751041] systemd[1]: Reached target Reactivate sysinit units.6366worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6367worker2 # [ 191.760251] dbus-broker-launch[503]: Caught SIGHUP, trigger reload.6368worker2 # [ 191.768968] systemd[1]: Reloading D-Bus System Message Bus...6369worker2 # [ 191.771410] systemd[1]: Reloading Nix worker-protocol over gRPC...6370worker2 # [ 191.786685] dbus-broker-launch[503]: Looking up NSS user entry for 'systemd-timesync'...6371worker2 # [ 191.792404] dbus-broker-launch[503]: NSS returned no entry for 'systemd-timesync'6372worker2 # [ 191.802198] dbus-broker-launch[503]: Invalid user-name in /nix/store/602n2r0g7ngz2pj9sxi9qamwpihwymx6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6373worker2 # [ 191.817178] systemd[1]: Reloaded Nix worker-protocol over gRPC.6374worker2 # [ 191.849809] systemd[1]: Reloaded D-Bus System Message Bus.6375worker2 # [ 191.855759] systemd[1]: Reached target Remote File Systems.6376worker2 # [ 191.858942] systemd[1]: Starting linger-users.service...6377worker2 # [ 191.864398] systemd[1]: Reached target Local File Systems.6378worker2 # [ 191.868497] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6379worker2 # [ 191.880461] systemd[1]: Starting Create SUID/SGID Wrappers...6380worker2 # [ 191.883419] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6381worker2 # [ 191.892180] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676382worker2 # [ 191.900264] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6383worker2 # [ 191.903536] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6384worker2 # [ 191.908625] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6385worker2 # [ 191.911621] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6386worker2 # [ 191.916681] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6387worker2 # [ 191.950146] systemd[1]: linger-users.service: Deactivated successfully.6388worker2 # [ 191.951094] systemd[1]: Finished linger-users.service.6389worker2 # [ 192.406588] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6390worker2 # [ 192.407672] systemd[1]: Finished Create SUID/SGID Wrappers.6391worker2 # [ 192.580078] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:03Z level=info event=draining6392worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6393worker2 # [ 192.663335] nixos[1979]: finished switching to system configuration /nix/store/3y6vfndpcj9aa9jcwmk2jinamvf8rfcl-nixos-system-worker2-test6394worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.68 seconds)6395worker2: must succeed: pgrep -f 'read -t [9]0 x'6396worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)6397client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&26398worker2 # [ 192.999304] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:535766399client # don't know how to build these paths:6400client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6401worker1 # [ 193.025357] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046402worker2 # [ 193.032462] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:03Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06403worker2 # [ 193.034357] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:03Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16404worker1 # [ 193.101822] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046405worker1 # [ 193.116537] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046406client # copying 1 paths to 'lb:50051'...6407worker1 # [ 193.132405] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046408client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6409worker1 # [ 193.157163] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6410worker1 # [ 193.374192] nix-daemon[844]: accepted connection from pid 1622, user nix-grpc-daemon (trusted)6411worker1 # [ 193.393052] nix-daemon[844]: reaped child process 1625, status = succeeded6412worker1 # [ 193.571964] nix-daemon[844]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6413worker1 # [ 193.589772] nix-daemon[844]: reaped child process 1630, status = succeeded6414lb # [ 193.566816] niks3-server[1182]: 2026/09/22 11:00:04 INFO Received uploads request method=POST path=/api/pending_closures6415worker1 # [ 193.614570] nix-grpc-daemon[1546]: time=2026-09-22T11:00:04.085Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6416worker1 # [ 193.616160] nix-grpc-daemon[1546]: time=2026-09-22T11:00:04.085Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6417lb # [ 193.595095] niks3-server[1182]: 2026/09/22 11:00:04 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6418lb # [ 193.609590] niks3-server[1182]: 2026/09/22 11:00:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6419worker1 # [ 193.653316] nix-grpc-daemon[1546]: time=2026-09-22T11:00:04.124Z level=INFO msg="Uploading 1 narinfos"6420lb # [ 193.622688] niks3-server[1182]: 2026/09/22 11:00:04 INFO Signed narinfos id=55 count=16421lb # [ 193.630012] niks3-server[1182]: 2026/09/22 11:00:04 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6422lb # [ 193.641565] niks3-server[1182]: 2026/09/22 11:00:04 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6423lb # [ 193.646334] niks3-server[1182]: 2026/09/22 11:00:04 INFO Completed upload id=556424worker1 # [ 193.683047] nix-grpc-daemon[1546]: time=2026-09-22T11:00:04.154Z level=INFO msg="Upload complete. (509ms)"6425lb # [ 193.649080] niks3-server[1182]: 2026/09/22 11:00:04 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6426worker1 # [ 193.693381] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=5286427worker1 # [ 193.706766] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046428worker1 # [ 193.713314] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6429worker1 # [ 193.718543] nix-daemon[844]: reaped child process 1620, status = succeeded6430client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6431client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6432worker1 # [ 193.732398] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6433lb # [ 194.215049] envoy[933]: conn peer=[2001:db8:1::1]:47728 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=851 ms=35036434client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6435client # farm-a> LOG-farm-a-during-drain6436worker1 # [ 195.160385] nix-daemon[844]: reaped child process 1634, status = succeeded6437worker1 # [ 195.322015] nix-daemon[844]: accepted connection from pid 1650, user nix-grpc-daemon (trusted)6438worker1 # [ 195.340849] nix-daemon[844]: reaped child process 1653, status = succeeded6439lb # [ 195.317729] niks3-server[1182]: 2026/09/22 11:00:05 INFO Received uploads request method=POST path=/api/pending_closures6440worker1 # [ 195.367270] nix-grpc-daemon[1546]: time=2026-09-22T11:00:05.835Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6441worker1 # [ 195.369513] nix-grpc-daemon[1546]: time=2026-09-22T11:00:05.835Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6442lb # [ 195.351877] niks3-server[1182]: 2026/09/22 11:00:05 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6443lb # [ 195.368962] niks3-server[1182]: 2026/09/22 11:00:05 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6444worker1 # [ 195.418621] nix-grpc-daemon[1546]: time=2026-09-22T11:00:05.888Z level=INFO msg="Uploading 1 narinfos"6445lb # [ 195.383142] niks3-server[1182]: 2026/09/22 11:00:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6446lb # [ 195.391351] niks3-server[1182]: 2026/09/22 11:00:05 INFO Signed narinfos id=56 count=16447worker1 # [ 195.438127] nix-grpc-daemon[1546]: time=2026-09-22T11:00:05.909Z level=INFO msg="Upload complete. (295ms)"6448lb # [ 195.401719] niks3-server[1182]: 2026/09/22 11:00:05 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6449lb # [ 195.406218] niks3-server[1182]: 2026/09/22 11:00:05 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6450lb # [ 195.411787] niks3-server[1182]: 2026/09/22 11:00:05 INFO Completed upload id=566451worker1 # [ 195.448928] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16452lb # [ 195.414705] niks3-server[1182]: 2026/09/22 11:00:05 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6453worker2 # [ 195.441122] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:05Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06454worker2 # [ 195.442939] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:05Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16455worker1 # [ 195.462985] nix-daemon[844]: reaped child process 1632, status = succeeded6456worker1 # [ 195.505693] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046457worker1 # [ 195.520559] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046458worker1 # [ 195.535543] nix-grpc-daemon[1546]: time=2026-09-22T11:00:06.006Z level=INFO msg="All 1 paths already cached"6459client # copying 1 paths to 'lb:50051'...6460worker1 # [ 195.546802] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046461client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6462worker1 # [ 195.569727] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6463worker1 # [ 195.777406] nix-daemon[844]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6464worker1 # [ 195.795365] nix-daemon[844]: reaped child process 1660, status = succeeded6465worker1 # [ 195.969691] nix-daemon[844]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)6466worker1 # [ 195.987806] nix-daemon[844]: reaped child process 1665, status = succeeded6467lb # [ 195.963171] niks3-server[1182]: 2026/09/22 11:00:06 INFO Received uploads request method=POST path=/api/pending_closures6468worker1 # [ 196.010914] nix-grpc-daemon[1546]: time=2026-09-22T11:00:06.481Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6469worker1 # [ 196.012533] nix-grpc-daemon[1546]: time=2026-09-22T11:00:06.481Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6470lb # [ 196.001413] niks3-server[1182]: 2026/09/22 11:00:06 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6471lb # [ 196.012663] niks3-server[1182]: 2026/09/22 11:00:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6472worker1 # [ 196.055306] nix-grpc-daemon[1546]: time=2026-09-22T11:00:06.526Z level=INFO msg="Uploading 1 narinfos"6473lb # [ 196.024086] niks3-server[1182]: 2026/09/22 11:00:06 INFO Signed narinfos id=57 count=16474lb # [ 196.026585] niks3-server[1182]: 2026/09/22 11:00:06 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6475lb # [ 196.035791] niks3-server[1182]: 2026/09/22 11:00:06 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6476lb # [ 196.043173] niks3-server[1182]: 2026/09/22 11:00:06 INFO Completed upload id=576477lb # [ 196.046025] niks3-server[1182]: 2026/09/22 11:00:06 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6478worker1 # [ 196.079175] nix-grpc-daemon[1546]: time=2026-09-22T11:00:06.550Z level=INFO msg="Upload complete. (492ms)"6479worker1 # [ 196.088482] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=6406480worker1 # [ 196.103065] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046481worker1 # [ 196.109729] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6482worker1 # [ 196.114721] nix-daemon[844]: reaped child process 1655, status = succeeded6483client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6484worker1 # [ 196.126010] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6485client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6486client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6487client # farm-b> LOG-farm-b-during-drain6488worker1 # [ 196.962669] nix-daemon[844]: reaped child process 1669, status = succeeded6489worker1 # [ 197.132117] nix-daemon[844]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6490worker1 # [ 197.150842] nix-daemon[844]: reaped child process 1688, status = succeeded6491lb # [ 197.126989] niks3-server[1182]: 2026/09/22 11:00:07 INFO Received uploads request method=POST path=/api/pending_closures6492worker1 # [ 197.173023] nix-grpc-daemon[1546]: time=2026-09-22T11:00:07.643Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6493worker1 # [ 197.175263] nix-grpc-daemon[1546]: time=2026-09-22T11:00:07.645Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6494lb # [ 197.159274] niks3-server[1182]: 2026/09/22 11:00:07 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6495lb # [ 197.172978] niks3-server[1182]: 2026/09/22 11:00:07 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6496lb # [ 197.188769] niks3-server[1182]: 2026/09/22 11:00:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6497worker1 # [ 197.231861] nix-grpc-daemon[1546]: time=2026-09-22T11:00:07.702Z level=INFO msg="Uploading 1 narinfos"6498lb # [ 197.200773] niks3-server[1182]: 2026/09/22 11:00:07 INFO Signed narinfos id=58 count=16499lb # [ 197.207377] niks3-server[1182]: 2026/09/22 11:00:07 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6500worker1 # [ 197.253737] nix-grpc-daemon[1546]: time=2026-09-22T11:00:07.724Z level=INFO msg="Upload complete. (315ms)"6501lb # [ 197.217658] niks3-server[1182]: 2026/09/22 11:00:07 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6502lb # [ 197.221983] niks3-server[1182]: 2026/09/22 11:00:07 INFO Completed upload id=586503lb # [ 197.226415] niks3-server[1182]: 2026/09/22 11:00:07 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6504worker2 # [ 197.257118] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:07Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06505worker2 # [ 197.259042] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:07Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16506worker1 # [ 197.265107] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16507worker1 # [ 197.285036] nix-daemon[844]: reaped child process 1667, status = succeeded6508worker1 # [ 197.320631] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046509worker1 # [ 197.336778] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046510worker1 # [ 197.351923] nix-grpc-daemon[1546]: time=2026-09-22T11:00:07.823Z level=INFO msg="All 2 paths already cached"6511client # copying 1 paths to 'lb:50051'...6512client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6513worker1 # [ 197.362907] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046514worker1 # [ 197.380830] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6515worker1 # [ 197.583328] nix-daemon[844]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6516worker1 # [ 197.600337] nix-daemon[844]: reaped child process 1695, status = succeeded6517worker1 # [ 197.770351] nix-daemon[844]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6518worker1 # [ 197.786273] nix-daemon[844]: reaped child process 1700, status = succeeded6519lb # [ 197.762468] niks3-server[1182]: 2026/09/22 11:00:08 INFO Received uploads request method=POST path=/api/pending_closures6520worker1 # [ 197.808168] nix-grpc-daemon[1546]: time=2026-09-22T11:00:08.279Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6521worker1 # [ 197.810017] nix-grpc-daemon[1546]: time=2026-09-22T11:00:08.280Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6522lb # [ 197.791385] niks3-server[1182]: 2026/09/22 11:00:08 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6523worker1 # [ 197.840569] nix-grpc-daemon[1546]: time=2026-09-22T11:00:08.311Z level=INFO msg="Uploading 1 narinfos"6524lb # [ 197.806418] niks3-server[1182]: 2026/09/22 11:00:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6525lb # [ 197.818269] niks3-server[1182]: 2026/09/22 11:00:08 INFO Signed narinfos id=59 count=16526lb # [ 197.827786] niks3-server[1182]: 2026/09/22 11:00:08 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6527worker1 # [ 197.869665] nix-grpc-daemon[1546]: time=2026-09-22T11:00:08.340Z level=INFO msg="Upload complete. (471ms)"6528lb # [ 197.833642] niks3-server[1182]: 2026/09/22 11:00:08 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6529lb # [ 197.838972] niks3-server[1182]: 2026/09/22 11:00:08 INFO Completed upload id=596530worker1 # [ 197.875213] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=7686531lb # [ 197.842299] niks3-server[1182]: 2026/09/22 11:00:08 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6532worker1 # [ 197.887137] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046533worker1 # [ 197.893228] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6534worker1 # [ 197.897660] nix-daemon[844]: reaped child process 1690, status = succeeded6535client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6536worker1 # [ 197.909401] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6537client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6538client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6539client # farm-top> LOG-farm-top-during-drain6540worker1 # [ 199.132583] nix-daemon[844]: reaped child process 1704, status = succeeded6541worker1 # [ 199.293064] nix-daemon[844]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6542worker1 # [ 199.313200] nix-daemon[844]: reaped child process 1723, status = succeeded6543lb # [ 199.289057] niks3-server[1182]: 2026/09/22 11:00:09 INFO Received uploads request method=POST path=/api/pending_closures6544worker1 # [ 199.336385] nix-grpc-daemon[1546]: time=2026-09-22T11:00:09.806Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6545worker1 # [ 199.337835] nix-grpc-daemon[1546]: time=2026-09-22T11:00:09.806Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6546lb # [ 199.327541] niks3-server[1182]: 2026/09/22 11:00:09 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6547lb # [ 199.340113] niks3-server[1182]: 2026/09/22 11:00:09 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6548worker1 # [ 199.391949] nix-grpc-daemon[1546]: time=2026-09-22T11:00:09.862Z level=INFO msg="Uploading 1 narinfos"6549lb # [ 199.349806] niks3-server[1182]: 2026/09/22 11:00:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6550lb # [ 199.359778] niks3-server[1182]: 2026/09/22 11:00:09 INFO Signed narinfos id=60 count=16551lb # [ 199.366178] niks3-server[1182]: 2026/09/22 11:00:09 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6552lb # [ 199.375363] niks3-server[1182]: 2026/09/22 11:00:09 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6553lb # [ 199.381065] niks3-server[1182]: 2026/09/22 11:00:09 INFO Completed upload id=606554worker1 # [ 199.417821] nix-grpc-daemon[1546]: time=2026-09-22T11:00:09.889Z level=INFO msg="Upload complete. (299ms)"6555lb # [ 199.384526] niks3-server[1182]: 2026/09/22 11:00:09 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6556worker1 # [ 199.429159] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16557worker1 # [ 199.447172] nix-daemon[844]: reaped child process 1702, status = succeeded6558client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.73 seconds)6559client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath6560client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)6561worker2: must succeed: pkill -f 'read -t [9]0 x'6562worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6563client: waiting for success: ! systemctl is-active sw6564worker2 # [ 199.863081] nix-daemon[866]: reaped child process 1947, status = succeeded6565worker2 # [ 200.028180] nix-daemon[866]: accepted connection from pid 2126, user nix-grpc-daemon (trusted)6566worker2 # [ 200.047873] nix-daemon[866]: reaped child process 2129, status = succeeded6567lb # [ 200.039695] niks3-server[1182]: 2026/09/22 11:00:10 INFO Received uploads request method=POST path=/api/pending_closures6568worker2 # [ 200.070166] nix-grpc-daemon[1879]: time=2026-09-22T11:00:10.528Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6569worker2 # [ 200.071680] nix-grpc-daemon[1879]: time=2026-09-22T11:00:10.528Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6570lb # [ 200.076058] niks3-server[1182]: 2026/09/22 11:00:10 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6571lb # [ 200.087921] niks3-server[1182]: 2026/09/22 11:00:10 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6572lb # [ 200.098286] niks3-server[1182]: 2026/09/22 11:00:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6573lb # [ 200.104771] niks3-server[1182]: 2026/09/22 11:00:10 INFO Signed narinfos id=61 count=16574worker2 # [ 200.125290] nix-grpc-daemon[1879]: time=2026-09-22T11:00:10.583Z level=INFO msg="Uploading 1 narinfos"6575lb # [ 200.111104] niks3-server[1182]: 2026/09/22 11:00:10 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6576lb # [ 200.121718] niks3-server[1182]: 2026/09/22 11:00:10 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6577lb # [ 200.129265] niks3-server[1182]: 2026/09/22 11:00:10 INFO Completed upload id=616578worker2 # [ 200.148570] nix-grpc-daemon[1879]: time=2026-09-22T11:00:10.607Z level=INFO msg="Upload complete. (306ms)"6579lb # [ 200.132216] niks3-server[1182]: 2026/09/22 11:00:10 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6580worker2 # [ 200.159675] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53576 duration_s=13 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16581worker2 # [ 200.176719] nix-daemon[866]: reaped child process 1945, status = succeeded6582client # [ 200.169884] systemd[1]: sw.service: Deactivated successfully.6583client # [ 200.175217] systemd[1]: sw.service: Consumed 179ms CPU time over 15.043s wall clock time, 41.2M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic.6584worker2 # [ 200.583123] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:11Z level=info event=signal_exit6585lb # [ 200.796705] niks3-server[1182]: 2026/09/22 11:00:11 INFO lead: released remote=192.168.1.4:473546586client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds)6587client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6588client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)6589worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6590worker2 # [ 201.025150] systemd[1]: Starting scheduler-token.service...6591worker2 # [ 201.109845] systemd[1]: scheduler-token.service: Deactivated successfully.6592worker2 # [ 201.111258] systemd[1]: Finished scheduler-token.service.6593worker2 # [ 201.112490] systemd[1]: scheduler-token.service: Consumed 30ms CPU time over 84ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6594lb # [ 201.444271] niks3-server[1182]: 2026/09/22 11:00:11 INFO lead: acquired remote=192.168.1.3:458626595worker1 # [ 201.482422] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:11Z level=info event=scheduler_take_over6596worker2 # [ 201.809589] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:12Z level=info event=scheduler_restarting6597worker1 # [ 201.826674] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:12Z level=info event=scheduler_restarting addr=lb:500516598worker2 # [ 201.820212] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:535606599worker2 # [ 201.825711] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:12Z level=info event=worker_gone addr=192.168.1.3:500516600worker2 # [ 201.830358] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:12Z level=info event=scheduler_restarting addr=lb:500516601worker2 # [ 201.834666] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:535606602worker2 # [ 201.839618] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:12Z level=info event=worker_gone addr=192.168.1.4:500516603worker2 # [ 201.864128] nix-daemon[866]: reaped child process 1850, status = succeeded6604worker1 # [ 202.027802] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:12Z level=info event=scheduler_connected addr=lb:500516605worker1 # [ 202.033124] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6606worker2 # [ 202.037590] nix-grpc-daemon[1796]: ts=2026-09-22T11:00:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6607worker2 # [ 202.081195] nix-grpc-daemon[1796]: E0000 00:00:1790074812.540176 1841 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6608worker1 # [ 202.233980] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:12Z level=info event=scheduler_connected addr=lb:500516609worker1 # [ 202.252225] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:576906610worker1 # [ 202.258157] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06611worker2 # [ 202.632981] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6612worker2 # [ 202.635988] systemd[1]: nix-grpc-daemon.service: Consumed 850ms CPU time over 28.428s wall clock time, 34.7M memory peak, 24.2K incoming IP traffic, 31.8K outgoing IP traffic.6613worker2 # [ 202.660531] systemd[1]: Starting Nix worker-protocol over gRPC...6614worker2 # [ 202.855686] nix-grpc-daemon[2146]: ts=2026-09-22T11:00:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6615worker2 # [ 202.858306] nix-grpc-daemon[2146]: ts=2026-09-22T11:00:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6616worker2 # [ 202.862321] nix-grpc-daemon[2146]: ts=2026-09-22T11:00:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16617worker2 # [ 202.869560] nix-grpc-daemon[2146]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6618worker2 # [ 202.871064] nix-grpc-daemon[2146]: E0000 00:00:1790074813.328099 2146 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6619worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.95 seconds)6620lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6621lb: must succeed: curl -sf localhost:9901/clusters6622worker2 # [ 202.890277] nix-grpc-daemon[2146]: E0000 00:00:1790074813.334117 2146 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6623worker2 # [ 202.893535] nix-grpc-daemon[2146]: ts=2026-09-22T11:00:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6624worker2 # [ 202.895753] systemd[1]: Started Nix worker-protocol over gRPC.6625worker2 # [ 202.908262] nix-grpc-daemon[2146]: E0000 00:00:1790074813.356281 2187 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6626worker2 # [ 202.929628] nix-grpc-daemon[2146]: E0000 00:00:1790074813.373829 2187 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6627worker2 # [ 202.932412] nix-grpc-daemon[2146]: E0000 00:00:1790074813.373984 2187 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6628worker2 # [ 202.934849] nix-grpc-daemon[2146]: E0000 00:00:1790074813.388220 2187 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6629worker2 # [ 202.944232] nix-grpc-daemon[2146]: ts=2026-09-22T11:00:13Z level=info event=scheduler_connected addr=lb:500516630worker1 # [ 202.973497] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:13Z level=info event=oidc_jwks provider=mock keys=16631worker1 # [ 202.973764] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:13Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:576906632worker1 # [ 202.973922] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06633lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)6634lb: must succeed: curl -sf localhost:9901/clusters6635lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)6636lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.20 seconds)6637lb: must succeed: curl -sf localhost:9901/clusters6638lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6639worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6640worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6641(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 19.14 seconds)6642subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6643client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16644lb # [ 204.216366] envoy[933]: conn peer=[2001:db8:1::1]:37910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9638 tx=4413 ms=64276645lb # [ 204.216723] envoy[933]: conn peer=[2001:db8:1::1]:47742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=1584 ms=147606646lb # [ 204.216845] envoy[933]: conn peer=[2001:db8:1::4]:43438 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1520 tx=344 ms=273816647worker1 # [ 204.514149] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:602046648worker2 # [ 204.513395] nix-grpc-daemon[2146]: ts=2026-09-22T11:00:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388926649worker2 # [ 204.531092] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)6650worker2 # [ 204.555934] nix-grpc-daemon[2146]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5)6651worker2 # [ 204.631653] nix-grpc-daemon[2146]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 171 ms (attempt 2/5)6652worker2 # [ 204.802961] nix-grpc-daemon[2146]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 329 ms (attempt 3/5)6653worker2 # [ 205.132366] nix-grpc-daemon[2146]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 578 ms (attempt 4/5)6654worker2 # [ 205.713833] nix-grpc-daemon[2146]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6655worker2 # [ 205.719686] nix-grpc-daemon[2146]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6656worker1 # [ 205.792575] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:16Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06657worker1 # [ 205.797908] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:16Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16658worker1 # [ 205.803768] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:16Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06659worker1 # [ 205.809142] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:16Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06660worker1 # [ 205.836591] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046661worker1 # [ 205.850714] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046662worker1 # [ 205.865020] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046663worker1 # [ 205.879773] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6664worker1 # [ 206.090544] nix-daemon[844]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)6665worker1 # [ 206.108689] nix-daemon[844]: reaped child process 1735, status = succeeded6666worker1 # [ 206.283524] nix-daemon[844]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6667worker1 # [ 206.301366] nix-daemon[844]: reaped child process 1740, status = succeeded6668lb # [ 206.276825] niks3-server[1182]: 2026/09/22 11:00:16 INFO Received uploads request method=POST path=/api/pending_closures6669worker1 # [ 206.323572] nix-grpc-daemon[1546]: time=2026-09-22T11:00:16.794Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6670worker1 # [ 206.325175] nix-grpc-daemon[1546]: time=2026-09-22T11:00:16.794Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6671lb # [ 206.305696] niks3-server[1182]: 2026/09/22 11:00:16 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6672lb # [ 206.321939] niks3-server[1182]: 2026/09/22 11:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6673lb # [ 206.336350] niks3-server[1182]: 2026/09/22 11:00:16 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6674lb # [ 206.347385] niks3-server[1182]: 2026/09/22 11:00:16 INFO Signed narinfos id=62 count=16675worker1 # [ 206.384399] nix-grpc-daemon[1546]: time=2026-09-22T11:00:16.855Z level=INFO msg="Uploading 1 narinfos"6676lb # [ 206.356569] niks3-server[1182]: 2026/09/22 11:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6677lb # [ 206.364350] niks3-server[1182]: 2026/09/22 11:00:16 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6678lb # [ 206.375517] niks3-server[1182]: 2026/09/22 11:00:16 INFO Completed upload id=626679worker1 # [ 206.414947] nix-grpc-daemon[1546]: time=2026-09-22T11:00:16.886Z level=INFO msg="Upload complete. (518ms)"6680worker1 # [ 206.424347] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=5126681worker1 # [ 206.444355] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046682worker1 # [ 206.451810] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6683worker1 # [ 206.458023] nix-daemon[844]: reaped child process 1730, status = succeeded6684worker1 # [ 206.471495] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6685worker1 # [ 207.901010] nix-daemon[844]: reaped child process 1744, status = succeeded6686worker1 # [ 208.067716] nix-daemon[844]: accepted connection from pid 1760, user nix-grpc-daemon (trusted)6687worker1 # [ 208.086337] nix-daemon[844]: reaped child process 1763, status = succeeded6688lb # [ 208.062132] niks3-server[1182]: 2026/09/22 11:00:18 INFO Received uploads request method=POST path=/api/pending_closures6689worker1 # [ 208.112374] nix-grpc-daemon[1546]: time=2026-09-22T11:00:18.581Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6690worker1 # [ 208.113851] nix-grpc-daemon[1546]: time=2026-09-22T11:00:18.581Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6691lb # [ 208.104549] niks3-server[1182]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6692lb # [ 208.117282] niks3-server[1182]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6693lb # [ 208.130527] niks3-server[1182]: 2026/09/22 11:00:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6694worker1 # [ 208.171418] nix-grpc-daemon[1546]: time=2026-09-22T11:00:18.642Z level=INFO msg="Uploading 1 narinfos"6695lb # [ 208.140044] niks3-server[1182]: 2026/09/22 11:00:18 INFO Signed narinfos id=63 count=16696lb # [ 208.142589] niks3-server[1182]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6697lb # [ 208.154162] niks3-server[1182]: 2026/09/22 11:00:18 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6698lb # [ 208.159525] niks3-server[1182]: 2026/09/22 11:00:18 INFO Completed upload id=636699worker1 # [ 208.196255] nix-grpc-daemon[1546]: time=2026-09-22T11:00:18.667Z level=INFO msg="Upload complete. (309ms)"6700lb # [ 208.162507] niks3-server[1182]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6701worker1 # [ 208.217162] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=16702worker1 # [ 208.231909] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:18Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16703worker1 # [ 208.236416] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:18Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06704worker1 # [ 208.239650] nix-daemon[844]: reaped child process 1742, status = succeeded6705worker1 # [ 208.252679] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046706worker1 # [ 208.266070] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046707worker1 # [ 208.280913] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046708worker1 # [ 208.294522] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6709worker1 # [ 208.492353] nix-daemon[844]: accepted connection from pid 1769, user nix-grpc-daemon (trusted)6710worker1 # [ 208.509121] nix-daemon[844]: reaped child process 1772, status = succeeded6711worker1 # [ 208.680099] nix-daemon[844]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6712worker1 # [ 208.695080] nix-daemon[844]: reaped child process 1777, status = succeeded6713lb # [ 208.673185] niks3-server[1182]: 2026/09/22 11:00:19 INFO Received uploads request method=POST path=/api/pending_closures6714worker1 # [ 208.718621] nix-grpc-daemon[1546]: time=2026-09-22T11:00:19.189Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6715worker1 # [ 208.720215] nix-grpc-daemon[1546]: time=2026-09-22T11:00:19.189Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6716lb # [ 208.699002] niks3-server[1182]: 2026/09/22 11:00:19 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6717worker1 # [ 208.749287] nix-grpc-daemon[1546]: time=2026-09-22T11:00:19.220Z level=INFO msg="Uploading 1 narinfos"6718lb # [ 208.715041] niks3-server[1182]: 2026/09/22 11:00:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6719lb # [ 208.724990] niks3-server[1182]: 2026/09/22 11:00:19 INFO Signed narinfos id=64 count=16720worker1 # [ 208.769255] nix-grpc-daemon[1546]: time=2026-09-22T11:00:19.240Z level=INFO msg="Upload complete. (459ms)"6721lb # [ 208.731910] niks3-server[1182]: 2026/09/22 11:00:19 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6722lb # [ 208.740718] niks3-server[1182]: 2026/09/22 11:00:19 INFO Completed upload id=646723worker1 # [ 208.775649] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=5126724lb # [ 208.743026] niks3-server[1182]: 2026/09/22 11:00:19 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6725worker1 # [ 208.785902] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046726worker1 # [ 208.790543] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6727worker1 # [ 208.794236] nix-daemon[844]: reaped child process 1767, status = succeeded6728worker1 # [ 208.803439] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6729worker1 # [ 210.229812] nix-daemon[844]: reaped child process 1781, status = succeeded6730worker1 # [ 210.393654] nix-daemon[844]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6731worker1 # [ 210.410297] nix-daemon[844]: reaped child process 1800, status = succeeded6732lb # [ 210.386075] niks3-server[1182]: 2026/09/22 11:00:20 INFO Received uploads request method=POST path=/api/pending_closures6733worker1 # [ 210.432361] nix-grpc-daemon[1546]: time=2026-09-22T11:00:20.903Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6734worker1 # [ 210.437060] nix-grpc-daemon[1546]: time=2026-09-22T11:00:20.905Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6735lb # [ 210.426914] niks3-server[1182]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6736worker1 # [ 210.477161] nix-grpc-daemon[1546]: time=2026-09-22T11:00:20.948Z level=INFO msg="Uploading 1 narinfos"6737lb # [ 210.438421] niks3-server[1182]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6738lb # [ 210.452933] niks3-server[1182]: 2026/09/22 11:00:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6739lb # [ 210.460715] niks3-server[1182]: 2026/09/22 11:00:20 INFO Signed narinfos id=65 count=16740lb # [ 210.469036] niks3-server[1182]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6741lb # [ 210.475582] niks3-server[1182]: 2026/09/22 11:00:20 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6742lb # [ 210.481994] niks3-server[1182]: 2026/09/22 11:00:20 INFO Completed upload id=656743worker1 # [ 210.511875] nix-grpc-daemon[1546]: time=2026-09-22T11:00:20.983Z level=INFO msg="Upload complete. (298ms)"6744lb # [ 210.485973] niks3-server[1182]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6745worker1 # [ 210.525728] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=16746worker1 # [ 210.544252] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:20Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16747worker1 # [ 210.552214] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046748worker1 # [ 210.557323] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:21Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06749worker1 # [ 210.562007] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046750worker1 # [ 210.566189] nix-daemon[844]: reaped child process 1779, status = succeeded6751worker1 # [ 210.569048] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046752worker1 # [ 210.582432] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6753worker1 # [ 210.762866] nix-daemon[844]: accepted connection from pid 1806, user nix-grpc-daemon (trusted)6754worker1 # [ 210.780469] nix-daemon[844]: reaped child process 1809, status = succeeded6755worker1 # [ 210.953930] nix-daemon[844]: accepted connection from pid 1811, user nix-grpc-daemon (trusted)6756worker1 # [ 210.969746] nix-daemon[844]: reaped child process 1814, status = succeeded6757lb # [ 210.944196] niks3-server[1182]: 2026/09/22 11:00:21 INFO Received uploads request method=POST path=/api/pending_closures6758worker1 # [ 210.989388] nix-grpc-daemon[1546]: time=2026-09-22T11:00:21.460Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6759worker1 # [ 210.990843] nix-grpc-daemon[1546]: time=2026-09-22T11:00:21.460Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6760lb # [ 210.972260] niks3-server[1182]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6761lb # [ 210.981893] niks3-server[1182]: 2026/09/22 11:00:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6762worker1 # [ 211.025970] nix-grpc-daemon[1546]: time=2026-09-22T11:00:21.497Z level=INFO msg="Uploading 1 narinfos"6763lb # [ 210.994675] niks3-server[1182]: 2026/09/22 11:00:21 INFO Signed narinfos id=66 count=16764lb # [ 211.001134] niks3-server[1182]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6765lb # [ 211.010375] niks3-server[1182]: 2026/09/22 11:00:21 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6766worker1 # [ 211.051243] nix-grpc-daemon[1546]: time=2026-09-22T11:00:21.522Z level=INFO msg="Upload complete. (464ms)"6767lb # [ 211.014938] niks3-server[1182]: 2026/09/22 11:00:21 INFO Completed upload id=666768lb # [ 211.018070] niks3-server[1182]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6769worker1 # [ 211.060226] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=5126770worker1 # [ 211.071506] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046771worker1 # [ 211.077550] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6772worker1 # [ 211.082024] nix-daemon[844]: reaped child process 1804, status = succeeded6773worker1 # [ 211.092664] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6774worker1 # [ 212.735223] nix-daemon[844]: reaped child process 1818, status = succeeded6775worker1 # [ 212.902563] nix-daemon[844]: accepted connection from pid 1834, user nix-grpc-daemon (trusted)6776worker1 # [ 212.919034] nix-daemon[844]: reaped child process 1837, status = succeeded6777lb # [ 212.894857] niks3-server[1182]: 2026/09/22 11:00:23 INFO Received uploads request method=POST path=/api/pending_closures6778worker1 # [ 212.942627] nix-grpc-daemon[1546]: time=2026-09-22T11:00:23.412Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6779worker1 # [ 212.944442] nix-grpc-daemon[1546]: time=2026-09-22T11:00:23.412Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6780lb # [ 212.934044] niks3-server[1182]: 2026/09/22 11:00:23 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6781lb # [ 212.943489] niks3-server[1182]: 2026/09/22 11:00:23 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6782lb # [ 212.952564] niks3-server[1182]: 2026/09/22 11:00:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6783worker1 # [ 212.994073] nix-grpc-daemon[1546]: time=2026-09-22T11:00:23.464Z level=INFO msg="Uploading 1 narinfos"6784lb # [ 212.960558] niks3-server[1182]: 2026/09/22 11:00:23 INFO Signed narinfos id=67 count=16785lb # [ 212.962723] niks3-server[1182]: 2026/09/22 11:00:23 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6786lb # [ 212.971265] niks3-server[1182]: 2026/09/22 11:00:23 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6787lb # [ 212.979587] niks3-server[1182]: 2026/09/22 11:00:23 INFO Completed upload id=676788lb # [ 212.982740] niks3-server[1182]: 2026/09/22 11:00:23 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6789worker1 # [ 213.016282] nix-grpc-daemon[1546]: time=2026-09-22T11:00:23.487Z level=INFO msg="Upload complete. (311ms)"6790worker1 # [ 213.034046] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=16791worker1 # [ 213.052792] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:23Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16792worker1 # [ 213.062724] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046793worker1 # [ 213.068284] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:23Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06794worker1 # [ 213.073316] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046795worker1 # [ 213.077766] nix-daemon[844]: reaped child process 1816, status = succeeded6796worker1 # [ 213.080974] nix-grpc-daemon[1546]: time=2026-09-22T11:00:23.543Z level=INFO msg="All 1 paths already cached"6797worker1 # [ 213.090131] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046798worker1 # [ 213.099687] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6799worker1 # [ 213.276741] nix-daemon[844]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6800worker1 # [ 213.293372] nix-daemon[844]: reaped child process 1846, status = succeeded6801worker1 # [ 213.464191] nix-daemon[844]: accepted connection from pid 1848, user nix-grpc-daemon (trusted)6802worker1 # [ 213.481967] nix-daemon[844]: reaped child process 1851, status = succeeded6803lb # [ 213.458047] niks3-server[1182]: 2026/09/22 11:00:23 INFO Received uploads request method=POST path=/api/pending_closures6804worker1 # [ 213.503654] nix-grpc-daemon[1546]: time=2026-09-22T11:00:23.974Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6805worker1 # [ 213.505359] nix-grpc-daemon[1546]: time=2026-09-22T11:00:23.974Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6806lb # [ 213.488131] niks3-server[1182]: 2026/09/22 11:00:23 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6807worker1 # [ 213.533883] nix-grpc-daemon[1546]: time=2026-09-22T11:00:24.004Z level=INFO msg="Uploading 1 narinfos"6808lb # [ 213.501188] niks3-server[1182]: 2026/09/22 11:00:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6809lb # [ 213.512718] niks3-server[1182]: 2026/09/22 11:00:23 INFO Signed narinfos id=68 count=16810lb # [ 213.523564] niks3-server[1182]: 2026/09/22 11:00:23 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6811worker1 # [ 213.564940] nix-grpc-daemon[1546]: time=2026-09-22T11:00:24.036Z level=INFO msg="Upload complete. (461ms)"6812lb # [ 213.529157] niks3-server[1182]: 2026/09/22 11:00:23 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6813lb # [ 213.535957] niks3-server[1182]: 2026/09/22 11:00:23 INFO Completed upload id=686814lb # [ 213.539353] niks3-server[1182]: 2026/09/22 11:00:23 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6815worker1 # [ 213.572191] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=6246816worker1 # [ 213.580718] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046817worker1 # [ 213.590189] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6818worker1 # [ 213.595201] nix-daemon[844]: reaped child process 1841, status = succeeded6819worker1 # [ 213.607491] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6820worker1 # [ 214.439237] nix-daemon[844]: reaped child process 1855, status = succeeded6821worker1 # [ 214.611017] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6822worker1 # [ 214.629254] nix-daemon[844]: reaped child process 1874, status = succeeded6823lb # [ 214.606409] niks3-server[1182]: 2026/09/22 11:00:25 INFO Received uploads request method=POST path=/api/pending_closures6824worker1 # [ 214.652413] nix-grpc-daemon[1546]: time=2026-09-22T11:00:25.123Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6825worker1 # [ 214.653887] nix-grpc-daemon[1546]: time=2026-09-22T11:00:25.123Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6826lb # [ 214.643940] niks3-server[1182]: 2026/09/22 11:00:25 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6827lb # [ 214.657203] niks3-server[1182]: 2026/09/22 11:00:25 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6828lb # [ 214.668696] niks3-server[1182]: 2026/09/22 11:00:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6829lb # [ 214.676475] niks3-server[1182]: 2026/09/22 11:00:25 INFO Signed narinfos id=69 count=16830worker1 # [ 214.711993] nix-grpc-daemon[1546]: time=2026-09-22T11:00:25.182Z level=INFO msg="Uploading 1 narinfos"6831lb # [ 214.679494] niks3-server[1182]: 2026/09/22 11:00:25 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6832lb # [ 214.691315] niks3-server[1182]: 2026/09/22 11:00:25 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6833lb # [ 214.699853] niks3-server[1182]: 2026/09/22 11:00:25 INFO Completed upload id=696834worker1 # [ 214.735922] nix-grpc-daemon[1546]: time=2026-09-22T11:00:25.207Z level=INFO msg="Upload complete. (319ms)"6835lb # [ 214.703150] niks3-server[1182]: 2026/09/22 11:00:25 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6836worker1 # [ 214.749126] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=16837worker1 # [ 214.766952] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:25Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16838worker1 # [ 214.771827] nix-daemon[844]: reaped child process 1853, status = succeeded6839worker1 # [ 214.792144] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046840worker1 # [ 214.799446] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:25Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06841worker1 # [ 214.810375] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046842worker1 # [ 214.823403] nix-grpc-daemon[1546]: time=2026-09-22T11:00:25.294Z level=INFO msg="All 1 paths already cached"6843worker1 # [ 214.832766] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046844worker1 # [ 214.845518] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6845worker1 # [ 215.038351] nix-daemon[844]: accepted connection from pid 1880, user nix-grpc-daemon (trusted)6846worker1 # [ 215.055237] nix-daemon[844]: reaped child process 1883, status = succeeded6847worker1 # [ 215.226990] nix-daemon[844]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6848worker1 # [ 215.243716] nix-daemon[844]: reaped child process 1888, status = succeeded6849lb # [ 215.219104] niks3-server[1182]: 2026/09/22 11:00:25 INFO Received uploads request method=POST path=/api/pending_closures6850worker1 # [ 215.264890] nix-grpc-daemon[1546]: time=2026-09-22T11:00:25.735Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6851worker1 # [ 215.266354] nix-grpc-daemon[1546]: time=2026-09-22T11:00:25.735Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6852lb # [ 215.246358] niks3-server[1182]: 2026/09/22 11:00:25 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6853worker1 # [ 215.295406] nix-grpc-daemon[1546]: time=2026-09-22T11:00:25.766Z level=INFO msg="Uploading 1 narinfos"6854lb # [ 215.263310] niks3-server[1182]: 2026/09/22 11:00:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6855lb # [ 215.274529] niks3-server[1182]: 2026/09/22 11:00:25 INFO Signed narinfos id=70 count=16856lb # [ 215.283400] niks3-server[1182]: 2026/09/22 11:00:25 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6857lb # [ 215.287331] niks3-server[1182]: 2026/09/22 11:00:25 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6858worker1 # [ 215.327755] nix-grpc-daemon[1546]: time=2026-09-22T11:00:25.798Z level=INFO msg="Upload complete. (468ms)"6859lb # [ 215.291744] niks3-server[1182]: 2026/09/22 11:00:25 INFO Completed upload id=706860lb # [ 215.294347] niks3-server[1182]: 2026/09/22 11:00:25 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6861worker1 # [ 215.336248] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=6246862worker1 # [ 215.346869] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046863worker1 # [ 215.352543] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6864worker1 # [ 215.357159] nix-daemon[844]: reaped child process 1878, status = succeeded6865worker1 # [ 215.369910] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6866worker1 # [ 216.617854] nix-daemon[844]: reaped child process 1892, status = succeeded6867worker1 # [ 216.791595] nix-daemon[844]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6868worker1 # [ 216.811140] nix-daemon[844]: reaped child process 1911, status = succeeded6869lb # [ 216.787136] niks3-server[1182]: 2026/09/22 11:00:27 INFO Received uploads request method=POST path=/api/pending_closures6870worker1 # [ 216.835425] nix-grpc-daemon[1546]: time=2026-09-22T11:00:27.305Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6871worker1 # [ 216.837713] nix-grpc-daemon[1546]: time=2026-09-22T11:00:27.305Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6872lb # [ 216.824482] niks3-server[1182]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6873lb # [ 216.837514] niks3-server[1182]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6874lb # [ 216.846735] niks3-server[1182]: 2026/09/22 11:00:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6875lb # [ 216.852742] niks3-server[1182]: 2026/09/22 11:00:27 INFO Signed narinfos id=71 count=16876worker1 # [ 216.889632] nix-grpc-daemon[1546]: time=2026-09-22T11:00:27.360Z level=INFO msg="Uploading 1 narinfos"6877lb # [ 216.858485] niks3-server[1182]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6878lb # [ 216.867598] niks3-server[1182]: 2026/09/22 11:00:27 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6879worker1 # [ 216.910414] nix-grpc-daemon[1546]: time=2026-09-22T11:00:27.381Z level=INFO msg="Upload complete. (329ms)"6880lb # [ 216.875608] niks3-server[1182]: 2026/09/22 11:00:27 INFO Completed upload id=716881lb # [ 216.878593] niks3-server[1182]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6882worker1 # [ 216.927655] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16883worker1 # [ 216.948909] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:27Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16884worker1 # [ 216.959234] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046885worker1 # [ 216.967234] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:27Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06886worker1 # [ 216.973825] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046887worker1 # [ 216.988921] nix-daemon[844]: reaped child process 1890, status = succeeded6888worker1 # [ 216.992602] nix-grpc-daemon[1546]: time=2026-09-22T11:00:27.443Z level=INFO msg="All 1 paths already cached"6889worker1 # [ 216.999254] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6890worker1 # [ 217.003227] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046891worker1 # [ 217.183072] nix-daemon[844]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6892worker1 # [ 217.201301] nix-daemon[844]: reaped child process 1920, status = succeeded6893worker1 # [ 217.372190] nix-daemon[844]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6894worker1 # [ 217.387456] nix-daemon[844]: reaped child process 1925, status = succeeded6895lb # [ 217.365674] niks3-server[1182]: 2026/09/22 11:00:27 INFO Received uploads request method=POST path=/api/pending_closures6896worker1 # [ 217.411226] nix-grpc-daemon[1546]: time=2026-09-22T11:00:27.882Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6897worker1 # [ 217.412844] nix-grpc-daemon[1546]: time=2026-09-22T11:00:27.882Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6898lb # [ 217.391041] niks3-server[1182]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6899worker1 # [ 217.440902] nix-grpc-daemon[1546]: time=2026-09-22T11:00:27.911Z level=INFO msg="Uploading 1 narinfos"6900lb # [ 217.405285] niks3-server[1182]: 2026/09/22 11:00:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6901lb # [ 217.419329] niks3-server[1182]: 2026/09/22 11:00:27 INFO Signed narinfos id=72 count=16902lb # [ 217.430613] niks3-server[1182]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6903worker1 # [ 217.471684] nix-grpc-daemon[1546]: time=2026-09-22T11:00:27.942Z level=INFO msg="Upload complete. (463ms)"6904lb # [ 217.436102] niks3-server[1182]: 2026/09/22 11:00:27 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6905lb # [ 217.441526] niks3-server[1182]: 2026/09/22 11:00:27 INFO Completed upload id=726906worker1 # [ 217.477990] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=6246907lb # [ 217.444954] niks3-server[1182]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6908worker1 # [ 217.487617] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046909worker1 # [ 217.499066] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6910worker1 # [ 217.503542] nix-daemon[844]: reaped child process 1915, status = succeeded6911worker1 # [ 217.514376] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6912worker1 # [ 219.162356] nix-daemon[844]: reaped child process 1929, status = succeeded6913worker1 # [ 219.350049] nix-daemon[844]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)6914worker1 # [ 219.368394] nix-daemon[844]: reaped child process 1948, status = succeeded6915lb # [ 219.347146] niks3-server[1182]: 2026/09/22 11:00:29 INFO Received uploads request method=POST path=/api/pending_closures6916worker1 # [ 219.397271] nix-grpc-daemon[1546]: time=2026-09-22T11:00:29.866Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6917worker1 # [ 219.398871] nix-grpc-daemon[1546]: time=2026-09-22T11:00:29.866Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6918lb # [ 219.385824] niks3-server[1182]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6919lb # [ 219.397482] niks3-server[1182]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6920lb # [ 219.408174] niks3-server[1182]: 2026/09/22 11:00:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6921worker1 # [ 219.452288] nix-grpc-daemon[1546]: time=2026-09-22T11:00:29.923Z level=INFO msg="Uploading 1 narinfos"6922lb # [ 219.415507] niks3-server[1182]: 2026/09/22 11:00:29 INFO Signed narinfos id=73 count=16923lb # [ 219.421268] niks3-server[1182]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6924lb # [ 219.431855] niks3-server[1182]: 2026/09/22 11:00:29 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6925lb # [ 219.440470] niks3-server[1182]: 2026/09/22 11:00:29 INFO Completed upload id=736926worker1 # [ 219.477111] nix-grpc-daemon[1546]: time=2026-09-22T11:00:29.948Z level=INFO msg="Upload complete. (350ms)"6927lb # [ 219.443536] niks3-server[1182]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6928worker1 # [ 219.495586] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16929worker1 # [ 219.513018] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:29Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16930worker1 # [ 219.522859] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046931worker1 # [ 219.527548] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:29Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06932worker1 # [ 219.532723] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046933worker1 # [ 219.536697] nix-daemon[844]: reaped child process 1927, status = succeeded6934worker1 # [ 219.538418] nix-grpc-daemon[1546]: time=2026-09-22T11:00:30.002Z level=INFO msg="All 2 paths already cached"6935worker1 # [ 219.540843] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046936worker1 # [ 219.550204] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6937worker1 # [ 219.733169] nix-daemon[844]: accepted connection from pid 1954, user nix-grpc-daemon (trusted)6938worker1 # [ 219.751738] nix-daemon[844]: reaped child process 1957, status = succeeded6939worker1 # [ 219.920819] nix-daemon[844]: accepted connection from pid 1959, user nix-grpc-daemon (trusted)6940worker1 # [ 219.938759] nix-daemon[844]: reaped child process 1962, status = succeeded6941lb # [ 219.913579] niks3-server[1182]: 2026/09/22 11:00:30 INFO Received uploads request method=POST path=/api/pending_closures6942worker1 # [ 219.958929] nix-grpc-daemon[1546]: time=2026-09-22T11:00:30.429Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6943worker1 # [ 219.960496] nix-grpc-daemon[1546]: time=2026-09-22T11:00:30.429Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6944lb # [ 219.940601] niks3-server[1182]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6945worker1 # [ 219.988854] nix-grpc-daemon[1546]: time=2026-09-22T11:00:30.459Z level=INFO msg="Uploading 1 narinfos"6946lb # [ 219.957020] niks3-server[1182]: 2026/09/22 11:00:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6947lb # [ 219.967289] niks3-server[1182]: 2026/09/22 11:00:30 INFO Signed narinfos id=74 count=16948lb # [ 219.974581] niks3-server[1182]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6949worker1 # [ 220.016290] nix-grpc-daemon[1546]: time=2026-09-22T11:00:30.487Z level=INFO msg="Upload complete. (456ms)"6950lb # [ 219.980525] niks3-server[1182]: 2026/09/22 11:00:30 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6951lb # [ 219.985788] niks3-server[1182]: 2026/09/22 11:00:30 INFO Completed upload id=746952worker1 # [ 220.024219] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=7526953lb # [ 219.989136] niks3-server[1182]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6954worker1 # [ 220.036072] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046955worker1 # [ 220.044919] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6956worker1 # [ 220.047596] nix-daemon[844]: reaped child process 1952, status = succeeded6957worker1 # [ 220.058263] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6958worker1 # [ 221.488842] nix-daemon[844]: reaped child process 1966, status = succeeded6959worker1 # [ 221.658291] nix-daemon[844]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6960worker1 # [ 221.675537] nix-daemon[844]: reaped child process 1985, status = succeeded6961lb # [ 221.651572] niks3-server[1182]: 2026/09/22 11:00:32 INFO Received uploads request method=POST path=/api/pending_closures6962worker1 # [ 221.698367] nix-grpc-daemon[1546]: time=2026-09-22T11:00:32.168Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6963worker1 # [ 221.699887] nix-grpc-daemon[1546]: time=2026-09-22T11:00:32.168Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6964lb # [ 221.683124] niks3-server[1182]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6965lb # [ 221.701637] niks3-server[1182]: 2026/09/22 11:00:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6966worker1 # [ 221.743975] nix-grpc-daemon[1546]: time=2026-09-22T11:00:32.214Z level=INFO msg="Uploading 1 narinfos"6967lb # [ 221.714986] niks3-server[1182]: 2026/09/22 11:00:32 INFO Signed narinfos id=75 count=16968lb # [ 221.721365] niks3-server[1182]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6969lb # [ 221.727300] niks3-server[1182]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6970lb # [ 221.734774] niks3-server[1182]: 2026/09/22 11:00:32 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6971lb # [ 221.739508] niks3-server[1182]: 2026/09/22 11:00:32 INFO Completed upload id=756972worker1 # [ 221.775562] nix-grpc-daemon[1546]: time=2026-09-22T11:00:32.246Z level=INFO msg="Upload complete. (306ms)"6973lb # [ 221.742408] niks3-server[1182]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6974worker1 # [ 221.788993] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16975worker1 # [ 221.803551] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:32Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16976worker1 # [ 221.808533] nix-daemon[844]: reaped child process 1964, status = succeeded6977worker1 # [ 221.828831] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602046978worker1 # [ 221.842918] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602046979worker1 # [ 221.857365] nix-grpc-daemon[1546]: time=2026-09-22T11:00:32.328Z level=INFO msg="All 2 paths already cached"6980worker1 # [ 221.866917] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602046981worker1 # [ 221.881105] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6982worker1 # [ 222.074900] nix-daemon[844]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6983worker1 # [ 222.093014] nix-daemon[844]: reaped child process 1992, status = succeeded6984worker1 # [ 222.264110] nix-daemon[844]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6985worker1 # [ 222.281300] nix-daemon[844]: reaped child process 1997, status = succeeded6986lb # [ 222.256114] niks3-server[1182]: 2026/09/22 11:00:32 INFO Received uploads request method=POST path=/api/pending_closures6987worker1 # [ 222.301177] nix-grpc-daemon[1546]: time=2026-09-22T11:00:32.772Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6988worker1 # [ 222.302653] nix-grpc-daemon[1546]: time=2026-09-22T11:00:32.772Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6989lb # [ 222.284598] niks3-server[1182]: 2026/09/22 11:00:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6990worker1 # [ 222.327367] nix-grpc-daemon[1546]: time=2026-09-22T11:00:32.798Z level=INFO msg="Uploading 1 narinfos"6991lb # [ 222.297743] niks3-server[1182]: 2026/09/22 11:00:32 INFO Signed narinfos id=76 count=16992lb # [ 222.306552] niks3-server[1182]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6993lb # [ 222.319812] niks3-server[1182]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6994lb # [ 222.324532] niks3-server[1182]: 2026/09/22 11:00:32 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6995worker1 # [ 222.366252] nix-grpc-daemon[1546]: time=2026-09-22T11:00:32.837Z level=INFO msg="Upload complete. (471ms)"6996lb # [ 222.331185] niks3-server[1182]: 2026/09/22 11:00:32 INFO Completed upload id=766997worker1 # [ 222.372264] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=7526998lb # [ 222.335024] niks3-server[1182]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6999worker1 # [ 222.380734] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602047000worker1 # [ 222.391064] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)7001worker1 # [ 222.395620] nix-daemon[844]: reaped child process 1987, status = succeeded7002worker1 # [ 222.409070] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)7003worker1 # [ 224.053716] nix-daemon[844]: reaped child process 2001, status = succeeded7004worker1 # [ 224.224728] nix-daemon[844]: accepted connection from pid 2017, user nix-grpc-daemon (trusted)7005worker1 # [ 224.243742] nix-daemon[844]: reaped child process 2020, status = succeeded7006lb # [ 224.223161] niks3-server[1182]: 2026/09/22 11:00:34 INFO Received uploads request method=POST path=/api/pending_closures7007worker1 # [ 224.271467] nix-grpc-daemon[1546]: time=2026-09-22T11:00:34.739Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7008worker1 # [ 224.273149] nix-grpc-daemon[1546]: time=2026-09-22T11:00:34.739Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"7009lb # [ 224.261737] niks3-server[1182]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv7010lb # [ 224.272395] niks3-server[1182]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst7011worker1 # [ 224.319561] nix-grpc-daemon[1546]: time=2026-09-22T11:00:34.790Z level=INFO msg="Uploading 1 narinfos"7012lb # [ 224.282762] niks3-server[1182]: 2026/09/22 11:00:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7013lb # [ 224.287947] niks3-server[1182]: 2026/09/22 11:00:34 INFO Signed narinfos id=77 count=17014lb # [ 224.290144] niks3-server[1182]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls7015lb # [ 224.299449] niks3-server[1182]: 2026/09/22 11:00:34 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7016lb # [ 224.306510] niks3-server[1182]: 2026/09/22 11:00:34 INFO Completed upload id=777017lb # [ 224.309707] niks3-server[1182]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo7018worker1 # [ 224.341046] nix-grpc-daemon[1546]: time=2026-09-22T11:00:34.812Z level=INFO msg="Upload complete. (321ms)"7019worker1 # [ 224.353205] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=17020worker1 # [ 224.368237] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:34Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=17021worker1 # [ 224.373296] nix-daemon[844]: reaped child process 1999, status = succeeded7022worker1 # [ 224.397353] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602047023worker1 # [ 224.411496] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602047024worker1 # [ 224.427017] nix-grpc-daemon[1546]: time=2026-09-22T11:00:34.897Z level=INFO msg="All 2 paths already cached"7025worker1 # [ 224.436436] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602047026worker1 # [ 224.449758] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)7027worker1 # [ 224.653499] nix-daemon[844]: accepted connection from pid 2024, user nix-grpc-daemon (trusted)7028worker1 # [ 224.672249] nix-daemon[844]: reaped child process 2027, status = succeeded7029worker1 # [ 224.843017] nix-daemon[844]: accepted connection from pid 2029, user nix-grpc-daemon (trusted)7030worker1 # [ 224.860909] nix-daemon[844]: reaped child process 2032, status = succeeded7031lb # [ 224.838185] niks3-server[1182]: 2026/09/22 11:00:35 INFO Received uploads request method=POST path=/api/pending_closures7032worker1 # [ 224.882899] nix-grpc-daemon[1546]: time=2026-09-22T11:00:35.353Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7033worker1 # [ 224.884420] nix-grpc-daemon[1546]: time=2026-09-22T11:00:35.353Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"7034lb # [ 224.861383] niks3-server[1182]: 2026/09/22 11:00:35 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst7035lb # [ 224.872216] niks3-server[1182]: 2026/09/22 11:00:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7036worker1 # [ 224.914273] nix-grpc-daemon[1546]: time=2026-09-22T11:00:35.385Z level=INFO msg="Uploading 1 narinfos"7037lb # [ 224.877540] niks3-server[1182]: 2026/09/22 11:00:35 INFO Signed narinfos id=78 count=17038lb # [ 224.882225] niks3-server[1182]: 2026/09/22 11:00:35 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls7039lb # [ 224.889052] niks3-server[1182]: 2026/09/22 11:00:35 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7040lb # [ 224.896572] niks3-server[1182]: 2026/09/22 11:00:35 INFO Completed upload id=787041worker1 # [ 224.933094] nix-grpc-daemon[1546]: time=2026-09-22T11:00:35.404Z level=INFO msg="Upload complete. (466ms)"7042lb # [ 224.898981] niks3-server[1182]: 2026/09/22 11:00:35 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo7043worker1 # [ 224.940119] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=0 paths=1 nar_bytes_in=7527044worker1 # [ 224.949848] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602047045worker1 # [ 224.954931] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)7046worker1 # [ 224.958733] nix-daemon[844]: reaped child process 2022, status = succeeded7047worker1 # [ 224.968908] nix-daemon[844]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)7048worker1 # [ 225.809006] nix-daemon[844]: reaped child process 2036, status = succeeded7049worker1 # [ 225.991448] nix-daemon[844]: accepted connection from pid 2052, user nix-grpc-daemon (trusted)7050worker1 # [ 226.008187] nix-daemon[844]: reaped child process 2055, status = succeeded7051lb # [ 225.989428] niks3-server[1182]: 2026/09/22 11:00:36 INFO Received uploads request method=POST path=/api/pending_closures7052worker1 # [ 226.035609] nix-grpc-daemon[1546]: time=2026-09-22T11:00:36.505Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7053worker1 # [ 226.037336] nix-grpc-daemon[1546]: time=2026-09-22T11:00:36.505Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7054lb # [ 226.021836] niks3-server[1182]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7055lb # [ 226.035557] niks3-server[1182]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7056lb # [ 226.046790] niks3-server[1182]: 2026/09/22 11:00:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7057worker1 # [ 226.088529] nix-grpc-daemon[1546]: time=2026-09-22T11:00:36.559Z level=INFO msg="Uploading 1 narinfos"7058lb # [ 226.055284] niks3-server[1182]: 2026/09/22 11:00:36 INFO Signed narinfos id=79 count=17059lb # [ 226.061829] niks3-server[1182]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7060lb # [ 226.070635] niks3-server[1182]: 2026/09/22 11:00:36 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7061lb # [ 226.077461] niks3-server[1182]: 2026/09/22 11:00:36 INFO Completed upload id=797062lb # [ 226.080486] niks3-server[1182]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7063worker1 # [ 226.113296] nix-grpc-daemon[1546]: time=2026-09-22T11:00:36.584Z level=INFO msg="Upload complete. (333ms)"7064worker1 # [ 226.125311] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60204 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=17065worker1 # [ 226.142165] nix-daemon[844]: reaped child process 2034, status = succeeded7066client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 22.00 seconds)7067worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7068worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7069client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'7070client # [ 226.347662] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7071client # Running as unit: gpu.service; invocation ID: c2857c971fe44315925ab46d34d999647072client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)7073lb: must succeed: curl -sf localhost:9901/clusters7074lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7075lb: must succeed: curl -sf localhost:9901/clusters7076lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7077worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7078worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7079worker2 # [ 226.626981] nix-grpc-daemon[2146]: ts=2026-09-22T11:00:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:388927080client # [ 226.602342] nix[1532]: don't know how to build these paths:7081client # [ 226.603448] nix[1532]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7082worker1 # [ 226.656722] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602047083client # [ 226.632522] nix[1532]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker7084worker1 # [ 226.681321] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:37Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07085lb: must succeed: curl -sf localhost:9901/clusters7086lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7087lb: must succeed: curl -sf localhost:9901/clusters7088lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7089worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7090worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7091client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7092lb # [ 234.218215] envoy[933]: conn peer=[2001:db8:1::1]:38794 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27174 tx=12470 ms=216757093client # [ 256.634459] nix[1532]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}7094client # [ 256.647713] nix[1532]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7095client # [ 256.688264] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7096client # [ 256.693662] systemd[1]: gpu.service: Failed with result 'exit-code'.7097client # [ 256.699951] systemd[1]: gpu.service: Consumed 180ms CPU time over 30.346s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7098client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.26 seconds)7099client: must succeed: journalctl -u gpu -o cat7100client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7101lb: must succeed: curl -sf localhost:9901/clusters7102lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7103lb: must succeed: curl -sf localhost:9901/clusters7104lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7105worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7106worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7107(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 53.11 seconds)7108subtest: low disk drains a worker and builds go to the other7109worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7110worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.08 seconds)7111worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7112worker1 # [ 258.526023] nix-grpc-daemon[1394]: ts=2026-09-22T11:01:08Z level=info event=unhealthy reason=min_free7113worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)7114client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&27115client # don't know how to build these paths:7116client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7117worker1 # [ 259.830055] nix-grpc-daemon[1394]: ts=2026-09-22T11:01:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:602047118worker2 # [ 259.818921] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388927119worker1 # [ 259.861596] nix-grpc-daemon[1394]: ts=2026-09-22T11:01:10Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07120worker1 # [ 259.867584] nix-grpc-daemon[1394]: ts=2026-09-22T11:01:10Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17121worker2 # [ 259.892475] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388927122worker2 # [ 259.909040] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388927123client # copying 1 paths to 'lb:50051'...7124client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7125worker2 # [ 259.925435] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:388927126worker2 # [ 259.941533] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7127worker2 # [ 259.969535] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:10Z level=info event=niks3_push_started pid=22067128worker2 # [ 260.191554] nix-grpc-daemon[2206]: time=2026-09-22T11:01:10.650Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""7129worker2 # [ 260.370301] nix-daemon[866]: accepted connection from pid 2219, user nix-grpc-daemon (trusted)7130worker2 # [ 260.388141] nix-daemon[866]: reaped child process 2222, status = succeeded7131worker2 # [ 260.564756] nix-daemon[866]: accepted connection from pid 2224, user nix-grpc-daemon (trusted)7132worker2 # [ 260.581665] nix-daemon[866]: reaped child process 2227, status = succeeded7133lb # [ 260.577226] niks3-server[1182]: 2026/09/22 11:01:11 INFO Received uploads request method=POST path=/api/pending_closures7134worker2 # [ 260.607179] nix-grpc-daemon[2206]: time=2026-09-22T11:01:11.065Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7135worker2 # [ 260.610192] nix-grpc-daemon[2206]: time=2026-09-22T11:01:11.065Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7136lb # [ 260.616895] niks3-server[1182]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7137worker2 # [ 260.651741] nix-grpc-daemon[2206]: time=2026-09-22T11:01:11.110Z level=INFO msg="Uploading 1 narinfos"7138lb # [ 260.631259] niks3-server[1182]: 2026/09/22 11:01:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7139lb # [ 260.642995] niks3-server[1182]: 2026/09/22 11:01:11 INFO Signed narinfos id=80 count=17140lb # [ 260.649432] niks3-server[1182]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7141lb # [ 260.663184] niks3-server[1182]: 2026/09/22 11:01:11 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7142lb # [ 260.667413] niks3-server[1182]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7143lb # [ 260.671887] niks3-server[1182]: 2026/09/22 11:01:11 INFO Completed upload id=807144worker2 # [ 260.691475] nix-grpc-daemon[2206]: time=2026-09-22T11:01:11.150Z level=INFO msg="Upload complete. (495ms)"7145worker2 # [ 260.701812] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38892 duration_s=0 paths=1 nar_bytes_in=5127146worker2 # [ 260.712772] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388927147worker2 # [ 260.725141] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7148worker2 # [ 260.733646] nix-daemon[866]: reaped child process 2204, status = succeeded7149client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7150client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7151worker2 # [ 260.749324] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7152client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7153client # farm-a> LOG-farm-a-drain7154worker2 # [ 261.798705] nix-daemon[866]: reaped child process 2231, status = succeeded7155worker2 # [ 261.978744] nix-daemon[866]: accepted connection from pid 2247, user nix-grpc-daemon (trusted)7156worker2 # [ 261.998214] nix-daemon[866]: reaped child process 2250, status = succeeded7157lb # [ 261.993277] niks3-server[1182]: 2026/09/22 11:01:12 INFO Received uploads request method=POST path=/api/pending_closures7158worker2 # [ 262.024435] nix-grpc-daemon[2206]: time=2026-09-22T11:01:12.482Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7159worker2 # [ 262.025917] nix-grpc-daemon[2206]: time=2026-09-22T11:01:12.482Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7160lb # [ 262.041424] niks3-server[1182]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7161worker2 # [ 262.074458] nix-grpc-daemon[2206]: time=2026-09-22T11:01:12.532Z level=INFO msg="Uploading 1 narinfos"7162lb # [ 262.051112] niks3-server[1182]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7163lb # [ 262.066231] niks3-server[1182]: 2026/09/22 11:01:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7164lb # [ 262.078734] niks3-server[1182]: 2026/09/22 11:01:12 INFO Signed narinfos id=81 count=17165worker2 # [ 262.101752] nix-grpc-daemon[2206]: time=2026-09-22T11:01:12.560Z level=INFO msg="Upload complete. (341ms)"7166lb # [ 262.082380] niks3-server[1182]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7167lb # [ 262.087119] niks3-server[1182]: 2026/09/22 11:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7168lb # [ 262.094805] niks3-server[1182]: 2026/09/22 11:01:12 INFO Completed upload id=817169lb # [ 262.098246] niks3-server[1182]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7170worker1 # [ 262.142724] nix-grpc-daemon[1394]: ts=2026-09-22T11:01:12Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07171worker2 # [ 262.113312] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38892 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=17172worker2 # [ 262.132562] nix-daemon[866]: reaped child process 2229, status = succeeded7173worker1 # [ 262.148676] nix-grpc-daemon[1394]: ts=2026-09-22T11:01:12Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17174worker2 # [ 262.172082] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388927175worker2 # [ 262.188404] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388927176worker2 # [ 262.201913] nix-grpc-daemon[2206]: time=2026-09-22T11:01:12.661Z level=INFO msg="All 1 paths already cached"7177client # copying 1 paths to 'lb:50051'...7178client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7179worker2 # [ 262.212848] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:388927180worker2 # [ 262.226593] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7181worker2 # [ 262.433847] nix-daemon[866]: accepted connection from pid 2254, user nix-grpc-daemon (trusted)7182worker2 # [ 262.454081] nix-daemon[866]: reaped child process 2257, status = succeeded7183worker2 # [ 262.631310] nix-daemon[866]: accepted connection from pid 2259, user nix-grpc-daemon (trusted)7184worker2 # [ 262.648880] nix-daemon[866]: reaped child process 2262, status = succeeded7185lb # [ 262.642871] niks3-server[1182]: 2026/09/22 11:01:13 INFO Received uploads request method=POST path=/api/pending_closures7186worker2 # [ 262.671821] nix-grpc-daemon[2206]: time=2026-09-22T11:01:13.130Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7187worker2 # [ 262.673582] nix-grpc-daemon[2206]: time=2026-09-22T11:01:13.130Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7188lb # [ 262.669632] niks3-server[1182]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7189lb # [ 262.683472] niks3-server[1182]: 2026/09/22 11:01:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7190worker2 # [ 262.711554] nix-grpc-daemon[2206]: time=2026-09-22T11:01:13.170Z level=INFO msg="Uploading 1 narinfos"7191lb # [ 262.695818] niks3-server[1182]: 2026/09/22 11:01:13 INFO Signed narinfos id=82 count=17192lb # [ 262.701900] niks3-server[1182]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7193lb # [ 262.710505] niks3-server[1182]: 2026/09/22 11:01:13 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7194lb # [ 262.718546] niks3-server[1182]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7195lb # [ 262.723118] niks3-server[1182]: 2026/09/22 11:01:13 INFO Completed upload id=827196worker2 # [ 262.743085] nix-grpc-daemon[2206]: time=2026-09-22T11:01:13.202Z level=INFO msg="Upload complete. (499ms)"7197worker2 # [ 262.752170] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38892 duration_s=0 paths=1 nar_bytes_in=6327198worker2 # [ 262.766214] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388927199worker2 # [ 262.772604] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7200worker2 # [ 262.777382] nix-daemon[866]: reaped child process 2252, status = succeeded7201client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7202worker2 # [ 262.789004] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7203client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7204client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7205client # farm-b> LOG-farm-b-drain7206worker2 # [ 263.635438] nix-daemon[866]: reaped child process 2266, status = succeeded7207worker2 # [ 263.825560] nix-daemon[866]: accepted connection from pid 2282, user nix-grpc-daemon (trusted)7208worker2 # [ 263.843898] nix-daemon[866]: reaped child process 2285, status = succeeded7209lb # [ 263.836657] niks3-server[1182]: 2026/09/22 11:01:14 INFO Received uploads request method=POST path=/api/pending_closures7210worker2 # [ 263.867013] nix-grpc-daemon[2206]: time=2026-09-22T11:01:14.325Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7211worker2 # [ 263.869008] nix-grpc-daemon[2206]: time=2026-09-22T11:01:14.325Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7212lb # [ 263.872477] niks3-server[1182]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7213lb # [ 263.885247] niks3-server[1182]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7214worker2 # [ 263.917809] nix-grpc-daemon[2206]: time=2026-09-22T11:01:14.376Z level=INFO msg="Uploading 1 narinfos"7215lb # [ 263.898798] niks3-server[1182]: 2026/09/22 11:01:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7216lb # [ 263.907586] niks3-server[1182]: 2026/09/22 11:01:14 INFO Signed narinfos id=83 count=17217lb # [ 263.912883] niks3-server[1182]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7218lb # [ 263.919443] niks3-server[1182]: 2026/09/22 11:01:14 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7219lb # [ 263.926037] niks3-server[1182]: 2026/09/22 11:01:14 INFO Completed upload id=837220worker2 # [ 263.945515] nix-grpc-daemon[2206]: time=2026-09-22T11:01:14.404Z level=INFO msg="Upload complete. (341ms)"7221lb # [ 263.928277] niks3-server[1182]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7222worker1 # [ 263.979458] nix-grpc-daemon[1394]: ts=2026-09-22T11:01:14Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07223worker1 # [ 263.985403] nix-grpc-daemon[1394]: ts=2026-09-22T11:01:14Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17224worker2 # [ 263.957185] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38892 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=17225worker2 # [ 263.979022] nix-daemon[866]: reaped child process 2264, status = succeeded7226worker2 # [ 264.007663] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388927227worker2 # [ 264.024126] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388927228worker2 # [ 264.038503] nix-grpc-daemon[2206]: time=2026-09-22T11:01:14.497Z level=INFO msg="All 2 paths already cached"7229client # copying 1 paths to 'lb:50051'...7230client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7231worker2 # [ 264.049755] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:388927232worker2 # [ 264.063862] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7233lb # [ 264.219339] envoy[933]: conn peer=[2001:db8:1::1]:35176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300627234worker2 # [ 264.268407] nix-daemon[866]: accepted connection from pid 2289, user nix-grpc-daemon (trusted)7235worker2 # [ 264.289718] nix-daemon[866]: reaped child process 2292, status = succeeded7236worker2 # [ 264.464108] nix-daemon[866]: accepted connection from pid 2294, user nix-grpc-daemon (trusted)7237worker2 # [ 264.484455] nix-daemon[866]: reaped child process 2297, status = succeeded7238lb # [ 264.477691] niks3-server[1182]: 2026/09/22 11:01:14 INFO Received uploads request method=POST path=/api/pending_closures7239worker2 # [ 264.507320] nix-grpc-daemon[2206]: time=2026-09-22T11:01:14.966Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7240worker2 # [ 264.508946] nix-grpc-daemon[2206]: time=2026-09-22T11:01:14.966Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7241lb # [ 264.507731] niks3-server[1182]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7242lb # [ 264.522005] niks3-server[1182]: 2026/09/22 11:01:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7243worker2 # [ 264.550535] nix-grpc-daemon[2206]: time=2026-09-22T11:01:15.009Z level=INFO msg="Uploading 1 narinfos"7244lb # [ 264.534127] niks3-server[1182]: 2026/09/22 11:01:14 INFO Signed narinfos id=84 count=17245lb # [ 264.540462] niks3-server[1182]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7246lb # [ 264.549243] niks3-server[1182]: 2026/09/22 11:01:15 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7247lb # [ 264.555799] niks3-server[1182]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7248lb # [ 264.561536] niks3-server[1182]: 2026/09/22 11:01:15 INFO Completed upload id=847249worker2 # [ 264.581405] nix-grpc-daemon[2206]: time=2026-09-22T11:01:15.040Z level=INFO msg="Upload complete. (499ms)"7250worker2 # [ 264.590652] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38892 duration_s=0 paths=1 nar_bytes_in=7607251worker2 # [ 264.600707] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388927252worker2 # [ 264.612798] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7253worker2 # [ 264.617884] nix-daemon[866]: reaped child process 2287, status = succeeded7254client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7255worker2 # [ 264.629549] nix-daemon[866]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7256client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7257client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7258client # farm-top> LOG-farm-top-drain7259worker2 # [ 265.858795] nix-daemon[866]: reaped child process 2301, status = succeeded7260worker2 # [ 266.024072] nix-daemon[866]: accepted connection from pid 2317, user nix-grpc-daemon (trusted)7261worker2 # [ 266.044983] nix-daemon[866]: reaped child process 2320, status = succeeded7262lb # [ 266.037199] niks3-server[1182]: 2026/09/22 11:01:16 INFO Received uploads request method=POST path=/api/pending_closures7263worker2 # [ 266.066005] nix-grpc-daemon[2206]: time=2026-09-22T11:01:16.523Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7264worker2 # [ 266.067507] nix-grpc-daemon[2206]: time=2026-09-22T11:01:16.523Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7265lb # [ 266.067425] niks3-server[1182]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7266lb # [ 266.082951] niks3-server[1182]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7267worker2 # [ 266.119325] nix-grpc-daemon[2206]: time=2026-09-22T11:01:16.578Z level=INFO msg="Uploading 1 narinfos"7268lb # [ 266.095436] niks3-server[1182]: 2026/09/22 11:01:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7269lb # [ 266.104583] niks3-server[1182]: 2026/09/22 11:01:16 INFO Signed narinfos id=85 count=17270lb # [ 266.106886] niks3-server[1182]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7271lb # [ 266.116648] niks3-server[1182]: 2026/09/22 11:01:16 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7272lb # [ 266.123044] niks3-server[1182]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7273lb # [ 266.128383] niks3-server[1182]: 2026/09/22 11:01:16 INFO Completed upload id=857274worker2 # [ 266.148385] nix-grpc-daemon[2206]: time=2026-09-22T11:01:16.607Z level=INFO msg="Upload complete. (306ms)"7275worker2 # [ 266.161117] nix-grpc-daemon[2146]: ts=2026-09-22T11:01:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38892 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=17276worker2 # [ 266.177502] nix-daemon[866]: reaped child process 2299, status = succeeded7277client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.66 seconds)7278client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath7279client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.24 seconds)7280worker1: must succeed: rm /nix/.rw-store/fill7281worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7282worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7283worker1 # [ 267.531309] nix-grpc-daemon[1394]: ts=2026-09-22T11:01:17Z level=info event=healthy reason=min_free7284worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)7285(finished: subtest: low disk drains a worker and builds go to the other, in 10.50 seconds)7286(finished: run the VM test script, in 268.76 seconds)7287test script finished in 269.02s7288cleanup7289kill QemuMachine (pid 50)7290client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)7291kill QemuMachine (pid 49)7292lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)7293kill QemuMachine (pid 51)7294worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)7295kill QemuMachine (pid 52)7296worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)7297(finished: cleanup, in 1.06 seconds)7298additionally exposed symbols:7299 client, lb, worker1, worker2,7300 vlan1,7301 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh