vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #461
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Formatting '/build/vm-state-worker1/tmp.ZP74nZ2PVq', fmt=raw size=107374182414worker1 # mke2fs 1.47.4 (6-Mar-2025)15worker1 # Discarding device blocks: 0/262144 done16worker1: QEMU running (pid 51)17worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes18lb: QEMU running (pid 50)19worker2 # Disk image does not exist, creating the virtualisation disk image...20worker1 # Filesystem UUID: 60c27afa-cc42-4a7a-9fbb-12f3061e83b421worker2: QEMU running (pid 52)22lb # Disk image does not exist, creating the virtualisation disk image...23client: QEMU running (pid 49)24client # Disk image does not exist, creating the virtualisation disk image...25lb # Formatting '/build/vm-state-lb/tmp.gz8VOxGIu9', fmt=raw size=107374182426worker2 # Formatting '/build/vm-state-worker2/tmp.L3QXVz14Nu', fmt=raw size=107374182427worker1 # Superblock backups stored on blocks:28client # Formatting '/build/vm-state-client/tmp.bDBQ9H5Jzb', fmt=raw size=107374182429lb # mke2fs 1.47.4 (6-Mar-2025)30worker1 # 32768, 98304, 163840, 22937631client # mke2fs 1.47.4 (6-Mar-2025)32lb # Discarding device blocks: 0/262144 done33worker1 # 34client # Discarding device blocks: 0/262144 done35lb # Creating filesystem with 262144 4k blocks and 65536 inodes36worker1 # Allocating group tables: 0/8 done37client # Creating filesystem with 262144 4k blocks and 65536 inodes38lb # Filesystem UUID: cd1d32ea-968a-4845-8c6c-e9ad2a45865839worker1 # Writing inode tables: 0/8 done40client # Filesystem UUID: 5e27ae01-54fe-4082-93cd-eed420088bcd41lb # Superblock backups stored on blocks:42lb # 32768, 98304, 163840, 22937643worker1 # Creating journal (8192 blocks): done44(finished: start all VMs, in 0.52 seconds)45worker2 # mke2fs 1.47.4 (6-Mar-2025)46worker1 # Writing superblocks and filesystem accounting information: 0/8 done47client: waiting for unit network-addresses-eth1.service48worker2 # Discarding device blocks: 0/262144 done49lb # 50client: waiting for the VM to finish booting51worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes52lb # Allocating group tables: 0/8 done53client # Superblock backups stored on blocks:54lb # Writing inode tables: 0/8 done55client # 32768, 98304, 163840, 22937656worker2 # Filesystem UUID: 94753198-4bef-46d3-8096-7b888aaeec4f57worker1 # 58lb # Creating journal (8192 blocks): done59client # 60worker2 # Superblock backups stored on blocks:61worker1 # Virtualisation disk image created.62lb # Writing superblocks and filesystem accounting information: 0/8 done63client # Allocating group tables: 0/8 done64lb # 65client # Writing inode tables: 0/8 done66lb # Virtualisation disk image created.67client # Creating journal (8192 blocks): done68worker2 # 32768, 98304, 163840, 22937669client # Writing superblocks and filesystem accounting information: 0/8 done70worker2 # 71client # 72worker2 # Allocating group tables: 0/8 done73client # Virtualisation disk image created.74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker1 # [ 0.000000] KASLR enabled82worker1 # [ 0.000000] random: crng init done83worker1 # [ 0.000000] Machine model: linux,dummy-virt84worker1 # [ 0.000000] efi: UEFI not found.85worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88worker1 # [ 0.000000] Zone ranges:89worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90worker1 # [ 0.000000] DMA32 empty91worker1 # [ 0.000000] Normal empty92worker1 # [ 0.000000] Device empty93worker1 # [ 0.000000] Movable zone start for each node94worker1 # [ 0.000000] Early memory node ranges95worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098worker1 # [ 0.000000] psci: probing for conduit method from DT.99worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.100worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101worker1 # [ 0.000000] psci: Trusted OS migration not required102worker1 # [ 0.000000] psci: SMC Calling Convention v1.1103worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105worker1 # [ 0.000000] Detected PIPT I-cache on CPU0106worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface108worker1 # [ 0.000000] CPU features: detected: Spectre-v4109worker1 # [ 0.000000] CPU features: detected: Spectre-BHB110worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112worker1 # [ 0.000000] alternatives: applying boot alternatives113worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5zd1y7cz4jr6rf6lhvvzl3jz666fjcb5-nixos-system-worker1-test/init regInfo=/nix/store/2ad7c9kk7bzpgdb8r5y81xd71pygh303-closure-info/registration console=ttyAMA0,115200n8 console=tty0114worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/2ad7c9kk7bzpgdb8r5y81xd71pygh303-closure-info/registration", will be passed to user space.115worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119worker1 # [ 0.000000] software IO TLB: area num 1.120worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)122worker1 # [ 0.000000] Fallback order for Node 0: 0123worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]124worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216125worker1 # [ 0.000000] Policy zone: DMA126worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off127worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026128worker2 # [ 0.000000] KASLR enabled129worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1130worker2 # [ 0.000000] random: crng init done131worker1 # [ 0.000000] allocated 3145728 bytes of page_ext132worker2 # [ 0.000000] Machine model: linux,dummy-virt133worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages134worker2 # [ 0.000000] efi: UEFI not found.135worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups136worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT137worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.138worker1 # [ 0.000000] rcu: RCU event tracing is enabled.139worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]140worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]141worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.142worker2 # [ 0.000000] Zone ranges:143worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.144worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]145worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.146worker2 # [ 0.000000] DMA32 empty147worker2 # [ 0.000000] Normal empty148worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.149worker2 # [ 0.000000] Device empty150worker2 # [ 0.000000] Movable zone start for each node151worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.152worker2 # [ 0.000000] Early memory node ranges153worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1154worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]155worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.156worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]157worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000158worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.159worker2 # [ 0.000000] psci: probing for conduit method from DT.160worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.161worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.162worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs163client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]164worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0165worker2 # [ 0.000000] psci: Trusted OS migration not required166worker1 # [ 0.000000] GICv3: 256 SPIs implemented167worker2 # [ 0.000000] psci: SMC Calling Convention v1.1168worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented169client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026170worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq171worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)172client # [ 0.000000] KASLR enabled173worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI174client # [ 0.000000] random: crng init done175worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296176worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0177client # [ 0.000000] Machine model: linux,dummy-virt178worker2 # [ 0.000000] Detected PIPT I-cache on CPU0179client # [ 0.000000] efi: UEFI not found.180worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000181client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT182worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)183worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]184worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface185client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]186worker2 # [ 0.000000] CPU features: detected: Spectre-v4187worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)188client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]189worker2 # [ 0.000000] CPU features: detected: Spectre-BHB190client # [ 0.000000] Zone ranges:191worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)192worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38193client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]194client # [ 0.000000] DMA32 empty195worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000196worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23197client # [ 0.000000] Normal empty198client # [ 0.000000] Device empty199worker2 # [ 0.000000] alternatives: applying boot alternatives200worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000201client # [ 0.000000] Movable zone start for each node202client # [ 0.000000] Early memory node ranges203worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.204client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]205lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]206worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns207client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]208worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).209client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000210lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026211client # [ 0.000000] psci: probing for conduit method from DT.212worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gh79m3xzyqyvpdkgcwhw7570p93z0llj-nixos-system-worker2-test/init regInfo=/nix/store/bj7i3c8j1fg5v0svy6zi2d643q7qsc6l-closure-info/registration console=ttyAMA0,115200n8 console=tty0213lb # [ 0.000000] KASLR enabled214client # [ 0.000000] psci: PSCIv1.3 detected in firmware.215lb # [ 0.000000] random: crng init done216worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns217lb # [ 0.000000] Machine model: linux,dummy-virt218client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs219lb # [ 0.000000] efi: UEFI not found.220worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns221client # [ 0.000000] psci: Trusted OS migration not required222worker1 # [ 0.000036] arm-pv: using stolen time PV223client # [ 0.000000] psci: SMC Calling Convention v1.1224lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT225worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bj7i3c8j1fg5v0svy6zi2d643q7qsc6l-closure-info/registration", will be passed to user space.226lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]227client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)228worker1 # [ 0.000665] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)229worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes230worker1 # [ 0.000880] Console: colour dummy device 80x25231lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]232client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296233lb # [ 0.000000] Zone ranges:234worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)235worker1 # [ 0.000888] printk: legacy console [tty0] enabled236client # [ 0.000000] Detected PIPT I-cache on CPU0237lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]238worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)239lb # [ 0.000000] DMA32 empty240client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)241worker1 # [ 0.001103] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)242lb # [ 0.000000] Normal empty243worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB244worker1 # [ 0.001109] pid_max: default: 32768 minimum: 301245lb # [ 0.000000] Device empty246client # [ 0.000000] CPU features: detected: GICv3 CPU interface247worker2 # [ 0.000000] software IO TLB: area num 1.248lb # [ 0.000000] Movable zone start for each node249client # [ 0.000000] CPU features: detected: Spectre-v4250worker1 # [ 0.001187] LSM: initializing lsm=capability,landlock,yama,bpf,ima251lb # [ 0.000000] Early memory node ranges252client # [ 0.000000] CPU features: detected: Spectre-BHB253worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB254worker1 # [ 0.001356] landlock: Up and running.255lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]256worker1 # [ 0.001359] Yama: becoming mindful.257client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38258worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)259worker1 # [ 0.001991] LSM support for eBPF active260lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]261worker2 # [ 0.000000] Fallback order for Node 0: 0262client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23263worker1 # [ 0.002153] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)264lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000265client # [ 0.000000] alternatives: applying boot alternatives266worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216267worker2 # [ 0.000000] Policy zone: DMA268lb # [ 0.000000] psci: probing for conduit method from DT.269worker1 # [ 0.002201] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)270lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.271worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off272worker1 # [ 0.003355] cacheinfo: Unable to detect cache hierarchy for CPU 0273lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs274worker1 # [ 0.004109] rcu: Hierarchical SRCU implementation.275worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1276lb # [ 0.000000] psci: Trusted OS migration not required277worker2 # [ 0.000000] allocated 3145728 bytes of page_ext278worker1 # [ 0.004113] rcu: Max phase no-delay instances is 1000.279lb # [ 0.000000] psci: SMC Calling Convention v1.1280worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages281worker1 # [ 0.005306] fsl-mc MSI: its@8080000 domain created282client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gkhkardj3d6k7q4gs4n9bbjjpdx2saf4-nixos-system-client-test/init regInfo=/nix/store/wgb28qz9jy19gamz4ya93y7989416b0w-closure-info/registration console=ttyAMA0,115200n8 console=tty0283worker1 # [ 0.005397] EFI services will not be available.284worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups285lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)286worker1 # [ 0.005475] smp: Bringing up secondary CPUs ...287worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.288worker1 # [ 0.005484] smp: Brought up 1 node, 1 CPU289lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296290worker2 # [ 0.000000] rcu: RCU event tracing is enabled.291lb # [ 0.000000] Detected PIPT I-cache on CPU0292worker1 # [ 0.005487] SMP: Total of 1 processors activated.293client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/wgb28qz9jy19gamz4ya93y7989416b0w-closure-info/registration", will be passed to user space.294worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.295worker1 # [ 0.005493] CPU: All CPU(s) started at EL1296worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.297lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)298worker1 # [ 0.005504] CPU features: detected: Branch Target Identification299worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.300client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes301lb # [ 0.000000] CPU features: detected: GICv3 CPU interface302worker1 # [ 0.005509] CPU features: detected: ARMv8.4 Translation Table Level303worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.304lb # [ 0.000000] CPU features: detected: Spectre-v4305client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)306lb # [ 0.000000] CPU features: detected: Spectre-BHB307worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.308worker1 # [ 0.005512] CPU features: detected: Instruction cache invalidation not required for I/D coherence309lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38310client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)311worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1312worker1 # [ 0.005516] CPU features: detected: Data cache clean to the PoU not required for I/D coherence313client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB314lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23315client # [ 0.000000] software IO TLB: area num 1.316worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.317lb # [ 0.000000] alternatives: applying boot alternatives318worker1 # [ 0.005519] CPU features: detected: Common not Private translations319client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB320worker1 # [ 0.005523] CPU features: detected: CRC32 instructions321worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.322client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)323worker1 # [ 0.005525] CPU features: detected: Data cache clean to Point of Deep Persistence324client # [ 0.000000] Fallback order for Node 0: 0325worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.326worker1 # [ 0.005529] CPU features: detected: Data cache clean to Point of Persistence327worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0328client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216329client # [ 0.000000] Policy zone: DMA330worker2 # [ 0.000000] GICv3: 256 SPIs implemented331worker1 # [ 0.005532] CPU features: detected: Data independent timing control (DIT)332lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/px4c3ifawqq4xs8a4zhy519i865fkk09-nixos-system-lb-test/init regInfo=/nix/store/qp69c7d7ks4qa26wilaji9gbrwmdkkah-closure-info/registration console=ttyAMA0,115200n8 console=tty0333worker1 # [ 0.005535] CPU features: detected: E0PD334worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented335client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off336worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq337worker1 # [ 0.005538] CPU features: detected: Enhanced Counter Virtualization338client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1339worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI340client # [ 0.000000] allocated 3145728 bytes of page_ext341worker1 # [ 0.005540] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)342worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0343client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages344worker1 # [ 0.005544] CPU features: detected: Enhanced Virtualization Traps345worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000346lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qp69c7d7ks4qa26wilaji9gbrwmdkkah-closure-info/registration", will be passed to user space.347client # [ 0.000000] ftrace: allocated 294 pages with 4 groups348worker1 # [ 0.005547] CPU features: detected: Fine Grained Traps349worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]350client # [ 0.000000] rcu: Hierarchical RCU implementation.351lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes352client # [ 0.000000] rcu: RCU event tracing is enabled.353worker1 # [ 0.005550] CPU features: detected: Generic authentication (architected QARMA5 algorithm)354worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)355lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)356worker1 # [ 0.005555] CPU features: detected: RCpc load-acquire (LDAPR)357client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.358client # [ 0.000000] Trampoline variant of Tasks RCU enabled.359lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)360worker1 # [ 0.005558] CPU features: detected: LSE atomic instructions361worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)362client # [ 0.000000] Rude variant of Tasks RCU enabled.363worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000364lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB365worker1 # [ 0.005561] CPU features: detected: Privileged Access Never366client # [ 0.000000] Tracing variant of Tasks RCU enabled.367lb # [ 0.000000] software IO TLB: area num 1.368worker1 # [ 0.005564] CPU features: detected: PMUv3369worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000370client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.371worker1 # [ 0.005567] CPU features: detected: RAS Extension Support372lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB373worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.374worker1 # [ 0.005569] CPU features: detected: RASv1p1 Extension Support375client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1376lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)377worker1 # [ 0.005572] CPU features: detected: Random Number Generator378worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns379lb # [ 0.000000] Fallback order for Node 0: 0380client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.381worker1 # [ 0.005574] CPU features: detected: Speculation barrier (SB)382worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).383lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216384lb # [ 0.000000] Policy zone: DMA385worker1 # [ 0.005577] CPU features: detected: Stage-2 Force Write-Back386client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.387worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns388lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off389worker1 # [ 0.005580] CPU features: detected: TLB range maintenance instructions390client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.391lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1392worker1 # [ 0.005585] CPU features: detected: Speculative Store Bypassing Safe (SSBS)393worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns394client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0395worker2 # [ 0.000031] arm-pv: using stolen time PV396lb # [ 0.000000] allocated 3145728 bytes of page_ext397client # [ 0.000000] GICv3: 256 SPIs implemented398worker1 # [ 0.005625] alternatives: applying system-wide alternatives399lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages400client # [ 0.000000] GICv3: 0 Extended SPIs implemented401worker1 # [ 0.008614] CPU features: detected: BBM Level 2 without TLB conflict abort402worker2 # [ 0.000521] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)403client # [ 0.000000] Root IRQ handler: gic_handle_irq404lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups405worker2 # [ 0.000709] Console: colour dummy device 80x25406client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI407lb # [ 0.000000] rcu: Hierarchical RCU implementation.408worker2 # [ 0.000717] printk: legacy console [tty0] enabled409client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0410lb # [ 0.000000] rcu: RCU event tracing is enabled.411worker1 # [ 0.008766] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)412client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000413lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.414worker2 # [ 0.000925] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)415client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]416lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.417worker2 # [ 0.000932] pid_max: default: 32768 minimum: 301418lb # [ 0.000000] Rude variant of Tasks RCU enabled.419client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)420worker2 # [ 0.001018] LSM: initializing lsm=capability,landlock,yama,bpf,ima421lb # [ 0.000000] Tracing variant of Tasks RCU enabled.422worker2 # [ 0.001177] landlock: Up and running.423worker2 # [ 0.001180] Yama: becoming mindful.424client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)425lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.426worker2 # [ 0.001720] LSM support for eBPF active427client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000428lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1429worker2 # [ 0.001861] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)430client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000431lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.432worker2 # [ 0.001894] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)433client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.434worker2 # [ 0.003002] cacheinfo: Unable to detect cache hierarchy for CPU 0435lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.436worker2 # [ 0.003749] rcu: Hierarchical SRCU implementation.437client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns438worker2 # [ 0.003753] rcu: Max phase no-delay instances is 1000.439lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.440client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).441worker2 # [ 0.005009] fsl-mc MSI: its@8080000 domain created442lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0443worker2 # [ 0.005103] EFI services will not be available.444worker1 # [ 0.009199] devtmpfs: initialized445lb # [ 0.000000] GICv3: 256 SPIs implemented446worker2 # [ 0.005189] smp: Bringing up secondary CPUs ...447client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns448lb # [ 0.000000] GICv3: 0 Extended SPIs implemented449worker2 # [ 0.005197] smp: Brought up 1 node, 1 CPU450worker1 # [ 0.011110] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)451lb # [ 0.000000] Root IRQ handler: gic_handle_irq452worker2 # [ 0.005201] SMP: Total of 1 processors activated.453client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns454client # [ 0.000031] arm-pv: using stolen time PV455worker2 # [ 0.005206] CPU: All CPU(s) started at EL1456worker1 # [ 0.011134] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).457lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI458lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0459worker2 # [ 0.005218] CPU features: detected: Branch Target Identification460worker1 # [ 0.011318] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL461client # [ 0.000392] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)462lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000463worker1 # [ 0.011322] 0 pages in range for non-PLT usage464worker2 # [ 0.005224] CPU features: detected: ARMv8.4 Translation Table Level465client # [ 0.000552] Console: colour dummy device 80x25466lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]467worker1 # [ 0.011323] 508288 pages in range for PLT usage468client # [ 0.000560] printk: legacy console [tty0] enabled469worker2 # [ 0.005227] CPU features: detected: Instruction cache invalidation not required for I/D coherence470worker1 # [ 0.011447] pinctrl core: initialized pinctrl subsystem471lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)472worker1 # [ 0.012236] DMI not present or invalid.473client # [ 0.000756] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)474worker2 # [ 0.005230] CPU features: detected: Data cache clean to the PoU not required for I/D coherence475worker1 # [ 0.016445] NET: Registered PF_NETLINK/PF_ROUTE protocol family476client # [ 0.000762] pid_max: default: 32768 minimum: 301477worker2 # [ 0.005234] CPU features: detected: Common not Private translations478lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)479worker1 # [ 0.018702] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations480client # [ 0.000832] LSM: initializing lsm=capability,landlock,yama,bpf,ima481worker2 # [ 0.005237] CPU features: detected: CRC32 instructions482lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000483client # [ 0.000970] landlock: Up and running.484client # [ 0.000972] Yama: becoming mindful.485worker1 # [ 0.018886] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations486worker2 # [ 0.005240] CPU features: detected: Data cache clean to Point of Deep Persistence487lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000488client # [ 0.001456] LSM support for eBPF active489worker1 # [ 0.019096] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations490worker2 # [ 0.005243] CPU features: detected: Data cache clean to Point of Persistence491lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.492client # [ 0.001624] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)493worker1 # [ 0.019121] audit: initializing netlink subsys (disabled)494worker2 # [ 0.005247] CPU features: detected: Data independent timing control (DIT)495worker2 # [ 0.005250] CPU features: detected: E0PD496worker1 # [ 0.019763] thermal_sys: Registered thermal governor 'fair_share'497client # [ 0.001657] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)498lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns499client # [ 0.002773] cacheinfo: Unable to detect cache hierarchy for CPU 0500worker2 # [ 0.005252] CPU features: detected: Enhanced Counter Virtualization501worker1 # [ 0.019765] thermal_sys: Registered thermal governor 'bang_bang'502lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).503client # [ 0.003518] rcu: Hierarchical SRCU implementation.504worker1 # [ 0.019769] thermal_sys: Registered thermal governor 'step_wise'505worker2 # [ 0.005255] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)506client # [ 0.003522] rcu: Max phase no-delay instances is 1000.507lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns508worker1 # [ 0.019774] thermal_sys: Registered thermal governor 'user_space'509worker2 # [ 0.005258] CPU features: detected: Enhanced Virtualization Traps510client # [ 0.004734] fsl-mc MSI: its@8080000 domain created511worker2 # [ 0.005261] CPU features: detected: Fine Grained Traps512client # [ 0.004826] EFI services will not be available.513worker1 # [ 0.019777] thermal_sys: Registered thermal governor 'power_allocator'514lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns515client # [ 0.004891] smp: Bringing up secondary CPUs ...516lb # [ 0.000029] arm-pv: using stolen time PV517worker2 # [ 0.005265] CPU features: detected: Generic authentication (architected QARMA5 algorithm)518worker1 # [ 0.019809] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1519client # [ 0.004899] smp: Brought up 1 node, 1 CPU520worker1 # [ 0.019819] cpuidle: using governor ladder521worker2 # [ 0.005271] CPU features: detected: RCpc load-acquire (LDAPR)522client # [ 0.004902] SMP: Total of 1 processors activated.523worker1 # [ 0.019824] cpuidle: using governor menu524lb # [ 0.000420] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)525client # [ 0.004908] CPU: All CPU(s) started at EL1526worker2 # [ 0.005274] CPU features: detected: LSE atomic instructions527lb # [ 0.000593] Console: colour dummy device 80x25528worker1 # [ 0.020024] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.529client # [ 0.004918] CPU features: detected: Branch Target Identification530worker2 # [ 0.005277] CPU features: detected: Privileged Access Never531lb # [ 0.000600] printk: legacy console [tty0] enabled532worker1 # [ 0.020040] ASID allocator initialised with 65536 entries533worker2 # [ 0.005279] CPU features: detected: PMUv3534client # [ 0.004923] CPU features: detected: ARMv8.4 Translation Table Level535worker1 # [ 0.021358] Serial: AMBA PL011 UART driver536worker2 # [ 0.005282] CPU features: detected: RAS Extension Support537lb # [ 0.000786] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)538client # [ 0.004927] CPU features: detected: Instruction cache invalidation not required for I/D coherence539worker1 # [ 0.026794] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1540worker2 # [ 0.005285] CPU features: detected: RASv1p1 Extension Support541lb # [ 0.000793] pid_max: default: 32768 minimum: 301542worker1 # [ 0.026971] printk: console [ttyAMA0] enabled543worker2 # [ 0.005287] CPU features: detected: Random Number Generator544lb # [ 0.000860] LSM: initializing lsm=capability,landlock,yama,bpf,ima545client # [ 0.004930] CPU features: detected: Data cache clean to the PoU not required for I/D coherence546lb # [ 0.000990] landlock: Up and running.547worker2 # [ 0.005290] CPU features: detected: Speculation barrier (SB)548lb # [ 0.000993] Yama: becoming mindful.549client # [ 0.004934] CPU features: detected: Common not Private translations550worker1 # [ 0.149355] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages551worker2 # [ 0.005293] CPU features: detected: Stage-2 Force Write-Back552lb # [ 0.001469] LSM support for eBPF active553client # [ 0.004937] CPU features: detected: CRC32 instructions554worker1 # [ 0.149374] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page555worker2 # [ 0.005296] CPU features: detected: TLB range maintenance instructions556lb # [ 0.001632] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)557client # [ 0.004940] CPU features: detected: Data cache clean to Point of Deep Persistence558worker1 # [ 0.149379] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages559worker2 # [ 0.005301] CPU features: detected: Speculative Store Bypassing Safe (SSBS)560client # [ 0.004944] CPU features: detected: Data cache clean to Point of Persistence561lb # [ 0.001664] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)562worker2 # [ 0.005340] alternatives: applying system-wide alternatives563worker1 # [ 0.149383] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page564lb # [ 0.002798] cacheinfo: Unable to detect cache hierarchy for CPU 0565client # [ 0.004947] CPU features: detected: Data independent timing control (DIT)566worker2 # [ 0.008451] CPU features: detected: BBM Level 2 without TLB conflict abort567client # [ 0.004950] CPU features: detected: E0PD568worker1 # [ 0.149387] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages569lb # [ 0.003524] rcu: Hierarchical SRCU implementation.570worker1 # [ 0.149391] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page571client # [ 0.004952] CPU features: detected: Enhanced Counter Virtualization572lb # [ 0.003528] rcu: Max phase no-delay instances is 1000.573lb # [ 0.004727] fsl-mc MSI: its@8080000 domain created574worker1 # [ 0.149395] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages575worker2 # [ 0.008688] Memory: 1406496K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)576client # [ 0.004955] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)577lb # [ 0.004826] EFI services will not be available.578worker2 # [ 0.009086] devtmpfs: initialized579worker1 # [ 0.149400] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page580client # [ 0.004959] CPU features: detected: Enhanced Virtualization Traps581lb # [ 0.004892] smp: Bringing up secondary CPUs ...582worker2 # [ 0.011140] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)583client # [ 0.004962] CPU features: detected: Fine Grained Traps584lb # [ 0.004900] smp: Brought up 1 node, 1 CPU585lb # [ 0.004903] SMP: Total of 1 processors activated.586client # [ 0.004966] CPU features: detected: Generic authentication (architected QARMA5 algorithm)587lb # [ 0.004909] CPU: All CPU(s) started at EL1588client # [ 0.004971] CPU features: detected: RCpc load-acquire (LDAPR)589worker1 # [ 0.156617] fbcon: Taking over console590lb # [ 0.004921] CPU features: detected: Branch Target Identification591worker1 # [ 0.156634] ACPI: Interpreter disabled.592client # [ 0.004974] CPU features: detected: LSE atomic instructions593lb # [ 0.004926] CPU features: detected: ARMv8.4 Translation Table Level594client # [ 0.004977] CPU features: detected: Privileged Access Never595client # [ 0.004980] CPU features: detected: PMUv3596lb # [ 0.004930] CPU features: detected: Instruction cache invalidation not required for I/D coherence597client # [ 0.004982] CPU features: detected: RAS Extension Support598client # [ 0.004985] CPU features: detected: RASv1p1 Extension Support599lb # [ 0.004933] CPU features: detected: Data cache clean to the PoU not required for I/D coherence600client # [ 0.004988] CPU features: detected: Random Number Generator601lb # [ 0.004937] CPU features: detected: Common not Private translations602lb # [ 0.004940] CPU features: detected: CRC32 instructions603client # [ 0.004990] CPU features: detected: Speculation barrier (SB)604client # [ 0.004993] CPU features: detected: Stage-2 Force Write-Back605lb # [ 0.004943] CPU features: detected: Data cache clean to Point of Deep Persistence606client # [ 0.004996] CPU features: detected: TLB range maintenance instructions607lb # [ 0.004947] CPU features: detected: Data cache clean to Point of Persistence608client # [ 0.005001] CPU features: detected: Speculative Store Bypassing Safe (SSBS)609lb # [ 0.004950] CPU features: detected: Data independent timing control (DIT)610worker2 # [ 0.011169] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).611client # [ 0.005038] alternatives: applying system-wide alternatives612lb # [ 0.004953] CPU features: detected: E0PD613worker1 # [ 0.165227] iommu: Default domain type: Translated614worker2 # [ 0.011362] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL615lb # [ 0.004956] CPU features: detected: Enhanced Counter Virtualization616client # [ 0.008036] CPU features: detected: BBM Level 2 without TLB conflict abort617worker2 # [ 0.011367] 0 pages in range for non-PLT usage618worker1 # [ 0.165238] iommu: DMA domain TLB invalidation policy: strict mode619worker2 # [ 0.011368] 508288 pages in range for PLT usage620lb # [ 0.004959] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)621worker1 # [ 0.166952] SCSI subsystem initialized622worker2 # [ 0.011463] pinctrl core: initialized pinctrl subsystem623lb # [ 0.004962] CPU features: detected: Enhanced Virtualization Traps624client # [ 0.008194] Memory: 1406556K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)625worker2 # [ 0.012272] DMI not present or invalid.626client # [ 0.008614] devtmpfs: initialized627lb # [ 0.004965] CPU features: detected: Fine Grained Traps628worker2 # [ 0.015354] NET: Registered PF_NETLINK/PF_ROUTE protocol family629worker1 # [ 0.167587] usbcore: registered new interface driver usbfs630client # [ 0.010459] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)631lb # [ 0.004968] CPU features: detected: Generic authentication (architected QARMA5 algorithm)632worker1 # [ 0.167611] usbcore: registered new interface driver hub633worker2 # [ 0.017612] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations634worker1 # [ 0.167637] usbcore: registered new device driver usb635lb # [ 0.004974] CPU features: detected: RCpc load-acquire (LDAPR)636client # [ 0.010487] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).637worker2 # [ 0.017789] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations638worker1 # [ 0.167906] pps_core: LinuxPPS API ver. 1 registered639lb # [ 0.004977] CPU features: detected: LSE atomic instructions640client # [ 0.010695] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL641worker2 # [ 0.017989] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations642lb # [ 0.004980] CPU features: detected: Privileged Access Never643client # [ 0.010699] 0 pages in range for non-PLT usage644worker1 # [ 0.167912] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>645lb # [ 0.004983] CPU features: detected: PMUv3646worker2 # [ 0.018012] audit: initializing netlink subsys (disabled)647client # [ 0.010700] 508288 pages in range for PLT usage648worker1 # [ 0.167922] PTP clock support registered649lb # [ 0.004985] CPU features: detected: RAS Extension Support650worker1 # [ 0.167962] EDAC MC: Ver: 3.0.0651worker2 # [ 0.018637] thermal_sys: Registered thermal governor 'fair_share'652client # [ 0.010810] pinctrl core: initialized pinctrl subsystem653client # [ 0.011560] DMI not present or invalid.654worker2 # [ 0.018639] thermal_sys: Registered thermal governor 'bang_bang'655lb # [ 0.004988] CPU features: detected: RASv1p1 Extension Support656worker1 # [ 0.172486] scmi_core: SCMI protocol bus registered657client # [ 0.014628] NET: Registered PF_NETLINK/PF_ROUTE protocol family658worker2 # [ 0.018643] thermal_sys: Registered thermal governor 'step_wise'659lb # [ 0.004990] CPU features: detected: Random Number Generator660worker1 # [ 0.173420] FPGA manager framework661client # [ 0.017028] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations662worker2 # [ 0.018649] thermal_sys: Registered thermal governor 'user_space'663lb # [ 0.004993] CPU features: detected: Speculation barrier (SB)664worker1 # [ 0.174407] vgaarb: loaded665lb # [ 0.004996] CPU features: detected: Stage-2 Force Write-Back666worker2 # [ 0.018651] thermal_sys: Registered thermal governor 'power_allocator'667client # [ 0.017231] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations668lb # [ 0.004999] CPU features: detected: TLB range maintenance instructions669worker2 # [ 0.018683] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1670client # [ 0.017435] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations671worker1 # [ 0.175056] clocksource: Switched to clocksource arch_sys_counter672worker2 # [ 0.018691] cpuidle: using governor ladder673lb # [ 0.005004] CPU features: detected: Speculative Store Bypassing Safe (SSBS)674client # [ 0.017461] audit: initializing netlink subsys (disabled)675worker1 # [ 0.175657] VFS: Disk quotas dquot_6.6.0676worker2 # [ 0.018697] cpuidle: using governor menu677lb # [ 0.005042] alternatives: applying system-wide alternatives678client # [ 0.018019] thermal_sys: Registered thermal governor 'fair_share'679worker1 # [ 0.175690] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)680worker2 # [ 0.018884] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.681client # [ 0.018020] thermal_sys: Registered thermal governor 'bang_bang'682lb # [ 0.008106] CPU features: detected: BBM Level 2 without TLB conflict abort683worker2 # [ 0.018899] ASID allocator initialised with 65536 entries684worker2 # [ 0.020094] Serial: AMBA PL011 UART driver685client # [ 0.018024] thermal_sys: Registered thermal governor 'step_wise'686worker1 # [ 0.179036] netfs: FS-Cache loaded687client # [ 0.018029] thermal_sys: Registered thermal governor 'user_space'688worker2 # [ 0.025285] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1689lb # [ 0.008302] Memory: 1406556K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)690worker1 # [ 0.179587] pnp: PnP ACPI: disabled691lb # [ 0.008712] devtmpfs: initialized692worker2 # [ 0.025444] printk: console [ttyAMA0] enabled693lb # [ 0.010593] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)694worker2 # [ 0.144663] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages695lb # [ 0.010616] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).696worker2 # [ 0.144684] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page697lb # [ 0.010787] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL698worker2 # [ 0.144689] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages699lb # [ 0.010792] 0 pages in range for non-PLT usage700worker2 # [ 0.144693] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page701lb # [ 0.010793] 508288 pages in range for PLT usage702worker2 # [ 0.144698] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages703lb # [ 0.010905] pinctrl core: initialized pinctrl subsystem704lb # [ 0.011659] DMI not present or invalid.705worker2 # [ 0.144702] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page706lb # [ 0.015024] NET: Registered PF_NETLINK/PF_ROUTE protocol family707worker2 # [ 0.144706] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages708lb # [ 0.017245] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations709worker2 # [ 0.144711] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page710worker1 # [ 0.184054] NET: Registered PF_INET protocol family711lb # [ 0.017438] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations712client # [ 0.018032] thermal_sys: Registered thermal governor 'power_allocator'713worker1 # [ 0.184353] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)714lb # [ 0.017643] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations715client # [ 0.018057] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1716lb # [ 0.017668] audit: initializing netlink subsys (disabled)717worker2 # [ 0.152267] fbcon: Taking over console718client # [ 0.018065] cpuidle: using governor ladder719worker2 # [ 0.152283] ACPI: Interpreter disabled.720client # [ 0.018071] cpuidle: using governor menu721worker2 # [ 0.154150] iommu: Default domain type: Translated722client # [ 0.018272] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.723worker2 # [ 0.154159] iommu: DMA domain TLB invalidation policy: strict mode724client # [ 0.018287] ASID allocator initialised with 65536 entries725client # [ 0.019428] Serial: AMBA PL011 UART driver726worker2 # [ 0.155901] SCSI subsystem initialized727client # [ 0.024574] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1728client # [ 0.024699] printk: console [ttyAMA0] enabled729client # [ 0.144172] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages730client # [ 0.144193] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page731client # [ 0.144198] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages732client # [ 0.144202] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page733client # [ 0.144207] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages734lb # [ 0.018263] thermal_sys: Registered thermal governor 'fair_share'735client # [ 0.144211] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page736lb # [ 0.018265] thermal_sys: Registered thermal governor 'bang_bang'737client # [ 0.144215] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages738lb # [ 0.018268] thermal_sys: Registered thermal governor 'step_wise'739client # [ 0.144219] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page740lb # [ 0.018274] thermal_sys: Registered thermal governor 'user_space'741worker2 # [ 0.163513] usbcore: registered new interface driver usbfs742worker2 # [ 0.163548] usbcore: registered new interface driver hub743lb # [ 0.018277] thermal_sys: Registered thermal governor 'power_allocator'744worker2 # [ 0.163567] usbcore: registered new device driver usb745lb # [ 0.018309] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1746worker2 # [ 0.163872] pps_core: LinuxPPS API ver. 1 registered747client # [ 0.151771] fbcon: Taking over console748lb # [ 0.018318] cpuidle: using governor ladder749client # [ 0.151786] ACPI: Interpreter disabled.750lb # [ 0.018324] cpuidle: using governor menu751worker2 # [ 0.163878] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>752client # [ 0.153572] iommu: Default domain type: Translated753worker2 # [ 0.163888] PTP clock support registered754lb # [ 0.018523] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.755worker2 # [ 0.163927] EDAC MC: Ver: 3.0.0756client # [ 0.153581] iommu: DMA domain TLB invalidation policy: strict mode757lb # [ 0.018539] ASID allocator initialised with 65536 entries758lb # [ 0.019671] Serial: AMBA PL011 UART driver759worker2 # [ 0.168564] scmi_core: SCMI protocol bus registered760client # [ 0.155269] SCSI subsystem initialized761lb # [ 0.024947] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1762worker2 # [ 0.169522] FPGA manager framework763worker2 # [ 0.170470] vgaarb: loaded764lb # [ 0.025105] printk: console [ttyAMA0] enabled765lb # [ 0.148560] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages766worker2 # [ 0.171129] clocksource: Switched to clocksource arch_sys_counter767lb # [ 0.148578] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page768worker2 # [ 0.171740] VFS: Disk quotas dquot_6.6.0769lb # [ 0.148583] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages770worker2 # [ 0.171766] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)771lb # [ 0.148587] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page772worker2 # [ 0.174176] netfs: FS-Cache loaded773worker2 # [ 0.174278] pnp: PnP ACPI: disabled774lb # [ 0.148592] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages775lb # [ 0.148596] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page776lb # [ 0.148600] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages777lb # [ 0.148604] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page778client # [ 0.162917] usbcore: registered new interface driver usbfs779client # [ 0.162948] usbcore: registered new interface driver hub780client # [ 0.162978] usbcore: registered new device driver usb781client # [ 0.163274] pps_core: LinuxPPS API ver. 1 registered782lb # [ 0.156279] fbcon: Taking over console783lb # [ 0.156295] ACPI: Interpreter disabled.784client # [ 0.163279] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>785client # [ 0.163297] PTP clock support registered786lb # [ 0.158231] iommu: Default domain type: Translated787client # [ 0.163338] EDAC MC: Ver: 3.0.0788lb # [ 0.158241] iommu: DMA domain TLB invalidation policy: strict mode789client # [ 0.167988] scmi_core: SCMI protocol bus registered790client # [ 0.168930] FPGA manager framework791worker2 # [ 0.180325] NET: Registered PF_INET protocol family792client # [ 0.169943] vgaarb: loaded793worker2 # [ 0.180600] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)794client # [ 0.170587] clocksource: Switched to clocksource arch_sys_counter795client # [ 0.171928] VFS: Disk quotas dquot_6.6.0796client # [ 0.171958] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)797client # [ 0.173505] netfs: FS-Cache loaded798worker1 # [ 0.213946] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)799client # [ 0.173623] pnp: PnP ACPI: disabled800lb # [ 0.165203] SCSI subsystem initialized801worker1 # [ 0.214000] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)802lb # [ 0.165919] usbcore: registered new interface driver usbfs803worker1 # [ 0.214034] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)804lb # [ 0.165954] usbcore: registered new interface driver hub805worker1 # [ 0.214107] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)806lb # [ 0.165971] usbcore: registered new device driver usb807worker1 # [ 0.214253] TCP: Hash tables configured (established 16384 bind 16384)808lb # [ 0.166266] pps_core: LinuxPPS API ver. 1 registered809worker1 # [ 0.214343] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)810lb # [ 0.166272] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>811lb # [ 0.166282] PTP clock support registered812worker1 # [ 0.214380] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)813lb # [ 0.166329] EDAC MC: Ver: 3.0.0814worker1 # [ 0.214417] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)815lb # [ 0.171091] scmi_core: SCMI protocol bus registered816worker1 # [ 0.214508] NET: Registered PF_UNIX/PF_LOCAL protocol family817worker1 # [ 0.214554] NET: Registered PF_XDP protocol family818lb # [ 0.172082] FPGA manager framework819worker1 # [ 0.214573] PCI: CLS 0 bytes, default 64820lb # [ 0.173065] vgaarb: loaded821worker1 # [ 0.214845] Trying to unpack rootfs image as initramfs...822client # [ 0.179479] NET: Registered PF_INET protocol family823lb # [ 0.173710] clocksource: Switched to clocksource arch_sys_counter824client # [ 0.179753] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)825lb # [ 0.176672] VFS: Disk quotas dquot_6.6.0826lb # [ 0.176701] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)827worker1 # [ 0.232940] kvm [1]: HYP mode not available828lb # [ 0.180527] netfs: FS-Cache loaded829lb # [ 0.180657] pnp: PnP ACPI: disabled830lb # [ 0.184585] NET: Registered PF_INET protocol family831lb # [ 0.184850] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)832worker2 # [ 0.211366] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833worker2 # [ 0.211413] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834worker2 # [ 0.211442] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)835worker2 # [ 0.211518] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)836worker2 # [ 0.211661] TCP: Hash tables configured (established 16384 bind 16384)837worker2 # [ 0.211741] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)838client # [ 0.209969] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)839worker2 # [ 0.211776] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)840worker2 # [ 0.211811] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)841client # [ 0.210020] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)842worker2 # [ 0.211931] NET: Registered PF_UNIX/PF_LOCAL protocol family843client # [ 0.210052] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)844worker2 # [ 0.211970] NET: Registered PF_XDP protocol family845worker2 # [ 0.211989] PCI: CLS 0 bytes, default 64846client # [ 0.210129] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)847worker2 # [ 0.212223] Trying to unpack rootfs image as initramfs...848client # [ 0.210290] TCP: Hash tables configured (established 16384 bind 16384)849client # [ 0.210378] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)850client # [ 0.210415] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)851client # [ 0.210451] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)852worker2 # [ 0.228688] kvm [1]: HYP mode not available853client # [ 0.210541] NET: Registered PF_UNIX/PF_LOCAL protocol family854client # [ 0.210564] NET: Registered PF_XDP protocol family855client # [ 0.210582] PCI: CLS 0 bytes, default 64856client # [ 0.220078] Trying to unpack rootfs image as initramfs...857client # [ 0.227386] kvm [1]: HYP mode not available858lb # [ 0.215560] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)859lb # [ 0.215615] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)860lb # [ 0.215646] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)861lb # [ 0.215731] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)862lb # [ 0.215875] TCP: Hash tables configured (established 16384 bind 16384)863lb # [ 0.215962] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)864lb # [ 0.216000] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865lb # [ 0.216047] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866lb # [ 0.216152] NET: Registered PF_UNIX/PF_LOCAL protocol family867lb # [ 0.216193] NET: Registered PF_XDP protocol family868lb # [ 0.216211] PCI: CLS 0 bytes, default 64869lb # [ 0.216473] Trying to unpack rootfs image as initramfs...870lb # [ 0.231634] kvm [1]: HYP mode not available871worker1 # [ 0.339647] Initialise system trusted keyrings872worker1 # [ 0.340463] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker1 # [ 0.341729] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker1 # [ 0.342501] 9p: Installing v9fs 9p2000 file system support875worker1 # [ 0.371185] Key type asymmetric registered876worker1 # [ 0.371213] Asymmetric key parser 'x509' registered877worker1 # [ 0.371292] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)878worker1 # [ 0.373445] io scheduler mq-deadline registered879worker1 # [ 0.373457] io scheduler kyber registered880worker2 # [ 0.338818] Initialise system trusted keyrings881worker2 # [ 0.339627] workingset: timestamp_bits=42 max_order=19 bucket_order=0882worker2 # [ 0.340900] squashfs: version 4.0 (2009/01/31) Phillip Lougher883worker2 # [ 0.341658] 9p: Installing v9fs 9p2000 file system support884worker1 # [ 0.383200] pl061_gpio 9030000.pl061: PL061 GPIO chip registered885client # [ 0.335222] Initialise system trusted keyrings886client # [ 0.335973] workingset: timestamp_bits=42 max_order=19 bucket_order=0887worker1 # [ 0.384623] ledtrig-cpu: registered to indicate activity on CPUs888client # [ 0.337262] squashfs: version 4.0 (2009/01/31) Phillip Lougher889worker1 # [ 0.385005] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:890client # [ 0.338022] 9p: Installing v9fs 9p2000 file system support891worker1 # [ 0.385021] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000892worker1 # [ 0.385033] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000893worker1 # [ 0.385041] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000894worker1 # [ 0.385064] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits895worker1 # [ 0.385089] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]896worker1 # [ 0.385171] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00897worker1 # [ 0.385180] pci_bus 0000:00: root bus resource [bus 00-ff]898worker1 # [ 0.385186] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]899worker1 # [ 0.385191] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]900worker1 # [ 0.385196] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]901worker1 # [ 0.385283] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint902worker1 # [ 0.385721] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint903worker1 # [ 0.385910] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]904worker1 # [ 0.385926] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]905lb # [ 0.342255] Initialise system trusted keyrings906worker1 # [ 0.385955] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]907worker2 # [ 0.365604] Key type asymmetric registered908lb # [ 0.343032] workingset: timestamp_bits=42 max_order=19 bucket_order=0909worker1 # [ 0.385971] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]910worker2 # [ 0.365631] Asymmetric key parser 'x509' registered911lb # [ 0.344297] squashfs: version 4.0 (2009/01/31) Phillip Lougher912worker1 # [ 0.386417] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint913worker2 # [ 0.365704] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)914worker1 # [ 0.386601] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]915lb # [ 0.345069] 9p: Installing v9fs 9p2000 file system support916worker1 # [ 0.386617] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]917worker1 # [ 0.386646] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]918worker1 # [ 0.406289] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint919worker1 # [ 0.406480] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]920worker1 # [ 0.406496] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]921worker1 # [ 0.406526] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]922worker1 # [ 0.407015] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint923worker2 # [ 0.371193] io scheduler mq-deadline registered924worker2 # [ 0.371215] io scheduler kyber registered925client # [ 0.366738] Key type asymmetric registered926client # [ 0.366768] Asymmetric key parser 'x509' registered927client # [ 0.366839] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)928client # [ 0.368974] io scheduler mq-deadline registered929worker1 # [ 0.411442] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]930client # [ 0.368986] io scheduler kyber registered931worker1 # [ 0.411461] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]932worker1 # [ 0.411494] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]933worker1 # [ 0.411961] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint934worker1 # [ 0.412147] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]935worker1 # [ 0.412163] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]936worker1 # [ 0.412203] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]937lb # [ 0.365677] Key type asymmetric registered938worker1 # [ 0.412659] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint939worker1 # [ 0.412843] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]940worker1 # [ 0.412859] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]941worker1 # [ 0.412887] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]942worker2 # [ 0.383281] pl061_gpio 9030000.pl061: PL061 GPIO chip registered943worker2 # [ 0.383923] ledtrig-cpu: registered to indicate activity on CPUs944worker1 # [ 0.413341] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint945worker1 # [ 0.413529] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]946worker2 # [ 0.384336] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:947client # [ 0.378717] pl061_gpio 9030000.pl061: PL061 GPIO chip registered948worker1 # [ 0.413544] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]949worker2 # [ 0.384354] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000950worker1 # [ 0.413574] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]951worker2 # [ 0.384366] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000952worker1 # [ 0.413590] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]953worker1 # [ 0.414086] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint954worker2 # [ 0.384383] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000955client # [ 0.380040] ledtrig-cpu: registered to indicate activity on CPUs956lb # [ 0.373742] Asymmetric key parser 'x509' registered957worker1 # [ 0.414275] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]958worker2 # [ 0.384404] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits959client # [ 0.380411] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:960lb # [ 0.373849] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)961worker1 # [ 0.414305] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]962lb # [ 0.375456] io scheduler mq-deadline registered963worker2 # [ 0.384427] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]964client # [ 0.380427] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000965lb # [ 0.375467] io scheduler kyber registered966worker1 # [ 0.414757] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint967worker2 # [ 0.384502] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00968worker1 # [ 0.414946] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]969client # [ 0.380439] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000970worker2 # [ 0.384511] pci_bus 0000:00: root bus resource [bus 00-ff]971worker1 # [ 0.414975] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]972worker2 # [ 0.384517] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]973client # [ 0.380447] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000974worker1 # [ 0.415377] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint975worker2 # [ 0.384522] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]976client # [ 0.380477] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits977worker1 # [ 0.415561] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]978worker2 # [ 0.384527] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]979client # [ 0.380500] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]980worker1 # [ 0.415815] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint981worker2 # [ 0.384636] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint982worker1 # [ 0.416109] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]983client # [ 0.380573] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00984worker2 # [ 0.385094] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint985worker1 # [ 0.416126] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]986client # [ 0.380582] pci_bus 0000:00: root bus resource [bus 00-ff]987worker2 # [ 0.385281] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]988client # [ 0.380588] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]989worker1 # [ 0.416155] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]990worker2 # [ 0.385301] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]991client # [ 0.380592] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]992worker1 # [ 0.416629] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint993worker2 # [ 0.385331] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker1 # [ 0.416815] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]995client # [ 0.380597] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]996worker2 # [ 0.385348] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]997worker1 # [ 0.416831] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]998client # [ 0.380653] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint999worker2 # [ 0.385806] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1000worker1 # [ 0.416860] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1001worker2 # [ 0.385992] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1002client # [ 0.381088] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1003worker1 # [ 0.417430] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1004client # [ 0.381277] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1005worker2 # [ 0.386008] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1006lb # [ 0.385850] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1007worker1 # [ 0.417441] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1008client # [ 0.381293] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1009worker2 # [ 0.386037] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1010client # [ 0.381322] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1011worker1 # [ 0.417447] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1012worker2 # [ 0.386492] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1013client # [ 0.381338] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1014worker2 # [ 0.386673] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1015worker1 # [ 0.417491] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1016worker2 # [ 0.386688] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1017client # [ 0.381803] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1018worker1 # [ 0.417537] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1019lb # [ 0.387270] ledtrig-cpu: registered to indicate activity on CPUs1020client # [ 0.381987] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1021worker2 # [ 0.386718] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1022client # [ 0.382002] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1023worker1 # [ 0.417583] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1024lb # [ 0.387669] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1025worker2 # [ 0.387203] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1026client # [ 0.382032] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1027worker1 # [ 0.417630] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1028worker2 # [ 0.387384] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1029lb # [ 0.387688] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001030worker2 # [ 0.387400] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1031client # [ 0.382504] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1032worker1 # [ 0.417676] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1033lb # [ 0.387709] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001034worker2 # [ 0.387432] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035worker1 # [ 0.417724] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1036client # [ 0.402001] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1037lb # [ 0.387717] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001038worker2 # [ 0.387896] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1039worker1 # [ 0.417771] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1040client # [ 0.402021] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1041worker2 # [ 0.388091] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1042lb # [ 0.387749] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1043client # [ 0.402050] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1044worker1 # [ 0.417818] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1045worker2 # [ 0.388108] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1046lb # [ 0.387774] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1047client # [ 0.402538] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1048worker2 # [ 0.388137] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049worker1 # [ 0.417866] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1050lb # [ 0.387856] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001051worker2 # [ 0.388602] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1052worker1 # [ 0.417940] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1053lb # [ 0.387865] pci_bus 0000:00: root bus resource [bus 00-ff]1054worker2 # [ 0.388784] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1055client # [ 0.405691] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1056worker1 # [ 0.417986] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1057lb # [ 0.387871] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1058worker2 # [ 0.388799] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1059client # [ 0.405710] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1060worker1 # [ 0.418008] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1061lb # [ 0.387876] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1062worker2 # [ 0.388829] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1063client # [ 0.405742] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064worker1 # [ 0.418029] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1065lb # [ 0.387881] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1066worker2 # [ 0.389282] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1067client # [ 0.406201] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1068worker1 # [ 0.418051] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1069worker2 # [ 0.389464] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1070client # [ 0.406387] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1071lb # [ 0.387941] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1072worker1 # [ 0.418075] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1073worker2 # [ 0.389479] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1074client # [ 0.406403] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1075worker1 # [ 0.418096] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1076lb # [ 0.388385] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1077worker2 # [ 0.389509] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078client # [ 0.406432] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1079lb # [ 0.388572] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1080worker1 # [ 0.418118] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1081worker2 # [ 0.389525] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1082lb # [ 0.388589] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1083worker1 # [ 0.418139] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1084worker2 # [ 0.390018] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1085lb # [ 0.388619] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086worker1 # [ 0.418161] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1087worker2 # [ 0.390204] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1088lb # [ 0.388635] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1089worker1 # [ 0.418183] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1090worker2 # [ 0.390234] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1091lb # [ 0.389098] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1092worker1 # [ 0.418206] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1093worker2 # [ 0.390689] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1094lb # [ 0.389284] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1095worker1 # [ 0.418229] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1096worker2 # [ 0.390873] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1097lb # [ 0.389300] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1098worker1 # [ 0.418250] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1099worker2 # [ 0.390902] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100lb # [ 0.389330] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101worker1 # [ 0.418271] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1102worker1 # [ 0.418292] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1103worker1 # [ 0.418313] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1104lb # [ 0.408684] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1105worker1 # [ 0.418334] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1106lb # [ 0.408878] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1107worker1 # [ 0.418355] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1108lb # [ 0.408895] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1109worker1 # [ 0.418377] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1110lb # [ 0.408925] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111worker1 # [ 0.418398] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1112worker1 # [ 0.418420] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1113lb # [ 0.409403] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1114worker1 # [ 0.418446] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1115lb # [ 0.409587] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1116lb # [ 0.409603] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1117worker1 # [ 0.418455] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1118worker1 # [ 0.418460] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1119lb # [ 0.409636] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1120worker2 # [ 0.435364] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1121worker2 # [ 0.435558] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1122client # [ 0.418984] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1123worker2 # [ 0.435821] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1124client # [ 0.419195] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1125worker2 # [ 0.436134] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1126client # [ 0.419212] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1127worker2 # [ 0.436152] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1128client # [ 0.419241] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129worker2 # [ 0.436182] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1130client # [ 0.419763] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1131worker2 # [ 0.436652] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1132client # [ 0.419952] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1133worker2 # [ 0.436833] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1134client # [ 0.419968] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1135worker2 # [ 0.436849] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1136client # [ 0.419997] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137worker2 # [ 0.436878] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1138client # [ 0.420013] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1139worker2 # [ 0.437455] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1140client # [ 0.420465] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1141worker2 # [ 0.437466] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1142client # [ 0.420652] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1143worker1 # [ 0.483715] pci 0000:00:0a.0: enabling device (0000 -> 0002)1144worker2 # [ 0.437471] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1145client # [ 0.420681] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1146worker2 # [ 0.437517] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1147client # [ 0.421129] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1148worker2 # [ 0.437563] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1149client # [ 0.421318] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1150client # [ 0.421347] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1151worker2 # [ 0.437611] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1152lb # [ 0.422185] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1153client # [ 0.421737] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1154worker2 # [ 0.437657] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1155lb # [ 0.422391] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1156client # [ 0.421921] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1157lb # [ 0.422408] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1158worker2 # [ 0.437704] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1159client # [ 0.422167] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1160lb # [ 0.422439] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1161worker2 # [ 0.437752] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1162client # [ 0.422462] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1163lb # [ 0.422912] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1164worker2 # [ 0.437804] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1165client # [ 0.422479] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1166lb # [ 0.423097] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1167client # [ 0.422508] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1168worker2 # [ 0.437852] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1169lb # [ 0.423113] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1170client # [ 0.422996] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1171worker2 # [ 0.437899] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1172lb # [ 0.423143] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1173client # [ 0.423179] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1174worker2 # [ 0.437977] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1175lb # [ 0.423631] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1176client # [ 0.423195] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1177worker2 # [ 0.438024] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1178lb # [ 0.423818] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1179client # [ 0.423224] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180worker2 # [ 0.438048] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1181lb # [ 0.423834] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1182client # [ 0.423831] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1183worker2 # [ 0.438070] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1184lb # [ 0.423864] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1185client # [ 0.423841] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1186worker2 # [ 0.438092] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1187lb # [ 0.423880] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1188client # [ 0.423847] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1189worker2 # [ 0.438116] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1190lb # [ 0.424345] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1191client # [ 0.423893] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1192worker2 # [ 0.438138] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1193lb # [ 0.424533] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1194worker2 # [ 0.438160] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1195client # [ 0.423940] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1196lb # [ 0.424563] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1197worker2 # [ 0.438183] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1198client # [ 0.423987] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1199lb # [ 0.425016] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1200worker2 # [ 0.438206] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1201lb # [ 0.425206] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1202client # [ 0.424034] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1203worker2 # [ 0.438228] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1204lb # [ 0.425236] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1205client # [ 0.424085] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1206worker2 # [ 0.438251] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1207lb # [ 0.425649] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1208client # [ 0.424131] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1209worker2 # [ 0.438274] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1210lb # [ 0.425850] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1211worker2 # [ 0.438295] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1212client # [ 0.424177] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1213worker2 # [ 0.438317] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1214lb # [ 0.426105] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1215client # [ 0.424224] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1216worker2 # [ 0.438339] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1217lb # [ 0.426383] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1218client # [ 0.424271] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1219lb # [ 0.426400] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1220worker2 # [ 0.438360] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1221worker2 # [ 0.438382] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1222lb # [ 0.426430] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1223client # [ 0.424344] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1224worker2 # [ 0.438404] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1225client # [ 0.424391] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1226lb # [ 0.426919] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1227worker2 # [ 0.438426] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1228client # [ 0.424412] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1229lb # [ 0.427105] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1230worker2 # [ 0.438447] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1231lb # [ 0.427121] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1232client # [ 0.424434] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1233worker2 # [ 0.438469] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1234lb # [ 0.427151] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1235client # [ 0.424457] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1236worker2 # [ 0.438495] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1237worker1 # [ 0.509285] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1238client # [ 0.424482] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1239lb # [ 0.427731] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1240worker2 # [ 0.438505] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1241client # [ 0.424504] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1242lb # [ 0.427742] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1243worker2 # [ 0.438509] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1244client # [ 0.424525] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1245lb # [ 0.427748] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1246client # [ 0.424547] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1247lb # [ 0.427792] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1248client # [ 0.424568] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1249lb # [ 0.427840] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1250client # [ 0.424590] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1251lb # [ 0.427887] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1252client # [ 0.424613] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1253client # [ 0.424635] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1254lb # [ 0.427934] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1255client # [ 0.424656] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1256lb # [ 0.427980] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1257client # [ 0.424676] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1258client # [ 0.424697] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1259lb # [ 0.428027] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1260worker1 # [ 0.515348] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1261client # [ 0.424719] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1262lb # [ 0.428074] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1263worker1 # [ 0.517586] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1264client # [ 0.424739] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1265lb # [ 0.428122] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1266worker2 # [ 0.483772] pci 0000:00:0a.0: enabling device (0000 -> 0002)1267client # [ 0.424760] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1268client # [ 0.424782] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1269lb # [ 0.428169] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1270client # [ 0.424803] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1271lb # [ 0.428241] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1272client # [ 0.424825] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1273lb # [ 0.428287] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1274client # [ 0.424852] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1275lb # [ 0.428309] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1276client # [ 0.424861] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1277lb # [ 0.428331] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1278client # [ 0.424865] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1279client # [ 0.426046] pci 0000:00:0a.0: enabling device (0000 -> 0002)1280lb # [ 0.428353] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1281lb # [ 0.428378] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1282lb # [ 0.428400] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1283lb # [ 0.428422] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1284lb # [ 0.428444] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1285lb # [ 0.428466] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1286worker1 # [ 0.527690] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1287lb # [ 0.428488] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1288worker1 # [ 0.529764] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1289lb # [ 0.428511] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1290lb # [ 0.428534] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1291lb # [ 0.428556] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1292lb # [ 0.428577] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1293lb # [ 0.428599] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1294lb # [ 0.428620] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1295lb # [ 0.428642] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1296lb # [ 0.428664] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1297lb # [ 0.428686] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1298lb # [ 0.428707] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1299lb # [ 0.428730] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1300lb # [ 0.428756] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1301lb # [ 0.428766] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1302lb # [ 0.428770] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1303worker2 # [ 0.504831] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1304worker1 # [ 0.539638] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1305worker1 # [ 0.542720] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1306worker1 # [ 0.544867] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1307worker1 # [ 0.546912] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1308lb # [ 0.490038] pci 0000:00:0a.0: enabling device (0000 -> 0002)1309client # [ 0.502071] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1310worker2 # [ 0.515480] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1311worker2 # [ 0.517818] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1312worker1 # [ 0.556543] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1313client # [ 0.512035] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1314worker1 # [ 0.563190] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1315worker2 # [ 0.527776] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1316worker2 # [ 0.529929] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1317client # [ 0.516538] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1318worker2 # [ 0.532101] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1319client # [ 0.518545] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1320client # [ 0.520630] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1321worker2 # [ 0.534307] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1322worker1 # [ 0.576344] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1323lb # [ 0.514897] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1324lb # [ 0.517097] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1325worker1 # [ 0.578864] msm_serial: driver initialized1326worker1 # [ 0.579012] SuperH (H)SCI(F) driver initialized1327worker2 # [ 0.544565] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1328worker2 # [ 0.546695] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1329worker1 # [ 0.580206] STM32 USART driver initialized1330lb # [ 0.528116] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1331lb # [ 0.531083] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1332lb # [ 0.533108] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1333worker2 # [ 0.556520] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1334client # [ 0.538837] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1335worker2 # [ 0.558793] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1336client # [ 0.541202] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1337client # [ 0.543435] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1338client # [ 0.545884] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1339lb # [ 0.543302] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1340worker2 # [ 0.571300] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1341client # [ 0.556118] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1342lb # [ 0.549789] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1343lb # [ 0.551843] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1344worker1 # [ 0.613946] loop: module loaded1345worker1 # [ 0.614154] virtio_blk virtio5: 1/0/0 default/read/poll queues1346worker1 # [ 0.614952] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1347client # [ 0.562691] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1348worker2 # [ 0.580539] msm_serial: driver initialized1349worker2 # [ 0.580727] SuperH (H)SCI(F) driver initialized1350worker2 # [ 0.580780] STM32 USART driver initialized1351worker1 # [ 0.619775] megasas: 07.734.00.00-rc11352lb # [ 0.561936] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1353lb # [ 0.563823] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1354worker1 # [ 0.620512] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1355client # [ 0.575918] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1356client # [ 0.578516] msm_serial: driver initialized1357client # [ 0.579193] SuperH (H)SCI(F) driver initialized1358client # [ 0.579250] STM32 USART driver initialized1359worker1 # [ 0.632237] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001360worker1 # [ 0.632283] Intel/Sharp Extended Query Table at 0x00311361worker1 # [ 0.634013] Using buffer write method1362lb # [ 0.574337] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1363worker1 # [ 0.634106] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1364lb # [ 0.586458] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1365worker2 # [ 0.609998] loop: module loaded1366lb # [ 0.589232] msm_serial: driver initialized1367worker2 # [ 0.610187] virtio_blk virtio5: 1/0/0 default/read/poll queues1368lb # [ 0.589408] SuperH (H)SCI(F) driver initialized1369worker2 # [ 0.610951] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1370lb # [ 0.589463] STM32 USART driver initialized1371worker1 # [ 0.649938] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001372worker1 # [ 0.649995] Intel/Sharp Extended Query Table at 0x00311373worker2 # [ 0.615819] megasas: 07.734.00.00-rc11374worker2 # [ 0.616630] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1375worker2 # [ 0.618532] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001376worker2 # [ 0.618558] Intel/Sharp Extended Query Table at 0x00311377worker1 # [ 0.659710] Using buffer write method1378worker1 # [ 0.659760] Concatenating MTD devices:1379worker1 # [ 0.659765] (0): "0.flash"1380worker1 # [ 0.659769] (1): "0.flash"1381worker1 # [ 0.659773] into device "0.flash"1382client # [ 0.612196] loop: module loaded1383client # [ 0.612390] virtio_blk virtio5: 1/0/0 default/read/poll queues1384client # [ 0.613167] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1385worker2 # [ 0.629022] Using buffer write method1386worker2 # [ 0.629098] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1387worker2 # [ 0.630602] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001388worker2 # [ 0.630652] Intel/Sharp Extended Query Table at 0x00311389client # [ 0.619273] megasas: 07.734.00.00-rc11390client # [ 0.620043] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1391worker2 # [ 0.640088] Using buffer write method1392worker2 # [ 0.640123] Concatenating MTD devices:1393worker2 # [ 0.640127] (0): "0.flash"1394worker2 # [ 0.640131] (1): "0.flash"1395worker2 # [ 0.640134] into device "0.flash"1396lb # [ 0.620521] loop: module loaded1397lb # [ 0.620715] virtio_blk virtio5: 1/0/0 default/read/poll queues1398lb # [ 0.621498] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1399client # [ 0.631839] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001400client # [ 0.631878] Intel/Sharp Extended Query Table at 0x00311401client # [ 0.634060] Using buffer write method1402client # [ 0.634137] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1403client # [ 0.639303] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001404client # [ 0.639340] Intel/Sharp Extended Query Table at 0x00311405client # [ 0.641101] Using buffer write method1406client # [ 0.641128] Concatenating MTD devices:1407client # [ 0.641132] (0): "0.flash"1408client # [ 0.641136] (1): "0.flash"1409client # [ 0.641139] into device "0.flash"1410lb # [ 0.634446] megasas: 07.734.00.00-rc11411lb # [ 0.635172] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1412lb # [ 0.637551] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001413lb # [ 0.637580] Intel/Sharp Extended Query Table at 0x00311414lb # [ 0.639098] Using buffer write method1415lb # [ 0.639169] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1416lb # [ 0.640848] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417lb # [ 0.640878] Intel/Sharp Extended Query Table at 0x00311418lb # [ 0.658404] Using buffer write method1419lb # [ 0.658446] Concatenating MTD devices:1420lb # [ 0.658450] (0): "0.flash"1421lb # [ 0.658454] (1): "0.flash"1422lb # [ 0.658457] into device "0.flash"1423worker1 # [ 0.909069] Freeing initrd memory: 26148K1424worker1 # [ 0.915220] tun: Universal TUN/TAP device driver, 1.61425worker1 # [ 0.919147] thunder_xcv, ver 1.01426worker1 # [ 0.919194] thunder_bgx, ver 1.01427worker1 # [ 0.919217] nicpf, ver 1.01428worker1 # [ 0.919773] e1000: Intel(R) PRO/1000 Network Driver1429worker1 # [ 0.919780] e1000: Copyright (c) 1999-2006 Intel Corporation.1430worker1 # [ 0.919807] e1000e: Intel(R) PRO/1000 Network Driver1431worker1 # [ 0.919816] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432worker1 # [ 0.919846] igb: Intel(R) Gigabit Ethernet Network Driver1433worker1 # [ 0.919852] igb: Copyright (c) 2007-2014 Intel Corporation.1434worker1 # [ 0.919876] igbvf: Intel(R) Gigabit Virtual Function Network Driver1435worker1 # [ 0.919882] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1436worker1 # [ 0.920014] sky2: driver version 1.301437worker1 # [ 0.921587] usbcore: registered new interface driver usb-storage1438worker1 # [ 0.921659] usbcore: registered new interface driver usbserial_generic1439worker1 # [ 0.921673] usbserial: USB Serial support registered for generic1440worker1 # [ 0.922253] hv_vmbus: registering driver hyperv_keyboard1441worker1 # [ 0.923562] ehci-pci 0000:00:0a.0: EHCI Host Controller1442worker2 # [ 0.898337] Freeing initrd memory: 26148K1443worker1 # [ 0.923592] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11444worker1 # [ 0.923826] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001445worker1 # [ 0.936317] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001446worker1 # [ 0.936679] hub 1-0:1.0: USB hub found1447worker1 # [ 0.936699] hub 1-0:1.0: 6 ports detected1448worker1 # [ 0.938929] rtc-pl031 9010000.pl031: registered as rtc01449worker1 # [ 0.938959] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T21:08:24 UTC (1789938504)1450worker2 # [ 0.904385] tun: Universal TUN/TAP device driver, 1.61451worker1 # [ 0.941139] i2c_dev: i2c /dev entries driver1452worker1 # [ 0.944393] sdhci: Secure Digital Host Controller Interface driver1453worker1 # [ 0.944404] sdhci: Copyright(c) Pierre Ossman1454worker2 # [ 0.908136] thunder_xcv, ver 1.01455worker1 # [ 0.944667] Synopsys Designware Multimedia Card Interface Driver1456worker2 # [ 0.908183] thunder_bgx, ver 1.01457worker2 # [ 0.908206] nicpf, ver 1.01458worker1 # [ 0.945027] sdhci-pltfm: SDHCI platform and OF driver helper1459worker2 # [ 0.908761] e1000: Intel(R) PRO/1000 Network Driver1460worker1 # [ 0.946680] hid: raw HID events driver (C) Jiri Kosina1461worker2 # [ 0.908770] e1000: Copyright (c) 1999-2006 Intel Corporation.1462worker1 # [ 0.946918] usbcore: registered new interface driver usbhid1463worker2 # [ 0.908800] e1000e: Intel(R) PRO/1000 Network Driver1464worker1 # [ 0.946924] usbhid: USB HID core driver1465client # [ 0.901397] Freeing initrd memory: 26160K1466worker2 # [ 0.908809] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1467worker2 # [ 0.908837] igb: Intel(R) Gigabit Ethernet Network Driver1468worker2 # [ 0.908843] igb: Copyright (c) 2007-2014 Intel Corporation.1469worker2 # [ 0.908869] igbvf: Intel(R) Gigabit Virtual Function Network Driver1470worker2 # [ 0.908875] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1471worker2 # [ 0.909007] sky2: driver version 1.301472worker1 # [ 0.952584] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1473worker2 # [ 0.910609] usbcore: registered new interface driver usb-storage1474worker1 # [ 0.954059] drop_monitor: Initializing network drop monitor service1475worker2 # [ 0.910694] usbcore: registered new interface driver usbserial_generic1476worker1 # [ 0.954211] NET: Registered PF_INET6 protocol family1477worker2 # [ 0.910709] usbserial: USB Serial support registered for generic1478worker1 # [ 0.957361] Segment Routing with IPv61479client # [ 0.907474] tun: Universal TUN/TAP device driver, 1.61480worker2 # [ 0.911928] ehci-pci 0000:00:0a.0: EHCI Host Controller1481worker1 # [ 0.957385] In-situ OAM (IOAM) with IPv61482worker1 # [ 0.957434] NET: Registered PF_PACKET protocol family1483worker2 # [ 0.912014] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11484worker2 # [ 0.912184] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001485worker2 # [ 0.923774] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001486worker2 # [ 0.924218] hub 1-0:1.0: USB hub found1487worker2 # [ 0.924234] hub 1-0:1.0: 6 ports detected1488worker1 # [ 0.959268] 9pnet: Installing 9P2000 support1489worker2 # [ 0.926470] hv_vmbus: registering driver hyperv_keyboard1490worker1 # [ 0.961527] Key type dns_resolver registered1491client # [ 0.911338] thunder_xcv, ver 1.01492worker2 # [ 0.928051] rtc-pl031 9010000.pl031: registered as rtc01493client # [ 0.911386] thunder_bgx, ver 1.01494client # [ 0.911408] nicpf, ver 1.01495worker2 # [ 0.928081] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T21:08:24 UTC (1789938504)1496client # [ 0.911955] e1000: Intel(R) PRO/1000 Network Driver1497worker2 # [ 0.928391] i2c_dev: i2c /dev entries driver1498client # [ 0.911962] e1000: Copyright (c) 1999-2006 Intel Corporation.1499client # [ 0.911991] e1000e: Intel(R) PRO/1000 Network Driver1500client # [ 0.911999] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1501client # [ 0.912027] igb: Intel(R) Gigabit Ethernet Network Driver1502client # [ 0.912032] igb: Copyright (c) 2007-2014 Intel Corporation.1503client # [ 0.912055] igbvf: Intel(R) Gigabit Virtual Function Network Driver1504worker1 # [ 0.968460] registered taskstats version 11505worker2 # [ 0.933265] sdhci: Secure Digital Host Controller Interface driver1506client # [ 0.912061] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1507worker1 # [ 0.968640] Loading compiled-in X.509 certificates1508client # [ 0.912194] sky2: driver version 1.301509worker2 # [ 0.933275] sdhci: Copyright(c) Pierre Ossman1510client # [ 0.913785] usbcore: registered new interface driver usb-storage1511worker2 # [ 0.933541] Synopsys Designware Multimedia Card Interface Driver1512worker2 # [ 0.933921] sdhci-pltfm: SDHCI platform and OF driver helper1513client # [ 0.913880] usbcore: registered new interface driver usbserial_generic1514client # [ 0.913895] usbserial: USB Serial support registered for generic1515client # [ 0.914497] hv_vmbus: registering driver hyperv_keyboard1516worker2 # [ 0.938308] hid: raw HID events driver (C) Jiri Kosina1517client # [ 0.915314] ehci-pci 0000:00:0a.0: EHCI Host Controller1518worker2 # [ 0.938584] usbcore: registered new interface driver usbhid1519client # [ 0.915347] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11520worker2 # [ 0.938591] usbhid: USB HID core driver1521client # [ 0.915513] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001522worker1 # [ 0.977264] Demotion targets for Node 0: null1523worker1 # [ 0.977378] Key type .fscrypt registered1524client # [ 0.927764] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001525worker1 # [ 0.977384] Key type fscrypt-provisioning registered1526lb # [ 0.918412] Freeing initrd memory: 26140K1527client # [ 0.928112] hub 1-0:1.0: USB hub found1528worker1 # [ 0.977486] ima: No TPM chip found, activating TPM-bypass!1529client # [ 0.928132] hub 1-0:1.0: 6 ports detected1530worker2 # [ 0.941451] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1531worker1 # [ 0.977506] ima: Allocated hash algorithm: sha11532worker2 # [ 0.942951] drop_monitor: Initializing network drop monitor service1533worker1 # [ 0.977526] ima: No architecture policies found1534worker2 # [ 0.943085] NET: Registered PF_INET6 protocol family1535worker1 # [ 0.981658] input: gpio-keys as /devices/platform/gpio-keys/input/input01536client # [ 0.931058] rtc-pl031 9010000.pl031: registered as rtc01537worker2 # [ 0.946072] Segment Routing with IPv61538worker2 # [ 0.946092] In-situ OAM (IOAM) with IPv61539client # [ 0.931086] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T21:08:24 UTC (1789938504)1540worker2 # [ 0.946132] NET: Registered PF_PACKET protocol family1541client # [ 0.931390] i2c_dev: i2c /dev entries driver1542worker2 # [ 0.947969] 9pnet: Installing 9P2000 support1543worker2 # [ 0.950170] Key type dns_resolver registered1544client # [ 0.936269] sdhci: Secure Digital Host Controller Interface driver1545client # [ 0.936280] sdhci: Copyright(c) Pierre Ossman1546lb # [ 0.924476] tun: Universal TUN/TAP device driver, 1.61547client # [ 0.936544] Synopsys Designware Multimedia Card Interface Driver1548lb # [ 0.927653] thunder_xcv, ver 1.01549lb # [ 0.927698] thunder_bgx, ver 1.01550client # [ 0.936913] sdhci-pltfm: SDHCI platform and OF driver helper1551lb # [ 0.927720] nicpf, ver 1.01552client # [ 0.938577] hid: raw HID events driver (C) Jiri Kosina1553lb # [ 0.928273] e1000: Intel(R) PRO/1000 Network Driver1554lb # [ 0.928281] e1000: Copyright (c) 1999-2006 Intel Corporation.1555client # [ 0.942161] usbcore: registered new interface driver usbhid1556lb # [ 0.928306] e1000e: Intel(R) PRO/1000 Network Driver1557client # [ 0.942172] usbhid: USB HID core driver1558lb # [ 0.928315] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1559lb # [ 0.928347] igb: Intel(R) Gigabit Ethernet Network Driver1560lb # [ 0.928352] igb: Copyright (c) 2007-2014 Intel Corporation.1561lb # [ 0.928376] igbvf: Intel(R) Gigabit Virtual Function Network Driver1562worker2 # [ 0.957052] registered taskstats version 11563worker2 # [ 0.957206] Loading compiled-in X.509 certificates1564lb # [ 0.928381] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1565client # [ 0.944502] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1566lb # [ 0.928511] sky2: driver version 1.301567lb # [ 0.930439] ehci-pci 0000:00:0a.0: EHCI Host Controller1568client # [ 0.946030] drop_monitor: Initializing network drop monitor service1569client # [ 0.946179] NET: Registered PF_INET6 protocol family1570lb # [ 0.930488] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11571client # [ 0.949285] Segment Routing with IPv61572lb # [ 0.930775] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001573client # [ 0.949305] In-situ OAM (IOAM) with IPv61574client # [ 0.949336] NET: Registered PF_PACKET protocol family1575lb # [ 0.941552] usbcore: registered new interface driver usb-storage1576lb # [ 0.941674] usbcore: registered new interface driver usbserial_generic1577worker1 # [ 0.999708] clk: Disabling unused clocks1578worker2 # [ 0.965582] Demotion targets for Node 0: null1579lb # [ 0.941688] usbserial: USB Serial support registered for generic1580worker1 # [ 0.999735] PM: genpd: Disabling unused power domains1581worker2 # [ 0.965693] Key type .fscrypt registered1582lb # [ 0.944014] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001583worker2 # [ 0.965700] Key type fscrypt-provisioning registered1584lb # [ 0.945085] hub 1-0:1.0: USB hub found1585worker2 # [ 0.965793] ima: No TPM chip found, activating TPM-bypass!1586lb # [ 0.945603] hub 1-0:1.0: 6 ports detected1587worker1 # [ 1.003980] Freeing unused kernel memory: 4736K1588client # [ 0.951190] 9pnet: Installing 9P2000 support1589worker2 # [ 0.965813] ima: Allocated hash algorithm: sha11590worker1 # [ 1.004187] Run /init as init process1591client # [ 0.953457] Key type dns_resolver registered1592worker2 # [ 0.965834] ima: No architecture policies found1593worker2 # [ 0.969954] input: gpio-keys as /devices/platform/gpio-keys/input/input01594lb # [ 0.947114] hv_vmbus: registering driver hyperv_keyboard1595lb # [ 0.948647] rtc-pl031 9010000.pl031: registered as rtc01596lb # [ 0.948675] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T21:08:24 UTC (1789938504)1597lb # [ 0.948983] i2c_dev: i2c /dev entries driver1598client # [ 0.960337] registered taskstats version 11599client # [ 0.960486] Loading compiled-in X.509 certificates1600lb # [ 0.954031] sdhci: Secure Digital Host Controller Interface driver1601lb # [ 0.954041] sdhci: Copyright(c) Pierre Ossman1602lb # [ 0.954305] Synopsys Designware Multimedia Card Interface Driver1603lb # [ 0.954676] sdhci-pltfm: SDHCI platform and OF driver helper1604lb # [ 0.956362] hid: raw HID events driver (C) Jiri Kosina1605client # [ 0.968958] Demotion targets for Node 0: null1606client # [ 0.969071] Key type .fscrypt registered1607lb # [ 0.956596] usbcore: registered new interface driver usbhid1608client # [ 0.969078] Key type fscrypt-provisioning registered1609lb # [ 0.956603] usbhid: USB HID core driver1610client # [ 0.969185] ima: No TPM chip found, activating TPM-bypass!1611worker1 # [ 1.021400] systemd[1]: Successfully made /usr/ read-only.1612client # [ 0.969205] ima: Allocated hash algorithm: sha11613client # [ 0.969228] ima: No architecture policies found1614lb # [ 0.962450] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1615client # [ 0.973408] input: gpio-keys as /devices/platform/gpio-keys/input/input01616lb # [ 0.963976] drop_monitor: Initializing network drop monitor service1617lb # [ 0.964191] NET: Registered PF_INET6 protocol family1618worker2 # [ 0.987734] clk: Disabling unused clocks1619lb # [ 0.967324] Segment Routing with IPv61620lb # [ 0.967344] In-situ OAM (IOAM) with IPv61621worker2 # [ 0.987765] PM: genpd: Disabling unused power domains1622lb # [ 0.967374] NET: Registered PF_PACKET protocol family1623lb # [ 0.969046] 9pnet: Installing 9P2000 support1624worker2 # [ 0.992070] Freeing unused kernel memory: 4736K1625worker2 # [ 0.992275] Run /init as init process1626lb # [ 0.972103] Key type dns_resolver registered1627lb # [ 0.978620] registered taskstats version 11628lb # [ 0.978767] Loading compiled-in X.509 certificates1629client # [ 0.992056] clk: Disabling unused clocks1630client # [ 0.992085] PM: genpd: Disabling unused power domains1631worker2 # [ 1.009465] systemd[1]: Successfully made /usr/ read-only.1632client # [ 0.996330] Freeing unused kernel memory: 4736K1633lb # [ 0.987510] Demotion targets for Node 0: null1634client # [ 0.996536] Run /init as init process1635lb # [ 0.987633] Key type .fscrypt registered1636lb # [ 0.987640] Key type fscrypt-provisioning registered1637lb # [ 0.987737] ima: No TPM chip found, activating TPM-bypass!1638lb # [ 0.987756] ima: Allocated hash algorithm: sha11639lb # [ 0.987778] ima: No architecture policies found1640lb # [ 0.991970] input: gpio-keys as /devices/platform/gpio-keys/input/input01641client # [ 1.012901] systemd[1]: Successfully made /usr/ read-only.1642lb # [ 1.011104] clk: Disabling unused clocks1643lb # [ 1.011133] PM: genpd: Disabling unused power domains1644lb # [ 1.015426] Freeing unused kernel memory: 4736K1645lb # [ 1.015647] Run /init as init process1646lb # [ 1.032137] systemd[1]: Successfully made /usr/ read-only.1647worker1 # [ 1.183133] usb 1-1: new high-speed USB device number 2 using ehci-pci1648worker2 # [ 1.171211] usb 1-1: new high-speed USB device number 2 using ehci-pci1649client # [ 1.174677] usb 1-1: new high-speed USB device number 2 using ehci-pci1650lb # [ 1.193758] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker1 # [ 1.335438] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652worker2 # [ 1.323358] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653worker1 # [ 1.356231] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1654worker1 # [ 1.368217] systemd[1]: Detected virtualization qemu.1655worker1 # [ 1.370183] systemd[1]: Detected architecture arm64.1656worker1 # [ 1.372125] systemd[1]: Running in initrd.1657worker1 # [ 1.374670] systemd[1]: Initializing machine ID from random generator.1658worker1 # [ 1.377562] systemd[1]: Hostname set to <worker1>.1659client # [ 1.326702] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660worker2 # [ 1.344341] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1661worker2 # [ 1.356178] systemd[1]: Detected virtualization qemu.1662worker2 # [ 1.358150] systemd[1]: Detected architecture arm64.1663worker2 # [ 1.360113] systemd[1]: Running in initrd.1664worker2 # [ 1.362615] systemd[1]: Initializing machine ID from random generator.1665worker2 # [ 1.365486] systemd[1]: Hostname set to <worker2>.1666lb # [ 1.343645] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11667client # [ 1.347728] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668client # [ 1.359677] systemd[1]: Detected virtualization qemu.1669client # [ 1.361667] systemd[1]: Detected architecture arm64.1670client # [ 1.363635] systemd[1]: Running in initrd.1671client # [ 1.366235] systemd[1]: Initializing machine ID from random generator.1672client # [ 1.369330] systemd[1]: Hostname set to <client>.1673worker1 # [ 1.423382] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01674lb # [ 1.367116] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1675lb # [ 1.379442] systemd[1]: Detected virtualization qemu.1676lb # [ 1.381554] systemd[1]: Detected architecture arm64.1677lb # [ 1.383598] systemd[1]: Running in initrd.1678lb # [ 1.386271] systemd[1]: Initializing machine ID from random generator.1679lb # [ 1.389253] systemd[1]: Hostname set to <lb>.1680worker2 # [ 1.411453] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681client # [ 1.418897] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682lb # [ 1.434028] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker1 # [ 1.547142] usb 1-2: new high-speed USB device number 3 using ehci-pci1684worker2 # [ 1.535231] usb 1-2: new high-speed USB device number 3 using ehci-pci1685client # [ 1.542678] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.557798] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker1 # [ 1.690968] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688worker2 # [ 1.670212] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689worker1 # [ 1.707479] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21690worker1 # [ 1.713234] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01691worker2 # [ 1.693717] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21692client # [ 1.682782] systemd[1]: bpf-restrict-fs: LSM BPF program attached1693worker2 # [ 1.699402] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694client # [ 1.700918] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21695client # [ 1.711047] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01696lb # [ 1.706927] systemd[1]: bpf-restrict-fs: LSM BPF program attached1697lb # [ 1.724242] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21698lb # [ 1.729209] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01699worker1 # [ 1.803803] systemd[1]: Queued start job for default target Initrd Default Target.1700worker1 # [ 1.813385] systemd[1]: Created slice Slice /system/modprobe.1701worker2 # [ 1.778272] systemd[1]: Queued start job for default target Initrd Default Target.1702worker1 # [ 1.814562] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1703worker1 # [ 1.815930] systemd[1]: Expecting device /dev/disk/by-label/nixos...1704worker1 # [ 1.816969] systemd[1]: Reached target Path Units.1705worker1 # [ 1.817751] systemd[1]: Reached target Slice Units.1706worker1 # [ 1.818555] systemd[1]: Reached target Swaps.1707worker1 # [ 1.819312] systemd[1]: Reached target Timer Units.1708worker1 # [ 1.820301] systemd[1]: Listening on D-Bus System Message Bus Socket.1709worker1 # [ 1.821480] systemd[1]: Listening on Journal Socket (/dev/log).1710worker1 # [ 1.822576] systemd[1]: Listening on Journal Sockets.1711worker1 # [ 1.823567] systemd[1]: Listening on udev Control Socket.1712worker1 # [ 1.824571] systemd[1]: Listening on udev Kernel Socket.1713worker1 # [ 1.825433] systemd[1]: Reached target Socket Units.1714worker2 # [ 1.789537] systemd[1]: Created slice Slice /system/modprobe.1715worker2 # [ 1.790718] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1716worker1 # [ 1.827989] systemd[1]: Starting Create List of Static Device Nodes...1717worker2 # [ 1.792133] systemd[1]: Expecting device /dev/disk/by-label/nixos...1718worker2 # [ 1.792173] systemd[1]: Reached target Path Units.1719worker2 # [ 1.792194] systemd[1]: Reached target Slice Units.1720worker2 # [ 1.792213] systemd[1]: Reached target Swaps.1721worker2 # [ 1.792232] systemd[1]: Reached target Timer Units.1722worker2 # [ 1.792455] systemd[1]: Listening on D-Bus System Message Bus Socket.1723worker2 # [ 1.792607] systemd[1]: Listening on Journal Socket (/dev/log).1724worker2 # [ 1.792745] systemd[1]: Listening on Journal Sockets.1725worker2 # [ 1.792881] systemd[1]: Listening on udev Control Socket.1726worker2 # [ 1.792992] systemd[1]: Listening on udev Kernel Socket.1727worker2 # [ 1.793013] systemd[1]: Reached target Socket Units.1728worker1 # [ 1.838544] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1729worker2 # [ 1.803663] systemd[1]: Starting Create List of Static Device Nodes...1730worker1 # [ 1.839895] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1731client # [ 1.793520] systemd[1]: Queued start job for default target Initrd Default Target.1732worker2 # [ 1.811362] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1733worker2 # [ 1.812547] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1734worker1 # [ 1.852748] systemd[1]: Mounting Kernel Configuration File System...1735worker2 # [ 1.819472] systemd[1]: Mounting Kernel Configuration File System...1736client # [ 1.806717] systemd[1]: Created slice Slice /system/modprobe.1737client # [ 1.806942] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1738client # [ 1.806977] systemd[1]: Expecting device /dev/disk/by-label/nixos...1739client # [ 1.807009] systemd[1]: Reached target Path Units.1740client # [ 1.807029] systemd[1]: Reached target Slice Units.1741client # [ 1.807044] systemd[1]: Reached target Swaps.1742client # [ 1.807063] systemd[1]: Reached target Timer Units.1743client # [ 1.807243] systemd[1]: Listening on D-Bus System Message Bus Socket.1744client # [ 1.807405] systemd[1]: Listening on Journal Socket (/dev/log).1745client # [ 1.807539] systemd[1]: Listening on Journal Sockets.1746client # [ 1.807671] systemd[1]: Listening on udev Control Socket.1747client # [ 1.807781] systemd[1]: Listening on udev Kernel Socket.1748client # [ 1.807802] systemd[1]: Reached target Socket Units.1749client # [ 1.820451] systemd[1]: Starting Create List of Static Device Nodes...1750worker2 # [ 1.838724] systemd[1]: Starting Journal Service...1751worker1 # [ 1.875330] systemd[1]: Starting Journal Service...1752lb # [ 1.817626] systemd[1]: Queued start job for default target Initrd Default Target.1753client # [ 1.827402] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1754worker1 # [ 1.879200] systemd[1]: Starting Load Kernel Modules...1755client # [ 1.828498] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1756worker1 # [ 1.880096] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1757worker2 # [ 1.848678] systemd[1]: Starting Load Kernel Modules...1758worker2 # [ 1.848809] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1759lb # [ 1.828584] systemd[1]: Created slice Slice /system/modprobe.1760client # [ 1.838783] systemd[1]: Mounting Kernel Configuration File System...1761lb # [ 1.830140] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1762lb # [ 1.831737] systemd[1]: Expecting device /dev/disk/by-label/nixos...1763lb # [ 1.833016] systemd[1]: Reached target Path Units.1764lb # [ 1.834070] systemd[1]: Reached target Slice Units.1765lb # [ 1.835072] systemd[1]: Reached target Swaps.1766lb # [ 1.836014] systemd[1]: Reached target Timer Units.1767lb # [ 1.837246] systemd[1]: Listening on D-Bus System Message Bus Socket.1768client # [ 1.848275] systemd[1]: Starting Journal Service...1769lb # [ 1.838854] systemd[1]: Listening on Journal Socket (/dev/log).1770lb # [ 1.839033] systemd[1]: Listening on Journal Sockets.1771lb # [ 1.839198] systemd[1]: Listening on udev Control Socket.1772lb # [ 1.839344] systemd[1]: Listening on udev Kernel Socket.1773worker1 # [ 1.903677] systemd[1]: Starting Coldplug All udev Devices...1774lb # [ 1.839371] systemd[1]: Reached target Socket Units.1775lb # [ 1.846578] systemd[1]: Starting Create List of Static Device Nodes...1776lb # [ 1.857965] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1777worker2 # [ 1.879436] systemd[1]: Starting Coldplug All udev Devices...1778lb # [ 1.859260] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1779worker1 # [ 1.919554] systemd[1]: Finished Create List of Static Device Nodes.1780worker1 # [ 1.921519] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1781lb # [ 1.863118] systemd[1]: Mounting Kernel Configuration File System...1782worker1 # [ 1.926542] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1783worker2 # [ 1.892108] systemd-journald[73]: Collecting audit messages is disabled.1784worker2 # [ 1.893089] systemd[1]: Finished Create List of Static Device Nodes.1785client # [ 1.880265] systemd[1]: Starting Load Kernel Modules...1786worker2 # [ 1.894043] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1787client # [ 1.880406] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1788worker1 # [ 1.935567] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1789worker1 # [ 1.935934] systemd[1]: Mounted Kernel Configuration File System.1790worker2 # [ 1.907366] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1791worker1 # [ 1.943485] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1792worker1 # [ 1.948370] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1793client # [ 1.901041] systemd[1]: Starting Coldplug All udev Devices...1794client # [ 1.904346] systemd-journald[73]: Collecting audit messages is disabled.1795lb # [ 1.898045] systemd[1]: Starting Journal Service...1796worker1 # [ 1.957289] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01797worker1 # [ 1.957535] [drm] features: -virgl +edid -resource_blob -host_visible1798worker1 # [ 1.957543] [drm] features: -context_init1799worker1 # [ 1.958265] [drm] number of scanouts: 11800worker1 # [ 1.958282] [drm] number of cap sets: 01801worker1 # [ 1.968695] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1802worker1 # [ 1.968719] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01803client # [ 1.919306] systemd[1]: Finished Create List of Static Device Nodes.1804lb # [ 1.913192] systemd[1]: Starting Load Kernel Modules...1805worker2 # [ 1.935384] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1806client # [ 1.920334] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1807lb # [ 1.914219] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1808client # [ 1.920643] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1809client # [ 1.920879] systemd[1]: Mounted Kernel Configuration File System.1810worker2 # [ 1.939472] systemd[1]: Mounted Kernel Configuration File System.1811worker2 # [ 1.945684] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1812lb # [ 1.923029] systemd[1]: Starting Coldplug All udev Devices...1813worker1 # [ 1.984953] systemd-journald[73]: Collecting audit messages is disabled.1814client # [ 1.935018] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1815worker2 # [ 1.967238] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1816worker1 # [ 2.010370] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1817worker2 # [ 1.974987] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01818lb # [ 1.949932] systemd[1]: Finished Create List of Static Device Nodes.1819lb # [ 1.950988] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1820lb # [ 1.951308] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1821lb # [ 1.951561] systemd[1]: Mounted Kernel Configuration File System.1822lb # [ 1.956394] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1823client # [ 1.966256] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1824worker1 # [ 2.016189] systemd[1]: Starting Create Static Device Nodes in /dev...1825worker2 # [ 1.983891] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1826worker2 # [ 1.987304] [drm] features: -virgl +edid -resource_blob -host_visible1827worker2 # [ 1.987323] [drm] features: -context_init1828worker2 # [ 1.988097] [drm] number of scanouts: 11829worker2 # [ 1.988116] [drm] number of cap sets: 01830client # [ 1.978715] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1831lb # [ 1.971786] systemd-journald[73]: Collecting audit messages is disabled.1832worker1 # [ 2.031230] Console: switching to colour frame buffer device 160x501833worker1 # [ 2.031913] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1834worker2 # [ 1.995559] systemd[1]: Starting Create Static Device Nodes in /dev...1835lb # [ 1.977803] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1836worker2 # [ 2.000508] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1837worker2 # [ 2.000532] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01838client # [ 1.990673] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01839client # [ 1.990909] [drm] features: -virgl +edid -resource_blob -host_visible1840client # [ 1.990919] [drm] features: -context_init1841client # [ 1.991717] [drm] number of scanouts: 11842client # [ 1.991738] [drm] number of cap sets: 01843worker1 # [ 2.047974] systemd[1]: Finished Load Kernel Modules.1844worker1 # [ 2.050297] systemd[1]: Starting Apply Kernel Variables...1845lb # [ 1.993810] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1846client # [ 2.007345] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1847client # [ 2.010022] systemd[1]: Starting Create Static Device Nodes in /dev...1848lb # [ 2.005953] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01849lb # [ 2.006208] [drm] features: -virgl +edid -resource_blob -host_visible1850lb # [ 2.006219] [drm] features: -context_init1851lb # [ 2.006968] [drm] number of scanouts: 11852lb # [ 2.006986] [drm] number of cap sets: 01853client # [ 2.015131] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1854client # [ 2.015158] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01855worker2 # [ 2.027547] Console: switching to colour frame buffer device 160x501856worker1 # [ 2.071214] systemd[1]: Finished Create Static Device Nodes in /dev.1857worker1 # [ 2.073542] systemd[1]: Reached target Preparation for Local File Systems.1858lb # [ 2.017860] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1859lb # [ 2.020539] systemd[1]: Starting Create Static Device Nodes in /dev...1860worker1 # [ 2.079330] systemd[1]: Reached target Local File Systems.1861worker2 # [ 2.051871] systemd[1]: Finished Create Static Device Nodes in /dev.1862worker2 # [ 2.052902] systemd[1]: Reached target Preparation for Local File Systems.1863worker2 # [ 2.053774] systemd[1]: Reached target Local File Systems.1864worker1 # [ 2.091460] systemd[1]: Starting Rule-based Manager for Device Events and Files...1865worker2 # [ 2.059771] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1866lb # [ 2.034008] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1867client # [ 2.042863] Console: switching to colour frame buffer device 160x501868lb # [ 2.034030] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01869worker2 # [ 2.064042] systemd[1]: Starting Rule-based Manager for Device Events and Files...1870client # [ 2.049479] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1871worker1 # [ 2.111359] systemd[1]: Finished Apply Kernel Variables.1872worker2 # [ 2.084700] systemd[1]: Finished Load Kernel Modules.1873worker2 # [ 2.074467] systemd-modules-load[74]: Inserted module 'dm_mod'1874lb # [ 2.062065] Console: switching to colour frame buffer device 160x501875client # [ 2.078877] systemd[1]: Finished Create Static Device Nodes in /dev.1876client # [ 2.079307] systemd[1]: Reached target Preparation for Local File Systems.1877client # [ 2.079348] systemd[1]: Reached target Local File Systems.1878client # [ 2.084670] systemd[1]: Starting Rule-based Manager for Device Events and Files...1879worker2 # [ 2.099612] systemd[1]: Starting Apply Kernel Variables...1880worker2 # [ 2.099781] systemd[1]: Started Journal Service.1881worker2 # [ 2.088566] systemd-modules-load[74]: Module 'virtio_balloon' is built in1882worker2 # [ 2.089755] systemd-modules-load[74]: Module 'virtio_console' is built in1883worker2 # [ 2.090811] systemd-modules-load[74]: Inserted module 'virtio_gpu'1884client # [ 2.077936] systemd-modules-load[75]: Inserted module 'dm_mod'1885lb # [ 2.082455] systemd[1]: Finished Create Static Device Nodes in /dev.1886worker2 # [ 2.091778] systemd-modules-load[74]: Module 'virtio_rng' is built in1887lb # [ 2.082846] systemd[1]: Reached target Preparation for Local File Systems.1888lb # [ 2.082873] systemd[1]: Reached target Local File Systems.1889client # [ 2.084346] systemd-modules-load[75]: Module 'virtio_balloon' is built in1890lb # [ 2.090165] systemd[1]: Starting Rule-based Manager for Device Events and Files...1891client # [ 2.085481] systemd-modules-load[75]: Module 'virtio_console' is built in1892client # [ 2.086575] systemd-modules-load[75]: Inserted module 'virtio_gpu'1893client # [ 2.087567] systemd-modules-load[75]: Module 'virtio_rng' is built in1894client # [ 2.103200] systemd[1]: Finished Load Kernel Modules.1895lb # [ 2.094554] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1896client # [ 2.105390] systemd[1]: Started Journal Service.1897worker1 # [ 2.159282] systemd[1]: Started Journal Service.1898worker1 # [ 2.147638] systemd-modules-load[74]: Inserted module 'dm_mod'1899worker1 # [ 2.148982] systemd-modules-load[74]: Module 'virtio_balloon' is built in1900worker2 # [ 2.113181] systemd[1]: Starting Create System Files and Directories...1901worker1 # [ 2.150105] systemd-modules-load[74]: Module 'virtio_console' is built in1902worker1 # [ 2.151167] systemd-modules-load[74]: Inserted module 'virtio_gpu'1903client # [ 2.108560] systemd[1]: Starting Apply Kernel Variables...1904worker1 # [ 2.164260] systemd-modules-load[74]: Module 'virtio_rng' is built in1905worker1 # [ 2.169875] systemd[1]: Starting Create System Files and Directories...1906client # [ 2.122577] systemd[1]: Starting Create System Files and Directories...1907lb # [ 2.130079] systemd[1]: Finished Load Kernel Modules.1908worker1 # [ 2.178070] systemd-udevd[81]: Using default interface naming scheme 'v261'.1909lb # [ 2.133633] systemd[1]: Starting Apply Kernel Variables...1910worker2 # [ 2.159157] systemd[1]: Finished Create System Files and Directories.1911worker2 # [ 2.161694] systemd-udevd[80]: Using default interface naming scheme 'v261'.1912worker1 # [ 2.200763] systemd[1]: Finished Create System Files and Directories.1913worker2 # [ 2.164736] systemd[1]: Finished Apply Kernel Variables.1914lb # [ 2.162850] systemd[1]: Started Journal Service.1915lb # [ 2.160318] systemd-modules-load[74]: Inserted module 'dm_mod'1916client # [ 2.169128] systemd[1]: Finished Create System Files and Directories.1917lb # [ 2.161628] systemd-modules-load[74]: Module 'virtio_balloon' is built in1918lb # [ 2.162724] systemd-modules-load[74]: Module 'virtio_console' is built in1919worker1 # [ 2.220808] systemd[1]: Started Rule-based Manager for Device Events and Files.1920lb # [ 2.163824] systemd-modules-load[74]: Inserted module 'virtio_gpu'1921client # [ 2.172462] systemd[1]: Finished Apply Kernel Variables.1922worker2 # [ 2.194701] systemd[1]: Started Rule-based Manager for Device Events and Files.1923lb # [ 2.176487] systemd-modules-load[74]: Module 'virtio_rng' is built in1924lb # [ 2.177594] systemd[1]: Starting Create System Files and Directories...1925client # [ 2.186678] systemd-udevd[79]: Using default interface naming scheme 'v261'.1926lb # [ 2.189705] systemd[1]: Finished Apply Kernel Variables.1927lb # [ 2.208670] systemd-udevd[79]: Using default interface naming scheme 'v261'.1928client # [ 2.217855] systemd[1]: Started Rule-based Manager for Device Events and Files.1929lb # [ 2.211883] systemd[1]: Finished Create System Files and Directories.1930worker1 # [ 2.288146] systemd[1]: Starting Virtual Console Setup...1931worker2 # [ 2.251891] systemd[1]: Starting Virtual Console Setup...1932lb # [ 2.241019] systemd[1]: Started Rule-based Manager for Device Events and Files.1933client # [ 2.273662] systemd[1]: Starting Virtual Console Setup...1934worker1 # [ 2.332621] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1935worker1 # [ 2.336170] systemd[1]: Finished Virtual Console Setup.1936worker2 # [ 2.308537] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1937worker2 # [ 2.312161] systemd[1]: Finished Virtual Console Setup.1938lb # [ 2.308194] systemd[1]: Starting Virtual Console Setup...1939client # [ 2.328619] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1940client # [ 2.332080] systemd[1]: Finished Virtual Console Setup.1941lb # [ 2.352462] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942lb # [ 2.356178] systemd[1]: Finished Virtual Console Setup.1943worker1 # [ 2.921186] systemd[1]: Finished Coldplug All udev Devices.1944worker1 # [ 2.922187] systemd[1]: Reached target System Initialization.1945worker1 # [ 2.923187] systemd[1]: Reached target Basic System.1946worker2 # [ 2.916259] systemd[1]: Finished Coldplug All udev Devices.1947worker2 # [ 2.917260] systemd[1]: Reached target System Initialization.1948worker2 # [ 2.920236] systemd[1]: Reached target Basic System.1949client # [ 2.915779] systemd[1]: Finished Coldplug All udev Devices.1950client # [ 2.916799] systemd[1]: Reached target System Initialization.1951client # [ 2.920107] systemd[1]: Reached target Basic System.1952lb # [ 2.956763] systemd[1]: Finished Coldplug All udev Devices.1953lb # [ 2.958590] systemd[1]: Reached target System Initialization.1954lb # [ 2.965163] systemd[1]: Reached target Basic System.1955worker1 # [ 3.063998] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1956worker2 # [ 3.052444] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1957worker1 # [ 3.097056] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1958worker1 # [ 3.101193] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1959client # [ 3.056242] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960worker2 # [ 3.086599] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1961worker2 # [ 3.090174] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1962client # [ 3.089850] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1963client # [ 3.096860] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1964lb # [ 3.108440] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1965worker1 # [ 3.170264] systemd[1]: Found device /dev/disk/by-label/nixos.1966worker1 # [ 3.182319] systemd[1]: Reached target Initrd Root Device.1967worker1 # [ 3.188114] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1968worker2 # [ 3.161664] systemd[1]: Found device /dev/disk/by-label/nixos.1969worker2 # [ 3.168110] systemd[1]: Reached target Initrd Root Device.1970lb # [ 3.146976] (udev-worker)[106]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1971lb # [ 3.149540] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1972worker2 # [ 3.172193] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1973client # [ 3.180500] systemd[1]: Found device /dev/disk/by-label/nixos.1974client # [ 3.184539] systemd[1]: Reached target Initrd Root Device.1975worker1 # [ 3.235974] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1976client # [ 3.187768] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1977worker1 # [ 3.243653] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1978worker1 # [ 3.246014] systemd[1]: Mounting /sysroot...1979worker2 # [ 3.225512] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1980worker2 # [ 3.233407] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1981worker2 # [ 3.235991] systemd[1]: Mounting /sysroot...1982worker1 # [ 3.295520] EXT4-fs (vda): mounted filesystem 60c27afa-cc42-4a7a-9fbb-12f3061e83b4 r/w with ordered data mode. Quota mode: none.1983client # [ 3.235922] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1984worker1 # [ 3.286487] systemd[1]: Mounted /sysroot.1985worker1 # [ 3.288654] systemd[1]: Reached target Initrd Root File System.1986worker1 # [ 3.292750] systemd[1]: Starting Mountpoints Configured in the Real Root...1987client # [ 3.244103] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1988client # [ 3.247078] systemd[1]: Mounting /sysroot...1989lb # [ 3.246155] systemd[1]: Found device /dev/disk/by-label/nixos.1990lb # [ 3.248307] systemd[1]: Reached target Initrd Root Device.1991lb # [ 3.252207] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992worker2 # [ 3.292697] EXT4-fs (vda): mounted filesystem 94753198-4bef-46d3-8096-7b888aaeec4f r/w with ordered data mode. Quota mode: none.1993worker2 # [ 3.286631] systemd[1]: Mounted /sysroot.1994worker1 # [ 3.322780] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1995worker2 # [ 3.290026] systemd[1]: Reached target Initrd Root File System.1996worker1 # [ 3.327391] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1997worker1 # [ 3.331965] systemd[1]: Reloading...1998client # [ 3.298643] EXT4-fs (vda): mounted filesystem 5e27ae01-54fe-4082-93cd-eed420088bcd r/w with ordered data mode. Quota mode: none.1999worker2 # [ 3.300325] systemd[1]: Starting Mountpoints Configured in the Real Root...2000client # [ 3.293969] systemd[1]: Mounted /sysroot.
loading 1,481 hidden lines…
3482lb # [ 21.306543] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3483lb # [ 21.309537] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3484lb # [ 21.311479] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls3485lb # [ 21.317968] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3486lb # [ 21.320905] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default3487lb # [ 21.322602] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action3488lb # [ 21.324842] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3489lb # [ 21.327327] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3490lb # [ 21.329459] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder3491lb # [ 21.331764] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd3492lb # [ 21.339081] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router3493lb # [ 21.341078] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3494lb # [ 21.343225] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23495lb # [ 21.345288] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response3496lb # [ 21.347583] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable3497lb # [ 21.349875] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3498lb # [ 21.353855] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default3499lb # [ 21.355730] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo3500lb # [ 21.357534] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3501lb # [ 21.362891] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23502lb # [ 21.365768] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re23503lb # [ 21.369750] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3504lb # [ 21.371344] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple3505lb # [ 21.376955] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config3506lb # [ 21.380750] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3507lb # [ 21.382648] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3508lb # [ 21.386092] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3509lb # [ 21.391243] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3510lb # [ 21.394040] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle3511lb # [ 21.396339] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection3512lb # [ 21.398156] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy3513lb # [ 21.401277] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3514lb # [ 21.403982] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3515lb # [ 21.406278] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3516lb # [ 21.408314] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default3517lb # [ 21.410653] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3518lb # [ 21.412955] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port3519lb # [ 21.421425] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3520lb # [ 21.423868] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3521lb # [ 21.427352] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3522lb # [ 21.429324] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3523lb # [ 21.431268] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3524lb # [ 21.434959] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3525lb # [ 21.436861] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log3526lb # [ 21.445477] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert3527lb # [ 21.447407] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3528lb # [ 21.450466] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr3529lb # [ 21.452748] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3530lb # [ 21.454922] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy3531lb # [ 21.468790] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3532lb # [ 21.471416] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3533lb # [ 21.473593] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3534lb # [ 21.475490] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3535lb # [ 21.478012] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3536lb # [ 21.479944] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter3537lb # [ 21.482190] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3538lb # [ 21.486037] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic3539lb # [ 21.487891] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san3540lb # [ 21.495433] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3541lb # [ 21.507169] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3542lb # [ 21.510892] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3543lb # [ 21.513776] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router3544lb # [ 21.544727] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3545lb # [ 21.551854] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83546lb # [ 21.554403] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3547lb # [ 21.557018] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter3548lb # [ 21.559098] envoy[926]: [2026-09-20 21:08:44.756][926][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3549lb # [ 21.563287] envoy[926]: [2026-09-20 21:08:44.757][926][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift3550lb # [ 21.566931] envoy[926]: [2026-09-20 21:08:44.757][926][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3551lb # [ 21.571941] envoy[926]: [2026-09-20 21:08:44.757][926][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator3552lb # [ 21.574681] envoy[926]: [2026-09-20 21:08:44.757][926][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3553lb # [ 21.579660] envoy[926]: [2026-09-20 21:08:44.802][926][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection3554lb # [ 21.585629] envoy[926]: [2026-09-20 21:08:44.812][926][info][main] [source/server/server.cc:531] HTTP header map info:3555lb # [ 21.587103] envoy[926]: [2026-09-20 21:08:44.814][926][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3556lb # [ 21.602342] envoy[926]: [2026-09-20 21:08:44.814][926][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3557lb # [ 21.604501] envoy[926]: [2026-09-20 21:08:44.814][926][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3558lb # [ 21.613587] envoy[926]: [2026-09-20 21:08:44.814][926][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3559lb # [ 21.615596] envoy[926]: [2026-09-20 21:08:45.042][926][info][main] [source/server/server.cc:968] runtime: {}3560lb # [ 21.617253] envoy[926]: [2026-09-20 21:08:45.075][926][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3561lb # [ 21.619055] envoy[926]: [2026-09-20 21:08:45.075][926][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013562lb # [ 21.620881] envoy[926]: [2026-09-20 21:08:45.075][926][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013563lb # [ 21.622545] envoy[926]: [2026-09-20 21:08:45.109][926][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3564lb # [ 21.624488] envoy[926]: [2026-09-20 21:08:45.109][926][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3565lb # [ 21.626088] envoy[926]: [2026-09-20 21:08:45.109][926][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3566lb # [ 21.653038] envoy[926]: [2026-09-20 21:08:45.217][926][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3567worker2 # [ 21.678766] systemd[1]: Reached target Network is Online.3568lb # [ 21.662850] envoy[926]: [2026-09-20 21:08:45.227][926][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3569lb # [ 21.665697] envoy[926]: [2026-09-20 21:08:45.230][926][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3570worker1 # [ 21.724223] systemd[1]: Starting Nix worker-protocol over gRPC...3571lb # [ 21.669140] envoy[926]: [2026-09-20 21:08:45.230][926][info][main] [source/server/server.cc:1077] starting main dispatch loop3572worker2 # [ 21.689909] systemd[1]: Starting scheduler-token.service...3573worker2 # [ 21.814543] systemd[1]: scheduler-token.service: Deactivated successfully.3574worker2 # [ 21.818722] systemd[1]: Finished scheduler-token.service.3575worker2 # [ 21.825312] systemd[1]: scheduler-token.service: Consumed 53ms CPU time over 128ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3576worker2 # [ 21.832631] systemd[1]: Starting Nix worker-protocol over gRPC...3577worker1 # [ 21.964923] nix-grpc-daemon[774]: ts=2026-09-20T21:08:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3578worker1 # [ 21.977196] nix-grpc-daemon[774]: ts=2026-09-20T21:08:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3579worker1 # [ 21.979466] nix-grpc-daemon[774]: ts=2026-09-20T21:08:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13580worker1 # [ 22.028357] nix-grpc-daemon[774]: ts=2026-09-20T21:08:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3581worker1 # [ 22.032792] systemd[1]: Started Nix worker-protocol over gRPC.3582lb # [ 22.011093] envoy[926]: [2026-09-20 21:08:45.575][926][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3583lb # [ 22.017968] envoy[926]: [2026-09-20 21:08:45.575][926][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3584lb # [ 22.023520] envoy[926]: [2026-09-20 21:08:45.575][926][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3585worker2 # [ 22.053029] nix-grpc-daemon[775]: ts=2026-09-20T21:08:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3586lb # [ 22.034178] envoy[926]: [2026-09-20 21:08:45.575][926][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3587worker2 # [ 22.064974] nix-grpc-daemon[775]: ts=2026-09-20T21:08:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3588worker1 # [ 22.101533] nix-grpc-daemon[774]: ts=2026-09-20T21:08:45Z level=info event=scheduler_connected addr=lb:500513589worker2 # [ 22.067354] nix-grpc-daemon[775]: ts=2026-09-20T21:08:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13590worker1 # [ 22.105893] nix-grpc-daemon[774]: ts=2026-09-20T21:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3591worker2 # [ 22.116087] nix-grpc-daemon[775]: ts=2026-09-20T21:08:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3592worker2 # [ 22.121762] systemd[1]: Started Nix worker-protocol over gRPC.3593worker2 # [ 22.180404] nix-grpc-daemon[775]: ts=2026-09-20T21:08:45Z level=info event=scheduler_connected addr=lb:500513594worker2 # [ 22.182280] nix-grpc-daemon[775]: ts=2026-09-20T21:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3595worker1 # [ 22.307288] nix-grpc-daemon[774]: ts=2026-09-20T21:08:45Z level=info event=scheduler_connected addr=lb:500513596worker1 # [ 22.312754] nix-grpc-daemon[774]: ts=2026-09-20T21:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3597worker2 # [ 22.383853] nix-grpc-daemon[775]: ts=2026-09-20T21:08:45Z level=info event=scheduler_connected addr=lb:500513598worker2 # [ 22.389048] nix-grpc-daemon[775]: ts=2026-09-20T21:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3599worker1 # [ 22.513602] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513600worker1 # [ 22.520747] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3601worker2 # [ 22.589800] nix-grpc-daemon[775]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513602worker2 # [ 22.593181] nix-grpc-daemon[775]: ts=2026-09-20T21:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3603worker1 # [ 22.719870] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513604worker1 # [ 22.725112] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3605worker2 # [ 22.794501] nix-grpc-daemon[775]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513606worker2 # [ 22.799661] nix-grpc-daemon[775]: ts=2026-09-20T21:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3607worker1 # [ 22.925804] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513608worker1 # [ 22.930774] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3609worker2 # [ 23.000881] nix-grpc-daemon[775]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513610worker2 # [ 23.005933] nix-grpc-daemon[775]: ts=2026-09-20T21:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3611worker1 # [ 23.133455] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513612worker1 # [ 23.137558] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3613worker2 # [ 23.208780] nix-grpc-daemon[775]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513614worker2 # [ 23.212883] nix-grpc-daemon[775]: ts=2026-09-20T21:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3615worker1 # [ 23.333741] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513616worker1 # [ 23.338767] nix-grpc-daemon[774]: ts=2026-09-20T21:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3617worker2 # [ 23.409531] nix-grpc-daemon[775]: ts=2026-09-20T21:08:46Z level=info event=scheduler_connected addr=lb:500513618worker2 # [ 23.414604] nix-grpc-daemon[775]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3619worker1 # [ 23.539666] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_connected addr=lb:500513620worker1 # [ 23.544906] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3621worker2 # [ 23.615820] nix-grpc-daemon[775]: ts=2026-09-20T21:08:47Z level=info event=scheduler_connected addr=lb:500513622worker2 # [ 23.621115] nix-grpc-daemon[775]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3623worker1 # [ 23.745696] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_connected addr=lb:500513624worker1 # [ 23.750691] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3625worker2 # [ 23.822194] nix-grpc-daemon[775]: ts=2026-09-20T21:08:47Z level=info event=scheduler_connected addr=lb:500513626worker2 # [ 23.827269] nix-grpc-daemon[775]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3627worker1 # [ 23.951987] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_connected addr=lb:500513628worker1 # [ 23.957281] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3629worker2 # [ 24.028919] nix-grpc-daemon[775]: ts=2026-09-20T21:08:47Z level=info event=scheduler_connected addr=lb:500513630worker2 # [ 24.034709] nix-grpc-daemon[775]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3631worker1 # [ 24.159844] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_connected addr=lb:500513632worker1 # [ 24.164174] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3633worker2 # [ 24.237624] nix-grpc-daemon[775]: ts=2026-09-20T21:08:47Z level=info event=scheduler_connected addr=lb:500513634worker2 # [ 24.241809] nix-grpc-daemon[775]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3635worker1 # [ 24.360786] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_connected addr=lb:500513636worker1 # [ 24.366027] nix-grpc-daemon[774]: ts=2026-09-20T21:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3637worker2 # [ 24.438724] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513638worker2 # [ 24.444137] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3639worker1 # [ 24.566983] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513640worker1 # [ 24.572309] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3641worker2 # [ 24.645213] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513642worker2 # [ 24.650304] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3643worker1 # [ 24.773258] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513644worker1 # [ 24.778516] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3645worker2 # [ 24.851633] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513646worker2 # [ 24.856984] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3647worker1 # [ 24.979396] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513648worker1 # [ 24.984798] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3649worker2 # [ 25.058433] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513650worker2 # [ 25.063559] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3651worker1 # [ 25.187205] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513652worker1 # [ 25.192703] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3653worker2 # [ 25.266527] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513654worker2 # [ 25.270814] nix-grpc-daemon[775]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3655worker1 # [ 25.393704] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_connected addr=lb:500513656worker1 # [ 25.399307] nix-grpc-daemon[774]: ts=2026-09-20T21:08:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3657worker2 # [ 25.467639] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513658worker2 # [ 25.473023] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3659worker1 # [ 25.600497] nix-grpc-daemon[774]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513660worker1 # [ 25.605914] nix-grpc-daemon[774]: ts=2026-09-20T21:08:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3661worker2 # [ 25.674596] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513662worker2 # [ 25.680424] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3663worker1 # [ 25.806841] nix-grpc-daemon[774]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513664worker1 # [ 25.812301] nix-grpc-daemon[774]: ts=2026-09-20T21:08:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3665worker2 # [ 25.881288] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513666worker2 # [ 25.891644] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3667worker1 # [ 26.013312] nix-grpc-daemon[774]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513668worker1 # [ 26.018638] nix-grpc-daemon[774]: ts=2026-09-20T21:08:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3669worker2 # [ 26.089591] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513670worker2 # [ 26.094835] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3671worker1 # [ 26.221431] nix-grpc-daemon[774]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513672worker1 # [ 26.225647] nix-grpc-daemon[774]: ts=2026-09-20T21:08:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3673worker2 # [ 26.298149] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513674worker2 # [ 26.302392] nix-grpc-daemon[775]: ts=2026-09-20T21:08:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3675worker1 # [ 26.422016] nix-grpc-daemon[774]: ts=2026-09-20T21:08:49Z level=info event=scheduler_connected addr=lb:500513676worker1 # [ 26.427343] nix-grpc-daemon[774]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3677worker2 # [ 26.498826] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_connected addr=lb:500513678worker2 # [ 26.504273] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3679worker1 # [ 26.628169] nix-grpc-daemon[774]: ts=2026-09-20T21:08:50Z level=info event=scheduler_connected addr=lb:500513680worker1 # [ 26.633245] nix-grpc-daemon[774]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3681worker2 # [ 26.705295] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_connected addr=lb:500513682worker2 # [ 26.710365] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3683worker1 # [ 26.834187] nix-grpc-daemon[774]: ts=2026-09-20T21:08:50Z level=info event=scheduler_connected addr=lb:500513684worker1 # [ 26.839249] nix-grpc-daemon[774]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3685worker2 # [ 26.911629] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_connected addr=lb:500513686worker2 # [ 26.916774] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3687worker1 # [ 27.040106] nix-grpc-daemon[774]: ts=2026-09-20T21:08:50Z level=info event=scheduler_connected addr=lb:500513688worker1 # [ 27.045107] nix-grpc-daemon[774]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3689worker2 # [ 27.117887] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_connected addr=lb:500513690worker2 # [ 27.123432] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3691worker1 # [ 27.247778] nix-grpc-daemon[774]: ts=2026-09-20T21:08:50Z level=info event=scheduler_connected addr=lb:500513692worker1 # [ 27.252174] nix-grpc-daemon[774]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3693worker2 # [ 27.326207] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_connected addr=lb:500513694worker2 # [ 27.330310] nix-grpc-daemon[775]: ts=2026-09-20T21:08:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3695worker1 # [ 27.448880] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513696worker1 # [ 27.454070] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3697worker2 # [ 27.527066] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513698worker2 # [ 27.532136] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3699worker1 # [ 27.654924] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513700worker1 # [ 27.659713] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3701worker2 # [ 27.733662] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513702worker2 # [ 27.739018] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3703worker1 # [ 27.861072] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513704worker1 # [ 27.866544] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3705worker2 # [ 27.940913] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513706worker2 # [ 27.946406] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3707worker1 # [ 28.072168] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513708worker1 # [ 28.077096] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3709worker2 # [ 28.148750] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513710worker2 # [ 28.154692] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3711worker1 # [ 28.281390] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513712worker1 # [ 28.285762] nix-grpc-daemon[774]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3713worker2 # [ 28.358065] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_connected addr=lb:500513714worker2 # [ 28.363104] nix-grpc-daemon[775]: ts=2026-09-20T21:08:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3715worker1 # [ 28.482008] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513716worker1 # [ 28.488428] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3717worker2 # [ 28.564924] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513718worker2 # [ 28.570558] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3719worker1 # [ 28.689874] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513720worker1 # [ 28.695687] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3721worker2 # [ 28.772916] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513722worker2 # [ 28.778747] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3723worker1 # [ 28.896922] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513724worker1 # [ 28.902425] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3725worker2 # [ 28.980537] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513726worker2 # [ 28.985666] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker1 # [ 29.103400] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513728worker1 # [ 29.109013] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3729worker2 # [ 29.188314] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513730worker2 # [ 29.197011] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker1 # [ 29.312096] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513732worker1 # [ 29.314681] nix-grpc-daemon[774]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3733worker2 # [ 29.401676] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_connected addr=lb:500513734worker2 # [ 29.406027] nix-grpc-daemon[775]: ts=2026-09-20T21:08:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker1 # [ 29.513418] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_connected addr=lb:500513736worker1 # [ 29.519063] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3737worker2 # [ 29.603037] nix-grpc-daemon[775]: ts=2026-09-20T21:08:53Z level=info event=scheduler_connected addr=lb:500513738worker2 # [ 29.608771] nix-grpc-daemon[775]: ts=2026-09-20T21:08:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3739worker1 # [ 29.720401] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_connected addr=lb:500513740worker1 # [ 29.725934] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3741worker2 # [ 29.810470] nix-grpc-daemon[775]: ts=2026-09-20T21:08:53Z level=info event=scheduler_connected addr=lb:500513742worker2 # [ 29.816443] nix-grpc-daemon[775]: ts=2026-09-20T21:08:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3743worker1 # [ 29.927049] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_connected addr=lb:500513744worker1 # [ 29.933393] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3745worker2 # [ 30.018367] nix-grpc-daemon[775]: ts=2026-09-20T21:08:53Z level=info event=scheduler_connected addr=lb:500513746worker2 # [ 30.024965] nix-grpc-daemon[775]: ts=2026-09-20T21:08:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3747worker1 # [ 30.134670] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_connected addr=lb:500513748worker1 # [ 30.140714] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3749worker2 # [ 30.227061] nix-grpc-daemon[775]: ts=2026-09-20T21:08:53Z level=info event=scheduler_connected addr=lb:500513750worker2 # [ 30.233419] nix-grpc-daemon[775]: ts=2026-09-20T21:08:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751worker1 # [ 30.344895] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_connected addr=lb:500513752worker1 # [ 30.349465] nix-grpc-daemon[774]: ts=2026-09-20T21:08:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3753worker2 # [ 30.437644] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513754worker2 # [ 30.441956] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3755worker1 # [ 30.545889] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513756worker1 # [ 30.551159] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3757worker2 # [ 30.639590] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513758worker2 # [ 30.645798] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3759worker1 # [ 30.752501] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513760worker1 # [ 30.758647] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3761worker2 # [ 30.847241] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513762worker2 # [ 30.853177] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3763worker1 # [ 30.960980] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513764worker1 # [ 30.967412] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3765worker2 # [ 31.055352] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513766worker2 # [ 31.061328] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3767worker1 # [ 31.168181] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513768worker1 # [ 31.174203] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3769worker2 # [ 31.263228] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513770worker2 # [ 31.269861] nix-grpc-daemon[775]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3771lb # [ 31.314539] rustfs-bucket-start[1035]: mb s3://farm3772worker1 # [ 31.377859] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_connected addr=lb:500513773worker1 # [ 31.384122] nix-grpc-daemon[774]: ts=2026-09-20T21:08:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774lb # [ 31.339763] systemd[1]: Finished rustfs-bucket.service.3775lb # [ 31.360325] systemd[1]: Starting niks3 server...3776worker2 # [ 31.474582] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513777worker2 # [ 31.479046] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3778worker1 # [ 31.585196] nix-grpc-daemon[774]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513779worker1 # [ 31.591461] nix-grpc-daemon[774]: ts=2026-09-20T21:08:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3780worker2 # [ 31.675239] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513781worker2 # [ 31.681616] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3782lb # [ 31.680791] postgres[1052]: [1052] ERROR: relation "goose_db_version" does not exist at character 363783lb # [ 31.682139] postgres[1052]: [1052] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3784lb # [ 31.723933] niks3-server[1045]: 2026/09/20 21:08:55 OK 20241026095416_initial_model.sql (25.68ms)3785lb # [ 31.728509] niks3-server[1045]: 2026/09/20 21:08:55 OK 20251210153512_drop_unused_gin_index.sql (4.97ms)3786lb # [ 31.736963] niks3-server[1045]: 2026/09/20 21:08:55 OK 20251218171726_add_pins.sql (8.39ms)3787worker1 # [ 31.792643] nix-grpc-daemon[774]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513788lb # [ 31.743145] niks3-server[1045]: 2026/09/20 21:08:55 OK 20260628120000_add_object_size_and_stats.sql (6.1ms)3789worker1 # [ 31.798691] nix-grpc-daemon[774]: ts=2026-09-20T21:08:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3790lb # [ 31.750058] niks3-server[1045]: 2026/09/20 21:08:55 OK 20260905000000_add_claims.sql (6.86ms)3791lb # [ 31.755514] niks3-server[1045]: 2026/09/20 21:08:55 OK 20260920000000_drop_claims.sql (5.22ms)3792lb # [ 31.757722] niks3-server[1045]: 2026/09/20 21:08:55 goose: successfully migrated database to version: 202609200000003793lb # [ 31.763047] niks3-server[1045]: 2026/09/20 21:08:55 OK 1_commit_pending_closure.sql (7.53ms)3794lb # [ 31.766577] niks3-server[1045]: 2026/09/20 21:08:55 OK 2_object_stats_trigger.sql (3.37ms)3795lb # [ 31.767976] niks3-server[1045]: 2026/09/20 21:08:55 goose: up to current file version: 23796lb # [ 31.772400] niks3-server[1045]: 2026/09/20 21:08:55 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3797lb # [ 31.831026] niks3-server[1045]: 2026/09/20 21:08:55 INFO Read proxy enabled — serving cache objects from S33798lb # [ 31.835156] niks3-server[1045]: 2026/09/20 21:08:55 INFO Using socket-activated listener address=0.0.0.0:57513799lb # [ 31.840828] niks3-server[1045]: 2026/09/20 21:08:55 INFO systemd watchdog enabled interval=15s3800lb # [ 31.843668] niks3-server[1045]: 2026/09/20 21:08:55 INFO Starting HTTP server address=0.0.0.0:57513801lb # [ 31.852365] systemd[1]: Started niks3 server.3802lb # [ 31.853209] systemd[1]: Reached target Multi-User System.3803lb # [ 31.854006] systemd[1]: Startup finished in 1.006s (kernel) + 4.678s (initrd) + 26.155s (userspace) = 31.839s.3804worker2 # [ 31.883969] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513805lb # [ 31.868318] niks3-server[1045]: 2026/09/20 21:08:55 INFO lead: acquired remote=192.168.1.4:335123806worker2 # [ 31.892255] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3807worker2 # [ 31.898399] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_take_over3808worker1 # [ 31.995408] nix-grpc-daemon[774]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513809worker1 # [ 32.001399] nix-grpc-daemon[774]: ts=2026-09-20T21:08:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3810lb: (finished: waiting for unit niks3.service, in 16.97 seconds)3811lb: waiting for TCP port 5751 on localhost3812worker2 # [ 32.087385] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513813worker2 # [ 32.093565] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3814lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3815lb: (finished: waiting for TCP port 5751 on localhost, in 0.09 seconds)3816lb: waiting for unit envoy.service3817worker1 # [ 32.202836] nix-grpc-daemon[774]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513818worker1 # [ 32.208753] nix-grpc-daemon[774]: ts=2026-09-20T21:08:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3819lb: (finished: waiting for unit envoy.service, in 0.08 seconds)3820lb: waiting for TCP port 50051 on localhost3821lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3822lb # [ 32.226962] envoy[926]: conn peer=[::1]:43722 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=43823lb: (finished: waiting for TCP port 50051 on localhost, in 0.05 seconds)3824worker2 # [ 32.295794] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513825worker2 # [ 32.332220] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=oidc_jwks provider=mock keys=13826worker2 # [ 32.339038] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:405803827worker2 # [ 32.344862] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03828worker1 # [ 32.410243] nix-grpc-daemon[774]: ts=2026-09-20T21:08:55Z level=info event=scheduler_connected addr=lb:500513829worker2 # [ 32.376819] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:405803830worker2 # [ 32.386312] nix-grpc-daemon[775]: ts=2026-09-20T21:08:55Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03831worker2 # [ 32.484859] systemd[1]: Starting scheduler-token.service...3832worker2 # [ 32.625188] systemd[1]: scheduler-token.service: Deactivated successfully.3833worker2 # [ 32.630119] systemd[1]: Finished scheduler-token.service.3834worker2 # [ 32.631005] systemd[1]: scheduler-token.service: Consumed 40ms CPU time over 144ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3835lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3836lb: must succeed: curl -sf localhost:9901/clusters3837lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.14 seconds)3838lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.14 seconds)3839subtest: one node schedules and both builders hold a WorkerSession on it3840lb: must succeed: curl -sf localhost:9901/clusters3841lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3842worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3843worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3844lb: must succeed: curl -sf localhost:9901/clusters3845lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)3846lb: must succeed: curl -sf localhost:9901/clusters3847lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3848lb: must succeed: curl -sf localhost:9901/clusters3849lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3850lb: must succeed: curl -sf localhost:9901/clusters3851lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3852worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3853worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.07 seconds)3854worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3855worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3856lb: must succeed: curl -sf localhost:9901/clusters3857lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3858lb: must succeed: curl -sf localhost:9901/clusters3859lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3860worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3861worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.11 seconds)3862worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3863worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3864worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3865worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3866worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3867worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3868(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 1.05 seconds)3869subtest: balancer auth: client cert, bearer token, nothing, unknown CN3870worker1 # [ 34.317839] nix-grpc-daemon[774]: ts=2026-09-20T21:08:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564043871worker1 # [ 34.342427] systemd[1]: Started Nix Daemon.3872worker1 # [ 34.695108] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)3873worker1 # [ 34.745785] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)3874worker1 # [ 34.838282] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 2/5)3875worker1 # [ 34.904963] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 207 ms (attempt 3/5)3876worker1 # [ 35.120109] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 679 ms (attempt 4/5)3877worker1 # [ 35.803422] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3878worker2 # [ 35.825694] nix-grpc-daemon[775]: ts=2026-09-20T21:08:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:405823879worker2 # [ 35.839353] systemd[1]: Started Nix Daemon.3880worker2 # [ 36.196233] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3881worker2 # [ 36.230399] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 1/5)3882worker2 # [ 36.324927] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)3883worker2 # [ 36.368822] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 3/5)3884worker2 # [ 36.387596] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 315 ms (attempt 4/5)3885worker2 # [ 36.713115] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3886worker2 # [ 36.730766] nix-grpc-daemon[775]: ts=2026-09-20T21:09:00Z level=debug event=path_info_miss path=gkhkardj3d6k7q4gs4n9bbjjpdx2saf4-nixos-system-client-test error="error: path '/nix/store/gkhkardj3d6k7q4gs4n9bbjjpdx2saf4-nixos-system-client-test' is required, but there is no substituter that can build it"3887worker1 # [ 36.775658] nix-grpc-daemon[774]: ts=2026-09-20T21:09:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564043888worker1 # [ 36.793785] nix-grpc-daemon[774]: ts=2026-09-20T21:09:00Z level=debug event=path_info_miss path=gkhkardj3d6k7q4gs4n9bbjjpdx2saf4-nixos-system-client-test error="error: path '/nix/store/gkhkardj3d6k7q4gs4n9bbjjpdx2saf4-nixos-system-client-test' is required, but there is no substituter that can build it"3889client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3890client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.12 seconds)3891client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3892client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.06 seconds)3893worker2 # [ 37.237081] nix-grpc-daemon[775]: ts=2026-09-20T21:09:00Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:405823894worker2 # [ 37.258109] nix-grpc-daemon[775]: ts=2026-09-20T21:09:00Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:405823895worker2 # [ 37.270888] nix-grpc-daemon[775]: ts=2026-09-20T21:09:00Z level=debug event=path_info_miss path=gkhkardj3d6k7q4gs4n9bbjjpdx2saf4-nixos-system-client-test error="error: path '/nix/store/gkhkardj3d6k7q4gs4n9bbjjpdx2saf4-nixos-system-client-test' is required, but there is no substituter that can build it"3896worker1 # [ 37.325308] nix-grpc-daemon[774]: ts=2026-09-20T21:09:00Z level=info event=oidc_jwks provider=mock keys=13897worker1 # [ 37.331442] nix-grpc-daemon[774]: ts=2026-09-20T21:09:00Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:564043898worker1 # [ 37.339975] nix-grpc-daemon[774]: ts=2026-09-20T21:09:00Z level=debug event=path_info_miss path=gkhkardj3d6k7q4gs4n9bbjjpdx2saf4-nixos-system-client-test error="error: path '/nix/store/gkhkardj3d6k7q4gs4n9bbjjpdx2saf4-nixos-system-client-test' is required, but there is no substituter that can build it"3899client: must succeed: rm -r /var/lib/nix-grpc-store3900client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)3901worker2 # [ 37.625057] nix-grpc-daemon[775]: ts=2026-09-20T21:09:01Z level=info event=denied method=QueryMissing cn=- role=none3902worker1 # [ 37.933156] nix-grpc-daemon[774]: ts=2026-09-20T21:09:01Z level=info event=denied method=QueryMissing cn=stranger role=none3903(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.09 seconds)3904subtest: DAG build is scheduled across workers and lands in the cache3905client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23906worker1 # [ 38.206703] nix-grpc-daemon[774]: ts=2026-09-20T21:09:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564043907client # don't know how to build these paths:3908client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3909worker2 # [ 38.187251] nix-grpc-daemon[775]: ts=2026-09-20T21:09:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405823910worker2 # [ 38.232447] nix-grpc-daemon[775]: ts=2026-09-20T21:09:01Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03911worker2 # [ 38.241318] nix-grpc-daemon[775]: ts=2026-09-20T21:09:01Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=13912worker2 # [ 38.283713] nix-grpc-daemon[775]: ts=2026-09-20T21:09:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405823913worker2 # [ 38.306945] nix-grpc-daemon[775]: ts=2026-09-20T21:09:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405823914client # copying 1 paths to 'lb:50051'...3915worker2 # [ 38.324134] nix-grpc-daemon[775]: ts=2026-09-20T21:09:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405823916client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3917worker2 # [ 38.351628] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3918worker2 # [ 38.399168] nix-grpc-daemon[775]: ts=2026-09-20T21:09:01Z level=info event=niks3_push_started pid=8683919worker2 # [ 38.718151] nix-grpc-daemon[868]: time=2026-09-20T21:09:02.303Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""3920worker2 # [ 38.921848] nix-daemon[856]: accepted connection from pid 881, user nix-grpc-daemon (trusted)3921worker2 # [ 38.943963] nix-daemon[856]: reaped child process 884, status = succeeded3922worker2 # [ 39.122562] nix-daemon[856]: accepted connection from pid 886, user nix-grpc-daemon (trusted)3923worker2 # [ 39.146401] nix-daemon[856]: reaped child process 889, status = succeeded3924lb # [ 39.142160] niks3-server[1045]: 2026/09/20 21:09:02 INFO Received uploads request method=POST path=/api/pending_closures3925worker2 # [ 39.190992] nix-grpc-daemon[868]: time=2026-09-20T21:09:02.777Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3926worker2 # [ 39.192902] nix-grpc-daemon[868]: time=2026-09-20T21:09:02.779Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3927lb # [ 39.223838] niks3-server[1045]: 2026/09/20 21:09:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3928lb # [ 39.231498] niks3-server[1045]: 2026/09/20 21:09:02 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3929lb # [ 39.263981] niks3-server[1045]: 2026/09/20 21:09:02 INFO Signed narinfos id=1 count=13930worker2 # [ 39.291876] nix-grpc-daemon[868]: time=2026-09-20T21:09:02.878Z level=INFO msg="Uploading 1 narinfos"3931lb # [ 39.282829] niks3-server[1045]: 2026/09/20 21:09:02 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3932lb # [ 39.293112] niks3-server[1045]: 2026/09/20 21:09:02 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3933lb # [ 39.304388] niks3-server[1045]: 2026/09/20 21:09:02 INFO Completed upload id=13934worker2 # [ 39.326068] nix-grpc-daemon[868]: time=2026-09-20T21:09:02.912Z level=INFO msg="Upload complete. (599ms)"3935lb # [ 39.315841] niks3-server[1045]: 2026/09/20 21:09:02 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3936worker2 # [ 39.335929] nix-grpc-daemon[775]: ts=2026-09-20T21:09:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 paths=1 nar_bytes_in=5043937worker2 # [ 39.352630] nix-grpc-daemon[775]: ts=2026-09-20T21:09:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405823938worker2 # [ 39.362418] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3939worker2 # [ 39.368413] nix-daemon[856]: reaped child process 866, status = succeeded3940client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3941client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3942worker2 # [ 39.385195] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3943client # farm-a> LOG-farm-a-t13944worker2 # [ 40.824265] nix-daemon[856]: reaped child process 894, status = succeeded3945worker2 # [ 41.020174] nix-daemon[856]: accepted connection from pid 909, user nix-grpc-daemon (trusted)3946worker2 # [ 41.040891] nix-daemon[856]: reaped child process 912, status = succeeded3947lb # [ 41.033822] niks3-server[1045]: 2026/09/20 21:09:04 INFO Received uploads request method=POST path=/api/pending_closures3948worker2 # [ 41.076302] nix-grpc-daemon[868]: time=2026-09-20T21:09:04.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3949worker2 # [ 41.079218] nix-grpc-daemon[868]: time=2026-09-20T21:09:04.661Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3950lb # [ 41.095481] niks3-server[1045]: 2026/09/20 21:09:04 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3951lb # [ 41.119648] niks3-server[1045]: 2026/09/20 21:09:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3952worker2 # [ 41.149361] nix-grpc-daemon[868]: time=2026-09-20T21:09:04.734Z level=INFO msg="Uploading 1 narinfos"3953lb # [ 41.130391] niks3-server[1045]: 2026/09/20 21:09:04 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3954lb # [ 41.140107] niks3-server[1045]: 2026/09/20 21:09:04 INFO Signed narinfos id=2 count=13955lb # [ 41.145950] niks3-server[1045]: 2026/09/20 21:09:04 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3956lb # [ 41.158883] niks3-server[1045]: 2026/09/20 21:09:04 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3957lb # [ 41.164586] niks3-server[1045]: 2026/09/20 21:09:04 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo3958lb # [ 41.170360] niks3-server[1045]: 2026/09/20 21:09:04 INFO Completed upload id=23959worker2 # [ 41.192081] nix-grpc-daemon[868]: time=2026-09-20T21:09:04.778Z level=INFO msg="Upload complete. (387ms)"3960worker2 # [ 41.221025] nix-grpc-daemon[775]: ts=2026-09-20T21:09:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=13961worker2 # [ 41.239832] nix-grpc-daemon[775]: ts=2026-09-20T21:09:04Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=03962worker2 # [ 41.244539] nix-grpc-daemon[775]: ts=2026-09-20T21:09:04Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=13963worker2 # [ 41.249785] nix-daemon[856]: reaped child process 892, status = succeeded3964worker2 # [ 41.262880] nix-grpc-daemon[775]: ts=2026-09-20T21:09:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405823965worker2 # [ 41.281159] nix-grpc-daemon[775]: ts=2026-09-20T21:09:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405823966worker2 # [ 41.302334] nix-grpc-daemon[868]: time=2026-09-20T21:09:04.889Z level=INFO msg="All 1 paths already cached"3967client # copying 1 paths to 'lb:50051'...3968worker2 # [ 41.314891] nix-grpc-daemon[775]: ts=2026-09-20T21:09:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405823969client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...3970worker2 # [ 41.339089] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3971worker2 # [ 41.571004] nix-daemon[856]: accepted connection from pid 918, user nix-grpc-daemon (trusted)3972worker2 # [ 41.592562] nix-daemon[856]: reaped child process 921, status = succeeded3973lb # [ 41.654078] envoy[926]: conn peer=[2001:db8:1::1]:44976 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=25403974lb # [ 41.659628] envoy[926]: conn peer=[2001:db8:1::1]:44982 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=843975lb # [ 41.665254] envoy[926]: conn peer=[2001:db8:1::1]:44992 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=133976lb # [ 41.670007] envoy[926]: conn peer=[2001:db8:1::1]:45002 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=143977worker2 # [ 41.780096] nix-daemon[856]: accepted connection from pid 923, user nix-grpc-daemon (trusted)3978worker2 # [ 41.799825] nix-daemon[856]: reaped child process 926, status = succeeded3979lb # [ 41.791592] niks3-server[1045]: 2026/09/20 21:09:05 INFO Received uploads request method=POST path=/api/pending_closures3980worker2 # [ 41.825429] nix-grpc-daemon[868]: time=2026-09-20T21:09:05.411Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3981worker2 # [ 41.827069] nix-grpc-daemon[868]: time=2026-09-20T21:09:05.411Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"3982lb # [ 41.824211] niks3-server[1045]: 2026/09/20 21:09:05 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst3983lb # [ 41.838225] niks3-server[1045]: 2026/09/20 21:09:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign3984worker2 # [ 41.867389] nix-grpc-daemon[868]: time=2026-09-20T21:09:05.453Z level=INFO msg="Uploading 1 narinfos"3985lb # [ 41.852774] niks3-server[1045]: 2026/09/20 21:09:05 INFO Signed narinfos id=3 count=13986lb # [ 41.855119] niks3-server[1045]: 2026/09/20 21:09:05 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls3987lb # [ 41.864692] niks3-server[1045]: 2026/09/20 21:09:05 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete3988lb # [ 41.872505] niks3-server[1045]: 2026/09/20 21:09:05 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo3989lb # [ 41.878594] niks3-server[1045]: 2026/09/20 21:09:05 INFO Completed upload id=33990worker2 # [ 41.900134] nix-grpc-daemon[868]: time=2026-09-20T21:09:05.487Z level=INFO msg="Upload complete. (542ms)"3991worker2 # [ 41.907184] nix-grpc-daemon[775]: ts=2026-09-20T21:09:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=0 paths=1 nar_bytes_in=6243992worker2 # [ 41.922522] nix-grpc-daemon[775]: ts=2026-09-20T21:09:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405823993worker2 # [ 41.929796] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3994worker2 # [ 41.935040] nix-daemon[856]: reaped child process 916, status = succeeded3995client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...3996worker2 # [ 41.947234] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3997client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3998client # farm-b> LOG-farm-b-t13999worker2 # [ 43.190977] nix-daemon[856]: reaped child process 930, status = succeeded4000worker2 # [ 43.378624] nix-daemon[856]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4001worker2 # [ 43.401911] nix-daemon[856]: reaped child process 949, status = succeeded4002lb # [ 43.394534] niks3-server[1045]: 2026/09/20 21:09:06 INFO Received uploads request method=POST path=/api/pending_closures4003worker2 # [ 43.428760] nix-grpc-daemon[868]: time=2026-09-20T21:09:07.014Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4004worker2 # [ 43.431577] nix-grpc-daemon[868]: time=2026-09-20T21:09:07.014Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4005lb # [ 43.441328] niks3-server[1045]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4006worker2 # [ 43.476180] nix-grpc-daemon[868]: time=2026-09-20T21:09:07.062Z level=INFO msg="Uploading 1 narinfos"4007lb # [ 43.454714] niks3-server[1045]: 2026/09/20 21:09:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4008lb # [ 43.467682] niks3-server[1045]: 2026/09/20 21:09:07 INFO Signed narinfos id=4 count=14009lb # [ 43.475633] niks3-server[1045]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4010lb # [ 43.492253] niks3-server[1045]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4011lb # [ 43.495999] niks3-server[1045]: 2026/09/20 21:09:07 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4012lb # [ 43.505895] niks3-server[1045]: 2026/09/20 21:09:07 INFO Completed upload id=44013worker2 # [ 43.526793] nix-grpc-daemon[868]: time=2026-09-20T21:09:07.113Z level=INFO msg="Upload complete. (365ms)"4014lb # [ 43.510698] niks3-server[1045]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4015worker2 # [ 43.549420] nix-grpc-daemon[775]: ts=2026-09-20T21:09:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14016worker2 # [ 43.567530] nix-grpc-daemon[775]: ts=2026-09-20T21:09:07Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04017worker2 # [ 43.572683] nix-grpc-daemon[775]: ts=2026-09-20T21:09:07Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=14018worker2 # [ 43.577052] nix-daemon[856]: reaped child process 928, status = succeeded4019worker2 # [ 43.596184] nix-grpc-daemon[775]: ts=2026-09-20T21:09:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824020worker2 # [ 43.612937] nix-grpc-daemon[775]: ts=2026-09-20T21:09:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405824021worker2 # [ 43.629985] nix-grpc-daemon[868]: time=2026-09-20T21:09:07.216Z level=INFO msg="All 2 paths already cached"4022client # copying 1 paths to 'lb:50051'...4023client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4024worker2 # [ 43.643013] nix-grpc-daemon[775]: ts=2026-09-20T21:09:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405824025worker2 # [ 43.662735] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4026worker2 # [ 43.881166] nix-daemon[856]: accepted connection from pid 955, user nix-grpc-daemon (trusted)4027worker2 # [ 43.902978] nix-daemon[856]: reaped child process 958, status = succeeded4028worker2 # [ 44.098214] nix-daemon[856]: accepted connection from pid 960, user nix-grpc-daemon (trusted)4029worker2 # [ 44.119270] nix-daemon[856]: reaped child process 963, status = succeeded4030lb # [ 44.111433] niks3-server[1045]: 2026/09/20 21:09:07 INFO Received uploads request method=POST path=/api/pending_closures4031worker2 # [ 44.144315] nix-grpc-daemon[868]: time=2026-09-20T21:09:07.731Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4032worker2 # [ 44.146647] nix-grpc-daemon[868]: time=2026-09-20T21:09:07.733Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4033lb # [ 44.144372] niks3-server[1045]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4034worker2 # [ 44.190478] nix-grpc-daemon[868]: time=2026-09-20T21:09:07.777Z level=INFO msg="Uploading 1 narinfos"4035lb # [ 44.163008] niks3-server[1045]: 2026/09/20 21:09:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4036lb # [ 44.178797] niks3-server[1045]: 2026/09/20 21:09:07 INFO Signed narinfos id=5 count=14037lb # [ 44.185477] niks3-server[1045]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4038lb # [ 44.192399] niks3-server[1045]: 2026/09/20 21:09:07 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4039lb # [ 44.206591] niks3-server[1045]: 2026/09/20 21:09:07 INFO Completed upload id=54040worker2 # [ 44.228730] nix-grpc-daemon[868]: time=2026-09-20T21:09:07.815Z level=INFO msg="Upload complete. (549ms)"4041lb # [ 44.210812] niks3-server[1045]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4042worker2 # [ 44.240600] nix-grpc-daemon[775]: ts=2026-09-20T21:09:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=0 paths=1 nar_bytes_in=7524043worker2 # [ 44.248636] nix-grpc-daemon[775]: ts=2026-09-20T21:09:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824044worker2 # [ 44.265188] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4045worker2 # [ 44.268765] nix-daemon[856]: reaped child process 953, status = succeeded4046client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4047client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4048worker2 # [ 44.286415] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4049client # farm-top> LOG-farm-top-t14050worker2 # [ 45.320765] nix-daemon[856]: reaped child process 967, status = succeeded4051worker2 # [ 45.506855] nix-daemon[856]: accepted connection from pid 982, user nix-grpc-daemon (trusted)4052worker2 # [ 45.530734] nix-daemon[856]: reaped child process 985, status = succeeded4053lb # [ 45.522875] niks3-server[1045]: 2026/09/20 21:09:09 INFO Received uploads request method=POST path=/api/pending_closures4054worker2 # [ 45.556553] nix-grpc-daemon[868]: time=2026-09-20T21:09:09.142Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4055worker2 # [ 45.558346] nix-grpc-daemon[868]: time=2026-09-20T21:09:09.142Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4056lb # [ 45.576218] niks3-server[1045]: 2026/09/20 21:09:09 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4057lb # [ 45.587889] niks3-server[1045]: 2026/09/20 21:09:09 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4058lb # [ 45.599364] niks3-server[1045]: 2026/09/20 21:09:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4059worker2 # [ 45.621277] nix-grpc-daemon[868]: time=2026-09-20T21:09:09.207Z level=INFO msg="Uploading 1 narinfos"4060lb # [ 45.608660] niks3-server[1045]: 2026/09/20 21:09:09 INFO Signed narinfos id=6 count=14061lb # [ 45.614844] niks3-server[1045]: 2026/09/20 21:09:09 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4062lb # [ 45.628184] niks3-server[1045]: 2026/09/20 21:09:09 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4063lb # [ 45.637327] niks3-server[1045]: 2026/09/20 21:09:09 INFO Completed upload id=64064lb # [ 45.641566] niks3-server[1045]: 2026/09/20 21:09:09 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4065worker2 # [ 45.658978] nix-grpc-daemon[868]: time=2026-09-20T21:09:09.245Z level=INFO msg="Upload complete. (358ms)"4066worker2 # [ 45.673200] nix-grpc-daemon[775]: ts=2026-09-20T21:09:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14067worker2 # [ 45.694994] nix-daemon[856]: reaped child process 965, status = succeeded4068client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.79 seconds)4069client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4070client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.26 seconds)4071client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4072client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4073client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4074client # copying 3 paths...4075client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4076client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4077client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4078client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.33 seconds)4079(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.54 seconds)4080subtest: repeat is answered Cached by the scheduler without building4081worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4082worker1 # finding garbage collector roots...4083worker1 # deleting specified paths...4084worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4085worker1 # deleting unused links...4086worker1 # note: hard linking is currently saving 0.0 KiB4087worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds)4088worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4089worker2 # finding garbage collector roots...4090worker2 # deleting specified paths...4091worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4092worker2 # deleting unused links...4093worker2 # note: hard linking is currently saving 0.0 KiB4094worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.29 seconds)4095client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24096client # don't know how to build these paths:4097client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4098worker1 # [ 47.393739] nix-grpc-daemon[774]: ts=2026-09-20T21:09:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564044099worker1 # [ 47.402663] nix-grpc-daemon[774]: ts=2026-09-20T21:09:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564044100worker2 # [ 47.395651] nix-grpc-daemon[775]: ts=2026-09-20T21:09:10Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14101client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.37 seconds)4102client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4103client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.22 seconds)4104lb: must succeed: curl -sf localhost:9901/clusters4105lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)4106lb: must succeed: curl -sf localhost:9901/clusters4107lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4108worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4109worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)4110(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.56 seconds)4111subtest: inputs already in the cache are not sent to the scheduler4112lb: must succeed: curl -sf localhost:9901/clusters4113lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4114lb: must succeed: curl -sf localhost:9901/clusters4115lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4116worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4117worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4118client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24119client # don't know how to build these paths:4120client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4121worker1 # [ 48.548501] nix-grpc-daemon[774]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564044122worker1 # [ 48.555649] nix-grpc-daemon[774]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564044123worker2 # [ 48.544326] nix-grpc-daemon[775]: ts=2026-09-20T21:09:12Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04124worker2 # [ 48.547759] nix-grpc-daemon[775]: ts=2026-09-20T21:09:12Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14125worker2 # [ 48.595388] nix-grpc-daemon[775]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824126worker2 # [ 48.615501] nix-grpc-daemon[775]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405824127worker2 # [ 48.632955] nix-grpc-daemon[868]: time=2026-09-20T21:09:12.218Z level=INFO msg="All 2 paths already cached"4128client # copying 1 paths to 'lb:50051'...4129client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4130worker2 # [ 48.645962] nix-grpc-daemon[775]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405824131worker2 # [ 48.662775] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4132worker2 # [ 48.875705] nix-daemon[856]: accepted connection from pid 1015, user nix-grpc-daemon (trusted)4133worker2 # [ 48.895777] nix-daemon[856]: reaped child process 1018, status = succeeded4134worker2 # [ 49.100114] nix-daemon[856]: accepted connection from pid 1020, user nix-grpc-daemon (trusted)4135worker2 # [ 49.121105] nix-daemon[856]: reaped child process 1023, status = succeeded4136lb # [ 49.114609] niks3-server[1045]: 2026/09/20 21:09:12 INFO Received uploads request method=POST path=/api/pending_closures4137worker2 # [ 49.147405] nix-grpc-daemon[868]: time=2026-09-20T21:09:12.733Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4138worker2 # [ 49.149179] nix-grpc-daemon[868]: time=2026-09-20T21:09:12.733Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4139lb # [ 49.146459] niks3-server[1045]: 2026/09/20 21:09:12 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4140lb # [ 49.164323] niks3-server[1045]: 2026/09/20 21:09:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4141worker2 # [ 49.191591] nix-grpc-daemon[868]: time=2026-09-20T21:09:12.778Z level=INFO msg="Uploading 1 narinfos"4142lb # [ 49.176785] niks3-server[1045]: 2026/09/20 21:09:12 INFO Signed narinfos id=7 count=14143lb # [ 49.182025] niks3-server[1045]: 2026/09/20 21:09:12 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4144lb # [ 49.194908] niks3-server[1045]: 2026/09/20 21:09:12 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4145lb # [ 49.201881] niks3-server[1045]: 2026/09/20 21:09:12 INFO Completed upload id=74146worker2 # [ 49.223487] nix-grpc-daemon[868]: time=2026-09-20T21:09:12.810Z level=INFO msg="Upload complete. (542ms)"4147lb # [ 49.205346] niks3-server[1045]: 2026/09/20 21:09:12 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4148worker2 # [ 49.232508] nix-grpc-daemon[775]: ts=2026-09-20T21:09:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=0 paths=1 nar_bytes_in=7524149worker2 # [ 49.249780] nix-grpc-daemon[775]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824150worker2 # [ 49.256866] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4151worker2 # [ 49.262416] nix-daemon[856]: reaped child process 1013, status = succeeded4152client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4153client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4154worker2 # [ 49.275694] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4155client # farm-top> LOG-farm-top-t1b4156worker2 # [ 50.908856] nix-daemon[856]: reaped child process 1027, status = succeeded4157worker2 # [ 51.088262] nix-daemon[856]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)4158worker2 # [ 51.109435] nix-daemon[856]: reaped child process 1045, status = succeeded4159lb # [ 51.105710] niks3-server[1045]: 2026/09/20 21:09:14 INFO Received uploads request method=POST path=/api/pending_closures4160worker2 # [ 51.143596] nix-grpc-daemon[868]: time=2026-09-20T21:09:14.726Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4161worker2 # [ 51.145629] nix-grpc-daemon[868]: time=2026-09-20T21:09:14.726Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4162lb # [ 51.149480] niks3-server[1045]: 2026/09/20 21:09:14 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4163lb # [ 51.156236] niks3-server[1045]: 2026/09/20 21:09:14 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4164lb # [ 51.164499] niks3-server[1045]: 2026/09/20 21:09:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4165worker2 # [ 51.186402] nix-grpc-daemon[868]: time=2026-09-20T21:09:14.772Z level=INFO msg="Uploading 1 narinfos"4166lb # [ 51.166979] niks3-server[1045]: 2026/09/20 21:09:14 INFO Signed narinfos id=8 count=14167lb # [ 51.176634] niks3-server[1045]: 2026/09/20 21:09:14 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4168lb # [ 51.187658] niks3-server[1045]: 2026/09/20 21:09:14 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4169worker2 # [ 51.213567] nix-grpc-daemon[868]: time=2026-09-20T21:09:14.800Z level=INFO msg="Upload complete. (322ms)"4170lb # [ 51.194048] niks3-server[1045]: 2026/09/20 21:09:14 INFO Completed upload id=84171lb # [ 51.196972] niks3-server[1045]: 2026/09/20 21:09:14 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4172worker2 # [ 51.220650] nix-grpc-daemon[775]: ts=2026-09-20T21:09:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14173worker2 # [ 51.230911] nix-daemon[856]: reaped child process 1025, status = succeeded4174client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 3.05 seconds)4175client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4176client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.25 seconds)4177lb: must succeed: curl -sf localhost:9901/clusters4178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4179lb: must succeed: curl -sf localhost:9901/clusters4180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4181worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4182lb # [ 51.654162] envoy[926]: conn peer=[2001:db8:1::1]:45012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9566 tx=3881 ms=75344183lb # [ 51.656426] envoy[926]: conn peer=[2001:db8:1::1]:33732 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=614184lb # [ 51.658228] envoy[926]: conn peer=[2001:db8:1::1]:33744 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4159 tx=1730 ms=27374185worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4186(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.71 seconds)4187subtest: two clients wanting the same drv build it once4188client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4189client # [ 51.809996] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4190client # Running as unit: dedup1.service; invocation ID: 1f129ff850a3480d9857f2cc56beb2a04191client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.11 seconds)4192client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4193worker1 # [ 52.147223] nix-grpc-daemon[774]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564044194client # [ 52.107245] nix[878]: don't know how to build these paths:4195client # [ 52.109392] nix[878]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4196worker2 # [ 52.134808] nix-grpc-daemon[775]: ts=2026-09-20T21:09:15Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04197worker2 # [ 52.135249] nix-grpc-daemon[775]: ts=2026-09-20T21:09:15Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14198worker1 # [ 52.220284] nix-grpc-daemon[774]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564044199worker1 # [ 52.243346] nix-grpc-daemon[774]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564044200client # [ 52.211318] nix[878]: copying 1 paths to 'lb:50051'...4201worker1 # [ 52.262309] nix-grpc-daemon[774]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564044202client # [ 52.214263] nix[878]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4203worker1 # [ 52.282827] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4204worker1 # [ 52.317310] nix-grpc-daemon[774]: ts=2026-09-20T21:09:15Z level=info event=niks3_push_started pid=8614205worker1 # [ 52.613535] nix-grpc-daemon[861]: time=2026-09-20T21:09:16.187Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""4206worker1 # [ 52.822600] nix-daemon[835]: accepted connection from pid 874, user nix-grpc-daemon (trusted)4207worker1 # [ 52.847709] nix-daemon[835]: reaped child process 877, status = succeeded4208client # [ 52.938508] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4209client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4210worker1 # [ 53.048365] nix-daemon[835]: accepted connection from pid 879, user nix-grpc-daemon (trusted)4211worker1 # [ 53.079870] nix-daemon[835]: reaped child process 885, status = succeeded4212lb # [ 53.042846] niks3-server[1045]: 2026/09/20 21:09:16 INFO Received uploads request method=POST path=/api/pending_closures4213worker1 # [ 53.111462] nix-grpc-daemon[861]: time=2026-09-20T21:09:16.687Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4214worker1 # [ 53.115206] nix-grpc-daemon[861]: time=2026-09-20T21:09:16.687Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4215worker1 # [ 53.165409] nix-grpc-daemon[861]: time=2026-09-20T21:09:16.740Z level=INFO msg="Uploading 1 narinfos"4216lb # [ 53.096903] niks3-server[1045]: 2026/09/20 21:09:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4217lb # [ 53.109958] niks3-server[1045]: 2026/09/20 21:09:16 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4218lb # [ 53.120205] niks3-server[1045]: 2026/09/20 21:09:16 INFO Signed narinfos id=9 count=14219lb # [ 53.126362] niks3-server[1045]: 2026/09/20 21:09:16 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4220lb # [ 53.137688] niks3-server[1045]: 2026/09/20 21:09:16 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4221lb # [ 53.143939] niks3-server[1045]: 2026/09/20 21:09:16 INFO Completed upload id=94222worker1 # [ 53.201931] nix-grpc-daemon[861]: time=2026-09-20T21:09:16.777Z level=INFO msg="Upload complete. (579ms)"4223lb # [ 53.147289] niks3-server[1045]: 2026/09/20 21:09:16 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4224worker1 # [ 53.208929] nix-grpc-daemon[774]: ts=2026-09-20T21:09:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=0 paths=1 nar_bytes_in=4324225worker1 # [ 53.221699] nix-grpc-daemon[774]: ts=2026-09-20T21:09:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564044226worker1 # [ 53.227880] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4227worker1 # [ 53.232824] nix-daemon[835]: reaped child process 859, status = succeeded4228worker1 # [ 53.245471] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4229client # Running as unit: dedup2.service; invocation ID: 33cb3ed38fcc4886b4f92c74d9c537cd4230client # [ 53.201035] nix[878]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4231client # [ 53.305185] nix[894]: don't know how to build these paths:4232client # [ 53.306275] nix[894]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4233worker2 # [ 53.315476] nix-grpc-daemon[775]: ts=2026-09-20T21:09:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:405824234worker2 # [ 53.332712] nix-grpc-daemon[775]: ts=2026-09-20T21:09:16Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04235worker1 # [ 53.381849] nix-grpc-daemon[774]: ts=2026-09-20T21:09:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564044236client # [ 53.340299] nix[894]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4237client: must succeed: sleep 24238client: (finished: must succeed: sleep 2, in 2.07 seconds)4239client: waiting for success: ! systemctl is-active dedup1 dedup24240worker1 # [ 57.887480] nix-daemon[835]: reaped child process 890, status = succeeded4241worker1 # [ 58.067075] nix-daemon[835]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4242worker1 # [ 58.086999] nix-daemon[835]: reaped child process 922, status = succeeded4243lb # [ 58.044754] niks3-server[1045]: 2026/09/20 21:09:21 INFO Received uploads request method=POST path=/api/pending_closures4244worker1 # [ 58.117619] nix-grpc-daemon[861]: time=2026-09-20T21:09:21.691Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4245worker1 # [ 58.119823] nix-grpc-daemon[861]: time=2026-09-20T21:09:21.691Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4246lb # [ 58.087880] niks3-server[1045]: 2026/09/20 21:09:21 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4247lb # [ 58.112685] niks3-server[1045]: 2026/09/20 21:09:21 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4248lb # [ 58.129982] niks3-server[1045]: 2026/09/20 21:09:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4249worker1 # [ 58.195118] nix-grpc-daemon[861]: time=2026-09-20T21:09:21.769Z level=INFO msg="Uploading 1 narinfos"4250lb # [ 58.138348] niks3-server[1045]: 2026/09/20 21:09:21 INFO Signed narinfos id=10 count=14251lb # [ 58.141918] niks3-server[1045]: 2026/09/20 21:09:21 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4252lb # [ 58.156150] niks3-server[1045]: 2026/09/20 21:09:21 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4253lb # [ 58.167901] niks3-server[1045]: 2026/09/20 21:09:21 INFO Completed upload id=104254worker1 # [ 58.224197] nix-grpc-daemon[861]: time=2026-09-20T21:09:21.800Z level=INFO msg="Upload complete. (359ms)"4255lb # [ 58.173449] niks3-server[1045]: 2026/09/20 21:09:21 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4256worker1 # [ 58.237423] nix-grpc-daemon[774]: ts=2026-09-20T21:09:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14257worker1 # [ 58.258361] nix-grpc-daemon[774]: ts=2026-09-20T21:09:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14258worker1 # [ 58.265355] nix-daemon[835]: reaped child process 888, status = succeeded4259client # [ 58.261328] systemd[1]: dedup2.service: Deactivated successfully.4260client # [ 58.266226] systemd[1]: dedup2.service: Consumed 205ms CPU time over 5.327s wall clock time, 33.2M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4261client # [ 58.277816] systemd[1]: dedup1.service: Deactivated successfully.4262client # [ 58.279728] systemd[1]: dedup1.service: Consumed 199ms CPU time over 6.466s wall clock time, 41.2M memory peak, 6.9K incoming IP traffic, 8.1K outgoing IP traffic.4263client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.15 seconds)4264client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4265client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4266worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4267worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4268worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4269worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4270worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4271worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4272worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4273worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4274(finished: subtest: two clients wanting the same drv build it once, in 7.61 seconds)4275subtest: build hook: nix-daemon with builders = grpc://lb4276client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4277client # [ 59.555005] systemd[1]: Started Nix Daemon.4278client # [ 59.751583] nix-daemon[936]: accepted connection from pid 933, user root (trusted)4279worker2 # [ 59.999763] nix-grpc-daemon[775]: ts=2026-09-20T21:09:23Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:405824280worker1 # [ 60.041884] nix-grpc-daemon[774]: ts=2026-09-20T21:09:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564044281worker2 # [ 60.028356] nix-grpc-daemon[775]: ts=2026-09-20T21:09:23Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04282worker2 # [ 60.034824] nix-grpc-daemon[775]: ts=2026-09-20T21:09:23Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14283worker2 # [ 60.082489] nix-grpc-daemon[775]: ts=2026-09-20T21:09:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824284worker2 # [ 60.100877] nix-grpc-daemon[775]: ts=2026-09-20T21:09:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405824285worker2 # [ 60.117855] nix-grpc-daemon[775]: ts=2026-09-20T21:09:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405824286worker2 # [ 60.134218] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4287worker2 # [ 60.357154] nix-daemon[856]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)4288worker2 # [ 60.380576] nix-daemon[856]: reaped child process 1088, status = succeeded4289worker2 # [ 60.576577] nix-daemon[856]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4290worker2 # [ 60.598300] nix-daemon[856]: reaped child process 1093, status = succeeded4291lb # [ 60.589175] niks3-server[1045]: 2026/09/20 21:09:24 INFO Received uploads request method=POST path=/api/pending_closures4292worker2 # [ 60.621389] nix-grpc-daemon[868]: time=2026-09-20T21:09:24.207Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4293worker2 # [ 60.622953] nix-grpc-daemon[868]: time=2026-09-20T21:09:24.207Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4294lb # [ 60.625596] niks3-server[1045]: 2026/09/20 21:09:24 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4295lb # [ 60.640485] niks3-server[1045]: 2026/09/20 21:09:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4296worker2 # [ 60.666488] nix-grpc-daemon[868]: time=2026-09-20T21:09:24.253Z level=INFO msg="Uploading 1 narinfos"4297lb # [ 60.648230] niks3-server[1045]: 2026/09/20 21:09:24 INFO Signed narinfos id=11 count=14298lb # [ 60.654115] niks3-server[1045]: 2026/09/20 21:09:24 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4299lb # [ 60.667147] niks3-server[1045]: 2026/09/20 21:09:24 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4300worker2 # [ 60.695365] nix-grpc-daemon[868]: time=2026-09-20T21:09:24.282Z level=INFO msg="Upload complete. (543ms)"4301lb # [ 60.673826] niks3-server[1045]: 2026/09/20 21:09:24 INFO Completed upload id=114302worker2 # [ 60.700244] nix-grpc-daemon[775]: ts=2026-09-20T21:09:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=0 paths=1 nar_bytes_in=5124303lb # [ 60.677776] niks3-server[1045]: 2026/09/20 21:09:24 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4304worker2 # [ 60.710511] nix-grpc-daemon[775]: ts=2026-09-20T21:09:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824305worker2 # [ 60.715145] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4306worker2 # [ 60.718743] nix-daemon[856]: reaped child process 1083, status = succeeded4307worker2 # [ 60.729656] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4308lb # [ 61.653097] envoy[926]: conn peer=[2001:db8:1::1]:48572 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1373 ms=61144309lb # [ 61.658308] envoy[926]: conn peer=[2001:db8:1::1]:48584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1085 ms=49154310worker2 # [ 61.779167] nix-daemon[856]: reaped child process 1097, status = succeeded4311worker2 # [ 61.975591] nix-daemon[856]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)4312worker2 # [ 61.998469] nix-daemon[856]: reaped child process 1115, status = succeeded4313lb # [ 61.990182] niks3-server[1045]: 2026/09/20 21:09:25 INFO Received uploads request method=POST path=/api/pending_closures4314worker2 # [ 62.023661] nix-grpc-daemon[868]: time=2026-09-20T21:09:25.607Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4315worker2 # [ 62.025949] nix-grpc-daemon[868]: time=2026-09-20T21:09:25.608Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4316lb # [ 62.029393] niks3-server[1045]: 2026/09/20 21:09:25 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4317lb # [ 62.045300] niks3-server[1045]: 2026/09/20 21:09:25 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4318lb # [ 62.060326] niks3-server[1045]: 2026/09/20 21:09:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4319worker2 # [ 62.090130] nix-grpc-daemon[868]: time=2026-09-20T21:09:25.676Z level=INFO msg="Uploading 1 narinfos"4320lb # [ 62.071634] niks3-server[1045]: 2026/09/20 21:09:25 INFO Signed narinfos id=12 count=14321lb # [ 62.077494] niks3-server[1045]: 2026/09/20 21:09:25 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4322lb # [ 62.088220] niks3-server[1045]: 2026/09/20 21:09:25 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4323lb # [ 62.094840] niks3-server[1045]: 2026/09/20 21:09:25 INFO Completed upload id=124324worker2 # [ 62.116558] nix-grpc-daemon[868]: time=2026-09-20T21:09:25.703Z level=INFO msg="Upload complete. (378ms)"4325lb # [ 62.097794] niks3-server[1045]: 2026/09/20 21:09:25 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4326worker1 # [ 62.169314] nix-grpc-daemon[774]: ts=2026-09-20T21:09:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564044327worker2 # [ 62.133441] nix-grpc-daemon[775]: ts=2026-09-20T21:09:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14328worker2 # [ 62.152229] nix-daemon[856]: reaped child process 1095, status = succeeded4329worker1 # [ 62.189772] nix-grpc-daemon[774]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4330worker2 # [ 62.206839] nix-grpc-daemon[775]: ts=2026-09-20T21:09:25Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:405824331worker2 # [ 62.216351] nix-grpc-daemon[775]: ts=2026-09-20T21:09:25Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=0 paths=1 nar_bytes_out=1284332worker1 # [ 62.545890] nix-grpc-daemon[774]: ts=2026-09-20T21:09:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:564044333worker1 # [ 62.554224] nix-grpc-daemon[774]: ts=2026-09-20T21:09:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564044334worker1 # [ 62.562650] nix-grpc-daemon[861]: time=2026-09-20T21:09:26.136Z level=INFO msg="All 1 paths already cached"4335worker2 # [ 62.541789] nix-grpc-daemon[775]: ts=2026-09-20T21:09:26Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04336worker2 # [ 62.551209] nix-grpc-daemon[775]: ts=2026-09-20T21:09:26Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14337worker2 # [ 62.559422] nix-grpc-daemon[775]: ts=2026-09-20T21:09:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824338worker2 # [ 62.572907] nix-grpc-daemon[775]: ts=2026-09-20T21:09:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405824339worker2 # [ 62.590163] nix-grpc-daemon[868]: time=2026-09-20T21:09:26.177Z level=INFO msg="All 1 paths already cached"4340worker2 # [ 62.601123] nix-grpc-daemon[775]: ts=2026-09-20T21:09:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405824341worker2 # [ 62.614854] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4342worker2 # [ 62.834593] nix-daemon[856]: accepted connection from pid 1121, user nix-grpc-daemon (trusted)4343worker2 # [ 62.859665] nix-daemon[856]: reaped child process 1124, status = succeeded4344worker2 # [ 63.035839] nix-daemon[856]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)4345worker2 # [ 63.061174] nix-daemon[856]: reaped child process 1129, status = succeeded4346lb # [ 63.054023] niks3-server[1045]: 2026/09/20 21:09:26 INFO Received uploads request method=POST path=/api/pending_closures4347worker2 # [ 63.091607] nix-grpc-daemon[868]: time=2026-09-20T21:09:26.677Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4348worker2 # [ 63.093462] nix-grpc-daemon[868]: time=2026-09-20T21:09:26.677Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4349lb # [ 63.094960] niks3-server[1045]: 2026/09/20 21:09:26 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4350worker2 # [ 63.127414] nix-grpc-daemon[868]: time=2026-09-20T21:09:26.714Z level=INFO msg="Uploading 1 narinfos"4351lb # [ 63.110039] niks3-server[1045]: 2026/09/20 21:09:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4352lb # [ 63.120925] niks3-server[1045]: 2026/09/20 21:09:26 INFO Signed narinfos id=13 count=14353lb # [ 63.132363] niks3-server[1045]: 2026/09/20 21:09:26 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4354lb # [ 63.138100] niks3-server[1045]: 2026/09/20 21:09:26 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4355lb # [ 63.145832] niks3-server[1045]: 2026/09/20 21:09:26 INFO Completed upload id=134356worker2 # [ 63.167800] nix-grpc-daemon[868]: time=2026-09-20T21:09:26.754Z level=INFO msg="Upload complete. (530ms)"4357worker2 # [ 63.178542] nix-grpc-daemon[775]: ts=2026-09-20T21:09:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=0 paths=1 nar_bytes_in=6244358lb # [ 63.159715] niks3-server[1045]: 2026/09/20 21:09:26 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4359worker2 # [ 63.193431] nix-grpc-daemon[775]: ts=2026-09-20T21:09:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824360worker2 # [ 63.201842] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4361worker2 # [ 63.207459] nix-daemon[856]: reaped child process 1119, status = succeeded4362worker2 # [ 63.224701] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4363worker2 # [ 64.673641] nix-daemon[856]: reaped child process 1133, status = succeeded4364worker2 # [ 64.867264] nix-daemon[856]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)4365worker2 # [ 64.888034] nix-daemon[856]: reaped child process 1151, status = succeeded4366lb # [ 64.881820] niks3-server[1045]: 2026/09/20 21:09:28 INFO Received uploads request method=POST path=/api/pending_closures4367worker2 # [ 64.913943] nix-grpc-daemon[868]: time=2026-09-20T21:09:28.499Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4368worker2 # [ 64.915735] nix-grpc-daemon[868]: time=2026-09-20T21:09:28.499Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4369lb # [ 64.916820] niks3-server[1045]: 2026/09/20 21:09:28 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4370lb # [ 64.931052] niks3-server[1045]: 2026/09/20 21:09:28 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4371lb # [ 64.946103] niks3-server[1045]: 2026/09/20 21:09:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4372worker2 # [ 64.971874] nix-grpc-daemon[868]: time=2026-09-20T21:09:28.558Z level=INFO msg="Uploading 1 narinfos"4373lb # [ 64.955361] niks3-server[1045]: 2026/09/20 21:09:28 INFO Signed narinfos id=14 count=14374lb # [ 64.961214] niks3-server[1045]: 2026/09/20 21:09:28 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4375lb # [ 64.970337] niks3-server[1045]: 2026/09/20 21:09:28 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4376lb # [ 64.976257] niks3-server[1045]: 2026/09/20 21:09:28 INFO Completed upload id=144377worker2 # [ 64.997173] nix-grpc-daemon[868]: time=2026-09-20T21:09:28.584Z level=INFO msg="Upload complete. (359ms)"4378lb # [ 64.979484] niks3-server[1045]: 2026/09/20 21:09:28 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4379worker2 # [ 65.014560] nix-grpc-daemon[775]: ts=2026-09-20T21:09:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14380worker2 # [ 65.028956] nix-grpc-daemon[775]: ts=2026-09-20T21:09:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:405824381worker2 # [ 65.035651] nix-daemon[856]: reaped child process 1131, status = succeeded4382worker1 # [ 65.077279] nix-grpc-daemon[774]: ts=2026-09-20T21:09:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:564044383worker1 # [ 65.092622] nix-grpc-daemon[774]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4384worker1 # [ 65.113098] nix-grpc-daemon[774]: ts=2026-09-20T21:09:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=0 paths=1 nar_bytes_out=1764385worker1 # [ 65.449331] nix-grpc-daemon[774]: ts=2026-09-20T21:09:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:564044386worker2 # [ 65.417637] nix-grpc-daemon[775]: ts=2026-09-20T21:09:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405824387worker2 # [ 65.428144] nix-grpc-daemon[868]: time=2026-09-20T21:09:29.014Z level=INFO msg="All 2 paths already cached"4388worker2 # [ 65.448101] nix-grpc-daemon[775]: ts=2026-09-20T21:09:29Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04389worker2 # [ 65.455554] nix-grpc-daemon[775]: ts=2026-09-20T21:09:29Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14390worker2 # [ 65.464830] nix-grpc-daemon[775]: ts=2026-09-20T21:09:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824391worker2 # [ 65.477442] nix-grpc-daemon[775]: ts=2026-09-20T21:09:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405824392worker2 # [ 65.490356] nix-grpc-daemon[868]: time=2026-09-20T21:09:29.077Z level=INFO msg="All 2 paths already cached"4393worker2 # [ 65.499517] nix-grpc-daemon[775]: ts=2026-09-20T21:09:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405824394worker2 # [ 65.509759] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4395worker2 # [ 65.702200] nix-daemon[856]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)4396worker2 # [ 65.724430] nix-daemon[856]: reaped child process 1160, status = succeeded4397worker2 # [ 65.902805] nix-daemon[856]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)4398worker2 # [ 65.922495] nix-daemon[856]: reaped child process 1165, status = succeeded4399lb # [ 65.915420] niks3-server[1045]: 2026/09/20 21:09:29 INFO Received uploads request method=POST path=/api/pending_closures4400worker2 # [ 65.947259] nix-grpc-daemon[868]: time=2026-09-20T21:09:29.533Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4401worker2 # [ 65.949006] nix-grpc-daemon[868]: time=2026-09-20T21:09:29.533Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4402lb # [ 65.949453] niks3-server[1045]: 2026/09/20 21:09:29 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4403lb # [ 65.960630] niks3-server[1045]: 2026/09/20 21:09:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4404worker2 # [ 65.989671] nix-grpc-daemon[868]: time=2026-09-20T21:09:29.576Z level=INFO msg="Uploading 1 narinfos"4405lb # [ 65.968226] niks3-server[1045]: 2026/09/20 21:09:29 INFO Signed narinfos id=15 count=14406lb # [ 65.977055] niks3-server[1045]: 2026/09/20 21:09:29 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4407lb # [ 65.986906] niks3-server[1045]: 2026/09/20 21:09:29 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4408worker2 # [ 66.015531] nix-grpc-daemon[868]: time=2026-09-20T21:09:29.602Z level=INFO msg="Upload complete. (493ms)"4409lb # [ 65.994378] niks3-server[1045]: 2026/09/20 21:09:29 INFO Completed upload id=154410lb # [ 65.997610] niks3-server[1045]: 2026/09/20 21:09:29 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4411worker2 # [ 66.025107] nix-grpc-daemon[775]: ts=2026-09-20T21:09:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=0 paths=1 nar_bytes_in=7524412worker2 # [ 66.039641] nix-grpc-daemon[775]: ts=2026-09-20T21:09:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824413worker2 # [ 66.046826] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4414worker2 # [ 66.052814] nix-daemon[856]: reaped child process 1155, status = succeeded4415worker2 # [ 66.068700] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4416worker2 # [ 67.117687] nix-daemon[856]: reaped child process 1169, status = succeeded4417worker2 # [ 67.293985] nix-daemon[856]: accepted connection from pid 1184, user nix-grpc-daemon (trusted)4418worker2 # [ 67.316322] nix-daemon[856]: reaped child process 1187, status = succeeded4419lb # [ 67.308811] niks3-server[1045]: 2026/09/20 21:09:30 INFO Received uploads request method=POST path=/api/pending_closures4420worker2 # [ 67.344516] nix-grpc-daemon[868]: time=2026-09-20T21:09:30.927Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4421worker2 # [ 67.346490] nix-grpc-daemon[868]: time=2026-09-20T21:09:30.927Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4422lb # [ 67.356320] niks3-server[1045]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4423lb # [ 67.369608] niks3-server[1045]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4424worker2 # [ 67.404630] nix-grpc-daemon[868]: time=2026-09-20T21:09:30.991Z level=INFO msg="Uploading 1 narinfos"4425lb # [ 67.382683] niks3-server[1045]: 2026/09/20 21:09:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4426lb # [ 67.392970] niks3-server[1045]: 2026/09/20 21:09:30 INFO Signed narinfos id=16 count=14427lb # [ 67.398732] niks3-server[1045]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4428lb # [ 67.410154] niks3-server[1045]: 2026/09/20 21:09:30 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4429lb # [ 67.419776] niks3-server[1045]: 2026/09/20 21:09:30 INFO Completed upload id=164430worker2 # [ 67.441324] nix-grpc-daemon[868]: time=2026-09-20T21:09:31.028Z level=INFO msg="Upload complete. (361ms)"4431lb # [ 67.423093] niks3-server[1045]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4432worker2 # [ 67.453324] nix-grpc-daemon[775]: ts=2026-09-20T21:09:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14433worker2 # [ 67.473272] nix-daemon[856]: reaped child process 1167, status = succeeded4434worker1 # [ 67.537806] nix-grpc-daemon[774]: ts=2026-09-20T21:09:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564044435worker1 # [ 67.551519] nix-grpc-daemon[774]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4436worker2 # [ 67.562644] nix-grpc-daemon[775]: ts=2026-09-20T21:09:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:405824437worker2 # [ 67.569812] nix-grpc-daemon[775]: ts=2026-09-20T21:09:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=0 paths=1 nar_bytes_out=2324438client # [ 67.657443] nix-daemon[936]: reaped child process 938, status = succeeded4439client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.35 seconds)4440client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4441client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4442client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4443client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.06 seconds)4444client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4445client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4446(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.49 seconds)4447subtest: an input only one worker has still reaches the builder4448client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4449client # this derivation will be built:4450client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4451client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4452client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.47 seconds)4453client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4454client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.21 seconds)4455worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4456worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.26 seconds)4457worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4458worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4459client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24460client # [ 69.030698] nix-daemon[936]: accepted connection from pid 1019, user root (trusted)4461client # this derivation will be built:4462client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4463worker1 # [ 69.309779] nix-grpc-daemon[774]: ts=2026-09-20T21:09:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:564044464client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4465worker2 # [ 69.278429] nix-grpc-daemon[775]: ts=2026-09-20T21:09:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405824466client # copying 1 paths...4467client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4468worker1 # [ 69.335485] nix-grpc-daemon[774]: ts=2026-09-20T21:09:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564044469worker1 # [ 69.351520] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4470worker1 # [ 69.573530] nix-daemon[835]: accepted connection from pid 943, user nix-grpc-daemon (trusted)4471worker1 # [ 69.595664] nix-daemon[835]: reaped child process 946, status = succeeded4472lb # [ 69.551250] niks3-server[1045]: 2026/09/20 21:09:33 INFO Received uploads request method=POST path=/api/pending_closures4473worker1 # [ 69.620091] nix-grpc-daemon[861]: time=2026-09-20T21:09:33.195Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4474worker1 # [ 69.621676] nix-grpc-daemon[861]: time=2026-09-20T21:09:33.195Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4475lb # [ 69.581590] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4476lb # [ 69.599703] niks3-server[1045]: 2026/09/20 21:09:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4477worker1 # [ 69.663625] nix-grpc-daemon[861]: time=2026-09-20T21:09:33.239Z level=INFO msg="Uploading 1 narinfos"4478lb # [ 69.613174] niks3-server[1045]: 2026/09/20 21:09:33 INFO Signed narinfos id=17 count=14479lb # [ 69.620540] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4480lb # [ 69.628367] niks3-server[1045]: 2026/09/20 21:09:33 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4481lb # [ 69.637782] niks3-server[1045]: 2026/09/20 21:09:33 INFO Completed upload id=174482worker1 # [ 69.694580] nix-grpc-daemon[861]: time=2026-09-20T21:09:33.270Z level=INFO msg="Upload complete. (327ms)"4483lb # [ 69.640351] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4484worker1 # [ 69.704351] nix-grpc-daemon[774]: ts=2026-09-20T21:09:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=0 paths=1 nar_bytes_in=1204485worker1 # [ 69.716457] nix-daemon[835]: reaped child process 941, status = succeeded4486worker2 # [ 69.685627] nix-grpc-daemon[775]: ts=2026-09-20T21:09:33Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04487worker2 # [ 69.692633] nix-grpc-daemon[775]: ts=2026-09-20T21:09:33Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14488worker1 # [ 69.770849] nix-grpc-daemon[774]: ts=2026-09-20T21:09:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564044489worker1 # [ 69.789366] nix-grpc-daemon[774]: ts=2026-09-20T21:09:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564044490client # copying 2 paths to 'lb:50051'...4491client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4492worker1 # [ 69.810713] nix-grpc-daemon[774]: ts=2026-09-20T21:09:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564044493client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4494worker1 # [ 69.823859] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4495worker1 # [ 70.037581] nix-daemon[835]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4496worker1 # [ 70.061315] nix-daemon[835]: reaped child process 953, status = succeeded4497worker1 # [ 70.247903] nix-daemon[835]: accepted connection from pid 955, user nix-grpc-daemon (trusted)4498worker1 # [ 70.270921] nix-daemon[835]: reaped child process 958, status = succeeded4499lb # [ 70.228787] niks3-server[1045]: 2026/09/20 21:09:33 INFO Received uploads request method=POST path=/api/pending_closures4500lb # [ 70.244783] niks3-server[1045]: 2026/09/20 21:09:33 INFO Received uploads request method=POST path=/api/pending_closures4501worker1 # [ 70.312544] nix-grpc-daemon[861]: time=2026-09-20T21:09:33.888Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4502worker1 # [ 70.315159] nix-grpc-daemon[861]: time=2026-09-20T21:09:33.890Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4503worker1 # [ 70.317299] nix-grpc-daemon[861]: time=2026-09-20T21:09:33.890Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4504lb # [ 70.288053] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4505lb # [ 70.305966] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4506lb # [ 70.318042] niks3-server[1045]: 2026/09/20 21:09:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4507lb # [ 70.324201] niks3-server[1045]: 2026/09/20 21:09:33 INFO Signed narinfos id=18 count=14508worker1 # [ 70.381153] nix-grpc-daemon[861]: time=2026-09-20T21:09:33.955Z level=INFO msg="Uploading 3 narinfos"4509lb # [ 70.326125] niks3-server[1045]: 2026/09/20 21:09:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4510lb # [ 70.335614] niks3-server[1045]: 2026/09/20 21:09:33 INFO Signed narinfos id=19 count=24511lb # [ 70.340435] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4512lb # [ 70.342655] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4513lb # [ 70.359820] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4514lb # [ 70.369916] niks3-server[1045]: 2026/09/20 21:09:33 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4515lb # [ 70.374822] niks3-server[1045]: 2026/09/20 21:09:33 INFO Completed upload id=184516lb # [ 70.376455] niks3-server[1045]: 2026/09/20 21:09:33 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4517lb # [ 70.379017] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4518lb # [ 70.382352] niks3-server[1045]: 2026/09/20 21:09:33 INFO Completed upload id=194519worker1 # [ 70.439660] nix-grpc-daemon[861]: time=2026-09-20T21:09:34.015Z level=INFO msg="Upload complete. (592ms)"4520lb # [ 70.384428] niks3-server[1045]: 2026/09/20 21:09:33 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4521worker1 # [ 70.456291] nix-grpc-daemon[774]: ts=2026-09-20T21:09:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=0 paths=2 nar_bytes_in=9844522worker1 # [ 70.473067] nix-grpc-daemon[774]: ts=2026-09-20T21:09:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564044523worker1 # [ 70.483949] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4524worker1 # [ 70.489900] nix-daemon[835]: reaped child process 948, status = succeeded4525client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4526client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4527worker1 # [ 70.506051] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4528worker1 # [ 71.569065] nix-daemon[835]: reaped child process 962, status = succeeded4529lb # [ 71.657870] envoy[926]: conn peer=[2001:db8:1::1]:41830 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=334 ms=324530lb # [ 71.663199] envoy[926]: conn peer=[2001:db8:1::1]:48586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3431 tx=1795 ms=22394531lb # [ 71.668335] envoy[926]: conn peer=[2001:db8:1::1]:41856 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=994532lb # [ 71.674276] envoy[926]: conn peer=[2001:db8:1::1]:41844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=1878 ms=26264533lb # [ 71.679352] envoy[926]: conn peer=[2001:db8:1::1]:41872 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4326 tx=2224 ms=21784534lb # [ 71.684377] envoy[926]: conn peer=[2001:db8:1::1]:41882 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=314535worker1 # [ 71.760702] nix-daemon[835]: accepted connection from pid 977, user nix-grpc-daemon (trusted)4536worker1 # [ 71.782323] nix-daemon[835]: reaped child process 980, status = succeeded4537lb # [ 71.741784] niks3-server[1045]: 2026/09/20 21:09:35 INFO Received uploads request method=POST path=/api/pending_closures4538worker1 # [ 71.813930] nix-grpc-daemon[861]: time=2026-09-20T21:09:35.387Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4539lb # [ 71.777970] niks3-server[1045]: 2026/09/20 21:09:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4540worker1 # [ 71.845915] nix-grpc-daemon[861]: time=2026-09-20T21:09:35.421Z level=INFO msg="Uploading 1 narinfos"4541lb # [ 71.794847] niks3-server[1045]: 2026/09/20 21:09:35 INFO Signed narinfos id=20 count=14542lb # [ 71.801118] niks3-server[1045]: 2026/09/20 21:09:35 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4543lb # [ 71.807372] niks3-server[1045]: 2026/09/20 21:09:35 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4544lb # [ 71.816314] niks3-server[1045]: 2026/09/20 21:09:35 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4545worker1 # [ 71.873562] nix-grpc-daemon[861]: time=2026-09-20T21:09:35.449Z level=INFO msg="Upload complete. (350ms)"4546lb # [ 71.820219] niks3-server[1045]: 2026/09/20 21:09:35 INFO Completed upload id=204547lb # [ 71.822904] niks3-server[1045]: 2026/09/20 21:09:35 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4548worker1 # [ 71.885509] nix-grpc-daemon[774]: ts=2026-09-20T21:09:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14549worker1 # [ 71.905270] nix-daemon[835]: reaped child process 960, status = succeeded4550client # copying 1 paths...4551worker1 # [ 71.931696] nix-grpc-daemon[774]: ts=2026-09-20T21:09:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564044552client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4553worker2 # [ 71.927956] nix-grpc-daemon[775]: ts=2026-09-20T21:09:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:405824554worker2 # [ 71.947973] nix-grpc-daemon[775]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4555worker2 # [ 71.974694] nix-grpc-daemon[775]: ts=2026-09-20T21:09:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=0 paths=1 nar_bytes_out=1204556client # [ 72.104922] nix-daemon[936]: reaped child process 1022, status = succeeded4557client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.33 seconds)4558client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24559client # [ 72.337390] nix-daemon[936]: accepted connection from pid 1041, user root (trusted)4560client # this derivation will be built:4561client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4562worker2 # [ 72.585613] nix-grpc-daemon[775]: ts=2026-09-20T21:09:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:405824563client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4564worker2 # [ 72.593280] nix-grpc-daemon[775]: ts=2026-09-20T21:09:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405824565worker2 # [ 72.623684] nix-grpc-daemon[775]: ts=2026-09-20T21:09:36Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04566worker2 # [ 72.632372] nix-grpc-daemon[775]: ts=2026-09-20T21:09:36Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14567worker2 # [ 72.642646] nix-grpc-daemon[775]: ts=2026-09-20T21:09:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824568worker2 # [ 72.658035] nix-grpc-daemon[775]: ts=2026-09-20T21:09:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405824569client # copying 1 paths to 'lb:50051'...4570client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4571worker2 # [ 72.684145] nix-grpc-daemon[775]: ts=2026-09-20T21:09:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405824572worker2 # [ 72.700905] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4573worker2 # [ 72.726628] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)4574worker2 # [ 72.732062] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)4575worker2 # [ 72.842106] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 374 ms (attempt 3/5)4576worker2 # [ 73.225388] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 288 ms (attempt 4/5)4577worker2 # [ 73.513480] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4578worker2 # [ 73.524236] nix-grpc-daemon[775]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4579worker2 # [ 73.792658] nix-daemon[856]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)4580worker2 # [ 73.813813] nix-daemon[856]: reaped child process 1206, status = succeeded4581worker2 # [ 74.009974] nix-daemon[856]: accepted connection from pid 1208, user nix-grpc-daemon (trusted)4582worker2 # [ 74.031744] nix-daemon[856]: reaped child process 1211, status = succeeded4583lb # [ 74.021414] niks3-server[1045]: 2026/09/20 21:09:37 INFO Received uploads request method=POST path=/api/pending_closures4584worker2 # [ 74.054950] nix-grpc-daemon[868]: time=2026-09-20T21:09:37.640Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4585worker2 # [ 74.056739] nix-grpc-daemon[868]: time=2026-09-20T21:09:37.640Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4586lb # [ 74.056551] niks3-server[1045]: 2026/09/20 21:09:37 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4587lb # [ 74.074701] niks3-server[1045]: 2026/09/20 21:09:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4588worker2 # [ 74.102203] nix-grpc-daemon[868]: time=2026-09-20T21:09:37.688Z level=INFO msg="Uploading 1 narinfos"4589lb # [ 74.083597] niks3-server[1045]: 2026/09/20 21:09:37 INFO Signed narinfos id=21 count=14590lb # [ 74.092119] niks3-server[1045]: 2026/09/20 21:09:37 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4591lb # [ 74.103360] niks3-server[1045]: 2026/09/20 21:09:37 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4592lb # [ 74.110230] niks3-server[1045]: 2026/09/20 21:09:37 INFO Completed upload id=214593worker2 # [ 74.131500] nix-grpc-daemon[868]: time=2026-09-20T21:09:37.718Z level=INFO msg="Upload complete. (570ms)"4594lb # [ 74.113327] niks3-server[1045]: 2026/09/20 21:09:37 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4595worker2 # [ 74.140539] nix-grpc-daemon[775]: ts=2026-09-20T21:09:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 paths=1 nar_bytes_in=5684596worker2 # [ 74.153343] nix-grpc-daemon[775]: ts=2026-09-20T21:09:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405824597worker2 # [ 74.171166] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4598worker2 # [ 74.178213] nix-daemon[856]: reaped child process 1197, status = succeeded4599worker2 # [ 74.189338] nix-grpc-daemon[775]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4600client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4601client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4602worker2 # [ 74.206563] nix-daemon[856]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4603worker2 # [ 75.237516] nix-daemon[856]: reaped child process 1216, status = succeeded4604worker2 # [ 75.429744] nix-daemon[856]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)4605worker2 # [ 75.454378] nix-daemon[856]: reaped child process 1234, status = succeeded4606lb # [ 75.450494] niks3-server[1045]: 2026/09/20 21:09:39 INFO Received uploads request method=POST path=/api/pending_closures4607worker2 # [ 75.480672] nix-grpc-daemon[868]: time=2026-09-20T21:09:39.066Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4608lb # [ 75.479584] niks3-server[1045]: 2026/09/20 21:09:39 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4609lb # [ 75.496592] niks3-server[1045]: 2026/09/20 21:09:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4610worker2 # [ 75.525481] nix-grpc-daemon[868]: time=2026-09-20T21:09:39.111Z level=INFO msg="Uploading 1 narinfos"4611lb # [ 75.506225] niks3-server[1045]: 2026/09/20 21:09:39 INFO Signed narinfos id=22 count=14612lb # [ 75.518520] niks3-server[1045]: 2026/09/20 21:09:39 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4613worker2 # [ 75.555742] nix-grpc-daemon[868]: time=2026-09-20T21:09:39.142Z level=INFO msg="Upload complete. (333ms)"4614lb # [ 75.534783] niks3-server[1045]: 2026/09/20 21:09:39 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4615lb # [ 75.541495] niks3-server[1045]: 2026/09/20 21:09:39 INFO Completed upload id=224616lb # [ 75.544226] niks3-server[1045]: 2026/09/20 21:09:39 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4617worker2 # [ 75.567151] nix-grpc-daemon[775]: ts=2026-09-20T21:09:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40582 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14618client # copying 1 paths...4619worker1 # [ 75.612970] nix-grpc-daemon[774]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564044620worker2 # [ 75.583412] nix-daemon[856]: reaped child process 1213, status = succeeded4621worker1 # [ 75.628498] nix-grpc-daemon[774]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4622client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4623worker1 # [ 75.682932] nix-grpc-daemon[774]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:564044624worker1 # [ 75.688961] nix-grpc-daemon[774]: ts=2026-09-20T21:09:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=0 paths=1 nar_bytes_out=1204625client # [ 75.770430] nix-daemon[936]: reaped child process 1044, status = succeeded4626client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.67 seconds)4627client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24628client # [ 76.016843] nix-daemon[936]: accepted connection from pid 1063, user root (trusted)4629client # this derivation will be built:4630client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4631worker2 # [ 76.277665] nix-grpc-daemon[775]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:405824632client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4633worker1 # [ 76.318079] nix-grpc-daemon[774]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564044634worker1 # [ 76.334260] nix-grpc-daemon[861]: time=2026-09-20T21:09:39.909Z level=INFO msg="All 1 paths already cached"4635worker2 # [ 76.319154] nix-grpc-daemon[775]: ts=2026-09-20T21:09:39Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04636worker2 # [ 76.327390] nix-grpc-daemon[775]: ts=2026-09-20T21:09:39Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14637worker1 # [ 76.368570] nix-grpc-daemon[774]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564044638worker1 # [ 76.386279] nix-grpc-daemon[774]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564044639worker1 # [ 76.405294] nix-grpc-daemon[861]: time=2026-09-20T21:09:39.981Z level=INFO msg="All 1 paths already cached"4640client # copying 1 paths to 'lb:50051'...4641client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4642worker1 # [ 76.419008] nix-grpc-daemon[774]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564044643worker1 # [ 76.435493] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4644worker1 # [ 76.650076] nix-daemon[835]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4645worker1 # [ 76.675934] nix-daemon[835]: reaped child process 988, status = succeeded4646worker1 # [ 76.878346] nix-daemon[835]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4647worker1 # [ 76.899538] nix-daemon[835]: reaped child process 993, status = succeeded4648lb # [ 76.858087] niks3-server[1045]: 2026/09/20 21:09:40 INFO Received uploads request method=POST path=/api/pending_closures4649worker1 # [ 76.926276] nix-grpc-daemon[861]: time=2026-09-20T21:09:40.502Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4650worker1 # [ 76.928760] nix-grpc-daemon[861]: time=2026-09-20T21:09:40.504Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4651lb # [ 76.894955] niks3-server[1045]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4652lb # [ 76.909521] niks3-server[1045]: 2026/09/20 21:09:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4653worker1 # [ 76.974611] nix-grpc-daemon[861]: time=2026-09-20T21:09:40.550Z level=INFO msg="Uploading 1 narinfos"4654lb # [ 76.921867] niks3-server[1045]: 2026/09/20 21:09:40 INFO Signed narinfos id=23 count=14655lb # [ 76.928088] niks3-server[1045]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4656lb # [ 76.935828] niks3-server[1045]: 2026/09/20 21:09:40 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4657lb # [ 76.945718] niks3-server[1045]: 2026/09/20 21:09:40 INFO Completed upload id=234658worker1 # [ 77.003531] nix-grpc-daemon[861]: time=2026-09-20T21:09:40.579Z level=INFO msg="Upload complete. (549ms)"4659lb # [ 76.948536] niks3-server[1045]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4660worker1 # [ 77.013052] nix-grpc-daemon[774]: ts=2026-09-20T21:09:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=0 paths=1 nar_bytes_in=5684661worker1 # [ 77.025596] nix-grpc-daemon[774]: ts=2026-09-20T21:09:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564044662worker1 # [ 77.031725] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4663worker1 # [ 77.037091] nix-daemon[835]: reaped child process 983, status = succeeded4664client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4665client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4666worker1 # [ 77.052240] nix-daemon[835]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4667worker1 # [ 78.497648] nix-daemon[835]: reaped child process 997, status = succeeded4668worker1 # [ 78.692290] nix-daemon[835]: accepted connection from pid 1012, user nix-grpc-daemon (trusted)4669worker1 # [ 78.712382] nix-daemon[835]: reaped child process 1015, status = succeeded4670lb # [ 78.672906] niks3-server[1045]: 2026/09/20 21:09:42 INFO Received uploads request method=POST path=/api/pending_closures4671worker1 # [ 78.743325] nix-grpc-daemon[861]: time=2026-09-20T21:09:42.318Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4672worker1 # [ 78.777719] nix-grpc-daemon[861]: time=2026-09-20T21:09:42.353Z level=INFO msg="Uploading 1 narinfos"4673lb # [ 78.710558] niks3-server[1045]: 2026/09/20 21:09:42 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4674lb # [ 78.727714] niks3-server[1045]: 2026/09/20 21:09:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4675lb # [ 78.736991] niks3-server[1045]: 2026/09/20 21:09:42 INFO Signed narinfos id=24 count=14676lb # [ 78.741653] niks3-server[1045]: 2026/09/20 21:09:42 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4677lb # [ 78.753570] niks3-server[1045]: 2026/09/20 21:09:42 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4678lb # [ 78.761807] niks3-server[1045]: 2026/09/20 21:09:42 INFO Completed upload id=244679worker1 # [ 78.819792] nix-grpc-daemon[861]: time=2026-09-20T21:09:42.395Z level=INFO msg="Upload complete. (355ms)"4680lb # [ 78.765427] niks3-server[1045]: 2026/09/20 21:09:42 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4681worker1 # [ 78.833461] nix-grpc-daemon[774]: ts=2026-09-20T21:09:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14682worker1 # [ 78.851133] nix-daemon[835]: reaped child process 995, status = succeeded4683client # copying 1 paths...4684worker1 # [ 78.881682] nix-grpc-daemon[774]: ts=2026-09-20T21:09:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564044685client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4686worker1 # [ 78.912998] nix-grpc-daemon[774]: ts=2026-09-20T21:09:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:564044687worker1 # [ 78.919683] nix-grpc-daemon[774]: ts=2026-09-20T21:09:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56404 duration_s=0 paths=1 nar_bytes_out=1204688client # [ 79.001104] nix-daemon[936]: reaped child process 1066, status = succeeded4689client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 3.23 seconds)4690(finished: subtest: an input only one worker has still reaches the builder, in 11.20 seconds)4691subtest: upload whose reference the worker lacks is completed from the cache4692client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4693client # this derivation will be built:4694client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4695client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4696client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.51 seconds)4697client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4698client # this derivation will be built:4699client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4700client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4701client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.49 seconds)4702client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4703client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.21 seconds)4704worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4705worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4706worker2 # [ 80.564866] systemd[1]: Stopping Nix worker-protocol over gRPC...4707worker2 # [ 81.171062] nix-grpc-daemon[775]: ts=2026-09-20T21:09:44Z level=info event=signal_exit4708worker2 # [ 81.415745] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_yield4709worker1 # [ 81.457482] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_restarting addr=lb:500514710worker2 # [ 81.421793] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:405804711lb # [ 81.401787] niks3-server[1045]: 2026/09/20 21:09:44 INFO lead: released remote=192.168.1.4:335124712worker2 # [ 81.428369] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=worker_gone addr=192.168.1.3:500514713worker2 # [ 81.432530] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_restarting addr=lb:500514714worker2 # [ 81.438878] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:405804715worker2 # [ 81.443535] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=worker_gone addr=192.168.1.4:500514716worker1 # [ 81.657099] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_connected addr=lb:500514717worker1 # [ 81.662533] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4718worker2 # [ 81.629524] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_connected addr=lb:500514719worker2 # [ 81.635104] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4720lb # [ 81.661345] envoy[926]: conn peer=[2001:db8:1::1]:41884 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4219 tx=1768 ms=27634721lb # [ 81.667258] envoy[926]: conn peer=[2001:db8:1::1]:55158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1154722lb # [ 81.672231] envoy[926]: conn peer=[2001:db8:1::1]:55168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3697 tx=1919 ms=31234723lb # [ 81.678456] envoy[926]: conn peer=[2001:db8:1::1]:55172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=684724lb # [ 81.683331] envoy[926]: conn peer=[2001:db8:1::1]:55174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3698 tx=1801 ms=26684725lb # [ 81.688297] envoy[926]: conn peer=[2001:db8:1::1]:55178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=724726worker1 # [ 81.863497] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_connected addr=lb:500514727worker1 # [ 81.869273] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4728worker2 # [ 81.836546] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_connected addr=lb:500514729worker2 # [ 81.841939] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4730lb # [ 81.849843] niks3-server[1045]: 2026/09/20 21:09:45 INFO lead: acquired remote=192.168.1.3:527924731worker1 # [ 81.907970] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_take_over4732worker1 # [ 82.070082] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_connected addr=lb:500514733worker1 # [ 82.075596] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4734worker2 # [ 82.043547] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_connected addr=lb:500514735worker2 # [ 82.049334] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4736worker1 # [ 82.276672] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_connected addr=lb:500514737worker1 # [ 82.282736] nix-grpc-daemon[774]: ts=2026-09-20T21:09:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4738worker2 # [ 82.252421] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_connected addr=lb:500514739worker2 # [ 82.256646] nix-grpc-daemon[775]: ts=2026-09-20T21:09:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4740worker2 # [ 82.422248] nix-grpc-daemon[775]: ts=2026-09-20T21:09:46Z level=info event=scheduler_restarting4741worker1 # [ 82.484351] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=scheduler_connected addr=lb:500514742worker1 # [ 82.487418] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4743worker2 # [ 82.482908] nix-grpc-daemon[775]: ts=2026-09-20T21:09:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4744worker2 # [ 82.496891] nix-daemon[856]: reaped child process 859, status = succeeded4745worker1 # [ 82.688932] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=scheduler_connected addr=lb:500514746worker1 # [ 82.694656] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4747worker1 # [ 82.895512] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=scheduler_connected addr=lb:500514748worker1 # [ 82.901332] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4749worker1 # [ 83.102240] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=scheduler_connected addr=lb:500514750worker1 # [ 83.107928] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4751worker1 # [ 83.308804] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=scheduler_connected addr=lb:500514752worker1 # [ 83.327947] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:535664753worker1 # [ 83.334319] nix-grpc-daemon[774]: ts=2026-09-20T21:09:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04754worker2 # [ 83.324873] nix-grpc-daemon[775]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4755worker2 # [ 83.329608] nix-grpc-daemon[775]: E0000 00:00:1789938586.910066 818 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4756worker2 # [ 83.329796] nix-grpc-daemon[775]: E0000 00:00:1789938586.910419 818 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4757worker2 # [ 83.406102] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4758worker2 # [ 83.436767] systemd[1]: Stopped Nix worker-protocol over gRPC.4759worker2 # [ 83.440874] systemd[1]: nix-grpc-daemon.service: Consumed 3.915s CPU time over 1min 1.569s wall clock time, 62.8M memory peak, 90.8K incoming IP traffic, 226.8K outgoing IP traffic.4760lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4761lb: must succeed: curl -sf localhost:9901/clusters4762worker2 # [ 83.446783] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4763worker2 # [ 83.448858] systemd[1]: Closed Nix worker-protocol over gRPC.4764worker2 # [ 83.450269] systemd[1]: nix-grpc-daemon.socket: Consumed 618us CPU time over 1min 11.878s wall clock time, 256K memory peak, 93K incoming IP traffic, 73.3K outgoing IP traffic.4765lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4766lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4767client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24768worker1 # [ 83.849869] nix-grpc-daemon[774]: ts=2026-09-20T21:09:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564044769client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4770worker1 # [ 83.858654] nix-grpc-daemon[774]: ts=2026-09-20T21:09:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564044771client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.32 seconds)4772worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4773worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4774worker2 # [ 83.970917] systemd[1]: Listening on Nix worker-protocol over gRPC.4775worker1 # [ 84.088738] systemd[1]: Stopping Nix worker-protocol over gRPC...4776worker2 # [ 84.393453] systemd[1]: Starting scheduler-token.service...4777worker2 # [ 84.513003] systemd[1]: scheduler-token.service: Deactivated successfully.4778worker2 # [ 84.520833] systemd[1]: Finished scheduler-token.service.4779worker2 # [ 84.522738] systemd[1]: scheduler-token.service: Consumed 58ms CPU time over 122ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4780worker2 # [ 84.529474] systemd[1]: Starting Nix worker-protocol over gRPC...4781worker2 # [ 84.721761] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4782worker2 # [ 84.724671] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4783worker2 # [ 84.727955] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14784worker2 # [ 84.736903] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4785worker2 # [ 84.738755] systemd[1]: Started Nix worker-protocol over gRPC.4786worker2 # [ 84.785566] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:48Z level=info event=scheduler_connected addr=lb:500514787worker1 # [ 84.825192] nix-grpc-daemon[774]: ts=2026-09-20T21:09:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:535664788worker1 # [ 84.831703] nix-grpc-daemon[774]: ts=2026-09-20T21:09:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04789worker1 # [ 85.064275] nix-grpc-daemon[774]: ts=2026-09-20T21:09:48Z level=info event=signal_exit4790worker1 # [ 85.331664] nix-grpc-daemon[774]: ts=2026-09-20T21:09:48Z level=info event=scheduler_yield4791lb # [ 85.277491] niks3-server[1045]: 2026/09/20 21:09:48 INFO lead: released remote=192.168.1.3:527924792worker2 # [ 85.301264] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:48Z level=info event=scheduler_restarting addr=lb:500514793worker1 # [ 85.343715] nix-grpc-daemon[774]: ts=2026-09-20T21:09:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:535664794worker1 # [ 85.350169] nix-grpc-daemon[774]: ts=2026-09-20T21:09:48Z level=info event=worker_gone addr=192.168.1.4:500514795worker1 # [ 85.356183] nix-grpc-daemon[774]: ts=2026-09-20T21:09:48Z level=info event=scheduler_restarting addr=lb:500514796worker1 # [ 85.360152] nix-grpc-daemon[774]: ts=2026-09-20T21:09:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:535664797worker1 # [ 85.364527] nix-grpc-daemon[774]: ts=2026-09-20T21:09:48Z level=info event=worker_gone addr=192.168.1.3:500514798worker2 # [ 85.503192] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_connected addr=lb:500514799worker2 # [ 85.509077] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4800worker1 # [ 85.551057] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=scheduler_connected addr=lb:500514801worker1 # [ 85.556765] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4802worker2 # [ 85.710186] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_connected addr=lb:500514803worker2 # [ 85.715614] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4804worker1 # [ 85.757974] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=scheduler_connected addr=lb:500514805worker1 # [ 85.763435] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4806worker2 # [ 85.919005] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_connected addr=lb:500514807worker2 # [ 85.923341] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4808worker1 # [ 85.964362] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=scheduler_connected addr=lb:500514809worker1 # [ 85.969694] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4810worker2 # [ 86.120390] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_connected addr=lb:500514811worker2 # [ 86.125802] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4812worker1 # [ 86.172723] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=scheduler_connected addr=lb:500514813worker1 # [ 86.176909] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4814worker1 # [ 86.337788] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=scheduler_restarting4815worker2 # [ 86.327224] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_connected addr=lb:500514816worker2 # [ 86.332859] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4817worker1 # [ 86.397918] nix-daemon[835]: reaped child process 837, status = succeeded4818worker1 # [ 86.400700] nix-grpc-daemon[774]: ts=2026-09-20T21:09:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4819worker2 # [ 86.533992] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_connected addr=lb:500514820worker2 # [ 86.539308] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4821worker2 # [ 86.740460] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_connected addr=lb:500514822worker2 # [ 86.746572] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4823worker2 # [ 86.950022] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_connected addr=lb:500514824worker2 # [ 86.954427] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4825worker2 # [ 87.151498] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_connected addr=lb:500514826worker2 # [ 87.157298] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4827worker1 # [ 87.237622] nix-grpc-daemon[774]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4828worker1 # [ 87.242354] nix-grpc-daemon[774]: E0000 00:00:1789938590.811774 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true4829worker1 # [ 87.242516] nix-grpc-daemon[774]: E0000 00:00:1789938590.812318 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4830worker1 # [ 87.364623] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4831worker1 # [ 87.394422] systemd[1]: Stopped Nix worker-protocol over gRPC.4832worker2 # [ 87.358592] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_connected addr=lb:500514833worker1 # [ 87.401045] systemd[1]: nix-grpc-daemon.service: Consumed 2.117s CPU time over 1min 5.643s wall clock time, 51.7M memory peak, 54.4K incoming IP traffic, 96.9K outgoing IP traffic.4834worker2 # [ 87.364640] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4835worker1 # [ 87.406465] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4836worker1 # [ 87.408693] systemd[1]: Closed Nix worker-protocol over gRPC.4837worker1 # [ 87.410455] systemd[1]: nix-grpc-daemon.socket: Consumed 625us CPU time over 1min 15.817s wall clock time, 256K memory peak, 69.5K incoming IP traffic, 62.1K outgoing IP traffic.4838lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4839lb: must succeed: curl -sf localhost:9901/clusters4840lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4841lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4842client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24843worker2 # [ 87.565912] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_connected addr=lb:500514844worker2 # [ 87.572729] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4845worker2 # [ 87.768885] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457224846worker2 # [ 87.781722] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_connected addr=lb:500514847worker2 # [ 87.796508] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4848worker2 # [ 87.809685] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)4849worker2 # [ 87.824693] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)4850worker2 # [ 87.879725] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 2/5)4851worker2 # [ 87.914767] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 182 ms (attempt 3/5)4852worker2 # [ 87.982188] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_connected addr=lb:500514853worker2 # [ 87.986635] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4854worker2 # [ 88.105096] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 387 ms (attempt 4/5)4855worker2 # [ 88.183514] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_connected addr=lb:500514856worker2 # [ 88.189347] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4857worker2 # [ 88.390692] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_connected addr=lb:500514858worker2 # [ 88.396734] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4859worker2 # [ 88.500994] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4860client # copying 2 paths...4861client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4862worker2 # [ 88.530190] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457224863client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4864worker2 # [ 88.549565] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)4865worker2 # [ 88.583505] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=niks3_push_started pid=13164866worker2 # [ 88.601612] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=scheduler_connected addr=lb:500514867worker2 # [ 88.612480] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4868worker2 # [ 88.809152] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=scheduler_connected addr=lb:500514869worker2 # [ 88.813541] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4870worker2 # [ 88.829705] nix-grpc-daemon[1316]: time=2026-09-20T21:09:52.415Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4871worker2 # [ 89.012627] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=scheduler_connected addr=lb:500514872worker2 # [ 89.016616] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4873worker2 # [ 89.022360] nix-daemon[856]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)4874worker2 # [ 89.043305] nix-daemon[856]: reaped child process 1332, status = succeeded4875lb # [ 89.035119] niks3-server[1045]: 2026/09/20 21:09:52 INFO Received uploads request method=POST path=/api/pending_closures4876lb # [ 89.045673] niks3-server[1045]: 2026/09/20 21:09:52 INFO Received uploads request method=POST path=/api/pending_closures4877worker2 # [ 89.079288] nix-grpc-daemon[1316]: time=2026-09-20T21:09:52.663Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4878worker2 # [ 89.082132] nix-grpc-daemon[1316]: time=2026-09-20T21:09:52.663Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4879worker2 # [ 89.084426] nix-grpc-daemon[1316]: time=2026-09-20T21:09:52.664Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4880lb # [ 89.104700] niks3-server[1045]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4881lb # [ 89.119671] niks3-server[1045]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4882lb # [ 89.131252] niks3-server[1045]: 2026/09/20 21:09:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4883worker2 # [ 89.160337] nix-grpc-daemon[1316]: time=2026-09-20T21:09:52.745Z level=INFO msg="Uploading 3 narinfos"4884lb # [ 89.141780] niks3-server[1045]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4885lb # [ 89.149376] niks3-server[1045]: 2026/09/20 21:09:52 INFO Signed narinfos id=25 count=14886lb # [ 89.151452] niks3-server[1045]: 2026/09/20 21:09:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4887lb # [ 89.160325] niks3-server[1045]: 2026/09/20 21:09:52 INFO Signed narinfos id=26 count=24888lb # [ 89.162211] niks3-server[1045]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4889lb # [ 89.172563] niks3-server[1045]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4890lb # [ 89.183092] niks3-server[1045]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4891lb # [ 89.188335] niks3-server[1045]: 2026/09/20 21:09:52 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4892lb # [ 89.193584] niks3-server[1045]: 2026/09/20 21:09:52 INFO Completed upload id=254893worker2 # [ 89.216562] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=scheduler_connected addr=lb:500514894lb # [ 89.197732] niks3-server[1045]: 2026/09/20 21:09:52 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4895lb # [ 89.201185] niks3-server[1045]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4896lb # [ 89.203399] niks3-server[1045]: 2026/09/20 21:09:52 INFO Completed upload id=264897worker2 # [ 89.225357] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4898worker2 # [ 89.236525] nix-grpc-daemon[1316]: time=2026-09-20T21:09:52.809Z level=INFO msg="Upload complete. (389ms)"4899worker2 # [ 89.251696] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45722 duration_s=0 paths=2 nar_bytes_in=3124900worker2 # [ 89.257470] nix-daemon[856]: reaped child process 1314, status = succeeded4901client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.80 seconds)4902worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4903worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)4904worker1 # [ 89.436822] systemd[1]: Listening on Nix worker-protocol over gRPC.4905worker1 # [ 89.444323] systemd[1]: Starting Nix worker-protocol over gRPC...4906worker2 # [ 89.423957] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500514907worker2 # [ 89.429636] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4908worker1 # [ 89.643717] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4909worker1 # [ 89.646509] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4910worker1 # [ 89.650954] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14911worker1 # [ 89.659878] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4912worker1 # [ 89.668818] systemd[1]: Started Nix worker-protocol over gRPC.4913worker2 # [ 89.630685] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500514914lb # [ 89.613853] niks3-server[1045]: 2026/09/20 21:09:53 INFO lead: acquired remote=192.168.1.3:516924915worker1 # [ 89.675982] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=scheduler_take_over4916worker2 # [ 89.637243] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4917lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4918lb: must succeed: curl -sf localhost:9901/clusters4919worker1 # [ 89.709611] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500514920worker1 # [ 89.712635] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4921lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4922worker2 # [ 89.838523] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500514923worker2 # [ 89.844509] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4924worker1 # [ 89.913933] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500514925worker1 # [ 89.919452] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4926worker2 # [ 90.046252] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500514927worker2 # [ 90.049423] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4928worker1 # [ 90.120656] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500514929worker1 # [ 90.126669] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4930worker2 # [ 90.250855] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500514931worker2 # [ 90.256755] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4932worker1 # [ 90.327568] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500514933worker1 # [ 90.333424] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4934worker2 # [ 90.457607] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:54Z level=info event=scheduler_connected addr=lb:500514935worker1 # [ 90.510300] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:54Z level=info event=oidc_jwks provider=mock keys=14936worker1 # [ 90.515342] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:536704937worker1 # [ 90.520969] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04938worker1 # [ 90.534175] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:54Z level=info event=scheduler_connected addr=lb:500514939worker1 # [ 90.540299] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:536704940worker1 # [ 90.545617] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04941lb: must succeed: curl -sf localhost:9901/clusters4942lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4943lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.19 seconds)4944(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.81 seconds)4945subtest: scheduler down: the other node takes the lock and keeps it4946lb: must succeed: curl -sf localhost:9901/clusters4947lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4948lb: must succeed: curl -sf localhost:9901/clusters4949lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4950lb: must succeed: curl -sf localhost:9901/clusters4951lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4952lb: must succeed: curl -sf localhost:9901/clusters4953lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4954worker1 # [ 91.173755] systemd[1]: Stopping Nix worker-protocol over gRPC...4955worker1 # [ 91.665636] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=signal_exit4956lb # [ 91.664782] envoy[926]: conn peer=[2001:db8:1::4]:47004 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=43011 tx=2671 ms=602874957lb # [ 91.670161] envoy[926]: conn peer=[2001:db8:1::1]:36710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=274958lb # [ 91.675053] envoy[926]: conn peer=[2001:db8:1::3]:37840 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=9677 tx=2641 ms=642874959lb # [ 91.681040] envoy[926]: conn peer=[2001:db8:1::1]:36714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=305 ms=14834960worker1 # [ 91.870696] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=scheduler_yield4961lb # [ 91.817505] niks3-server[1045]: 2026/09/20 21:09:55 INFO lead: released remote=192.168.1.3:516924962worker1 # [ 91.880859] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=scheduler_restarting addr=lb:500514963worker1 # [ 91.886612] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:536704964worker2 # [ 91.853210] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:55Z level=info event=scheduler_restarting addr=lb:500514965worker1 # [ 91.892479] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=worker_gone addr=192.168.1.3:500514966worker1 # [ 91.896504] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:536704967worker1 # [ 91.900879] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=worker_gone addr=192.168.1.4:500514968worker1 # [ 92.082185] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=scheduler_connected addr=lb:500514969worker2 # [ 92.053158] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:55Z level=info event=scheduler_connected addr=lb:500514970worker1 # [ 92.090070] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4971worker2 # [ 92.059128] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4972worker1 # [ 92.291237] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=scheduler_connected addr=lb:500514973worker2 # [ 92.260571] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:55Z level=info event=scheduler_connected addr=lb:500514974worker1 # [ 92.299014] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4975worker2 # [ 92.266208] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4976worker1 # [ 92.500215] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:56Z level=info event=scheduler_connected addr=lb:500514977worker2 # [ 92.467808] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_connected addr=lb:500514978worker1 # [ 92.505986] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4979worker2 # [ 92.473818] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4980worker1 # [ 92.708888] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:56Z level=info event=scheduler_connected addr=lb:500514981worker2 # [ 92.675188] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_connected addr=lb:500514982worker1 # [ 92.713552] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4983worker2 # [ 92.681037] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4984worker1 # [ 92.876162] nix-grpc-daemon[1038]: ts=2026-09-20T21:09:56Z level=info event=scheduler_restarting4985worker2 # [ 92.884123] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_connected addr=lb:500514986worker2 # [ 92.888346] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4987worker2 # [ 93.085212] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_connected addr=lb:500514988worker2 # [ 93.090684] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4989worker2 # [ 93.291796] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_connected addr=lb:500514990worker2 # [ 93.297331] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4991worker1 # [ 93.365190] nix-grpc-daemon[1038]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4992worker1 # [ 93.369924] nix-grpc-daemon[1038]: E0000 00:00:1789938596.940938 1081 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true4993worker1 # [ 93.378393] nix-grpc-daemon[1038]: E0000 00:00:1789938596.954246 1081 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4994worker2 # [ 93.498383] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_connected addr=lb:500514995worker2 # [ 93.504134] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4996worker1 # [ 93.697760] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4997worker1 # [ 93.708272] systemd[1]: Stopped Nix worker-protocol over gRPC.4998worker1 # [ 93.710764] systemd[1]: nix-grpc-daemon.service: Consumed 233ms CPU time over 4.252s wall clock time, 13.1M memory peak, 10.1K incoming IP traffic, 9.5K outgoing IP traffic.4999worker1 # [ 93.726430] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5000worker1 # [ 93.729549] systemd[1]: Closed Nix worker-protocol over gRPC.5001worker1 # [ 93.732565] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 4.301s wall clock time, 256K memory peak, 16.1K incoming IP traffic, 11.5K outgoing IP traffic.5002worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5003worker2 # [ 93.705215] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_connected addr=lb:500515004worker2 # [ 93.711989] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5005worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.10 seconds)5006worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5007worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5008worker2 # [ 93.913950] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_connected addr=lb:500515009worker2 # [ 93.917495] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5010worker2 # [ 94.115461] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_connected addr=lb:500515011worker2 # [ 94.121192] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5012worker2 # [ 94.322321] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_connected addr=lb:500515013worker2 # [ 94.327876] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5014worker2 # [ 94.528953] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:58Z level=info event=scheduler_connected addr=lb:500515015worker2 # [ 94.534501] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5016worker2 # [ 94.735260] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:58Z level=info event=scheduler_connected addr=lb:500515017worker2 # [ 94.738573] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5018worker2 # [ 94.761870] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:58Z level=info event=scheduler_take_over5019lb # [ 94.741475] niks3-server[1045]: 2026/09/20 21:09:58 INFO lead: acquired remote=192.168.1.4:519245020worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5021worker2 # [ 94.941807] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:58Z level=info event=scheduler_connected addr=lb:500515022worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)5023worker2 # [ 94.972454] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:58Z level=info event=oidc_jwks provider=mock keys=15024worker2 # [ 94.977360] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:58Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:416165025worker2 # [ 94.981438] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:58Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05026worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5027worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5028client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25029client # don't know how to build these paths:5030client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5031worker2 # [ 96.354806] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457225032worker2 # [ 96.359886] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457225033worker2 # [ 96.388227] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:59Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05034worker2 # [ 96.391932] nix-grpc-daemon[1260]: ts=2026-09-20T21:09:59Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15035worker2 # [ 96.439446] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457225036worker2 # [ 96.453738] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457225037client # copying 1 paths to 'lb:50051'...5038client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5039worker2 # [ 96.474468] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457225040worker2 # [ 96.489964] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5041worker2 # [ 96.704734] nix-daemon[856]: accepted connection from pid 1356, user nix-grpc-daemon (trusted)5042worker2 # [ 96.730188] nix-daemon[856]: reaped child process 1359, status = succeeded5043worker2 # [ 96.941533] nix-daemon[856]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5044worker2 # [ 96.965318] nix-daemon[856]: reaped child process 1364, status = succeeded5045lb # [ 96.954996] niks3-server[1045]: 2026/09/20 21:10:00 INFO Received uploads request method=POST path=/api/pending_closures5046worker2 # [ 96.987085] nix-grpc-daemon[1316]: time=2026-09-20T21:10:00.573Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5047worker2 # [ 96.988896] nix-grpc-daemon[1316]: time=2026-09-20T21:10:00.573Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5048lb # [ 96.987651] niks3-server[1045]: 2026/09/20 21:10:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5049lb # [ 97.009369] niks3-server[1045]: 2026/09/20 21:10:00 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5050lb # [ 97.032217] niks3-server[1045]: 2026/09/20 21:10:00 INFO Signed narinfos id=27 count=15051worker2 # [ 97.054472] nix-grpc-daemon[1316]: time=2026-09-20T21:10:00.640Z level=INFO msg="Uploading 1 narinfos"5052lb # [ 97.055429] niks3-server[1045]: 2026/09/20 21:10:00 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5053lb # [ 97.066515] niks3-server[1045]: 2026/09/20 21:10:00 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5054lb # [ 97.076353] niks3-server[1045]: 2026/09/20 21:10:00 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5055lb # [ 97.083752] niks3-server[1045]: 2026/09/20 21:10:00 INFO Completed upload id=275056worker2 # [ 97.105495] nix-grpc-daemon[1316]: time=2026-09-20T21:10:00.692Z level=INFO msg="Upload complete. (599ms)"5057worker2 # [ 97.117187] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45722 duration_s=0 paths=1 nar_bytes_in=5205058worker2 # [ 97.132792] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457225059worker2 # [ 97.146323] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5060worker2 # [ 97.154717] nix-daemon[856]: reaped child process 1354, status = succeeded5061client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5062client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5063worker2 # [ 97.169737] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5064client # farm-a> LOG-farm-a-on-standby5065worker2 # [ 98.617097] nix-daemon[856]: reaped child process 1368, status = succeeded5066worker2 # [ 98.808688] nix-daemon[856]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)5067worker2 # [ 98.829605] nix-daemon[856]: reaped child process 1386, status = succeeded5068lb # [ 98.821840] niks3-server[1045]: 2026/09/20 21:10:02 INFO Received uploads request method=POST path=/api/pending_closures5069worker2 # [ 98.862901] nix-grpc-daemon[1316]: time=2026-09-20T21:10:02.447Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5070worker2 # [ 98.864848] nix-grpc-daemon[1316]: time=2026-09-20T21:10:02.447Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5071lb # [ 98.874863] niks3-server[1045]: 2026/09/20 21:10:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5072worker2 # [ 98.905248] nix-grpc-daemon[1316]: time=2026-09-20T21:10:02.491Z level=INFO msg="Uploading 1 narinfos"5073lb # [ 98.893202] niks3-server[1045]: 2026/09/20 21:10:02 INFO Signed narinfos id=28 count=15074lb # [ 98.899585] niks3-server[1045]: 2026/09/20 21:10:02 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5075lb # [ 98.908974] niks3-server[1045]: 2026/09/20 21:10:02 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5076lb # [ 98.912906] niks3-server[1045]: 2026/09/20 21:10:02 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5077lb # [ 98.921776] niks3-server[1045]: 2026/09/20 21:10:02 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5078lb # [ 98.926711] niks3-server[1045]: 2026/09/20 21:10:02 INFO Completed upload id=285079worker2 # [ 98.948455] nix-grpc-daemon[1316]: time=2026-09-20T21:10:02.535Z level=INFO msg="Upload complete. (365ms)"5080lb # [ 98.929429] niks3-server[1045]: 2026/09/20 21:10:02 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5081worker2 # [ 98.972411] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45722 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15082worker2 # [ 98.983781] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:02Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05083worker2 # [ 98.987251] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:02Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15084worker2 # [ 98.991072] nix-daemon[856]: reaped child process 1366, status = succeeded5085worker2 # [ 99.015175] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457225086worker2 # [ 99.032556] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457225087worker2 # [ 99.048568] nix-grpc-daemon[1316]: time=2026-09-20T21:10:02.635Z level=INFO msg="All 1 paths already cached"5088client # copying 1 paths to 'lb:50051'...5089client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5090worker2 # [ 99.060123] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457225091worker2 # [ 99.075441] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5092worker2 # [ 99.293214] nix-daemon[856]: accepted connection from pid 1392, user nix-grpc-daemon (trusted)5093worker2 # [ 99.316067] nix-daemon[856]: reaped child process 1395, status = succeeded5094worker2 # [ 99.509207] nix-daemon[856]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)5095worker2 # [ 99.528651] nix-daemon[856]: reaped child process 1400, status = succeeded5096lb # [ 99.522126] niks3-server[1045]: 2026/09/20 21:10:03 INFO Received uploads request method=POST path=/api/pending_closures5097worker2 # [ 99.556705] nix-grpc-daemon[1316]: time=2026-09-20T21:10:03.142Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5098worker2 # [ 99.559188] nix-grpc-daemon[1316]: time=2026-09-20T21:10:03.142Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5099lb # [ 99.555236] niks3-server[1045]: 2026/09/20 21:10:03 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5100lb # [ 99.571236] niks3-server[1045]: 2026/09/20 21:10:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5101worker2 # [ 99.599004] nix-grpc-daemon[1316]: time=2026-09-20T21:10:03.185Z level=INFO msg="Uploading 1 narinfos"5102lb # [ 99.586928] niks3-server[1045]: 2026/09/20 21:10:03 INFO Signed narinfos id=29 count=15103lb # [ 99.593783] niks3-server[1045]: 2026/09/20 21:10:03 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5104lb # [ 99.606266] niks3-server[1045]: 2026/09/20 21:10:03 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5105lb # [ 99.612202] niks3-server[1045]: 2026/09/20 21:10:03 INFO Completed upload id=295106worker2 # [ 99.633860] nix-grpc-daemon[1316]: time=2026-09-20T21:10:03.220Z level=INFO msg="Upload complete. (541ms)"5107lb # [ 99.615600] niks3-server[1045]: 2026/09/20 21:10:03 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5108worker2 # [ 99.645470] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45722 duration_s=0 paths=1 nar_bytes_in=6405109worker2 # [ 99.659912] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457225110worker2 # [ 99.671091] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5111worker2 # [ 99.673568] nix-daemon[856]: reaped child process 1390, status = succeeded5112client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5113client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5114worker2 # [ 99.689900] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5115client # farm-b> LOG-farm-b-on-standby5116worker2 # [ 100.337730] nix-daemon[856]: reaped child process 1404, status = succeeded5117worker2 # [ 100.550039] nix-daemon[856]: accepted connection from pid 1419, user nix-grpc-daemon (trusted)5118worker2 # [ 100.572285] nix-daemon[856]: reaped child process 1422, status = succeeded5119lb # [ 100.570098] niks3-server[1045]: 2026/09/20 21:10:04 INFO Received uploads request method=POST path=/api/pending_closures5120worker2 # [ 100.603565] nix-grpc-daemon[1316]: time=2026-09-20T21:10:04.189Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5121worker2 # [ 100.605615] nix-grpc-daemon[1316]: time=2026-09-20T21:10:04.189Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5122lb # [ 100.610956] niks3-server[1045]: 2026/09/20 21:10:04 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5123worker2 # [ 100.649850] nix-grpc-daemon[1316]: time=2026-09-20T21:10:04.236Z level=INFO msg="Uploading 1 narinfos"5124lb # [ 100.628914] niks3-server[1045]: 2026/09/20 21:10:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5125lb # [ 100.641554] niks3-server[1045]: 2026/09/20 21:10:04 INFO Signed narinfos id=30 count=15126lb # [ 100.643779] niks3-server[1045]: 2026/09/20 21:10:04 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5127lb # [ 100.652833] niks3-server[1045]: 2026/09/20 21:10:04 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5128worker2 # [ 100.683057] nix-grpc-daemon[1316]: time=2026-09-20T21:10:04.269Z level=INFO msg="Upload complete. (381ms)"5129lb # [ 100.661424] niks3-server[1045]: 2026/09/20 21:10:04 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5130lb # [ 100.665046] niks3-server[1045]: 2026/09/20 21:10:04 INFO Completed upload id=305131lb # [ 100.667538] niks3-server[1045]: 2026/09/20 21:10:04 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5132worker2 # [ 100.693332] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45722 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15133worker2 # [ 100.709543] nix-daemon[856]: reaped child process 1402, status = succeeded5134worker2 # [ 100.740146] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:04Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05135worker2 # [ 100.746557] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:04Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15136worker2 # [ 100.795298] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457225137worker2 # [ 100.815162] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457225138worker2 # [ 100.834338] nix-grpc-daemon[1316]: time=2026-09-20T21:10:04.421Z level=INFO msg="All 2 paths already cached"5139client # copying 1 paths to 'lb:50051'...5140client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5141worker2 # [ 100.845507] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457225142worker2 # [ 100.861400] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5143worker2 # [ 101.080101] nix-daemon[856]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)5144worker2 # [ 101.105385] nix-daemon[856]: reaped child process 1431, status = succeeded5145worker2 # [ 101.308173] nix-daemon[856]: accepted connection from pid 1433, user nix-grpc-daemon (trusted)5146worker2 # [ 101.329473] nix-daemon[856]: reaped child process 1436, status = succeeded5147lb # [ 101.324185] niks3-server[1045]: 2026/09/20 21:10:04 INFO Received uploads request method=POST path=/api/pending_closures5148worker2 # [ 101.357357] nix-grpc-daemon[1316]: time=2026-09-20T21:10:04.943Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5149worker2 # [ 101.359125] nix-grpc-daemon[1316]: time=2026-09-20T21:10:04.943Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5150lb # [ 101.357325] niks3-server[1045]: 2026/09/20 21:10:04 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5151lb # [ 101.373587] niks3-server[1045]: 2026/09/20 21:10:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5152worker2 # [ 101.403359] nix-grpc-daemon[1316]: time=2026-09-20T21:10:04.989Z level=INFO msg="Uploading 1 narinfos"5153lb # [ 101.387644] niks3-server[1045]: 2026/09/20 21:10:04 INFO Signed narinfos id=31 count=15154lb # [ 101.395918] niks3-server[1045]: 2026/09/20 21:10:04 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5155lb # [ 101.407128] niks3-server[1045]: 2026/09/20 21:10:04 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5156lb # [ 101.415693] niks3-server[1045]: 2026/09/20 21:10:04 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5157lb # [ 101.424151] niks3-server[1045]: 2026/09/20 21:10:04 INFO Completed upload id=315158worker2 # [ 101.445892] nix-grpc-daemon[1316]: time=2026-09-20T21:10:05.032Z level=INFO msg="Upload complete. (565ms)"5159worker2 # [ 101.455344] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45722 duration_s=0 paths=1 nar_bytes_in=7685160worker2 # [ 101.472393] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457225161worker2 # [ 101.478783] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5162worker2 # [ 101.485127] nix-daemon[856]: reaped child process 1426, status = succeeded5163client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5164worker2 # [ 101.498749] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5165client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5166lb # [ 101.666843] envoy[926]: conn peer=[2001:db8:1::3]:42424 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1568 tx=513 ms=32195167client # farm-top> LOG-farm-top-on-standby5168worker2 # [ 102.537335] nix-daemon[856]: reaped child process 1440, status = succeeded5169worker2 # [ 102.717529] nix-daemon[856]: accepted connection from pid 1455, user nix-grpc-daemon (trusted)5170worker2 # [ 102.739736] nix-daemon[856]: reaped child process 1458, status = succeeded5171lb # [ 102.734322] niks3-server[1045]: 2026/09/20 21:10:06 INFO Received uploads request method=POST path=/api/pending_closures5172worker2 # [ 102.769860] nix-grpc-daemon[1316]: time=2026-09-20T21:10:06.354Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5173worker2 # [ 102.771776] nix-grpc-daemon[1316]: time=2026-09-20T21:10:06.354Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5174lb # [ 102.765236] niks3-server[1045]: 2026/09/20 21:10:06 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5175lb # [ 102.781646] niks3-server[1045]: 2026/09/20 21:10:06 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5176lb # [ 102.786702] niks3-server[1045]: 2026/09/20 21:10:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5177worker2 # [ 102.813797] nix-grpc-daemon[1316]: time=2026-09-20T21:10:06.400Z level=INFO msg="Uploading 1 narinfos"5178lb # [ 102.796105] niks3-server[1045]: 2026/09/20 21:10:06 INFO Signed narinfos id=32 count=15179lb # [ 102.798804] niks3-server[1045]: 2026/09/20 21:10:06 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5180lb # [ 102.808645] niks3-server[1045]: 2026/09/20 21:10:06 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5181lb # [ 102.813758] niks3-server[1045]: 2026/09/20 21:10:06 INFO Completed upload id=325182worker2 # [ 102.835318] nix-grpc-daemon[1316]: time=2026-09-20T21:10:06.422Z level=INFO msg="Upload complete. (320ms)"5183lb # [ 102.816659] niks3-server[1045]: 2026/09/20 21:10:06 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5184worker2 # [ 102.849208] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45722 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15185worker2 # [ 102.863426] nix-daemon[856]: reaped child process 1438, status = succeeded5186client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 6.86 seconds)5187client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5188client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.25 seconds)5189worker1 # [ 103.288995] systemd[1]: Listening on Nix worker-protocol over gRPC.5190worker1 # [ 103.296913] systemd[1]: Starting Nix worker-protocol over gRPC...5191worker1 # [ 103.503401] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5192worker1 # [ 103.506008] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5193worker1 # [ 103.510087] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15194worker1 # [ 103.518268] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5195worker1 # [ 103.525584] systemd[1]: Started Nix worker-protocol over gRPC.5196lb: must succeed: curl -sf localhost:9901/clusters5197worker1 # [ 103.568143] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:07Z level=info event=scheduler_connected addr=lb:500515198worker2 # [ 103.536176] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:416165199worker2 # [ 103.542064] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05200lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)5201worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5202worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5203lb: must succeed: curl -sf localhost:9901/clusters5204lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5205lb: must succeed: curl -sf localhost:9901/clusters5206lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5207worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5208worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5209(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.09 seconds)5210subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5211client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5212client # [ 103.992246] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5213client # Running as unit: rs.service; invocation ID: 234799375d4648829ce251ff30a1c9735214client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds)5215worker2 # [ 104.281523] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457225216client # [ 104.276469] nix[1167]: don't know how to build these paths:5217client # [ 104.277709] nix[1167]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5218worker2 # [ 104.302702] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:07Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05219worker2 # [ 104.310297] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:07Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15220worker2 # [ 104.350243] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457225221worker2 # [ 104.368365] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457225222client # [ 104.368413] nix[1167]: copying 1 paths to 'lb:50051'...5223client # [ 104.374206] nix[1167]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5224worker2 # [ 104.386944] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457225225worker2 # [ 104.403544] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5226worker2 # [ 104.624601] nix-daemon[856]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5227worker2 # [ 104.647998] nix-daemon[856]: reaped child process 1474, status = succeeded5228worker2 # [ 104.838478] nix-daemon[856]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)5229worker2 # [ 104.860720] nix-daemon[856]: reaped child process 1479, status = succeeded5230lb # [ 104.853601] niks3-server[1045]: 2026/09/20 21:10:08 INFO Received uploads request method=POST path=/api/pending_closures5231worker2 # [ 104.885462] nix-grpc-daemon[1316]: time=2026-09-20T21:10:08.471Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5232worker2 # [ 104.886973] nix-grpc-daemon[1316]: time=2026-09-20T21:10:08.471Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5233lb # [ 104.885074] niks3-server[1045]: 2026/09/20 21:10:08 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5234worker2 # [ 104.929646] nix-grpc-daemon[1316]: time=2026-09-20T21:10:08.516Z level=INFO msg="Uploading 1 narinfos"5235lb # [ 104.902568] niks3-server[1045]: 2026/09/20 21:10:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5236lb # [ 104.915139] niks3-server[1045]: 2026/09/20 21:10:08 INFO Signed narinfos id=33 count=15237lb # [ 104.921925] niks3-server[1045]: 2026/09/20 21:10:08 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5238lb # [ 104.934025] niks3-server[1045]: 2026/09/20 21:10:08 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5239lb # [ 104.939923] niks3-server[1045]: 2026/09/20 21:10:08 INFO Completed upload id=335240worker2 # [ 104.961533] nix-grpc-daemon[1316]: time=2026-09-20T21:10:08.548Z level=INFO msg="Upload complete. (540ms)"5241lb # [ 104.942702] niks3-server[1045]: 2026/09/20 21:10:08 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5242worker2 # [ 104.971639] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45722 duration_s=0 paths=1 nar_bytes_in=4165243worker2 # [ 104.981056] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457225244worker2 # [ 104.993549] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5245worker2 # [ 104.998857] nix-daemon[856]: reaped child process 1469, status = succeeded5246client # [ 104.997473] nix[1167]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5247worker2 # [ 105.014343] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5248lb: must succeed: curl -sf localhost:9901/clusters5249lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5250lb: must succeed: curl -sf localhost:9901/clusters5251lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5252lb: must succeed: curl -sf localhost:9901/clusters5253lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5254lb: must succeed: curl -sf localhost:9901/clusters5255lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5256worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5257worker1 # [ 107.050073] systemd-journald[273]: Received client request to rotate journal, rotating.5258worker1 # [ 107.069960] systemd-journald[273]: Vacuuming done, freed 0B of archived journals from /var/log/journal/5c2d0510de1242d19e1f2d15138c1773.5259worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.08 seconds)5260worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5261worker2 # [ 107.091471] systemd[1]: Stopping Nix worker-protocol over gRPC...5262worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5263client: must succeed: sleep 5.05264worker2 # [ 107.759160] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:11Z level=info event=draining5265lb # [ 111.667725] envoy[926]: conn peer=[2001:db8:1::1]:60468 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9610 tx=3841 ms=65175266client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5267worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5268worker2 # [ 112.649657] nix-daemon[856]: reaped child process 1483, status = succeeded5269worker2 # [ 112.851776] nix-daemon[856]: accepted connection from pid 1523, user nix-grpc-daemon (trusted)5270worker2 # [ 112.879990] nix-daemon[856]: reaped child process 1526, status = succeeded5271lb # [ 112.874443] niks3-server[1045]: 2026/09/20 21:10:16 INFO Received uploads request method=POST path=/api/pending_closures5272worker2 # [ 112.908849] nix-grpc-daemon[1316]: time=2026-09-20T21:10:16.495Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5273worker2 # [ 112.914595] nix-grpc-daemon[1316]: time=2026-09-20T21:10:16.497Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5274lb # [ 112.937862] niks3-server[1045]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5275lb # [ 112.948437] niks3-server[1045]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5276lb # [ 112.960923] niks3-server[1045]: 2026/09/20 21:10:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5277lb # [ 112.965652] niks3-server[1045]: 2026/09/20 21:10:16 INFO Signed narinfos id=34 count=15278worker2 # [ 112.987573] nix-grpc-daemon[1316]: time=2026-09-20T21:10:16.573Z level=INFO msg="Uploading 1 narinfos"5279lb # [ 112.969142] niks3-server[1045]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5280lb # [ 112.982060] niks3-server[1045]: 2026/09/20 21:10:16 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5281lb # [ 112.992152] niks3-server[1045]: 2026/09/20 21:10:16 INFO Completed upload id=345282worker2 # [ 113.014066] nix-grpc-daemon[1316]: time=2026-09-20T21:10:16.600Z level=INFO msg="Upload complete. (382ms)"5283lb # [ 112.996596] niks3-server[1045]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5284worker2 # [ 113.033745] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45722 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15285worker2 # [ 113.053623] nix-daemon[856]: reaped child process 1481, status = succeeded5286client # [ 113.071009] systemd[1]: rs.service: Deactivated successfully.5287client # [ 113.076208] systemd[1]: rs.service: Consumed 200ms CPU time over 9.083s wall clock time, 41.2M memory peak, 6.9K incoming IP traffic, 8.2K outgoing IP traffic.5288worker2 # [ 113.761463] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:17Z level=info event=signal_exit5289worker2 # [ 113.998710] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:17Z level=info event=scheduler_yield5290lb # [ 113.981492] niks3-server[1045]: 2026/09/20 21:10:17 INFO lead: released remote=192.168.1.4:519245291worker2 # [ 114.005248] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:17Z level=info event=scheduler_restarting addr=lb:500515292worker1 # [ 114.042802] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:17Z level=info event=scheduler_restarting addr=lb:500515293worker2 # [ 114.017190] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:416165294worker2 # [ 114.023604] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:17Z level=info event=worker_gone addr=192.168.1.4:500515295worker2 # [ 114.028296] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:416165296worker2 # [ 114.033970] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:17Z level=info event=worker_gone addr=192.168.1.3:500515297worker2 # [ 114.206674] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:17Z level=info event=scheduler_connected addr=lb:500515298worker1 # [ 114.242958] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:17Z level=info event=scheduler_connected addr=lb:500515299worker2 # [ 114.212627] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5300worker1 # [ 114.248980] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5301worker1 # [ 114.449855] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_connected addr=lb:500515302worker2 # [ 114.413898] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:18Z level=info event=scheduler_connected addr=lb:500515303worker2 # [ 114.419507] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5304worker1 # [ 114.455517] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5305worker1 # [ 114.656683] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_connected addr=lb:500515306worker2 # [ 114.620912] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:18Z level=info event=scheduler_connected addr=lb:500515307worker1 # [ 114.662357] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5308worker2 # [ 114.626859] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5309worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.25 seconds)5310client: waiting for success: ! systemctl is-active rs5311client: (finished: waiting for success: ! systemctl is-active rs, in 0.08 seconds)5312client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5313worker1 # [ 114.863306] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_connected addr=lb:500515314worker2 # [ 114.827804] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:18Z level=info event=scheduler_connected addr=lb:500515315worker1 # [ 114.869359] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5316worker2 # [ 114.833565] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5318client: must fail: journalctl -u rs | grep -q 'reconnecting'5319client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5320lb: must succeed: curl -sf localhost:9901/clusters5321lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5322worker2 # [ 115.005385] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:18Z level=info event=scheduler_restarting5323worker1 # [ 115.072460] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_connected addr=lb:500515324worker1 # [ 115.076503] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5325worker2 # [ 115.065975] nix-grpc-daemon[1260]: ts=2026-09-20T21:10:18Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5326worker2 # [ 115.071023] nix-daemon[856]: reaped child process 1309, status = succeeded5327worker1 # [ 115.272725] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_connected addr=lb:500515328worker1 # [ 115.276341] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5329worker1 # [ 115.476946] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:19Z level=info event=scheduler_connected addr=lb:500515330worker1 # [ 115.479440] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5331worker1 # [ 115.681330] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:19Z level=info event=scheduler_connected addr=lb:500515332worker1 # [ 115.687711] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5333worker2 # [ 115.769654] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5334worker2 # [ 115.773423] systemd[1]: Stopped Nix worker-protocol over gRPC.5335worker2 # [ 115.780853] systemd[1]: nix-grpc-daemon.service: Consumed 2.237s CPU time over 31.236s wall clock time, 36.4M memory peak, 63.1K incoming IP traffic, 160.1K outgoing IP traffic.5336worker2 # [ 115.802280] systemd[1]: Starting scheduler-token.service...5337worker1 # [ 115.889054] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:19Z level=info event=scheduler_connected addr=lb:500515338worker1 # [ 115.895739] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5339worker2 # [ 115.916745] systemd[1]: scheduler-token.service: Deactivated successfully.5340worker2 # [ 115.920879] systemd[1]: Finished scheduler-token.service.5341worker2 # [ 115.922297] systemd[1]: scheduler-token.service: Consumed 49ms CPU time over 118ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5342worker2 # [ 115.931852] systemd[1]: Starting Nix worker-protocol over gRPC...5343lb: must succeed: curl -sf localhost:9901/clusters5344worker1 # [ 116.098009] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:19Z level=info event=scheduler_connected addr=lb:500515345lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5346worker1 # [ 116.103726] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5347worker2 # [ 116.113549] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5348worker2 # [ 116.116443] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5349worker2 # [ 116.120886] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15350worker2 # [ 116.130281] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5351lb # [ 116.128528] niks3-server[1045]: 2026/09/20 21:10:19 INFO lead: acquired remote=192.168.1.4:511925352worker2 # [ 116.155121] systemd[1]: Started Nix worker-protocol over gRPC.5353worker2 # [ 116.165721] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=scheduler_take_over5354worker2 # [ 116.181091] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=scheduler_connected addr=lb:500515355worker2 # [ 116.190250] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=oidc_jwks provider=mock keys=15356worker2 # [ 116.191869] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:462045357worker2 # [ 116.194169] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05358worker2 # [ 116.270811] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:462045359worker1 # [ 116.304986] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:19Z level=info event=scheduler_connected addr=lb:500515360worker2 # [ 116.272948] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05361lb: must succeed: curl -sf localhost:9901/clusters5362lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5363worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5364worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5365(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.31 seconds)5366subtest: scheduler restart mid-build: build finishes, no double build5367client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5368client # [ 117.305773] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5369client # Running as unit: mid.service; invocation ID: edbbfd23e09a432ba891c9f17afc88ab5370client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.10 seconds)5371worker1 # [ 117.645310] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:518225372worker1 # [ 117.661672] nix-daemon[835]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5373client # [ 117.633768] nix[1202]: don't know how to build these paths:5374client # [ 117.635560] nix[1202]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5375worker2 # [ 117.666721] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:21Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05376worker2 # [ 117.672709] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:21Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15377worker2 # [ 117.729895] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462105378worker2 # [ 117.748259] nix-daemon[856]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5379worker2 # [ 117.780474] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5)5380worker2 # [ 117.800570] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 2/5)5381worker2 # [ 117.948193] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 397 ms (attempt 3/5)5382worker2 # [ 118.353524] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 490 ms (attempt 4/5)5383worker2 # [ 118.853813] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5384worker2 # [ 118.878420] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462105385client # [ 118.880965] nix[1202]: copying 1 paths to 'lb:50051'...5386worker2 # [ 118.898516] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:462105387client # [ 118.887881] nix[1202]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5388worker2 # [ 118.921179] nix-daemon[856]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5389worker2 # [ 118.952757] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:22Z level=info event=niks3_push_started pid=16055390worker2 # [ 119.190110] nix-grpc-daemon[1605]: time=2026-09-20T21:10:22.776Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5391worker2 # [ 119.378588] nix-daemon[856]: accepted connection from pid 1618, user nix-grpc-daemon (trusted)5392worker2 # [ 119.399767] nix-daemon[856]: reaped child process 1621, status = succeeded5393worker2 # [ 119.590673] nix-daemon[856]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)5394worker2 # [ 119.610972] nix-daemon[856]: reaped child process 1626, status = succeeded5395lb # [ 119.603848] niks3-server[1045]: 2026/09/20 21:10:23 INFO Received uploads request method=POST path=/api/pending_closures5396worker2 # [ 119.636106] nix-grpc-daemon[1605]: time=2026-09-20T21:10:23.222Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5397worker2 # [ 119.637719] nix-grpc-daemon[1605]: time=2026-09-20T21:10:23.222Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5398lb # [ 119.648557] niks3-server[1045]: 2026/09/20 21:10:23 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5399worker2 # [ 119.684725] nix-grpc-daemon[1605]: time=2026-09-20T21:10:23.270Z level=INFO msg="Uploading 1 narinfos"5400lb # [ 119.662217] niks3-server[1045]: 2026/09/20 21:10:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5401lb # [ 119.675202] niks3-server[1045]: 2026/09/20 21:10:23 INFO Signed narinfos id=35 count=15402lb # [ 119.684341] niks3-server[1045]: 2026/09/20 21:10:23 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5403lb # [ 119.695064] niks3-server[1045]: 2026/09/20 21:10:23 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5404lb # [ 119.710344] niks3-server[1045]: 2026/09/20 21:10:23 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5405lb # [ 119.719622] niks3-server[1045]: 2026/09/20 21:10:23 INFO Completed upload id=355406worker2 # [ 119.741469] nix-grpc-daemon[1605]: time=2026-09-20T21:10:23.328Z level=INFO msg="Upload complete. (546ms)"5407worker2 # [ 119.752653] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46210 duration_s=0 paths=1 nar_bytes_in=4245408worker2 # [ 119.768636] nix-grpc-daemon[1536]: ts=2026-09-20T21:10:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462105409worker2 # [ 119.779269] nix-daemon[856]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5410worker2 # [ 119.786538] nix-daemon[856]: reaped child process 1603, status = succeeded5411client # [ 119.790949] nix[1202]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5412worker2 # [ 119.810510] nix-daemon[856]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5413lb: must succeed: curl -sf localhost:9901/clusters5414lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5415lb: must succeed: curl -sf localhost:9901/clusters5416lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5417worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5418worker2 # [ 121.392986] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1536 (nix-grpc-daemon) on client request.5419worker2 # [ 121.394974] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5420lb # [ 121.389738] niks3-server[1045]: 2026/09/20 21:10:24 INFO lead: released remote=192.168.1.4:511925421client # [ 121.410289] nix[1202]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5422worker2 # [ 121.429508] systemd[1]: Starting scheduler-token.service...5423worker1 # [ 121.459595] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:25Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5424client # [ 121.416945] nix[1202]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5425worker2 # [ 121.448174] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5426worker2 # [ 121.449864] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5427worker2 # [ 121.456285] systemd[1]: nix-grpc-daemon.service: Consumed 558ms CPU time over 5.513s wall clock time, 34.2M memory peak, 13.6K incoming IP traffic, 17.7K outgoing IP traffic.5428worker2 # [ 121.479726] nix-daemon[856]: reaped child process 1628, status = succeeded5429worker2 # [ 121.481725] nix-daemon[856]: reaped child process 1594, status = succeeded5430worker2 # [ 121.492607] nix-daemon[1633]: unexpected Nix daemon error: error: interrupted by the user5431worker2 # [ 121.504455] nix-daemon[856]: reaped child process 1633, status = failed with exit code 15432worker2 # [ 121.543081] systemd[1]: scheduler-token.service: Deactivated successfully.5433worker2 # [ 121.546240] systemd[1]: Finished scheduler-token.service.5434worker2 # [ 121.547104] systemd[1]: scheduler-token.service: Consumed 29ms CPU time over 115ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5435worker2 # [ 121.552412] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5436worker2 # [ 121.555522] systemd[1]: Starting scheduler-token.service...5437worker1 # [ 121.659539] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:25Z level=info event=scheduler_connected addr=lb:500515438worker2 # [ 121.637434] systemd[1]: scheduler-token.service: Deactivated successfully.5439worker2 # [ 121.642486] systemd[1]: Finished scheduler-token.service.5440worker2 # [ 121.643805] systemd[1]: scheduler-token.service: Consumed 27ms CPU time over 83ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5441worker2 # [ 121.648512] systemd[1]: Starting Nix worker-protocol over gRPC...5442lb # [ 121.667277] envoy[926]: conn peer=[2001:db8:1::1]:59212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=1352 ms=87645443lb # [ 121.672629] envoy[926]: conn peer=[2001:db8:1::4]:34418 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=35491 tx=2101 ms=302625444lb # [ 121.677914] envoy[926]: conn peer=[2001:db8:1::4]:56896 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1101 tx=226 ms=52485445worker2 # [ 121.833077] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5446worker2 # [ 121.836821] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5447worker2 # [ 121.839446] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15448worker2 # [ 121.848070] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5449worker2 # [ 121.849929] systemd[1]: Started Nix worker-protocol over gRPC.5450lb # [ 121.842986] niks3-server[1045]: 2026/09/20 21:10:25 INFO lead: acquired remote=192.168.1.4:536765451worker2 # [ 121.870393] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=scheduler_take_over5452worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.52 seconds)5453client: must succeed: sleep 5.05454worker2 # [ 121.884715] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:409025455worker2 # [ 121.886610] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05456worker2 # [ 121.899348] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=scheduler_connected addr=lb:500515457worker2 # [ 121.905386] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=oidc_jwks provider=mock keys=15458worker2 # [ 121.907142] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:409025459worker2 # [ 121.909040] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05460worker2 # [ 121.932076] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05461worker2 # [ 121.934174] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:25Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15462worker1 # [ 122.020189] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518225463worker1 # [ 122.041012] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5)5464worker1 # [ 122.051983] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 2/5)5465worker1 # [ 122.112808] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 268 ms (attempt 3/5)5466worker1 # [ 122.388408] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 4/5)5467worker1 # [ 122.566592] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5468worker1 # [ 122.588698] nix-grpc-daemon[1102]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5469worker1 # [ 122.627621] nix-daemon[835]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5470client # [ 122.597536] nix[1202]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5471worker1 # [ 122.648346] nix-daemon[835]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5472client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5473worker1 # [ 127.271963] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:30Z level=info event=niks3_push_started pid=12535474worker1 # [ 127.294016] nix-daemon[835]: reaped child process 1229, status = succeeded5475client: waiting for success: ! systemctl is-active mid5476worker1 # [ 127.490319] nix-grpc-daemon[1253]: time=2026-09-20T21:10:31.065Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5477worker1 # [ 127.675116] nix-daemon[835]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)5478worker1 # [ 127.697681] nix-daemon[835]: reaped child process 1269, status = succeeded5479lb # [ 127.657496] niks3-server[1045]: 2026/09/20 21:10:31 INFO Received uploads request method=POST path=/api/pending_closures5480worker1 # [ 127.728121] nix-grpc-daemon[1253]: time=2026-09-20T21:10:31.302Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5481worker1 # [ 127.729745] nix-grpc-daemon[1253]: time=2026-09-20T21:10:31.302Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5482lb # [ 127.722302] niks3-server[1045]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5483worker1 # [ 127.810570] nix-grpc-daemon[1253]: time=2026-09-20T21:10:31.384Z level=INFO msg="Uploading 1 narinfos"5484lb # [ 127.750602] niks3-server[1045]: 2026/09/20 21:10:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5485lb # [ 127.759653] niks3-server[1045]: 2026/09/20 21:10:31 INFO Signed narinfos id=36 count=15486lb # [ 127.769014] niks3-server[1045]: 2026/09/20 21:10:31 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5487lb # [ 127.790620] niks3-server[1045]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5488lb # [ 127.805592] niks3-server[1045]: 2026/09/20 21:10:31 INFO Completed upload id=365489worker1 # [ 127.863738] nix-grpc-daemon[1253]: time=2026-09-20T21:10:31.439Z level=INFO msg="Upload complete. (368ms)"5490lb # [ 127.810153] niks3-server[1045]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5491lb # [ 127.821670] niks3-server[1045]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5492worker1 # [ 127.880281] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51822 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15493worker1 # [ 127.902021] nix-daemon[835]: reaped child process 1227, status = succeeded5494client # [ 127.882855] systemd[1]: mid.service: Deactivated successfully.5495client # [ 127.888257] systemd[1]: mid.service: Consumed 216ms CPU time over 10.581s wall clock time, 41.2M memory peak, 8.1K incoming IP traffic, 9.8K outgoing IP traffic.5496client: (finished: waiting for success: ! systemctl is-active mid, in 1.16 seconds)5497client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5498client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5499(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.24 seconds)5500subtest: interrupting the client stops the build on the worker5501client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5502client # [ 128.517155] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5503client # Running as unit: intr.service; invocation ID: e84eadf60b194e8d956a74460566c5195504client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5505worker1 # [ 128.818747] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:518225506client # [ 128.778428] nix[1237]: don't know how to build these paths:5507client # [ 128.779452] nix[1237]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5508worker2 # [ 128.805505] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:32Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05509worker2 # [ 128.812311] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:32Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15510worker2 # [ 128.865103] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409245511worker2 # [ 128.880239] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5512worker2 # [ 128.906205] nix-grpc-daemon[1672]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)5513worker2 # [ 128.958635] nix-grpc-daemon[1672]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 2/5)5514worker2 # [ 129.012595] nix-grpc-daemon[1672]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 3/5)5515worker2 # [ 129.019979] nix-grpc-daemon[1672]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 461 ms (attempt 4/5)5516worker2 # [ 129.481799] nix-grpc-daemon[1672]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5517worker2 # [ 129.504360] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409245518client # [ 129.508201] nix[1237]: copying 1 paths to 'lb:50051'...5519worker2 # [ 129.521791] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409245520client # [ 129.508514] nix[1237]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5521worker2 # [ 129.539733] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5522worker2 # [ 129.571242] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:33Z level=info event=niks3_push_started pid=17445523worker2 # [ 129.790314] nix-grpc-daemon[1744]: time=2026-09-20T21:10:33.376Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5524worker2 # [ 129.988620] nix-daemon[856]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)5525worker2 # [ 130.007452] nix-daemon[856]: reaped child process 1764, status = succeeded5526worker2 # [ 130.196323] nix-daemon[856]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)5527worker2 # [ 130.215980] nix-daemon[856]: reaped child process 1769, status = succeeded5528lb # [ 130.213808] niks3-server[1045]: 2026/09/20 21:10:33 INFO Received uploads request method=POST path=/api/pending_closures5529worker2 # [ 130.251079] nix-grpc-daemon[1744]: time=2026-09-20T21:10:33.836Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5530worker2 # [ 130.252969] nix-grpc-daemon[1744]: time=2026-09-20T21:10:33.836Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5531lb # [ 130.261846] niks3-server[1045]: 2026/09/20 21:10:33 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5532worker2 # [ 130.295828] nix-grpc-daemon[1744]: time=2026-09-20T21:10:33.881Z level=INFO msg="Uploading 1 narinfos"5533lb # [ 130.273365] niks3-server[1045]: 2026/09/20 21:10:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5534lb # [ 130.285719] niks3-server[1045]: 2026/09/20 21:10:33 INFO Signed narinfos id=37 count=15535lb # [ 130.292536] niks3-server[1045]: 2026/09/20 21:10:33 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5536lb # [ 130.304673] niks3-server[1045]: 2026/09/20 21:10:33 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5537lb # [ 130.315449] niks3-server[1045]: 2026/09/20 21:10:33 INFO Completed upload id=375538lb # [ 130.318433] niks3-server[1045]: 2026/09/20 21:10:33 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5539worker2 # [ 130.337107] nix-grpc-daemon[1744]: time=2026-09-20T21:10:33.923Z level=INFO msg="Upload complete. (542ms)"5540worker2 # [ 130.350510] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=0 paths=1 nar_bytes_in=4245541worker2 # [ 130.367671] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409245542worker2 # [ 130.374494] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5543worker2 # [ 130.381747] nix-daemon[856]: reaped child process 1742, status = succeeded5544client # [ 130.380770] nix[1237]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5545worker2 # [ 130.396210] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5546lb # [ 131.666630] envoy[926]: conn peer=[2001:db8:1::1]:35084 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3841 tx=1658 ms=102645547client: must succeed: systemctl kill -s INT intr5548worker2 # [ 132.275914] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:35Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5549client # [ 132.262319] systemd[1]: intr.service: Sent signal SIGINT to main process 1237 (nix) on client request.5550client # [ 132.265519] nix[1237]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5551client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5552client # [ 132.277127] nix[1237]: error: interrupted by the user5553client # [ 132.312560] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5554client # [ 132.317509] systemd[1]: intr.service: Failed with result 'exit-code'.5555client # [ 132.319847] systemd[1]: intr.service: Consumed 168ms CPU time over 3.798s wall clock time, 41.2M memory peak, 6.2K incoming IP traffic, 8K outgoing IP traffic.5556worker2 # [ 132.444496] nix-daemon[1773]: unexpected Nix daemon error: error: interrupted by the user5557worker2 # [ 132.448525] nix-daemon[856]: reaped child process 1771, status = succeeded5558worker2 # [ 132.456136] nix-daemon[856]: reaped child process 1773, status = failed with exit code 15559(finished: subtest: interrupting the client stops the build on the worker, in 3.98 seconds)5560subtest: a deploy mid-build drains: the build finishes, the new generation takes over5561client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5562client # [ 132.514016] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5563client # Running as unit: sw.service; invocation ID: 231b9a5f67f0402895064c1ba1d1a2285564client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5565client # [ 132.788068] nix[1257]: don't know how to build these paths:5566client # [ 132.789043] nix[1257]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5567worker1 # [ 132.835371] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:518225568worker2 # [ 132.810171] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:36Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05569worker2 # [ 132.817278] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:36Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15570worker1 # [ 132.898778] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518225571worker1 # [ 132.915556] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518225572client # [ 132.880154] nix[1257]: copying 1 paths to 'lb:50051'...5573worker1 # [ 132.931594] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518225574client # [ 132.885695] nix[1257]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5575worker1 # [ 132.946995] nix-daemon[835]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5576worker1 # [ 133.167169] nix-daemon[835]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5577worker1 # [ 133.189072] nix-daemon[835]: reaped child process 1300, status = succeeded5578worker1 # [ 133.366981] nix-daemon[835]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5579worker1 # [ 133.385447] nix-daemon[835]: reaped child process 1305, status = succeeded5580lb # [ 133.342878] niks3-server[1045]: 2026/09/20 21:10:36 INFO Received uploads request method=POST path=/api/pending_closures5581worker1 # [ 133.412185] nix-grpc-daemon[1253]: time=2026-09-20T21:10:36.987Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5582worker1 # [ 133.413672] nix-grpc-daemon[1253]: time=2026-09-20T21:10:36.987Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5583lb # [ 133.377921] niks3-server[1045]: 2026/09/20 21:10:36 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5584worker1 # [ 133.445534] nix-grpc-daemon[1253]: time=2026-09-20T21:10:37.021Z level=INFO msg="Uploading 1 narinfos"5585lb # [ 133.390547] niks3-server[1045]: 2026/09/20 21:10:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5586lb # [ 133.402264] niks3-server[1045]: 2026/09/20 21:10:36 INFO Signed narinfos id=38 count=15587lb # [ 133.412162] niks3-server[1045]: 2026/09/20 21:10:36 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5588worker1 # [ 133.474993] nix-grpc-daemon[1253]: time=2026-09-20T21:10:37.050Z level=INFO msg="Upload complete. (510ms)"5589lb # [ 133.417789] niks3-server[1045]: 2026/09/20 21:10:36 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5590lb # [ 133.422365] niks3-server[1045]: 2026/09/20 21:10:36 INFO Completed upload id=385591lb # [ 133.425566] niks3-server[1045]: 2026/09/20 21:10:36 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5592worker1 # [ 133.483939] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51822 duration_s=0 paths=1 nar_bytes_in=4165593worker1 # [ 133.495466] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518225594worker1 # [ 133.502994] nix-daemon[835]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5595worker1 # [ 133.508717] nix-daemon[835]: reaped child process 1295, status = succeeded5596client # [ 133.470082] nix[1257]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5597worker1 # [ 133.524265] nix-daemon[835]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5598worker1: must succeed: systemctl reload nix-grpc-daemon.service5599worker1 # [ 135.293616] systemd[1]: Reloading Nix worker-protocol over gRPC...5600worker1 # [ 135.340973] systemd[1]: Reloaded Nix worker-protocol over gRPC.5601worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.11 seconds)5602worker1: must succeed: pgrep -f 'read -t [9]0 x'5603worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5604client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25605worker1 # [ 135.535994] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:39Z level=info event=draining5606client # don't know how to build these paths:5607client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5608worker1 # [ 135.676747] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518225609worker2 # [ 135.636799] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409245610worker2 # [ 135.663685] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:39Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05611worker2 # [ 135.671192] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:39Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15612worker2 # [ 135.714498] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409245613worker2 # [ 135.731209] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409245614client # copying 1 paths to 'lb:50051'...5615worker2 # [ 135.749148] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409245616client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5617worker2 # [ 135.774153] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5618worker2 # [ 135.987098] nix-daemon[856]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)5619worker2 # [ 136.010165] nix-daemon[856]: reaped child process 1826, status = succeeded5620worker2 # [ 136.194637] nix-daemon[856]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)5621worker2 # [ 136.214891] nix-daemon[856]: reaped child process 1831, status = succeeded5622lb # [ 136.205459] niks3-server[1045]: 2026/09/20 21:10:39 INFO Received uploads request method=POST path=/api/pending_closures5623worker2 # [ 136.238799] nix-grpc-daemon[1744]: time=2026-09-20T21:10:39.824Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5624worker2 # [ 136.240509] nix-grpc-daemon[1744]: time=2026-09-20T21:10:39.824Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5625lb # [ 136.239568] niks3-server[1045]: 2026/09/20 21:10:39 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5626worker2 # [ 136.280871] nix-grpc-daemon[1744]: time=2026-09-20T21:10:39.867Z level=INFO msg="Uploading 1 narinfos"5627lb # [ 136.259099] niks3-server[1045]: 2026/09/20 21:10:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5628lb # [ 136.268296] niks3-server[1045]: 2026/09/20 21:10:39 INFO Signed narinfos id=39 count=15629lb # [ 136.277069] niks3-server[1045]: 2026/09/20 21:10:39 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5630lb # [ 136.287432] niks3-server[1045]: 2026/09/20 21:10:39 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5631lb # [ 136.297409] niks3-server[1045]: 2026/09/20 21:10:39 INFO Completed upload id=395632lb # [ 136.301575] niks3-server[1045]: 2026/09/20 21:10:39 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5633worker2 # [ 136.316781] nix-grpc-daemon[1744]: time=2026-09-20T21:10:39.903Z level=INFO msg="Upload complete. (524ms)"5634worker2 # [ 136.329679] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=0 paths=1 nar_bytes_in=5285635worker2 # [ 136.343454] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409245636worker2 # [ 136.350521] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5637worker2 # [ 136.355931] nix-daemon[856]: reaped child process 1821, status = succeeded5638client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5639client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5640worker2 # [ 136.371801] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5641client # farm-a> LOG-farm-a-during-drain5642worker2 # [ 137.200289] nix-daemon[856]: reaped child process 1835, status = succeeded5643worker2 # [ 137.372172] nix-daemon[856]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)5644worker2 # [ 137.393175] nix-daemon[856]: reaped child process 1853, status = succeeded5645lb # [ 137.383753] niks3-server[1045]: 2026/09/20 21:10:40 INFO Received uploads request method=POST path=/api/pending_closures5646worker2 # [ 137.419666] nix-grpc-daemon[1744]: time=2026-09-20T21:10:41.003Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5647worker2 # [ 137.422041] nix-grpc-daemon[1744]: time=2026-09-20T21:10:41.003Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5648lb # [ 137.450995] niks3-server[1045]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5649lb # [ 137.462105] niks3-server[1045]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5650lb # [ 137.471214] niks3-server[1045]: 2026/09/20 21:10:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5651lb # [ 137.476425] niks3-server[1045]: 2026/09/20 21:10:41 INFO Signed narinfos id=40 count=15652worker2 # [ 137.497870] nix-grpc-daemon[1744]: time=2026-09-20T21:10:41.083Z level=INFO msg="Uploading 1 narinfos"5653lb # [ 137.480294] niks3-server[1045]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5654lb # [ 137.490015] niks3-server[1045]: 2026/09/20 21:10:41 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5655lb # [ 137.497142] niks3-server[1045]: 2026/09/20 21:10:41 INFO Completed upload id=405656worker2 # [ 137.516444] nix-grpc-daemon[1744]: time=2026-09-20T21:10:41.103Z level=INFO msg="Upload complete. (333ms)"5657lb # [ 137.499383] niks3-server[1045]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5658worker2 # [ 137.535460] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15659worker2 # [ 137.552209] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:41Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05660worker2 # [ 137.555941] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:41Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15661worker2 # [ 137.559702] nix-daemon[856]: reaped child process 1833, status = succeeded5662worker2 # [ 137.588199] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409245663worker2 # [ 137.605176] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409245664worker2 # [ 137.620559] nix-grpc-daemon[1744]: time=2026-09-20T21:10:41.207Z level=INFO msg="All 1 paths already cached"5665client # copying 1 paths to 'lb:50051'...5666worker2 # [ 137.631682] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409245667client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5668worker2 # [ 137.655427] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5669worker2 # [ 137.871183] nix-daemon[856]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)5670worker2 # [ 137.893256] nix-daemon[856]: reaped child process 1862, status = succeeded5671worker2 # [ 138.072628] nix-daemon[856]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)5672worker2 # [ 138.094396] nix-daemon[856]: reaped child process 1867, status = succeeded5673lb # [ 138.083767] niks3-server[1045]: 2026/09/20 21:10:41 INFO Received uploads request method=POST path=/api/pending_closures5674worker2 # [ 138.114084] nix-grpc-daemon[1744]: time=2026-09-20T21:10:41.701Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5675worker2 # [ 138.116353] nix-grpc-daemon[1744]: time=2026-09-20T21:10:41.702Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5676lb # [ 138.113090] niks3-server[1045]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5677lb # [ 138.128493] niks3-server[1045]: 2026/09/20 21:10:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5678worker2 # [ 138.156080] nix-grpc-daemon[1744]: time=2026-09-20T21:10:41.742Z level=INFO msg="Uploading 1 narinfos"5679lb # [ 138.141019] niks3-server[1045]: 2026/09/20 21:10:41 INFO Signed narinfos id=41 count=15680lb # [ 138.148547] niks3-server[1045]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5681lb # [ 138.159466] niks3-server[1045]: 2026/09/20 21:10:41 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5682worker2 # [ 138.186564] nix-grpc-daemon[1744]: time=2026-09-20T21:10:41.773Z level=INFO msg="Upload complete. (512ms)"5683lb # [ 138.165832] niks3-server[1045]: 2026/09/20 21:10:41 INFO Completed upload id=415684lb # [ 138.170415] niks3-server[1045]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5685worker2 # [ 138.197466] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=0 paths=1 nar_bytes_in=6405686worker2 # [ 138.211878] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409245687worker2 # [ 138.218291] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5688worker2 # [ 138.223248] nix-daemon[856]: reaped child process 1857, status = succeeded5689client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5690worker2 # [ 138.236962] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5691client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5692client # farm-b> LOG-farm-b-during-drain5693worker2 # [ 139.080766] nix-daemon[856]: reaped child process 1871, status = succeeded5694worker2 # [ 139.259876] nix-daemon[856]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)5695worker2 # [ 139.280580] nix-daemon[856]: reaped child process 1889, status = succeeded5696lb # [ 139.270921] niks3-server[1045]: 2026/09/20 21:10:42 INFO Received uploads request method=POST path=/api/pending_closures5697worker2 # [ 139.303283] nix-grpc-daemon[1744]: time=2026-09-20T21:10:42.889Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5698worker2 # [ 139.305050] nix-grpc-daemon[1744]: time=2026-09-20T21:10:42.889Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5699lb # [ 139.309954] niks3-server[1045]: 2026/09/20 21:10:42 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5700lb # [ 139.322732] niks3-server[1045]: 2026/09/20 21:10:42 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5701lb # [ 139.332949] niks3-server[1045]: 2026/09/20 21:10:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5702worker2 # [ 139.361092] nix-grpc-daemon[1744]: time=2026-09-20T21:10:42.947Z level=INFO msg="Uploading 1 narinfos"5703lb # [ 139.339182] niks3-server[1045]: 2026/09/20 21:10:42 INFO Signed narinfos id=42 count=15704lb # [ 139.343960] niks3-server[1045]: 2026/09/20 21:10:42 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5705lb # [ 139.357556] niks3-server[1045]: 2026/09/20 21:10:42 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5706lb # [ 139.363774] niks3-server[1045]: 2026/09/20 21:10:42 INFO Completed upload id=425707worker2 # [ 139.384510] nix-grpc-daemon[1744]: time=2026-09-20T21:10:42.971Z level=INFO msg="Upload complete. (334ms)"5708lb # [ 139.366922] niks3-server[1045]: 2026/09/20 21:10:42 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5709worker2 # [ 139.403153] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15710worker2 # [ 139.418961] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:42Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05711worker2 # [ 139.422939] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:42Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15712worker2 # [ 139.426727] nix-daemon[856]: reaped child process 1869, status = succeeded5713worker2 # [ 139.450662] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409245714worker2 # [ 139.473558] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409245715worker2 # [ 139.489840] nix-grpc-daemon[1744]: time=2026-09-20T21:10:43.076Z level=INFO msg="All 2 paths already cached"5716client # copying 1 paths to 'lb:50051'...5717worker2 # [ 139.501374] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409245718client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5719worker2 # [ 139.525007] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5720worker2 # [ 139.743834] nix-daemon[856]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)5721worker2 # [ 139.764203] nix-daemon[856]: reaped child process 1898, status = succeeded5722worker2 # [ 139.941987] nix-daemon[856]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)5723worker2 # [ 139.962480] nix-daemon[856]: reaped child process 1903, status = succeeded5724lb # [ 139.952383] niks3-server[1045]: 2026/09/20 21:10:43 INFO Received uploads request method=POST path=/api/pending_closures5725worker2 # [ 139.984067] nix-grpc-daemon[1744]: time=2026-09-20T21:10:43.570Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5726worker2 # [ 139.985724] nix-grpc-daemon[1744]: time=2026-09-20T21:10:43.570Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5727lb # [ 139.982759] niks3-server[1045]: 2026/09/20 21:10:43 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5728worker2 # [ 140.018860] nix-grpc-daemon[1744]: time=2026-09-20T21:10:43.605Z level=INFO msg="Uploading 1 narinfos"5729lb # [ 139.998258] niks3-server[1045]: 2026/09/20 21:10:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5730lb # [ 140.009474] niks3-server[1045]: 2026/09/20 21:10:43 INFO Signed narinfos id=43 count=15731lb # [ 140.016982] niks3-server[1045]: 2026/09/20 21:10:43 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5732lb # [ 140.024491] niks3-server[1045]: 2026/09/20 21:10:43 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5733worker2 # [ 140.051969] nix-grpc-daemon[1744]: time=2026-09-20T21:10:43.638Z level=INFO msg="Upload complete. (508ms)"5734lb # [ 140.033483] niks3-server[1045]: 2026/09/20 21:10:43 INFO Completed upload id=435735lb # [ 140.037757] niks3-server[1045]: 2026/09/20 21:10:43 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5736worker2 # [ 140.058901] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=0 paths=1 nar_bytes_in=7685737worker2 # [ 140.074419] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409245738worker2 # [ 140.081254] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5739worker2 # [ 140.086901] nix-daemon[856]: reaped child process 1893, status = succeeded5740client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5741client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5742worker2 # [ 140.102262] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)5743client # farm-top> LOG-farm-top-during-drain5744worker2 # [ 141.351674] nix-daemon[856]: reaped child process 1907, status = succeeded5745worker2 # [ 141.544570] nix-daemon[856]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)5746worker2 # [ 141.562646] nix-daemon[856]: reaped child process 1925, status = succeeded5747lb # [ 141.558492] niks3-server[1045]: 2026/09/20 21:10:45 INFO Received uploads request method=POST path=/api/pending_closures5748worker2 # [ 141.590580] nix-grpc-daemon[1744]: time=2026-09-20T21:10:45.175Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5749worker2 # [ 141.593561] nix-grpc-daemon[1744]: time=2026-09-20T21:10:45.175Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5750lb # [ 141.605210] niks3-server[1045]: 2026/09/20 21:10:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5751worker2 # [ 141.636041] nix-grpc-daemon[1744]: time=2026-09-20T21:10:45.222Z level=INFO msg="Uploading 1 narinfos"5752lb # [ 141.614980] niks3-server[1045]: 2026/09/20 21:10:45 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5753lb # [ 141.621637] niks3-server[1045]: 2026/09/20 21:10:45 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5754lb # [ 141.630030] niks3-server[1045]: 2026/09/20 21:10:45 INFO Signed narinfos id=44 count=15755lb # [ 141.633649] niks3-server[1045]: 2026/09/20 21:10:45 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5756lb # [ 141.640701] niks3-server[1045]: 2026/09/20 21:10:45 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5757lb # [ 141.647933] niks3-server[1045]: 2026/09/20 21:10:45 INFO Completed upload id=445758worker2 # [ 141.668844] nix-grpc-daemon[1744]: time=2026-09-20T21:10:45.255Z level=INFO msg="Upload complete. (354ms)"5759lb # [ 141.651127] niks3-server[1045]: 2026/09/20 21:10:45 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5760worker2 # [ 141.680639] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15761lb # [ 141.666652] envoy[926]: conn peer=[2001:db8:1::1]:47166 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2931 tx=733 ms=35235762lb # [ 141.670873] envoy[926]: conn peer=[2001:db8:1::1]:46796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9569 tx=3877 ms=60625763worker2 # [ 141.696954] nix-daemon[856]: reaped child process 1905, status = succeeded5764client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.37 seconds)5765client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5766client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.24 seconds)5767worker1: must succeed: pkill -f 'read -t [9]0 x'5768worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5769client: waiting for success: ! systemctl is-active sw5770worker1 # [ 142.158954] nix-daemon[835]: reaped child process 1309, status = succeeded5771worker1 # [ 142.323345] nix-daemon[835]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5772worker1 # [ 142.343891] nix-daemon[835]: reaped child process 1353, status = succeeded5773lb # [ 142.297394] niks3-server[1045]: 2026/09/20 21:10:45 INFO Received uploads request method=POST path=/api/pending_closures5774worker1 # [ 142.365962] nix-grpc-daemon[1253]: time=2026-09-20T21:10:45.941Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5775worker1 # [ 142.368387] nix-grpc-daemon[1253]: time=2026-09-20T21:10:45.943Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5776lb # [ 142.339910] niks3-server[1045]: 2026/09/20 21:10:45 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5777lb # [ 142.349572] niks3-server[1045]: 2026/09/20 21:10:45 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5778worker1 # [ 142.419126] nix-grpc-daemon[1253]: time=2026-09-20T21:10:45.994Z level=INFO msg="Uploading 1 narinfos"5779lb # [ 142.363376] niks3-server[1045]: 2026/09/20 21:10:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5780lb # [ 142.370833] niks3-server[1045]: 2026/09/20 21:10:45 INFO Signed narinfos id=45 count=15781lb # [ 142.373797] niks3-server[1045]: 2026/09/20 21:10:45 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5782lb # [ 142.379860] niks3-server[1045]: 2026/09/20 21:10:45 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5783lb # [ 142.384254] niks3-server[1045]: 2026/09/20 21:10:45 INFO Completed upload id=455784worker1 # [ 142.441541] nix-grpc-daemon[1253]: time=2026-09-20T21:10:46.017Z level=INFO msg="Upload complete. (299ms)"5785lb # [ 142.386834] niks3-server[1045]: 2026/09/20 21:10:45 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5786worker1 # [ 142.453012] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51822 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15787worker1 # [ 142.468794] nix-daemon[835]: reaped child process 1307, status = succeeded5788client # [ 142.446342] systemd[1]: sw.service: Deactivated successfully.5789client # [ 142.450926] systemd[1]: sw.service: Consumed 189ms CPU time over 9.935s wall clock time, 41.2M memory peak, 6.7K incoming IP traffic, 8.1K outgoing IP traffic.5790worker1 # [ 142.537305] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:46Z level=info event=signal_exit5791client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds)5792client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5793client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5794worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5795worker1 # [ 143.777670] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:47Z level=info event=scheduler_restarting5796worker2 # [ 143.740664] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:409025797worker2 # [ 143.741045] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:47Z level=info event=worker_gone addr=192.168.1.3:500515798worker1 # [ 143.802487] nix-grpc-daemon[1102]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5799worker1 # [ 143.809015] nix-grpc-daemon[1102]: E0000 00:00:1789938647.378026 1145 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5800worker1 # [ 143.816420] nix-grpc-daemon[1102]: E0000 00:00:1789938647.378217 1145 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5801worker1 # [ 143.823770] nix-grpc-daemon[1102]: ts=2026-09-20T21:10:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5802worker1 # [ 143.846272] nix-daemon[835]: reaped child process 1205, status = succeeded5803worker1 # [ 144.044947] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5804worker1 # [ 144.047950] systemd[1]: nix-grpc-daemon.service: Consumed 983ms CPU time over 40.746s wall clock time, 33M memory peak, 23.9K incoming IP traffic, 34K outgoing IP traffic.5805worker1 # [ 144.071495] systemd[1]: Starting Nix worker-protocol over gRPC...5806worker1 # [ 144.276593] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5807worker1 # [ 144.278670] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5808worker1 # [ 144.282970] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15809worker1 # [ 144.290735] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5810worker1 # [ 144.298679] systemd[1]: Started Nix worker-protocol over gRPC.5811worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.04 seconds)5812lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5813lb: must succeed: curl -sf localhost:9901/clusters5814worker1 # [ 144.337848] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:47Z level=info event=scheduler_connected addr=lb:500515815worker2 # [ 144.303952] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:409025816worker2 # [ 144.306738] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05817lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5818lb: must succeed: curl -sf localhost:9901/clusters5819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5820lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5821lb: must succeed: curl -sf localhost:9901/clusters5822lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5823worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5824worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5825(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.13 seconds)5826subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5827client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15828worker1 # [ 145.905296] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427625829worker1 # [ 145.920873] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5830worker2 # [ 145.907322] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409245831worker2 # [ 145.954198] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:49Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05832worker2 # [ 145.960367] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:49Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15833worker2 # [ 145.966268] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:49Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05834worker2 # [ 145.971959] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:49Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05835worker1 # [ 146.038075] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427625836worker1 # [ 146.058498] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 1/5)5837worker1 # [ 146.093104] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 2/5)5838worker1 # [ 146.198155] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 3/5)5839worker1 # [ 146.304767] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 775 ms (attempt 4/5)5840worker1 # [ 147.081915] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5841worker1 # [ 147.104874] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427625842worker1 # [ 147.120148] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427625843worker1 # [ 147.137025] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5844worker1 # [ 147.166770] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:50Z level=info event=niks3_push_started pid=14235845worker1 # [ 147.379908] nix-grpc-daemon[1423]: time=2026-09-20T21:10:50.955Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5846worker1 # [ 147.559267] nix-daemon[835]: accepted connection from pid 1436, user nix-grpc-daemon (trusted)5847worker1 # [ 147.576722] nix-daemon[835]: reaped child process 1439, status = succeeded5848worker1 # [ 147.760208] nix-daemon[835]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5849worker1 # [ 147.777606] nix-daemon[835]: reaped child process 1444, status = succeeded5850lb # [ 147.735688] niks3-server[1045]: 2026/09/20 21:10:51 INFO Received uploads request method=POST path=/api/pending_closures5851worker1 # [ 147.806013] nix-grpc-daemon[1423]: time=2026-09-20T21:10:51.380Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5852worker1 # [ 147.807754] nix-grpc-daemon[1423]: time=2026-09-20T21:10:51.380Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5853lb # [ 147.773797] niks3-server[1045]: 2026/09/20 21:10:51 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5854worker1 # [ 147.846526] nix-grpc-daemon[1423]: time=2026-09-20T21:10:51.422Z level=INFO msg="Uploading 1 narinfos"5855lb # [ 147.788671] niks3-server[1045]: 2026/09/20 21:10:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5856lb # [ 147.800267] niks3-server[1045]: 2026/09/20 21:10:51 INFO Signed narinfos id=46 count=15857lb # [ 147.807633] niks3-server[1045]: 2026/09/20 21:10:51 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5858lb # [ 147.822451] niks3-server[1045]: 2026/09/20 21:10:51 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5859worker1 # [ 147.886023] nix-grpc-daemon[1423]: time=2026-09-20T21:10:51.461Z level=INFO msg="Upload complete. (502ms)"5860lb # [ 147.830285] niks3-server[1045]: 2026/09/20 21:10:51 INFO Completed upload id=465861lb # [ 147.833116] niks3-server[1045]: 2026/09/20 21:10:51 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5862worker1 # [ 147.896388] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=0 paths=1 nar_bytes_in=5125863worker1 # [ 147.908907] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427625864worker1 # [ 147.920762] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5865worker1 # [ 147.926016] nix-daemon[835]: reaped child process 1421, status = succeeded5866worker1 # [ 147.939715] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5867worker1 # [ 149.191545] nix-daemon[835]: reaped child process 1448, status = succeeded5868worker1 # [ 149.379324] nix-daemon[835]: accepted connection from pid 1463, user nix-grpc-daemon (trusted)5869worker1 # [ 149.399630] nix-daemon[835]: reaped child process 1466, status = succeeded5870lb # [ 149.355066] niks3-server[1045]: 2026/09/20 21:10:52 INFO Received uploads request method=POST path=/api/pending_closures5871worker1 # [ 149.424425] nix-grpc-daemon[1423]: time=2026-09-20T21:10:52.998Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5872worker1 # [ 149.425939] nix-grpc-daemon[1423]: time=2026-09-20T21:10:52.998Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5873lb # [ 149.402824] niks3-server[1045]: 2026/09/20 21:10:52 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5874lb # [ 149.419479] niks3-server[1045]: 2026/09/20 21:10:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5875worker1 # [ 149.485155] nix-grpc-daemon[1423]: time=2026-09-20T21:10:53.059Z level=INFO msg="Uploading 1 narinfos"5876lb # [ 149.430527] niks3-server[1045]: 2026/09/20 21:10:52 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5877lb # [ 149.439331] niks3-server[1045]: 2026/09/20 21:10:52 INFO Signed narinfos id=47 count=15878lb # [ 149.443888] niks3-server[1045]: 2026/09/20 21:10:53 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5879worker1 # [ 149.511007] nix-grpc-daemon[1423]: time=2026-09-20T21:10:53.086Z level=INFO msg="Upload complete. (359ms)"5880lb # [ 149.453859] niks3-server[1045]: 2026/09/20 21:10:53 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5881lb # [ 149.458145] niks3-server[1045]: 2026/09/20 21:10:53 INFO Completed upload id=475882lb # [ 149.461724] niks3-server[1045]: 2026/09/20 21:10:53 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5883worker2 # [ 149.485325] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:53Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15884worker1 # [ 149.525079] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=15885worker2 # [ 149.499628] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:53Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05886worker1 # [ 149.541423] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427625887worker1 # [ 149.547479] nix-daemon[835]: reaped child process 1446, status = succeeded5888worker1 # [ 149.555886] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427625889worker1 # [ 149.569281] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427625890worker1 # [ 149.582121] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5891worker1 # [ 149.777167] nix-daemon[835]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)5892worker1 # [ 149.796777] nix-daemon[835]: reaped child process 1473, status = succeeded5893worker1 # [ 149.972385] nix-daemon[835]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5894worker1 # [ 149.990424] nix-daemon[835]: reaped child process 1478, status = succeeded5895lb # [ 149.945630] niks3-server[1045]: 2026/09/20 21:10:53 INFO Received uploads request method=POST path=/api/pending_closures5896worker1 # [ 150.012061] nix-grpc-daemon[1423]: time=2026-09-20T21:10:53.588Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5897worker1 # [ 150.014173] nix-grpc-daemon[1423]: time=2026-09-20T21:10:53.589Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5898lb # [ 149.976151] niks3-server[1045]: 2026/09/20 21:10:53 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5899worker1 # [ 150.048132] nix-grpc-daemon[1423]: time=2026-09-20T21:10:53.623Z level=INFO msg="Uploading 1 narinfos"5900lb # [ 149.992315] niks3-server[1045]: 2026/09/20 21:10:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5901lb # [ 150.005052] niks3-server[1045]: 2026/09/20 21:10:53 INFO Signed narinfos id=48 count=15902lb # [ 150.012089] niks3-server[1045]: 2026/09/20 21:10:53 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5903lb # [ 150.021009] niks3-server[1045]: 2026/09/20 21:10:53 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5904worker1 # [ 150.083169] nix-grpc-daemon[1423]: time=2026-09-20T21:10:53.659Z level=INFO msg="Upload complete. (487ms)"5905lb # [ 150.026113] niks3-server[1045]: 2026/09/20 21:10:53 INFO Completed upload id=485906lb # [ 150.030150] niks3-server[1045]: 2026/09/20 21:10:53 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5907worker1 # [ 150.092290] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=0 paths=1 nar_bytes_in=5125908worker1 # [ 150.104831] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427625909worker1 # [ 150.111055] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5910worker1 # [ 150.116578] nix-daemon[835]: reaped child process 1468, status = succeeded5911worker1 # [ 150.131009] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5912worker1 # [ 151.184671] nix-daemon[835]: reaped child process 1482, status = succeeded5913worker1 # [ 151.366325] nix-daemon[835]: accepted connection from pid 1497, user nix-grpc-daemon (trusted)5914worker1 # [ 151.384534] nix-daemon[835]: reaped child process 1500, status = succeeded5915lb # [ 151.339101] niks3-server[1045]: 2026/09/20 21:10:54 INFO Received uploads request method=POST path=/api/pending_closures5916worker1 # [ 151.407556] nix-grpc-daemon[1423]: time=2026-09-20T21:10:54.983Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5917worker1 # [ 151.410079] nix-grpc-daemon[1423]: time=2026-09-20T21:10:54.985Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5918lb # [ 151.385156] niks3-server[1045]: 2026/09/20 21:10:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5919worker1 # [ 151.454629] nix-grpc-daemon[1423]: time=2026-09-20T21:10:55.030Z level=INFO msg="Uploading 1 narinfos"5920lb # [ 151.402589] niks3-server[1045]: 2026/09/20 21:10:54 INFO Signed narinfos id=49 count=15921lb # [ 151.411416] niks3-server[1045]: 2026/09/20 21:10:54 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5922lb # [ 151.425858] niks3-server[1045]: 2026/09/20 21:10:54 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5923worker1 # [ 151.488342] nix-grpc-daemon[1423]: time=2026-09-20T21:10:55.064Z level=INFO msg="Upload complete. (345ms)"5924lb # [ 151.431383] niks3-server[1045]: 2026/09/20 21:10:54 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5925lb # [ 151.435253] niks3-server[1045]: 2026/09/20 21:10:54 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5926lb # [ 151.439791] niks3-server[1045]: 2026/09/20 21:10:54 INFO Completed upload id=495927lb # [ 151.442786] niks3-server[1045]: 2026/09/20 21:10:54 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5928worker2 # [ 151.463547] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:55Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15929worker1 # [ 151.502059] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=15930worker2 # [ 151.477339] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:55Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05931worker1 # [ 151.523291] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427625932worker1 # [ 151.529672] nix-daemon[835]: reaped child process 1480, status = succeeded5933worker1 # [ 151.533333] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427625934worker1 # [ 151.545807] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427625935worker1 # [ 151.559481] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5936lb # [ 151.667123] envoy[926]: conn peer=[2001:db8:1::1]:46782 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2935 tx=1309 ms=96325937lb # [ 151.672405] envoy[926]: conn peer=[2001:db8:1::3]:60328 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2527 tx=862 ms=402325938worker1 # [ 151.753990] nix-daemon[835]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5939worker1 # [ 151.773029] nix-daemon[835]: reaped child process 1507, status = succeeded5940worker1 # [ 151.954376] nix-daemon[835]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)5941worker1 # [ 151.973130] nix-daemon[835]: reaped child process 1512, status = succeeded5942lb # [ 151.926729] niks3-server[1045]: 2026/09/20 21:10:55 INFO Received uploads request method=POST path=/api/pending_closures5943worker1 # [ 151.996965] nix-grpc-daemon[1423]: time=2026-09-20T21:10:55.572Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5944worker1 # [ 151.998473] nix-grpc-daemon[1423]: time=2026-09-20T21:10:55.572Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"5945lb # [ 151.957522] niks3-server[1045]: 2026/09/20 21:10:55 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst5946lb # [ 151.973725] niks3-server[1045]: 2026/09/20 21:10:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5947worker1 # [ 152.038207] nix-grpc-daemon[1423]: time=2026-09-20T21:10:55.613Z level=INFO msg="Uploading 1 narinfos"5948lb # [ 151.987304] niks3-server[1045]: 2026/09/20 21:10:55 INFO Signed narinfos id=50 count=15949lb # [ 151.994904] niks3-server[1045]: 2026/09/20 21:10:55 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls5950lb # [ 152.005836] niks3-server[1045]: 2026/09/20 21:10:55 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5951lb # [ 152.010591] niks3-server[1045]: 2026/09/20 21:10:55 INFO Completed upload id=505952worker1 # [ 152.068002] nix-grpc-daemon[1423]: time=2026-09-20T21:10:55.643Z level=INFO msg="Upload complete. (495ms)"5953lb # [ 152.013438] niks3-server[1045]: 2026/09/20 21:10:55 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo5954worker1 # [ 152.076368] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=0 paths=1 nar_bytes_in=5125955worker1 # [ 152.090638] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427625956worker1 # [ 152.098354] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5957worker1 # [ 152.103442] nix-daemon[835]: reaped child process 1502, status = succeeded5958worker1 # [ 152.118262] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5959worker1 # [ 153.163087] nix-daemon[835]: reaped child process 1516, status = succeeded5960worker1 # [ 153.354856] nix-daemon[835]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5961worker1 # [ 153.372914] nix-daemon[835]: reaped child process 1534, status = succeeded5962lb # [ 153.327972] niks3-server[1045]: 2026/09/20 21:10:56 INFO Received uploads request method=POST path=/api/pending_closures5963worker1 # [ 153.399888] nix-grpc-daemon[1423]: time=2026-09-20T21:10:56.973Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5964worker1 # [ 153.401815] nix-grpc-daemon[1423]: time=2026-09-20T21:10:56.973Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"5965lb # [ 153.366734] niks3-server[1045]: 2026/09/20 21:10:56 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst5966lb # [ 153.381823] niks3-server[1045]: 2026/09/20 21:10:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5967worker1 # [ 153.444994] nix-grpc-daemon[1423]: time=2026-09-20T21:10:57.020Z level=INFO msg="Uploading 1 narinfos"5968lb # [ 153.392875] niks3-server[1045]: 2026/09/20 21:10:56 INFO Signed narinfos id=51 count=15969lb # [ 153.394960] niks3-server[1045]: 2026/09/20 21:10:56 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv5970worker1 # [ 153.464000] nix-grpc-daemon[1423]: time=2026-09-20T21:10:57.039Z level=INFO msg="Upload complete. (335ms)"5971lb # [ 153.407357] niks3-server[1045]: 2026/09/20 21:10:56 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls5972lb # [ 153.410070] niks3-server[1045]: 2026/09/20 21:10:56 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5973lb # [ 153.414708] niks3-server[1045]: 2026/09/20 21:10:56 INFO Completed upload id=515974lb # [ 153.416776] niks3-server[1045]: 2026/09/20 21:10:56 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo5975worker2 # [ 153.437276] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:57Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=15976worker1 # [ 153.476926] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=15977worker2 # [ 153.451074] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:57Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=05978worker1 # [ 153.492165] nix-daemon[835]: reaped child process 1514, status = succeeded5979worker1 # [ 153.530685] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427625980worker1 # [ 153.550032] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427625981worker1 # [ 153.566538] nix-grpc-daemon[1423]: time=2026-09-20T21:10:57.142Z level=INFO msg="All 1 paths already cached"5982worker1 # [ 153.576097] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427625983worker1 # [ 153.590571] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5984worker1 # [ 153.800779] nix-daemon[835]: accepted connection from pid 1538, user nix-grpc-daemon (trusted)5985worker1 # [ 153.819092] nix-daemon[835]: reaped child process 1541, status = succeeded5986worker1 # [ 154.000602] nix-daemon[835]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5987worker1 # [ 154.019652] nix-daemon[835]: reaped child process 1546, status = succeeded5988lb # [ 153.974808] niks3-server[1045]: 2026/09/20 21:10:57 INFO Received uploads request method=POST path=/api/pending_closures5989worker1 # [ 154.045176] nix-grpc-daemon[1423]: time=2026-09-20T21:10:57.620Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5990worker1 # [ 154.046675] nix-grpc-daemon[1423]: time=2026-09-20T21:10:57.620Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"5991lb # [ 154.008164] niks3-server[1045]: 2026/09/20 21:10:57 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst5992worker1 # [ 154.073561] nix-grpc-daemon[1423]: time=2026-09-20T21:10:57.649Z level=INFO msg="Uploading 1 narinfos"5993lb # [ 154.017052] niks3-server[1045]: 2026/09/20 21:10:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5994lb # [ 154.019828] niks3-server[1045]: 2026/09/20 21:10:57 INFO Signed narinfos id=52 count=15995lb # [ 154.028669] niks3-server[1045]: 2026/09/20 21:10:57 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls5996lb # [ 154.037327] niks3-server[1045]: 2026/09/20 21:10:57 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5997lb # [ 154.042130] niks3-server[1045]: 2026/09/20 21:10:57 INFO Completed upload id=525998worker1 # [ 154.099440] nix-grpc-daemon[1423]: time=2026-09-20T21:10:57.675Z level=INFO msg="Upload complete. (490ms)"5999lb # [ 154.044158] niks3-server[1045]: 2026/09/20 21:10:57 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6000worker1 # [ 154.106946] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=0 paths=1 nar_bytes_in=6246001worker1 # [ 154.118611] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626002worker1 # [ 154.123757] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6003worker1 # [ 154.128451] nix-daemon[835]: reaped child process 1536, status = succeeded6004worker1 # [ 154.141285] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6005worker1 # [ 155.362761] nix-daemon[835]: reaped child process 1550, status = succeeded6006worker1 # [ 155.525578] nix-daemon[835]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)6007worker1 # [ 155.543336] nix-daemon[835]: reaped child process 1568, status = succeeded6008lb # [ 155.498108] niks3-server[1045]: 2026/09/20 21:10:59 INFO Received uploads request method=POST path=/api/pending_closures6009worker1 # [ 155.568047] nix-grpc-daemon[1423]: time=2026-09-20T21:10:59.141Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6010worker1 # [ 155.569513] nix-grpc-daemon[1423]: time=2026-09-20T21:10:59.141Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6011lb # [ 155.541586] niks3-server[1045]: 2026/09/20 21:10:59 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6012worker1 # [ 155.611404] nix-grpc-daemon[1423]: time=2026-09-20T21:10:59.187Z level=INFO msg="Uploading 1 narinfos"6013lb # [ 155.556236] niks3-server[1045]: 2026/09/20 21:10:59 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6014lb # [ 155.566797] niks3-server[1045]: 2026/09/20 21:10:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6015lb # [ 155.571293] niks3-server[1045]: 2026/09/20 21:10:59 INFO Signed narinfos id=53 count=16016lb # [ 155.578182] niks3-server[1045]: 2026/09/20 21:10:59 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6017lb # [ 155.582926] niks3-server[1045]: 2026/09/20 21:10:59 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6018worker1 # [ 155.646377] nix-grpc-daemon[1423]: time=2026-09-20T21:10:59.222Z level=INFO msg="Upload complete. (295ms)"6019lb # [ 155.590554] niks3-server[1045]: 2026/09/20 21:10:59 INFO Completed upload id=536020lb # [ 155.594772] niks3-server[1045]: 2026/09/20 21:10:59 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6021worker2 # [ 155.618197] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:59Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16022worker1 # [ 155.656935] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16023worker2 # [ 155.631379] nix-grpc-daemon[1672]: ts=2026-09-20T21:10:59Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06024worker1 # [ 155.673931] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626025worker1 # [ 155.679374] nix-daemon[835]: reaped child process 1548, status = succeeded6026worker1 # [ 155.686944] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427626027worker1 # [ 155.698635] nix-grpc-daemon[1423]: time=2026-09-20T21:10:59.274Z level=INFO msg="All 1 paths already cached"6028worker1 # [ 155.706321] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427626029worker1 # [ 155.717590] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6030worker1 # [ 155.906697] nix-daemon[835]: accepted connection from pid 1572, user nix-grpc-daemon (trusted)6031worker1 # [ 155.924370] nix-daemon[835]: reaped child process 1575, status = succeeded6032worker1 # [ 156.098784] nix-daemon[835]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6033worker1 # [ 156.115871] nix-daemon[835]: reaped child process 1580, status = succeeded6034lb # [ 156.069790] niks3-server[1045]: 2026/09/20 21:10:59 INFO Received uploads request method=POST path=/api/pending_closures6035worker1 # [ 156.136669] nix-grpc-daemon[1423]: time=2026-09-20T21:10:59.712Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6036worker1 # [ 156.138154] nix-grpc-daemon[1423]: time=2026-09-20T21:10:59.712Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6037lb # [ 156.099185] niks3-server[1045]: 2026/09/20 21:10:59 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6038lb # [ 156.115132] niks3-server[1045]: 2026/09/20 21:10:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6039worker1 # [ 156.179043] nix-grpc-daemon[1423]: time=2026-09-20T21:10:59.754Z level=INFO msg="Uploading 1 narinfos"6040lb # [ 156.127641] niks3-server[1045]: 2026/09/20 21:10:59 INFO Signed narinfos id=54 count=16041lb # [ 156.136098] niks3-server[1045]: 2026/09/20 21:10:59 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6042lb # [ 156.142642] niks3-server[1045]: 2026/09/20 21:10:59 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6043lb # [ 156.148345] niks3-server[1045]: 2026/09/20 21:10:59 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6044lb # [ 156.153018] niks3-server[1045]: 2026/09/20 21:10:59 INFO Completed upload id=546045worker1 # [ 156.210875] nix-grpc-daemon[1423]: time=2026-09-20T21:10:59.786Z level=INFO msg="Upload complete. (480ms)"6046worker1 # [ 156.219117] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=0 paths=1 nar_bytes_in=6246047worker1 # [ 156.233663] nix-grpc-daemon[1367]: ts=2026-09-20T21:10:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626048worker1 # [ 156.240502] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6049worker1 # [ 156.246010] nix-daemon[835]: reaped child process 1570, status = succeeded6050worker1 # [ 156.258314] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6051worker1 # [ 157.687411] nix-daemon[835]: reaped child process 1584, status = succeeded6052worker1 # [ 157.847233] nix-daemon[835]: accepted connection from pid 1599, user nix-grpc-daemon (trusted)6053worker1 # [ 157.865780] nix-daemon[835]: reaped child process 1602, status = succeeded6054lb # [ 157.823144] niks3-server[1045]: 2026/09/20 21:11:01 INFO Received uploads request method=POST path=/api/pending_closures6055worker1 # [ 157.894958] nix-grpc-daemon[1423]: time=2026-09-20T21:11:01.467Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6056worker1 # [ 157.896551] nix-grpc-daemon[1423]: time=2026-09-20T21:11:01.467Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6057lb # [ 157.863764] niks3-server[1045]: 2026/09/20 21:11:01 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6058lb # [ 157.874323] niks3-server[1045]: 2026/09/20 21:11:01 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6059lb # [ 157.888549] niks3-server[1045]: 2026/09/20 21:11:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6060worker1 # [ 157.952520] nix-grpc-daemon[1423]: time=2026-09-20T21:11:01.528Z level=INFO msg="Uploading 1 narinfos"6061lb # [ 157.899173] niks3-server[1045]: 2026/09/20 21:11:01 INFO Signed narinfos id=55 count=16062lb # [ 157.904665] niks3-server[1045]: 2026/09/20 21:11:01 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6063lb # [ 157.912482] niks3-server[1045]: 2026/09/20 21:11:01 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6064lb # [ 157.918227] niks3-server[1045]: 2026/09/20 21:11:01 INFO Completed upload id=556065worker1 # [ 157.976104] nix-grpc-daemon[1423]: time=2026-09-20T21:11:01.552Z level=INFO msg="Upload complete. (303ms)"6066lb # [ 157.920744] niks3-server[1045]: 2026/09/20 21:11:01 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6067worker2 # [ 157.951872] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:01Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16068worker1 # [ 157.988616] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16069worker2 # [ 157.961077] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:01Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06070worker1 # [ 158.005628] nix-daemon[835]: reaped child process 1582, status = succeeded6071worker1 # [ 158.034195] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626072worker1 # [ 158.049318] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427626073worker1 # [ 158.063859] nix-grpc-daemon[1423]: time=2026-09-20T21:11:01.639Z level=INFO msg="All 1 paths already cached"6074worker1 # [ 158.072904] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427626075worker1 # [ 158.087195] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6076worker1 # [ 158.288054] nix-daemon[835]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6077worker1 # [ 158.305846] nix-daemon[835]: reaped child process 1609, status = succeeded6078worker1 # [ 158.476917] nix-daemon[835]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6079worker1 # [ 158.494032] nix-daemon[835]: reaped child process 1614, status = succeeded6080lb # [ 158.448694] niks3-server[1045]: 2026/09/20 21:11:02 INFO Received uploads request method=POST path=/api/pending_closures6081worker1 # [ 158.515534] nix-grpc-daemon[1423]: time=2026-09-20T21:11:02.091Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6082worker1 # [ 158.517146] nix-grpc-daemon[1423]: time=2026-09-20T21:11:02.091Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6083lb # [ 158.478057] niks3-server[1045]: 2026/09/20 21:11:02 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6084lb # [ 158.491109] niks3-server[1045]: 2026/09/20 21:11:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6085worker1 # [ 158.554325] nix-grpc-daemon[1423]: time=2026-09-20T21:11:02.130Z level=INFO msg="Uploading 1 narinfos"6086lb # [ 158.502671] niks3-server[1045]: 2026/09/20 21:11:02 INFO Signed narinfos id=56 count=16087lb # [ 158.508604] niks3-server[1045]: 2026/09/20 21:11:02 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6088lb # [ 158.515233] niks3-server[1045]: 2026/09/20 21:11:02 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6089lb # [ 158.521631] niks3-server[1045]: 2026/09/20 21:11:02 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6090lb # [ 158.526914] niks3-server[1045]: 2026/09/20 21:11:02 INFO Completed upload id=566091worker1 # [ 158.584700] nix-grpc-daemon[1423]: time=2026-09-20T21:11:02.160Z level=INFO msg="Upload complete. (478ms)"6092worker1 # [ 158.592937] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=0 paths=1 nar_bytes_in=6246093worker1 # [ 158.605748] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626094worker1 # [ 158.611765] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6095worker1 # [ 158.616837] nix-daemon[835]: reaped child process 1604, status = succeeded6096worker1 # [ 158.630587] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6097worker1 # [ 159.477008] nix-daemon[835]: reaped child process 1618, status = succeeded6098worker1 # [ 159.661806] nix-daemon[835]: accepted connection from pid 1633, user nix-grpc-daemon (trusted)6099worker1 # [ 159.678565] nix-daemon[835]: reaped child process 1636, status = succeeded6100lb # [ 159.633260] niks3-server[1045]: 2026/09/20 21:11:03 INFO Received uploads request method=POST path=/api/pending_closures6101worker1 # [ 159.703472] nix-grpc-daemon[1423]: time=2026-09-20T21:11:03.276Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6102worker1 # [ 159.705343] nix-grpc-daemon[1423]: time=2026-09-20T21:11:03.276Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6103lb # [ 159.671418] niks3-server[1045]: 2026/09/20 21:11:03 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6104lb # [ 159.681584] niks3-server[1045]: 2026/09/20 21:11:03 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6105lb # [ 159.691357] niks3-server[1045]: 2026/09/20 21:11:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6106worker1 # [ 159.753811] nix-grpc-daemon[1423]: time=2026-09-20T21:11:03.329Z level=INFO msg="Uploading 1 narinfos"6107lb # [ 159.700592] niks3-server[1045]: 2026/09/20 21:11:03 INFO Signed narinfos id=57 count=16108lb # [ 159.702722] niks3-server[1045]: 2026/09/20 21:11:03 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6109lb # [ 159.710682] niks3-server[1045]: 2026/09/20 21:11:03 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6110lb # [ 159.716926] niks3-server[1045]: 2026/09/20 21:11:03 INFO Completed upload id=576111worker1 # [ 159.774649] nix-grpc-daemon[1423]: time=2026-09-20T21:11:03.350Z level=INFO msg="Upload complete. (333ms)"6112lb # [ 159.719380] niks3-server[1045]: 2026/09/20 21:11:03 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6113worker2 # [ 159.747497] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:03Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16114worker1 # [ 159.787332] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16115worker2 # [ 159.761447] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:03Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06116worker1 # [ 159.807462] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626117worker1 # [ 159.812945] nix-daemon[835]: reaped child process 1616, status = succeeded6118worker1 # [ 159.821121] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427626119worker1 # [ 159.834588] nix-grpc-daemon[1423]: time=2026-09-20T21:11:03.410Z level=INFO msg="All 2 paths already cached"6120worker1 # [ 159.843073] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427626121worker1 # [ 159.855906] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6122worker1 # [ 160.055358] nix-daemon[835]: accepted connection from pid 1640, user nix-grpc-daemon (trusted)6123worker1 # [ 160.075125] nix-daemon[835]: reaped child process 1643, status = succeeded6124worker1 # [ 160.248532] nix-daemon[835]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6125worker1 # [ 160.265266] nix-daemon[835]: reaped child process 1648, status = succeeded6126lb # [ 160.218625] niks3-server[1045]: 2026/09/20 21:11:03 INFO Received uploads request method=POST path=/api/pending_closures6127worker1 # [ 160.285359] nix-grpc-daemon[1423]: time=2026-09-20T21:11:03.861Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6128worker1 # [ 160.286811] nix-grpc-daemon[1423]: time=2026-09-20T21:11:03.861Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6129lb # [ 160.248652] niks3-server[1045]: 2026/09/20 21:11:03 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6130worker1 # [ 160.322290] nix-grpc-daemon[1423]: time=2026-09-20T21:11:03.898Z level=INFO msg="Uploading 1 narinfos"6131lb # [ 160.264769] niks3-server[1045]: 2026/09/20 21:11:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6132lb # [ 160.276929] niks3-server[1045]: 2026/09/20 21:11:03 INFO Signed narinfos id=58 count=16133lb # [ 160.283308] niks3-server[1045]: 2026/09/20 21:11:03 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6134worker1 # [ 160.349894] nix-grpc-daemon[1423]: time=2026-09-20T21:11:03.925Z level=INFO msg="Upload complete. (480ms)"6135lb # [ 160.294218] niks3-server[1045]: 2026/09/20 21:11:03 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6136worker1 # [ 160.356055] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=0 paths=1 nar_bytes_in=7526137lb # [ 160.300323] niks3-server[1045]: 2026/09/20 21:11:03 INFO Completed upload id=586138lb # [ 160.304153] niks3-server[1045]: 2026/09/20 21:11:03 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6139worker1 # [ 160.369678] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626140worker1 # [ 160.377998] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6141worker1 # [ 160.380732] nix-daemon[835]: reaped child process 1638, status = succeeded6142worker1 # [ 160.393786] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6143worker1 # [ 161.438215] nix-daemon[835]: reaped child process 1652, status = succeeded6144worker1 # [ 161.621961] nix-daemon[835]: accepted connection from pid 1667, user nix-grpc-daemon (trusted)6145worker1 # [ 161.639517] nix-daemon[835]: reaped child process 1670, status = succeeded6146lb # [ 161.594503] niks3-server[1045]: 2026/09/20 21:11:05 INFO Received uploads request method=POST path=/api/pending_closures6147worker1 # [ 161.662957] nix-grpc-daemon[1423]: time=2026-09-20T21:11:05.237Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6148worker1 # [ 161.665163] nix-grpc-daemon[1423]: time=2026-09-20T21:11:05.237Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6149lb # [ 161.631946] niks3-server[1045]: 2026/09/20 21:11:05 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6150lb # [ 161.644592] niks3-server[1045]: 2026/09/20 21:11:05 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6151lb # [ 161.654407] niks3-server[1045]: 2026/09/20 21:11:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6152worker1 # [ 161.718104] nix-grpc-daemon[1423]: time=2026-09-20T21:11:05.293Z level=INFO msg="Uploading 1 narinfos"6153lb # [ 161.663922] niks3-server[1045]: 2026/09/20 21:11:05 INFO Signed narinfos id=59 count=16154lb # [ 161.670766] niks3-server[1045]: 2026/09/20 21:11:05 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6155lb # [ 161.678251] niks3-server[1045]: 2026/09/20 21:11:05 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6156lb # [ 161.686346] niks3-server[1045]: 2026/09/20 21:11:05 INFO Completed upload id=596157worker1 # [ 161.743283] nix-grpc-daemon[1423]: time=2026-09-20T21:11:05.319Z level=INFO msg="Upload complete. (339ms)"6158lb # [ 161.689359] niks3-server[1045]: 2026/09/20 21:11:05 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6159worker2 # [ 161.717200] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:05Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16160worker1 # [ 161.757142] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16161worker1 # [ 161.773906] nix-daemon[835]: reaped child process 1650, status = succeeded6162worker1 # [ 161.805932] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626163worker1 # [ 161.821589] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427626164worker1 # [ 161.836617] nix-grpc-daemon[1423]: time=2026-09-20T21:11:05.412Z level=INFO msg="All 2 paths already cached"6165worker1 # [ 161.845903] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427626166worker1 # [ 161.860145] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6167worker1 # [ 162.054921] nix-daemon[835]: accepted connection from pid 1674, user nix-grpc-daemon (trusted)6168worker1 # [ 162.073630] nix-daemon[835]: reaped child process 1677, status = succeeded6169worker1 # [ 162.248329] nix-daemon[835]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6170worker1 # [ 162.263735] nix-daemon[835]: reaped child process 1682, status = succeeded6171lb # [ 162.220261] niks3-server[1045]: 2026/09/20 21:11:05 INFO Received uploads request method=POST path=/api/pending_closures6172worker1 # [ 162.287929] nix-grpc-daemon[1423]: time=2026-09-20T21:11:05.863Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6173worker1 # [ 162.289639] nix-grpc-daemon[1423]: time=2026-09-20T21:11:05.863Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6174lb # [ 162.249181] niks3-server[1045]: 2026/09/20 21:11:05 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6175lb # [ 162.262802] niks3-server[1045]: 2026/09/20 21:11:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6176worker1 # [ 162.326689] nix-grpc-daemon[1423]: time=2026-09-20T21:11:05.902Z level=INFO msg="Uploading 1 narinfos"6177lb # [ 162.273099] niks3-server[1045]: 2026/09/20 21:11:05 INFO Signed narinfos id=60 count=16178lb # [ 162.275864] niks3-server[1045]: 2026/09/20 21:11:05 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6179lb # [ 162.286078] niks3-server[1045]: 2026/09/20 21:11:05 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6180lb # [ 162.293289] niks3-server[1045]: 2026/09/20 21:11:05 INFO Completed upload id=606181worker1 # [ 162.350987] nix-grpc-daemon[1423]: time=2026-09-20T21:11:05.926Z level=INFO msg="Upload complete. (474ms)"6182lb # [ 162.296256] niks3-server[1045]: 2026/09/20 21:11:05 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6183worker1 # [ 162.360241] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=0 paths=1 nar_bytes_in=7526184worker1 # [ 162.371327] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626185worker1 # [ 162.378434] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6186worker1 # [ 162.383234] nix-daemon[835]: reaped child process 1672, status = succeeded6187worker1 # [ 162.394638] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6188worker1 # [ 163.442352] nix-daemon[835]: reaped child process 1686, status = succeeded6189worker1 # [ 163.615912] nix-daemon[835]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6190worker1 # [ 163.631300] nix-daemon[835]: reaped child process 1704, status = succeeded6191lb # [ 163.589624] niks3-server[1045]: 2026/09/20 21:11:07 INFO Received uploads request method=POST path=/api/pending_closures6192worker1 # [ 163.657778] nix-grpc-daemon[1423]: time=2026-09-20T21:11:07.232Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6193worker1 # [ 163.659609] nix-grpc-daemon[1423]: time=2026-09-20T21:11:07.232Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6194lb # [ 163.626733] niks3-server[1045]: 2026/09/20 21:11:07 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6195lb # [ 163.638843] niks3-server[1045]: 2026/09/20 21:11:07 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6196lb # [ 163.648190] niks3-server[1045]: 2026/09/20 21:11:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6197worker1 # [ 163.711186] nix-grpc-daemon[1423]: time=2026-09-20T21:11:07.286Z level=INFO msg="Uploading 1 narinfos"6198lb # [ 163.655555] niks3-server[1045]: 2026/09/20 21:11:07 INFO Signed narinfos id=61 count=16199lb # [ 163.661475] niks3-server[1045]: 2026/09/20 21:11:07 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6200lb # [ 163.668099] niks3-server[1045]: 2026/09/20 21:11:07 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6201lb # [ 163.676686] niks3-server[1045]: 2026/09/20 21:11:07 INFO Completed upload id=616202worker1 # [ 163.733888] nix-grpc-daemon[1423]: time=2026-09-20T21:11:07.309Z level=INFO msg="Upload complete. (327ms)"6203lb # [ 163.679605] niks3-server[1045]: 2026/09/20 21:11:07 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6204worker2 # [ 163.708859] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:07Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16205worker1 # [ 163.745127] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16206worker1 # [ 163.765410] nix-daemon[835]: reaped child process 1684, status = succeeded6207worker1 # [ 163.789776] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626208worker1 # [ 163.804873] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427626209worker1 # [ 163.819836] nix-grpc-daemon[1423]: time=2026-09-20T21:11:07.395Z level=INFO msg="All 2 paths already cached"6210worker1 # [ 163.828795] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427626211worker1 # [ 163.842211] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6212worker1 # [ 164.037554] nix-daemon[835]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6213worker1 # [ 164.055551] nix-daemon[835]: reaped child process 1711, status = succeeded6214worker1 # [ 164.231942] nix-daemon[835]: accepted connection from pid 1713, user nix-grpc-daemon (trusted)6215worker1 # [ 164.250130] nix-daemon[835]: reaped child process 1716, status = succeeded6216lb # [ 164.208438] niks3-server[1045]: 2026/09/20 21:11:07 INFO Received uploads request method=POST path=/api/pending_closures6217worker1 # [ 164.277018] nix-grpc-daemon[1423]: time=2026-09-20T21:11:07.852Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6218worker1 # [ 164.278511] nix-grpc-daemon[1423]: time=2026-09-20T21:11:07.852Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6219lb # [ 164.238476] niks3-server[1045]: 2026/09/20 21:11:07 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6220lb # [ 164.253348] niks3-server[1045]: 2026/09/20 21:11:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6221worker1 # [ 164.317380] nix-grpc-daemon[1423]: time=2026-09-20T21:11:07.893Z level=INFO msg="Uploading 1 narinfos"6222lb # [ 164.264346] niks3-server[1045]: 2026/09/20 21:11:07 INFO Signed narinfos id=62 count=16223lb # [ 164.267511] niks3-server[1045]: 2026/09/20 21:11:07 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6224lb # [ 164.279488] niks3-server[1045]: 2026/09/20 21:11:07 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6225lb # [ 164.285885] niks3-server[1045]: 2026/09/20 21:11:07 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6226lb # [ 164.291505] niks3-server[1045]: 2026/09/20 21:11:07 INFO Completed upload id=626227worker1 # [ 164.349326] nix-grpc-daemon[1423]: time=2026-09-20T21:11:07.925Z level=INFO msg="Upload complete. (491ms)"6228worker1 # [ 164.357893] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=0 paths=1 nar_bytes_in=7526229worker1 # [ 164.371953] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427626230worker1 # [ 164.378566] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6231worker1 # [ 164.383338] nix-daemon[835]: reaped child process 1706, status = succeeded6232worker1 # [ 164.397091] nix-daemon[835]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6233worker1 # [ 165.634174] nix-daemon[835]: reaped child process 1720, status = succeeded6234worker1 # [ 165.813186] nix-daemon[835]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6235worker1 # [ 165.831198] nix-daemon[835]: reaped child process 1738, status = succeeded6236lb # [ 165.786131] niks3-server[1045]: 2026/09/20 21:11:09 INFO Received uploads request method=POST path=/api/pending_closures6237worker1 # [ 165.856426] nix-grpc-daemon[1423]: time=2026-09-20T21:11:09.429Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6238worker1 # [ 165.857905] nix-grpc-daemon[1423]: time=2026-09-20T21:11:09.429Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6239lb # [ 165.821523] niks3-server[1045]: 2026/09/20 21:11:09 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6240lb # [ 165.834609] niks3-server[1045]: 2026/09/20 21:11:09 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6241worker1 # [ 165.906474] nix-grpc-daemon[1423]: time=2026-09-20T21:11:09.482Z level=INFO msg="Uploading 1 narinfos"6242lb # [ 165.848518] niks3-server[1045]: 2026/09/20 21:11:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6243lb # [ 165.858134] niks3-server[1045]: 2026/09/20 21:11:09 INFO Signed narinfos id=63 count=16244lb # [ 165.864447] niks3-server[1045]: 2026/09/20 21:11:09 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6245lb # [ 165.871782] niks3-server[1045]: 2026/09/20 21:11:09 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6246lb # [ 165.878662] niks3-server[1045]: 2026/09/20 21:11:09 INFO Completed upload id=636247worker1 # [ 165.936570] nix-grpc-daemon[1423]: time=2026-09-20T21:11:09.512Z level=INFO msg="Upload complete. (328ms)"6248lb # [ 165.882662] niks3-server[1045]: 2026/09/20 21:11:09 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6249worker1 # [ 165.949023] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42762 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16250worker1 # [ 165.962745] nix-daemon[835]: reaped child process 1718, status = succeeded6251client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 20.44 seconds)6252worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6253worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6254client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6255client # [ 166.162408] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6256client # Running as unit: gpu.service; invocation ID: 35de7de818d4444bb76a5aaa99ebf7ee6257client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6258lb: must succeed: curl -sf localhost:9901/clusters6259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6260lb: must succeed: curl -sf localhost:9901/clusters6261lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6262worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6263worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6264client # [ 166.417134] nix[1336]: don't know how to build these paths:6265client # [ 166.418096] nix[1336]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6266worker2 # [ 166.429258] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409246267worker2 # [ 166.435814] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409246268client # [ 166.446145] nix[1336]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6269worker2 # [ 166.458544] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:10Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06270lb: must succeed: curl -sf localhost:9901/clusters6271lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6272lb: must succeed: curl -sf localhost:9901/clusters6273lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6274worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6275worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6276client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6277lb # [ 171.669544] envoy[926]: conn peer=[2001:db8:1::1]:47116 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27088 tx=10830 ms=201076278client # [ 196.450232] nix[1336]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6279client # [ 196.467814] nix[1336]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6280client # [ 196.509399] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6281client # [ 196.517114] systemd[1]: gpu.service: Failed with result 'exit-code'.6282client # [ 196.520368] systemd[1]: gpu.service: Consumed 184ms CPU time over 30.352s wall clock time, 37.5M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6283client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.26 seconds)6284client: must succeed: journalctl -u gpu -o cat6285client: (finished: must succeed: journalctl -u gpu -o cat, in 0.04 seconds)6286lb: must succeed: curl -sf localhost:9901/clusters6287lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6288lb: must succeed: curl -sf localhost:9901/clusters6289lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6290worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6291worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6292(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.55 seconds)6293subtest: low disk drains a worker and builds go to the other6294worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6295worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.13 seconds)6296worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6297worker1 # [ 198.321682] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:41Z level=info event=unhealthy reason=min_free6298worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.05 seconds)6299client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26300worker1 # [ 198.625400] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427626301client # don't know how to build these paths:6302client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6303worker1 # [ 198.637662] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427626304worker2 # [ 198.634509] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:42Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06305worker2 # [ 198.635397] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:42Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16306worker2 # [ 198.679138] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409246307worker2 # [ 198.695265] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409246308client # copying 1 paths to 'lb:50051'...6309client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6310worker2 # [ 198.710723] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409246311worker2 # [ 198.726539] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6312worker2 # [ 198.944555] nix-daemon[856]: accepted connection from pid 1959, user nix-grpc-daemon (trusted)6313worker2 # [ 198.962430] nix-daemon[856]: reaped child process 1962, status = succeeded6314worker2 # [ 199.137124] nix-daemon[856]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6315worker2 # [ 199.153349] nix-daemon[856]: reaped child process 1967, status = succeeded6316lb # [ 199.146684] niks3-server[1045]: 2026/09/20 21:11:42 INFO Received uploads request method=POST path=/api/pending_closures6317worker2 # [ 199.179974] nix-grpc-daemon[1744]: time=2026-09-20T21:11:42.765Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6318worker2 # [ 199.181642] nix-grpc-daemon[1744]: time=2026-09-20T21:11:42.766Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6319lb # [ 199.178618] niks3-server[1045]: 2026/09/20 21:11:42 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6320lb # [ 199.198993] niks3-server[1045]: 2026/09/20 21:11:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6321worker2 # [ 199.226014] nix-grpc-daemon[1744]: time=2026-09-20T21:11:42.812Z level=INFO msg="Uploading 1 narinfos"6322lb # [ 199.208703] niks3-server[1045]: 2026/09/20 21:11:42 INFO Signed narinfos id=64 count=16323lb # [ 199.215399] niks3-server[1045]: 2026/09/20 21:11:42 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6324lb # [ 199.223355] niks3-server[1045]: 2026/09/20 21:11:42 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6325lb # [ 199.229374] niks3-server[1045]: 2026/09/20 21:11:42 INFO Completed upload id=646326worker2 # [ 199.249541] nix-grpc-daemon[1744]: time=2026-09-20T21:11:42.836Z level=INFO msg="Upload complete. (500ms)"6327lb # [ 199.231510] niks3-server[1045]: 2026/09/20 21:11:42 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6328worker2 # [ 199.259284] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=0 paths=1 nar_bytes_in=5126329worker2 # [ 199.271398] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409246330worker2 # [ 199.280975] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6331worker2 # [ 199.283845] nix-daemon[856]: reaped child process 1957, status = succeeded6332client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6333client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6334worker2 # [ 199.297803] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6335worker2 # [ 200.118380] nix-daemon[856]: reaped child process 1971, status = succeeded6336client # farm-a> LOG-farm-a-drain6337worker2 # [ 200.282630] nix-daemon[856]: accepted connection from pid 1986, user nix-grpc-daemon (trusted)6338worker2 # [ 200.298551] nix-daemon[856]: reaped child process 1989, status = succeeded6339lb # [ 200.290105] niks3-server[1045]: 2026/09/20 21:11:43 INFO Received uploads request method=POST path=/api/pending_closures6340worker2 # [ 200.324377] nix-grpc-daemon[1744]: time=2026-09-20T21:11:43.909Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6341worker2 # [ 200.325865] nix-grpc-daemon[1744]: time=2026-09-20T21:11:43.909Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6342lb # [ 200.340289] niks3-server[1045]: 2026/09/20 21:11:43 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6343worker2 # [ 200.370091] nix-grpc-daemon[1744]: time=2026-09-20T21:11:43.956Z level=INFO msg="Uploading 1 narinfos"6344lb # [ 200.350934] niks3-server[1045]: 2026/09/20 21:11:43 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6345lb # [ 200.364145] niks3-server[1045]: 2026/09/20 21:11:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6346lb # [ 200.366853] niks3-server[1045]: 2026/09/20 21:11:43 INFO Signed narinfos id=65 count=16347worker2 # [ 200.393786] nix-grpc-daemon[1744]: time=2026-09-20T21:11:43.980Z level=INFO msg="Upload complete. (286ms)"6348lb # [ 200.372810] niks3-server[1045]: 2026/09/20 21:11:43 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6349lb # [ 200.376850] niks3-server[1045]: 2026/09/20 21:11:43 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6350lb # [ 200.381148] niks3-server[1045]: 2026/09/20 21:11:43 INFO Completed upload id=656351lb # [ 200.382972] niks3-server[1045]: 2026/09/20 21:11:43 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6352worker2 # [ 200.404931] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16353worker2 # [ 200.417327] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:44Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06354worker2 # [ 200.424320] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:44Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16355worker2 # [ 200.429484] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409246356worker2 # [ 200.434136] nix-daemon[856]: reaped child process 1969, status = succeeded6357worker2 # [ 200.435845] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409246358worker2 # [ 200.442970] nix-grpc-daemon[1744]: time=2026-09-20T21:11:44.029Z level=INFO msg="All 1 paths already cached"6359client # copying 1 paths to 'lb:50051'...6360worker2 # [ 200.450584] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409246361client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6362worker2 # [ 200.465292] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6363worker2 # [ 200.652551] nix-daemon[856]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)6364worker2 # [ 200.669923] nix-daemon[856]: reaped child process 1998, status = succeeded6365worker2 # [ 200.853097] nix-daemon[856]: accepted connection from pid 2000, user nix-grpc-daemon (trusted)6366worker2 # [ 200.869054] nix-daemon[856]: reaped child process 2003, status = succeeded6367lb # [ 200.859240] niks3-server[1045]: 2026/09/20 21:11:44 INFO Received uploads request method=POST path=/api/pending_closures6368worker2 # [ 200.889493] nix-grpc-daemon[1744]: time=2026-09-20T21:11:44.476Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6369worker2 # [ 200.891008] nix-grpc-daemon[1744]: time=2026-09-20T21:11:44.476Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6370lb # [ 200.883909] niks3-server[1045]: 2026/09/20 21:11:44 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6371worker2 # [ 200.919070] nix-grpc-daemon[1744]: time=2026-09-20T21:11:44.505Z level=INFO msg="Uploading 1 narinfos"6372lb # [ 200.899210] niks3-server[1045]: 2026/09/20 21:11:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6373lb # [ 200.910734] niks3-server[1045]: 2026/09/20 21:11:44 INFO Signed narinfos id=66 count=16374lb # [ 200.917032] niks3-server[1045]: 2026/09/20 21:11:44 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6375worker2 # [ 200.946396] nix-grpc-daemon[1744]: time=2026-09-20T21:11:44.533Z level=INFO msg="Upload complete. (470ms)"6376lb # [ 200.925589] niks3-server[1045]: 2026/09/20 21:11:44 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6377lb # [ 200.930609] niks3-server[1045]: 2026/09/20 21:11:44 INFO Completed upload id=666378worker2 # [ 200.951800] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=0 paths=1 nar_bytes_in=6326379lb # [ 200.933534] niks3-server[1045]: 2026/09/20 21:11:44 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6380worker2 # [ 200.963252] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409246381worker2 # [ 200.966569] nix-daemon[856]: reaped child process 1993, status = succeeded6382worker2 # [ 200.974283] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6383client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6384client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6385worker2 # [ 200.988214] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6386lb # [ 201.670290] envoy[926]: conn peer=[2001:db8:1::1]:53242 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300616387client # farm-b> LOG-farm-b-drain6388worker2 # [ 202.425962] nix-daemon[856]: reaped child process 2007, status = succeeded6389worker2 # [ 202.612448] nix-daemon[856]: accepted connection from pid 2022, user nix-grpc-daemon (trusted)6390worker2 # [ 202.628246] nix-daemon[856]: reaped child process 2025, status = succeeded6391lb # [ 202.623242] niks3-server[1045]: 2026/09/20 21:11:46 INFO Received uploads request method=POST path=/api/pending_closures6392worker2 # [ 202.656396] nix-grpc-daemon[1744]: time=2026-09-20T21:11:46.240Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6393worker2 # [ 202.657911] nix-grpc-daemon[1744]: time=2026-09-20T21:11:46.240Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6394lb # [ 202.662870] niks3-server[1045]: 2026/09/20 21:11:46 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6395lb # [ 202.674813] niks3-server[1045]: 2026/09/20 21:11:46 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6396lb # [ 202.688113] niks3-server[1045]: 2026/09/20 21:11:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6397worker2 # [ 202.716930] nix-grpc-daemon[1744]: time=2026-09-20T21:11:46.303Z level=INFO msg="Uploading 1 narinfos"6398lb # [ 202.700198] niks3-server[1045]: 2026/09/20 21:11:46 INFO Signed narinfos id=67 count=16399lb # [ 202.708089] niks3-server[1045]: 2026/09/20 21:11:46 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6400lb # [ 202.717257] niks3-server[1045]: 2026/09/20 21:11:46 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6401lb # [ 202.721998] niks3-server[1045]: 2026/09/20 21:11:46 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6402lb # [ 202.725971] niks3-server[1045]: 2026/09/20 21:11:46 INFO Completed upload id=676403worker2 # [ 202.747198] nix-grpc-daemon[1744]: time=2026-09-20T21:11:46.334Z level=INFO msg="Upload complete. (342ms)"6404worker2 # [ 202.768328] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16405worker2 # [ 202.780354] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:46Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06406worker2 # [ 202.786079] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:46Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16407worker2 # [ 202.790528] nix-daemon[856]: reaped child process 2005, status = succeeded6408worker2 # [ 202.809539] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409246409worker2 # [ 202.824672] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409246410worker2 # [ 202.838771] nix-grpc-daemon[1744]: time=2026-09-20T21:11:46.425Z level=INFO msg="All 2 paths already cached"6411client # copying 1 paths to 'lb:50051'...6412client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6413worker2 # [ 202.848943] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409246414worker2 # [ 202.860181] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6415worker2 # [ 203.068195] nix-daemon[856]: accepted connection from pid 2031, user nix-grpc-daemon (trusted)6416worker2 # [ 203.084004] nix-daemon[856]: reaped child process 2034, status = succeeded6417worker2 # [ 203.262259] nix-daemon[856]: accepted connection from pid 2036, user nix-grpc-daemon (trusted)6418worker2 # [ 203.277659] nix-daemon[856]: reaped child process 2039, status = succeeded6419lb # [ 203.268178] niks3-server[1045]: 2026/09/20 21:11:46 INFO Received uploads request method=POST path=/api/pending_closures6420worker2 # [ 203.299490] nix-grpc-daemon[1744]: time=2026-09-20T21:11:46.886Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6421worker2 # [ 203.301070] nix-grpc-daemon[1744]: time=2026-09-20T21:11:46.886Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6422lb # [ 203.300239] niks3-server[1045]: 2026/09/20 21:11:46 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6423worker2 # [ 203.328984] nix-grpc-daemon[1744]: time=2026-09-20T21:11:46.915Z level=INFO msg="Uploading 1 narinfos"6424lb # [ 203.314041] niks3-server[1045]: 2026/09/20 21:11:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6425lb # [ 203.328170] niks3-server[1045]: 2026/09/20 21:11:46 INFO Signed narinfos id=68 count=16426worker2 # [ 203.353536] nix-grpc-daemon[1744]: time=2026-09-20T21:11:46.940Z level=INFO msg="Upload complete. (472ms)"6427lb # [ 203.331066] niks3-server[1045]: 2026/09/20 21:11:46 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6428worker2 # [ 203.360121] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=0 paths=1 nar_bytes_in=7606429lb # [ 203.342154] niks3-server[1045]: 2026/09/20 21:11:46 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6430lb # [ 203.345938] niks3-server[1045]: 2026/09/20 21:11:46 INFO Completed upload id=686431lb # [ 203.348115] niks3-server[1045]: 2026/09/20 21:11:46 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6432worker2 # [ 203.370169] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409246433worker2 # [ 203.375771] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6434worker2 # [ 203.380331] nix-daemon[856]: reaped child process 2029, status = succeeded6435client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6436worker2 # [ 203.389845] nix-daemon[856]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6437client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6438worker2 # [ 204.813022] nix-daemon[856]: reaped child process 2043, status = succeeded6439client # farm-top> LOG-farm-top-drain6440worker2 # [ 204.974875] nix-daemon[856]: accepted connection from pid 2058, user nix-grpc-daemon (trusted)6441worker2 # [ 204.991354] nix-daemon[856]: reaped child process 2061, status = succeeded6442lb # [ 204.986173] niks3-server[1045]: 2026/09/20 21:11:48 INFO Received uploads request method=POST path=/api/pending_closures6443worker2 # [ 205.017616] nix-grpc-daemon[1744]: time=2026-09-20T21:11:48.603Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6444worker2 # [ 205.019066] nix-grpc-daemon[1744]: time=2026-09-20T21:11:48.603Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6445lb # [ 205.022569] niks3-server[1045]: 2026/09/20 21:11:48 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6446lb # [ 205.034286] niks3-server[1045]: 2026/09/20 21:11:48 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6447worker2 # [ 205.066017] nix-grpc-daemon[1744]: time=2026-09-20T21:11:48.652Z level=INFO msg="Uploading 1 narinfos"6448lb # [ 205.044679] niks3-server[1045]: 2026/09/20 21:11:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6449lb # [ 205.051505] niks3-server[1045]: 2026/09/20 21:11:48 INFO Signed narinfos id=69 count=16450lb # [ 205.056573] niks3-server[1045]: 2026/09/20 21:11:48 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6451lb # [ 205.063187] niks3-server[1045]: 2026/09/20 21:11:48 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6452worker2 # [ 205.089041] nix-grpc-daemon[1744]: time=2026-09-20T21:11:48.675Z level=INFO msg="Upload complete. (288ms)"6453lb # [ 205.068362] niks3-server[1045]: 2026/09/20 21:11:48 INFO Completed upload id=696454lb # [ 205.071413] niks3-server[1045]: 2026/09/20 21:11:48 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6455worker2 # [ 205.099269] nix-grpc-daemon[1672]: ts=2026-09-20T21:11:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40924 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16456worker2 # [ 205.114184] nix-daemon[856]: reaped child process 2041, status = succeeded6457client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.83 seconds)6458client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6459client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.22 seconds)6460worker1: must succeed: rm /nix/.rw-store/fill6461worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6462worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6463worker1 # [ 206.326297] nix-grpc-daemon[1367]: ts=2026-09-20T21:11:49Z level=info event=healthy reason=min_free6464worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)6465(finished: subtest: low disk drains a worker and builds go to the other, in 9.60 seconds)6466(finished: run the VM test script, in 207.64 seconds)6467test script finished in 207.93s6468cleanup6469kill QemuMachine (pid 49)6470client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6471kill QemuMachine (pid 50)6472lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6473kill QemuMachine (pid 51)6474worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6475kill QemuMachine (pid 52)6476worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6477(finished: cleanup, in 1.13 seconds)6478additionally exposed symbols:6479 client, lb, worker1, worker2,6480 vlan1,6481 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh