vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #462
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14worker2 # Disk image does not exist, creating the virtualisation disk image...15client: QEMU running (pid 49)16lb: QEMU running (pid 50)17worker1: QEMU running (pid 51)18client # Disk image does not exist, creating the virtualisation disk image...19worker2: QEMU running (pid 52)20lb # Formatting '/build/vm-state-lb/tmp.E5uFvtwuku', fmt=raw size=107374182421client # Formatting '/build/vm-state-client/tmp.FYRARm0Yff', fmt=raw size=107374182422worker2 # Formatting '/build/vm-state-worker2/tmp.BsWXkj5nYl', fmt=raw size=107374182423worker1 # Formatting '/build/vm-state-worker1/tmp.MYLEPhjQT0', fmt=raw size=107374182424lb # mke2fs 1.47.4 (6-Mar-2025)25(finished: start all VMs, in 0.54 seconds)26worker1 # mke2fs 1.47.4 (6-Mar-2025)27lb # Discarding device blocks: 0/262144 done28client: waiting for unit network-addresses-eth1.service29worker1 # Discarding device blocks: 0/262144 done30lb # Creating filesystem with 262144 4k blocks and 65536 inodes31client: waiting for the VM to finish booting32worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes33lb # Filesystem UUID: dc39a975-c708-411b-9408-140d620138b134worker2 # mke2fs 1.47.4 (6-Mar-2025)35client # mke2fs 1.47.4 (6-Mar-2025)36worker1 # Filesystem UUID: a505a470-475f-4e3c-be79-dfe4ace5a1ec37lb # Superblock backups stored on blocks:38worker2 # Discarding device blocks: 0/262144 done39client # Discarding device blocks: 0/262144 done40worker1 # Superblock backups stored on blocks:41lb # 32768, 98304, 163840, 22937642worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes43client # Creating filesystem with 262144 4k blocks and 65536 inodes44worker1 # 32768, 98304, 163840, 22937645lb # 46worker2 # Filesystem UUID: d99f2033-5ec8-4d15-85ec-bb532d39d4c947client # Filesystem UUID: f76c1d46-0987-4c81-bfb2-08ffb764ef4b48worker1 # 49lb # Allocating group tables: 0/8 done50worker2 # Superblock backups stored on blocks:51client # Superblock backups stored on blocks:52worker1 # Allocating group tables: 0/8 done53lb # Writing inode tables: 0/8 done54worker2 # 32768, 98304, 163840, 22937655client # 32768, 98304, 163840, 22937656worker1 # Writing inode tables: 0/8 done57lb # Creating journal (8192 blocks): done58worker2 # 59client # 60worker1 # Creating journal (8192 blocks): done61lb # Writing superblocks and filesystem accounting information: 0/8 done62worker2 # Allocating group tables: 0/8 done63client # Allocating group tables: 0/8 done64client # Writing inode tables: 0/8 done65worker1 # Writing superblocks and filesystem accounting information: 0/8 done66worker2 # Writing inode tables: 0/8 done67client # Creating journal (8192 blocks): done68lb # 69client # Writing superblocks and filesystem accounting information: 0/8 done70lb # Virtualisation disk image created.71worker2 # Creating journal (8192 blocks): done72worker1 # 73worker2 # Writing superblocks and filesystem accounting information: 0/8 done74worker1 # Virtualisation disk image created.75worker2 # 76client # 77worker2 # Virtualisation disk image created.78client # Virtualisation disk image created.79worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker1 # [ 0.000000] KASLR enabled82worker1 # [ 0.000000] random: crng init done83worker1 # [ 0.000000] Machine model: linux,dummy-virt84worker1 # [ 0.000000] efi: UEFI not found.85worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88worker1 # [ 0.000000] Zone ranges:89worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90worker1 # [ 0.000000] DMA32 empty91worker1 # [ 0.000000] Normal empty92worker1 # [ 0.000000] Device empty93worker1 # [ 0.000000] Movable zone start for each node94worker1 # [ 0.000000] Early memory node ranges95worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098worker1 # [ 0.000000] psci: probing for conduit method from DT.99worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.100worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101worker1 # [ 0.000000] psci: Trusted OS migration not required102worker1 # [ 0.000000] psci: SMC Calling Convention v1.1103worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105worker1 # [ 0.000000] Detected PIPT I-cache on CPU0106worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface108worker1 # [ 0.000000] CPU features: detected: Spectre-v4109worker1 # [ 0.000000] CPU features: detected: Spectre-BHB110worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112worker1 # [ 0.000000] alternatives: applying boot alternatives113worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5vbszsyzv004gnsrdf7b2mfaix2sa0dq-nixos-system-worker1-test/init regInfo=/nix/store/w5mlawikgfxdl58sa9ysfv5h6mcrr1x6-closure-info/registration console=ttyAMA0,115200n8 console=tty0114worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/w5mlawikgfxdl58sa9ysfv5h6mcrr1x6-closure-info/registration", will be passed to user space.115worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119worker1 # [ 0.000000] software IO TLB: area num 1.120worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)122worker1 # [ 0.000000] Fallback order for Node 0: 0123worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216124worker1 # [ 0.000000] Policy zone: DMA125worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off126worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1127worker1 # [ 0.000000] allocated 3145728 bytes of page_ext128worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages129worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups130worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.131worker1 # [ 0.000000] rcu: RCU event tracing is enabled.132worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.133worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.134worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.135worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.136worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.137worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]138worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1139worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026140worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.141worker2 # [ 0.000000] KASLR enabled142worker2 # [ 0.000000] random: crng init done143worker2 # [ 0.000000] Machine model: linux,dummy-virt144worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.145worker2 # [ 0.000000] efi: UEFI not found.146lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]147worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.148worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT149worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0150worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]151worker1 # [ 0.000000] GICv3: 256 SPIs implemented152lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026153worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]154worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented155lb # [ 0.000000] KASLR enabled156worker2 # [ 0.000000] Zone ranges:157lb # [ 0.000000] random: crng init done158worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq159worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]160lb # [ 0.000000] Machine model: linux,dummy-virt161worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI162worker2 # [ 0.000000] DMA32 empty163lb # [ 0.000000] efi: UEFI not found.164worker2 # [ 0.000000] Normal empty165worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0166worker2 # [ 0.000000] Device empty167lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT168worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000169worker2 # [ 0.000000] Movable zone start for each node170worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]171worker2 # [ 0.000000] Early memory node ranges172lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]173worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]174lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]175worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)176lb # [ 0.000000] Zone ranges:177worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]178client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]179lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]180worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000181lb # [ 0.000000] DMA32 empty182worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)183lb # [ 0.000000] Normal empty184worker2 # [ 0.000000] psci: probing for conduit method from DT.185lb # [ 0.000000] Device empty186worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000187client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026188worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.189lb # [ 0.000000] Movable zone start for each node190client # [ 0.000000] KASLR enabled191worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000192lb # [ 0.000000] Early memory node ranges193client # [ 0.000000] random: crng init done194worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs195client # [ 0.000000] Machine model: linux,dummy-virt196worker2 # [ 0.000000] psci: Trusted OS migration not required197lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]198worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.199client # [ 0.000000] efi: UEFI not found.200worker2 # [ 0.000000] psci: SMC Calling Convention v1.1201lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]202client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT203worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns204lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000205worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)206client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]207worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).208lb # [ 0.000000] psci: probing for conduit method from DT.209worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296210lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.211client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]212worker2 # [ 0.000000] Detected PIPT I-cache on CPU0213client # [ 0.000000] Zone ranges:214lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs215worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns216client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]217worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)218lb # [ 0.000000] psci: Trusted OS migration not required219client # [ 0.000000] DMA32 empty220worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns221lb # [ 0.000000] psci: SMC Calling Convention v1.1222client # [ 0.000000] Normal empty223worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface224worker1 # [ 0.000041] arm-pv: using stolen time PV225client # [ 0.000000] Device empty226worker2 # [ 0.000000] CPU features: detected: Spectre-v4227client # [ 0.000000] Movable zone start for each node228lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)229worker2 # [ 0.000000] CPU features: detected: Spectre-BHB230client # [ 0.000000] Early memory node ranges231worker1 # [ 0.000947] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)232lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296233worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38234worker1 # [ 0.001116] Console: colour dummy device 80x25235client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]236lb # [ 0.000000] Detected PIPT I-cache on CPU0237worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23238worker1 # [ 0.001124] printk: legacy console [tty0] enabled239client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]240worker2 # [ 0.000000] alternatives: applying boot alternatives241lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)242client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000243worker1 # [ 0.001324] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)244lb # [ 0.000000] CPU features: detected: GICv3 CPU interface245client # [ 0.000000] psci: probing for conduit method from DT.246worker1 # [ 0.001330] pid_max: default: 32768 minimum: 301247lb # [ 0.000000] CPU features: detected: Spectre-v4248client # [ 0.000000] psci: PSCIv1.3 detected in firmware.249lb # [ 0.000000] CPU features: detected: Spectre-BHB250worker1 # [ 0.001406] LSM: initializing lsm=capability,landlock,yama,bpf,ima251client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs252worker1 # [ 0.001613] landlock: Up and running.253lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38254client # [ 0.000000] psci: Trusted OS migration not required255worker1 # [ 0.001615] Yama: becoming mindful.256client # [ 0.000000] psci: SMC Calling Convention v1.1257lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23258worker1 # [ 0.002520] LSM support for eBPF active259worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6yvsrrlv2v3fnzjcmy4bw3slivzrwrqr-nixos-system-worker2-test/init regInfo=/nix/store/b8ic37vid8d5yxv85cz60grxwxgw2vyx-closure-info/registration console=ttyAMA0,115200n8 console=tty0260lb # [ 0.000000] alternatives: applying boot alternatives261worker1 # [ 0.002720] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)262client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)263client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296264worker1 # [ 0.002753] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)265client # [ 0.000000] Detected PIPT I-cache on CPU0266worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/b8ic37vid8d5yxv85cz60grxwxgw2vyx-closure-info/registration", will be passed to user space.267worker1 # [ 0.003861] cacheinfo: Unable to detect cache hierarchy for CPU 0268worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes269worker1 # [ 0.004621] rcu: Hierarchical SRCU implementation.270client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)271worker1 # [ 0.004625] rcu: Max phase no-delay instances is 1000.272client # [ 0.000000] CPU features: detected: GICv3 CPU interface273worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)274lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7q1s737l14q4l5xb2660hk29x3d20j19-nixos-system-lb-test/init regInfo=/nix/store/cbbgdriybx566s3b1zm5jsl6z5kvfm7z-closure-info/registration console=ttyAMA0,115200n8 console=tty0275worker1 # [ 0.005798] fsl-mc MSI: its@8080000 domain created276client # [ 0.000000] CPU features: detected: Spectre-v4277worker1 # [ 0.005883] EFI services will not be available.278client # [ 0.000000] CPU features: detected: Spectre-BHB279worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)280worker1 # [ 0.005959] smp: Bringing up secondary CPUs ...281client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38282worker1 # [ 0.005967] smp: Brought up 1 node, 1 CPU283worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB284client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23285worker1 # [ 0.005970] SMP: Total of 1 processors activated.286lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/cbbgdriybx566s3b1zm5jsl6z5kvfm7z-closure-info/registration", will be passed to user space.287worker2 # [ 0.000000] software IO TLB: area num 1.288client # [ 0.000000] alternatives: applying boot alternatives289worker1 # [ 0.005975] CPU: All CPU(s) started at EL1290lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes291worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB292worker1 # [ 0.005987] CPU features: detected: Branch Target Identification293lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)294worker1 # [ 0.005992] CPU features: detected: ARMv8.4 Translation Table Level295worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)296worker2 # [ 0.000000] Fallback order for Node 0: 0297lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)298worker1 # [ 0.005995] CPU features: detected: Instruction cache invalidation not required for I/D coherence299worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216300lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB301worker2 # [ 0.000000] Policy zone: DMA302worker1 # [ 0.005999] CPU features: detected: Data cache clean to the PoU not required for I/D coherence303lb # [ 0.000000] software IO TLB: area num 1.304client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9b0ww2bi5z80fdwbhp6j472q7cahjy3h-nixos-system-client-test/init regInfo=/nix/store/sss4564bccwij88k7328qm31pya0512n-closure-info/registration console=ttyAMA0,115200n8 console=tty0305worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off306worker1 # [ 0.006002] CPU features: detected: Common not Private translations307lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB308worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1309worker1 # [ 0.006005] CPU features: detected: CRC32 instructions310worker2 # [ 0.000000] allocated 3145728 bytes of page_ext311lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)312client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/sss4564bccwij88k7328qm31pya0512n-closure-info/registration", will be passed to user space.313worker1 # [ 0.006008] CPU features: detected: Data cache clean to Point of Deep Persistence314lb # [ 0.000000] Fallback order for Node 0: 0315worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages316client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes317worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups318worker1 # [ 0.006012] CPU features: detected: Data cache clean to Point of Persistence319lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216320lb # [ 0.000000] Policy zone: DMA321worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.322client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)323worker1 # [ 0.006015] CPU features: detected: Data independent timing control (DIT)324worker2 # [ 0.000000] rcu: RCU event tracing is enabled.325worker1 # [ 0.006017] CPU features: detected: E0PD326lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off327client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)328worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.329lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1330worker1 # [ 0.006020] CPU features: detected: Enhanced Counter Virtualization331worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.332lb # [ 0.000000] allocated 3145728 bytes of page_ext333worker1 # [ 0.006023] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)334worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.335client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB336lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages337worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.338worker1 # [ 0.006026] CPU features: detected: Enhanced Virtualization Traps339client # [ 0.000000] software IO TLB: area num 1.340lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups341worker1 # [ 0.006029] CPU features: detected: Fine Grained Traps342client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB343lb # [ 0.000000] rcu: Hierarchical RCU implementation.344worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.345lb # [ 0.000000] rcu: RCU event tracing is enabled.346worker1 # [ 0.006033] CPU features: detected: Generic authentication (architected QARMA5 algorithm)347client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)348worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1349lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.350client # [ 0.000000] Fallback order for Node 0: 0351worker1 # [ 0.006037] CPU features: detected: RCpc load-acquire (LDAPR)352lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.353worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.354worker1 # [ 0.006040] CPU features: detected: LSE atomic instructions355client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216356lb # [ 0.000000] Rude variant of Tasks RCU enabled.357client # [ 0.000000] Policy zone: DMA358worker1 # [ 0.006043] CPU features: detected: Privileged Access Never359worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.360lb # [ 0.000000] Tracing variant of Tasks RCU enabled.361worker1 # [ 0.006046] CPU features: detected: PMUv3362client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off363lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.364worker1 # [ 0.006049] CPU features: detected: RAS Extension Support365worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.366client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1367client # [ 0.000000] allocated 3145728 bytes of page_ext368worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0369lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1370worker2 # [ 0.000000] GICv3: 256 SPIs implemented371client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages372worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented373lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.374client # [ 0.000000] ftrace: allocated 294 pages with 4 groups375worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq376client # [ 0.000000] rcu: Hierarchical RCU implementation.377worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI378lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.379client # [ 0.000000] rcu: RCU event tracing is enabled.380worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0381client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.382lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.383worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000384client # [ 0.000000] Trampoline variant of Tasks RCU enabled.385lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0386worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]387client # [ 0.000000] Rude variant of Tasks RCU enabled.388lb # [ 0.000000] GICv3: 256 SPIs implemented389client # [ 0.000000] Tracing variant of Tasks RCU enabled.390lb # [ 0.000000] GICv3: 0 Extended SPIs implemented391worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)392lb # [ 0.000000] Root IRQ handler: gic_handle_irq393client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.394lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI395worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)396client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1397lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0398worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000399lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000400client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.401worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000402lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]403worker1 # [ 0.006051] CPU features: detected: RASv1p1 Extension Support404worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.405client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.406worker1 # [ 0.006054] CPU features: detected: Random Number Generator407lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)408worker1 # [ 0.006057] CPU features: detected: Speculation barrier (SB)409client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.410worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns411worker1 # [ 0.006060] CPU features: detected: Stage-2 Force Write-Back412lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)413client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0414worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).415client # [ 0.000000] GICv3: 256 SPIs implemented416worker1 # [ 0.006062] CPU features: detected: TLB range maintenance instructions417lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000418client # [ 0.000000] GICv3: 0 Extended SPIs implemented419worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns420lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000421client # [ 0.000000] Root IRQ handler: gic_handle_irq422worker1 # [ 0.006066] CPU features: detected: Speculative Store Bypassing Safe (SSBS)423client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI424worker1 # [ 0.006108] alternatives: applying system-wide alternatives425lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.426worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns427client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0428worker2 # [ 0.000030] arm-pv: using stolen time PV429worker1 # [ 0.009082] CPU features: detected: BBM Level 2 without TLB conflict abort430client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000431lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns432client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]433worker2 # [ 0.000442] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)434lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).435worker2 # [ 0.000607] Console: colour dummy device 80x25436worker1 # [ 0.009306] Memory: 1406756K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)437worker1 # [ 0.009693] devtmpfs: initialized438worker2 # [ 0.000614] printk: legacy console [tty0] enabled439client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)440lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns441worker1 # [ 0.011566] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)442client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)443worker2 # [ 0.000823] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)444lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns445worker2 # [ 0.000829] pid_max: default: 32768 minimum: 301446lb # [ 0.000029] arm-pv: using stolen time PV447client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000448worker1 # [ 0.011589] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).449worker1 # [ 0.011767] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL450worker2 # [ 0.000905] LSM: initializing lsm=capability,landlock,yama,bpf,ima451client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000452worker2 # [ 0.001032] landlock: Up and running.453lb # [ 0.000394] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)454worker1 # [ 0.011771] 0 pages in range for non-PLT usage455worker2 # [ 0.001034] Yama: becoming mindful.456client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.457worker1 # [ 0.011772] 508288 pages in range for PLT usage458lb # [ 0.000581] Console: colour dummy device 80x25459worker2 # [ 0.001466] LSM support for eBPF active460lb # [ 0.000589] printk: legacy console [tty0] enabled461worker1 # [ 0.011881] pinctrl core: initialized pinctrl subsystem462client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns463worker1 # [ 0.012638] DMI not present or invalid.464worker2 # [ 0.001605] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)465client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).466worker1 # [ 0.016925] NET: Registered PF_NETLINK/PF_ROUTE protocol family467lb # [ 0.000785] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)468worker2 # [ 0.001637] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)469lb # [ 0.000792] pid_max: default: 32768 minimum: 301470worker1 # [ 0.019178] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations471worker2 # [ 0.002701] cacheinfo: Unable to detect cache hierarchy for CPU 0472client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns473lb # [ 0.000867] LSM: initializing lsm=capability,landlock,yama,bpf,ima474worker2 # [ 0.003365] rcu: Hierarchical SRCU implementation.475worker1 # [ 0.019350] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations476lb # [ 0.001011] landlock: Up and running.477lb # [ 0.001014] Yama: becoming mindful.478worker2 # [ 0.003370] rcu: Max phase no-delay instances is 1000.479client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns480client # [ 0.000031] arm-pv: using stolen time PV481lb # [ 0.001507] LSM support for eBPF active482worker1 # [ 0.019550] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations483worker2 # [ 0.004555] fsl-mc MSI: its@8080000 domain created484worker2 # [ 0.004644] EFI services will not be available.485worker1 # [ 0.019570] audit: initializing netlink subsys (disabled)486lb # [ 0.001661] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)487worker2 # [ 0.004725] smp: Bringing up secondary CPUs ...488client # [ 0.000414] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)489worker1 # [ 0.020156] thermal_sys: Registered thermal governor 'fair_share'490worker2 # [ 0.004733] smp: Brought up 1 node, 1 CPU491client # [ 0.000543] Console: colour dummy device 80x25492lb # [ 0.001694] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)493worker2 # [ 0.004736] SMP: Total of 1 processors activated.494worker1 # [ 0.020158] thermal_sys: Registered thermal governor 'bang_bang'495client # [ 0.000550] printk: legacy console [tty0] enabled496lb # [ 0.002810] cacheinfo: Unable to detect cache hierarchy for CPU 0497worker2 # [ 0.004772] CPU: All CPU(s) started at EL1498worker1 # [ 0.020162] thermal_sys: Registered thermal governor 'step_wise'499lb # [ 0.003514] rcu: Hierarchical SRCU implementation.500worker2 # [ 0.004782] CPU features: detected: Branch Target Identification501client # [ 0.000743] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)502worker1 # [ 0.020167] thermal_sys: Registered thermal governor 'user_space'503lb # [ 0.003518] rcu: Max phase no-delay instances is 1000.504client # [ 0.000750] pid_max: default: 32768 minimum: 301505worker2 # [ 0.004787] CPU features: detected: ARMv8.4 Translation Table Level506lb # [ 0.004717] fsl-mc MSI: its@8080000 domain created507worker1 # [ 0.020170] thermal_sys: Registered thermal governor 'power_allocator'508client # [ 0.000817] LSM: initializing lsm=capability,landlock,yama,bpf,ima509lb # [ 0.004816] EFI services will not be available.510client # [ 0.000954] landlock: Up and running.511worker1 # [ 0.020193] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1512worker2 # [ 0.004791] CPU features: detected: Instruction cache invalidation not required for I/D coherence513lb # [ 0.004885] smp: Bringing up secondary CPUs ...514client # [ 0.000957] Yama: becoming mindful.515worker1 # [ 0.020203] cpuidle: using governor ladder516lb # [ 0.004893] smp: Brought up 1 node, 1 CPU517client # [ 0.001428] LSM support for eBPF active518worker1 # [ 0.020209] cpuidle: using governor menu519worker2 # [ 0.004794] CPU features: detected: Data cache clean to the PoU not required for I/D coherence520lb # [ 0.004896] SMP: Total of 1 processors activated.521lb # [ 0.004901] CPU: All CPU(s) started at EL1522worker1 # [ 0.020410] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.523worker2 # [ 0.004797] CPU features: detected: Common not Private translations524client # [ 0.001602] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)525lb # [ 0.004912] CPU features: detected: Branch Target Identification526worker2 # [ 0.004800] CPU features: detected: CRC32 instructions527worker1 # [ 0.020426] ASID allocator initialised with 65536 entries528client # [ 0.001634] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)529worker1 # [ 0.021736] Serial: AMBA PL011 UART driver530lb # [ 0.004917] CPU features: detected: ARMv8.4 Translation Table Level531worker2 # [ 0.004803] CPU features: detected: Data cache clean to Point of Deep Persistence532client # [ 0.002712] cacheinfo: Unable to detect cache hierarchy for CPU 0533worker1 # [ 0.027081] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1534client # [ 0.003439] rcu: Hierarchical SRCU implementation.535worker2 # [ 0.004807] CPU features: detected: Data cache clean to Point of Persistence536lb # [ 0.004920] CPU features: detected: Instruction cache invalidation not required for I/D coherence537worker1 # [ 0.027201] printk: console [ttyAMA0] enabled538client # [ 0.003443] rcu: Max phase no-delay instances is 1000.539worker2 # [ 0.004811] CPU features: detected: Data independent timing control (DIT)540lb # [ 0.004924] CPU features: detected: Data cache clean to the PoU not required for I/D coherence541client # [ 0.004622] fsl-mc MSI: its@8080000 domain created542worker2 # [ 0.004814] CPU features: detected: E0PD543client # [ 0.004712] EFI services will not be available.544worker1 # [ 0.149076] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages545lb # [ 0.004928] CPU features: detected: Common not Private translations546worker2 # [ 0.004816] CPU features: detected: Enhanced Counter Virtualization547client # [ 0.004784] smp: Bringing up secondary CPUs ...548lb # [ 0.004931] CPU features: detected: CRC32 instructions549worker1 # [ 0.149093] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page550client # [ 0.004791] smp: Brought up 1 node, 1 CPU551worker2 # [ 0.004819] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)552client # [ 0.004794] SMP: Total of 1 processors activated.553lb # [ 0.004934] CPU features: detected: Data cache clean to Point of Deep Persistence554worker1 # [ 0.149098] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages555worker2 # [ 0.004823] CPU features: detected: Enhanced Virtualization Traps556client # [ 0.004799] CPU: All CPU(s) started at EL1557worker1 # [ 0.149103] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page558lb # [ 0.004937] CPU features: detected: Data cache clean to Point of Persistence559worker2 # [ 0.004826] CPU features: detected: Fine Grained Traps560client # [ 0.004810] CPU features: detected: Branch Target Identification561worker1 # [ 0.149107] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages562lb # [ 0.004941] CPU features: detected: Data independent timing control (DIT)563client # [ 0.004815] CPU features: detected: ARMv8.4 Translation Table Level564worker2 # [ 0.004829] CPU features: detected: Generic authentication (architected QARMA5 algorithm)565lb # [ 0.004943] CPU features: detected: E0PD566worker1 # [ 0.149111] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page567worker2 # [ 0.004833] CPU features: detected: RCpc load-acquire (LDAPR)568client # [ 0.004818] CPU features: detected: Instruction cache invalidation not required for I/D coherence569lb # [ 0.004946] CPU features: detected: Enhanced Counter Virtualization570worker1 # [ 0.149116] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages571worker2 # [ 0.004836] CPU features: detected: LSE atomic instructions572lb # [ 0.004949] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)573worker1 # [ 0.149120] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page574worker2 # [ 0.004840] CPU features: detected: Privileged Access Never575client # [ 0.004822] CPU features: detected: Data cache clean to the PoU not required for I/D coherence576worker2 # [ 0.004842] CPU features: detected: PMUv3577lb # [ 0.004952] CPU features: detected: Enhanced Virtualization Traps578client # [ 0.004826] CPU features: detected: Common not Private translations579worker2 # [ 0.004845] CPU features: detected: RAS Extension Support580lb # [ 0.004955] CPU features: detected: Fine Grained Traps581client # [ 0.004829] CPU features: detected: CRC32 instructions582worker2 # [ 0.004848] CPU features: detected: RASv1p1 Extension Support583lb # [ 0.004959] CPU features: detected: Generic authentication (architected QARMA5 algorithm)584worker1 # [ 0.156319] fbcon: Taking over console585client # [ 0.004832] CPU features: detected: Data cache clean to Point of Deep Persistence586worker2 # [ 0.004850] CPU features: detected: Random Number Generator587worker1 # [ 0.156331] ACPI: Interpreter disabled.588lb # [ 0.004964] CPU features: detected: RCpc load-acquire (LDAPR)589client # [ 0.004836] CPU features: detected: Data cache clean to Point of Persistence590worker2 # [ 0.004853] CPU features: detected: Speculation barrier (SB)591lb # [ 0.004967] CPU features: detected: LSE atomic instructions592worker2 # [ 0.004855] CPU features: detected: Stage-2 Force Write-Back593client # [ 0.004839] CPU features: detected: Data independent timing control (DIT)594lb # [ 0.004970] CPU features: detected: Privileged Access Never595client # [ 0.004842] CPU features: detected: E0PD596worker2 # [ 0.004858] CPU features: detected: TLB range maintenance instructions597lb # [ 0.004972] CPU features: detected: PMUv3598client # [ 0.004844] CPU features: detected: Enhanced Counter Virtualization599lb # [ 0.004975] CPU features: detected: RAS Extension Support600worker2 # [ 0.004863] CPU features: detected: Speculative Store Bypassing Safe (SSBS)601client # [ 0.004847] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)602lb # [ 0.004978] CPU features: detected: RASv1p1 Extension Support603client # [ 0.004851] CPU features: detected: Enhanced Virtualization Traps604lb # [ 0.004980] CPU features: detected: Random Number Generator605client # [ 0.004854] CPU features: detected: Fine Grained Traps606lb # [ 0.004983] CPU features: detected: Speculation barrier (SB)607lb # [ 0.004986] CPU features: detected: Stage-2 Force Write-Back608client # [ 0.004858] CPU features: detected: Generic authentication (architected QARMA5 algorithm)609worker1 # [ 0.163348] iommu: Default domain type: Translated610lb # [ 0.004989] CPU features: detected: TLB range maintenance instructions611client # [ 0.004863] CPU features: detected: RCpc load-acquire (LDAPR)612worker1 # [ 0.163357] iommu: DMA domain TLB invalidation policy: strict mode613client # [ 0.004866] CPU features: detected: LSE atomic instructions614lb # [ 0.004993] CPU features: detected: Speculative Store Bypassing Safe (SSBS)615worker1 # [ 0.165013] SCSI subsystem initialized616client # [ 0.004869] CPU features: detected: Privileged Access Never617lb # [ 0.005031] alternatives: applying system-wide alternatives618client # [ 0.004871] CPU features: detected: PMUv3619lb # [ 0.008028] CPU features: detected: BBM Level 2 without TLB conflict abort620client # [ 0.004874] CPU features: detected: RAS Extension Support621worker1 # [ 0.165656] usbcore: registered new interface driver usbfs622client # [ 0.004877] CPU features: detected: RASv1p1 Extension Support623worker1 # [ 0.165693] usbcore: registered new interface driver hub624worker1 # [ 0.165711] usbcore: registered new device driver usb625client # [ 0.004880] CPU features: detected: Random Number Generator626lb # [ 0.008242] Memory: 1406504K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)627lb # [ 0.008678] devtmpfs: initialized628worker1 # [ 0.165978] pps_core: LinuxPPS API ver. 1 registered629client # [ 0.004882] CPU features: detected: Speculation barrier (SB)630client # [ 0.004885] CPU features: detected: Stage-2 Force Write-Back631lb # [ 0.010535] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)632worker2 # [ 0.004899] alternatives: applying system-wide alternatives633worker1 # [ 0.165984] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>634client # [ 0.004888] CPU features: detected: TLB range maintenance instructions635worker1 # [ 0.165994] PTP clock support registered636worker2 # [ 0.007769] CPU features: detected: BBM Level 2 without TLB conflict abort637worker1 # [ 0.166043] EDAC MC: Ver: 3.0.0638client # [ 0.004893] CPU features: detected: Speculative Store Bypassing Safe (SSBS)639worker1 # [ 0.170479] scmi_core: SCMI protocol bus registered640worker2 # [ 0.008059] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)641worker2 # [ 0.008418] devtmpfs: initialized642worker1 # [ 0.171437] FPGA manager framework643worker1 # [ 0.172384] vgaarb: loaded644worker2 # [ 0.010292] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)645worker1 # [ 0.172998] clocksource: Switched to clocksource arch_sys_counter646worker2 # [ 0.010314] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).647worker2 # [ 0.010488] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL648worker2 # [ 0.010493] 0 pages in range for non-PLT usage649worker2 # [ 0.010494] 508288 pages in range for PLT usage650worker2 # [ 0.010578] pinctrl core: initialized pinctrl subsystem651worker1 # [ 0.176057] VFS: Disk quotas dquot_6.6.0652worker2 # [ 0.011274] DMI not present or invalid.653worker1 # [ 0.176092] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)654worker2 # [ 0.014446] NET: Registered PF_NETLINK/PF_ROUTE protocol family655lb # [ 0.010558] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).656worker2 # [ 0.016628] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations657lb # [ 0.010757] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL658worker2 # [ 0.016820] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations659client # [ 0.004929] alternatives: applying system-wide alternatives660lb # [ 0.010762] 0 pages in range for non-PLT usage661lb # [ 0.010763] 508288 pages in range for PLT usage662worker1 # [ 0.179666] netfs: FS-Cache loaded663client # [ 0.007990] CPU features: detected: BBM Level 2 without TLB conflict abort664worker2 # [ 0.017017] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations665worker1 # [ 0.179782] pnp: PnP ACPI: disabled666lb # [ 0.010876] pinctrl core: initialized pinctrl subsystem667worker2 # [ 0.017037] audit: initializing netlink subsys (disabled)668lb # [ 0.011633] DMI not present or invalid.669worker2 # [ 0.017600] thermal_sys: Registered thermal governor 'fair_share'670client # [ 0.008187] Memory: 1406780K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)671lb # [ 0.014925] NET: Registered PF_NETLINK/PF_ROUTE protocol family672client # [ 0.008588] devtmpfs: initialized673worker2 # [ 0.017602] thermal_sys: Registered thermal governor 'bang_bang'674lb # [ 0.017149] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations675worker2 # [ 0.017606] thermal_sys: Registered thermal governor 'step_wise'676client # [ 0.010346] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)677lb # [ 0.017338] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations678worker2 # [ 0.017611] thermal_sys: Registered thermal governor 'user_space'679client # [ 0.010368] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).680worker2 # [ 0.017614] thermal_sys: Registered thermal governor 'power_allocator'681lb # [ 0.017548] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations682client # [ 0.010553] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL683lb # [ 0.017569] audit: initializing netlink subsys (disabled)684client # [ 0.010558] 0 pages in range for non-PLT usage685worker2 # [ 0.017637] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1686worker1 # [ 0.183643] NET: Registered PF_INET protocol family687worker2 # [ 0.017647] cpuidle: using governor ladder688client # [ 0.010559] 508288 pages in range for PLT usage689lb # [ 0.018132] thermal_sys: Registered thermal governor 'fair_share'690worker2 # [ 0.017653] cpuidle: using governor menu691worker1 # [ 0.183892] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)692client # [ 0.010680] pinctrl core: initialized pinctrl subsystem693lb # [ 0.018134] thermal_sys: Registered thermal governor 'bang_bang'694client # [ 0.011434] DMI not present or invalid.695worker2 # [ 0.017839] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.696lb # [ 0.018138] thermal_sys: Registered thermal governor 'step_wise'697client # [ 0.014648] NET: Registered PF_NETLINK/PF_ROUTE protocol family698worker2 # [ 0.017854] ASID allocator initialised with 65536 entries699lb # [ 0.018143] thermal_sys: Registered thermal governor 'user_space'700worker2 # [ 0.019022] Serial: AMBA PL011 UART driver701client # [ 0.016838] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations702lb # [ 0.018145] thermal_sys: Registered thermal governor 'power_allocator'703worker2 # [ 0.024108] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1704client # [ 0.017029] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations705worker2 # [ 0.024278] printk: console [ttyAMA0] enabled706lb # [ 0.018169] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1707lb # [ 0.018178] cpuidle: using governor ladder708client # [ 0.017227] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations709lb # [ 0.018183] cpuidle: using governor menu710client # [ 0.017249] audit: initializing netlink subsys (disabled)711worker2 # [ 0.144059] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages712lb # [ 0.018362] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.713client # [ 0.017771] thermal_sys: Registered thermal governor 'fair_share'714worker2 # [ 0.144076] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page715lb # [ 0.018376] ASID allocator initialised with 65536 entries716client # [ 0.017773] thermal_sys: Registered thermal governor 'bang_bang'717lb # [ 0.019491] Serial: AMBA PL011 UART driver718worker2 # [ 0.144081] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages719client # [ 0.017777] thermal_sys: Registered thermal governor 'step_wise'720worker2 # [ 0.144085] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page721lb # [ 0.024711] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1722client # [ 0.017782] thermal_sys: Registered thermal governor 'user_space'723lb # [ 0.024858] printk: console [ttyAMA0] enabled724worker2 # [ 0.144089] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages725client # [ 0.017785] thermal_sys: Registered thermal governor 'power_allocator'726worker2 # [ 0.144093] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page727client # [ 0.017817] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1728lb # [ 0.145108] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages729worker2 # [ 0.144098] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages730client # [ 0.017825] cpuidle: using governor ladder731client # [ 0.017831] cpuidle: using governor menu732lb # [ 0.145127] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page733worker2 # [ 0.144102] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page734lb # [ 0.145132] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages735client # [ 0.018013] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.736client # [ 0.018027] ASID allocator initialised with 65536 entries737lb # [ 0.145137] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page738client # [ 0.019284] Serial: AMBA PL011 UART driver739lb # [ 0.145141] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages740worker2 # [ 0.151372] fbcon: Taking over console741worker2 # [ 0.151386] ACPI: Interpreter disabled.742client # [ 0.024403] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1743lb # [ 0.145145] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page744client # [ 0.024522] printk: console [ttyAMA0] enabled745lb # [ 0.145149] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages746lb # [ 0.145154] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page747client # [ 0.144260] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages748client # [ 0.144280] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page749client # [ 0.144285] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages750lb # [ 0.152729] fbcon: Taking over console751client # [ 0.144290] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page752lb # [ 0.152743] ACPI: Interpreter disabled.753client # [ 0.144294] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages754lb # [ 0.154606] iommu: Default domain type: Translated755worker2 # [ 0.158400] iommu: Default domain type: Translated756client # [ 0.144299] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page757lb # [ 0.154616] iommu: DMA domain TLB invalidation policy: strict mode758worker2 # [ 0.158410] iommu: DMA domain TLB invalidation policy: strict mode759client # [ 0.144303] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages760lb # [ 0.156310] SCSI subsystem initialized761worker2 # [ 0.160048] SCSI subsystem initialized762client # [ 0.144308] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page763worker2 # [ 0.160714] usbcore: registered new interface driver usbfs764worker2 # [ 0.160765] usbcore: registered new interface driver hub765client # [ 0.151821] fbcon: Taking over console766worker2 # [ 0.160783] usbcore: registered new device driver usb767client # [ 0.151836] ACPI: Interpreter disabled.768worker2 # [ 0.161044] pps_core: LinuxPPS API ver. 1 registered769worker2 # [ 0.161050] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>770worker2 # [ 0.161060] PTP clock support registered771worker2 # [ 0.161117] EDAC MC: Ver: 3.0.0772worker2 # [ 0.165700] scmi_core: SCMI protocol bus registered773worker2 # [ 0.166671] FPGA manager framework774lb # [ 0.163481] usbcore: registered new interface driver usbfs775worker2 # [ 0.167670] vgaarb: loaded776lb # [ 0.163513] usbcore: registered new interface driver hub777client # [ 0.158031] iommu: Default domain type: Translated778worker2 # [ 0.168297] clocksource: Switched to clocksource arch_sys_counter779lb # [ 0.163531] usbcore: registered new device driver usb780client # [ 0.158040] iommu: DMA domain TLB invalidation policy: strict mode781lb # [ 0.163804] pps_core: LinuxPPS API ver. 1 registered782client # [ 0.159704] SCSI subsystem initialized783lb # [ 0.163810] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>784lb # [ 0.163819] PTP clock support registered785client # [ 0.160391] usbcore: registered new interface driver usbfs786lb # [ 0.163865] EDAC MC: Ver: 3.0.0787worker2 # [ 0.171149] VFS: Disk quotas dquot_6.6.0788client # [ 0.160424] usbcore: registered new interface driver hub789lb # [ 0.168483] scmi_core: SCMI protocol bus registered790worker2 # [ 0.171182] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)791client # [ 0.160451] usbcore: registered new device driver usb792client # [ 0.160717] pps_core: LinuxPPS API ver. 1 registered793lb # [ 0.169426] FPGA manager framework794lb # [ 0.170392] vgaarb: loaded795client # [ 0.160723] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>796client # [ 0.160734] PTP clock support registered797lb # [ 0.171019] clocksource: Switched to clocksource arch_sys_counter798client # [ 0.160783] EDAC MC: Ver: 3.0.0799worker2 # [ 0.174715] netfs: FS-Cache loaded800worker2 # [ 0.174837] pnp: PnP ACPI: disabled801worker1 # [ 0.212274] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)802client # [ 0.165405] scmi_core: SCMI protocol bus registered803lb # [ 0.172748] VFS: Disk quotas dquot_6.6.0804worker1 # [ 0.212316] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)805client # [ 0.166393] FPGA manager framework806lb # [ 0.172777] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)807client # [ 0.167373] vgaarb: loaded808worker1 # [ 0.212345] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)809worker1 # [ 0.212418] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)810client # [ 0.168015] clocksource: Switched to clocksource arch_sys_counter811worker1 # [ 0.212562] TCP: Hash tables configured (established 16384 bind 16384)812lb # [ 0.176422] netfs: FS-Cache loaded813worker1 # [ 0.212640] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)814lb # [ 0.176553] pnp: PnP ACPI: disabled815worker2 # [ 0.178662] NET: Registered PF_INET protocol family816worker1 # [ 0.212684] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)817worker2 # [ 0.178916] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)818worker1 # [ 0.212720] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)819worker1 # [ 0.212813] NET: Registered PF_UNIX/PF_LOCAL protocol family820worker1 # [ 0.212835] NET: Registered PF_XDP protocol family821worker1 # [ 0.212852] PCI: CLS 0 bytes, default 64822client # [ 0.171809] VFS: Disk quotas dquot_6.6.0823worker1 # [ 0.213099] Trying to unpack rootfs image as initramfs...824client # [ 0.171847] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)825client # [ 0.175434] netfs: FS-Cache loaded826client # [ 0.175538] pnp: PnP ACPI: disabled827lb # [ 0.180443] NET: Registered PF_INET protocol family828lb # [ 0.180723] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)829client # [ 0.179448] NET: Registered PF_INET protocol family830client # [ 0.179709] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)831worker1 # [ 0.230681] kvm [1]: HYP mode not available832worker2 # [ 0.207832] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833worker2 # [ 0.207877] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834worker2 # [ 0.207902] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)835worker2 # [ 0.207974] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)836worker2 # [ 0.208116] TCP: Hash tables configured (established 16384 bind 16384)837worker2 # [ 0.208224] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)838worker2 # [ 0.208260] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)839worker2 # [ 0.214653] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)840worker2 # [ 0.214752] NET: Registered PF_UNIX/PF_LOCAL protocol family841worker2 # [ 0.214780] NET: Registered PF_XDP protocol family842worker2 # [ 0.214799] PCI: CLS 0 bytes, default 64843lb # [ 0.209589] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)844worker2 # [ 0.215026] Trying to unpack rootfs image as initramfs...845lb # [ 0.209634] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)846worker2 # [ 0.223585] kvm [1]: HYP mode not available847lb # [ 0.209663] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)848lb # [ 0.209738] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)849lb # [ 0.209881] TCP: Hash tables configured (established 16384 bind 16384)850client # [ 0.209706] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)851lb # [ 0.209965] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)852client # [ 0.209762] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)853lb # [ 0.210010] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)854client # [ 0.209794] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)855lb # [ 0.210046] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)856client # [ 0.209881] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)857lb # [ 0.210134] NET: Registered PF_UNIX/PF_LOCAL protocol family858client # [ 0.210025] TCP: Hash tables configured (established 16384 bind 16384)859lb # [ 0.210157] NET: Registered PF_XDP protocol family860lb # [ 0.210175] PCI: CLS 0 bytes, default 64861client # [ 0.210112] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)862lb # [ 0.210423] Trying to unpack rootfs image as initramfs...863client # [ 0.210148] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)864client # [ 0.210184] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)865client # [ 0.210302] NET: Registered PF_UNIX/PF_LOCAL protocol family866client # [ 0.210330] NET: Registered PF_XDP protocol family867client # [ 0.210350] PCI: CLS 0 bytes, default 64868client # [ 0.210602] Trying to unpack rootfs image as initramfs...869lb # [ 0.228940] kvm [1]: HYP mode not available870client # [ 0.225864] kvm [1]: HYP mode not available871worker1 # [ 0.315757] Initialise system trusted keyrings872worker1 # [ 0.325063] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker1 # [ 0.326481] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker1 # [ 0.327243] 9p: Installing v9fs 9p2000 file system support875worker1 # [ 0.339811] Key type asymmetric registered876worker1 # [ 0.339822] Asymmetric key parser 'x509' registered877worker1 # [ 0.339868] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)878worker1 # [ 0.349074] io scheduler mq-deadline registered879worker1 # [ 0.349093] io scheduler kyber registered880worker1 # [ 0.357159] pl061_gpio 9030000.pl061: PL061 GPIO chip registered881worker2 # [ 0.316916] Initialise system trusted keyrings882worker2 # [ 0.317706] workingset: timestamp_bits=42 max_order=19 bucket_order=0883worker2 # [ 0.318954] squashfs: version 4.0 (2009/01/31) Phillip Lougher884worker2 # [ 0.319729] 9p: Installing v9fs 9p2000 file system support885worker1 # [ 0.365045] ledtrig-cpu: registered to indicate activity on CPUs886lb # [ 0.324289] Initialise system trusted keyrings887worker1 # [ 0.365510] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:888lb # [ 0.325042] workingset: timestamp_bits=42 max_order=19 bucket_order=0889worker1 # [ 0.365529] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000890lb # [ 0.326305] squashfs: version 4.0 (2009/01/31) Phillip Lougher891worker1 # [ 0.365541] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000892client # [ 0.321047] Initialise system trusted keyrings893worker1 # [ 0.365549] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000894client # [ 0.321805] workingset: timestamp_bits=42 max_order=19 bucket_order=0895worker1 # [ 0.365569] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits896client # [ 0.323091] squashfs: version 4.0 (2009/01/31) Phillip Lougher897worker1 # [ 0.365594] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]898client # [ 0.323875] 9p: Installing v9fs 9p2000 file system support899worker1 # [ 0.365682] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00900lb # [ 0.327149] 9p: Installing v9fs 9p2000 file system support901worker1 # [ 0.365692] pci_bus 0000:00: root bus resource [bus 00-ff]902worker1 # [ 0.365697] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]903worker1 # [ 0.365702] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]904worker1 # [ 0.365707] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]905worker1 # [ 0.365786] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint906worker1 # [ 0.366225] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint907worker1 # [ 0.366423] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]908worker1 # [ 0.366439] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]909worker1 # [ 0.366468] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]910worker1 # [ 0.366484] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]911worker1 # [ 0.366946] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint912worker1 # [ 0.367130] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]913worker1 # [ 0.367145] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]914worker1 # [ 0.367174] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]915worker1 # [ 0.367628] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint916worker1 # [ 0.367809] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]917worker1 # [ 0.367824] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]918worker2 # [ 0.348377] Key type asymmetric registered919worker1 # [ 0.367853] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]920worker2 # [ 0.348403] Asymmetric key parser 'x509' registered921worker1 # [ 0.368308] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint922worker2 # [ 0.348469] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)923worker1 # [ 0.368490] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]924worker2 # [ 0.350589] io scheduler mq-deadline registered925worker1 # [ 0.368506] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]926worker2 # [ 0.350600] io scheduler kyber registered927worker1 # [ 0.368538] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]928worker1 # [ 0.368992] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint929worker1 # [ 0.369189] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]930worker1 # [ 0.369205] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]931worker1 # [ 0.369234] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]932worker1 # [ 0.369709] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint933worker1 # [ 0.369891] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]934worker1 # [ 0.369907] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]935worker1 # [ 0.369935] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]936lb # [ 0.355067] Key type asymmetric registered937lb # [ 0.355094] Asymmetric key parser 'x509' registered938worker1 # [ 0.370391] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint939lb # [ 0.355164] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)940worker1 # [ 0.370573] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]941lb # [ 0.357321] io scheduler mq-deadline registered942worker1 # [ 0.370589] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]943lb # [ 0.357332] io scheduler kyber registered944worker2 # [ 0.360442] pl061_gpio 9030000.pl061: PL061 GPIO chip registered945worker1 # [ 0.370617] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]946worker1 # [ 0.370633] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]947client # [ 0.352491] Key type asymmetric registered948client # [ 0.352517] Asymmetric key parser 'x509' registered949worker1 # [ 0.371100] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint950worker1 # [ 0.371286] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]951client # [ 0.352597] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)952worker2 # [ 0.361790] ledtrig-cpu: registered to indicate activity on CPUs953client # [ 0.354697] io scheduler mq-deadline registered954worker1 # [ 0.371315] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]955client # [ 0.354708] io scheduler kyber registered956worker2 # [ 0.362174] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:957worker1 # [ 0.371764] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint958worker2 # [ 0.362190] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000959worker1 # [ 0.371948] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]960worker1 # [ 0.371977] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]961worker2 # [ 0.362202] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000962worker1 # [ 0.372367] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint963worker2 # [ 0.362211] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000964worker1 # [ 0.372548] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]965worker2 # [ 0.362231] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits966worker1 # [ 0.372791] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint967worker2 # [ 0.362255] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]968lb # [ 0.367155] pl061_gpio 9030000.pl061: PL061 GPIO chip registered969worker2 # [ 0.362340] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00970worker2 # [ 0.362350] pci_bus 0000:00: root bus resource [bus 00-ff]971worker2 # [ 0.362356] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]972worker2 # [ 0.362361] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]973lb # [ 0.368537] ledtrig-cpu: registered to indicate activity on CPUs974worker2 # [ 0.362366] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]975client # [ 0.364129] pl061_gpio 9030000.pl061: PL061 GPIO chip registered976lb # [ 0.368913] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:977worker2 # [ 0.362423] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint978lb # [ 0.368930] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000979worker2 # [ 0.362853] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint980worker2 # [ 0.363049] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]981lb # [ 0.368942] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000982worker2 # [ 0.363065] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]983client # [ 0.365462] ledtrig-cpu: registered to indicate activity on CPUs984lb # [ 0.368950] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000985worker2 # [ 0.363095] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]986client # [ 0.365859] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:987worker1 # [ 0.421109] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]988worker2 # [ 0.363112] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]989lb # [ 0.368979] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits990worker1 # [ 0.421136] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]991client # [ 0.365876] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000992worker2 # [ 0.363562] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint993worker1 # [ 0.421165] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994lb # [ 0.369002] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]995client # [ 0.365889] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000996worker2 # [ 0.363744] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]997lb # [ 0.369081] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00998worker1 # [ 0.421664] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint999worker2 # [ 0.363759] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1000client # [ 0.365898] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001001worker1 # [ 0.421849] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1002lb # [ 0.369090] pci_bus 0000:00: root bus resource [bus 00-ff]1003worker2 # [ 0.363789] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1004worker1 # [ 0.421865] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1005lb # [ 0.369096] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1006client # [ 0.365917] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1007worker2 # [ 0.364233] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1008lb # [ 0.369101] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1009worker1 # [ 0.421894] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1010client # [ 0.365942] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1011worker2 # [ 0.384294] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1012worker1 # [ 0.422518] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1013lb # [ 0.369107] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1014client # [ 0.366016] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001015worker1 # [ 0.422530] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1016lb # [ 0.369163] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1017client # [ 0.366025] pci_bus 0000:00: root bus resource [bus 00-ff]1018worker1 # [ 0.422535] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1019client # [ 0.366031] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1020lb # [ 0.369614] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1021client # [ 0.366036] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1022lb # [ 0.369813] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1023worker1 # [ 0.422581] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1024lb # [ 0.369829] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1025client # [ 0.366042] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1026worker1 # [ 0.422628] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1027worker2 # [ 0.384923] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1028lb # [ 0.369859] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1029client # [ 0.366098] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1030worker1 # [ 0.422675] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1031worker2 # [ 0.384955] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1032lb # [ 0.369874] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1033client # [ 0.366550] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1034worker1 # [ 0.422723] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1035worker2 # [ 0.385421] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1036lb # [ 0.370336] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1037client # [ 0.366752] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1038worker2 # [ 0.385602] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1039lb # [ 0.370523] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1040worker1 # [ 0.422770] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1041client # [ 0.366778] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1042worker2 # [ 0.385618] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1043lb # [ 0.370539] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1044worker1 # [ 0.422827] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1045client # [ 0.366807] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046worker2 # [ 0.385651] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1047lb # [ 0.370569] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1048client # [ 0.366823] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1049worker1 # [ 0.422875] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1050worker2 # [ 0.386160] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1051client # [ 0.367298] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1052worker2 # [ 0.386344] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1053worker1 # [ 0.422925] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1054client # [ 0.367484] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1055worker2 # [ 0.386360] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1056lb # [ 0.389895] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1057worker1 # [ 0.422971] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1058client # [ 0.367500] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1059lb # [ 0.390087] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1060worker2 # [ 0.386389] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1061worker1 # [ 0.423066] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1062client # [ 0.367529] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1063lb # [ 0.390102] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1064worker2 # [ 0.386843] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1065worker1 # [ 0.423112] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1066lb # [ 0.390132] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067client # [ 0.367990] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1068worker2 # [ 0.387029] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1069worker1 # [ 0.423134] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1070worker2 # [ 0.387045] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1071lb # [ 0.390597] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1072client # [ 0.387586] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1073worker1 # [ 0.423155] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1074lb # [ 0.390787] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1075worker2 # [ 0.387075] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1076client # [ 0.387606] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1077worker1 # [ 0.423179] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1078lb # [ 0.390803] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1079worker2 # [ 0.387531] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1080client # [ 0.387635] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081worker1 # [ 0.423203] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1082lb # [ 0.390835] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1083worker2 # [ 0.387720] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1084worker1 # [ 0.423225] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1085worker2 # [ 0.387735] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1086worker2 # [ 0.387765] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1087worker1 # [ 0.423246] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1088worker2 # [ 0.387781] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1089worker1 # [ 0.423268] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1090worker1 # [ 0.423290] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1091worker2 # [ 0.388239] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1092worker1 # [ 0.423311] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1093worker1 # [ 0.423334] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1094worker1 # [ 0.423357] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1095worker1 # [ 0.423378] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1096worker1 # [ 0.423398] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1097worker1 # [ 0.423419] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1098worker1 # [ 0.423440] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1099worker1 # [ 0.423461] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1100worker1 # [ 0.423482] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1101client # [ 0.396140] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1102worker1 # [ 0.423505] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1103client # [ 0.396333] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1104worker1 # [ 0.423526] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1105client # [ 0.396349] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1106worker1 # [ 0.423548] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1107client # [ 0.396382] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108worker2 # [ 0.408458] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1109worker1 # [ 0.423574] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1110worker2 # [ 0.408495] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111client # [ 0.396852] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1112worker1 # [ 0.423583] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1113client # [ 0.397035] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1114worker1 # [ 0.423588] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1115worker2 # [ 0.408965] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1116client # [ 0.397051] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1117lb # [ 0.403343] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118worker1 # [ 0.424760] pci 0000:00:0a.0: enabling device (0000 -> 0002)1119worker2 # [ 0.409153] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1120lb # [ 0.403535] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1121client # [ 0.397080] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122worker2 # [ 0.409182] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123lb # [ 0.403552] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1124client # [ 0.397537] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1125worker2 # [ 0.409582] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1126lb # [ 0.403582] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127client # [ 0.397721] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1128worker2 # [ 0.409763] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1129client # [ 0.397736] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1130lb # [ 0.404039] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1131worker2 # [ 0.410011] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1132lb # [ 0.404228] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1133client # [ 0.397766] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1134worker2 # [ 0.410261] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1135lb # [ 0.404243] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1136client # [ 0.398226] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1137worker2 # [ 0.410278] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1138lb # [ 0.404273] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1139client # [ 0.398409] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1140worker2 # [ 0.410307] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1141client # [ 0.398424] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1142lb # [ 0.404754] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1143worker2 # [ 0.410758] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1144client # [ 0.398453] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1145lb # [ 0.404938] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1146worker2 # [ 0.410939] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1147client # [ 0.398469] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1148lb # [ 0.404954] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1149worker2 # [ 0.410955] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1150lb # [ 0.404982] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1151client # [ 0.398931] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1152worker2 # [ 0.410985] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1153lb # [ 0.404998] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1154client # [ 0.399122] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1155worker1 # [ 0.469119] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 43323 usecs1156worker2 # [ 0.411556] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1157client # [ 0.399151] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158lb # [ 0.405461] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1159worker2 # [ 0.411567] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1160lb # [ 0.405655] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1161client # [ 0.399607] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1162worker2 # [ 0.411573] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1163lb # [ 0.405684] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164client # [ 0.399795] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1165worker2 # [ 0.411619] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1166client # [ 0.399824] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167lb # [ 0.406140] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1168worker2 # [ 0.411666] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1169lb # [ 0.406330] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1170lb # [ 0.406359] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171worker2 # [ 0.411715] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1172lb # [ 0.406761] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1173worker2 # [ 0.411763] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1174lb # [ 0.406946] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1175worker2 # [ 0.411810] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1176lb # [ 0.407206] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1177worker2 # [ 0.411858] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1178lb # [ 0.407457] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1179worker2 # [ 0.411905] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1180lb # [ 0.407474] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1181lb # [ 0.407504] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1182worker2 # [ 0.411952] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1183client # [ 0.418245] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1184lb # [ 0.407966] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1185worker2 # [ 0.411999] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1186client # [ 0.418429] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1187lb # [ 0.408161] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1188worker2 # [ 0.412060] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1189client # [ 0.418675] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1190lb # [ 0.408176] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1191client # [ 0.418945] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1192worker2 # [ 0.412106] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1193lb # [ 0.408205] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1194client # [ 0.418962] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1195worker2 # [ 0.412127] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1196lb # [ 0.408782] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1197client # [ 0.418991] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1198worker2 # [ 0.412149] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1199lb # [ 0.408794] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1200worker2 # [ 0.412171] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1201client # [ 0.419474] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1202lb # [ 0.408800] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1203client # [ 0.419661] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1204worker2 # [ 0.412195] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1205client # [ 0.419677] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1206lb # [ 0.408845] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1207worker2 # [ 0.412218] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1208client # [ 0.419706] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1209worker2 # [ 0.412240] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1210lb # [ 0.408892] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1211client # [ 0.420298] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1212worker2 # [ 0.412262] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1213lb # [ 0.408938] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1214worker2 # [ 0.412284] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1215client # [ 0.420309] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1216lb # [ 0.408984] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1217client # [ 0.420315] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1218lb # [ 0.409031] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1219client # [ 0.420361] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1220lb # [ 0.409078] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1221client # [ 0.420407] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1222lb # [ 0.409124] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1223client # [ 0.420457] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1224worker1 # [ 0.489037] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1225lb # [ 0.409170] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1226client # [ 0.420503] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1227worker1 # [ 0.491210] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1228lb # [ 0.409217] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1229client # [ 0.420549] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1230lb # [ 0.409280] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1231client # [ 0.420595] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1232lb # [ 0.409326] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1233client # [ 0.420641] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1234lb # [ 0.409347] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1235worker2 # [ 0.448368] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1236client # [ 0.420687] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1237lb # [ 0.409369] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1238worker2 # [ 0.448416] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1239client # [ 0.420734] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1240lb # [ 0.409390] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1241worker2 # [ 0.448441] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1242client # [ 0.420805] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1243lb # [ 0.409413] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1244worker2 # [ 0.448465] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1245client # [ 0.420851] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1246lb # [ 0.409435] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1247worker2 # [ 0.448487] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1248client # [ 0.420872] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1249lb # [ 0.409457] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1250worker2 # [ 0.448510] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1251client # [ 0.420893] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1252lb # [ 0.409485] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1253worker2 # [ 0.448532] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1254client # [ 0.420915] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1255lb # [ 0.409507] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1256worker2 # [ 0.448555] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1257client # [ 0.420939] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1258worker2 # [ 0.448578] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1259lb # [ 0.409530] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1260worker2 # [ 0.448600] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1261client # [ 0.420961] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1262lb # [ 0.409552] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1263worker2 # [ 0.448623] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1264worker1 # [ 0.501210] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1265client # [ 0.420983] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1266lb # [ 0.409574] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1267worker2 # [ 0.448646] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1268client # [ 0.421004] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1269worker1 # [ 0.503530] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1270lb # [ 0.409595] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1271worker2 # [ 0.448680] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1272client # [ 0.421026] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1273lb # [ 0.409617] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1274worker2 # [ 0.448689] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1275lb # [ 0.409638] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1276client # [ 0.421048] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1277worker2 # [ 0.448694] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1278lb # [ 0.409659] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1279client # [ 0.421070] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1280worker2 # [ 0.449890] pci 0000:00:0a.0: enabling device (0000 -> 0002)1281lb # [ 0.409680] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1282client # [ 0.421092] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1283lb # [ 0.409701] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1284client # [ 0.421113] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1285lb # [ 0.409723] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1286client # [ 0.421133] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1287lb # [ 0.409744] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1288client # [ 0.421154] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1289lb # [ 0.409766] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1290client # [ 0.421175] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1291lb # [ 0.409793] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1292client # [ 0.421197] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1293lb # [ 0.409803] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1294client # [ 0.421218] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1295lb # [ 0.409807] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1296client # [ 0.421240] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1297lb # [ 0.410968] pci 0000:00:0a.0: enabling device (0000 -> 0002)1298client # [ 0.421261] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1299client # [ 0.421282] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1300client # [ 0.421309] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1301client # [ 0.421319] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1302client # [ 0.421324] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1303client # [ 0.422480] pci 0000:00:0a.0: enabling device (0000 -> 0002)1304worker1 # [ 0.513557] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1305lb # [ 0.471160] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 58784 usecs1306worker1 # [ 0.516113] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1307worker1 # [ 0.518324] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1308worker1 # [ 0.520460] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1309worker1 # [ 0.530815] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1310worker1 # [ 0.532589] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1311worker2 # [ 0.486860] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1312worker2 # [ 0.489871] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1313worker2 # [ 0.491790] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1314client # [ 0.486232] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1315lb # [ 0.491531] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1316lb # [ 0.493746] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1317worker1 # [ 0.543070] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1318worker2 # [ 0.501691] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1319worker2 # [ 0.503546] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1320lb # [ 0.504948] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1321client # [ 0.496265] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1322client # [ 0.499906] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1323worker1 # [ 0.555856] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1324worker1 # [ 0.557477] msm_serial: driver initialized1325worker1 # [ 0.557606] SuperH (H)SCI(F) driver initialized1326worker2 # [ 0.515002] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1327worker1 # [ 0.557658] STM32 USART driver initialized1328lb # [ 0.511306] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1329lb # [ 0.513453] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1330worker2 # [ 0.517889] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1331worker2 # [ 0.520020] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1332client # [ 0.509314] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1333client # [ 0.511795] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1334lb # [ 0.523662] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1335lb # [ 0.525902] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1336worker2 # [ 0.530147] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1337worker2 # [ 0.531901] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1338client # [ 0.521456] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1339lb # [ 0.537406] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1340worker2 # [ 0.543174] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1341worker1 # [ 0.591911] loop: module loaded1342worker1 # [ 0.592084] virtio_blk virtio5: 1/0/0 default/read/poll queues1343worker1 # [ 0.592809] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1344client # [ 0.532574] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1345client # [ 0.534693] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1346client # [ 0.536816] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1347client # [ 0.538689] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1348worker2 # [ 0.553545] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1349worker2 # [ 0.556117] msm_serial: driver initialized1350worker2 # [ 0.556276] SuperH (H)SCI(F) driver initialized1351worker1 # [ 0.597635] megasas: 07.734.00.00-rc11352worker1 # [ 0.598340] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1353lb # [ 0.547317] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1354lb # [ 0.550709] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1355worker2 # [ 0.557405] STM32 USART driver initialized1356lb # [ 0.553447] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1357client # [ 0.549550] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1358worker1 # [ 0.610094] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001359worker1 # [ 0.610127] Intel/Sharp Extended Query Table at 0x00311360worker1 # [ 0.611706] Using buffer write method1361worker1 # [ 0.611768] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1362lb # [ 0.571538] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1363client # [ 0.566606] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1364lb # [ 0.574305] msm_serial: driver initialized1365lb # [ 0.574511] SuperH (H)SCI(F) driver initialized1366lb # [ 0.574566] STM32 USART driver initialized1367client # [ 0.569145] msm_serial: driver initialized1368client # [ 0.569279] SuperH (H)SCI(F) driver initialized1369client # [ 0.569334] STM32 USART driver initialized1370worker1 # [ 0.626103] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001371worker1 # [ 0.626132] Intel/Sharp Extended Query Table at 0x00311372worker2 # [ 0.588915] loop: module loaded1373worker2 # [ 0.589087] virtio_blk virtio5: 1/0/0 default/read/poll queues1374worker1 # [ 0.637542] Using buffer write method1375worker1 # [ 0.637574] Concatenating MTD devices:1376worker2 # [ 0.589871] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1377worker1 # [ 0.637579] (0): "0.flash"1378worker1 # [ 0.637583] (1): "0.flash"1379worker1 # [ 0.637586] into device "0.flash"1380worker2 # [ 0.600919] megasas: 07.734.00.00-rc11381worker2 # [ 0.601631] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1382client # [ 0.598950] loop: module loaded1383lb # [ 0.605787] loop: module loaded1384worker2 # [ 0.603733] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001385client # [ 0.599133] virtio_blk virtio5: 1/0/0 default/read/poll queues1386lb # [ 0.605967] virtio_blk virtio5: 1/0/0 default/read/poll queues1387worker2 # [ 0.603757] Intel/Sharp Extended Query Table at 0x00311388worker2 # [ 0.605645] Using buffer write method1389client # [ 0.599920] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1390lb # [ 0.606705] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1391worker2 # [ 0.605710] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1392worker2 # [ 0.607403] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001393worker2 # [ 0.607432] Intel/Sharp Extended Query Table at 0x00311394lb # [ 0.611617] megasas: 07.734.00.00-rc11395lb # [ 0.612308] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1396worker2 # [ 0.625889] Using buffer write method1397worker2 # [ 0.625926] Concatenating MTD devices:1398worker2 # [ 0.625931] (0): "0.flash"1399worker2 # [ 0.625935] (1): "0.flash"1400worker2 # [ 0.625938] into device "0.flash"1401client # [ 0.612637] megasas: 07.734.00.00-rc11402client # [ 0.613375] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1403client # [ 0.615594] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001404client # [ 0.615621] Intel/Sharp Extended Query Table at 0x00311405lb # [ 0.623076] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406client # [ 0.617333] Using buffer write method1407lb # [ 0.623104] Intel/Sharp Extended Query Table at 0x00311408lb # [ 0.625908] Using buffer write method1409client # [ 0.617405] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1410lb # [ 0.625988] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1411client # [ 0.619224] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001412lb # [ 0.627697] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001413client # [ 0.619256] Intel/Sharp Extended Query Table at 0x00311414lb # [ 0.627721] Intel/Sharp Extended Query Table at 0x00311415lb # [ 0.629308] Using buffer write method1416lb # [ 0.629334] Concatenating MTD devices:1417lb # [ 0.629338] (0): "0.flash"1418lb # [ 0.629342] (1): "0.flash"1419lb # [ 0.629346] into device "0.flash"1420client # [ 0.637679] Using buffer write method1421client # [ 0.637716] Concatenating MTD devices:1422client # [ 0.637720] (0): "0.flash"1423client # [ 0.637724] (1): "0.flash"1424client # [ 0.637728] into device "0.flash"1425worker1 # [ 0.861279] Freeing initrd memory: 26148K1426worker1 # [ 0.867272] tun: Universal TUN/TAP device driver, 1.61427worker1 # [ 0.870945] thunder_xcv, ver 1.01428worker1 # [ 0.870988] thunder_bgx, ver 1.01429worker1 # [ 0.871011] nicpf, ver 1.01430worker1 # [ 0.871543] e1000: Intel(R) PRO/1000 Network Driver1431worker1 # [ 0.871550] e1000: Copyright (c) 1999-2006 Intel Corporation.1432worker1 # [ 0.871574] e1000e: Intel(R) PRO/1000 Network Driver1433worker1 # [ 0.871583] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1434worker1 # [ 0.871611] igb: Intel(R) Gigabit Ethernet Network Driver1435worker1 # [ 0.871617] igb: Copyright (c) 2007-2014 Intel Corporation.1436worker1 # [ 0.871640] igbvf: Intel(R) Gigabit Virtual Function Network Driver1437worker1 # [ 0.871646] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1438worker1 # [ 0.871774] sky2: driver version 1.301439worker1 # [ 0.873732] ehci-pci 0000:00:0a.0: EHCI Host Controller1440worker1 # [ 0.873757] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11441worker1 # [ 0.874002] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001442worker1 # [ 0.883636] usbcore: registered new interface driver usb-storage1443worker1 # [ 0.883769] usbcore: registered new interface driver usbserial_generic1444worker1 # [ 0.883783] usbserial: USB Serial support registered for generic1445worker1 # [ 0.884359] hv_vmbus: registering driver hyperv_keyboard1446worker1 # [ 0.887983] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001447worker1 # [ 0.888303] hub 1-0:1.0: USB hub found1448worker1 # [ 0.888321] hub 1-0:1.0: 6 ports detected1449worker1 # [ 0.890618] rtc-pl031 9010000.pl031: registered as rtc01450worker1 # [ 0.890646] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T04:23:53 UTC (1789964633)1451worker1 # [ 0.890973] i2c_dev: i2c /dev entries driver1452worker2 # [ 0.847354] Freeing initrd memory: 26148K1453worker1 # [ 0.895815] sdhci: Secure Digital Host Controller Interface driver1454worker1 # [ 0.895824] sdhci: Copyright(c) Pierre Ossman1455worker1 # [ 0.896081] Synopsys Designware Multimedia Card Interface Driver1456worker1 # [ 0.896443] sdhci-pltfm: SDHCI platform and OF driver helper1457worker2 # [ 0.853209] tun: Universal TUN/TAP device driver, 1.61458worker1 # [ 0.900773] hid: raw HID events driver (C) Jiri Kosina1459worker1 # [ 0.901663] usbcore: registered new interface driver usbhid1460worker1 # [ 0.901672] usbhid: USB HID core driver1461worker1 # [ 0.903870] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1462worker1 # [ 0.906409] drop_monitor: Initializing network drop monitor service1463worker2 # [ 0.856726] thunder_xcv, ver 1.01464worker1 # [ 0.906547] NET: Registered PF_INET6 protocol family1465worker2 # [ 0.856776] thunder_bgx, ver 1.01466worker2 # [ 0.856799] nicpf, ver 1.01467worker2 # [ 0.857339] e1000: Intel(R) PRO/1000 Network Driver1468worker1 # [ 0.908684] Segment Routing with IPv61469worker1 # [ 0.908706] In-situ OAM (IOAM) with IPv61470worker2 # [ 0.857346] e1000: Copyright (c) 1999-2006 Intel Corporation.1471worker1 # [ 0.908739] NET: Registered PF_PACKET protocol family1472worker2 # [ 0.857375] e1000e: Intel(R) PRO/1000 Network Driver1473worker2 # [ 0.857383] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1474worker2 # [ 0.857420] igb: Intel(R) Gigabit Ethernet Network Driver1475worker2 # [ 0.857425] igb: Copyright (c) 2007-2014 Intel Corporation.1476worker2 # [ 0.857448] igbvf: Intel(R) Gigabit Virtual Function Network Driver1477worker1 # [ 0.910582] 9pnet: Installing 9P2000 support1478worker2 # [ 0.857453] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1479worker1 # [ 0.912716] Key type dns_resolver registered1480worker2 # [ 0.857590] sky2: driver version 1.301481worker2 # [ 0.859163] usbcore: registered new interface driver usb-storage1482worker2 # [ 0.859245] usbcore: registered new interface driver usbserial_generic1483worker2 # [ 0.859259] usbserial: USB Serial support registered for generic1484worker2 # [ 0.859844] hv_vmbus: registering driver hyperv_keyboard1485worker2 # [ 0.861076] ehci-pci 0000:00:0a.0: EHCI Host Controller1486worker2 # [ 0.861101] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11487lb # [ 0.867731] Freeing initrd memory: 26140K1488worker2 # [ 0.861398] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001489worker2 # [ 0.872781] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001490worker2 # [ 0.873117] hub 1-0:1.0: USB hub found1491worker1 # [ 0.919560] registered taskstats version 11492worker2 # [ 0.873132] hub 1-0:1.0: 6 ports detected1493client # [ 0.863754] Freeing initrd memory: 26160K1494worker1 # [ 0.919712] Loading compiled-in X.509 certificates1495worker2 # [ 0.876136] rtc-pl031 9010000.pl031: registered as rtc01496lb # [ 0.873640] tun: Universal TUN/TAP device driver, 1.61497worker2 # [ 0.876163] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T04:23:53 UTC (1789964633)1498worker2 # [ 0.878226] i2c_dev: i2c /dev entries driver1499client # [ 0.869735] tun: Universal TUN/TAP device driver, 1.61500lb # [ 0.877194] thunder_xcv, ver 1.01501worker1 # [ 0.928111] Demotion targets for Node 0: null1502lb # [ 0.877237] thunder_bgx, ver 1.01503lb # [ 0.877259] nicpf, ver 1.01504worker1 # [ 0.928217] Key type .fscrypt registered1505lb # [ 0.877799] e1000: Intel(R) PRO/1000 Network Driver1506worker1 # [ 0.928223] Key type fscrypt-provisioning registered1507lb # [ 0.877807] e1000: Copyright (c) 1999-2006 Intel Corporation.1508worker1 # [ 0.928314] ima: No TPM chip found, activating TPM-bypass!1509worker2 # [ 0.881443] sdhci: Secure Digital Host Controller Interface driver1510worker1 # [ 0.928333] ima: Allocated hash algorithm: sha11511lb # [ 0.877831] e1000e: Intel(R) PRO/1000 Network Driver1512worker2 # [ 0.881453] sdhci: Copyright(c) Pierre Ossman1513worker1 # [ 0.928352] ima: No architecture policies found1514lb # [ 0.877839] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1515worker2 # [ 0.881718] Synopsys Designware Multimedia Card Interface Driver1516lb # [ 0.877869] igb: Intel(R) Gigabit Ethernet Network Driver1517worker2 # [ 0.882088] sdhci-pltfm: SDHCI platform and OF driver helper1518worker1 # [ 0.932444] input: gpio-keys as /devices/platform/gpio-keys/input/input01519client # [ 0.873392] thunder_xcv, ver 1.01520lb # [ 0.877875] igb: Copyright (c) 2007-2014 Intel Corporation.1521worker2 # [ 0.883735] hid: raw HID events driver (C) Jiri Kosina1522client # [ 0.873435] thunder_bgx, ver 1.01523client # [ 0.873459] nicpf, ver 1.01524worker2 # [ 0.883974] usbcore: registered new interface driver usbhid1525lb # [ 0.877898] igbvf: Intel(R) Gigabit Virtual Function Network Driver1526worker2 # [ 0.883981] usbhid: USB HID core driver1527client # [ 0.874014] e1000: Intel(R) PRO/1000 Network Driver1528lb # [ 0.877904] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1529lb # [ 0.878034] sky2: driver version 1.301530client # [ 0.874021] e1000: Copyright (c) 1999-2006 Intel Corporation.1531client # [ 0.874045] e1000e: Intel(R) PRO/1000 Network Driver1532client # [ 0.874053] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1533client # [ 0.874082] igb: Intel(R) Gigabit Ethernet Network Driver1534client # [ 0.874088] igb: Copyright (c) 2007-2014 Intel Corporation.1535lb # [ 0.886618] usbcore: registered new interface driver usb-storage1536worker2 # [ 0.889414] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1537client # [ 0.874112] igbvf: Intel(R) Gigabit Virtual Function Network Driver1538lb # [ 0.886673] usbcore: registered new interface driver usbserial_generic1539worker2 # [ 0.890917] drop_monitor: Initializing network drop monitor service1540client # [ 0.874118] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1541lb # [ 0.886687] usbserial: USB Serial support registered for generic1542worker2 # [ 0.891054] NET: Registered PF_INET6 protocol family1543client # [ 0.874249] sky2: driver version 1.301544lb # [ 0.887711] ehci-pci 0000:00:0a.0: EHCI Host Controller1545client # [ 0.875815] usbcore: registered new interface driver usb-storage1546worker2 # [ 0.894028] Segment Routing with IPv61547worker2 # [ 0.894049] In-situ OAM (IOAM) with IPv61548lb # [ 0.887736] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11549client # [ 0.875863] usbcore: registered new interface driver usbserial_generic1550worker2 # [ 0.894077] NET: Registered PF_PACKET protocol family1551lb # [ 0.888015] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001552client # [ 0.875879] usbserial: USB Serial support registered for generic1553client # [ 0.876724] ehci-pci 0000:00:0a.0: EHCI Host Controller1554worker2 # [ 0.895668] 9pnet: Installing 9P2000 support1555lb # [ 0.892765] hv_vmbus: registering driver hyperv_keyboard1556client # [ 0.876751] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11557client # [ 0.876909] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001558lb # [ 0.894267] rtc-pl031 9010000.pl031: registered as rtc01559client # [ 0.888519] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001560lb # [ 0.894300] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T04:23:53 UTC (1789964633)1561client # [ 0.888866] hub 1-0:1.0: USB hub found1562lb # [ 0.894598] i2c_dev: i2c /dev entries driver1563client # [ 0.888881] hub 1-0:1.0: 6 ports detected1564worker2 # [ 0.898489] Key type dns_resolver registered1565client # [ 0.891417] hv_vmbus: registering driver hyperv_keyboard1566client # [ 0.892933] rtc-pl031 9010000.pl031: registered as rtc01567client # [ 0.892959] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T04:23:53 UTC (1789964633)1568client # [ 0.893265] i2c_dev: i2c /dev entries driver1569worker1 # [ 0.949703] clk: Disabling unused clocks1570worker1 # [ 0.949729] PM: genpd: Disabling unused power domains1571lb # [ 0.899521] sdhci: Secure Digital Host Controller Interface driver1572lb # [ 0.899532] sdhci: Copyright(c) Pierre Ossman1573lb # [ 0.899796] Synopsys Designware Multimedia Card Interface Driver1574worker1 # [ 0.953949] Freeing unused kernel memory: 4736K1575lb # [ 0.900160] sdhci-pltfm: SDHCI platform and OF driver helper1576worker2 # [ 0.904504] registered taskstats version 11577worker1 # [ 0.954152] Run /init as init process1578lb # [ 0.901810] hid: raw HID events driver (C) Jiri Kosina1579worker2 # [ 0.904641] Loading compiled-in X.509 certificates1580lb # [ 0.902042] usbcore: registered new interface driver usbhid1581lb # [ 0.902048] usbhid: USB HID core driver1582client # [ 0.898155] sdhci: Secure Digital Host Controller Interface driver1583client # [ 0.898165] sdhci: Copyright(c) Pierre Ossman1584client # [ 0.898429] Synopsys Designware Multimedia Card Interface Driver1585lb # [ 0.907489] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001586lb # [ 0.907804] hub 1-0:1.0: USB hub found1587client # [ 0.898806] sdhci-pltfm: SDHCI platform and OF driver helper1588lb # [ 0.907824] hub 1-0:1.0: 6 ports detected1589worker2 # [ 0.912951] Demotion targets for Node 0: null1590worker2 # [ 0.913051] Key type .fscrypt registered1591client # [ 0.903116] hid: raw HID events driver (C) Jiri Kosina1592worker2 # [ 0.913058] Key type fscrypt-provisioning registered1593client # [ 0.903390] usbcore: registered new interface driver usbhid1594lb # [ 0.909899] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1595worker2 # [ 0.913151] ima: No TPM chip found, activating TPM-bypass!1596client # [ 0.903396] usbhid: USB HID core driver1597worker2 # [ 0.913171] ima: Allocated hash algorithm: sha11598worker2 # [ 0.913191] ima: No architecture policies found1599lb # [ 0.912447] drop_monitor: Initializing network drop monitor service1600lb # [ 0.912579] NET: Registered PF_INET6 protocol family1601worker2 # [ 0.917060] input: gpio-keys as /devices/platform/gpio-keys/input/input01602lb # [ 0.914443] Segment Routing with IPv61603lb # [ 0.914465] In-situ OAM (IOAM) with IPv61604client # [ 0.906191] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1605lb # [ 0.914493] NET: Registered PF_PACKET protocol family1606client # [ 0.907672] drop_monitor: Initializing network drop monitor service1607client # [ 0.907793] NET: Registered PF_INET6 protocol family1608client # [ 0.910946] Segment Routing with IPv61609client # [ 0.910964] In-situ OAM (IOAM) with IPv61610client # [ 0.910992] NET: Registered PF_PACKET protocol family1611lb # [ 0.916151] 9pnet: Installing 9P2000 support1612lb # [ 0.918282] Key type dns_resolver registered1613worker1 # [ 0.971115] systemd[1]: Successfully made /usr/ read-only.1614client # [ 0.912844] 9pnet: Installing 9P2000 support1615client # [ 0.914923] Key type dns_resolver registered1616lb # [ 0.924948] registered taskstats version 11617lb # [ 0.925105] Loading compiled-in X.509 certificates1618client # [ 0.921788] registered taskstats version 11619client # [ 0.921937] Loading compiled-in X.509 certificates1620worker2 # [ 0.934184] clk: Disabling unused clocks1621worker2 # [ 0.934204] PM: genpd: Disabling unused power domains1622lb # [ 0.933818] Demotion targets for Node 0: null1623lb # [ 0.933919] Key type .fscrypt registered1624lb # [ 0.933925] Key type fscrypt-provisioning registered1625worker2 # [ 0.938350] Freeing unused kernel memory: 4736K1626worker2 # [ 0.938547] Run /init as init process1627lb # [ 0.934026] ima: No TPM chip found, activating TPM-bypass!1628lb # [ 0.934046] ima: Allocated hash algorithm: sha11629lb # [ 0.934067] ima: No architecture policies found1630client # [ 0.930464] Demotion targets for Node 0: null1631client # [ 0.930569] Key type .fscrypt registered1632lb # [ 0.937990] input: gpio-keys as /devices/platform/gpio-keys/input/input01633client # [ 0.930576] Key type fscrypt-provisioning registered1634client # [ 0.930668] ima: No TPM chip found, activating TPM-bypass!1635client # [ 0.930687] ima: Allocated hash algorithm: sha11636client # [ 0.930707] ima: No architecture policies found1637client # [ 0.934713] input: gpio-keys as /devices/platform/gpio-keys/input/input01638worker2 # [ 0.954490] systemd[1]: Successfully made /usr/ read-only.1639lb # [ 0.955882] clk: Disabling unused clocks1640lb # [ 0.955912] PM: genpd: Disabling unused power domains1641lb # [ 0.960145] Freeing unused kernel memory: 4736K1642lb # [ 0.960368] Run /init as init process1643client # [ 0.952919] clk: Disabling unused clocks1644client # [ 0.952946] PM: genpd: Disabling unused power domains1645client # [ 0.957173] Freeing unused kernel memory: 4736K1646client # [ 0.957364] Run /init as init process1647lb # [ 0.977180] systemd[1]: Successfully made /usr/ read-only.1648client # [ 0.974115] systemd[1]: Successfully made /usr/ read-only.1649worker1 # [ 1.133077] usb 1-1: new high-speed USB device number 2 using ehci-pci1650worker2 # [ 1.120373] usb 1-1: new high-speed USB device number 2 using ehci-pci1651client # [ 1.136101] usb 1-1: new high-speed USB device number 2 using ehci-pci1652lb # [ 1.155107] usb 1-1: new high-speed USB device number 2 using ehci-pci1653worker1 # [ 1.285392] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11654worker1 # [ 1.305872] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1655worker1 # [ 1.317072] systemd[1]: Detected virtualization qemu.1656worker1 # [ 1.318946] systemd[1]: Detected architecture arm64.1657worker2 # [ 1.270552] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11658worker1 # [ 1.320773] systemd[1]: Running in initrd.1659worker1 # [ 1.323208] systemd[1]: Initializing machine ID from random generator.1660worker1 # [ 1.325839] systemd[1]: Hostname set to <worker1>.1661worker2 # [ 1.289225] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1662worker2 # [ 1.300437] systemd[1]: Detected virtualization qemu.1663client # [ 1.288150] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11664worker2 # [ 1.302308] systemd[1]: Detected architecture arm64.1665worker2 # [ 1.304090] systemd[1]: Running in initrd.1666worker2 # [ 1.306553] systemd[1]: Initializing machine ID from random generator.1667worker2 # [ 1.309257] systemd[1]: Hostname set to <worker2>.1668lb # [ 1.307323] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11669lb # [ 1.312880] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1670client # [ 1.308926] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1671worker1 # [ 1.373256] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01672lb # [ 1.325024] systemd[1]: Detected virtualization qemu.1673client # [ 1.320893] systemd[1]: Detected virtualization qemu.1674lb # [ 1.325110] systemd[1]: Detected architecture arm64.1675client # [ 1.322877] systemd[1]: Detected architecture arm64.1676lb # [ 1.325134] systemd[1]: Running in initrd.1677client # [ 1.324830] systemd[1]: Running in initrd.1678lb # [ 1.326052] systemd[1]: Initializing machine ID from random generator.1679lb # [ 1.326326] systemd[1]: Hostname set to <lb>.1680client # [ 1.327371] systemd[1]: Initializing machine ID from random generator.1681client # [ 1.330341] systemd[1]: Hostname set to <client>.1682worker2 # [ 1.352599] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683client # [ 1.376286] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01684lb # [ 1.391294] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01685worker1 # [ 1.493090] usb 1-2: new high-speed USB device number 3 using ehci-pci1686worker2 # [ 1.472374] usb 1-2: new high-speed USB device number 3 using ehci-pci1687client # [ 1.496090] usb 1-2: new high-speed USB device number 3 using ehci-pci1688lb # [ 1.511076] usb 1-2: new high-speed USB device number 3 using ehci-pci1689worker1 # [ 1.621500] systemd[1]: bpf-restrict-fs: LSM BPF program attached1690worker1 # [ 1.653604] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21691worker1 # [ 1.658676] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01692worker2 # [ 1.630567] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21693worker2 # [ 1.636112] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694lb # [ 1.636609] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695worker2 # [ 1.640776] systemd[1]: bpf-restrict-fs: LSM BPF program attached1696client # [ 1.630884] systemd[1]: bpf-restrict-fs: LSM BPF program attached1697client # [ 1.657229] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21698client # [ 1.661491] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01699lb # [ 1.674283] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21700lb # [ 1.678994] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01701worker1 # [ 1.737512] systemd[1]: Queued start job for default target Initrd Default Target.1702worker1 # [ 1.747493] systemd[1]: Created slice Slice /system/modprobe.1703worker1 # [ 1.748716] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1704worker1 # [ 1.750171] systemd[1]: Expecting device /dev/disk/by-label/nixos...1705worker1 # [ 1.751265] systemd[1]: Reached target Path Units.1706worker1 # [ 1.752111] systemd[1]: Reached target Slice Units.1707worker1 # [ 1.752966] systemd[1]: Reached target Swaps.1708worker1 # [ 1.753803] systemd[1]: Reached target Timer Units.1709worker1 # [ 1.754834] systemd[1]: Listening on D-Bus System Message Bus Socket.1710worker1 # [ 1.756088] systemd[1]: Listening on Journal Socket (/dev/log).1711worker1 # [ 1.757268] systemd[1]: Listening on Journal Sockets.1712worker1 # [ 1.758278] systemd[1]: Listening on udev Control Socket.1713worker1 # [ 1.759428] systemd[1]: Listening on udev Kernel Socket.1714worker1 # [ 1.760366] systemd[1]: Reached target Socket Units.1715worker1 # [ 1.762883] systemd[1]: Starting Create List of Static Device Nodes...1716worker1 # [ 1.773178] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1717worker1 # [ 1.773265] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1718worker2 # [ 1.733010] systemd[1]: Queued start job for default target Initrd Default Target.1719worker1 # [ 1.777642] systemd[1]: Mounting Kernel Configuration File System...1720worker2 # [ 1.742583] systemd[1]: Created slice Slice /system/modprobe.1721worker1 # [ 1.789210] systemd[1]: Starting Journal Service...1722worker2 # [ 1.743708] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1723worker2 # [ 1.745029] systemd[1]: Expecting device /dev/disk/by-label/nixos...1724worker2 # [ 1.746026] systemd[1]: Reached target Path Units.1725worker2 # [ 1.746793] systemd[1]: Reached target Slice Units.1726worker2 # [ 1.747573] systemd[1]: Reached target Swaps.1727worker2 # [ 1.748297] systemd[1]: Reached target Timer Units.1728worker2 # [ 1.749280] systemd[1]: Listening on D-Bus System Message Bus Socket.1729worker2 # [ 1.750514] systemd[1]: Listening on Journal Socket (/dev/log).1730worker2 # [ 1.751573] systemd[1]: Listening on Journal Sockets.1731client # [ 1.741665] systemd[1]: Queued start job for default target Initrd Default Target.1732worker2 # [ 1.752541] systemd[1]: Listening on udev Control Socket.1733worker2 # [ 1.752652] systemd[1]: Listening on udev Kernel Socket.1734worker2 # [ 1.752676] systemd[1]: Reached target Socket Units.1735lb # [ 1.752765] systemd[1]: Queued start job for default target Initrd Default Target.1736worker2 # [ 1.756601] systemd[1]: Starting Create List of Static Device Nodes...1737client # [ 1.750421] systemd[1]: Created slice Slice /system/modprobe.1738client # [ 1.751714] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1739worker2 # [ 1.764505] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1740lb # [ 1.761423] systemd[1]: Created slice Slice /system/modprobe.1741client # [ 1.753285] systemd[1]: Expecting device /dev/disk/by-label/nixos...1742worker1 # [ 1.811790] systemd[1]: Starting Load Kernel Modules...1743worker2 # [ 1.765604] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1744lb # [ 1.762670] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1745client # [ 1.753324] systemd[1]: Reached target Path Units.1746worker1 # [ 1.812711] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1747client # [ 1.753346] systemd[1]: Reached target Slice Units.1748lb # [ 1.764082] systemd[1]: Expecting device /dev/disk/by-label/nixos...1749client # [ 1.753368] systemd[1]: Reached target Swaps.1750lb # [ 1.765178] systemd[1]: Reached target Path Units.1751client # [ 1.753388] systemd[1]: Reached target Timer Units.1752lb # [ 1.766036] systemd[1]: Reached target Slice Units.1753client # [ 1.753585] systemd[1]: Listening on D-Bus System Message Bus Socket.1754lb # [ 1.766904] systemd[1]: Reached target Swaps.1755client # [ 1.753747] systemd[1]: Listening on Journal Socket (/dev/log).1756lb # [ 1.767845] systemd[1]: Reached target Timer Units.1757client # [ 1.753895] systemd[1]: Listening on Journal Sockets.1758lb # [ 1.768895] systemd[1]: Listening on D-Bus System Message Bus Socket.1759worker2 # [ 1.772565] systemd[1]: Mounting Kernel Configuration File System...1760client # [ 1.754046] systemd[1]: Listening on udev Control Socket.1761lb # [ 1.770162] systemd[1]: Listening on Journal Socket (/dev/log).1762client # [ 1.754165] systemd[1]: Listening on udev Kernel Socket.1763client # [ 1.754190] systemd[1]: Reached target Socket Units.1764client # [ 1.765880] systemd[1]: Starting Create List of Static Device Nodes...1765lb # [ 1.771374] systemd[1]: Listening on Journal Sockets.1766lb # [ 1.771520] systemd[1]: Listening on udev Control Socket.1767lb # [ 1.771640] systemd[1]: Listening on udev Kernel Socket.1768lb # [ 1.771664] systemd[1]: Reached target Socket Units.1769lb # [ 1.776701] systemd[1]: Starting Create List of Static Device Nodes...1770client # [ 1.772908] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1771client # [ 1.774145] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1772worker1 # [ 1.833163] systemd[1]: Starting Coldplug All udev Devices...1773worker1 # [ 1.836389] systemd[1]: Finished Create List of Static Device Nodes.1774lb # [ 1.787207] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1775lb # [ 1.788391] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1776client # [ 1.784177] systemd[1]: Mounting Kernel Configuration File System...1777worker1 # [ 1.839400] systemd-journald[73]: Collecting audit messages is disabled.1778worker1 # [ 1.840232] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1779lb # [ 1.791791] systemd[1]: Mounting Kernel Configuration File System...1780worker2 # [ 1.802760] systemd[1]: Starting Journal Service...1781client # [ 1.793465] systemd[1]: Starting Journal Service...1782worker1 # [ 1.853208] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1783worker1 # [ 1.853552] systemd[1]: Mounted Kernel Configuration File System.1784worker2 # [ 1.809417] systemd[1]: Starting Load Kernel Modules...1785worker2 # [ 1.810303] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1786worker1 # [ 1.857659] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1787lb # [ 1.811540] systemd[1]: Starting Journal Service...1788worker2 # [ 1.816215] systemd[1]: Starting Coldplug All udev Devices...1789lb # [ 1.824713] systemd[1]: Starting Load Kernel Modules...1790lb # [ 1.825675] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1791worker2 # [ 1.828812] systemd[1]: Finished Create List of Static Device Nodes.1792worker2 # [ 1.829876] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1793client # [ 1.821328] systemd[1]: Starting Load Kernel Modules...1794client # [ 1.822238] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1795lb # [ 1.838550] systemd[1]: Starting Coldplug All udev Devices...1796worker2 # [ 1.840556] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1797worker2 # [ 1.840944] systemd[1]: Mounted Kernel Configuration File System.1798worker2 # [ 1.846152] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1799worker1 # [ 1.893626] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1800worker1 # [ 1.899755] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1801client # [ 1.844775] systemd[1]: Starting Coldplug All udev Devices...1802lb # [ 1.851369] systemd[1]: Finished Create List of Static Device Nodes.1803lb # [ 1.852245] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1804lb # [ 1.852533] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1805lb # [ 1.852753] systemd[1]: Mounted Kernel Configuration File System.1806worker1 # [ 1.901807] systemd[1]: Starting Create Static Device Nodes in /dev...1807client # [ 1.855115] systemd-journald[73]: Collecting audit messages is disabled.1808worker2 # [ 1.866137] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1809lb # [ 1.863751] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1810worker1 # [ 1.913103] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1811client # [ 1.860230] systemd[1]: Finished Create List of Static Device Nodes.1812client # [ 1.861172] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1813worker1 # [ 1.927027] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01814client # [ 1.869711] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1815worker1 # [ 1.927267] [drm] features: -virgl +edid -resource_blob -host_visible1816worker2 # [ 1.883896] systemd-journald[73]: Collecting audit messages is disabled.1817client # [ 1.873786] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1818client # [ 1.874133] systemd[1]: Mounted Kernel Configuration File System.1819worker1 # [ 1.927276] [drm] features: -context_init1820worker1 # [ 1.927987] [drm] number of scanouts: 11821worker1 # [ 1.928005] [drm] number of cap sets: 01822worker2 # [ 1.887447] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1823worker1 # [ 1.938716] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1824worker1 # [ 1.938738] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01825worker2 # [ 1.896138] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01826client # [ 1.884536] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1827lb # [ 1.893656] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1828worker2 # [ 1.900449] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1829worker2 # [ 1.902913] systemd[1]: Starting Create Static Device Nodes in /dev...1830lb # [ 1.902461] systemd-journald[73]: Collecting audit messages is disabled.1831worker2 # [ 1.912554] [drm] features: -virgl +edid -resource_blob -host_visible1832worker2 # [ 1.912572] [drm] features: -context_init1833worker2 # [ 1.913418] [drm] number of scanouts: 11834worker2 # [ 1.913436] [drm] number of cap sets: 01835client # [ 1.900437] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1836lb # [ 1.911868] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1837client # [ 1.905140] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01838worker1 # [ 1.961746] systemd[1]: Finished Create Static Device Nodes in /dev.1839worker1 # [ 1.962116] systemd[1]: Reached target Preparation for Local File Systems.1840client # [ 1.905370] [drm] features: -virgl +edid -resource_blob -host_visible1841client # [ 1.905380] [drm] features: -context_init1842lb # [ 1.915099] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1843worker1 # [ 1.962144] systemd[1]: Reached target Local File Systems.1844client # [ 1.906080] [drm] number of scanouts: 11845client # [ 1.906098] [drm] number of cap sets: 01846worker1 # [ 1.965953] systemd[1]: Starting Rule-based Manager for Device Events and Files...1847lb # [ 1.917309] systemd[1]: Starting Create Static Device Nodes in /dev...1848worker2 # [ 1.920765] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1849worker2 # [ 1.920782] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01850lb # [ 1.930816] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01851worker1 # [ 1.977280] Console: switching to colour frame buffer device 160x501852client # [ 1.924646] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1853client # [ 1.926180] systemd[1]: Starting Create Static Device Nodes in /dev...1854client # [ 1.931709] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1855client # [ 1.931731] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01856lb # [ 1.947129] [drm] features: -virgl +edid -resource_blob -host_visible1857worker1 # [ 1.997621] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1858lb # [ 1.947148] [drm] features: -context_init1859lb # [ 1.947905] [drm] number of scanouts: 11860lb # [ 1.947924] [drm] number of cap sets: 01861worker2 # [ 1.948601] Console: switching to colour frame buffer device 160x501862worker1 # [ 2.006416] systemd[1]: Finished Load Kernel Modules.1863lb # [ 1.950964] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1864lb # [ 1.950982] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01865worker2 # [ 1.964975] systemd[1]: Finished Create Static Device Nodes in /dev.1866worker2 # [ 1.965149] systemd[1]: Reached target Preparation for Local File Systems.1867worker2 # [ 1.965175] systemd[1]: Reached target Local File Systems.1868worker2 # [ 1.968985] systemd[1]: Starting Rule-based Manager for Device Events and Files...1869client # [ 1.960919] Console: switching to colour frame buffer device 160x501870worker1 # [ 2.021463] systemd[1]: Starting Apply Kernel Variables...1871lb # [ 1.975630] systemd[1]: Finished Create Static Device Nodes in /dev.1872lb # [ 1.975827] systemd[1]: Reached target Preparation for Local File Systems.1873lb # [ 1.975854] systemd[1]: Reached target Local File Systems.1874worker2 # [ 1.983281] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1875worker1 # [ 2.017860] systemd-modules-load[74]: Inserted module 'dm_mod'1876lb # [ 1.983412] systemd[1]: Starting Rule-based Manager for Device Events and Files...1877worker1 # [ 2.035984] systemd[1]: Started Journal Service.1878worker1 # [ 2.024266] systemd-modules-load[74]: Module 'virtio_balloon' is built in1879worker1 # [ 2.025380] systemd-modules-load[74]: Module 'virtio_console' is built in1880worker2 # [ 1.996550] systemd[1]: Finished Load Kernel Modules.1881worker1 # [ 2.026436] systemd-modules-load[74]: Inserted module 'virtio_gpu'1882worker1 # [ 2.027404] systemd-modules-load[74]: Module 'virtio_rng' is built in1883lb # [ 1.988362] Console: switching to colour frame buffer device 160x501884lb # [ 1.994846] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1885client # [ 1.988644] systemd[1]: Finished Create Static Device Nodes in /dev.1886client # [ 1.989020] systemd[1]: Reached target Preparation for Local File Systems.1887worker2 # [ 2.000635] systemd[1]: Starting Apply Kernel Variables...1888client # [ 1.989047] systemd[1]: Reached target Local File Systems.1889client # [ 1.993356] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1890worker1 # [ 2.048506] systemd[1]: Starting Create System Files and Directories...1891lb # [ 2.019725] systemd[1]: Finished Load Kernel Modules.1892client # [ 2.012175] systemd[1]: Starting Rule-based Manager for Device Events and Files...1893client # [ 2.012704] systemd[1]: Finished Load Kernel Modules.1894client # [ 2.016749] systemd[1]: Starting Apply Kernel Variables...1895lb # [ 2.027438] systemd[1]: Starting Apply Kernel Variables...1896worker1 # [ 2.064970] systemd-udevd[80]: Using default interface naming scheme 'v261'.1897worker1 # [ 2.077674] systemd[1]: Finished Apply Kernel Variables.1898client # [ 2.028916] systemd-modules-load[75]: Inserted module 'dm_mod'1899client # [ 2.046017] systemd[1]: Started Journal Service.1900worker2 # [ 2.056992] systemd[1]: Finished Apply Kernel Variables.1901client # [ 2.033195] systemd-modules-load[75]: Module 'virtio_balloon' is built in1902worker1 # [ 2.090114] systemd[1]: Finished Create System Files and Directories.1903worker2 # [ 2.066796] systemd[1]: Started Journal Service.1904client # [ 2.043345] systemd-modules-load[75]: Module 'virtio_console' is built in1905worker1 # [ 2.106134] systemd[1]: Started Rule-based Manager for Device Events and Files.1906client # [ 2.052474] systemd-modules-load[75]: Inserted module 'virtio_gpu'1907client # [ 2.053502] systemd-modules-load[75]: Module 'virtio_rng' is built in1908client # [ 2.054536] systemd[1]: Starting Create System Files and Directories...1909worker2 # [ 2.064285] systemd-modules-load[74]: Inserted module 'dm_mod'1910worker2 # [ 2.066923] systemd-modules-load[74]: Module 'virtio_balloon' is built in1911worker2 # [ 2.068156] systemd-modules-load[74]: Module 'virtio_console' is built in1912worker2 # [ 2.071045] systemd-modules-load[74]: Inserted module 'virtio_gpu'1913lb # [ 2.082893] systemd[1]: Finished Apply Kernel Variables.1914client # [ 2.068315] systemd[1]: Finished Apply Kernel Variables.1915worker2 # [ 2.080723] systemd-modules-load[74]: Module 'virtio_rng' is built in1916worker2 # [ 2.082423] systemd[1]: Starting Create System Files and Directories...1917worker2 # [ 2.083743] systemd-udevd[79]: Using default interface naming scheme 'v261'.1918lb # [ 2.095144] systemd[1]: Started Journal Service.1919lb # [ 2.088354] systemd-modules-load[74]: Inserted module 'dm_mod'1920lb # [ 2.089610] systemd-modules-load[74]: Module 'virtio_balloon' is built in1921lb # [ 2.090675] systemd-modules-load[74]: Module 'virtio_console' is built in1922lb # [ 2.091696] systemd-modules-load[74]: Inserted module 'virtio_gpu'1923client # [ 2.086256] systemd[1]: Finished Create System Files and Directories.1924lb # [ 2.097888] systemd-modules-load[74]: Module 'virtio_rng' is built in1925worker2 # [ 2.100198] systemd[1]: Finished Create System Files and Directories.1926lb # [ 2.106038] systemd-udevd[79]: Using default interface naming scheme 'v261'.1927client # [ 2.100426] systemd-udevd[80]: Using default interface naming scheme 'v261'.1928lb # [ 2.108212] systemd[1]: Starting Create System Files and Directories...1929worker2 # [ 2.116758] systemd[1]: Started Rule-based Manager for Device Events and Files.1930worker1 # [ 2.164278] systemd[1]: Starting Virtual Console Setup...1931lb # [ 2.132237] systemd[1]: Finished Create System Files and Directories.1932client # [ 2.125374] systemd[1]: Started Rule-based Manager for Device Events and Files.1933lb # [ 2.145259] systemd[1]: Started Rule-based Manager for Device Events and Files.1934worker1 # [ 2.216448] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1935worker2 # [ 2.172130] systemd[1]: Starting Virtual Console Setup...1936worker1 # [ 2.219868] systemd[1]: Finished Virtual Console Setup.1937client # [ 2.184804] systemd[1]: Starting Virtual Console Setup...1938lb # [ 2.203858] systemd[1]: Starting Virtual Console Setup...1939worker2 # [ 2.216459] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940worker2 # [ 2.219738] systemd[1]: Finished Virtual Console Setup.1941client # [ 2.240450] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942lb # [ 2.248435] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1943client # [ 2.243799] systemd[1]: Finished Virtual Console Setup.1944lb # [ 2.251837] systemd[1]: Finished Virtual Console Setup.1945worker1 # [ 2.801713] systemd[1]: Finished Coldplug All udev Devices.1946worker1 # [ 2.804178] systemd[1]: Reached target System Initialization.1947worker1 # [ 2.804997] systemd[1]: Reached target Basic System.1948worker2 # [ 2.817532] systemd[1]: Finished Coldplug All udev Devices.1949worker2 # [ 2.818475] systemd[1]: Reached target System Initialization.1950worker2 # [ 2.819254] systemd[1]: Reached target Basic System.1951client # [ 2.839254] systemd[1]: Finished Coldplug All udev Devices.1952client # [ 2.840469] systemd[1]: Reached target System Initialization.1953client # [ 2.844349] systemd[1]: Reached target Basic System.1954lb # [ 2.854120] systemd[1]: Finished Coldplug All udev Devices.1955lb # [ 2.855061] systemd[1]: Reached target System Initialization.1956lb # [ 2.855868] systemd[1]: Reached target Basic System.1957worker1 # [ 2.939467] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1958worker1 # [ 2.976414] (udev-worker)[106]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1959worker1 # [ 2.981074] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1960worker2 # [ 2.936105] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1961lb # [ 2.975894] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1962worker1 # [ 3.036587] systemd[1]: Found device /dev/disk/by-label/nixos.1963worker1 # [ 3.039367] systemd[1]: Reached target Initrd Root Device.1964worker2 # [ 2.997755] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1965worker1 # [ 3.044163] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1966worker2 # [ 3.004184] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1967client # [ 3.015202] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1968client # [ 3.020454] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969client # [ 3.024611] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1970lb # [ 3.034285] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1971lb # [ 3.040259] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1972worker1 # [ 3.094683] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1973worker2 # [ 3.052875] systemd[1]: Found device /dev/disk/by-label/nixos.1974worker1 # [ 3.101116] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1975worker2 # [ 3.059785] systemd[1]: Reached target Initrd Root Device.1976worker1 # [ 3.109271] systemd[1]: Mounting /sysroot...1977worker2 # [ 3.067887] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1978client # [ 3.084834] systemd[1]: Found device /dev/disk/by-label/nixos.1979client # [ 3.087405] systemd[1]: Reached target Initrd Root Device.1980worker1 # [ 3.159987] EXT4-fs (vda): mounted filesystem a505a470-475f-4e3c-be79-dfe4ace5a1ec r/w with ordered data mode. Quota mode: none.1981client # [ 3.089787] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982worker1 # [ 3.149686] systemd[1]: Mounted /sysroot.1983worker1 # [ 3.151489] systemd[1]: Reached target Initrd Root File System.1984lb # [ 3.110251] systemd[1]: Found device /dev/disk/by-label/nixos.1985worker1 # [ 3.160008] systemd[1]: Starting Mountpoints Configured in the Real Root...1986lb # [ 3.114126] systemd[1]: Reached target Initrd Root Device.1987worker2 # [ 3.117729] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1988lb # [ 3.117843] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989worker2 # [ 3.124103] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1990worker2 # [ 3.127306] systemd[1]: Mounting /sysroot...1991worker1 # [ 3.176363] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1992worker1 # [ 3.183004] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1993worker1 # [ 3.188813] systemd[1]: Reloading...1994client # [ 3.143825] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1995lb # [ 3.157938] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1996client # [ 3.151256] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1997worker2 # [ 3.179003] EXT4-fs (vda): mounted filesystem d99f2033-5ec8-4d15-85ec-bb532d39d4c9 r/w with ordered data mode. Quota mode: none.1998worker2 # [ 3.167669] systemd[1]: Mounted /sysroot.1999lb # [ 3.166199] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.2000lb # [ 3.168133] systemd[1]: Mounting /sysroot...
loading 1,541 hidden lines…
3542lb # [ 19.174414] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3543lb # [ 19.178443] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3544lb # [ 19.187478] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3545lb # [ 19.197246] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3546lb # [ 19.202770] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response3547lb # [ 19.208291] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3548lb # [ 19.215239] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3549lb # [ 19.219672] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3550lb # [ 19.224549] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23551lb # [ 19.231287] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3552lb # [ 19.234193] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3553lb # [ 19.236892] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3554lb # [ 19.238961] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3555lb # [ 19.241700] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3556lb # [ 19.252327] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3557lb # [ 19.262311] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3558lb # [ 19.266822] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls3559lb # [ 19.278437] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr3560worker2 # [ 19.292826] systemd[1]: Reached target Network is Online.3561lb # [ 19.289623] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3562lb # [ 19.296876] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3563worker1 # [ 19.344901] systemd[1]: Starting Nix worker-protocol over gRPC...3564lb # [ 19.299840] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3565worker2 # [ 19.302704] systemd[1]: Starting scheduler-token.service...3566lb # [ 19.304667] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3567lb # [ 19.308762] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector3568lb # [ 19.316339] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23569lb # [ 19.319054] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default3570lb # [ 19.329115] envoy[902]: [2026-09-21 04:24:11.469][902][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3571lb # [ 19.332907] envoy[902]: [2026-09-21 04:24:11.506][902][info][main] [source/server/server.cc:531] HTTP header map info:3572lb # [ 19.334613] envoy[902]: [2026-09-21 04:24:11.508][902][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3573lb # [ 19.352407] envoy[902]: [2026-09-21 04:24:11.508][902][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3574lb # [ 19.354189] envoy[902]: [2026-09-21 04:24:11.508][902][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3575lb # [ 19.363767] envoy[902]: [2026-09-21 04:24:11.510][902][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3576lb # [ 19.366342] envoy[902]: [2026-09-21 04:24:11.729][902][info][main] [source/server/server.cc:968] runtime: {}3577lb # [ 19.368206] envoy[902]: [2026-09-21 04:24:11.767][902][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3578lb # [ 19.370194] envoy[902]: [2026-09-21 04:24:11.767][902][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013579lb # [ 19.372178] envoy[902]: [2026-09-21 04:24:11.768][902][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013580lb # [ 19.373902] envoy[902]: [2026-09-21 04:24:11.775][902][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3581lb # [ 19.375694] envoy[902]: [2026-09-21 04:24:11.775][902][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3582lb # [ 19.379037] envoy[902]: [2026-09-21 04:24:11.775][902][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3583lb # [ 19.381125] envoy[902]: [2026-09-21 04:24:11.908][902][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3584lb # [ 19.383117] envoy[902]: [2026-09-21 04:24:11.945][902][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3585lb # [ 19.385353] envoy[902]: [2026-09-21 04:24:11.946][902][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3586lb # [ 19.389037] envoy[902]: [2026-09-21 04:24:11.946][902][info][main] [source/server/server.cc:1077] starting main dispatch loop3587worker2 # [ 19.413081] systemd[1]: scheduler-token.service: Deactivated successfully.3588worker2 # [ 19.416878] systemd[1]: Finished scheduler-token.service.3589worker2 # [ 19.418363] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 113ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3590worker2 # [ 19.427362] systemd[1]: Starting Nix worker-protocol over gRPC...3591worker1 # [ 19.589098] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3592worker1 # [ 19.600999] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3593worker1 # [ 19.603454] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13594worker1 # [ 19.648073] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3595worker1 # [ 19.652260] systemd[1]: Started Nix worker-protocol over gRPC.3596lb # [ 19.641626] envoy[902]: [2026-09-21 04:24:12.261][902][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3597worker2 # [ 19.647035] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3598lb # [ 19.648522] envoy[902]: [2026-09-21 04:24:12.261][902][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3599worker2 # [ 19.658857] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3600worker2 # [ 19.661195] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13601lb # [ 19.659246] envoy[902]: [2026-09-21 04:24:12.261][902][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3602worker1 # [ 19.715425] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=scheduler_connected addr=lb:500513603lb # [ 19.666708] envoy[902]: [2026-09-21 04:24:12.261][902][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3604worker1 # [ 19.725040] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3605worker2 # [ 19.710482] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3606worker2 # [ 19.723658] systemd[1]: Started Nix worker-protocol over gRPC.3607worker2 # [ 19.773914] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=scheduler_connected addr=lb:500513608worker2 # [ 19.775663] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3609worker1 # [ 19.926356] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=scheduler_connected addr=lb:500513610worker1 # [ 19.931595] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3611worker2 # [ 19.977285] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=scheduler_connected addr=lb:500513612worker2 # [ 19.982423] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3613worker1 # [ 20.132404] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=scheduler_connected addr=lb:500513614worker1 # [ 20.137123] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3615worker2 # [ 20.183625] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=scheduler_connected addr=lb:500513616worker2 # [ 20.188825] nix-grpc-daemon[776]: ts=2026-09-21T04:24:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3617worker1 # [ 20.338136] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=scheduler_connected addr=lb:500513618worker1 # [ 20.343148] nix-grpc-daemon[777]: ts=2026-09-21T04:24:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3619worker2 # [ 20.390050] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513620worker2 # [ 20.395202] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3621worker1 # [ 20.543945] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513622worker1 # [ 20.549104] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3623worker2 # [ 20.596477] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513624worker2 # [ 20.601451] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3625worker1 # [ 20.751685] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513626worker1 # [ 20.755839] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3627worker2 # [ 20.804209] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513628worker2 # [ 20.808195] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3629worker1 # [ 20.952317] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513630worker1 # [ 20.957292] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3631worker2 # [ 21.004922] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513632worker2 # [ 21.009886] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3633worker1 # [ 21.158173] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513634worker1 # [ 21.163246] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3635worker2 # [ 21.211127] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513636worker2 # [ 21.216353] nix-grpc-daemon[776]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3637worker1 # [ 21.364118] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_connected addr=lb:500513638worker1 # [ 21.369196] nix-grpc-daemon[777]: ts=2026-09-21T04:24:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3639worker2 # [ 21.417496] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_connected addr=lb:500513640worker2 # [ 21.422811] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3641worker1 # [ 21.570165] nix-grpc-daemon[777]: ts=2026-09-21T04:24:14Z level=info event=scheduler_connected addr=lb:500513642worker1 # [ 21.575209] nix-grpc-daemon[777]: ts=2026-09-21T04:24:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3643worker2 # [ 21.624137] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_connected addr=lb:500513644worker2 # [ 21.629433] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3645worker1 # [ 21.777679] nix-grpc-daemon[777]: ts=2026-09-21T04:24:14Z level=info event=scheduler_connected addr=lb:500513646worker1 # [ 21.781764] nix-grpc-daemon[777]: ts=2026-09-21T04:24:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3647worker2 # [ 21.832322] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_connected addr=lb:500513648worker2 # [ 21.836261] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3649worker1 # [ 21.978371] nix-grpc-daemon[777]: ts=2026-09-21T04:24:14Z level=info event=scheduler_connected addr=lb:500513650worker1 # [ 21.983912] nix-grpc-daemon[777]: ts=2026-09-21T04:24:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3651worker2 # [ 22.033035] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_connected addr=lb:500513652worker2 # [ 22.038164] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3653worker1 # [ 22.184780] nix-grpc-daemon[777]: ts=2026-09-21T04:24:14Z level=info event=scheduler_connected addr=lb:500513654worker1 # [ 22.189832] nix-grpc-daemon[777]: ts=2026-09-21T04:24:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3655worker2 # [ 22.239321] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_connected addr=lb:500513656worker2 # [ 22.244803] nix-grpc-daemon[776]: ts=2026-09-21T04:24:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3657worker1 # [ 22.390565] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513658worker1 # [ 22.395382] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3659worker2 # [ 22.446016] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513660worker2 # [ 22.451355] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3661worker1 # [ 22.596185] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513662worker1 # [ 22.600902] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3663worker2 # [ 22.652703] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513664worker2 # [ 22.657819] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3665worker1 # [ 22.803464] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513666worker1 # [ 22.807629] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3667worker2 # [ 22.860675] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513668worker2 # [ 22.864792] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3669worker1 # [ 23.003572] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513670worker1 # [ 23.008867] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3671worker2 # [ 23.061541] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513672worker2 # [ 23.066869] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3673worker1 # [ 23.210084] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513674worker1 # [ 23.215478] nix-grpc-daemon[777]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3675worker2 # [ 23.268958] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_connected addr=lb:500513676worker2 # [ 23.275011] nix-grpc-daemon[776]: ts=2026-09-21T04:24:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3677worker1 # [ 23.417176] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513678worker1 # [ 23.422766] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3679worker2 # [ 23.476273] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513680worker2 # [ 23.481739] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3681worker1 # [ 23.623770] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513682worker1 # [ 23.629348] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3683worker2 # [ 23.684450] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513684worker2 # [ 23.690605] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3685worker1 # [ 23.832562] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513686worker1 # [ 23.838000] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3687worker2 # [ 23.896214] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513688worker2 # [ 23.900577] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3689worker1 # [ 24.039811] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513690worker1 # [ 24.045738] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3691worker2 # [ 24.096971] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513692worker2 # [ 24.102870] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3693worker1 # [ 24.246916] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513694worker1 # [ 24.253916] nix-grpc-daemon[777]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3695worker2 # [ 24.304789] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_connected addr=lb:500513696worker2 # [ 24.310414] nix-grpc-daemon[776]: ts=2026-09-21T04:24:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3697worker1 # [ 24.455151] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513698worker1 # [ 24.461376] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3699worker2 # [ 24.511807] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513700worker2 # [ 24.518423] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3701worker1 # [ 24.662990] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513702worker1 # [ 24.669343] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3703worker2 # [ 24.720368] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513704worker2 # [ 24.726242] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3705worker1 # [ 24.875481] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513706worker1 # [ 24.880188] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3707worker2 # [ 24.932853] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513708worker2 # [ 24.937111] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3709worker1 # [ 25.074998] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513710worker1 # [ 25.081144] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3711worker2 # [ 25.134085] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513712worker2 # [ 25.140258] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3713worker1 # [ 25.282821] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513714worker1 # [ 25.288798] nix-grpc-daemon[777]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3715worker2 # [ 25.342267] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_connected addr=lb:500513716worker2 # [ 25.348605] nix-grpc-daemon[776]: ts=2026-09-21T04:24:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3717worker1 # [ 25.489913] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_connected addr=lb:500513718worker1 # [ 25.494139] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3719worker2 # [ 25.550348] nix-grpc-daemon[776]: ts=2026-09-21T04:24:18Z level=info event=scheduler_connected addr=lb:500513720worker2 # [ 25.556383] nix-grpc-daemon[776]: ts=2026-09-21T04:24:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3721worker1 # [ 25.695982] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_connected addr=lb:500513722worker1 # [ 25.701535] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3723worker2 # [ 25.757999] nix-grpc-daemon[776]: ts=2026-09-21T04:24:18Z level=info event=scheduler_connected addr=lb:500513724worker2 # [ 25.767467] nix-grpc-daemon[776]: ts=2026-09-21T04:24:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3725worker1 # [ 25.905096] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_connected addr=lb:500513726worker1 # [ 25.910695] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker2 # [ 25.965885] nix-grpc-daemon[776]: ts=2026-09-21T04:24:18Z level=info event=scheduler_connected addr=lb:500513728worker2 # [ 25.968449] nix-grpc-daemon[776]: ts=2026-09-21T04:24:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3729worker1 # [ 26.112115] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_connected addr=lb:500513730worker1 # [ 26.117778] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker2 # [ 26.172373] nix-grpc-daemon[776]: ts=2026-09-21T04:24:18Z level=info event=scheduler_connected addr=lb:500513732worker2 # [ 26.176838] nix-grpc-daemon[776]: ts=2026-09-21T04:24:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3733worker1 # [ 26.319267] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_connected addr=lb:500513734worker1 # [ 26.325264] nix-grpc-daemon[777]: ts=2026-09-21T04:24:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker2 # [ 26.373528] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513736worker2 # [ 26.378171] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3737worker1 # [ 26.527021] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513738worker1 # [ 26.533435] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3739worker2 # [ 26.579816] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513740worker2 # [ 26.585554] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3741worker1 # [ 26.734429] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513742worker1 # [ 26.740332] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3743worker2 # [ 26.786787] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513744worker2 # [ 26.792614] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3745worker1 # [ 26.946224] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513746worker1 # [ 26.950599] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3747worker2 # [ 26.995696] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513748worker2 # [ 27.001201] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3749worker1 # [ 27.146613] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513750worker1 # [ 27.152754] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751worker2 # [ 27.202934] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513752worker2 # [ 27.209486] nix-grpc-daemon[776]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3753worker1 # [ 27.354237] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500513754worker1 # [ 27.359784] nix-grpc-daemon[777]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3755worker2 # [ 27.411926] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500513756worker2 # [ 27.417525] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3757worker1 # [ 27.561590] nix-grpc-daemon[777]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500513758worker1 # [ 27.568549] nix-grpc-daemon[777]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3759worker2 # [ 27.618872] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500513760worker2 # [ 27.624563] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3761worker1 # [ 27.768644] nix-grpc-daemon[777]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500513762worker1 # [ 27.774488] nix-grpc-daemon[777]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3763worker2 # [ 27.826066] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500513764worker2 # [ 27.831728] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3765worker1 # [ 27.979063] nix-grpc-daemon[777]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500513766worker1 # [ 27.983547] nix-grpc-daemon[777]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3767worker2 # [ 28.036466] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500513768worker2 # [ 28.040797] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3769worker1 # [ 28.182100] nix-grpc-daemon[777]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500513770worker1 # [ 28.186552] nix-grpc-daemon[777]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3771worker2 # [ 28.238972] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500513772worker2 # [ 28.244966] nix-grpc-daemon[776]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3773worker1 # [ 28.383782] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513774worker1 # [ 28.389996] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3775worker2 # [ 28.445946] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513776worker2 # [ 28.448795] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3777worker1 # [ 28.592000] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513778worker1 # [ 28.597643] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3779worker2 # [ 28.650640] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513780worker2 # [ 28.656406] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3781worker1 # [ 28.798087] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513782worker1 # [ 28.800744] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3783worker2 # [ 28.857288] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513784worker2 # [ 28.862336] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3785worker1 # [ 29.003718] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513786worker1 # [ 29.007914] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3787worker2 # [ 29.064911] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513788worker2 # [ 29.069574] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3789worker1 # [ 29.204162] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513790worker1 # [ 29.209259] nix-grpc-daemon[777]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3791worker2 # [ 29.270237] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500513792worker2 # [ 29.273195] nix-grpc-daemon[776]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3793worker1 # [ 29.410204] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513794worker1 # [ 29.415357] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3795worker2 # [ 29.474364] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513796worker2 # [ 29.479459] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3797worker1 # [ 29.616254] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513798worker1 # [ 29.621432] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3799worker2 # [ 29.680216] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513800worker2 # [ 29.683190] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3801worker1 # [ 29.822238] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513802worker1 # [ 29.827448] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3803worker2 # [ 29.884303] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513804worker2 # [ 29.889391] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3805worker1 # [ 30.030109] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513806worker1 # [ 30.034210] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3807worker2 # [ 30.092272] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513808worker2 # [ 30.096358] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3809worker1 # [ 30.230602] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513810worker1 # [ 30.235729] nix-grpc-daemon[777]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3811worker2 # [ 30.292767] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500513812worker2 # [ 30.297712] nix-grpc-daemon[776]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3813worker1 # [ 30.436580] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513814worker1 # [ 30.441749] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3815worker2 # [ 30.498793] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513816worker2 # [ 30.503810] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3817worker1 # [ 30.642551] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513818worker1 # [ 30.647590] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3819worker2 # [ 30.704976] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513820worker2 # [ 30.709901] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3821worker1 # [ 30.848876] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513822worker1 # [ 30.853502] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3823worker2 # [ 30.910796] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513824worker2 # [ 30.915893] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3825worker1 # [ 31.052533] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513826worker1 # [ 31.056552] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3827worker2 # [ 31.118876] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513828worker2 # [ 31.122979] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3829worker1 # [ 31.252982] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513830worker1 # [ 31.258000] nix-grpc-daemon[777]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3831worker2 # [ 31.319539] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_connected addr=lb:500513832worker2 # [ 31.324649] nix-grpc-daemon[776]: ts=2026-09-21T04:24:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3833worker1 # [ 31.458902] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513834worker1 # [ 31.464173] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3835worker2 # [ 31.525634] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513836worker2 # [ 31.530609] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3837worker1 # [ 31.664921] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513838worker1 # [ 31.669960] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3839worker2 # [ 31.731709] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513840worker2 # [ 31.737006] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3841lb # [ 31.823416] rustfs-bucket-start[1021]: mb s3://farm3842worker1 # [ 31.870752] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513843worker1 # [ 31.875845] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3844lb # [ 31.832961] systemd[1]: Finished rustfs-bucket.service.3845lb # [ 31.844576] systemd[1]: Starting niks3 server...3846worker2 # [ 31.938151] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513847worker2 # [ 31.943437] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3848worker1 # [ 32.077755] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513849worker1 # [ 32.080444] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3850lb # [ 32.054244] postgres[1046]: [1046] ERROR: relation "goose_db_version" does not exist at character 363851lb # [ 32.055564] postgres[1046]: [1046] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3852lb # [ 32.090895] niks3-server[1038]: 2026/09/21 04:24:24 OK 20241026095416_initial_model.sql (21.32ms)3853lb # [ 32.094676] niks3-server[1038]: 2026/09/21 04:24:24 OK 20251210153512_drop_unused_gin_index.sql (2.31ms)3854lb # [ 32.100120] niks3-server[1038]: 2026/09/21 04:24:24 OK 20251218171726_add_pins.sql (5.37ms)3855lb # [ 32.104881] niks3-server[1038]: 2026/09/21 04:24:24 OK 20260628120000_add_object_size_and_stats.sql (4.72ms)3856lb # [ 32.110289] niks3-server[1038]: 2026/09/21 04:24:24 OK 20260905000000_add_claims.sql (5.38ms)3857lb # [ 32.114661] niks3-server[1038]: 2026/09/21 04:24:24 OK 20260920000000_drop_claims.sql (4.26ms)3858lb # [ 32.116644] niks3-server[1038]: 2026/09/21 04:24:24 goose: successfully migrated database to version: 202609200000003859lb # [ 32.121433] niks3-server[1038]: 2026/09/21 04:24:24 OK 1_commit_pending_closure.sql (6.74ms)3860lb # [ 32.123967] niks3-server[1038]: 2026/09/21 04:24:24 OK 2_object_stats_trigger.sql (2.41ms)3861lb # [ 32.125359] niks3-server[1038]: 2026/09/21 04:24:24 goose: up to current file version: 23862lb # [ 32.129414] niks3-server[1038]: 2026/09/21 04:24:24 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3863worker2 # [ 32.146600] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513864worker2 # [ 32.150660] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3865lb # [ 32.174385] niks3-server[1038]: 2026/09/21 04:24:24 INFO Read proxy enabled — serving cache objects from S33866lb # [ 32.176419] niks3-server[1038]: 2026/09/21 04:24:24 INFO Using socket-activated listener address=0.0.0.0:57513867lb # [ 32.180182] niks3-server[1038]: 2026/09/21 04:24:24 INFO systemd watchdog enabled interval=15s3868lb # [ 32.183737] systemd[1]: Started niks3 server.3869lb # [ 32.186585] niks3-server[1038]: 2026/09/21 04:24:24 INFO Starting HTTP server address=0.0.0.0:57513870lb # [ 32.187790] systemd[1]: Reached target Multi-User System.3871lb # [ 32.192750] systemd[1]: Startup finished in 949ms (kernel) + 5.446s (initrd) + 25.786s (userspace) = 32.183s.3872lb # [ 32.194065] niks3-server[1038]: 2026/09/21 04:24:24 INFO lead: acquired remote=192.168.1.4:364103873worker2 # [ 32.203137] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_take_over3874worker1 # [ 32.278658] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513875worker1 # [ 32.283784] nix-grpc-daemon[777]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3876worker2 # [ 32.347318] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_connected addr=lb:500513877worker2 # [ 32.352856] nix-grpc-daemon[776]: ts=2026-09-21T04:24:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3878worker1 # [ 32.484492] nix-grpc-daemon[777]: ts=2026-09-21T04:24:25Z level=info event=scheduler_connected addr=lb:500513879worker1 # [ 32.489490] nix-grpc-daemon[777]: ts=2026-09-21T04:24:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3880worker2 # [ 32.553740] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=scheduler_connected addr=lb:500513881worker2 # [ 32.560514] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3882worker1 # [ 32.690302] nix-grpc-daemon[777]: ts=2026-09-21T04:24:25Z level=info event=scheduler_connected addr=lb:500513883worker1 # [ 32.695426] nix-grpc-daemon[777]: ts=2026-09-21T04:24:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3884worker2 # [ 32.758963] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=scheduler_connected addr=lb:500513885worker2 # [ 32.764180] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3886worker1 # [ 32.896166] nix-grpc-daemon[777]: ts=2026-09-21T04:24:25Z level=info event=scheduler_connected addr=lb:500513887worker2 # [ 32.860296] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:588503888worker2 # [ 32.866488] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03889worker2 # [ 32.965188] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=scheduler_connected addr=lb:500513890worker2 # [ 32.987089] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=oidc_jwks provider=mock keys=13891worker2 # [ 32.991247] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:588503892lb: (finished: waiting for unit niks3.service, in 19.53 seconds)3893lb: waiting for TCP port 5751 on localhost3894worker2 # [ 32.996781] nix-grpc-daemon[776]: ts=2026-09-21T04:24:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03895lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3896lb: (finished: waiting for TCP port 5751 on localhost, in 0.09 seconds)3897lb: waiting for unit envoy.service3898lb: (finished: waiting for unit envoy.service, in 0.06 seconds)3899lb: waiting for TCP port 50051 on localhost3900lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3901lb # [ 33.185247] envoy[902]: conn peer=[::1]:57830 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33902lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3903worker2 # [ 33.375926] systemd[1]: Starting scheduler-token.service...3904worker2 # [ 33.465258] systemd[1]: scheduler-token.service: Deactivated successfully.3905worker2 # [ 33.468309] systemd[1]: Finished scheduler-token.service.3906worker2 # [ 33.469206] systemd[1]: scheduler-token.service: Consumed 40ms CPU time over 92ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3907lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3908lb: must succeed: curl -sf localhost:9901/clusters3909lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3910lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3911subtest: one node schedules and both builders hold a WorkerSession on it3912lb: must succeed: curl -sf localhost:9901/clusters3913lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3914worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3915worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)3916lb: must succeed: curl -sf localhost:9901/clusters3917lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3918lb: must succeed: curl -sf localhost:9901/clusters3919lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3920lb: must succeed: curl -sf localhost:9901/clusters3921lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3922lb: must succeed: curl -sf localhost:9901/clusters3923lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)3924worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3925worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.07 seconds)3926worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3927worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3928lb: must succeed: curl -sf localhost:9901/clusters3929lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3930lb: must succeed: curl -sf localhost:9901/clusters3931lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3932worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3933worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3934worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3935worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)3936worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3937worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3938worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3939worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)3940(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.87 seconds)3941subtest: balancer auth: client cert, bearer token, nothing, unknown CN3942worker1 # [ 34.897712] nix-grpc-daemon[777]: ts=2026-09-21T04:24:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:580423943worker1 # [ 34.917217] systemd[1]: Started Nix Daemon.3944worker1 # [ 35.222147] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3945worker1 # [ 35.247140] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 1/5)3946worker1 # [ 35.293303] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 2/5)3947worker1 # [ 35.334060] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 3/5)3948worker1 # [ 35.507743] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 220 ms (attempt 4/5)3949worker1 # [ 35.729931] nix-grpc-daemon[777]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3950worker1 # [ 35.736929] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3951worker2 # [ 35.734096] nix-grpc-daemon[776]: ts=2026-09-21T04:24:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588643952worker2 # [ 35.747667] systemd[1]: Started Nix Daemon.3953worker2 # [ 36.055210] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)3954worker2 # [ 36.077627] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)3955worker2 # [ 36.088206] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 2/5)3956worker2 # [ 36.122726] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 3/5)3957worker2 # [ 36.296965] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 264 ms (attempt 4/5)3958worker2 # [ 36.570433] nix-grpc-daemon[776]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3959worker2 # [ 36.575920] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3960worker2 # [ 36.588679] nix-grpc-daemon[776]: ts=2026-09-21T04:24:29Z level=debug event=path_info_miss path=9b0ww2bi5z80fdwbhp6j472q7cahjy3h-nixos-system-client-test error="error: path '/nix/store/9b0ww2bi5z80fdwbhp6j472q7cahjy3h-nixos-system-client-test' is required, but there is no substituter that can build it"3961worker2 # [ 36.601720] nix-grpc-daemon[776]: ts=2026-09-21T04:24:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588643962worker2 # [ 36.614248] nix-grpc-daemon[776]: ts=2026-09-21T04:24:29Z level=debug event=path_info_miss path=9b0ww2bi5z80fdwbhp6j472q7cahjy3h-nixos-system-client-test error="error: path '/nix/store/9b0ww2bi5z80fdwbhp6j472q7cahjy3h-nixos-system-client-test' is required, but there is no substituter that can build it"3963client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3964client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)3965client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3966client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)3967worker1 # [ 37.101250] nix-grpc-daemon[777]: ts=2026-09-21T04:24:29Z level=info event=oidc_jwks provider=mock keys=13968worker1 # [ 37.107223] nix-grpc-daemon[777]: ts=2026-09-21T04:24:29Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:580423969worker2 # [ 37.078788] nix-grpc-daemon[776]: ts=2026-09-21T04:24:29Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:588643970worker1 # [ 37.139751] nix-grpc-daemon[777]: ts=2026-09-21T04:24:29Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:580423971worker2 # [ 37.090374] nix-grpc-daemon[776]: ts=2026-09-21T04:24:29Z level=debug event=path_info_miss path=9b0ww2bi5z80fdwbhp6j472q7cahjy3h-nixos-system-client-test error="error: path '/nix/store/9b0ww2bi5z80fdwbhp6j472q7cahjy3h-nixos-system-client-test' is required, but there is no substituter that can build it"3972worker1 # [ 37.154604] nix-grpc-daemon[777]: ts=2026-09-21T04:24:29Z level=debug event=path_info_miss path=9b0ww2bi5z80fdwbhp6j472q7cahjy3h-nixos-system-client-test error="error: path '/nix/store/9b0ww2bi5z80fdwbhp6j472q7cahjy3h-nixos-system-client-test' is required, but there is no substituter that can build it"3973client: must succeed: rm -r /var/lib/nix-grpc-store3974client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.08 seconds)3975lb # [ 39.290280] envoy[902]: conn peer=[2001:db8:1::1]:35576 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=18193976lb # [ 39.293648] envoy[902]: conn peer=[2001:db8:1::1]:35586 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=853977worker2 # [ 39.494317] nix-grpc-daemon[776]: ts=2026-09-21T04:24:32Z level=info event=denied method=QueryMissing cn=- role=none3978worker2 # [ 39.757602] nix-grpc-daemon[776]: ts=2026-09-21T04:24:32Z level=info event=denied method=QueryMissing cn=stranger role=none3979(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.33 seconds)3980subtest: DAG build is scheduled across workers and lands in the cache3981client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23982worker2 # [ 40.011659] nix-grpc-daemon[776]: ts=2026-09-21T04:24:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588643983client # don't know how to build these paths:3984client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3985worker2 # [ 40.025998] nix-grpc-daemon[776]: ts=2026-09-21T04:24:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588643986worker2 # [ 40.064133] nix-grpc-daemon[776]: ts=2026-09-21T04:24:32Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03987worker2 # [ 40.070053] nix-grpc-daemon[776]: ts=2026-09-21T04:24:32Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=13988worker1 # [ 40.163617] nix-grpc-daemon[777]: ts=2026-09-21T04:24:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580423989worker1 # [ 40.181858] nix-grpc-daemon[777]: ts=2026-09-21T04:24:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580423990client # copying 1 paths to 'lb:50051'...3991worker1 # [ 40.196665] nix-grpc-daemon[777]: ts=2026-09-21T04:24:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580423992client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3993worker1 # [ 40.226078] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3994worker1 # [ 40.260169] nix-grpc-daemon[777]: ts=2026-09-21T04:24:32Z level=info event=niks3_push_started pid=8523995worker1 # [ 40.550189] nix-grpc-daemon[852]: time=2026-09-21T04:24:33.174Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""3996worker1 # [ 40.741504] nix-daemon[838]: accepted connection from pid 865, user nix-grpc-daemon (trusted)3997worker1 # [ 40.761517] nix-daemon[838]: reaped child process 868, status = succeeded3998worker1 # [ 40.933716] nix-daemon[838]: accepted connection from pid 870, user nix-grpc-daemon (trusted)3999worker1 # [ 40.949181] nix-daemon[838]: reaped child process 873, status = succeeded4000lb # [ 40.914782] niks3-server[1038]: 2026/09/21 04:24:33 INFO Received uploads request method=POST path=/api/pending_closures4001worker1 # [ 40.989448] nix-grpc-daemon[852]: time=2026-09-21T04:24:33.615Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4002worker1 # [ 40.992681] nix-grpc-daemon[852]: time=2026-09-21T04:24:33.617Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4003lb # [ 40.986052] niks3-server[1038]: 2026/09/21 04:24:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4004lb # [ 40.998954] niks3-server[1038]: 2026/09/21 04:24:33 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4005lb # [ 41.040124] niks3-server[1038]: 2026/09/21 04:24:33 INFO Signed narinfos id=1 count=14006worker1 # [ 41.090819] nix-grpc-daemon[852]: time=2026-09-21T04:24:33.716Z level=INFO msg="Uploading 1 narinfos"4007lb # [ 41.053149] niks3-server[1038]: 2026/09/21 04:24:33 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4008lb # [ 41.059031] niks3-server[1038]: 2026/09/21 04:24:33 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4009lb # [ 41.073374] niks3-server[1038]: 2026/09/21 04:24:33 INFO Completed upload id=14010worker1 # [ 41.121280] nix-grpc-daemon[852]: time=2026-09-21T04:24:33.747Z level=INFO msg="Upload complete. (563ms)"4011lb # [ 41.081234] niks3-server[1038]: 2026/09/21 04:24:33 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4012worker1 # [ 41.134067] nix-grpc-daemon[777]: ts=2026-09-21T04:24:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=0 paths=1 nar_bytes_in=5044013worker1 # [ 41.151060] nix-grpc-daemon[777]: ts=2026-09-21T04:24:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424014worker1 # [ 41.163823] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4015worker1 # [ 41.167873] nix-daemon[838]: reaped child process 850, status = succeeded4016client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4017client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4018worker1 # [ 41.182334] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4019client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4020client # farm-a> LOG-farm-a-t14021worker1 # [ 42.015113] nix-daemon[838]: reaped child process 877, status = succeeded4022worker1 # [ 42.170241] nix-daemon[838]: accepted connection from pid 893, user nix-grpc-daemon (trusted)4023worker1 # [ 42.186378] nix-daemon[838]: reaped child process 897, status = succeeded4024lb # [ 42.151853] niks3-server[1038]: 2026/09/21 04:24:34 INFO Received uploads request method=POST path=/api/pending_closures4025worker1 # [ 42.215622] nix-grpc-daemon[852]: time=2026-09-21T04:24:34.840Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4026worker1 # [ 42.217816] nix-grpc-daemon[852]: time=2026-09-21T04:24:34.840Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4027lb # [ 42.213753] niks3-server[1038]: 2026/09/21 04:24:34 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4028worker1 # [ 42.273283] nix-grpc-daemon[852]: time=2026-09-21T04:24:34.898Z level=INFO msg="Uploading 1 narinfos"4029lb # [ 42.227757] niks3-server[1038]: 2026/09/21 04:24:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4030lb # [ 42.238042] niks3-server[1038]: 2026/09/21 04:24:34 INFO Signed narinfos id=2 count=14031lb # [ 42.242692] niks3-server[1038]: 2026/09/21 04:24:34 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4032lb # [ 42.251458] niks3-server[1038]: 2026/09/21 04:24:34 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4033worker1 # [ 42.302890] nix-grpc-daemon[852]: time=2026-09-21T04:24:34.929Z level=INFO msg="Upload complete. (306ms)"4034lb # [ 42.255774] niks3-server[1038]: 2026/09/21 04:24:34 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4035lb # [ 42.259977] niks3-server[1038]: 2026/09/21 04:24:34 INFO Completed upload id=24036lb # [ 42.263564] niks3-server[1038]: 2026/09/21 04:24:34 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4037worker1 # [ 42.313051] nix-grpc-daemon[777]: ts=2026-09-21T04:24:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14038worker2 # [ 42.280765] nix-grpc-daemon[776]: ts=2026-09-21T04:24:34Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04039worker1 # [ 42.330649] nix-daemon[838]: reaped child process 875, status = succeeded4040worker2 # [ 42.286021] nix-grpc-daemon[776]: ts=2026-09-21T04:24:34Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14041worker1 # [ 42.373281] nix-grpc-daemon[777]: ts=2026-09-21T04:24:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424042worker1 # [ 42.387184] nix-grpc-daemon[777]: ts=2026-09-21T04:24:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424043worker1 # [ 42.402648] nix-grpc-daemon[852]: time=2026-09-21T04:24:35.028Z level=INFO msg="All 1 paths already cached"4044client # copying 1 paths to 'lb:50051'...4045worker1 # [ 42.412476] nix-grpc-daemon[777]: ts=2026-09-21T04:24:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580424046client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4047worker1 # [ 42.434321] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4048worker1 # [ 42.644182] nix-daemon[838]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4049worker1 # [ 42.661346] nix-daemon[838]: reaped child process 904, status = succeeded4050worker1 # [ 42.833515] nix-daemon[838]: accepted connection from pid 906, user nix-grpc-daemon (trusted)4051worker1 # [ 42.848904] nix-daemon[838]: reaped child process 909, status = succeeded4052lb # [ 42.812669] niks3-server[1038]: 2026/09/21 04:24:35 INFO Received uploads request method=POST path=/api/pending_closures4053worker1 # [ 42.871022] nix-grpc-daemon[852]: time=2026-09-21T04:24:35.496Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4054worker1 # [ 42.872543] nix-grpc-daemon[852]: time=2026-09-21T04:24:35.496Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4055lb # [ 42.843016] niks3-server[1038]: 2026/09/21 04:24:35 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4056worker1 # [ 42.902290] nix-grpc-daemon[852]: time=2026-09-21T04:24:35.528Z level=INFO msg="Uploading 1 narinfos"4057lb # [ 42.859538] niks3-server[1038]: 2026/09/21 04:24:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4058lb # [ 42.869929] niks3-server[1038]: 2026/09/21 04:24:35 INFO Signed narinfos id=3 count=14059lb # [ 42.882423] niks3-server[1038]: 2026/09/21 04:24:35 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4060lb # [ 42.886809] niks3-server[1038]: 2026/09/21 04:24:35 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4061worker1 # [ 42.939249] nix-grpc-daemon[852]: time=2026-09-21T04:24:35.565Z level=INFO msg="Upload complete. (487ms)"4062lb # [ 42.892418] niks3-server[1038]: 2026/09/21 04:24:35 INFO Completed upload id=34063lb # [ 42.894774] niks3-server[1038]: 2026/09/21 04:24:35 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4064worker1 # [ 42.948250] nix-grpc-daemon[777]: ts=2026-09-21T04:24:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=0 paths=1 nar_bytes_in=6244065worker1 # [ 42.956482] nix-grpc-daemon[777]: ts=2026-09-21T04:24:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424066worker1 # [ 42.964728] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4067worker1 # [ 42.969584] nix-daemon[838]: reaped child process 899, status = succeeded4068client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4069client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4070worker1 # [ 42.982578] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4071client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4072client # farm-b> LOG-farm-b-t14073worker1 # [ 44.209254] nix-daemon[838]: reaped child process 913, status = succeeded4074worker1 # [ 44.370451] nix-daemon[838]: accepted connection from pid 929, user nix-grpc-daemon (trusted)4075worker1 # [ 44.386935] nix-daemon[838]: reaped child process 932, status = succeeded4076lb # [ 44.357011] niks3-server[1038]: 2026/09/21 04:24:36 INFO Received uploads request method=POST path=/api/pending_closures4077worker1 # [ 44.417507] nix-grpc-daemon[852]: time=2026-09-21T04:24:37.042Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4078worker1 # [ 44.418956] nix-grpc-daemon[852]: time=2026-09-21T04:24:37.042Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4079lb # [ 44.398005] niks3-server[1038]: 2026/09/21 04:24:37 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4080worker1 # [ 44.462550] nix-grpc-daemon[852]: time=2026-09-21T04:24:37.088Z level=INFO msg="Uploading 1 narinfos"4081lb # [ 44.414604] niks3-server[1038]: 2026/09/21 04:24:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4082lb # [ 44.422470] niks3-server[1038]: 2026/09/21 04:24:37 INFO Signed narinfos id=4 count=14083lb # [ 44.428160] niks3-server[1038]: 2026/09/21 04:24:37 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4084lb # [ 44.431986] niks3-server[1038]: 2026/09/21 04:24:37 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4085lb # [ 44.442288] niks3-server[1038]: 2026/09/21 04:24:37 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4086worker1 # [ 44.489898] nix-grpc-daemon[852]: time=2026-09-21T04:24:37.116Z level=INFO msg="Upload complete. (297ms)"4087lb # [ 44.448830] niks3-server[1038]: 2026/09/21 04:24:37 INFO Completed upload id=44088lb # [ 44.451683] niks3-server[1038]: 2026/09/21 04:24:37 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4089worker1 # [ 44.500979] nix-grpc-daemon[777]: ts=2026-09-21T04:24:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14090worker2 # [ 44.462542] nix-grpc-daemon[776]: ts=2026-09-21T04:24:37Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04091worker1 # [ 44.517304] nix-daemon[838]: reaped child process 911, status = succeeded4092worker2 # [ 44.468544] nix-grpc-daemon[776]: ts=2026-09-21T04:24:37Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14093worker1 # [ 44.551115] nix-grpc-daemon[777]: ts=2026-09-21T04:24:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424094worker1 # [ 44.566883] nix-grpc-daemon[777]: ts=2026-09-21T04:24:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424095worker1 # [ 44.581958] nix-grpc-daemon[852]: time=2026-09-21T04:24:37.208Z level=INFO msg="All 2 paths already cached"4096client # copying 1 paths to 'lb:50051'...4097client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4098worker1 # [ 44.592071] nix-grpc-daemon[777]: ts=2026-09-21T04:24:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580424099worker1 # [ 44.609862] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4100worker1 # [ 44.811985] nix-daemon[838]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4101worker1 # [ 44.831803] nix-daemon[838]: reaped child process 939, status = succeeded4102worker1 # [ 45.005356] nix-daemon[838]: accepted connection from pid 941, user nix-grpc-daemon (trusted)4103worker1 # [ 45.022667] nix-daemon[838]: reaped child process 944, status = succeeded4104lb # [ 44.991436] niks3-server[1038]: 2026/09/21 04:24:37 INFO Received uploads request method=POST path=/api/pending_closures4105worker1 # [ 45.049289] nix-grpc-daemon[852]: time=2026-09-21T04:24:37.675Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4106worker1 # [ 45.050873] nix-grpc-daemon[852]: time=2026-09-21T04:24:37.675Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4107lb # [ 45.019686] niks3-server[1038]: 2026/09/21 04:24:37 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4108worker1 # [ 45.080163] nix-grpc-daemon[852]: time=2026-09-21T04:24:37.706Z level=INFO msg="Uploading 1 narinfos"4109lb # [ 45.036974] niks3-server[1038]: 2026/09/21 04:24:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4110lb # [ 45.047238] niks3-server[1038]: 2026/09/21 04:24:37 INFO Signed narinfos id=5 count=14111lb # [ 45.052556] niks3-server[1038]: 2026/09/21 04:24:37 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4112lb # [ 45.063109] niks3-server[1038]: 2026/09/21 04:24:37 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4113lb # [ 45.068335] niks3-server[1038]: 2026/09/21 04:24:37 INFO Completed upload id=54114worker1 # [ 45.115942] nix-grpc-daemon[852]: time=2026-09-21T04:24:37.742Z level=INFO msg="Upload complete. (488ms)"4115lb # [ 45.071352] niks3-server[1038]: 2026/09/21 04:24:37 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4116worker1 # [ 45.124341] nix-grpc-daemon[777]: ts=2026-09-21T04:24:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=0 paths=1 nar_bytes_in=7524117worker1 # [ 45.133055] nix-grpc-daemon[777]: ts=2026-09-21T04:24:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424118worker1 # [ 45.142864] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4119worker1 # [ 45.148411] nix-daemon[838]: reaped child process 934, status = succeeded4120client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4121worker1 # [ 45.160813] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4122client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4123client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4124client # farm-top> LOG-farm-top-t14125worker1 # [ 45.994298] nix-daemon[838]: reaped child process 948, status = succeeded4126worker1 # [ 46.170600] nix-daemon[838]: accepted connection from pid 965, user nix-grpc-daemon (trusted)4127worker1 # [ 46.188215] nix-daemon[838]: reaped child process 968, status = succeeded4128lb # [ 46.157837] niks3-server[1038]: 2026/09/21 04:24:38 INFO Received uploads request method=POST path=/api/pending_closures4129worker1 # [ 46.215670] nix-grpc-daemon[852]: time=2026-09-21T04:24:38.840Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4130worker1 # [ 46.217306] nix-grpc-daemon[852]: time=2026-09-21T04:24:38.840Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4131lb # [ 46.186366] niks3-server[1038]: 2026/09/21 04:24:38 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4132lb # [ 46.202362] niks3-server[1038]: 2026/09/21 04:24:38 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4133worker1 # [ 46.267673] nix-grpc-daemon[852]: time=2026-09-21T04:24:38.893Z level=INFO msg="Uploading 1 narinfos"4134lb # [ 46.214597] niks3-server[1038]: 2026/09/21 04:24:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4135lb # [ 46.223728] niks3-server[1038]: 2026/09/21 04:24:38 INFO Signed narinfos id=6 count=14136lb # [ 46.229301] niks3-server[1038]: 2026/09/21 04:24:38 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4137lb # [ 46.240315] niks3-server[1038]: 2026/09/21 04:24:38 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4138lb # [ 46.243730] niks3-server[1038]: 2026/09/21 04:24:38 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4139lb # [ 46.247701] niks3-server[1038]: 2026/09/21 04:24:38 INFO Completed upload id=64140worker1 # [ 46.295254] nix-grpc-daemon[852]: time=2026-09-21T04:24:38.921Z level=INFO msg="Upload complete. (325ms)"4141worker1 # [ 46.305068] nix-grpc-daemon[777]: ts=2026-09-21T04:24:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14142worker1 # [ 46.322190] nix-daemon[838]: reaped child process 946, status = succeeded4143client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 6.53 seconds)4144client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4145client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.23 seconds)4146client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4147client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4148client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4149client # copying 3 paths...4150client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4151client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4152client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4153client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.29 seconds)4154(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.18 seconds)4155subtest: repeat is answered Cached by the scheduler without building4156worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4157worker1 # finding garbage collector roots...4158worker1 # deleting specified paths...4159worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4160worker1 # deleting unused links...4161worker1 # note: hard linking is currently saving 0.0 KiB4162worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.27 seconds)4163worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4164worker2 # finding garbage collector roots...4165worker2 # deleting specified paths...4166worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4167worker2 # deleting unused links...4168worker2 # note: hard linking is currently saving 0.0 KiB4169worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4170client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24171worker1 # [ 47.822553] nix-grpc-daemon[777]: ts=2026-09-21T04:24:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:580424172client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4173client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4174worker2 # [ 47.797952] nix-grpc-daemon[776]: ts=2026-09-21T04:24:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588644175worker2 # [ 47.828932] nix-grpc-daemon[776]: ts=2026-09-21T04:24:40Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14176client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.35 seconds)4177client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4178client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4179lb: must succeed: curl -sf localhost:9901/clusters4180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4181lb: must succeed: curl -sf localhost:9901/clusters4182lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4183worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4184worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4185(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.43 seconds)4186subtest: inputs already in the cache are not sent to the scheduler4187lb: must succeed: curl -sf localhost:9901/clusters4188lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4189lb: must succeed: curl -sf localhost:9901/clusters4190lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4191worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4192worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4193client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24194worker2 # [ 48.846434] nix-grpc-daemon[776]: ts=2026-09-21T04:24:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588644195client # don't know how to build these paths:4196client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4197worker1 # [ 48.905273] nix-grpc-daemon[777]: ts=2026-09-21T04:24:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424198worker1 # [ 48.918135] nix-grpc-daemon[852]: time=2026-09-21T04:24:41.543Z level=INFO msg="All 2 paths already cached"4199worker2 # [ 48.888799] nix-grpc-daemon[776]: ts=2026-09-21T04:24:41Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04200worker2 # [ 48.894803] nix-grpc-daemon[776]: ts=2026-09-21T04:24:41Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14201worker1 # [ 48.980478] nix-grpc-daemon[777]: ts=2026-09-21T04:24:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424202worker1 # [ 48.995722] nix-grpc-daemon[777]: ts=2026-09-21T04:24:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424203worker1 # [ 49.010305] nix-grpc-daemon[852]: time=2026-09-21T04:24:41.636Z level=INFO msg="All 2 paths already cached"4204client # copying 1 paths to 'lb:50051'...4205worker1 # [ 49.020118] nix-grpc-daemon[777]: ts=2026-09-21T04:24:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580424206client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4207worker1 # [ 49.039015] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4208worker1 # [ 49.249171] nix-daemon[838]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4209worker1 # [ 49.267675] nix-daemon[838]: reaped child process 988, status = succeeded4210lb # [ 49.289911] envoy[902]: conn peer=[2001:db8:1::1]:35602 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134211lb # [ 49.294660] envoy[902]: conn peer=[2001:db8:1::1]:35614 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=154212lb # [ 49.299505] envoy[902]: conn peer=[2001:db8:1::1]:35620 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9510 tx=4092 ms=62694213lb # [ 49.304394] envoy[902]: conn peer=[2001:db8:1::1]:37038 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=724214worker1 # [ 49.437118] nix-daemon[838]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4215worker1 # [ 49.453597] nix-daemon[838]: reaped child process 993, status = succeeded4216lb # [ 49.418556] niks3-server[1038]: 2026/09/21 04:24:42 INFO Received uploads request method=POST path=/api/pending_closures4217worker1 # [ 49.475413] nix-grpc-daemon[852]: time=2026-09-21T04:24:42.101Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4218worker1 # [ 49.477015] nix-grpc-daemon[852]: time=2026-09-21T04:24:42.101Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4219lb # [ 49.445084] niks3-server[1038]: 2026/09/21 04:24:42 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4220worker1 # [ 49.505673] nix-grpc-daemon[852]: time=2026-09-21T04:24:42.131Z level=INFO msg="Uploading 1 narinfos"4221lb # [ 49.461534] niks3-server[1038]: 2026/09/21 04:24:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4222lb # [ 49.471980] niks3-server[1038]: 2026/09/21 04:24:42 INFO Signed narinfos id=7 count=14223lb # [ 49.477115] niks3-server[1038]: 2026/09/21 04:24:42 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4224lb # [ 49.486868] niks3-server[1038]: 2026/09/21 04:24:42 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4225lb # [ 49.492966] niks3-server[1038]: 2026/09/21 04:24:42 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4226lb # [ 49.497468] niks3-server[1038]: 2026/09/21 04:24:42 INFO Completed upload id=74227worker1 # [ 49.544725] nix-grpc-daemon[852]: time=2026-09-21T04:24:42.170Z level=INFO msg="Upload complete. (487ms)"4228worker1 # [ 49.552708] nix-grpc-daemon[777]: ts=2026-09-21T04:24:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=0 paths=1 nar_bytes_in=7524229worker1 # [ 49.565394] nix-grpc-daemon[777]: ts=2026-09-21T04:24:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424230worker1 # [ 49.571514] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4231worker1 # [ 49.575975] nix-daemon[838]: reaped child process 983, status = succeeded4232client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4233worker1 # [ 49.588236] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4234client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4235client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4236client # farm-top> LOG-farm-top-t1b4237worker1 # [ 50.433695] nix-daemon[838]: reaped child process 997, status = succeeded4238worker1 # [ 50.607258] nix-daemon[838]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)4239worker1 # [ 50.624432] nix-daemon[838]: reaped child process 1016, status = succeeded4240lb # [ 50.593869] niks3-server[1038]: 2026/09/21 04:24:43 INFO Received uploads request method=POST path=/api/pending_closures4241worker1 # [ 50.651944] nix-grpc-daemon[852]: time=2026-09-21T04:24:43.275Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4242worker1 # [ 50.653600] nix-grpc-daemon[852]: time=2026-09-21T04:24:43.275Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4243lb # [ 50.629137] niks3-server[1038]: 2026/09/21 04:24:43 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4244lb # [ 50.641569] niks3-server[1038]: 2026/09/21 04:24:43 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4245lb # [ 50.648435] niks3-server[1038]: 2026/09/21 04:24:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4246worker1 # [ 50.700984] nix-grpc-daemon[852]: time=2026-09-21T04:24:43.326Z level=INFO msg="Uploading 1 narinfos"4247lb # [ 50.656374] niks3-server[1038]: 2026/09/21 04:24:43 INFO Signed narinfos id=8 count=14248lb # [ 50.664074] niks3-server[1038]: 2026/09/21 04:24:43 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4249lb # [ 50.673032] niks3-server[1038]: 2026/09/21 04:24:43 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4250lb # [ 50.680934] niks3-server[1038]: 2026/09/21 04:24:43 INFO Completed upload id=84251lb # [ 50.684226] niks3-server[1038]: 2026/09/21 04:24:43 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4252worker1 # [ 50.728623] nix-grpc-daemon[852]: time=2026-09-21T04:24:43.354Z level=INFO msg="Upload complete. (330ms)"4253worker1 # [ 50.741324] nix-grpc-daemon[777]: ts=2026-09-21T04:24:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14254worker1 # [ 50.758871] nix-daemon[838]: reaped child process 995, status = succeeded4255client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.15 seconds)4256client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4257client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds)4258lb: must succeed: curl -sf localhost:9901/clusters4259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4260lb: must succeed: curl -sf localhost:9901/clusters4261lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4262worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4263worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4264(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.76 seconds)4265subtest: two clients wanting the same drv build it once4266client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4267client # [ 51.237429] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4268client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4269client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4270worker1 # [ 51.558121] nix-grpc-daemon[777]: ts=2026-09-21T04:24:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:580424271client # Running as unit: dedup1.service; invocation ID: b0ca76e0e37f401991b26265a02e44ae4272client # [ 51.511092] nix[888]: don't know how to build these paths:4273client # [ 51.513090] nix[888]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4274worker2 # [ 51.533657] nix-grpc-daemon[776]: ts=2026-09-21T04:24:44Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04275worker2 # [ 51.534053] nix-grpc-daemon[776]: ts=2026-09-21T04:24:44Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14276worker2 # [ 51.547186] nix-grpc-daemon[776]: ts=2026-09-21T04:24:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588644277worker2 # [ 51.565753] nix-grpc-daemon[776]: ts=2026-09-21T04:24:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588644278client # [ 51.569777] nix[888]: copying 1 paths to 'lb:50051'...4279client # [ 51.571564] nix[888]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4280worker2 # [ 51.580047] nix-grpc-daemon[776]: ts=2026-09-21T04:24:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588644281worker2 # [ 51.595427] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4282worker2 # [ 51.622561] nix-grpc-daemon[776]: ts=2026-09-21T04:24:44Z level=info event=niks3_push_started pid=9114283worker2 # [ 51.883585] nix-grpc-daemon[911]: time=2026-09-21T04:24:44.522Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4284worker2 # [ 52.075422] nix-daemon[857]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4285worker2 # [ 52.091792] nix-daemon[857]: reaped child process 927, status = succeeded4286worker2 # [ 52.265269] nix-daemon[857]: accepted connection from pid 929, user nix-grpc-daemon (trusted)4287worker2 # [ 52.283245] nix-daemon[857]: reaped child process 932, status = succeeded4288lb # [ 52.293639] niks3-server[1038]: 2026/09/21 04:24:44 INFO Received uploads request method=POST path=/api/pending_closures4289worker2 # [ 52.306487] nix-grpc-daemon[911]: time=2026-09-21T04:24:44.946Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4290worker2 # [ 52.309791] nix-grpc-daemon[911]: time=2026-09-21T04:24:44.948Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4291lb # [ 52.327759] niks3-server[1038]: 2026/09/21 04:24:44 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4292lb # [ 52.341369] niks3-server[1038]: 2026/09/21 04:24:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4293worker2 # [ 52.350140] nix-grpc-daemon[911]: time=2026-09-21T04:24:44.989Z level=INFO msg="Uploading 1 narinfos"4294lb # [ 52.353847] niks3-server[1038]: 2026/09/21 04:24:44 INFO Signed narinfos id=9 count=14295lb # [ 52.360800] niks3-server[1038]: 2026/09/21 04:24:44 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4296lb # [ 52.370632] niks3-server[1038]: 2026/09/21 04:24:44 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4297client # [ 52.357608] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4298worker2 # [ 52.380823] nix-grpc-daemon[911]: time=2026-09-21T04:24:45.020Z level=INFO msg="Upload complete. (488ms)"4299lb # [ 52.378558] niks3-server[1038]: 2026/09/21 04:24:44 INFO Completed upload id=94300client # Running as unit: dedup2.service; invocation ID: 1841a1945e854f3e97fdbd2f59408c424301lb # [ 52.381388] niks3-server[1038]: 2026/09/21 04:24:45 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4302worker2 # [ 52.388270] nix-grpc-daemon[776]: ts=2026-09-21T04:24:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=0 paths=1 nar_bytes_in=4324303worker2 # [ 52.397011] nix-grpc-daemon[776]: ts=2026-09-21T04:24:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588644304client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4305worker2 # [ 52.406247] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4306worker2 # [ 52.410544] nix-daemon[857]: reaped child process 909, status = succeeded4307client # [ 52.410393] nix[888]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4308worker2 # [ 52.423287] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4309client: must succeed: sleep 24310client # [ 54.765057] nix[904]: don't know how to build these paths:4311client # [ 54.765246] nix[904]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4312worker1 # [ 54.818816] nix-grpc-daemon[777]: ts=2026-09-21T04:24:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:580424313worker2 # [ 54.780949] nix-grpc-daemon[776]: ts=2026-09-21T04:24:47Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04314worker2 # [ 54.785918] nix-grpc-daemon[776]: ts=2026-09-21T04:24:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588644315client # [ 54.780477] nix[904]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4316client: (finished: must succeed: sleep 2, in 2.92 seconds)4317client: waiting for success: ! systemctl is-active dedup1 dedup24318worker2 # [ 56.846724] nix-daemon[857]: reaped child process 936, status = succeeded4319worker2 # [ 57.002530] nix-daemon[857]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4320worker2 # [ 57.020655] nix-daemon[857]: reaped child process 971, status = succeeded4321lb # [ 57.031309] niks3-server[1038]: 2026/09/21 04:24:49 INFO Received uploads request method=POST path=/api/pending_closures4322worker2 # [ 57.044654] nix-grpc-daemon[911]: time=2026-09-21T04:24:49.683Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4323worker2 # [ 57.046434] nix-grpc-daemon[911]: time=2026-09-21T04:24:49.683Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4324lb # [ 57.078157] niks3-server[1038]: 2026/09/21 04:24:49 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4325worker2 # [ 57.094937] nix-grpc-daemon[911]: time=2026-09-21T04:24:49.733Z level=INFO msg="Uploading 1 narinfos"4326lb # [ 57.091765] niks3-server[1038]: 2026/09/21 04:24:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4327lb # [ 57.103066] niks3-server[1038]: 2026/09/21 04:24:49 INFO Signed narinfos id=10 count=14328lb # [ 57.110195] niks3-server[1038]: 2026/09/21 04:24:49 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4329lb # [ 57.118082] niks3-server[1038]: 2026/09/21 04:24:49 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4330lb # [ 57.125768] niks3-server[1038]: 2026/09/21 04:24:49 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4331lb # [ 57.132121] niks3-server[1038]: 2026/09/21 04:24:49 INFO Completed upload id=104332worker2 # [ 57.133671] nix-grpc-daemon[911]: time=2026-09-21T04:24:49.773Z level=INFO msg="Upload complete. (298ms)"4333lb # [ 57.135379] niks3-server[1038]: 2026/09/21 04:24:49 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4334worker2 # [ 57.147056] nix-grpc-daemon[776]: ts=2026-09-21T04:24:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14335worker2 # [ 57.160380] nix-grpc-daemon[776]: ts=2026-09-21T04:24:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=2 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14336worker2 # [ 57.166249] nix-daemon[857]: reaped child process 934, status = succeeded4337client # [ 57.201262] systemd[1]: dedup1.service: Deactivated successfully.4338client # [ 57.205692] systemd[1]: dedup1.service: Consumed 172ms CPU time over 5.969s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.4339client # [ 57.216432] systemd[1]: dedup2.service: Deactivated successfully.4340client # [ 57.218448] systemd[1]: dedup2.service: Consumed 232ms CPU time over 4.856s wall clock time, 33.4M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4341client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4342client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4343client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4344worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4345worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4346worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4347worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4348worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4349worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4350worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4351worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4352(finished: subtest: two clients wanting the same drv build it once, in 7.15 seconds)4353subtest: build hook: nix-daemon with builders = grpc://lb4354client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4355client # [ 58.523665] systemd[1]: Started Nix Daemon.4356client # [ 58.715818] nix-daemon[946]: accepted connection from pid 943, user root (trusted)4357worker1 # [ 58.992649] nix-grpc-daemon[777]: ts=2026-09-21T04:24:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:580424358worker2 # [ 58.951486] nix-grpc-daemon[776]: ts=2026-09-21T04:24:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588644359worker2 # [ 58.970911] nix-grpc-daemon[776]: ts=2026-09-21T04:24:51Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04360worker2 # [ 58.976989] nix-grpc-daemon[776]: ts=2026-09-21T04:24:51Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14361worker1 # [ 59.063279] nix-grpc-daemon[777]: ts=2026-09-21T04:24:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424362worker1 # [ 59.079066] nix-grpc-daemon[777]: ts=2026-09-21T04:24:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424363worker1 # [ 59.094271] nix-grpc-daemon[777]: ts=2026-09-21T04:24:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580424364worker1 # [ 59.109065] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4365worker1 # [ 59.314127] nix-daemon[838]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4366worker1 # [ 59.333155] nix-daemon[838]: reaped child process 1044, status = succeeded4367lb # [ 59.291758] envoy[902]: conn peer=[2001:db8:1::1]:37052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1783 ms=18604368lb # [ 59.296922] envoy[902]: conn peer=[2001:db8:1::1]:57992 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1085 ms=23914369lb # [ 59.301764] envoy[902]: conn peer=[2001:db8:1::1]:57990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1453 ms=56504370worker1 # [ 59.504759] nix-daemon[838]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4371worker1 # [ 59.519952] nix-daemon[838]: reaped child process 1049, status = succeeded4372lb # [ 59.485290] niks3-server[1038]: 2026/09/21 04:24:52 INFO Received uploads request method=POST path=/api/pending_closures4373worker1 # [ 59.542081] nix-grpc-daemon[852]: time=2026-09-21T04:24:52.167Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4374worker1 # [ 59.543518] nix-grpc-daemon[852]: time=2026-09-21T04:24:52.167Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4375lb # [ 59.514398] niks3-server[1038]: 2026/09/21 04:24:52 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4376lb # [ 59.526597] niks3-server[1038]: 2026/09/21 04:24:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4377worker1 # [ 59.582282] nix-grpc-daemon[852]: time=2026-09-21T04:24:52.208Z level=INFO msg="Uploading 1 narinfos"4378lb # [ 59.536705] niks3-server[1038]: 2026/09/21 04:24:52 INFO Signed narinfos id=11 count=14379lb # [ 59.543753] niks3-server[1038]: 2026/09/21 04:24:52 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4380lb # [ 59.552395] niks3-server[1038]: 2026/09/21 04:24:52 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4381lb # [ 59.561370] niks3-server[1038]: 2026/09/21 04:24:52 INFO Completed upload id=114382worker1 # [ 59.608339] nix-grpc-daemon[852]: time=2026-09-21T04:24:52.234Z level=INFO msg="Upload complete. (482ms)"4383lb # [ 59.563775] niks3-server[1038]: 2026/09/21 04:24:52 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4384worker1 # [ 59.615621] nix-grpc-daemon[777]: ts=2026-09-21T04:24:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=0 paths=1 nar_bytes_in=5124385worker1 # [ 59.629960] nix-grpc-daemon[777]: ts=2026-09-21T04:24:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424386worker1 # [ 59.639032] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4387worker1 # [ 59.641470] nix-daemon[838]: reaped child process 1039, status = succeeded4388worker1 # [ 59.654332] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4389worker1 # [ 60.698046] nix-daemon[838]: reaped child process 1053, status = succeeded4390worker1 # [ 60.879000] nix-daemon[838]: accepted connection from pid 1069, user nix-grpc-daemon (trusted)4391worker1 # [ 60.895683] nix-daemon[838]: reaped child process 1072, status = succeeded4392lb # [ 60.861164] niks3-server[1038]: 2026/09/21 04:24:53 INFO Received uploads request method=POST path=/api/pending_closures4393worker1 # [ 60.920758] nix-grpc-daemon[852]: time=2026-09-21T04:24:53.544Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4394worker1 # [ 60.927270] nix-grpc-daemon[852]: time=2026-09-21T04:24:53.544Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4395lb # [ 60.903723] niks3-server[1038]: 2026/09/21 04:24:53 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4396lb # [ 60.915854] niks3-server[1038]: 2026/09/21 04:24:53 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4397lb # [ 60.925571] niks3-server[1038]: 2026/09/21 04:24:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4398worker1 # [ 60.978127] nix-grpc-daemon[852]: time=2026-09-21T04:24:53.603Z level=INFO msg="Uploading 1 narinfos"4399lb # [ 60.935197] niks3-server[1038]: 2026/09/21 04:24:53 INFO Signed narinfos id=12 count=14400lb # [ 60.940116] niks3-server[1038]: 2026/09/21 04:24:53 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4401lb # [ 60.946497] niks3-server[1038]: 2026/09/21 04:24:53 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4402lb # [ 60.950833] niks3-server[1038]: 2026/09/21 04:24:53 INFO Completed upload id=124403worker1 # [ 60.997956] nix-grpc-daemon[852]: time=2026-09-21T04:24:53.624Z level=INFO msg="Upload complete. (334ms)"4404worker1 # [ 61.008467] nix-grpc-daemon[852]: time=2026-09-21T04:24:53.629Z level=WARN msg="Failed to register uploaded object" key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo error="server returned 404: object is not pending upload\n"4405worker1 # [ 61.029437] nix-grpc-daemon[777]: ts=2026-09-21T04:24:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14406worker1 # [ 61.036619] nix-grpc-daemon[777]: ts=2026-09-21T04:24:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:580424407worker1 # [ 61.041027] nix-daemon[838]: reaped child process 1051, status = succeeded4408worker2 # [ 61.001300] nix-grpc-daemon[776]: ts=2026-09-21T04:24:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588644409worker2 # [ 61.024890] nix-grpc-daemon[776]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4410worker2 # [ 61.046767] nix-grpc-daemon[776]: ts=2026-09-21T04:24:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=0 paths=1 nar_bytes_out=1284411worker2 # [ 61.372179] nix-grpc-daemon[776]: ts=2026-09-21T04:24:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:588644412worker1 # [ 61.425162] nix-grpc-daemon[777]: ts=2026-09-21T04:24:54Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:580424413worker1 # [ 61.437925] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4414worker2 # [ 61.464506] nix-grpc-daemon[776]: ts=2026-09-21T04:24:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588644415worker2 # [ 61.475096] nix-grpc-daemon[911]: time=2026-09-21T04:24:54.114Z level=INFO msg="All 1 paths already cached"4416worker2 # [ 61.496845] nix-grpc-daemon[776]: ts=2026-09-21T04:24:54Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04417worker2 # [ 61.502914] nix-grpc-daemon[776]: ts=2026-09-21T04:24:54Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14418worker1 # [ 61.551897] nix-grpc-daemon[777]: ts=2026-09-21T04:24:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424419worker1 # [ 61.568890] nix-grpc-daemon[777]: ts=2026-09-21T04:24:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424420worker1 # [ 61.583479] nix-grpc-daemon[852]: time=2026-09-21T04:24:54.209Z level=INFO msg="All 1 paths already cached"4421worker1 # [ 61.593297] nix-grpc-daemon[777]: ts=2026-09-21T04:24:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580424422worker1 # [ 61.611782] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4423worker1 # [ 61.827753] nix-daemon[838]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)4424worker1 # [ 61.845723] nix-daemon[838]: reaped child process 1083, status = succeeded4425worker1 # [ 62.018542] nix-daemon[838]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)4426worker1 # [ 62.034033] nix-daemon[838]: reaped child process 1088, status = succeeded4427lb # [ 61.998933] niks3-server[1038]: 2026/09/21 04:24:54 INFO Received uploads request method=POST path=/api/pending_closures4428worker1 # [ 62.056309] nix-grpc-daemon[852]: time=2026-09-21T04:24:54.681Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4429worker1 # [ 62.057739] nix-grpc-daemon[852]: time=2026-09-21T04:24:54.682Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4430lb # [ 62.028768] niks3-server[1038]: 2026/09/21 04:24:54 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4431worker1 # [ 62.089575] nix-grpc-daemon[852]: time=2026-09-21T04:24:54.715Z level=INFO msg="Uploading 1 narinfos"4432lb # [ 62.045321] niks3-server[1038]: 2026/09/21 04:24:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4433lb # [ 62.057059] niks3-server[1038]: 2026/09/21 04:24:54 INFO Signed narinfos id=13 count=14434lb # [ 62.059802] niks3-server[1038]: 2026/09/21 04:24:54 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4435lb # [ 62.067511] niks3-server[1038]: 2026/09/21 04:24:54 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4436lb # [ 62.074171] niks3-server[1038]: 2026/09/21 04:24:54 INFO Completed upload id=134437worker1 # [ 62.121798] nix-grpc-daemon[852]: time=2026-09-21T04:24:54.748Z level=INFO msg="Upload complete. (491ms)"4438lb # [ 62.077229] niks3-server[1038]: 2026/09/21 04:24:54 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4439worker1 # [ 62.133639] nix-grpc-daemon[777]: ts=2026-09-21T04:24:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=0 paths=1 nar_bytes_in=6244440worker1 # [ 62.143557] nix-grpc-daemon[777]: ts=2026-09-21T04:24:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424441worker1 # [ 62.154251] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4442worker1 # [ 62.157365] nix-daemon[838]: reaped child process 1078, status = succeeded4443worker1 # [ 62.171014] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4444worker1 # [ 63.491311] nix-daemon[838]: reaped child process 1092, status = succeeded4445worker1 # [ 63.771595] nix-daemon[838]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)4446worker1 # [ 63.795788] nix-daemon[838]: reaped child process 1111, status = succeeded4447lb # [ 63.766621] niks3-server[1038]: 2026/09/21 04:24:56 INFO Received uploads request method=POST path=/api/pending_closures4448worker1 # [ 63.832765] nix-grpc-daemon[852]: time=2026-09-21T04:24:56.457Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4449worker1 # [ 63.839717] nix-grpc-daemon[852]: time=2026-09-21T04:24:56.457Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4450lb # [ 63.859253] niks3-server[1038]: 2026/09/21 04:24:56 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4451lb # [ 63.882261] niks3-server[1038]: 2026/09/21 04:24:56 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4452worker1 # [ 63.944425] nix-grpc-daemon[852]: time=2026-09-21T04:24:56.569Z level=INFO msg="Uploading 1 narinfos"4453lb # [ 63.898526] niks3-server[1038]: 2026/09/21 04:24:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4454lb # [ 63.908844] niks3-server[1038]: 2026/09/21 04:24:56 INFO Signed narinfos id=14 count=14455lb # [ 63.914388] niks3-server[1038]: 2026/09/21 04:24:56 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4456lb # [ 63.923866] niks3-server[1038]: 2026/09/21 04:24:56 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4457worker1 # [ 63.971414] nix-grpc-daemon[852]: time=2026-09-21T04:24:56.597Z level=INFO msg="Upload complete. (585ms)"4458lb # [ 63.928863] niks3-server[1038]: 2026/09/21 04:24:56 INFO Completed upload id=144459lb # [ 63.932443] niks3-server[1038]: 2026/09/21 04:24:56 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4460worker1 # [ 63.985590] nix-grpc-daemon[777]: ts=2026-09-21T04:24:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14461worker1 # [ 64.003059] nix-daemon[838]: reaped child process 1090, status = succeeded4462worker2 # [ 63.986537] nix-grpc-daemon[776]: ts=2026-09-21T04:24:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588644463worker2 # [ 64.002324] nix-grpc-daemon[776]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4464worker2 # [ 64.049817] nix-grpc-daemon[776]: ts=2026-09-21T04:24:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588644465worker2 # [ 64.054834] nix-grpc-daemon[776]: ts=2026-09-21T04:24:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=0 paths=1 nar_bytes_out=1764466worker1 # [ 64.159029] nix-grpc-daemon[777]: ts=2026-09-21T04:24:56Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=2 bytes_in=312 bytes_out=2004467worker1 # [ 64.166331] nix-daemon[838]: reaped child process 1075, status = succeeded4468worker2 # [ 64.381239] nix-grpc-daemon[776]: ts=2026-09-21T04:24:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:588644469worker1 # [ 64.431243] nix-grpc-daemon[777]: ts=2026-09-21T04:24:57Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:580424470worker1 # [ 64.444908] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4471worker1 # [ 64.513677] nix-grpc-daemon[777]: ts=2026-09-21T04:24:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424472worker1 # [ 64.523879] nix-grpc-daemon[852]: time=2026-09-21T04:24:57.150Z level=INFO msg="All 2 paths already cached"4473worker2 # [ 64.501125] nix-grpc-daemon[776]: ts=2026-09-21T04:24:57Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04474worker1 # [ 64.551060] nix-grpc-daemon[777]: ts=2026-09-21T04:24:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424475worker2 # [ 64.506925] nix-grpc-daemon[776]: ts=2026-09-21T04:24:57Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14476worker1 # [ 64.567006] nix-grpc-daemon[777]: ts=2026-09-21T04:24:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424477worker1 # [ 64.581043] nix-grpc-daemon[852]: time=2026-09-21T04:24:57.207Z level=INFO msg="All 2 paths already cached"4478worker1 # [ 64.590073] nix-grpc-daemon[777]: ts=2026-09-21T04:24:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580424479worker1 # [ 64.605501] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4480worker1 # [ 64.809299] nix-daemon[838]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4481worker1 # [ 64.826949] nix-daemon[838]: reaped child process 1122, status = succeeded4482worker1 # [ 64.998837] nix-daemon[838]: accepted connection from pid 1124, user nix-grpc-daemon (trusted)4483worker1 # [ 65.015032] nix-daemon[838]: reaped child process 1127, status = succeeded4484lb # [ 64.982110] niks3-server[1038]: 2026/09/21 04:24:57 INFO Received uploads request method=POST path=/api/pending_closures4485worker1 # [ 65.037536] nix-grpc-daemon[852]: time=2026-09-21T04:24:57.663Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4486worker1 # [ 65.039049] nix-grpc-daemon[852]: time=2026-09-21T04:24:57.663Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4487lb # [ 65.007010] niks3-server[1038]: 2026/09/21 04:24:57 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4488lb # [ 65.020787] niks3-server[1038]: 2026/09/21 04:24:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4489worker1 # [ 65.074213] nix-grpc-daemon[852]: time=2026-09-21T04:24:57.700Z level=INFO msg="Uploading 1 narinfos"4490lb # [ 65.031133] niks3-server[1038]: 2026/09/21 04:24:57 INFO Signed narinfos id=15 count=14491lb # [ 65.037834] niks3-server[1038]: 2026/09/21 04:24:57 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4492lb # [ 65.047238] niks3-server[1038]: 2026/09/21 04:24:57 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4493lb # [ 65.052617] niks3-server[1038]: 2026/09/21 04:24:57 INFO Completed upload id=154494worker1 # [ 65.100282] nix-grpc-daemon[852]: time=2026-09-21T04:24:57.726Z level=INFO msg="Upload complete. (477ms)"4495lb # [ 65.055430] niks3-server[1038]: 2026/09/21 04:24:57 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4496worker1 # [ 65.108838] nix-grpc-daemon[777]: ts=2026-09-21T04:24:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=0 paths=1 nar_bytes_in=7524497worker1 # [ 65.120925] nix-grpc-daemon[777]: ts=2026-09-21T04:24:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424498worker1 # [ 65.129122] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4499worker1 # [ 65.131927] nix-daemon[838]: reaped child process 1117, status = succeeded4500worker1 # [ 65.145531] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4501worker1 # [ 66.776272] nix-daemon[838]: reaped child process 1131, status = succeeded4502worker1 # [ 66.937948] nix-daemon[838]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)4503worker1 # [ 66.954663] nix-daemon[838]: reaped child process 1150, status = succeeded4504lb # [ 66.925583] niks3-server[1038]: 2026/09/21 04:24:59 INFO Received uploads request method=POST path=/api/pending_closures4505worker1 # [ 66.984379] nix-grpc-daemon[852]: time=2026-09-21T04:24:59.607Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4506worker1 # [ 66.985849] nix-grpc-daemon[852]: time=2026-09-21T04:24:59.607Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4507lb # [ 66.965027] niks3-server[1038]: 2026/09/21 04:24:59 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4508lb # [ 66.977889] niks3-server[1038]: 2026/09/21 04:24:59 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4509worker1 # [ 67.037512] nix-grpc-daemon[852]: time=2026-09-21T04:24:59.663Z level=INFO msg="Uploading 1 narinfos"4510lb # [ 66.989701] niks3-server[1038]: 2026/09/21 04:24:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4511lb # [ 66.994960] niks3-server[1038]: 2026/09/21 04:24:59 INFO Signed narinfos id=16 count=14512lb # [ 66.999942] niks3-server[1038]: 2026/09/21 04:24:59 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4513lb # [ 67.011076] niks3-server[1038]: 2026/09/21 04:24:59 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4514lb # [ 67.016615] niks3-server[1038]: 2026/09/21 04:24:59 INFO Completed upload id=164515worker1 # [ 67.063345] nix-grpc-daemon[852]: time=2026-09-21T04:24:59.689Z level=INFO msg="Upload complete. (307ms)"4516lb # [ 67.019432] niks3-server[1038]: 2026/09/21 04:24:59 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4517worker2 # [ 67.034468] nix-grpc-daemon[776]: ts=2026-09-21T04:24:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588644518worker1 # [ 67.072984] nix-grpc-daemon[777]: ts=2026-09-21T04:24:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14519worker1 # [ 67.089022] nix-daemon[838]: reaped child process 1129, status = succeeded4520worker2 # [ 67.046921] nix-grpc-daemon[776]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4521worker1 # [ 67.136131] nix-grpc-daemon[777]: ts=2026-09-21T04:24:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:580424522worker1 # [ 67.141971] nix-grpc-daemon[777]: ts=2026-09-21T04:24:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=0 paths=1 nar_bytes_out=2324523worker1 # [ 67.164061] nix-grpc-daemon[777]: ts=2026-09-21T04:24:59Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=2 bytes_in=376 bytes_out=2004524worker1 # [ 67.169849] nix-daemon[838]: reaped child process 1114, status = succeeded4525client # [ 67.181487] nix-daemon[946]: reaped child process 948, status = succeeded4526client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.90 seconds)4527client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4528client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4529client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4530client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4531client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4532client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4533(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 9.03 seconds)4534subtest: an input only one worker has still reaches the builder4535client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4536client # this derivation will be built:4537client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4538client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4539client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.43 seconds)4540client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4541client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4542worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4543worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.24 seconds)4544worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4545worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4546client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24547client # [ 68.467985] nix-daemon[946]: accepted connection from pid 1032, user root (trusted)4548client # this derivation will be built:4549client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4550client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4551worker2 # [ 68.689835] nix-grpc-daemon[776]: ts=2026-09-21T04:25:01Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:588644552worker1 # [ 68.739832] nix-grpc-daemon[777]: ts=2026-09-21T04:25:01Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:580424553worker1 # [ 68.754181] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4554worker2 # [ 68.776472] nix-grpc-daemon[776]: ts=2026-09-21T04:25:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588644555client # copying 1 paths...4556client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4557worker2 # [ 68.794019] nix-grpc-daemon[776]: ts=2026-09-21T04:25:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588644558worker2 # [ 68.810439] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4559worker2 # [ 69.025270] nix-daemon[857]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4560worker2 # [ 69.046056] nix-daemon[857]: reaped child process 999, status = succeeded4561lb # [ 69.055642] niks3-server[1038]: 2026/09/21 04:25:01 INFO Received uploads request method=POST path=/api/pending_closures4562worker2 # [ 69.066323] nix-grpc-daemon[911]: time=2026-09-21T04:25:01.706Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4563worker2 # [ 69.068602] nix-grpc-daemon[911]: time=2026-09-21T04:25:01.708Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4564lb # [ 69.083894] niks3-server[1038]: 2026/09/21 04:25:01 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4565lb # [ 69.098608] niks3-server[1038]: 2026/09/21 04:25:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4566worker2 # [ 69.107269] nix-grpc-daemon[911]: time=2026-09-21T04:25:01.746Z level=INFO msg="Uploading 1 narinfos"4567lb # [ 69.111331] niks3-server[1038]: 2026/09/21 04:25:01 INFO Signed narinfos id=17 count=14568lb # [ 69.118042] niks3-server[1038]: 2026/09/21 04:25:01 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4569lb # [ 69.125112] niks3-server[1038]: 2026/09/21 04:25:01 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4570worker2 # [ 69.136208] nix-grpc-daemon[911]: time=2026-09-21T04:25:01.776Z level=INFO msg="Upload complete. (309ms)"4571lb # [ 69.135379] niks3-server[1038]: 2026/09/21 04:25:01 INFO Completed upload id=174572lb # [ 69.138372] niks3-server[1038]: 2026/09/21 04:25:01 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4573worker2 # [ 69.148146] nix-grpc-daemon[776]: ts=2026-09-21T04:25:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=0 paths=1 nar_bytes_in=1204574worker2 # [ 69.163399] nix-daemon[857]: reaped child process 994, status = succeeded4575worker2 # [ 69.168167] nix-grpc-daemon[776]: ts=2026-09-21T04:25:01Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04576worker2 # [ 69.172719] nix-grpc-daemon[776]: ts=2026-09-21T04:25:01Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14577worker2 # [ 69.218349] nix-grpc-daemon[776]: ts=2026-09-21T04:25:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588644578worker2 # [ 69.234003] nix-grpc-daemon[776]: ts=2026-09-21T04:25:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588644579client # copying 2 paths to 'lb:50051'...4580worker2 # [ 69.252622] nix-grpc-daemon[776]: ts=2026-09-21T04:25:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588644581client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4582client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4583worker2 # [ 69.266311] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4584lb # [ 69.291938] envoy[902]: conn peer=[2001:db8:1::1]:58008 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3401 tx=1718 ms=21604585lb # [ 69.300671] envoy[902]: conn peer=[2001:db8:1::1]:50432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=1104586lb # [ 69.306521] envoy[902]: conn peer=[2001:db8:1::1]:50456 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=644587lb # [ 69.312292] envoy[902]: conn peer=[2001:db8:1::1]:50446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4502 tx=2380 ms=27374588lb # [ 69.317326] envoy[902]: conn peer=[2001:db8:1::1]:50474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=294589lb # [ 69.322441] envoy[902]: conn peer=[2001:db8:1::1]:50462 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4956 tx=2596 ms=27324590worker2 # [ 69.462189] nix-daemon[857]: accepted connection from pid 1005, user nix-grpc-daemon (trusted)4591worker2 # [ 69.481240] nix-daemon[857]: reaped child process 1008, status = succeeded4592worker2 # [ 69.649449] nix-daemon[857]: accepted connection from pid 1010, user nix-grpc-daemon (trusted)4593worker2 # [ 69.667468] nix-daemon[857]: reaped child process 1013, status = succeeded4594lb # [ 69.676753] niks3-server[1038]: 2026/09/21 04:25:02 INFO Received uploads request method=POST path=/api/pending_closures4595lb # [ 69.686670] niks3-server[1038]: 2026/09/21 04:25:02 INFO Received uploads request method=POST path=/api/pending_closures4596worker2 # [ 69.698172] nix-grpc-daemon[911]: time=2026-09-21T04:25:02.337Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4597worker2 # [ 69.699630] nix-grpc-daemon[911]: time=2026-09-21T04:25:02.337Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4598worker2 # [ 69.701458] nix-grpc-daemon[911]: time=2026-09-21T04:25:02.337Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4599lb # [ 69.722783] niks3-server[1038]: 2026/09/21 04:25:02 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4600lb # [ 69.740238] niks3-server[1038]: 2026/09/21 04:25:02 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4601lb # [ 69.755429] niks3-server[1038]: 2026/09/21 04:25:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4602worker2 # [ 69.764469] nix-grpc-daemon[911]: time=2026-09-21T04:25:02.403Z level=INFO msg="Uploading 3 narinfos"4603lb # [ 69.764870] niks3-server[1038]: 2026/09/21 04:25:02 INFO Signed narinfos id=18 count=14604lb # [ 69.771170] niks3-server[1038]: 2026/09/21 04:25:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4605lb # [ 69.778522] niks3-server[1038]: 2026/09/21 04:25:02 INFO Signed narinfos id=19 count=24606lb # [ 69.782697] niks3-server[1038]: 2026/09/21 04:25:02 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4607lb # [ 69.789865] niks3-server[1038]: 2026/09/21 04:25:02 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4608lb # [ 69.802736] niks3-server[1038]: 2026/09/21 04:25:02 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4609lb # [ 69.810742] niks3-server[1038]: 2026/09/21 04:25:02 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4610lb # [ 69.818314] niks3-server[1038]: 2026/09/21 04:25:02 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4611lb # [ 69.822504] niks3-server[1038]: 2026/09/21 04:25:02 INFO Completed upload id=184612lb # [ 69.824684] niks3-server[1038]: 2026/09/21 04:25:02 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4613worker2 # [ 69.826885] nix-grpc-daemon[911]: time=2026-09-21T04:25:02.466Z level=INFO msg="Upload complete. (541ms)"4614lb # [ 69.827928] niks3-server[1038]: 2026/09/21 04:25:02 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4615lb # [ 69.835889] niks3-server[1038]: 2026/09/21 04:25:02 INFO Completed upload id=194616worker2 # [ 69.838931] nix-grpc-daemon[776]: ts=2026-09-21T04:25:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=0 paths=2 nar_bytes_in=9844617worker2 # [ 69.859354] nix-grpc-daemon[776]: ts=2026-09-21T04:25:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588644618worker2 # [ 69.870863] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4619worker2 # [ 69.880492] nix-daemon[857]: reaped child process 1003, status = succeeded4620client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4621worker2 # [ 69.890298] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4622client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4623client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4624worker2 # [ 71.314883] nix-daemon[857]: reaped child process 1017, status = succeeded4625worker2 # [ 71.470808] nix-daemon[857]: accepted connection from pid 1033, user nix-grpc-daemon (trusted)4626worker2 # [ 71.487330] nix-daemon[857]: reaped child process 1036, status = succeeded4627lb # [ 71.501167] niks3-server[1038]: 2026/09/21 04:25:04 INFO Received uploads request method=POST path=/api/pending_closures4628worker2 # [ 71.512410] nix-grpc-daemon[911]: time=2026-09-21T04:25:04.151Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4629lb # [ 71.530563] niks3-server[1038]: 2026/09/21 04:25:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4630worker2 # [ 71.542860] nix-grpc-daemon[911]: time=2026-09-21T04:25:04.182Z level=INFO msg="Uploading 1 narinfos"4631lb # [ 71.545096] niks3-server[1038]: 2026/09/21 04:25:04 INFO Signed narinfos id=20 count=14632lb # [ 71.551506] niks3-server[1038]: 2026/09/21 04:25:04 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4633lb # [ 71.558506] niks3-server[1038]: 2026/09/21 04:25:04 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4634worker2 # [ 71.570942] nix-grpc-daemon[911]: time=2026-09-21T04:25:04.210Z level=INFO msg="Upload complete. (267ms)"4635lb # [ 71.569271] niks3-server[1038]: 2026/09/21 04:25:04 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4636lb # [ 71.576165] niks3-server[1038]: 2026/09/21 04:25:04 INFO Completed upload id=204637client # copying 1 paths...4638worker2 # [ 71.580308] nix-grpc-daemon[776]: ts=2026-09-21T04:25:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14639lb # [ 71.579095] niks3-server[1038]: 2026/09/21 04:25:04 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4640client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4641worker2 # [ 71.590788] nix-grpc-daemon[776]: ts=2026-09-21T04:25:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588644642worker2 # [ 71.593956] nix-daemon[857]: reaped child process 1015, status = succeeded4643worker2 # [ 71.614171] nix-grpc-daemon[776]: ts=2026-09-21T04:25:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588644644worker2 # [ 71.618426] nix-grpc-daemon[776]: ts=2026-09-21T04:25:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=0 paths=1 nar_bytes_out=1204645worker1 # [ 71.731148] nix-grpc-daemon[777]: ts=2026-09-21T04:25:04Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=2 bytes_in=320 bytes_out=2004646worker1 # [ 71.738919] nix-daemon[838]: reaped child process 1158, status = succeeded4647client # [ 71.740616] nix-daemon[946]: reaped child process 1035, status = succeeded4648client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.51 seconds)4649client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24650client # [ 71.958615] nix-daemon[946]: accepted connection from pid 1056, user root (trusted)4651client # this derivation will be built:4652client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4653worker1 # [ 72.226854] nix-grpc-daemon[777]: ts=2026-09-21T04:25:04Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:580424654client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4655worker2 # [ 72.186989] nix-grpc-daemon[776]: ts=2026-09-21T04:25:04Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:588644656worker2 # [ 72.200507] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4657worker1 # [ 72.313678] nix-grpc-daemon[777]: ts=2026-09-21T04:25:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424658worker1 # [ 72.325867] nix-grpc-daemon[852]: time=2026-09-21T04:25:04.951Z level=INFO msg="All 1 paths already cached"4659worker2 # [ 72.302123] nix-grpc-daemon[776]: ts=2026-09-21T04:25:04Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04660worker2 # [ 72.308837] nix-grpc-daemon[776]: ts=2026-09-21T04:25:04Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14661worker1 # [ 72.358045] nix-grpc-daemon[777]: ts=2026-09-21T04:25:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424662worker1 # [ 72.373565] nix-grpc-daemon[777]: ts=2026-09-21T04:25:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580424663client # copying 1 paths to 'lb:50051'...4664client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4665worker1 # [ 72.395472] nix-grpc-daemon[777]: ts=2026-09-21T04:25:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580424666worker1 # [ 72.409641] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4667worker1 # [ 72.432356] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5)4668worker1 # [ 72.490590] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 157 ms (attempt 2/5)4669worker1 # [ 72.655062] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 3/5)4670worker1 # [ 72.720587] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 584 ms (attempt 4/5)4671worker1 # [ 73.313783] nix-grpc-daemon[777]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4672worker1 # [ 73.321310] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4673worker1 # [ 73.330136] nix-grpc-daemon[777]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4674worker1 # [ 73.549196] nix-daemon[838]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)4675worker1 # [ 73.566377] nix-daemon[838]: reaped child process 1170, status = succeeded4676worker1 # [ 73.737093] nix-daemon[838]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)4677worker1 # [ 73.752586] nix-daemon[838]: reaped child process 1175, status = succeeded4678lb # [ 73.717742] niks3-server[1038]: 2026/09/21 04:25:06 INFO Received uploads request method=POST path=/api/pending_closures4679worker1 # [ 73.775415] nix-grpc-daemon[852]: time=2026-09-21T04:25:06.401Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4680worker1 # [ 73.777068] nix-grpc-daemon[852]: time=2026-09-21T04:25:06.401Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4681lb # [ 73.751726] niks3-server[1038]: 2026/09/21 04:25:06 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4682lb # [ 73.766275] niks3-server[1038]: 2026/09/21 04:25:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4683worker1 # [ 73.821248] nix-grpc-daemon[852]: time=2026-09-21T04:25:06.447Z level=INFO msg="Uploading 1 narinfos"4684lb # [ 73.778098] niks3-server[1038]: 2026/09/21 04:25:06 INFO Signed narinfos id=21 count=14685lb # [ 73.785355] niks3-server[1038]: 2026/09/21 04:25:06 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4686worker1 # [ 73.844865] nix-grpc-daemon[852]: time=2026-09-21T04:25:06.471Z level=INFO msg="Upload complete. (497ms)"4687lb # [ 73.797653] niks3-server[1038]: 2026/09/21 04:25:06 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4688lb # [ 73.802595] niks3-server[1038]: 2026/09/21 04:25:06 INFO Completed upload id=214689lb # [ 73.805711] niks3-server[1038]: 2026/09/21 04:25:06 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4690worker1 # [ 73.852135] nix-grpc-daemon[777]: ts=2026-09-21T04:25:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=1 paths=1 nar_bytes_in=5684691worker1 # [ 73.863972] nix-grpc-daemon[777]: ts=2026-09-21T04:25:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580424692worker1 # [ 73.875909] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4693worker1 # [ 73.881592] nix-daemon[838]: reaped child process 1160, status = succeeded4694client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4695worker1 # [ 73.895221] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4696client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4697client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4698worker1 # [ 75.330205] nix-daemon[838]: reaped child process 1179, status = succeeded4699worker1 # [ 75.503323] nix-daemon[838]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)4700worker1 # [ 75.520431] nix-daemon[838]: reaped child process 1198, status = succeeded4701lb # [ 75.487395] niks3-server[1038]: 2026/09/21 04:25:08 INFO Received uploads request method=POST path=/api/pending_closures4702worker1 # [ 75.546733] nix-grpc-daemon[852]: time=2026-09-21T04:25:08.171Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4703lb # [ 75.516540] niks3-server[1038]: 2026/09/21 04:25:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4704worker1 # [ 75.571603] nix-grpc-daemon[852]: time=2026-09-21T04:25:08.197Z level=INFO msg="Uploading 1 narinfos"4705lb # [ 75.529160] niks3-server[1038]: 2026/09/21 04:25:08 INFO Signed narinfos id=22 count=14706lb # [ 75.536672] niks3-server[1038]: 2026/09/21 04:25:08 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4707lb # [ 75.549308] niks3-server[1038]: 2026/09/21 04:25:08 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4708worker1 # [ 75.601719] nix-grpc-daemon[852]: time=2026-09-21T04:25:08.227Z level=INFO msg="Upload complete. (296ms)"4709lb # [ 75.554752] niks3-server[1038]: 2026/09/21 04:25:08 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4710lb # [ 75.559737] niks3-server[1038]: 2026/09/21 04:25:08 INFO Completed upload id=224711lb # [ 75.562469] niks3-server[1038]: 2026/09/21 04:25:08 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4712worker1 # [ 75.613207] nix-grpc-daemon[777]: ts=2026-09-21T04:25:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14713worker1 # [ 75.626202] nix-daemon[838]: reaped child process 1177, status = succeeded4714client # copying 1 paths...4715worker1 # [ 75.657433] nix-grpc-daemon[777]: ts=2026-09-21T04:25:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:580424716client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4717worker2 # [ 75.639829] nix-grpc-daemon[776]: ts=2026-09-21T04:25:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588644718worker2 # [ 75.653886] nix-grpc-daemon[776]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4719worker2 # [ 75.670653] nix-grpc-daemon[776]: ts=2026-09-21T04:25:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=0 paths=1 nar_bytes_out=1204720worker2 # [ 75.736474] nix-grpc-daemon[776]: ts=2026-09-21T04:25:08Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=3 bytes_in=320 bytes_out=2004721worker2 # [ 75.745398] nix-daemon[857]: reaped child process 1039, status = succeeded4722client # [ 75.791959] nix-daemon[946]: reaped child process 1059, status = succeeded4723client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 4.05 seconds)4724client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24725client # [ 76.007823] nix-daemon[946]: accepted connection from pid 1080, user root (trusted)4726client # this derivation will be built:4727client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4728worker1 # [ 76.273162] nix-grpc-daemon[777]: ts=2026-09-21T04:25:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:580424729client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4730worker1 # [ 76.288926] nix-grpc-daemon[777]: ts=2026-09-21T04:25:08Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:580424731worker1 # [ 76.304114] nix-daemon[838]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4732worker2 # [ 76.271965] nix-grpc-daemon[776]: ts=2026-09-21T04:25:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588644733worker2 # [ 76.282893] nix-grpc-daemon[911]: time=2026-09-21T04:25:08.922Z level=INFO msg="All 1 paths already cached"4734worker2 # [ 76.302145] nix-grpc-daemon[776]: ts=2026-09-21T04:25:08Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04735worker2 # [ 76.311374] nix-grpc-daemon[776]: ts=2026-09-21T04:25:08Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14736worker2 # [ 76.320302] nix-grpc-daemon[776]: ts=2026-09-21T04:25:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588644737worker2 # [ 76.328136] nix-grpc-daemon[776]: ts=2026-09-21T04:25:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588644738worker2 # [ 76.338861] nix-grpc-daemon[911]: time=2026-09-21T04:25:08.978Z level=INFO msg="All 1 paths already cached"4739client # copying 1 paths to 'lb:50051'...4740client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4741worker2 # [ 76.348802] nix-grpc-daemon[776]: ts=2026-09-21T04:25:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588644742worker2 # [ 76.362953] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4743worker2 # [ 76.569524] nix-daemon[857]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)4744worker2 # [ 76.586923] nix-daemon[857]: reaped child process 1052, status = succeeded4745worker2 # [ 76.758149] nix-daemon[857]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)4746worker2 # [ 76.774331] nix-daemon[857]: reaped child process 1057, status = succeeded4747lb # [ 76.784890] niks3-server[1038]: 2026/09/21 04:25:09 INFO Received uploads request method=POST path=/api/pending_closures4748worker2 # [ 76.797333] nix-grpc-daemon[911]: time=2026-09-21T04:25:09.437Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4749worker2 # [ 76.798812] nix-grpc-daemon[911]: time=2026-09-21T04:25:09.437Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4750lb # [ 76.816218] niks3-server[1038]: 2026/09/21 04:25:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4751worker2 # [ 76.826259] nix-grpc-daemon[911]: time=2026-09-21T04:25:09.466Z level=INFO msg="Uploading 1 narinfos"4752lb # [ 76.829899] niks3-server[1038]: 2026/09/21 04:25:09 INFO Signed narinfos id=23 count=14753lb # [ 76.836215] niks3-server[1038]: 2026/09/21 04:25:09 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4754lb # [ 76.842320] niks3-server[1038]: 2026/09/21 04:25:09 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4755worker2 # [ 76.852536] nix-grpc-daemon[911]: time=2026-09-21T04:25:09.492Z level=INFO msg="Upload complete. (473ms)"4756lb # [ 76.851079] niks3-server[1038]: 2026/09/21 04:25:09 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4757lb # [ 76.856818] niks3-server[1038]: 2026/09/21 04:25:09 INFO Completed upload id=234758worker2 # [ 76.860661] nix-grpc-daemon[776]: ts=2026-09-21T04:25:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=0 paths=1 nar_bytes_in=5684759lb # [ 76.859765] niks3-server[1038]: 2026/09/21 04:25:09 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4760worker2 # [ 76.868880] nix-grpc-daemon[776]: ts=2026-09-21T04:25:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588644761worker2 # [ 76.880148] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4762worker2 # [ 76.882947] nix-daemon[857]: reaped child process 1047, status = succeeded4763client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4764worker2 # [ 76.895342] nix-daemon[857]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4765client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4766client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4767worker2 # [ 78.541009] nix-daemon[857]: reaped child process 1061, status = succeeded4768worker2 # [ 78.723076] nix-daemon[857]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)4769worker2 # [ 78.739407] nix-daemon[857]: reaped child process 1080, status = succeeded4770lb # [ 78.750584] niks3-server[1038]: 2026/09/21 04:25:11 INFO Received uploads request method=POST path=/api/pending_closures4771worker2 # [ 78.764391] nix-grpc-daemon[911]: time=2026-09-21T04:25:11.402Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4772lb # [ 78.781956] niks3-server[1038]: 2026/09/21 04:25:11 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4773worker2 # [ 78.793677] nix-grpc-daemon[911]: time=2026-09-21T04:25:11.433Z level=INFO msg="Uploading 1 narinfos"4774lb # [ 78.792556] niks3-server[1038]: 2026/09/21 04:25:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4775lb # [ 78.803566] niks3-server[1038]: 2026/09/21 04:25:11 INFO Signed narinfos id=24 count=14776lb # [ 78.808898] niks3-server[1038]: 2026/09/21 04:25:11 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4777lb # [ 78.817828] niks3-server[1038]: 2026/09/21 04:25:11 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4778lb # [ 78.822782] niks3-server[1038]: 2026/09/21 04:25:11 INFO Completed upload id=244779worker2 # [ 78.824447] nix-grpc-daemon[911]: time=2026-09-21T04:25:11.464Z level=INFO msg="Upload complete. (318ms)"4780lb # [ 78.825818] niks3-server[1038]: 2026/09/21 04:25:11 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4781client # copying 1 paths...4782worker2 # [ 78.836288] nix-grpc-daemon[776]: ts=2026-09-21T04:25:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14783client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4784worker2 # [ 78.850286] nix-grpc-daemon[776]: ts=2026-09-21T04:25:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:588644785worker2 # [ 78.854205] nix-daemon[857]: reaped child process 1059, status = succeeded4786worker2 # [ 78.874004] nix-grpc-daemon[776]: ts=2026-09-21T04:25:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:588644787worker2 # [ 78.879023] nix-grpc-daemon[776]: ts=2026-09-21T04:25:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58864 duration_s=0 paths=1 nar_bytes_out=1204788worker1 # [ 78.984271] nix-grpc-daemon[777]: ts=2026-09-21T04:25:11Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:58042 duration_s=2 bytes_in=320 bytes_out=2004789worker1 # [ 78.991465] nix-daemon[838]: reaped child process 1201, status = succeeded4790client # [ 78.989632] nix-daemon[946]: reaped child process 1083, status = succeeded4791client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 3.20 seconds)4792(finished: subtest: an input only one worker has still reaches the builder, in 11.67 seconds)4793subtest: upload whose reference the worker lacks is completed from the cache4794client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4795client # this derivation will be built:4796client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4797lb # [ 79.292844] envoy[902]: conn peer=[2001:db8:1::1]:45152 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=674798lb # [ 79.297793] envoy[902]: conn peer=[2001:db8:1::1]:50484 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4798 tx=2167 ms=29924799lb # [ 79.302584] envoy[902]: conn peer=[2001:db8:1::1]:45182 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=944800lb # [ 79.307920] envoy[902]: conn peer=[2001:db8:1::1]:45168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4306 tx=2193 ms=35504801lb # [ 79.312794] envoy[902]: conn peer=[2001:db8:1::1]:45192 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=614802lb # [ 79.317374] envoy[902]: conn peer=[2001:db8:1::1]:45190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4333 tx=2172 ms=27034803client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4804client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.44 seconds)4805client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4806client # this derivation will be built:4807client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4808client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4809client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.42 seconds)4810client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4811client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4812worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4813worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.23 seconds)4814worker2 # [ 80.378074] systemd[1]: Stopping Nix worker-protocol over gRPC...4815worker2 # [ 80.746989] nix-grpc-daemon[776]: ts=2026-09-21T04:25:13Z level=info event=signal_exit4816worker2 # [ 80.991586] nix-grpc-daemon[776]: ts=2026-09-21T04:25:13Z level=info event=scheduler_yield4817worker2 # [ 80.998026] nix-grpc-daemon[776]: ts=2026-09-21T04:25:13Z level=info event=scheduler_restarting addr=lb:500514818worker1 # [ 81.045533] nix-grpc-daemon[777]: ts=2026-09-21T04:25:13Z level=info event=scheduler_restarting addr=lb:500514819lb # [ 81.000271] niks3-server[1038]: 2026/09/21 04:25:13 INFO lead: released remote=192.168.1.4:364104820worker2 # [ 81.002288] nix-grpc-daemon[776]: ts=2026-09-21T04:25:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:588504821worker2 # [ 81.008536] nix-grpc-daemon[776]: ts=2026-09-21T04:25:13Z level=info event=worker_gone addr=192.168.1.4:500514822worker2 # [ 81.012582] nix-grpc-daemon[776]: ts=2026-09-21T04:25:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:588504823worker2 # [ 81.016930] nix-grpc-daemon[776]: ts=2026-09-21T04:25:13Z level=info event=worker_gone addr=192.168.1.3:500514824worker2 # [ 81.198304] nix-grpc-daemon[776]: ts=2026-09-21T04:25:13Z level=info event=scheduler_connected addr=lb:500514825worker1 # [ 81.245610] nix-grpc-daemon[777]: ts=2026-09-21T04:25:13Z level=info event=scheduler_connected addr=lb:500514826worker2 # [ 81.203586] nix-grpc-daemon[776]: ts=2026-09-21T04:25:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4827worker1 # [ 81.250861] nix-grpc-daemon[777]: ts=2026-09-21T04:25:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4828worker2 # [ 81.404672] nix-grpc-daemon[776]: ts=2026-09-21T04:25:14Z level=info event=scheduler_connected addr=lb:500514829worker1 # [ 81.451508] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_connected addr=lb:500514830worker2 # [ 81.409746] nix-grpc-daemon[776]: ts=2026-09-21T04:25:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4831worker1 # [ 81.456725] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4832worker2 # [ 81.610853] nix-grpc-daemon[776]: ts=2026-09-21T04:25:14Z level=info event=scheduler_connected addr=lb:500514833worker1 # [ 81.657464] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_connected addr=lb:500514834worker2 # [ 81.616093] nix-grpc-daemon[776]: ts=2026-09-21T04:25:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4835worker1 # [ 81.662559] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4836worker1 # [ 81.863300] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_connected addr=lb:500514837worker2 # [ 81.818717] nix-grpc-daemon[776]: ts=2026-09-21T04:25:14Z level=info event=scheduler_connected addr=lb:500514838worker2 # [ 81.822912] nix-grpc-daemon[776]: ts=2026-09-21T04:25:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4839worker1 # [ 81.868410] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4840worker2 # [ 81.997678] nix-grpc-daemon[776]: ts=2026-09-21T04:25:14Z level=info event=scheduler_restarting4841worker1 # [ 82.070956] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_connected addr=lb:500514842worker1 # [ 82.075093] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4843worker2 # [ 82.041819] nix-grpc-daemon[776]: ts=2026-09-21T04:25:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4844worker2 # [ 82.058372] nix-daemon[857]: reaped child process 860, status = succeeded4845worker1 # [ 82.234045] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_take_over4846lb # [ 82.187131] niks3-server[1038]: 2026/09/21 04:25:14 INFO lead: acquired remote=192.168.1.3:401764847worker1 # [ 82.271438] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_connected addr=lb:500514848worker1 # [ 82.276637] nix-grpc-daemon[777]: ts=2026-09-21T04:25:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4849worker2 # [ 82.381222] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4850worker2 # [ 82.387012] systemd[1]: Stopped Nix worker-protocol over gRPC.4851worker2 # [ 82.394889] systemd[1]: nix-grpc-daemon.service: Consumed 1.952s CPU time over 1min 2.947s wall clock time, 55.8M memory peak, 62.2K incoming IP traffic, 173.8K outgoing IP traffic.4852worker2 # [ 82.406904] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4853worker2 # [ 82.410284] systemd[1]: Closed Nix worker-protocol over gRPC.4854worker2 # [ 82.411866] systemd[1]: nix-grpc-daemon.socket: Consumed 573us CPU time over 1min 11.032s wall clock time, 256K memory peak, 78.9K incoming IP traffic, 68.4K outgoing IP traffic.4855lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4856lb: must succeed: curl -sf localhost:9901/clusters4857worker1 # [ 82.477504] nix-grpc-daemon[777]: ts=2026-09-21T04:25:15Z level=info event=scheduler_connected addr=lb:500514858worker1 # [ 82.482552] nix-grpc-daemon[777]: ts=2026-09-21T04:25:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4859lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4860lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4861client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24862worker1 # [ 82.683317] nix-grpc-daemon[777]: ts=2026-09-21T04:25:15Z level=info event=scheduler_connected addr=lb:500514863worker1 # [ 82.689064] nix-grpc-daemon[777]: ts=2026-09-21T04:25:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4864worker1 # [ 82.802602] nix-grpc-daemon[777]: ts=2026-09-21T04:25:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:580424865client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4866worker1 # [ 82.811046] nix-grpc-daemon[777]: ts=2026-09-21T04:25:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:580424867client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.29 seconds)4868worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4869worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)4870worker1 # [ 82.889732] nix-grpc-daemon[777]: ts=2026-09-21T04:25:15Z level=info event=scheduler_connected addr=lb:500514871worker1 # [ 82.908265] nix-grpc-daemon[777]: ts=2026-09-21T04:25:15Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:459144872worker1 # [ 82.913993] nix-grpc-daemon[777]: ts=2026-09-21T04:25:15Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04873worker2 # [ 82.893735] systemd[1]: Listening on Nix worker-protocol over gRPC.4874worker1 # [ 83.013883] systemd[1]: Stopping Nix worker-protocol over gRPC...4875worker1 # [ 83.689777] nix-grpc-daemon[777]: ts=2026-09-21T04:25:16Z level=info event=signal_exit4876worker2 # [ 83.879175] systemd[1]: Starting scheduler-token.service...4877worker1 # [ 83.940178] nix-grpc-daemon[777]: ts=2026-09-21T04:25:16Z level=info event=scheduler_yield4878worker1 # [ 83.949220] nix-grpc-daemon[777]: ts=2026-09-21T04:25:16Z level=info event=scheduler_restarting addr=lb:500514879lb # [ 83.902129] niks3-server[1038]: 2026/09/21 04:25:16 INFO lead: released remote=192.168.1.3:401764880worker1 # [ 83.954856] nix-grpc-daemon[777]: ts=2026-09-21T04:25:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:459144881worker1 # [ 83.959127] nix-grpc-daemon[777]: ts=2026-09-21T04:25:16Z level=info event=worker_gone addr=192.168.1.3:500514882worker2 # [ 83.981543] systemd[1]: scheduler-token.service: Deactivated successfully.4883worker2 # [ 83.988358] systemd[1]: Finished scheduler-token.service.4884worker2 # [ 83.991733] systemd[1]: scheduler-token.service: Consumed 50ms CPU time over 103ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4885worker2 # [ 83.997701] systemd[1]: Starting Nix worker-protocol over gRPC...4886worker1 # [ 84.154450] nix-grpc-daemon[777]: ts=2026-09-21T04:25:16Z level=info event=scheduler_connected addr=lb:500514887worker1 # [ 84.164496] nix-grpc-daemon[777]: ts=2026-09-21T04:25:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4888worker2 # [ 84.176431] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4889worker2 # [ 84.178502] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4890worker2 # [ 84.182676] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14891worker2 # [ 84.190836] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4892worker2 # [ 84.199824] systemd[1]: Started Nix worker-protocol over gRPC.4893worker2 # [ 84.205672] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=scheduler_take_over4894lb # [ 84.203779] niks3-server[1038]: 2026/09/21 04:25:16 INFO lead: acquired remote=192.168.1.4:424024895worker2 # [ 84.235124] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=scheduler_connected addr=lb:500514896worker2 # [ 84.242572] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=oidc_jwks provider=mock keys=14897worker2 # [ 84.243861] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:574144898worker2 # [ 84.245823] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04899worker1 # [ 84.365487] nix-grpc-daemon[777]: ts=2026-09-21T04:25:16Z level=info event=scheduler_connected addr=lb:500514900worker2 # [ 84.321611] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:574144901worker2 # [ 84.325881] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04902worker2 # [ 84.908138] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:574144903worker2 # [ 84.908552] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:17Z level=info event=worker_gone addr=192.168.1.3:500514904worker1 # [ 84.953095] nix-grpc-daemon[777]: ts=2026-09-21T04:25:17Z level=info event=scheduler_restarting4905worker1 # [ 84.989619] nix-grpc-daemon[777]: ts=2026-09-21T04:25:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4906worker1 # [ 85.008965] nix-daemon[838]: reaped child process 840, status = succeeded4907worker1 # [ 85.802488] nix-grpc-daemon[777]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4908worker1 # [ 85.802893] nix-grpc-daemon[777]: E0000 00:00:1789964718.428426 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 11, started true, shutdown true4909worker1 # [ 85.835500] nix-grpc-daemon[777]: E0000 00:00:1789964718.461628 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4910worker1 # [ 86.349586] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4911worker1 # [ 86.360342] systemd[1]: Stopped Nix worker-protocol over gRPC.4912worker1 # [ 86.362920] systemd[1]: nix-grpc-daemon.service: Consumed 3.673s CPU time over 1min 7.005s wall clock time, 64M memory peak, 90K incoming IP traffic, 164.7K outgoing IP traffic.4913worker1 # [ 86.380259] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4914worker1 # [ 86.383322] systemd[1]: Closed Nix worker-protocol over gRPC.4915worker1 # [ 86.386423] systemd[1]: nix-grpc-daemon.socket: Consumed 560us CPU time over 1min 15.694s wall clock time, 256K memory peak, 90.3K incoming IP traffic, 74.2K outgoing IP traffic.4916lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4917lb: must succeed: curl -sf localhost:9901/clusters4918lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4919lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4920client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24921worker2 # [ 86.695597] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:19Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:574304922worker2 # [ 86.711843] nix-daemon[857]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4923worker2 # [ 86.780617] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:574304924worker2 # [ 86.794759] nix-daemon[857]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4925worker2 # [ 86.821830] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5)4926worker2 # [ 86.919338] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 158 ms (attempt 2/5)4927worker2 # [ 87.084977] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 287 ms (attempt 3/5)4928worker2 # [ 87.379750] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 533 ms (attempt 4/5)4929worker2 # [ 87.922218] nix-grpc-daemon[1104]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4930worker2 # [ 87.928201] nix-grpc-daemon[1104]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4931client # copying 2 paths...4932client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4933worker2 # [ 87.950262] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:574304934client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4935worker2 # [ 87.970117] nix-daemon[857]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4936worker2 # [ 87.998879] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:20Z level=info event=niks3_push_started pid=11654937worker2 # [ 88.219352] nix-grpc-daemon[1165]: time=2026-09-21T04:25:20.858Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4938worker2 # [ 88.391232] nix-daemon[857]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)4939worker2 # [ 88.407788] nix-daemon[857]: reaped child process 1181, status = succeeded4940lb # [ 88.419244] niks3-server[1038]: 2026/09/21 04:25:21 INFO Received uploads request method=POST path=/api/pending_closures4941lb # [ 88.430017] niks3-server[1038]: 2026/09/21 04:25:21 INFO Received uploads request method=POST path=/api/pending_closures4942worker2 # [ 88.443218] nix-grpc-daemon[1165]: time=2026-09-21T04:25:21.081Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4943worker2 # [ 88.445768] nix-grpc-daemon[1165]: time=2026-09-21T04:25:21.081Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4944worker2 # [ 88.447622] nix-grpc-daemon[1165]: time=2026-09-21T04:25:21.081Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4945lb # [ 88.476423] niks3-server[1038]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4946lb # [ 88.498772] niks3-server[1038]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4947lb # [ 88.506488] niks3-server[1038]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4948worker2 # [ 88.520153] nix-grpc-daemon[1165]: time=2026-09-21T04:25:21.158Z level=INFO msg="Uploading 3 narinfos"4949lb # [ 88.520846] niks3-server[1038]: 2026/09/21 04:25:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4950lb # [ 88.532853] niks3-server[1038]: 2026/09/21 04:25:21 INFO Signed narinfos id=26 count=24951lb # [ 88.535150] niks3-server[1038]: 2026/09/21 04:25:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4952lb # [ 88.549797] niks3-server[1038]: 2026/09/21 04:25:21 INFO Signed narinfos id=25 count=14953lb # [ 88.566435] niks3-server[1038]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4954lb # [ 88.579207] niks3-server[1038]: 2026/09/21 04:25:21 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4955lb # [ 88.584670] niks3-server[1038]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4956lb # [ 88.593168] niks3-server[1038]: 2026/09/21 04:25:21 INFO Completed upload id=254957lb # [ 88.596925] niks3-server[1038]: 2026/09/21 04:25:21 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4958worker2 # [ 88.598421] nix-grpc-daemon[1165]: time=2026-09-21T04:25:21.238Z level=INFO msg="Upload complete. (374ms)"4959lb # [ 88.600304] niks3-server[1038]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4960lb # [ 88.603550] niks3-server[1038]: 2026/09/21 04:25:21 INFO Completed upload id=264961lb # [ 88.610008] niks3-server[1038]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4962worker2 # [ 88.610737] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57430 duration_s=0 paths=2 nar_bytes_in=3124963worker2 # [ 88.636981] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:21Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:57430 duration_s=1 bytes_in=400 bytes_out=2004964client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.22 seconds)4965worker2 # [ 88.646966] nix-daemon[857]: reaped child process 1163, status = succeeded4966worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4967worker2 # [ 88.649394] nix-daemon[857]: reaped child process 1154, status = succeeded4968worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)4969worker1 # [ 88.809088] systemd[1]: Listening on Nix worker-protocol over gRPC.4970worker1 # [ 88.815545] systemd[1]: Starting Nix worker-protocol over gRPC...4971worker1 # [ 89.012461] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4972worker1 # [ 89.015224] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4973worker1 # [ 89.019317] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14974worker1 # [ 89.027567] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4975worker1 # [ 89.033879] systemd[1]: Started Nix worker-protocol over gRPC.4976lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4977lb: must succeed: curl -sf localhost:9901/clusters4978worker1 # [ 89.073794] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:21Z level=info event=scheduler_connected addr=lb:500514979worker2 # [ 89.032312] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:574144980worker2 # [ 89.037129] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04981lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4982lb # [ 89.295840] envoy[902]: conn peer=[2001:db8:1::4]:51832 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=53490 tx=2737 ms=622564983lb # [ 89.305082] envoy[902]: conn peer=[2001:db8:1::1]:43976 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=204984lb # [ 89.310266] envoy[902]: conn peer=[2001:db8:1::3]:42854 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12220 tx=3401 ms=652564985lb # [ 89.315749] envoy[902]: conn peer=[2001:db8:1::1]:43986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1632 tx=617 ms=19304986lb: must succeed: curl -sf localhost:9901/clusters4987lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4988lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.18 seconds)4989(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.17 seconds)4990subtest: scheduler down: the other node takes the lock and keeps it4991lb: must succeed: curl -sf localhost:9901/clusters4992lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4993lb: must succeed: curl -sf localhost:9901/clusters4994lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4995lb: must succeed: curl -sf localhost:9901/clusters4996lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4997lb: must succeed: curl -sf localhost:9901/clusters4998lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4999worker2 # [ 90.450281] systemd[1]: Stopping Nix worker-protocol over gRPC...5000worker2 # [ 91.203120] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:23Z level=info event=signal_exit5001worker2 # [ 91.413337] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_yield5002lb # [ 91.413260] niks3-server[1038]: 2026/09/21 04:25:24 INFO lead: released remote=192.168.1.4:424025003worker1 # [ 91.463573] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:24Z level=info event=scheduler_restarting addr=lb:500515004worker2 # [ 91.427581] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:574145005worker2 # [ 91.434583] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=worker_gone addr=192.168.1.3:500515006worker2 # [ 91.438545] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_restarting addr=lb:500515007worker2 # [ 91.442266] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:574145008worker2 # [ 91.446378] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=worker_gone addr=192.168.1.4:500515009worker1 # [ 91.663632] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500515010worker1 # [ 91.669149] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5011worker2 # [ 91.635074] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500515012worker2 # [ 91.640379] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5013worker1 # [ 91.869883] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500515014worker1 # [ 91.875349] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5015worker2 # [ 91.841457] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500515016worker2 # [ 91.846345] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5017worker1 # [ 92.076234] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500515018worker1 # [ 92.081329] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5019worker2 # [ 92.047390] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500515020worker2 # [ 92.052612] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5021worker1 # [ 92.282261] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500515022worker1 # [ 92.287476] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5023worker2 # [ 92.255209] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500515024worker2 # [ 92.259380] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5025worker2 # [ 92.418339] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:25Z level=info event=scheduler_restarting5026worker1 # [ 92.489964] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500515027worker2 # [ 92.447836] nix-daemon[857]: reaped child process 1157, status = succeeded5028worker1 # [ 92.494208] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5029worker2 # [ 92.475148] nix-grpc-daemon[1104]: ts=2026-09-21T04:25:25Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5030worker1 # [ 92.690368] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500515031worker1 # [ 92.695469] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5032worker1 # [ 92.896343] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500515033worker1 # [ 92.901273] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5034worker1 # [ 93.102174] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500515035worker1 # [ 93.107275] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5036worker2 # [ 93.222530] nix-grpc-daemon[1104]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5037worker2 # [ 93.222879] nix-grpc-daemon[1104]: E0000 00:00:1789964725.862218 1147 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5038worker2 # [ 93.234381] nix-grpc-daemon[1104]: E0000 00:00:1789964725.874284 1147 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5039worker1 # [ 93.308112] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500515040worker1 # [ 93.313282] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5041worker1 # [ 93.515763] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_connected addr=lb:500515042worker1 # [ 93.520098] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5043worker1 # [ 93.716387] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_connected addr=lb:500515044worker1 # [ 93.721453] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5045worker1 # [ 93.922266] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_connected addr=lb:500515046worker1 # [ 93.927448] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5047worker1 # [ 94.039760] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_take_over5048lb # [ 93.993092] niks3-server[1038]: 2026/09/21 04:25:26 INFO lead: acquired remote=192.168.1.3:592145049worker1 # [ 94.128310] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_connected addr=lb:500515050worker1 # [ 94.133341] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5051worker2 # [ 94.235360] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5052worker2 # [ 94.239166] systemd[1]: Stopped Nix worker-protocol over gRPC.5053worker2 # [ 94.244854] systemd[1]: nix-grpc-daemon.service: Consumed 471ms CPU time over 10.235s wall clock time, 34.7M memory peak, 19.7K incoming IP traffic, 26.1K outgoing IP traffic.5054worker2 # [ 94.259868] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5055worker2 # [ 94.265282] systemd[1]: Closed Nix worker-protocol over gRPC.5056worker2 # [ 94.268202] systemd[1]: nix-grpc-daemon.socket: Consumed 712us CPU time over 11.373s wall clock time, 256K memory peak, 22.2K incoming IP traffic, 16.9K outgoing IP traffic.5057worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5058worker1 # [ 94.334581] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_connected addr=lb:500515059worker1 # [ 94.342652] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5060worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.10 seconds)5061worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5062worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5063worker1 # [ 94.543108] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:27Z level=info event=scheduler_connected addr=lb:500515064worker1 # [ 94.551254] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:584365065worker1 # [ 94.556592] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05066worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5067worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5068client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25069worker1 # [ 95.809120] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584405070worker1 # [ 95.823574] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5071client # don't know how to build these paths:5072client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5073worker1 # [ 95.844292] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584405074worker1 # [ 95.866859] nix-grpc-daemon[1224]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 1/5)5075worker1 # [ 95.920503] nix-grpc-daemon[1224]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)5076worker1 # [ 96.118696] nix-grpc-daemon[1224]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 3/5)5077worker1 # [ 96.260315] nix-grpc-daemon[1224]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 366 ms (attempt 4/5)5078worker1 # [ 96.627046] nix-grpc-daemon[1224]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5079worker1 # [ 96.633252] nix-grpc-daemon[1224]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5080worker1 # [ 96.663380] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:29Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05081worker1 # [ 96.670417] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:29Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15082worker1 # [ 96.708217] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584405083worker1 # [ 96.723363] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584405084client # copying 1 paths to 'lb:50051'...5085client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5086worker1 # [ 96.737558] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584405087worker1 # [ 96.751492] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5088worker1 # [ 96.779177] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:29Z level=info event=niks3_push_started pid=12955089worker1 # [ 97.006431] nix-grpc-daemon[1295]: time=2026-09-21T04:25:29.631Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5090worker1 # [ 97.186957] nix-daemon[838]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5091worker1 # [ 97.205205] nix-daemon[838]: reaped child process 1311, status = succeeded5092worker1 # [ 97.386530] nix-daemon[838]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5093worker1 # [ 97.405311] nix-daemon[838]: reaped child process 1316, status = succeeded5094lb # [ 97.370604] niks3-server[1038]: 2026/09/21 04:25:29 INFO Received uploads request method=POST path=/api/pending_closures5095worker1 # [ 97.431368] nix-grpc-daemon[1295]: time=2026-09-21T04:25:30.056Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5096worker1 # [ 97.433256] nix-grpc-daemon[1295]: time=2026-09-21T04:25:30.056Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5097lb # [ 97.418731] niks3-server[1038]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5098lb # [ 97.431886] niks3-server[1038]: 2026/09/21 04:25:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5099worker1 # [ 97.484513] nix-grpc-daemon[1295]: time=2026-09-21T04:25:30.110Z level=INFO msg="Uploading 1 narinfos"5100lb # [ 97.442549] niks3-server[1038]: 2026/09/21 04:25:30 INFO Signed narinfos id=27 count=15101lb # [ 97.447760] niks3-server[1038]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5102lb # [ 97.459076] niks3-server[1038]: 2026/09/21 04:25:30 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5103worker1 # [ 97.514546] nix-grpc-daemon[1295]: time=2026-09-21T04:25:30.140Z level=INFO msg="Upload complete. (504ms)"5104lb # [ 97.468298] niks3-server[1038]: 2026/09/21 04:25:30 INFO Completed upload id=275105lb # [ 97.472518] niks3-server[1038]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5106worker1 # [ 97.526645] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58440 duration_s=0 paths=1 nar_bytes_in=5205107worker1 # [ 97.540185] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584405108worker1 # [ 97.549187] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5109worker1 # [ 97.551889] nix-daemon[838]: reaped child process 1293, status = succeeded5110client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5111client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5112worker1 # [ 97.565191] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5113client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5114client # farm-a> LOG-farm-a-on-standby5115worker1 # [ 98.410637] nix-daemon[838]: reaped child process 1320, status = succeeded5116worker1 # [ 98.585385] nix-daemon[838]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)5117worker1 # [ 98.603855] nix-daemon[838]: reaped child process 1339, status = succeeded5118lb # [ 98.568939] niks3-server[1038]: 2026/09/21 04:25:31 INFO Received uploads request method=POST path=/api/pending_closures5119worker1 # [ 98.630321] nix-grpc-daemon[1295]: time=2026-09-21T04:25:31.256Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5120worker1 # [ 98.638429] nix-grpc-daemon[1295]: time=2026-09-21T04:25:31.256Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5121lb # [ 98.609012] niks3-server[1038]: 2026/09/21 04:25:31 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5122lb # [ 98.629132] niks3-server[1038]: 2026/09/21 04:25:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5123worker1 # [ 98.684248] nix-grpc-daemon[1295]: time=2026-09-21T04:25:31.309Z level=INFO msg="Uploading 1 narinfos"5124lb # [ 98.644956] niks3-server[1038]: 2026/09/21 04:25:31 INFO Signed narinfos id=28 count=15125lb # [ 98.652250] niks3-server[1038]: 2026/09/21 04:25:31 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5126lb # [ 98.659536] niks3-server[1038]: 2026/09/21 04:25:31 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5127lb # [ 98.668319] niks3-server[1038]: 2026/09/21 04:25:31 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5128lb # [ 98.673043] niks3-server[1038]: 2026/09/21 04:25:31 INFO Completed upload id=285129worker1 # [ 98.720797] nix-grpc-daemon[1295]: time=2026-09-21T04:25:31.346Z level=INFO msg="Upload complete. (344ms)"5130lb # [ 98.676878] niks3-server[1038]: 2026/09/21 04:25:31 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5131worker1 # [ 98.748805] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58440 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15132worker1 # [ 98.760327] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:31Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05133worker1 # [ 98.763959] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:31Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15134worker1 # [ 98.769575] nix-daemon[838]: reaped child process 1318, status = succeeded5135worker1 # [ 98.787434] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584405136worker1 # [ 98.802435] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584405137worker1 # [ 98.817530] nix-grpc-daemon[1295]: time=2026-09-21T04:25:31.443Z level=INFO msg="All 1 paths already cached"5138client # copying 1 paths to 'lb:50051'...5139client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5140worker1 # [ 98.827524] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584405141worker1 # [ 98.841318] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5142worker1 # [ 99.042078] nix-daemon[838]: accepted connection from pid 1345, user nix-grpc-daemon (trusted)5143worker1 # [ 99.058312] nix-daemon[838]: reaped child process 1348, status = succeeded5144worker1 # [ 99.227811] nix-daemon[838]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5145worker1 # [ 99.242297] nix-daemon[838]: reaped child process 1353, status = succeeded5146lb # [ 99.209991] niks3-server[1038]: 2026/09/21 04:25:31 INFO Received uploads request method=POST path=/api/pending_closures5147worker1 # [ 99.268559] nix-grpc-daemon[1295]: time=2026-09-21T04:25:31.894Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5148worker1 # [ 99.270045] nix-grpc-daemon[1295]: time=2026-09-21T04:25:31.894Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5149lb # [ 99.241912] niks3-server[1038]: 2026/09/21 04:25:31 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5150lb # [ 99.253301] niks3-server[1038]: 2026/09/21 04:25:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5151worker1 # [ 99.308254] nix-grpc-daemon[1295]: time=2026-09-21T04:25:31.934Z level=INFO msg="Uploading 1 narinfos"5152lb # [ 99.263798] niks3-server[1038]: 2026/09/21 04:25:31 INFO Signed narinfos id=29 count=15153lb # [ 99.270156] niks3-server[1038]: 2026/09/21 04:25:31 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5154lb # [ 99.277559] niks3-server[1038]: 2026/09/21 04:25:31 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5155lb # [ 99.283812] niks3-server[1038]: 2026/09/21 04:25:31 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5156lb # [ 99.289078] niks3-server[1038]: 2026/09/21 04:25:31 INFO Completed upload id=295157worker1 # [ 99.336678] nix-grpc-daemon[1295]: time=2026-09-21T04:25:31.962Z level=INFO msg="Upload complete. (479ms)"5158lb # [ 99.292529] envoy[902]: conn peer=[2001:db8:1::4]:44802 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4158 tx=349 ms=82295159worker1 # [ 99.344274] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58440 duration_s=0 paths=1 nar_bytes_in=6405160worker1 # [ 99.357732] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584405161worker1 # [ 99.363971] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5162worker1 # [ 99.369214] nix-daemon[838]: reaped child process 1343, status = succeeded5163client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5164client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5165worker1 # [ 99.383155] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5166client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5167client # farm-b> LOG-farm-b-on-standby5168worker1 # [ 100.434224] nix-daemon[838]: reaped child process 1357, status = succeeded5169worker1 # [ 100.615108] nix-daemon[838]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)5170worker1 # [ 100.632323] nix-daemon[838]: reaped child process 1376, status = succeeded5171lb # [ 100.597321] niks3-server[1038]: 2026/09/21 04:25:33 INFO Received uploads request method=POST path=/api/pending_closures5172worker1 # [ 100.655513] nix-grpc-daemon[1295]: time=2026-09-21T04:25:33.281Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5173worker1 # [ 100.657614] nix-grpc-daemon[1295]: time=2026-09-21T04:25:33.281Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5174lb # [ 100.633730] niks3-server[1038]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5175lb # [ 100.646376] niks3-server[1038]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5176lb # [ 100.655665] niks3-server[1038]: 2026/09/21 04:25:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5177worker1 # [ 100.708616] nix-grpc-daemon[1295]: time=2026-09-21T04:25:33.334Z level=INFO msg="Uploading 1 narinfos"5178lb # [ 100.662843] niks3-server[1038]: 2026/09/21 04:25:33 INFO Signed narinfos id=30 count=15179lb # [ 100.668265] niks3-server[1038]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5180lb # [ 100.674418] niks3-server[1038]: 2026/09/21 04:25:33 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5181lb # [ 100.682765] niks3-server[1038]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5182lb # [ 100.686402] niks3-server[1038]: 2026/09/21 04:25:33 INFO Completed upload id=305183worker1 # [ 100.734116] nix-grpc-daemon[1295]: time=2026-09-21T04:25:33.360Z level=INFO msg="Upload complete. (340ms)"5184worker1 # [ 100.752876] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58440 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15185worker1 # [ 100.767436] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:33Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05186worker1 # [ 100.771428] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:33Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15187worker1 # [ 100.775474] nix-daemon[838]: reaped child process 1355, status = succeeded5188worker1 # [ 100.796844] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584405189worker1 # [ 100.811556] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584405190worker1 # [ 100.825400] nix-grpc-daemon[1295]: time=2026-09-21T04:25:33.451Z level=INFO msg="All 2 paths already cached"5191client # copying 1 paths to 'lb:50051'...5192client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5193worker1 # [ 100.833961] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584405194worker1 # [ 100.847865] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5195worker1 # [ 101.054697] nix-daemon[838]: accepted connection from pid 1382, user nix-grpc-daemon (trusted)5196worker1 # [ 101.072252] nix-daemon[838]: reaped child process 1385, status = succeeded5197worker1 # [ 101.243628] nix-daemon[838]: accepted connection from pid 1387, user nix-grpc-daemon (trusted)5198worker1 # [ 101.259706] nix-daemon[838]: reaped child process 1390, status = succeeded5199lb # [ 101.223529] niks3-server[1038]: 2026/09/21 04:25:33 INFO Received uploads request method=POST path=/api/pending_closures5200worker1 # [ 101.280032] nix-grpc-daemon[1295]: time=2026-09-21T04:25:33.905Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5201worker1 # [ 101.281540] nix-grpc-daemon[1295]: time=2026-09-21T04:25:33.906Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5202lb # [ 101.255506] niks3-server[1038]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5203worker1 # [ 101.310318] nix-grpc-daemon[1295]: time=2026-09-21T04:25:33.936Z level=INFO msg="Uploading 1 narinfos"5204lb # [ 101.269613] niks3-server[1038]: 2026/09/21 04:25:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5205lb # [ 101.279139] niks3-server[1038]: 2026/09/21 04:25:33 INFO Signed narinfos id=31 count=15206lb # [ 101.287909] niks3-server[1038]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5207worker1 # [ 101.341328] nix-grpc-daemon[1295]: time=2026-09-21T04:25:33.967Z level=INFO msg="Upload complete. (477ms)"5208lb # [ 101.294350] niks3-server[1038]: 2026/09/21 04:25:33 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5209lb # [ 101.300197] niks3-server[1038]: 2026/09/21 04:25:33 INFO Completed upload id=315210lb # [ 101.303614] niks3-server[1038]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5211worker1 # [ 101.351759] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58440 duration_s=0 paths=1 nar_bytes_in=7685212worker1 # [ 101.360253] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584405213worker1 # [ 101.368569] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5214worker1 # [ 101.370992] nix-daemon[838]: reaped child process 1380, status = succeeded5215client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5216worker1 # [ 101.382788] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5217client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5218client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5219client # farm-top> LOG-farm-top-on-standby5220worker1 # [ 102.404933] nix-daemon[838]: reaped child process 1394, status = succeeded5221worker1 # [ 102.560718] nix-daemon[838]: accepted connection from pid 1410, user nix-grpc-daemon (trusted)5222worker1 # [ 102.578671] nix-daemon[838]: reaped child process 1413, status = succeeded5223lb # [ 102.543947] niks3-server[1038]: 2026/09/21 04:25:35 INFO Received uploads request method=POST path=/api/pending_closures5224worker1 # [ 102.602666] nix-grpc-daemon[1295]: time=2026-09-21T04:25:35.227Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5225worker1 # [ 102.604234] nix-grpc-daemon[1295]: time=2026-09-21T04:25:35.227Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5226lb # [ 102.574056] niks3-server[1038]: 2026/09/21 04:25:35 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5227lb # [ 102.591425] niks3-server[1038]: 2026/09/21 04:25:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5228worker1 # [ 102.644488] nix-grpc-daemon[1295]: time=2026-09-21T04:25:35.270Z level=INFO msg="Uploading 1 narinfos"5229lb # [ 102.604294] niks3-server[1038]: 2026/09/21 04:25:35 INFO Signed narinfos id=32 count=15230lb # [ 102.606831] niks3-server[1038]: 2026/09/21 04:25:35 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5231lb # [ 102.618894] niks3-server[1038]: 2026/09/21 04:25:35 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5232lb # [ 102.622540] niks3-server[1038]: 2026/09/21 04:25:35 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5233worker1 # [ 102.666152] nix-grpc-daemon[1295]: time=2026-09-21T04:25:35.292Z level=INFO msg="Upload complete. (274ms)"5234lb # [ 102.627797] niks3-server[1038]: 2026/09/21 04:25:35 INFO Completed upload id=325235lb # [ 102.629716] niks3-server[1038]: 2026/09/21 04:25:35 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5236worker1 # [ 102.674852] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58440 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15237worker1 # [ 102.689539] nix-daemon[838]: reaped child process 1392, status = succeeded5238client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 7.19 seconds)5239client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5240client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5241worker2 # [ 103.019723] systemd[1]: Listening on Nix worker-protocol over gRPC.5242worker2 # [ 103.029362] systemd[1]: Starting scheduler-token.service...5243worker2 # [ 103.128937] systemd[1]: scheduler-token.service: Deactivated successfully.5244worker2 # [ 103.135794] systemd[1]: Finished scheduler-token.service.5245worker2 # [ 103.139053] systemd[1]: scheduler-token.service: Consumed 40ms CPU time over 103ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5246worker2 # [ 103.145718] systemd[1]: Starting Nix worker-protocol over gRPC...5247worker2 # [ 103.325747] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5248worker2 # [ 103.328403] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5249worker2 # [ 103.332507] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15250worker2 # [ 103.340100] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5251worker2 # [ 103.352248] systemd[1]: Started Nix worker-protocol over gRPC.5252lb: must succeed: curl -sf localhost:9901/clusters5253worker2 # [ 103.393651] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:36Z level=info event=scheduler_connected addr=lb:500515254worker1 # [ 103.452185] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:36Z level=info event=oidc_jwks provider=mock keys=15255worker1 # [ 103.457933] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:584365256worker1 # [ 103.463305] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05257lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5258worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5259worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5260lb: must succeed: curl -sf localhost:9901/clusters5261lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5262lb: must succeed: curl -sf localhost:9901/clusters5263lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5264worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5265worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5266(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.56 seconds)5267subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5268client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5269client # [ 103.804500] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5270client # Running as unit: rs.service; invocation ID: 5ad6c5a3e2b345bcaa63e182b4e34f7e5271client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5272worker2 # [ 104.079929] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:371205273worker2 # [ 104.095646] nix-daemon[857]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5274client # [ 104.102933] nix[1188]: don't know how to build these paths:5275client # [ 104.104053] nix[1188]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5276worker1 # [ 104.172368] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:36Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05277worker1 # [ 104.172643] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:36Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15278worker1 # [ 104.216113] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584405279worker1 # [ 104.231653] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584405280client # [ 104.188325] nix[1188]: copying 1 paths to 'lb:50051'...5281client # [ 104.195123] nix[1188]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5282worker1 # [ 104.248411] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584405283worker1 # [ 104.263415] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5284worker1 # [ 104.468233] nix-daemon[838]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5285worker1 # [ 104.485808] nix-daemon[838]: reaped child process 1429, status = succeeded5286worker1 # [ 104.664256] nix-daemon[838]: accepted connection from pid 1431, user nix-grpc-daemon (trusted)5287worker1 # [ 104.679300] nix-daemon[838]: reaped child process 1434, status = succeeded5288lb # [ 104.647396] niks3-server[1038]: 2026/09/21 04:25:37 INFO Received uploads request method=POST path=/api/pending_closures5289worker1 # [ 104.705434] nix-grpc-daemon[1295]: time=2026-09-21T04:25:37.331Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5290worker1 # [ 104.706871] nix-grpc-daemon[1295]: time=2026-09-21T04:25:37.331Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5291lb # [ 104.678165] niks3-server[1038]: 2026/09/21 04:25:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5292worker1 # [ 104.734142] nix-grpc-daemon[1295]: time=2026-09-21T04:25:37.360Z level=INFO msg="Uploading 1 narinfos"5293lb # [ 104.690466] niks3-server[1038]: 2026/09/21 04:25:37 INFO Signed narinfos id=33 count=15294lb # [ 104.699329] niks3-server[1038]: 2026/09/21 04:25:37 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5295worker1 # [ 104.761747] nix-grpc-daemon[1295]: time=2026-09-21T04:25:37.387Z level=INFO msg="Upload complete. (481ms)"5296lb # [ 104.709566] niks3-server[1038]: 2026/09/21 04:25:37 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5297lb # [ 104.721183] niks3-server[1038]: 2026/09/21 04:25:37 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5298lb # [ 104.726676] niks3-server[1038]: 2026/09/21 04:25:37 INFO Completed upload id=335299lb # [ 104.729285] niks3-server[1038]: 2026/09/21 04:25:37 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5300worker1 # [ 104.768446] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58440 duration_s=0 paths=1 nar_bytes_in=4165301worker1 # [ 104.781857] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584405302worker1 # [ 104.788501] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5303worker1 # [ 104.793390] nix-daemon[838]: reaped child process 1424, status = succeeded5304client # [ 104.747301] nix[1188]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5305worker1 # [ 104.805621] nix-daemon[838]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5306lb: must succeed: curl -sf localhost:9901/clusters5307lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5308lb: must succeed: curl -sf localhost:9901/clusters5309lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5310lb: must succeed: curl -sf localhost:9901/clusters5311lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5312lb: must succeed: curl -sf localhost:9901/clusters5313lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5314worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5315worker2 # [ 106.762944] systemd-journald[273]: Received client request to rotate journal, rotating.5316worker2 # [ 106.783564] systemd-journald[273]: Vacuuming done, freed 0B of archived journals from /var/log/journal/5e232d78feb34bd5ac3ee60ea479abf5.5317worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5318worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5319worker1 # [ 106.890288] systemd[1]: Stopping Nix worker-protocol over gRPC...5320worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5321client: must succeed: sleep 5.05322worker1 # [ 107.040212] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:39Z level=info event=draining5323lb # [ 109.292684] envoy[902]: conn peer=[2001:db8:1::1]:60208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9623 tx=4144 ms=68985324client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5325worker1 # [ 112.234691] nix-daemon[838]: reaped child process 1438, status = succeeded5326worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5327worker1 # [ 112.397750] nix-daemon[838]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)5328worker1 # [ 112.415840] nix-daemon[838]: reaped child process 1482, status = succeeded5329lb # [ 112.381614] niks3-server[1038]: 2026/09/21 04:25:45 INFO Received uploads request method=POST path=/api/pending_closures5330worker1 # [ 112.440391] nix-grpc-daemon[1295]: time=2026-09-21T04:25:45.065Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5331worker1 # [ 112.443554] nix-grpc-daemon[1295]: time=2026-09-21T04:25:45.065Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5332lb # [ 112.420891] niks3-server[1038]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5333lb # [ 112.432299] niks3-server[1038]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5334worker1 # [ 112.492305] nix-grpc-daemon[1295]: time=2026-09-21T04:25:45.118Z level=INFO msg="Uploading 1 narinfos"5335lb # [ 112.444700] niks3-server[1038]: 2026/09/21 04:25:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5336lb # [ 112.453226] niks3-server[1038]: 2026/09/21 04:25:45 INFO Signed narinfos id=34 count=15337lb # [ 112.458831] niks3-server[1038]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5338lb # [ 112.465181] niks3-server[1038]: 2026/09/21 04:25:45 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5339lb # [ 112.473123] niks3-server[1038]: 2026/09/21 04:25:45 INFO Completed upload id=345340worker1 # [ 112.519620] nix-grpc-daemon[1295]: time=2026-09-21T04:25:45.145Z level=INFO msg="Upload complete. (295ms)"5341lb # [ 112.475610] niks3-server[1038]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5342worker1 # [ 112.532987] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58440 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15343worker1 # [ 112.546931] nix-daemon[838]: reaped child process 1436, status = succeeded5344client # [ 112.520301] systemd[1]: rs.service: Deactivated successfully.5345client # [ 112.524878] systemd[1]: rs.service: Consumed 181ms CPU time over 8.719s wall clock time, 41.2M memory peak, 7K incoming IP traffic, 8.4K outgoing IP traffic.5346worker1 # [ 113.042134] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:45Z level=info event=signal_exit5347worker1 # [ 113.264125] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:45Z level=info event=scheduler_yield5348lb # [ 113.218123] niks3-server[1038]: 2026/09/21 04:25:45 INFO lead: released remote=192.168.1.3:592145349worker1 # [ 113.272973] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:45Z level=info event=scheduler_restarting addr=lb:500515350worker2 # [ 113.229456] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:45Z level=info event=scheduler_restarting addr=lb:500515351worker1 # [ 113.278474] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:584365352worker1 # [ 113.283244] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:45Z level=info event=worker_gone addr=192.168.1.4:500515353worker1 # [ 113.287293] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:584365354worker1 # [ 113.291730] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:45Z level=info event=worker_gone addr=192.168.1.3:500515355worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.13 seconds)5356client: waiting for success: ! systemctl is-active rs5357lb # [ 113.352143] niks3-server[1038]: 2026/09/21 04:25:45 INFO lead: acquired remote=192.168.1.4:608965358worker2 # [ 113.358766] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:45Z level=info event=scheduler_take_over5359client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5360client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5361worker1 # [ 113.473898] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=scheduler_connected addr=lb:500515362worker2 # [ 113.429191] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_connected addr=lb:500515363worker1 # [ 113.479270] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker2 # [ 113.434338] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5365client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5366client: must fail: journalctl -u rs | grep -q 'reconnecting'5367client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5368lb: must succeed: curl -sf localhost:9901/clusters5369lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5370worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5371worker2 # [ 113.635679] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_connected addr=lb:500515372worker1 # [ 113.681317] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=scheduler_connected addr=lb:500515373worker2 # [ 113.640448] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5374worker1 # [ 113.686076] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5375worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5376worker1 # [ 113.882933] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=scheduler_connected addr=lb:500515377worker2 # [ 113.841373] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_connected addr=lb:500515378worker1 # [ 113.888288] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5379worker2 # [ 113.846717] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5380worker1 # [ 114.088926] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=scheduler_connected addr=lb:500515381worker2 # [ 114.048003] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_connected addr=lb:500515382worker1 # [ 114.093956] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5383worker2 # [ 114.053482] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker1 # [ 114.270257] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=scheduler_restarting5385worker2 # [ 114.256646] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_connected addr=lb:500515386worker2 # [ 114.260798] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5387worker1 # [ 114.323236] nix-grpc-daemon[1224]: ts=2026-09-21T04:25:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5388worker1 # [ 114.328828] nix-daemon[838]: reaped child process 1285, status = succeeded5389worker1 # [ 114.422180] nix-grpc-daemon[1224]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5390worker1 # [ 114.427358] nix-grpc-daemon[1224]: E0000 00:00:1789964747.048257 1267 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5391worker1 # [ 114.434178] nix-grpc-daemon[1224]: E0000 00:00:1789964747.053419 1267 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5392worker2 # [ 114.457244] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:47Z level=info event=scheduler_connected addr=lb:500515393worker2 # [ 114.474818] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:47Z level=info event=oidc_jwks provider=mock keys=15394worker2 # [ 114.479405] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:342285395worker2 # [ 114.484824] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05396lb: must succeed: curl -sf localhost:9901/clusters5397lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5398worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5399worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5400worker1 # [ 115.645555] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5401worker1 # [ 115.651360] systemd[1]: Stopped Nix worker-protocol over gRPC.5402worker1 # [ 115.655288] systemd[1]: nix-grpc-daemon.service: Consumed 1.800s CPU time over 26.828s wall clock time, 37.1M memory peak, 48.6K incoming IP traffic, 90.9K outgoing IP traffic.5403worker1 # [ 115.671656] systemd[1]: Starting Nix worker-protocol over gRPC...5404worker1 # [ 115.862361] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5405worker1 # [ 115.864923] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5406worker1 # [ 115.868995] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15407lb: must succeed: curl -sf localhost:9901/clusters5408worker1 # [ 115.876686] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5409worker1 # [ 115.878449] systemd[1]: Started Nix worker-protocol over gRPC.5410worker1 # [ 115.916182] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:48Z level=info event=scheduler_connected addr=lb:500515411worker2 # [ 115.874229] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:342285412worker2 # [ 115.879632] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05413lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5414worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5415worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5416(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.25 seconds)5417subtest: scheduler restart mid-build: build finishes, no double build5418client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5419client # [ 116.050483] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5420client # Running as unit: mid.service; invocation ID: 62d5d94176b64c24acd4ffb680b0d2c35421client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5422worker2 # [ 116.311138] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:371205423client # [ 116.309313] nix[1224]: don't know how to build these paths:5424client # [ 116.310323] nix[1224]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5425worker2 # [ 116.330941] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:48Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05426worker2 # [ 116.338080] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:48Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15427worker2 # [ 116.378325] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371205428worker2 # [ 116.398958] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 1/5)5429worker2 # [ 116.500660] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 2/5)5430worker2 # [ 116.520985] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 344 ms (attempt 3/5)5431worker2 # [ 116.872506] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 4/5)5432worker2 # [ 117.037315] nix-grpc-daemon[1205]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5433worker2 # [ 117.042744] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5434worker2 # [ 117.062008] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:371205435client # [ 117.067550] nix[1224]: copying 1 paths to 'lb:50051'...5436worker2 # [ 117.077034] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:371205437client # [ 117.067950] nix[1224]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5438worker2 # [ 117.092439] nix-daemon[857]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5439worker2 # [ 117.120088] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:49Z level=info event=niks3_push_started pid=13215440worker2 # [ 117.329093] nix-grpc-daemon[1321]: time=2026-09-21T04:25:49.967Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5441worker2 # [ 117.515382] nix-daemon[857]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5442worker2 # [ 117.531190] nix-daemon[857]: reaped child process 1341, status = succeeded5443worker2 # [ 117.709727] nix-daemon[857]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5444worker2 # [ 117.725559] nix-daemon[857]: reaped child process 1346, status = succeeded5445lb # [ 117.735588] niks3-server[1038]: 2026/09/21 04:25:50 INFO Received uploads request method=POST path=/api/pending_closures5446worker2 # [ 117.747721] nix-grpc-daemon[1321]: time=2026-09-21T04:25:50.386Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5447worker2 # [ 117.749462] nix-grpc-daemon[1321]: time=2026-09-21T04:25:50.386Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5448lb # [ 117.767604] niks3-server[1038]: 2026/09/21 04:25:50 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5449worker2 # [ 117.788689] nix-grpc-daemon[1321]: time=2026-09-21T04:25:50.428Z level=INFO msg="Uploading 1 narinfos"5450lb # [ 117.779948] niks3-server[1038]: 2026/09/21 04:25:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5451lb # [ 117.792341] niks3-server[1038]: 2026/09/21 04:25:50 INFO Signed narinfos id=35 count=15452lb # [ 117.802941] niks3-server[1038]: 2026/09/21 04:25:50 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5453worker2 # [ 117.813628] nix-grpc-daemon[1321]: time=2026-09-21T04:25:50.453Z level=INFO msg="Upload complete. (481ms)"5454lb # [ 117.818447] niks3-server[1038]: 2026/09/21 04:25:50 INFO Completed upload id=355455worker2 # [ 117.820115] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37120 duration_s=0 paths=1 nar_bytes_in=4245456worker2 # [ 117.827846] nix-grpc-daemon[1205]: ts=2026-09-21T04:25:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371205457worker2 # [ 117.838014] nix-daemon[857]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5458worker2 # [ 117.842632] nix-daemon[857]: reaped child process 1319, status = succeeded5459lb # [ 117.842732] niks3-server[1038]: 2026/09/21 04:25:50 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5460worker2 # [ 117.848232] nix-grpc-daemon[1321]: time=2026-09-21T04:25:50.480Z level=WARN msg="Failed to register uploaded object" key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo error="server returned 404: object is not pending upload\n"5461worker2 # [ 117.858522] nix-daemon[857]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5462client # [ 117.847061] nix[1224]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5463lb # [ 119.290457] envoy[902]: conn peer=[2001:db8:1::1]:44568 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1429 ms=84225464lb: must succeed: curl -sf localhost:9901/clusters5465lb # [ 119.295446] envoy[902]: conn peer=[2001:db8:1::3]:34616 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3890 tx=1213 ms=252385466lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5467lb: must succeed: curl -sf localhost:9901/clusters5468lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5469worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5470worker2 # [ 120.029585] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1205 (nix-grpc-daemon) on client request.5471worker2 # [ 120.031107] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5472lb # [ 120.048788] niks3-server[1038]: 2026/09/21 04:25:52 INFO lead: released remote=192.168.1.4:608965473worker2 # [ 120.061448] systemd[1]: Starting scheduler-token.service...5474client # [ 120.049127] nix[1224]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5475worker1 # [ 120.104168] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:52Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5476client # [ 120.055530] nix[1224]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5477worker2 # [ 120.081868] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5478worker2 # [ 120.084347] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5479worker2 # [ 120.088777] systemd[1]: nix-grpc-daemon.service: Consumed 552ms CPU time over 16.933s wall clock time, 34.4M memory peak, 16.8K incoming IP traffic, 31.4K outgoing IP traffic.5480worker2 # [ 120.107706] nix-daemon[857]: reaped child process 1348, status = succeeded5481worker2 # [ 120.109798] nix-daemon[857]: reaped child process 1261, status = succeeded5482worker2 # [ 120.116528] nix-daemon[1350]: unexpected Nix daemon error: error: interrupted by the user5483worker2 # [ 120.122499] nix-daemon[857]: reaped child process 1350, status = failed with exit code 15484worker2 # [ 120.167576] systemd[1]: scheduler-token.service: Deactivated successfully.5485worker2 # [ 120.169053] systemd[1]: Finished scheduler-token.service.5486worker2 # [ 120.171055] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 103ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5487worker2 # [ 120.173726] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5488worker2 # [ 120.175501] systemd[1]: Starting scheduler-token.service...5489worker2 # [ 120.248156] systemd[1]: scheduler-token.service: Deactivated successfully.5490worker2 # [ 120.249137] systemd[1]: Finished scheduler-token.service.5491worker2 # [ 120.249854] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 75ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5492worker2 # [ 120.254218] systemd[1]: Starting Nix worker-protocol over gRPC...5493worker1 # [ 120.304356] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:52Z level=info event=scheduler_connected addr=lb:500515494worker2 # [ 120.420466] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5495worker2 # [ 120.423115] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5496worker2 # [ 120.427146] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15497worker2 # [ 120.434905] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5498lb # [ 120.446717] niks3-server[1038]: 2026/09/21 04:25:53 INFO lead: acquired remote=192.168.1.4:492125499worker2 # [ 120.459041] systemd[1]: Started Nix worker-protocol over gRPC.5500worker2 # [ 120.459881] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=scheduler_take_over5501worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.49 seconds)5502worker2 # [ 120.471194] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:342365503client: must succeed: sleep 5.05504worker2 # [ 120.473930] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05505worker2 # [ 120.487210] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=scheduler_connected addr=lb:500515506worker2 # [ 120.493158] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=oidc_jwks provider=mock keys=15507worker2 # [ 120.494421] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:342365508worker2 # [ 120.496258] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05509worker2 # [ 120.576423] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05510worker2 # [ 120.580613] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:53Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15511worker1 # [ 120.682401] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392405512worker1 # [ 120.695955] nix-daemon[838]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5513worker1 # [ 120.720268] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5)5514worker1 # [ 120.749224] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 2/5)5515worker1 # [ 120.846710] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 3/5)5516worker1 # [ 120.911191] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 392 ms (attempt 4/5)5517worker1 # [ 121.311734] nix-grpc-daemon[1489]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5518worker1 # [ 121.317636] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5519worker1 # [ 121.334021] nix-grpc-daemon[1489]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5520worker1 # [ 121.360316] nix-daemon[838]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5521client # [ 121.321936] nix[1224]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5522worker1 # [ 121.377603] nix-daemon[838]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5523client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5524client: waiting for success: ! systemctl is-active mid5525worker1 # [ 126.005168] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:58Z level=info event=niks3_push_started pid=15925526worker1 # [ 126.053474] nix-daemon[838]: reaped child process 1567, status = succeeded5527worker1 # [ 126.241648] nix-grpc-daemon[1592]: time=2026-09-21T04:25:58.867Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5528worker1 # [ 126.416531] nix-daemon[838]: accepted connection from pid 1605, user nix-grpc-daemon (trusted)5529worker1 # [ 126.433962] nix-daemon[838]: reaped child process 1608, status = succeeded5530lb # [ 126.401017] niks3-server[1038]: 2026/09/21 04:25:59 INFO Received uploads request method=POST path=/api/pending_closures5531worker1 # [ 126.463816] nix-grpc-daemon[1592]: time=2026-09-21T04:25:59.088Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5532worker1 # [ 126.466597] nix-grpc-daemon[1592]: time=2026-09-21T04:25:59.088Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5533lb # [ 126.465872] niks3-server[1038]: 2026/09/21 04:25:59 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5534worker1 # [ 126.523350] nix-grpc-daemon[1592]: time=2026-09-21T04:25:59.148Z level=INFO msg="Uploading 1 narinfos"5535lb # [ 126.479540] niks3-server[1038]: 2026/09/21 04:25:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5536lb # [ 126.489579] niks3-server[1038]: 2026/09/21 04:25:59 INFO Signed narinfos id=36 count=15537lb # [ 126.495004] niks3-server[1038]: 2026/09/21 04:25:59 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5538lb # [ 126.508173] niks3-server[1038]: 2026/09/21 04:25:59 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5539worker1 # [ 126.560612] nix-grpc-daemon[1592]: time=2026-09-21T04:25:59.186Z level=INFO msg="Upload complete. (315ms)"5540lb # [ 126.513020] niks3-server[1038]: 2026/09/21 04:25:59 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5541lb # [ 126.517315] niks3-server[1038]: 2026/09/21 04:25:59 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5542lb # [ 126.523559] niks3-server[1038]: 2026/09/21 04:25:59 INFO Completed upload id=365543worker1 # [ 126.572369] nix-grpc-daemon[1489]: ts=2026-09-21T04:25:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39240 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15544worker1 # [ 126.590349] nix-daemon[838]: reaped child process 1565, status = succeeded5545client # [ 126.566870] systemd[1]: mid.service: Deactivated successfully.5546client # [ 126.571185] systemd[1]: mid.service: Consumed 182ms CPU time over 10.519s wall clock time, 43.2M memory peak, 8.6K incoming IP traffic, 10.4K outgoing IP traffic.5547client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5548client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5549client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5550(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.04 seconds)5551subtest: interrupting the client stops the build on the worker5552client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5553client # [ 127.060779] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5554client # Running as unit: intr.service; invocation ID: 067a4c0df3ce438cb7560d35e5b2c90d5555client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.05 seconds)5556worker2 # [ 127.325743] nix-grpc-daemon[1393]: ts=2026-09-21T04:25:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:340345557worker2 # [ 127.338929] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5558client # [ 127.347437] nix[1259]: don't know how to build these paths:5559client # [ 127.349028] nix[1259]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5560worker2 # [ 127.365377] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:00Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05561worker2 # [ 127.371995] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:00Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15562worker2 # [ 127.418207] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340345563worker2 # [ 127.435443] nix-grpc-daemon[1393]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)5564worker2 # [ 127.463745] nix-grpc-daemon[1393]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 2/5)5565worker2 # [ 127.655656] nix-grpc-daemon[1393]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 3/5)5566worker2 # [ 127.751339] nix-grpc-daemon[1393]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 172 ms (attempt 4/5)5567worker2 # [ 127.924233] nix-grpc-daemon[1393]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5568worker2 # [ 127.929932] nix-grpc-daemon[1393]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5569worker2 # [ 127.948932] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340345570client # [ 127.952139] nix[1259]: copying 1 paths to 'lb:50051'...5571worker2 # [ 127.963590] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340345572client # [ 127.957241] nix[1259]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5573worker2 # [ 127.978334] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5574worker2 # [ 128.005802] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:00Z level=info event=niks3_push_started pid=14665575worker2 # [ 128.221573] nix-grpc-daemon[1466]: time=2026-09-21T04:26:00.861Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5576worker2 # [ 128.412425] nix-daemon[857]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)5577worker2 # [ 128.435841] nix-daemon[857]: reaped child process 1486, status = succeeded5578worker2 # [ 128.608775] nix-daemon[857]: accepted connection from pid 1488, user nix-grpc-daemon (trusted)5579worker2 # [ 128.626877] nix-daemon[857]: reaped child process 1491, status = succeeded5580lb # [ 128.638479] niks3-server[1038]: 2026/09/21 04:26:01 INFO Received uploads request method=POST path=/api/pending_closures5581worker2 # [ 128.651476] nix-grpc-daemon[1466]: time=2026-09-21T04:26:01.290Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5582worker2 # [ 128.653488] nix-grpc-daemon[1466]: time=2026-09-21T04:26:01.290Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5583lb # [ 128.671715] niks3-server[1038]: 2026/09/21 04:26:01 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5584lb # [ 128.685486] niks3-server[1038]: 2026/09/21 04:26:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5585worker2 # [ 128.695747] nix-grpc-daemon[1466]: time=2026-09-21T04:26:01.335Z level=INFO msg="Uploading 1 narinfos"5586lb # [ 128.698419] niks3-server[1038]: 2026/09/21 04:26:01 INFO Signed narinfos id=37 count=15587lb # [ 128.705074] niks3-server[1038]: 2026/09/21 04:26:01 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5588lb # [ 128.717085] niks3-server[1038]: 2026/09/21 04:26:01 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5589worker2 # [ 128.725237] nix-grpc-daemon[1466]: time=2026-09-21T04:26:01.365Z level=INFO msg="Upload complete. (499ms)"5590lb # [ 128.723468] niks3-server[1038]: 2026/09/21 04:26:01 INFO Completed upload id=375591lb # [ 128.726405] niks3-server[1038]: 2026/09/21 04:26:01 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5592worker2 # [ 128.732754] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=0 paths=1 nar_bytes_in=4245593worker2 # [ 128.747729] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340345594worker2 # [ 128.754679] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5595worker2 # [ 128.759496] nix-daemon[857]: reaped child process 1464, status = succeeded5596client # [ 128.760291] nix[1259]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5597worker2 # [ 128.772585] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5598lb # [ 129.293354] envoy[902]: conn peer=[2001:db8:1::4]:33704 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5723 tx=511 ms=166775599lb # [ 129.298265] envoy[902]: conn peer=[2001:db8:1::1]:44412 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1853 ms=102305600client: must succeed: systemctl kill -s INT intr5601worker2 # [ 129.644150] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:02Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5602client # [ 129.631825] systemd[1]: intr.service: Sent signal SIGINT to main process 1259 (nix) on client request.5603client # [ 129.638848] nix[1259]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5604client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5605client # [ 129.648479] nix[1259]: error: interrupted by the user5606client # [ 129.689549] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5607client # [ 129.695170] systemd[1]: intr.service: Failed with result 'exit-code'.5608client # [ 129.697776] systemd[1]: intr.service: Consumed 162ms CPU time over 2.632s wall clock time, 41.4M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.5609worker2 # [ 129.805871] nix-daemon[1495]: unexpected Nix daemon error: error: interrupted by the user5610worker2 # [ 129.817357] nix-daemon[857]: reaped child process 1493, status = succeeded5611worker2 # [ 129.818527] nix-daemon[857]: reaped child process 1495, status = failed with exit code 15612(finished: subtest: interrupting the client stops the build on the worker, in 3.95 seconds)5613subtest: a deploy mid-build drains: the build finishes, the new generation takes over5614client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5615client # [ 131.038145] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5616client # Running as unit: sw.service; invocation ID: 84bc1d983f0c4ae9a67cef8e9cfce21b5617client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5618client # [ 131.306352] nix[1279]: don't know how to build these paths:5619client # [ 131.307367] nix[1279]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5620worker2 # [ 131.315631] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:340345621worker2 # [ 131.329454] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:03Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05622worker2 # [ 131.335541] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:03Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15623worker1 # [ 131.421155] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392405624worker1 # [ 131.438000] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392405625client # [ 131.394791] nix[1279]: copying 1 paths to 'lb:50051'...5626client # [ 131.400360] nix[1279]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5627worker1 # [ 131.454869] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392405628worker1 # [ 131.469765] nix-daemon[838]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5629worker1 # [ 131.672595] nix-daemon[838]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)5630worker1 # [ 131.691200] nix-daemon[838]: reaped child process 1639, status = succeeded5631worker1 # [ 131.863739] nix-daemon[838]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)5632worker1 # [ 131.881301] nix-daemon[838]: reaped child process 1644, status = succeeded5633lb # [ 131.846218] niks3-server[1038]: 2026/09/21 04:26:04 INFO Received uploads request method=POST path=/api/pending_closures5634worker1 # [ 131.905732] nix-grpc-daemon[1592]: time=2026-09-21T04:26:04.531Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5635worker1 # [ 131.907242] nix-grpc-daemon[1592]: time=2026-09-21T04:26:04.531Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5636lb # [ 131.885189] niks3-server[1038]: 2026/09/21 04:26:04 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5637worker1 # [ 131.945854] nix-grpc-daemon[1592]: time=2026-09-21T04:26:04.571Z level=INFO msg="Uploading 1 narinfos"5638lb # [ 131.901133] niks3-server[1038]: 2026/09/21 04:26:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5639lb # [ 131.912191] niks3-server[1038]: 2026/09/21 04:26:04 INFO Signed narinfos id=38 count=15640lb # [ 131.917921] niks3-server[1038]: 2026/09/21 04:26:04 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5641lb # [ 131.924963] niks3-server[1038]: 2026/09/21 04:26:04 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5642lb # [ 131.931386] niks3-server[1038]: 2026/09/21 04:26:04 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5643lb # [ 131.935308] niks3-server[1038]: 2026/09/21 04:26:04 INFO Completed upload id=385644worker1 # [ 131.983001] nix-grpc-daemon[1592]: time=2026-09-21T04:26:04.609Z level=INFO msg="Upload complete. (497ms)"5645worker1 # [ 131.992264] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39240 duration_s=0 paths=1 nar_bytes_in=4165646worker1 # [ 132.004662] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392405647worker1 # [ 132.013847] nix-daemon[838]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5648worker1 # [ 132.023274] nix-daemon[838]: reaped child process 1634, status = succeeded5649client # [ 131.972578] nix[1279]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5650worker1 # [ 132.030812] nix-daemon[838]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5651worker1: must succeed: systemctl reload nix-grpc-daemon.service5652worker1 # [ 133.775813] systemd[1]: Reloading Nix worker-protocol over gRPC...5653worker1 # [ 133.824893] systemd[1]: Reloaded Nix worker-protocol over gRPC.5654worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.10 seconds)5655worker1: must succeed: pgrep -f 'read -t [9]0 x'5656worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)5657client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25658worker1 # [ 133.886340] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:06Z level=info event=draining5659worker2 # [ 134.096610] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:340345660client # don't know how to build these paths:5661client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5662worker1 # [ 134.150905] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392405663worker2 # [ 134.128241] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:06Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05664worker2 # [ 134.135087] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:06Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15665worker2 # [ 134.174354] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340345666worker2 # [ 134.189335] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340345667client # copying 1 paths to 'lb:50051'...5668client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5669worker2 # [ 134.205355] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340345670worker2 # [ 134.220279] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5671worker2 # [ 134.431669] nix-daemon[857]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)5672worker2 # [ 134.449047] nix-daemon[857]: reaped child process 1549, status = succeeded5673worker2 # [ 134.622501] nix-daemon[857]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)5674worker2 # [ 134.638607] nix-daemon[857]: reaped child process 1554, status = succeeded5675lb # [ 134.649609] niks3-server[1038]: 2026/09/21 04:26:07 INFO Received uploads request method=POST path=/api/pending_closures5676worker2 # [ 134.661684] nix-grpc-daemon[1466]: time=2026-09-21T04:26:07.301Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5677worker2 # [ 134.663107] nix-grpc-daemon[1466]: time=2026-09-21T04:26:07.301Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5678worker2 # [ 134.686085] nix-grpc-daemon[1466]: time=2026-09-21T04:26:07.325Z level=INFO msg="Uploading 1 narinfos"5679lb # [ 134.683344] niks3-server[1038]: 2026/09/21 04:26:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5680lb # [ 134.697894] niks3-server[1038]: 2026/09/21 04:26:07 INFO Signed narinfos id=39 count=15681lb # [ 134.705955] niks3-server[1038]: 2026/09/21 04:26:07 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5682lb # [ 134.727221] niks3-server[1038]: 2026/09/21 04:26:07 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5683lb # [ 134.736104] niks3-server[1038]: 2026/09/21 04:26:07 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5684lb # [ 134.743040] niks3-server[1038]: 2026/09/21 04:26:07 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5685lb # [ 134.759820] niks3-server[1038]: 2026/09/21 04:26:07 INFO Completed upload id=395686worker2 # [ 134.762137] nix-grpc-daemon[1466]: time=2026-09-21T04:26:07.402Z level=INFO msg="Upload complete. (525ms)"5687worker2 # [ 134.772316] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=0 paths=1 nar_bytes_in=5285688worker2 # [ 134.788391] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340345689worker2 # [ 134.795890] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5690worker2 # [ 134.802009] nix-daemon[857]: reaped child process 1544, status = succeeded5691client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5692worker2 # [ 134.814740] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5693client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5694client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5695client # farm-a> LOG-farm-a-during-drain5696worker2 # [ 135.842889] nix-daemon[857]: reaped child process 1558, status = succeeded5697worker2 # [ 136.000318] nix-daemon[857]: accepted connection from pid 1574, user nix-grpc-daemon (trusted)5698worker2 # [ 136.017904] nix-daemon[857]: reaped child process 1577, status = succeeded5699lb # [ 136.027781] niks3-server[1038]: 2026/09/21 04:26:08 INFO Received uploads request method=POST path=/api/pending_closures5700worker2 # [ 136.044384] nix-grpc-daemon[1466]: time=2026-09-21T04:26:08.682Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5701worker2 # [ 136.045809] nix-grpc-daemon[1466]: time=2026-09-21T04:26:08.683Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5702lb # [ 136.078004] niks3-server[1038]: 2026/09/21 04:26:08 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5703lb # [ 136.087172] niks3-server[1038]: 2026/09/21 04:26:08 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5704lb # [ 136.097433] niks3-server[1038]: 2026/09/21 04:26:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5705worker2 # [ 136.106583] nix-grpc-daemon[1466]: time=2026-09-21T04:26:08.745Z level=INFO msg="Uploading 1 narinfos"5706lb # [ 136.103399] niks3-server[1038]: 2026/09/21 04:26:08 INFO Signed narinfos id=40 count=15707lb # [ 136.108337] niks3-server[1038]: 2026/09/21 04:26:08 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5708lb # [ 136.120293] niks3-server[1038]: 2026/09/21 04:26:08 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5709lb # [ 136.127573] niks3-server[1038]: 2026/09/21 04:26:08 INFO Completed upload id=405710worker2 # [ 136.129937] nix-grpc-daemon[1466]: time=2026-09-21T04:26:08.769Z level=INFO msg="Upload complete. (301ms)"5711lb # [ 136.130851] niks3-server[1038]: 2026/09/21 04:26:08 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5712worker2 # [ 136.147207] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15713worker2 # [ 136.163286] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:08Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05714worker2 # [ 136.166812] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:08Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15715worker2 # [ 136.170408] nix-daemon[857]: reaped child process 1556, status = succeeded5716worker2 # [ 136.196993] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340345717worker2 # [ 136.212395] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340345718worker2 # [ 136.226989] nix-grpc-daemon[1466]: time=2026-09-21T04:26:08.866Z level=INFO msg="All 1 paths already cached"5719client # copying 1 paths to 'lb:50051'...5720worker2 # [ 136.237059] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340345721client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5722worker2 # [ 136.266159] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5723worker2 # [ 136.477928] nix-daemon[857]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)5724worker2 # [ 136.494839] nix-daemon[857]: reaped child process 1586, status = succeeded5725worker2 # [ 136.667687] nix-daemon[857]: accepted connection from pid 1588, user nix-grpc-daemon (trusted)5726worker2 # [ 136.685017] nix-daemon[857]: reaped child process 1591, status = succeeded5727lb # [ 136.694237] niks3-server[1038]: 2026/09/21 04:26:09 INFO Received uploads request method=POST path=/api/pending_closures5728worker2 # [ 136.706215] nix-grpc-daemon[1466]: time=2026-09-21T04:26:09.345Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5729worker2 # [ 136.707681] nix-grpc-daemon[1466]: time=2026-09-21T04:26:09.345Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5730lb # [ 136.720206] niks3-server[1038]: 2026/09/21 04:26:09 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5731lb # [ 136.734416] niks3-server[1038]: 2026/09/21 04:26:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5732worker2 # [ 136.742795] nix-grpc-daemon[1466]: time=2026-09-21T04:26:09.382Z level=INFO msg="Uploading 1 narinfos"5733lb # [ 136.747332] niks3-server[1038]: 2026/09/21 04:26:09 INFO Signed narinfos id=41 count=15734lb # [ 136.752728] niks3-server[1038]: 2026/09/21 04:26:09 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5735lb # [ 136.761322] niks3-server[1038]: 2026/09/21 04:26:09 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5736worker2 # [ 136.767293] nix-grpc-daemon[1466]: time=2026-09-21T04:26:09.407Z level=INFO msg="Upload complete. (483ms)"5737lb # [ 136.765841] niks3-server[1038]: 2026/09/21 04:26:09 INFO Completed upload id=415738lb # [ 136.769119] niks3-server[1038]: 2026/09/21 04:26:09 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5739worker2 # [ 136.775692] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=0 paths=1 nar_bytes_in=6405740worker2 # [ 136.784959] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340345741worker2 # [ 136.790065] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5742worker2 # [ 136.794455] nix-daemon[857]: reaped child process 1581, status = succeeded5743client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5744client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5745worker2 # [ 136.806977] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5746client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5747client # farm-b> LOG-farm-b-during-drain5748worker2 # [ 138.029110] nix-daemon[857]: reaped child process 1595, status = succeeded5749worker2 # [ 138.182966] nix-daemon[857]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)5750worker2 # [ 138.199893] nix-daemon[857]: reaped child process 1614, status = succeeded5751lb # [ 138.212826] niks3-server[1038]: 2026/09/21 04:26:10 INFO Received uploads request method=POST path=/api/pending_closures5752worker2 # [ 138.224478] nix-grpc-daemon[1466]: time=2026-09-21T04:26:10.864Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5753worker2 # [ 138.227065] nix-grpc-daemon[1466]: time=2026-09-21T04:26:10.866Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5754lb # [ 138.244833] niks3-server[1038]: 2026/09/21 04:26:10 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5755lb # [ 138.261837] niks3-server[1038]: 2026/09/21 04:26:10 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5756lb # [ 138.276818] niks3-server[1038]: 2026/09/21 04:26:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5757worker2 # [ 138.285663] nix-grpc-daemon[1466]: time=2026-09-21T04:26:10.925Z level=INFO msg="Uploading 1 narinfos"5758lb # [ 138.289645] niks3-server[1038]: 2026/09/21 04:26:10 INFO Signed narinfos id=42 count=15759lb # [ 138.295161] niks3-server[1038]: 2026/09/21 04:26:10 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5760lb # [ 138.302105] niks3-server[1038]: 2026/09/21 04:26:10 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5761lb # [ 138.309387] niks3-server[1038]: 2026/09/21 04:26:10 INFO Completed upload id=425762worker2 # [ 138.311139] nix-grpc-daemon[1466]: time=2026-09-21T04:26:10.951Z level=INFO msg="Upload complete. (295ms)"5763lb # [ 138.311853] niks3-server[1038]: 2026/09/21 04:26:10 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5764worker2 # [ 138.328639] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15765worker2 # [ 138.343360] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:10Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05766worker2 # [ 138.347613] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:10Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15767worker2 # [ 138.351900] nix-daemon[857]: reaped child process 1593, status = succeeded5768worker2 # [ 138.370284] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340345769worker2 # [ 138.385001] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340345770worker2 # [ 138.399309] nix-grpc-daemon[1466]: time=2026-09-21T04:26:11.039Z level=INFO msg="All 2 paths already cached"5771client # copying 1 paths to 'lb:50051'...5772worker2 # [ 138.409601] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340345773client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5774worker2 # [ 138.424312] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5775worker2 # [ 138.623912] nix-daemon[857]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)5776worker2 # [ 138.639492] nix-daemon[857]: reaped child process 1623, status = succeeded5777worker2 # [ 138.811119] nix-daemon[857]: accepted connection from pid 1625, user nix-grpc-daemon (trusted)5778worker2 # [ 138.826732] nix-daemon[857]: reaped child process 1628, status = succeeded5779lb # [ 138.835860] niks3-server[1038]: 2026/09/21 04:26:11 INFO Received uploads request method=POST path=/api/pending_closures5780worker2 # [ 138.847074] nix-grpc-daemon[1466]: time=2026-09-21T04:26:11.486Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5781worker2 # [ 138.848657] nix-grpc-daemon[1466]: time=2026-09-21T04:26:11.486Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5782lb # [ 138.862510] niks3-server[1038]: 2026/09/21 04:26:11 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5783worker2 # [ 138.877317] nix-grpc-daemon[1466]: time=2026-09-21T04:26:11.517Z level=INFO msg="Uploading 1 narinfos"5784lb # [ 138.879846] niks3-server[1038]: 2026/09/21 04:26:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5785lb # [ 138.890741] niks3-server[1038]: 2026/09/21 04:26:11 INFO Signed narinfos id=43 count=15786lb # [ 138.894862] niks3-server[1038]: 2026/09/21 04:26:11 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5787lb # [ 138.904357] niks3-server[1038]: 2026/09/21 04:26:11 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5788worker2 # [ 138.910782] nix-grpc-daemon[1466]: time=2026-09-21T04:26:11.550Z level=INFO msg="Upload complete. (466ms)"5789lb # [ 138.909159] niks3-server[1038]: 2026/09/21 04:26:11 INFO Completed upload id=435790lb # [ 138.912905] niks3-server[1038]: 2026/09/21 04:26:11 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5791worker2 # [ 138.920494] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=0 paths=1 nar_bytes_in=7685792worker2 # [ 138.929107] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340345793worker2 # [ 138.935692] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5794worker2 # [ 138.940176] nix-daemon[857]: reaped child process 1618, status = succeeded5795client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5796worker2 # [ 138.952485] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5797client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5798lb # [ 139.292106] envoy[902]: conn peer=[2001:db8:1::1]:41820 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=853 ms=23645799client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5800client # farm-top> LOG-farm-top-during-drain5801worker2 # [ 139.995195] nix-daemon[857]: reaped child process 1632, status = succeeded5802worker2 # [ 140.175078] nix-daemon[857]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)5803worker2 # [ 140.193104] nix-daemon[857]: reaped child process 1651, status = succeeded5804lb # [ 140.205510] niks3-server[1038]: 2026/09/21 04:26:12 INFO Received uploads request method=POST path=/api/pending_closures5805worker2 # [ 140.218529] nix-grpc-daemon[1466]: time=2026-09-21T04:26:12.856Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5806worker2 # [ 140.220301] nix-grpc-daemon[1466]: time=2026-09-21T04:26:12.856Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5807lb # [ 140.244231] niks3-server[1038]: 2026/09/21 04:26:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5808worker2 # [ 140.257246] nix-grpc-daemon[1466]: time=2026-09-21T04:26:12.896Z level=INFO msg="Uploading 1 narinfos"5809lb # [ 140.259217] niks3-server[1038]: 2026/09/21 04:26:12 INFO Signed narinfos id=44 count=15810lb # [ 140.268659] niks3-server[1038]: 2026/09/21 04:26:12 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5811lb # [ 140.277457] niks3-server[1038]: 2026/09/21 04:26:12 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5812lb # [ 140.285366] niks3-server[1038]: 2026/09/21 04:26:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5813worker2 # [ 140.292849] nix-grpc-daemon[1466]: time=2026-09-21T04:26:12.932Z level=INFO msg="Upload complete. (330ms)"5814lb # [ 140.291079] niks3-server[1038]: 2026/09/21 04:26:12 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5815lb # [ 140.295864] niks3-server[1038]: 2026/09/21 04:26:12 INFO Completed upload id=445816lb # [ 140.299573] niks3-server[1038]: 2026/09/21 04:26:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5817worker2 # [ 140.305044] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15818worker2 # [ 140.322919] nix-daemon[857]: reaped child process 1630, status = succeeded5819client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.53 seconds)5820client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5821client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5822worker1: must succeed: pkill -f 'read -t [9]0 x'5823worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5824client: waiting for success: ! systemctl is-active sw5825worker1 # [ 140.880486] nix-daemon[838]: reaped child process 1648, status = succeeded5826worker1 # [ 141.061488] nix-daemon[838]: accepted connection from pid 1690, user nix-grpc-daemon (trusted)5827worker1 # [ 141.079513] nix-daemon[838]: reaped child process 1693, status = succeeded5828lb # [ 141.044050] niks3-server[1038]: 2026/09/21 04:26:13 INFO Received uploads request method=POST path=/api/pending_closures5829worker1 # [ 141.100981] nix-grpc-daemon[1592]: time=2026-09-21T04:26:13.726Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5830worker1 # [ 141.103728] nix-grpc-daemon[1592]: time=2026-09-21T04:26:13.729Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5831lb # [ 141.076668] niks3-server[1038]: 2026/09/21 04:26:13 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5832lb # [ 141.095139] niks3-server[1038]: 2026/09/21 04:26:13 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5833lb # [ 141.104201] niks3-server[1038]: 2026/09/21 04:26:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5834lb # [ 141.110105] niks3-server[1038]: 2026/09/21 04:26:13 INFO Signed narinfos id=45 count=15835worker1 # [ 141.158162] nix-grpc-daemon[1592]: time=2026-09-21T04:26:13.782Z level=INFO msg="Uploading 1 narinfos"5836lb # [ 141.114190] niks3-server[1038]: 2026/09/21 04:26:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5837lb # [ 141.124448] niks3-server[1038]: 2026/09/21 04:26:13 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5838lb # [ 141.132952] niks3-server[1038]: 2026/09/21 04:26:13 INFO Completed upload id=455839lb # [ 141.136687] niks3-server[1038]: 2026/09/21 04:26:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5840worker1 # [ 141.180587] nix-grpc-daemon[1592]: time=2026-09-21T04:26:13.806Z level=INFO msg="Upload complete. (330ms)"5841worker1 # [ 141.193395] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39240 duration_s=9 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15842worker1 # [ 141.212197] nix-daemon[838]: reaped child process 1646, status = succeeded5843client # [ 141.179538] systemd[1]: sw.service: Deactivated successfully.5844client # [ 141.185015] systemd[1]: sw.service: Consumed 182ms CPU time over 10.144s wall clock time, 43.2M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.5845client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds)5846client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5847client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)5848worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5849worker1 # [ 141.889392] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:14Z level=info event=signal_exit5850worker1 # [ 143.125039] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:15Z level=info event=scheduler_restarting5851worker2 # [ 143.078593] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:15Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:342365852worker2 # [ 143.078936] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:15Z level=info event=worker_gone addr=192.168.1.3:500515853worker1 # [ 143.151196] nix-grpc-daemon[1489]: ts=2026-09-21T04:26:15Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5854worker1 # [ 143.174412] nix-daemon[838]: reaped child process 1558, status = succeeded5855worker1 # [ 143.672997] nix-grpc-daemon[1489]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5856worker1 # [ 143.677978] nix-grpc-daemon[1489]: E0000 00:00:1789964776.299047 1532 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5857worker1 # [ 143.684325] nix-grpc-daemon[1489]: E0000 00:00:1789964776.304122 1532 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5858worker1 # [ 143.935738] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5859worker1 # [ 143.942889] systemd[1]: nix-grpc-daemon.service: Consumed 873ms CPU time over 28.261s wall clock time, 33.3M memory peak, 21K incoming IP traffic, 27.5K outgoing IP traffic.5860worker1 # [ 143.960116] systemd[1]: Starting Nix worker-protocol over gRPC...5861worker1 # [ 144.158589] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5862worker1 # [ 144.160581] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5863worker1 # [ 144.164352] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15864worker1 # [ 144.172060] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5865worker1 # [ 144.178981] systemd[1]: Started Nix worker-protocol over gRPC.5866worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.32 seconds)5867lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5868lb: must succeed: curl -sf localhost:9901/clusters5869worker1 # [ 144.215791] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:16Z level=info event=scheduler_connected addr=lb:500515870worker2 # [ 144.173133] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:342365871worker2 # [ 144.173345] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05872lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5873lb: must succeed: curl -sf localhost:9901/clusters5874lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5875lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5876lb: must succeed: curl -sf localhost:9901/clusters5877lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5878worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5879worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5880(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.48 seconds)5881subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5882client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15883worker1 # [ 145.777018] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446905884worker1 # [ 145.792487] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5885worker1 # [ 145.812094] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446905886worker1 # [ 145.827670] nix-grpc-daemon[1707]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)5887worker1 # [ 145.908300] nix-grpc-daemon[1707]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 2/5)5888worker1 # [ 146.049924] nix-grpc-daemon[1707]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 299 ms (attempt 3/5)5889worker1 # [ 146.349590] nix-grpc-daemon[1707]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 4/5)5890worker1 # [ 146.366145] nix-grpc-daemon[1707]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5891worker1 # [ 146.370293] nix-grpc-daemon[1707]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5892worker2 # [ 146.375521] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:19Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05893worker2 # [ 146.381480] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:19Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15894worker2 # [ 146.387394] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:19Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05895worker2 # [ 146.392830] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:19Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05896worker1 # [ 146.468560] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446905897worker1 # [ 146.483202] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446905898worker1 # [ 146.497428] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446905899worker1 # [ 146.511614] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5900worker1 # [ 146.539079] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:19Z level=info event=niks3_push_started pid=17645901worker1 # [ 146.753456] nix-grpc-daemon[1764]: time=2026-09-21T04:26:19.379Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5902worker1 # [ 146.924328] nix-daemon[838]: accepted connection from pid 1777, user nix-grpc-daemon (trusted)5903worker1 # [ 146.941325] nix-daemon[838]: reaped child process 1780, status = succeeded5904worker1 # [ 147.121341] nix-daemon[838]: accepted connection from pid 1782, user nix-grpc-daemon (trusted)5905worker1 # [ 147.137888] nix-daemon[838]: reaped child process 1785, status = succeeded5906lb # [ 147.102863] niks3-server[1038]: 2026/09/21 04:26:19 INFO Received uploads request method=POST path=/api/pending_closures5907worker1 # [ 147.160062] nix-grpc-daemon[1764]: time=2026-09-21T04:26:19.785Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5908worker1 # [ 147.161546] nix-grpc-daemon[1764]: time=2026-09-21T04:26:19.785Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5909lb # [ 147.134814] niks3-server[1038]: 2026/09/21 04:26:19 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5910lb # [ 147.151275] niks3-server[1038]: 2026/09/21 04:26:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5911worker1 # [ 147.205555] nix-grpc-daemon[1764]: time=2026-09-21T04:26:19.831Z level=INFO msg="Uploading 1 narinfos"5912lb # [ 147.163486] niks3-server[1038]: 2026/09/21 04:26:19 INFO Signed narinfos id=46 count=15913lb # [ 147.171183] niks3-server[1038]: 2026/09/21 04:26:19 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5914lb # [ 147.181699] niks3-server[1038]: 2026/09/21 04:26:19 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5915lb # [ 147.189402] niks3-server[1038]: 2026/09/21 04:26:19 INFO Completed upload id=465916worker1 # [ 147.237060] nix-grpc-daemon[1764]: time=2026-09-21T04:26:19.863Z level=INFO msg="Upload complete. (478ms)"5917lb # [ 147.192234] niks3-server[1038]: 2026/09/21 04:26:19 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5918worker1 # [ 147.244909] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 paths=1 nar_bytes_in=5125919worker1 # [ 147.260353] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446905920worker1 # [ 147.269600] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5921worker1 # [ 147.272331] nix-daemon[838]: reaped child process 1762, status = succeeded5922worker1 # [ 147.285444] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5923worker1 # [ 147.905806] nix-daemon[838]: reaped child process 1789, status = succeeded5924worker1 # [ 148.072420] nix-daemon[838]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)5925worker1 # [ 148.090426] nix-daemon[838]: reaped child process 1808, status = succeeded5926lb # [ 148.053860] niks3-server[1038]: 2026/09/21 04:26:20 INFO Received uploads request method=POST path=/api/pending_closures5927worker1 # [ 148.112391] nix-grpc-daemon[1764]: time=2026-09-21T04:26:20.736Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5928worker1 # [ 148.113891] nix-grpc-daemon[1764]: time=2026-09-21T04:26:20.736Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5929lb # [ 148.096521] niks3-server[1038]: 2026/09/21 04:26:20 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5930lb # [ 148.110413] niks3-server[1038]: 2026/09/21 04:26:20 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5931lb # [ 148.119159] niks3-server[1038]: 2026/09/21 04:26:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5932worker1 # [ 148.173195] nix-grpc-daemon[1764]: time=2026-09-21T04:26:20.798Z level=INFO msg="Uploading 1 narinfos"5933lb # [ 148.125268] niks3-server[1038]: 2026/09/21 04:26:20 INFO Signed narinfos id=47 count=15934lb # [ 148.131652] niks3-server[1038]: 2026/09/21 04:26:20 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5935lb # [ 148.142838] niks3-server[1038]: 2026/09/21 04:26:20 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5936lb # [ 148.149693] niks3-server[1038]: 2026/09/21 04:26:20 INFO Completed upload id=475937worker1 # [ 148.197284] nix-grpc-daemon[1764]: time=2026-09-21T04:26:20.823Z level=INFO msg="Upload complete. (302ms)"5938lb # [ 148.152817] niks3-server[1038]: 2026/09/21 04:26:20 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5939worker2 # [ 148.161799] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:20Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15940worker1 # [ 148.208310] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=15941worker2 # [ 148.173760] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:20Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05942worker1 # [ 148.224145] nix-daemon[838]: reaped child process 1787, status = succeeded5943worker1 # [ 148.260699] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446905944worker1 # [ 148.276309] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446905945worker1 # [ 148.289928] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446905946worker1 # [ 148.303493] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5947worker1 # [ 148.499793] nix-daemon[838]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)5948worker1 # [ 148.518234] nix-daemon[838]: reaped child process 1815, status = succeeded5949worker1 # [ 148.692841] nix-daemon[838]: accepted connection from pid 1817, user nix-grpc-daemon (trusted)5950worker1 # [ 148.708626] nix-daemon[838]: reaped child process 1820, status = succeeded5951lb # [ 148.674523] niks3-server[1038]: 2026/09/21 04:26:21 INFO Received uploads request method=POST path=/api/pending_closures5952worker1 # [ 148.730948] nix-grpc-daemon[1764]: time=2026-09-21T04:26:21.356Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5953worker1 # [ 148.732580] nix-grpc-daemon[1764]: time=2026-09-21T04:26:21.356Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5954lb # [ 148.700051] niks3-server[1038]: 2026/09/21 04:26:21 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5955lb # [ 148.713295] niks3-server[1038]: 2026/09/21 04:26:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5956worker1 # [ 148.768120] nix-grpc-daemon[1764]: time=2026-09-21T04:26:21.394Z level=INFO msg="Uploading 1 narinfos"5957lb # [ 148.726252] niks3-server[1038]: 2026/09/21 04:26:21 INFO Signed narinfos id=48 count=15958lb # [ 148.732269] niks3-server[1038]: 2026/09/21 04:26:21 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5959lb # [ 148.739999] niks3-server[1038]: 2026/09/21 04:26:21 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5960lb # [ 148.744946] niks3-server[1038]: 2026/09/21 04:26:21 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5961lb # [ 148.749122] niks3-server[1038]: 2026/09/21 04:26:21 INFO Completed upload id=485962worker1 # [ 148.796736] nix-grpc-daemon[1764]: time=2026-09-21T04:26:21.422Z level=INFO msg="Upload complete. (479ms)"5963worker1 # [ 148.803584] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 paths=1 nar_bytes_in=5125964worker1 # [ 148.815247] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446905965worker1 # [ 148.821614] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5966worker1 # [ 148.826243] nix-daemon[838]: reaped child process 1810, status = succeeded5967worker1 # [ 148.839663] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5968lb # [ 149.291773] envoy[902]: conn peer=[2001:db8:1::1]:46326 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9595 tx=4104 ms=62355969lb # [ 149.296973] envoy[902]: conn peer=[2001:db8:1::1]:46316 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1429 ms=98465970lb # [ 149.302711] envoy[902]: conn peer=[2001:db8:1::3]:40380 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=986 tx=448 ms=272355971worker1 # [ 149.681401] nix-daemon[838]: reaped child process 1824, status = succeeded5972worker1 # [ 149.855185] nix-daemon[838]: accepted connection from pid 1840, user nix-grpc-daemon (trusted)5973worker1 # [ 149.872116] nix-daemon[838]: reaped child process 1843, status = succeeded5974lb # [ 149.840574] niks3-server[1038]: 2026/09/21 04:26:22 INFO Received uploads request method=POST path=/api/pending_closures5975worker1 # [ 149.897347] nix-grpc-daemon[1764]: time=2026-09-21T04:26:22.522Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5976worker1 # [ 149.898815] nix-grpc-daemon[1764]: time=2026-09-21T04:26:22.522Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5977lb # [ 149.872092] niks3-server[1038]: 2026/09/21 04:26:22 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5978worker1 # [ 149.942253] nix-grpc-daemon[1764]: time=2026-09-21T04:26:22.567Z level=INFO msg="Uploading 1 narinfos"5979lb # [ 149.887547] niks3-server[1038]: 2026/09/21 04:26:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5980lb # [ 149.902701] niks3-server[1038]: 2026/09/21 04:26:22 INFO Signed narinfos id=49 count=15981lb # [ 149.909935] niks3-server[1038]: 2026/09/21 04:26:22 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5982lb # [ 149.923303] niks3-server[1038]: 2026/09/21 04:26:22 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5983worker1 # [ 149.976364] nix-grpc-daemon[1764]: time=2026-09-21T04:26:22.602Z level=INFO msg="Upload complete. (326ms)"5984lb # [ 149.928818] niks3-server[1038]: 2026/09/21 04:26:22 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5985lb # [ 149.934969] niks3-server[1038]: 2026/09/21 04:26:22 INFO Completed upload id=495986lb # [ 149.938326] niks3-server[1038]: 2026/09/21 04:26:22 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5987worker2 # [ 149.938521] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:22Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15988worker1 # [ 149.987141] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=15989worker2 # [ 149.950685] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:22Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05990worker1 # [ 150.007584] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446905991worker1 # [ 150.013617] nix-daemon[838]: reaped child process 1822, status = succeeded5992worker1 # [ 150.021348] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446905993worker1 # [ 150.033860] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446905994worker1 # [ 150.046288] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5995worker1 # [ 150.243065] nix-daemon[838]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)5996worker1 # [ 150.259907] nix-daemon[838]: reaped child process 1850, status = succeeded5997worker1 # [ 150.433818] nix-daemon[838]: accepted connection from pid 1852, user nix-grpc-daemon (trusted)5998worker1 # [ 150.450182] nix-daemon[838]: reaped child process 1855, status = succeeded5999lb # [ 150.413806] niks3-server[1038]: 2026/09/21 04:26:23 INFO Received uploads request method=POST path=/api/pending_closures6000worker1 # [ 150.469930] nix-grpc-daemon[1764]: time=2026-09-21T04:26:23.095Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6001worker1 # [ 150.471450] nix-grpc-daemon[1764]: time=2026-09-21T04:26:23.095Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6002lb # [ 150.439512] niks3-server[1038]: 2026/09/21 04:26:23 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6003worker1 # [ 150.500717] nix-grpc-daemon[1764]: time=2026-09-21T04:26:23.126Z level=INFO msg="Uploading 1 narinfos"6004lb # [ 150.457046] niks3-server[1038]: 2026/09/21 04:26:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6005lb # [ 150.466908] niks3-server[1038]: 2026/09/21 04:26:23 INFO Signed narinfos id=50 count=16006lb # [ 150.475496] niks3-server[1038]: 2026/09/21 04:26:23 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6007worker1 # [ 150.528325] nix-grpc-daemon[1764]: time=2026-09-21T04:26:23.154Z level=INFO msg="Upload complete. (469ms)"6008lb # [ 150.481262] niks3-server[1038]: 2026/09/21 04:26:23 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6009lb # [ 150.486137] niks3-server[1038]: 2026/09/21 04:26:23 INFO Completed upload id=506010worker1 # [ 150.533478] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 paths=1 nar_bytes_in=5126011lb # [ 150.488960] niks3-server[1038]: 2026/09/21 04:26:23 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6012worker1 # [ 150.544631] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906013worker1 # [ 150.550718] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6014worker1 # [ 150.554765] nix-daemon[838]: reaped child process 1845, status = succeeded6015worker1 # [ 150.564771] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6016worker1 # [ 151.611458] nix-daemon[838]: reaped child process 1859, status = succeeded6017worker1 # [ 151.788935] nix-daemon[838]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)6018worker1 # [ 151.804603] nix-daemon[838]: reaped child process 1878, status = succeeded6019lb # [ 151.773797] niks3-server[1038]: 2026/09/21 04:26:24 INFO Received uploads request method=POST path=/api/pending_closures6020worker1 # [ 151.832415] nix-grpc-daemon[1764]: time=2026-09-21T04:26:24.457Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6021worker1 # [ 151.833900] nix-grpc-daemon[1764]: time=2026-09-21T04:26:24.457Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6022lb # [ 151.807983] niks3-server[1038]: 2026/09/21 04:26:24 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6023lb # [ 151.819889] niks3-server[1038]: 2026/09/21 04:26:24 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6024lb # [ 151.830610] niks3-server[1038]: 2026/09/21 04:26:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6025worker1 # [ 151.883603] nix-grpc-daemon[1764]: time=2026-09-21T04:26:24.509Z level=INFO msg="Uploading 1 narinfos"6026lb # [ 151.838648] niks3-server[1038]: 2026/09/21 04:26:24 INFO Signed narinfos id=51 count=16027lb # [ 151.843883] niks3-server[1038]: 2026/09/21 04:26:24 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6028lb # [ 151.852619] niks3-server[1038]: 2026/09/21 04:26:24 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6029lb # [ 151.858209] niks3-server[1038]: 2026/09/21 04:26:24 INFO Completed upload id=516030worker1 # [ 151.905868] nix-grpc-daemon[1764]: time=2026-09-21T04:26:24.532Z level=INFO msg="Upload complete. (329ms)"6031lb # [ 151.861611] niks3-server[1038]: 2026/09/21 04:26:24 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6032worker2 # [ 151.870005] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:24Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16033worker2 # [ 151.883024] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:24Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06034worker1 # [ 151.917149] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16035worker1 # [ 151.935105] nix-daemon[838]: reaped child process 1857, status = succeeded6036worker1 # [ 151.967987] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906037worker1 # [ 151.983544] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446906038worker1 # [ 151.998596] nix-grpc-daemon[1764]: time=2026-09-21T04:26:24.624Z level=INFO msg="All 1 paths already cached"6039worker1 # [ 152.007445] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446906040worker1 # [ 152.021334] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6041worker1 # [ 152.221566] nix-daemon[838]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)6042worker1 # [ 152.238599] nix-daemon[838]: reaped child process 1885, status = succeeded6043worker1 # [ 152.410678] nix-daemon[838]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6044worker1 # [ 152.426742] nix-daemon[838]: reaped child process 1890, status = succeeded6045lb # [ 152.391194] niks3-server[1038]: 2026/09/21 04:26:25 INFO Received uploads request method=POST path=/api/pending_closures6046worker1 # [ 152.447846] nix-grpc-daemon[1764]: time=2026-09-21T04:26:25.073Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6047worker1 # [ 152.449562] nix-grpc-daemon[1764]: time=2026-09-21T04:26:25.073Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6048lb # [ 152.424397] niks3-server[1038]: 2026/09/21 04:26:25 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6049worker1 # [ 152.484489] nix-grpc-daemon[1764]: time=2026-09-21T04:26:25.110Z level=INFO msg="Uploading 1 narinfos"6050lb # [ 152.439032] niks3-server[1038]: 2026/09/21 04:26:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6051lb # [ 152.448219] niks3-server[1038]: 2026/09/21 04:26:25 INFO Signed narinfos id=52 count=16052lb # [ 152.450569] niks3-server[1038]: 2026/09/21 04:26:25 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6053worker1 # [ 152.506752] nix-grpc-daemon[1764]: time=2026-09-21T04:26:25.133Z level=INFO msg="Upload complete. (469ms)"6054lb # [ 152.459490] niks3-server[1038]: 2026/09/21 04:26:25 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6055lb # [ 152.463999] niks3-server[1038]: 2026/09/21 04:26:25 INFO Completed upload id=526056lb # [ 152.466487] niks3-server[1038]: 2026/09/21 04:26:25 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6057worker1 # [ 152.514586] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 paths=1 nar_bytes_in=6246058worker1 # [ 152.526006] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906059worker1 # [ 152.531533] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6060worker1 # [ 152.535562] nix-daemon[838]: reaped child process 1880, status = succeeded6061worker1 # [ 152.545778] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6062worker1 # [ 153.787426] nix-daemon[838]: reaped child process 1894, status = succeeded6063worker1 # [ 153.955609] nix-daemon[838]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6064worker1 # [ 153.974270] nix-daemon[838]: reaped child process 1913, status = succeeded6065lb # [ 153.938834] niks3-server[1038]: 2026/09/21 04:26:26 INFO Received uploads request method=POST path=/api/pending_closures6066worker1 # [ 153.996421] nix-grpc-daemon[1764]: time=2026-09-21T04:26:26.621Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6067worker1 # [ 153.997905] nix-grpc-daemon[1764]: time=2026-09-21T04:26:26.621Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6068lb # [ 153.978210] niks3-server[1038]: 2026/09/21 04:26:26 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6069lb # [ 153.988296] niks3-server[1038]: 2026/09/21 04:26:26 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6070lb # [ 154.001432] niks3-server[1038]: 2026/09/21 04:26:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6071worker1 # [ 154.053914] nix-grpc-daemon[1764]: time=2026-09-21T04:26:26.679Z level=INFO msg="Uploading 1 narinfos"6072lb # [ 154.010609] niks3-server[1038]: 2026/09/21 04:26:26 INFO Signed narinfos id=53 count=16073lb # [ 154.016449] niks3-server[1038]: 2026/09/21 04:26:26 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6074lb # [ 154.024095] niks3-server[1038]: 2026/09/21 04:26:26 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6075lb # [ 154.030936] niks3-server[1038]: 2026/09/21 04:26:26 INFO Completed upload id=536076worker1 # [ 154.078625] nix-grpc-daemon[1764]: time=2026-09-21T04:26:26.704Z level=INFO msg="Upload complete. (321ms)"6077lb # [ 154.034055] niks3-server[1038]: 2026/09/21 04:26:26 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6078worker2 # [ 154.043271] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:26Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16079worker1 # [ 154.093145] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16080worker2 # [ 154.056446] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:26Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06081worker1 # [ 154.108645] nix-daemon[838]: reaped child process 1892, status = succeeded6082worker1 # [ 154.143877] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906083worker1 # [ 154.159138] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446906084worker1 # [ 154.173261] nix-grpc-daemon[1764]: time=2026-09-21T04:26:26.799Z level=INFO msg="All 1 paths already cached"6085worker1 # [ 154.182625] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446906086worker1 # [ 154.196539] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6087worker1 # [ 154.395519] nix-daemon[838]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6088worker1 # [ 154.415362] nix-daemon[838]: reaped child process 1920, status = succeeded6089worker1 # [ 154.586712] nix-daemon[838]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6090worker1 # [ 154.602756] nix-daemon[838]: reaped child process 1925, status = succeeded6091lb # [ 154.566071] niks3-server[1038]: 2026/09/21 04:26:27 INFO Received uploads request method=POST path=/api/pending_closures6092worker1 # [ 154.622481] nix-grpc-daemon[1764]: time=2026-09-21T04:26:27.248Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6093worker1 # [ 154.623980] nix-grpc-daemon[1764]: time=2026-09-21T04:26:27.248Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6094lb # [ 154.594279] niks3-server[1038]: 2026/09/21 04:26:27 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6095worker1 # [ 154.660237] nix-grpc-daemon[1764]: time=2026-09-21T04:26:27.286Z level=INFO msg="Uploading 1 narinfos"6096lb # [ 154.605481] niks3-server[1038]: 2026/09/21 04:26:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6097lb # [ 154.617692] niks3-server[1038]: 2026/09/21 04:26:27 INFO Signed narinfos id=54 count=16098lb # [ 154.624359] niks3-server[1038]: 2026/09/21 04:26:27 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6099worker1 # [ 154.682574] nix-grpc-daemon[1764]: time=2026-09-21T04:26:27.308Z level=INFO msg="Upload complete. (469ms)"6100lb # [ 154.636633] niks3-server[1038]: 2026/09/21 04:26:27 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6101lb # [ 154.642061] niks3-server[1038]: 2026/09/21 04:26:27 INFO Completed upload id=546102worker1 # [ 154.689487] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 paths=1 nar_bytes_in=6246103lb # [ 154.645381] niks3-server[1038]: 2026/09/21 04:26:27 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6104worker1 # [ 154.702406] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906105worker1 # [ 154.711864] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6106worker1 # [ 154.714628] nix-daemon[838]: reaped child process 1915, status = succeeded6107worker1 # [ 154.726783] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6108worker1 # [ 155.967517] nix-daemon[838]: reaped child process 1929, status = succeeded6109worker1 # [ 156.143117] nix-daemon[838]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)6110worker1 # [ 156.160491] nix-daemon[838]: reaped child process 1948, status = succeeded6111lb # [ 156.130066] niks3-server[1038]: 2026/09/21 04:26:28 INFO Received uploads request method=POST path=/api/pending_closures6112worker1 # [ 156.187517] nix-grpc-daemon[1764]: time=2026-09-21T04:26:28.811Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6113worker1 # [ 156.190221] nix-grpc-daemon[1764]: time=2026-09-21T04:26:28.811Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6114lb # [ 156.165054] niks3-server[1038]: 2026/09/21 04:26:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6115worker1 # [ 156.220526] nix-grpc-daemon[1764]: time=2026-09-21T04:26:28.846Z level=INFO msg="Uploading 1 narinfos"6116lb # [ 156.182257] niks3-server[1038]: 2026/09/21 04:26:28 INFO Signed narinfos id=55 count=16117lb # [ 156.190947] niks3-server[1038]: 2026/09/21 04:26:28 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6118lb # [ 156.198212] niks3-server[1038]: 2026/09/21 04:26:28 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6119lb # [ 156.206912] niks3-server[1038]: 2026/09/21 04:26:28 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6120worker1 # [ 156.258657] nix-grpc-daemon[1764]: time=2026-09-21T04:26:28.884Z level=INFO msg="Upload complete. (322ms)"6121lb # [ 156.211543] niks3-server[1038]: 2026/09/21 04:26:28 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6122lb # [ 156.216269] niks3-server[1038]: 2026/09/21 04:26:28 INFO Completed upload id=556123lb # [ 156.219767] niks3-server[1038]: 2026/09/21 04:26:28 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6124worker2 # [ 156.224275] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:28Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16125worker1 # [ 156.273231] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16126worker2 # [ 156.238720] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:28Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06127worker1 # [ 156.292152] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906128worker1 # [ 156.298986] nix-daemon[838]: reaped child process 1927, status = succeeded6129worker1 # [ 156.303457] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446906130worker1 # [ 156.312007] nix-grpc-daemon[1764]: time=2026-09-21T04:26:28.938Z level=INFO msg="All 1 paths already cached"6131worker1 # [ 156.318149] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446906132worker1 # [ 156.329186] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6133worker1 # [ 156.517318] nix-daemon[838]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)6134worker1 # [ 156.536139] nix-daemon[838]: reaped child process 1955, status = succeeded6135worker1 # [ 156.708353] nix-daemon[838]: accepted connection from pid 1957, user nix-grpc-daemon (trusted)6136worker1 # [ 156.725196] nix-daemon[838]: reaped child process 1960, status = succeeded6137lb # [ 156.688564] niks3-server[1038]: 2026/09/21 04:26:29 INFO Received uploads request method=POST path=/api/pending_closures6138worker1 # [ 156.744581] nix-grpc-daemon[1764]: time=2026-09-21T04:26:29.370Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6139worker1 # [ 156.746047] nix-grpc-daemon[1764]: time=2026-09-21T04:26:29.370Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6140lb # [ 156.714257] niks3-server[1038]: 2026/09/21 04:26:29 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6141lb # [ 156.728367] niks3-server[1038]: 2026/09/21 04:26:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6142worker1 # [ 156.781868] nix-grpc-daemon[1764]: time=2026-09-21T04:26:29.407Z level=INFO msg="Uploading 1 narinfos"6143lb # [ 156.741186] niks3-server[1038]: 2026/09/21 04:26:29 INFO Signed narinfos id=56 count=16144lb # [ 156.746550] niks3-server[1038]: 2026/09/21 04:26:29 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6145lb # [ 156.754469] niks3-server[1038]: 2026/09/21 04:26:29 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6146worker1 # [ 156.806605] nix-grpc-daemon[1764]: time=2026-09-21T04:26:29.432Z level=INFO msg="Upload complete. (463ms)"6147lb # [ 156.759679] niks3-server[1038]: 2026/09/21 04:26:29 INFO Completed upload id=566148lb # [ 156.763538] niks3-server[1038]: 2026/09/21 04:26:29 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6149worker1 # [ 156.816864] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 paths=1 nar_bytes_in=6246150worker1 # [ 156.825340] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906151worker1 # [ 156.830774] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6152worker1 # [ 156.835124] nix-daemon[838]: reaped child process 1950, status = succeeded6153worker1 # [ 156.845728] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6154worker1 # [ 157.676639] nix-daemon[838]: reaped child process 1964, status = succeeded6155worker1 # [ 157.838337] nix-daemon[838]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)6156worker1 # [ 157.858994] nix-daemon[838]: reaped child process 1983, status = succeeded6157lb # [ 157.824789] niks3-server[1038]: 2026/09/21 04:26:30 INFO Received uploads request method=POST path=/api/pending_closures6158worker1 # [ 157.884463] nix-grpc-daemon[1764]: time=2026-09-21T04:26:30.509Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6159worker1 # [ 157.885964] nix-grpc-daemon[1764]: time=2026-09-21T04:26:30.509Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6160lb # [ 157.862424] niks3-server[1038]: 2026/09/21 04:26:30 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6161lb # [ 157.880214] niks3-server[1038]: 2026/09/21 04:26:30 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6162lb # [ 157.888546] niks3-server[1038]: 2026/09/21 04:26:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6163worker1 # [ 157.941533] nix-grpc-daemon[1764]: time=2026-09-21T04:26:30.567Z level=INFO msg="Uploading 1 narinfos"6164lb # [ 157.896898] niks3-server[1038]: 2026/09/21 04:26:30 INFO Signed narinfos id=57 count=16165lb # [ 157.902617] niks3-server[1038]: 2026/09/21 04:26:30 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6166lb # [ 157.910410] niks3-server[1038]: 2026/09/21 04:26:30 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6167lb # [ 157.918656] niks3-server[1038]: 2026/09/21 04:26:30 INFO Completed upload id=576168worker1 # [ 157.966384] nix-grpc-daemon[1764]: time=2026-09-21T04:26:30.592Z level=INFO msg="Upload complete. (310ms)"6169lb # [ 157.922069] niks3-server[1038]: 2026/09/21 04:26:30 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6170worker2 # [ 157.930986] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:30Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16171worker1 # [ 157.981236] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16172worker1 # [ 157.997326] nix-daemon[838]: reaped child process 1962, status = succeeded6173worker1 # [ 158.021974] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906174worker2 # [ 157.979935] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:30Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06175worker1 # [ 158.037954] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446906176worker1 # [ 158.053125] nix-grpc-daemon[1764]: time=2026-09-21T04:26:30.678Z level=INFO msg="All 2 paths already cached"6177worker1 # [ 158.062011] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446906178worker1 # [ 158.075575] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6179worker1 # [ 158.271477] nix-daemon[838]: accepted connection from pid 1987, user nix-grpc-daemon (trusted)6180worker1 # [ 158.291441] nix-daemon[838]: reaped child process 1990, status = succeeded6181worker1 # [ 158.466643] nix-daemon[838]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6182worker1 # [ 158.483944] nix-daemon[838]: reaped child process 1995, status = succeeded6183lb # [ 158.447813] niks3-server[1038]: 2026/09/21 04:26:31 INFO Received uploads request method=POST path=/api/pending_closures6184worker1 # [ 158.504110] nix-grpc-daemon[1764]: time=2026-09-21T04:26:31.130Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6185worker1 # [ 158.505821] nix-grpc-daemon[1764]: time=2026-09-21T04:26:31.130Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6186lb # [ 158.477565] niks3-server[1038]: 2026/09/21 04:26:31 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6187lb # [ 158.491681] niks3-server[1038]: 2026/09/21 04:26:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6188worker1 # [ 158.547398] nix-grpc-daemon[1764]: time=2026-09-21T04:26:31.173Z level=INFO msg="Uploading 1 narinfos"6189lb # [ 158.499960] niks3-server[1038]: 2026/09/21 04:26:31 INFO Signed narinfos id=58 count=16190lb # [ 158.508644] niks3-server[1038]: 2026/09/21 04:26:31 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6191lb # [ 158.517483] niks3-server[1038]: 2026/09/21 04:26:31 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6192lb # [ 158.523339] niks3-server[1038]: 2026/09/21 04:26:31 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6193lb # [ 158.528422] niks3-server[1038]: 2026/09/21 04:26:31 INFO Completed upload id=586194worker1 # [ 158.576158] nix-grpc-daemon[1764]: time=2026-09-21T04:26:31.202Z level=INFO msg="Upload complete. (486ms)"6195worker1 # [ 158.584604] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 paths=1 nar_bytes_in=7526196worker1 # [ 158.599847] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906197worker1 # [ 158.606189] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6198worker1 # [ 158.611024] nix-daemon[838]: reaped child process 1985, status = succeeded6199worker1 # [ 158.625680] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6200worker1 # [ 159.247763] nix-daemon[838]: reaped child process 1999, status = succeeded6201worker1 # [ 159.403979] nix-daemon[838]: accepted connection from pid 2015, user nix-grpc-daemon (trusted)6202worker1 # [ 159.422095] nix-daemon[838]: reaped child process 2018, status = succeeded6203lb # [ 159.387336] niks3-server[1038]: 2026/09/21 04:26:32 INFO Received uploads request method=POST path=/api/pending_closures6204worker1 # [ 159.448302] nix-grpc-daemon[1764]: time=2026-09-21T04:26:32.071Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6205worker1 # [ 159.449742] nix-grpc-daemon[1764]: time=2026-09-21T04:26:32.071Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6206lb # [ 159.425400] niks3-server[1038]: 2026/09/21 04:26:32 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6207lb # [ 159.437022] niks3-server[1038]: 2026/09/21 04:26:32 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6208worker1 # [ 159.494173] nix-grpc-daemon[1764]: time=2026-09-21T04:26:32.120Z level=INFO msg="Uploading 1 narinfos"6209lb # [ 159.450624] niks3-server[1038]: 2026/09/21 04:26:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6210lb # [ 159.457692] niks3-server[1038]: 2026/09/21 04:26:32 INFO Signed narinfos id=59 count=16211lb # [ 159.459838] niks3-server[1038]: 2026/09/21 04:26:32 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6212lb # [ 159.466736] niks3-server[1038]: 2026/09/21 04:26:32 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6213lb # [ 159.470945] niks3-server[1038]: 2026/09/21 04:26:32 INFO Completed upload id=596214worker1 # [ 159.518048] nix-grpc-daemon[1764]: time=2026-09-21T04:26:32.144Z level=INFO msg="Upload complete. (283ms)"6215lb # [ 159.473383] niks3-server[1038]: 2026/09/21 04:26:32 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6216worker2 # [ 159.481358] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:32Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16217worker1 # [ 159.527513] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16218worker1 # [ 159.540729] nix-daemon[838]: reaped child process 1997, status = succeeded6219worker1 # [ 159.545302] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906220worker1 # [ 159.554437] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446906221worker1 # [ 159.565484] nix-grpc-daemon[1764]: time=2026-09-21T04:26:32.191Z level=INFO msg="All 2 paths already cached"6222worker1 # [ 159.573470] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446906223worker1 # [ 159.584182] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6224worker1 # [ 159.769697] nix-daemon[838]: accepted connection from pid 2022, user nix-grpc-daemon (trusted)6225worker1 # [ 159.786149] nix-daemon[838]: reaped child process 2025, status = succeeded6226worker1 # [ 159.958238] nix-daemon[838]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)6227worker1 # [ 159.974108] nix-daemon[838]: reaped child process 2030, status = succeeded6228lb # [ 159.938580] niks3-server[1038]: 2026/09/21 04:26:32 INFO Received uploads request method=POST path=/api/pending_closures6229worker1 # [ 159.994807] nix-grpc-daemon[1764]: time=2026-09-21T04:26:32.620Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6230worker1 # [ 159.996398] nix-grpc-daemon[1764]: time=2026-09-21T04:26:32.620Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6231lb # [ 159.969284] niks3-server[1038]: 2026/09/21 04:26:32 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6232lb # [ 159.980411] niks3-server[1038]: 2026/09/21 04:26:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6233worker1 # [ 160.034321] nix-grpc-daemon[1764]: time=2026-09-21T04:26:32.660Z level=INFO msg="Uploading 1 narinfos"6234lb # [ 159.991238] niks3-server[1038]: 2026/09/21 04:26:32 INFO Signed narinfos id=60 count=16235lb # [ 159.998685] niks3-server[1038]: 2026/09/21 04:26:32 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6236lb # [ 160.007761] niks3-server[1038]: 2026/09/21 04:26:32 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6237lb # [ 160.013476] niks3-server[1038]: 2026/09/21 04:26:32 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6238lb # [ 160.017813] niks3-server[1038]: 2026/09/21 04:26:32 INFO Completed upload id=606239worker1 # [ 160.065414] nix-grpc-daemon[1764]: time=2026-09-21T04:26:32.691Z level=INFO msg="Upload complete. (468ms)"6240worker1 # [ 160.072679] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 paths=1 nar_bytes_in=7526241worker1 # [ 160.085666] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906242worker1 # [ 160.092581] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6243worker1 # [ 160.097553] nix-daemon[838]: reaped child process 2020, status = succeeded6244worker1 # [ 160.110980] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6245worker1 # [ 160.931264] nix-daemon[838]: reaped child process 2034, status = succeeded6246worker1 # [ 161.083286] nix-daemon[838]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)6247worker1 # [ 161.099993] nix-daemon[838]: reaped child process 2053, status = succeeded6248lb # [ 161.066714] niks3-server[1038]: 2026/09/21 04:26:33 INFO Received uploads request method=POST path=/api/pending_closures6249worker1 # [ 161.124328] nix-grpc-daemon[1764]: time=2026-09-21T04:26:33.749Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6250worker1 # [ 161.125777] nix-grpc-daemon[1764]: time=2026-09-21T04:26:33.749Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6251lb # [ 161.103300] niks3-server[1038]: 2026/09/21 04:26:33 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6252lb # [ 161.114332] niks3-server[1038]: 2026/09/21 04:26:33 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6253lb # [ 161.124625] niks3-server[1038]: 2026/09/21 04:26:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6254worker1 # [ 161.177734] nix-grpc-daemon[1764]: time=2026-09-21T04:26:33.803Z level=INFO msg="Uploading 1 narinfos"6255lb # [ 161.130025] niks3-server[1038]: 2026/09/21 04:26:33 INFO Signed narinfos id=61 count=16256lb # [ 161.134795] niks3-server[1038]: 2026/09/21 04:26:33 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6257lb # [ 161.143815] niks3-server[1038]: 2026/09/21 04:26:33 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6258lb # [ 161.152136] niks3-server[1038]: 2026/09/21 04:26:33 INFO Completed upload id=616259worker1 # [ 161.199826] nix-grpc-daemon[1764]: time=2026-09-21T04:26:33.825Z level=INFO msg="Upload complete. (280ms)"6260lb # [ 161.154847] niks3-server[1038]: 2026/09/21 04:26:33 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6261worker2 # [ 161.163485] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:33Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16262worker1 # [ 161.212755] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16263worker1 # [ 161.224837] nix-daemon[838]: reaped child process 2032, status = succeeded6264worker1 # [ 161.255867] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906265worker1 # [ 161.270339] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446906266worker1 # [ 161.284049] nix-grpc-daemon[1764]: time=2026-09-21T04:26:33.910Z level=INFO msg="All 2 paths already cached"6267worker1 # [ 161.293105] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446906268worker1 # [ 161.306669] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6269worker1 # [ 161.499026] nix-daemon[838]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)6270worker1 # [ 161.516582] nix-daemon[838]: reaped child process 2060, status = succeeded6271worker1 # [ 161.688870] nix-daemon[838]: accepted connection from pid 2062, user nix-grpc-daemon (trusted)6272worker1 # [ 161.703560] nix-daemon[838]: reaped child process 2065, status = succeeded6273lb # [ 161.671614] niks3-server[1038]: 2026/09/21 04:26:34 INFO Received uploads request method=POST path=/api/pending_closures6274worker1 # [ 161.728074] nix-grpc-daemon[1764]: time=2026-09-21T04:26:34.354Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6275worker1 # [ 161.729571] nix-grpc-daemon[1764]: time=2026-09-21T04:26:34.354Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6276lb # [ 161.697819] niks3-server[1038]: 2026/09/21 04:26:34 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6277worker1 # [ 161.758529] nix-grpc-daemon[1764]: time=2026-09-21T04:26:34.384Z level=INFO msg="Uploading 1 narinfos"6278lb # [ 161.715283] niks3-server[1038]: 2026/09/21 04:26:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6279lb # [ 161.726207] niks3-server[1038]: 2026/09/21 04:26:34 INFO Signed narinfos id=62 count=16280worker1 # [ 161.782240] nix-grpc-daemon[1764]: time=2026-09-21T04:26:34.408Z level=INFO msg="Upload complete. (460ms)"6281lb # [ 161.735402] niks3-server[1038]: 2026/09/21 04:26:34 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6282worker1 # [ 161.788427] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 paths=1 nar_bytes_in=7526283lb # [ 161.740933] niks3-server[1038]: 2026/09/21 04:26:34 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6284lb # [ 161.746351] niks3-server[1038]: 2026/09/21 04:26:34 INFO Completed upload id=626285lb # [ 161.748959] niks3-server[1038]: 2026/09/21 04:26:34 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6286worker1 # [ 161.798879] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446906287worker1 # [ 161.807477] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6288worker1 # [ 161.809523] nix-daemon[838]: reaped child process 2055, status = succeeded6289worker1 # [ 161.821430] nix-daemon[838]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6290worker1 # [ 162.448813] nix-daemon[838]: reaped child process 2069, status = succeeded6291worker1 # [ 162.613837] nix-daemon[838]: accepted connection from pid 2085, user nix-grpc-daemon (trusted)6292worker1 # [ 162.630230] nix-daemon[838]: reaped child process 2088, status = succeeded6293lb # [ 162.595393] niks3-server[1038]: 2026/09/21 04:26:35 INFO Received uploads request method=POST path=/api/pending_closures6294worker1 # [ 162.652393] nix-grpc-daemon[1764]: time=2026-09-21T04:26:35.278Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6295worker1 # [ 162.653834] nix-grpc-daemon[1764]: time=2026-09-21T04:26:35.278Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6296lb # [ 162.637606] niks3-server[1038]: 2026/09/21 04:26:35 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6297lb # [ 162.649970] niks3-server[1038]: 2026/09/21 04:26:35 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6298worker1 # [ 162.705652] nix-grpc-daemon[1764]: time=2026-09-21T04:26:35.331Z level=INFO msg="Uploading 1 narinfos"6299lb # [ 162.661550] niks3-server[1038]: 2026/09/21 04:26:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6300lb # [ 162.668957] niks3-server[1038]: 2026/09/21 04:26:35 INFO Signed narinfos id=63 count=16301lb # [ 162.671083] niks3-server[1038]: 2026/09/21 04:26:35 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6302lb # [ 162.677286] niks3-server[1038]: 2026/09/21 04:26:35 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6303lb # [ 162.682752] niks3-server[1038]: 2026/09/21 04:26:35 INFO Completed upload id=636304worker1 # [ 162.730457] nix-grpc-daemon[1764]: time=2026-09-21T04:26:35.356Z level=INFO msg="Upload complete. (301ms)"6305lb # [ 162.685168] niks3-server[1038]: 2026/09/21 04:26:35 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6306worker1 # [ 162.741060] nix-grpc-daemon[1707]: ts=2026-09-21T04:26:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44690 duration_s=0 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16307worker1 # [ 162.755336] nix-daemon[838]: reaped child process 2067, status = succeeded6308client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.31 seconds)6309worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6310worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6311client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6312client # [ 162.906672] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6313client # Running as unit: gpu.service; invocation ID: e8a3a37b686942a798babac8754255ed6314client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6315lb: must succeed: curl -sf localhost:9901/clusters6316lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6317lb: must succeed: curl -sf localhost:9901/clusters6318lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6319worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6320worker2 # [ 163.162545] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:340346321client # [ 163.159012] nix[1359]: don't know how to build these paths:6322client # [ 163.159972] nix[1359]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6323worker2 # [ 163.173624] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340346324worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)6325client # [ 163.186550] nix[1359]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6326worker2 # [ 163.195271] nix-grpc-daemon[1393]: ts=2026-09-21T04:26:35Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06327lb: must succeed: curl -sf localhost:9901/clusters6328lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6329lb: must succeed: curl -sf localhost:9901/clusters6330lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6331worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6332worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6333client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6334lb # [ 169.291114] envoy[902]: conn peer=[2001:db8:1::1]:33726 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27075 tx=11457 ms=169836335client # [ 193.191442] nix[1359]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6336client # [ 193.201045] nix[1359]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6337client # [ 193.250614] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6338client # [ 193.258458] systemd[1]: gpu.service: Failed with result 'exit-code'.6339client # [ 193.261585] systemd[1]: gpu.service: Consumed 177ms CPU time over 30.350s wall clock time, 37.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6340client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.25 seconds)6341client: must succeed: journalctl -u gpu -o cat6342client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6343lb: must succeed: curl -sf localhost:9901/clusters6344lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6345lb: must succeed: curl -sf localhost:9901/clusters6346lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6347worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6348worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6349(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.44 seconds)6350subtest: low disk drains a worker and builds go to the other6351worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6352worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.12 seconds)6353worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6354worker1 # [ 195.202565] nix-grpc-daemon[1707]: ts=2026-09-21T04:27:07Z level=info event=unhealthy reason=min_free6355worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6356client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26357client # don't know how to build these paths:6358client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6359worker2 # [ 196.407407] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:340346360worker1 # [ 196.457976] nix-grpc-daemon[1707]: ts=2026-09-21T04:27:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446906361worker2 # [ 196.438871] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:09Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06362worker2 # [ 196.445789] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:09Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16363worker2 # [ 196.486302] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340346364worker2 # [ 196.501588] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340346365client # copying 1 paths to 'lb:50051'...6366client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6367worker2 # [ 196.517331] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340346368worker2 # [ 196.532128] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6369worker2 # [ 196.744142] nix-daemon[857]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6370worker2 # [ 196.760691] nix-daemon[857]: reaped child process 1688, status = succeeded6371worker2 # [ 196.932132] nix-daemon[857]: accepted connection from pid 1690, user nix-grpc-daemon (trusted)6372worker2 # [ 196.947370] nix-daemon[857]: reaped child process 1693, status = succeeded6373lb # [ 196.961952] niks3-server[1038]: 2026/09/21 04:27:09 INFO Received uploads request method=POST path=/api/pending_closures6374worker2 # [ 196.972366] nix-grpc-daemon[1466]: time=2026-09-21T04:27:09.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6375worker2 # [ 196.973828] nix-grpc-daemon[1466]: time=2026-09-21T04:27:09.612Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6376lb # [ 197.001333] niks3-server[1038]: 2026/09/21 04:27:09 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6377worker2 # [ 197.025439] nix-grpc-daemon[1466]: time=2026-09-21T04:27:09.665Z level=INFO msg="Uploading 1 narinfos"6378lb # [ 197.016767] niks3-server[1038]: 2026/09/21 04:27:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6379lb # [ 197.029166] niks3-server[1038]: 2026/09/21 04:27:09 INFO Signed narinfos id=64 count=16380lb # [ 197.035365] niks3-server[1038]: 2026/09/21 04:27:09 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6381lb # [ 197.046384] niks3-server[1038]: 2026/09/21 04:27:09 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6382lb # [ 197.054980] niks3-server[1038]: 2026/09/21 04:27:09 INFO Completed upload id=646383worker2 # [ 197.057325] nix-grpc-daemon[1466]: time=2026-09-21T04:27:09.697Z level=INFO msg="Upload complete. (508ms)"6384lb # [ 197.058566] niks3-server[1038]: 2026/09/21 04:27:09 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6385worker2 # [ 197.066437] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=0 paths=1 nar_bytes_in=5126386worker2 # [ 197.082652] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340346387worker2 # [ 197.092354] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6388worker2 # [ 197.097300] nix-daemon[857]: reaped child process 1683, status = succeeded6389client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6390client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6391worker2 # [ 197.112239] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6392client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6393client # farm-a> LOG-farm-a-drain6394worker2 # [ 198.161957] nix-daemon[857]: reaped child process 1697, status = succeeded6395worker2 # [ 198.343271] nix-daemon[857]: accepted connection from pid 1713, user nix-grpc-daemon (trusted)6396worker2 # [ 198.361771] nix-daemon[857]: reaped child process 1716, status = succeeded6397lb # [ 198.373074] niks3-server[1038]: 2026/09/21 04:27:10 INFO Received uploads request method=POST path=/api/pending_closures6398worker2 # [ 198.388398] nix-grpc-daemon[1466]: time=2026-09-21T04:27:11.026Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6399worker2 # [ 198.389918] nix-grpc-daemon[1466]: time=2026-09-21T04:27:11.026Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6400lb # [ 198.418955] niks3-server[1038]: 2026/09/21 04:27:11 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6401lb # [ 198.432314] niks3-server[1038]: 2026/09/21 04:27:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6402worker2 # [ 198.439355] nix-grpc-daemon[1466]: time=2026-09-21T04:27:11.079Z level=INFO msg="Uploading 1 narinfos"6403lb # [ 198.440096] niks3-server[1038]: 2026/09/21 04:27:11 INFO Signed narinfos id=65 count=16404lb # [ 198.444104] niks3-server[1038]: 2026/09/21 04:27:11 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6405lb # [ 198.452363] niks3-server[1038]: 2026/09/21 04:27:11 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6406lb # [ 198.459673] niks3-server[1038]: 2026/09/21 04:27:11 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6407lb # [ 198.463258] niks3-server[1038]: 2026/09/21 04:27:11 INFO Completed upload id=656408lb # [ 198.465378] niks3-server[1038]: 2026/09/21 04:27:11 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6409worker2 # [ 198.465636] nix-grpc-daemon[1466]: time=2026-09-21T04:27:11.105Z level=INFO msg="Upload complete. (339ms)"6410worker2 # [ 198.485077] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16411worker2 # [ 198.500256] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:11Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06412worker2 # [ 198.503848] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:11Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16413worker2 # [ 198.508266] nix-daemon[857]: reaped child process 1695, status = succeeded6414worker2 # [ 198.526023] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340346415worker2 # [ 198.540371] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340346416worker2 # [ 198.554032] nix-grpc-daemon[1466]: time=2026-09-21T04:27:11.194Z level=INFO msg="All 1 paths already cached"6417client # copying 1 paths to 'lb:50051'...6418client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6419worker2 # [ 198.563957] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340346420worker2 # [ 198.577727] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6421worker2 # [ 198.780177] nix-daemon[857]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6422worker2 # [ 198.797946] nix-daemon[857]: reaped child process 1725, status = succeeded6423worker2 # [ 198.964124] nix-daemon[857]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6424worker2 # [ 198.981315] nix-daemon[857]: reaped child process 1730, status = succeeded6425lb # [ 198.989724] niks3-server[1038]: 2026/09/21 04:27:11 INFO Received uploads request method=POST path=/api/pending_closures6426worker2 # [ 199.000062] nix-grpc-daemon[1466]: time=2026-09-21T04:27:11.640Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6427worker2 # [ 199.001923] nix-grpc-daemon[1466]: time=2026-09-21T04:27:11.641Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6428lb # [ 199.017022] niks3-server[1038]: 2026/09/21 04:27:11 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6429lb # [ 199.029901] niks3-server[1038]: 2026/09/21 04:27:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6430worker2 # [ 199.038222] nix-grpc-daemon[1466]: time=2026-09-21T04:27:11.678Z level=INFO msg="Uploading 1 narinfos"6431lb # [ 199.040277] niks3-server[1038]: 2026/09/21 04:27:11 INFO Signed narinfos id=66 count=16432lb # [ 199.043415] niks3-server[1038]: 2026/09/21 04:27:11 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6433lb # [ 199.058365] niks3-server[1038]: 2026/09/21 04:27:11 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6434lb # [ 199.063598] niks3-server[1038]: 2026/09/21 04:27:11 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6435lb # [ 199.068166] niks3-server[1038]: 2026/09/21 04:27:11 INFO Completed upload id=666436worker2 # [ 199.070389] nix-grpc-daemon[1466]: time=2026-09-21T04:27:11.710Z level=INFO msg="Upload complete. (476ms)"6437worker2 # [ 199.076261] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=0 paths=1 nar_bytes_in=6326438worker2 # [ 199.089245] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340346439worker2 # [ 199.098790] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6440worker2 # [ 199.101099] nix-daemon[857]: reaped child process 1720, status = succeeded6441client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6442client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6443worker2 # [ 199.114860] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6444lb # [ 199.297580] envoy[902]: conn peer=[2001:db8:1::1]:50188 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300606445client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6446client # farm-b> LOG-farm-b-drain6447worker2 # [ 199.944203] nix-daemon[857]: reaped child process 1734, status = succeeded6448worker2 # [ 200.115078] nix-daemon[857]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6449worker2 # [ 200.131637] nix-daemon[857]: reaped child process 1753, status = succeeded6450lb # [ 200.141438] niks3-server[1038]: 2026/09/21 04:27:12 INFO Received uploads request method=POST path=/api/pending_closures6451worker2 # [ 200.155960] nix-grpc-daemon[1466]: time=2026-09-21T04:27:12.792Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6452worker2 # [ 200.157619] nix-grpc-daemon[1466]: time=2026-09-21T04:27:12.792Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6453lb # [ 200.174660] niks3-server[1038]: 2026/09/21 04:27:12 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6454lb # [ 200.188310] niks3-server[1038]: 2026/09/21 04:27:12 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6455lb # [ 200.199172] niks3-server[1038]: 2026/09/21 04:27:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6456worker2 # [ 200.206763] nix-grpc-daemon[1466]: time=2026-09-21T04:27:12.846Z level=INFO msg="Uploading 1 narinfos"6457lb # [ 200.209645] niks3-server[1038]: 2026/09/21 04:27:12 INFO Signed narinfos id=67 count=16458lb # [ 200.215702] niks3-server[1038]: 2026/09/21 04:27:12 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6459lb # [ 200.225004] niks3-server[1038]: 2026/09/21 04:27:12 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6460lb # [ 200.229767] niks3-server[1038]: 2026/09/21 04:27:12 INFO Completed upload id=676461worker2 # [ 200.231445] nix-grpc-daemon[1466]: time=2026-09-21T04:27:12.871Z level=INFO msg="Upload complete. (307ms)"6462lb # [ 200.232660] niks3-server[1038]: 2026/09/21 04:27:12 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6463worker2 # [ 200.246397] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16464worker2 # [ 200.260290] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:12Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06465worker2 # [ 200.263933] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:12Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16466worker2 # [ 200.268347] nix-daemon[857]: reaped child process 1732, status = succeeded6467worker2 # [ 200.294114] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340346468worker2 # [ 200.307537] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340346469worker2 # [ 200.320771] nix-grpc-daemon[1466]: time=2026-09-21T04:27:12.960Z level=INFO msg="All 2 paths already cached"6470client # copying 1 paths to 'lb:50051'...6471client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6472worker2 # [ 200.330661] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340346473worker2 # [ 200.344211] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6474worker2 # [ 200.546756] nix-daemon[857]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6475worker2 # [ 200.563728] nix-daemon[857]: reaped child process 1762, status = succeeded6476worker2 # [ 200.734813] nix-daemon[857]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6477worker2 # [ 200.750710] nix-daemon[857]: reaped child process 1767, status = succeeded6478lb # [ 200.759472] niks3-server[1038]: 2026/09/21 04:27:13 INFO Received uploads request method=POST path=/api/pending_closures6479worker2 # [ 200.770308] nix-grpc-daemon[1466]: time=2026-09-21T04:27:13.410Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6480worker2 # [ 200.771749] nix-grpc-daemon[1466]: time=2026-09-21T04:27:13.410Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6481lb # [ 200.785045] niks3-server[1038]: 2026/09/21 04:27:13 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6482worker2 # [ 200.800321] nix-grpc-daemon[1466]: time=2026-09-21T04:27:13.440Z level=INFO msg="Uploading 1 narinfos"6483lb # [ 200.798622] niks3-server[1038]: 2026/09/21 04:27:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6484lb # [ 200.810667] niks3-server[1038]: 2026/09/21 04:27:13 INFO Signed narinfos id=68 count=16485lb # [ 200.816855] niks3-server[1038]: 2026/09/21 04:27:13 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6486worker2 # [ 200.826729] nix-grpc-daemon[1466]: time=2026-09-21T04:27:13.466Z level=INFO msg="Upload complete. (466ms)"6487lb # [ 200.825561] niks3-server[1038]: 2026/09/21 04:27:13 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6488lb # [ 200.830545] niks3-server[1038]: 2026/09/21 04:27:13 INFO Completed upload id=686489lb # [ 200.833450] niks3-server[1038]: 2026/09/21 04:27:13 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6490worker2 # [ 200.836195] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=0 paths=1 nar_bytes_in=7606491worker2 # [ 200.839837] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340346492worker2 # [ 200.851664] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6493worker2 # [ 200.854173] nix-daemon[857]: reaped child process 1757, status = succeeded6494client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6495client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6496worker2 # [ 200.866032] nix-daemon[857]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6497client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6498client # farm-top> LOG-farm-top-drain6499worker2 # [ 202.086603] nix-daemon[857]: reaped child process 1771, status = succeeded6500worker2 # [ 202.246847] nix-daemon[857]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6501worker2 # [ 202.266205] nix-daemon[857]: reaped child process 1790, status = succeeded6502lb # [ 202.277944] niks3-server[1038]: 2026/09/21 04:27:14 INFO Received uploads request method=POST path=/api/pending_closures6503worker2 # [ 202.290977] nix-grpc-daemon[1466]: time=2026-09-21T04:27:14.930Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6504worker2 # [ 202.293826] nix-grpc-daemon[1466]: time=2026-09-21T04:27:14.932Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6505lb # [ 202.315106] niks3-server[1038]: 2026/09/21 04:27:14 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6506lb # [ 202.332949] niks3-server[1038]: 2026/09/21 04:27:14 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6507worker2 # [ 202.347902] nix-grpc-daemon[1466]: time=2026-09-21T04:27:14.987Z level=INFO msg="Uploading 1 narinfos"6508lb # [ 202.340907] niks3-server[1038]: 2026/09/21 04:27:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6509lb # [ 202.350747] niks3-server[1038]: 2026/09/21 04:27:14 INFO Signed narinfos id=69 count=16510lb # [ 202.355785] niks3-server[1038]: 2026/09/21 04:27:14 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6511worker2 # [ 202.365392] nix-grpc-daemon[1466]: time=2026-09-21T04:27:15.005Z level=INFO msg="Upload complete. (289ms)"6512lb # [ 202.363622] niks3-server[1038]: 2026/09/21 04:27:14 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6513lb # [ 202.367136] niks3-server[1038]: 2026/09/21 04:27:14 INFO Completed upload id=696514lb # [ 202.370441] niks3-server[1038]: 2026/09/21 04:27:14 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6515worker2 # [ 202.377072] nix-grpc-daemon[1393]: ts=2026-09-21T04:27:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34034 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16516worker2 # [ 202.391292] nix-daemon[857]: reaped child process 1769, status = succeeded6517client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.33 seconds)6518client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6519client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.24 seconds)6520worker1: must succeed: rm /nix/.rw-store/fill6521worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.21 seconds)6522worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6523worker1 # [ 203.206601] nix-grpc-daemon[1707]: ts=2026-09-21T04:27:15Z level=info event=healthy reason=min_free6524worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.14 seconds)6525(finished: subtest: low disk drains a worker and builds go to the other, in 10.31 seconds)6526(finished: run the VM test script, in 205.38 seconds)6527test script finished in 205.65s6528cleanup6529kill QemuMachine (pid 49)6530client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6531kill QemuMachine (pid 50)6532lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6533kill QemuMachine (pid 51)6534worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6535kill QemuMachine (pid 52)6536worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6537(finished: cleanup, in 1.00 seconds)6538additionally exposed symbols:6539 client, lb, worker1, worker2,6540 vlan1,6541 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh